Electronic device for encoding or decoding video data and non-transitory machine-readable medium

Abstract

An electronic device is provided for decoding video data. One or more computer-executable instructions, when executed by at least one processor of the electronic device, cause the electronic device to: receive the video data; determine, from a current frame in the video data, a block unit and vector reference blocks of the block unit; derive filter models of the block unit; determine first template matching costs; determine a first arrangement of the filter models based on the first template matching costs; and reconstruct the block unit based on the first arrangement. Each of the vector reference blocks is indicated by a corresponding one of block vector candidates of the block unit. Each of the filter models is derived based on a corresponding one of the vector reference blocks. Each of the first template matching costs is calculated using a corresponding one of the filter models of the block unit.

Description

ELECTRONIC DEVICE FOR ENCODING OR DECODING VIDEO DATA AND NON-TRANSITORY MACHINE-READABLE MEDIUM

[0001] The present disclosure generally relates to video coding, and more specifically, to techniques for predicting and / or reconstructing a block unit by arranging multiple filter models derived based on multiple reference blocks indicated by multiple block vector candidates of the block unit.

[0002] The present disclosure claims the benefit of and priority to U.S. Provisional Patent Application Serial No. 63 / 587,828, filed on October 4, 2023, entitled “Reconstruction-reordered Intra Block Copy with Local Illumination Compensation,” and U.S. Provisional Patent Application Serial No. 63 / 619,593, filed on January 10, 2024, entitled “Intra Block Copy Merge Mode with Filtered Intra Block Copy,” the contents of all of which are hereby incorporated herein fully by reference in their entirety for all purposes.

[0003] Filtered intra block copy (FIBC) mode is a coding tool for video coding, in which, an encoder and / or a decoder may predict each of multiple block samples in a current block by using a filter model. In addition, the encoder and / or the decoder may derive the filter model of the current block based on a block template of the current block and a reference template of a reference block, indicated by an intra block vector of the current block. However, in the FIBC mode, the encoder needs to provide some prediction syntaxes for indicating the prediction information of the current block to the decoder.

[0004] Therefore, an efficient operation mechanism for the FIBC mode may be required for the encoder and / or the decoder to decrease number of bits in the bitstream.

[0005] The present disclosure is directed to a device and method for predicting and / or reconstructing a block unit by arranging multiple filter models derived based on multiple reference blocks indicated by multiple block vector candidates of the block unit.

[0006] In a first aspect of the present disclosure, an electronic device for decoding video data is provided. The electronic device includes at least one processor; and one or more non-transitory computer-readable media coupled to the at least one processor and storing one or more computer-executable instructions that, when executed by the at least one processor, cause the electronic device to: receive the video data; determine a block unit from a current frame included in the video data; determine, from the current frame, multiple vector reference blocks of the block unit, each of the multiple vector reference blocks indicated by a corresponding one of multiple block vector candidates of the block unit; derive multiple filter models of the block unit, each of the multiple filter models derived based on a corresponding one of the multiple vector reference blocks; determine multiple first template matching costs, each of the multiple first template matching costs calculated using a corresponding one of the multiple filter models of the block unit; determine a first arrangement of the multiple filter models based on the multiple first template matching costs; and reconstruct the block unit based on the first arrangement of the multiple filter models.

[0007] In an implementation of the first aspect of the present disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine multiple neighboring blocks neighboring the block unit; determine multiple neighboring block vectors, each of the multiple neighboring block vectors indicating a corresponding one of multiple neighboring reference blocks in the current frame for reconstructing a corresponding one of the multiple neighboring blocks; and determine the multiple block vector candidates based on the multiple neighboring block vectors.

[0008] In another implementation of the first aspect of the present disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a first block template region, neighboring the block unit, and multiple first reference template regions, each of the multiple first reference template regions neighboring a corresponding one of the multiple vector reference blocks; and determine a second block template region, neighboring the block unit, and multiple second reference template regions, each of the multiple second reference template regions neighboring a corresponding one of the multiple vector reference blocks, wherein: each of the multiple filter models is derived further based on the first block template region and a corresponding one of the multiple first reference template regions, and each of the multiple first template matching costs is calculated further based on the second block template region and a corresponding one of the multiple second reference template regions using the corresponding one of the multiple filter models.

[0009] In another implementation of the first aspect of the present disclosure, the first block template region is identical to the second block template, and each of the multiple first reference template regions is identical to a corresponding one of the multiple second template regions.

[0010] In another implementation of the first aspect of the present disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a block-vector-based candidate list including multiple block-vector-based prediction candidates, wherein: each of the block-vector-based prediction candidates corresponds to a corresponding one of the multiple block vector candidates and a corresponding one of the multiple filter models, the multiple block-vector-based prediction candidates in the block-vector-based candidate list is ordered based on the first arrangement of the multiple filter models, and reconstructing the block unit is further based on the multiple block-vector-based prediction candidates ordered in the block-vector-based candidate list.

[0011] In another implementation of the first aspect of the present disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine multiple second template matching costs, each of the multiple second template matching costs calculated directly based on a corresponding one of the multiple block vector candidates without using the multiple filter models of the block unit; and determine a second arrangement of multiple block-vector-based prediction candidates based on the multiple first template matching costs and the multiple second template matching costs, wherein reconstructing the block unit is further based on the second arrangement of the multiple block-vector-based prediction candidates.

[0012] In another implementation of the first aspect of the present disclosure, the multiple block-vector-based prediction candidates includes at least one of multiple first block-vector-based candidates or multiple second block-vector-based candidates, each of the multiple first block-vector-based candidates corresponds to a corresponding one of the multiple block vector candidates and a corresponding one of the multiple filter models, and each of the multiple second block-vector-based candidates only corresponds to a corresponding one of the multiple block vector candidates without using the multiple filter models.

[0013] In another implementation of the first aspect of the present disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a filter flag indicating whether to derive the multiple filter models for determining the first arrangement of the multiple filter models based on the multiple first template matching costs.

[0014] In a second aspect of the present disclosure, a non-transitory machine-readable medium of an electronic device storing one or more computer-executable instructions for decoding video data is provided. The one or more computer-executable instructions, when executed by at least one processor of the electronic device, cause the electronic device to: receive the video data; determine a block unit from a current frame included in the video data; determine, from the current frame, multiple vector reference blocks of the block unit, each of the multiple vector reference blocks indicated by a corresponding one of multiple block vector candidates of the block unit; derive multiple filter models of the block unit, each of the multiple filter models derived based on a corresponding one of the multiple vector reference blocks; determine multiple first template matching costs, each of the multiple first template matching costs calculated using a corresponding one of the multiple filter models of the block unit; determine a first arrangement of the multiple filter models based on the multiple first template matching costs; and reconstruct the block unit based on the first arrangement of the multiple filter models.

[0015] In an implementation of the second aspect of the present disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine multiple neighboring blocks neighboring the block unit; determine multiple neighboring block vectors, each of the multiple neighboring block vectors indicating a corresponding one of multiple neighboring reference blocks in the current frame for reconstructing a corresponding one of the multiple neighboring blocks; and determine the multiple block vector candidates based on the multiple neighboring block vectors.

[0016] In another implementation of the second aspect of the present disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a first block template region, neighboring the block unit, and multiple first reference template regions, each of the multiple first reference template regions neighboring a corresponding one of the multiple vector reference blocks; and determine a second block template region, neighboring the block unit, and multiple second reference template regions, each of the multiple second reference template regions neighboring a corresponding one of the multiple vector reference blocks, wherein: each of the multiple filter models is derived further based on the first block template region and a corresponding one of the multiple first reference template regions, and each of the multiple first template matching costs is calculated further based on the second block template region and a corresponding one of the multiple second reference template regions using the corresponding one of the multiple filter models.

[0017] In another implementation of the second aspect of the present disclosure, the first block template region is identical to the second block template, and each of the multiple first reference template regions is identical to a corresponding one of the multiple second template regions.

[0018] In another implementation of the second aspect of the present disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a block-vector-based candidate list including multiple block-vector-based prediction candidates, wherein: each of the block-vector-based prediction candidates corresponds to a corresponding one of the multiple block vector candidates and a corresponding one of the multiple filter models, the multiple block-vector-based prediction candidates in the block-vector-based candidate list is ordered based on the first arrangement of the multiple filter models, and reconstructing the block unit is further based on the multiple block-vector-based prediction candidates ordered in the block-vector-based candidate list.

[0019] In another implementation of the second aspect of the present disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine multiple second template matching costs, each of the multiple second template matching costs calculated directly based on a corresponding one of the multiple block vector candidates without using the multiple filter models of the block unit; and determine a second arrangement of multiple block-vector-based prediction candidates based on the multiple first template matching costs and the multiple second template matching costs, wherein reconstructing the block unit is further based on the second arrangement of the multiple block-vector-based prediction candidates.

[0020] In another implementation of the second aspect of the present disclosure, the multiple block-vector-based prediction candidates includes at least one of multiple first block-vector-based candidates or multiple second block-vector-based candidates, each of the multiple first block-vector-based candidates corresponds to a corresponding one of the multiple block vector candidates and a corresponding one of the multiple filter models, and each of the multiple second block-vector-based candidates only corresponds to a corresponding one of the multiple block vector candidates without using the multiple filter models.

[0021] In another implementation of the second aspect of the present disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a filter flag indicating whether to derive the multiple filter models for determining the first arrangement of the multiple filter models based on the multiple first template matching costs.

[0022] In a third aspect of the present disclosure, an electronic device for encoding video data is provided. The electronic device includes at least one processor; and one or more non-transitory computer-readable media coupled to the at least one processor and storing one or more computer-executable instructions that, when executed by the at least one processor, cause the electronic device to: receive the video data; determine a block unit from a current frame included in the video data; determine, from the current frame, multiple vector reference blocks of the block unit, each of the multiple vector reference blocks indicated by a corresponding one of multiple block vector candidates of the block unit; derive multiple filter models of the block unit, each of the multiple filter models derived based on a corresponding one of the multiple vector reference blocks; determine multiple first template matching costs, each of the multiple first template matching costs calculated using a corresponding one of the multiple filter models of the block unit; determine a first arrangement of the multiple filter models based on the multiple first template matching costs; and reconstruct the block unit based on the first arrangement of the multiple filter models.

[0023] In an implementation of the third aspect of the present disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine multiple neighboring blocks neighboring the block unit; determine multiple neighboring block vectors, each of the multiple neighboring block vectors indicating a corresponding one of multiple neighboring reference blocks in the current frame for reconstructing a corresponding one of the multiple neighboring blocks; and determine the multiple block vector candidates based on the multiple neighboring block vectors.

[0024] In another implementation of the third aspect of the present disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a first block template region, neighboring the block unit, and multiple first reference template regions, each of the multiple first reference template regions neighboring a corresponding one of the multiple vector reference blocks; and determine a second block template region, neighboring the block unit, and multiple second reference template regions, each of the multiple second reference template regions neighboring a corresponding one of the multiple vector reference blocks, wherein: each of the multiple filter models is derived further based on the first block template region and a corresponding one of the multiple first reference template regions, and each of the multiple first template matching costs is calculated further based on the second block template region and a corresponding one of the multiple second reference template regions using the corresponding one of the multiple filter models.

[0025] In another implementation of the third aspect of the present disclosure, the first block template region is identical to the second block template, and each of the multiple first reference template regions is identical to a corresponding one of the multiple second template regions.

[0026] In another implementation of the third aspect of the present disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a block-vector-based candidate list including multiple block-vector-based prediction candidates, wherein: each of the block-vector-based prediction candidates corresponds to a corresponding one of the multiple block vector candidates and a corresponding one of the multiple filter models, the multiple block-vector-based prediction candidates in the block-vector-based candidate list is ordered based on the first arrangement of the multiple filter models, and reconstructing the block unit is further based on the multiple block-vector-based prediction candidates ordered in the block-vector-based candidate list.

[0027] In another implementation of the third aspect of the present disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine multiple second template matching costs, each of the multiple second template matching costs calculated directly based on a corresponding one of the multiple block vector candidates without using the multiple filter models of the block unit; and determine a second arrangement of multiple block-vector-based prediction candidates based on the multiple first template matching costs and the multiple second template matching costs, wherein reconstructing the block unit is further based on the second arrangement of the multiple block-vector-based prediction candidates.

[0028] In another implementation of the third aspect of the present disclosure, the multiple block-vector-based prediction candidates includes at least one of multiple first block-vector-based candidates or multiple second block-vector-based candidates, each of the multiple first block-vector-based candidates corresponds to a corresponding one of the multiple block vector candidates and a corresponding one of the multiple filter models, and each of the multiple second block-vector-based candidates only corresponds to a corresponding one of the multiple block vector candidates without using the multiple filter models.

[0029] In another implementation of the third aspect of the present disclosure, the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: determine a filter flag indicating whether to derive the multiple filter models for determining the first arrangement of the multiple filter models based on the multiple first template matching costs.

[0030] Aspects of the present disclosure are best understood from the following detailed disclosure and the corresponding figures. Various features are not drawn to scale and dimensions of various features may be arbitrarily increased or reduced for clarity of discussion.

[0031] FIG. 1 is a block diagram illustrating a system having a first electronic device and a second electronic device for encoding and decoding video data, in accordance with one or more example implementations of this disclosure.

[0032] FIG. 2 is a block diagram illustrating a decoder module of the second electronic device illustrated in FIG. 1, in accordance with one or more example implementations of this disclosure.

[0033] FIG. 3 is a flowchart illustrating a method / process for decoding and / or encoding video data by an electronic device, in accordance with one or more example implementations of this disclosure.

[0034] FIGS. 4A and 4B are schematic illustrations of a reference block of the block unit, indicated by a block vector, in accordance with one or more example implementations of this disclosure.

[0035] FIGS. 5A and 5B are schematic illustrations of the corresponding relationship between the block template region and the reference template region, in accordance with one or more example implementations of this disclosure.

[0036] FIGS. 6A-6C are schematic illustrations of a horizontal flipping relationship between the block template region and the reference template region, in accordance with one or more example implementations of this disclosure.

[0037] FIG. 7 is a flowchart illustrating a method / process for decoding and / or encoding video data by an electronic device, in accordance with one or more example implementations of this disclosure.

[0038] FIGS. 8A and 8B are schematic illustrations of the relative location between the block unit and the first block template region and the relative location between the reference block and the first reference template region, in accordance with one or more example implementations of this disclosure.

[0039] FIG. 9 is a block diagram illustrating an encoder module of the first electronic device illustrated in FIG. 1, in accordance with one or more example implementations of this disclosure.

[0040] The following disclosure contains specific information pertaining to implementations in the present disclosure. The figures and the corresponding detailed disclosure are directed to example implementations. However, the present disclosure is not limited to these example implementations. Other variations and implementations of the present disclosure will occur to those skilled in the art.

[0041] Unless noted otherwise, like or corresponding elements among the figures may be indicated by like or corresponding reference designators. The figures and illustrations in the present disclosure are generally not to scale and are not intended to correspond to actual relative dimensions.

[0042] For the purposes of consistency and ease of understanding, features are identified (although, in some examples, not illustrated) by reference designators in the exemplary figures. However, the features in different implementations may differ in other respects and shall not be narrowly confined to what is illustrated in the figures.

[0043] The disclosure uses the phrases “in one implementation,” or “in some implementations,” which may refer to one or more of the same or different implementations. The term “coupled” is defined as connected, whether directly or indirectly through intervening components, and is not necessarily limited to physical connections. The term “comprising” means “including, but not necessarily limited to” and specifically indicates open-ended inclusion or membership in the so-described combination, group, series, and the equivalent.

[0044] For purposes of explanation and non-limitation, specific details, such as functional entities, techniques, protocols, and standards, are set forth for providing an understanding of the disclosed technology. Detailed disclosure of well-known methods, technologies, systems, and architectures are omitted so as not to obscure the present disclosure with unnecessary details.

[0045] Persons skilled in the art will recognize that any disclosed coding function(s) or algorithm(s) described in the present disclosure may be implemented by hardware, software, or a combination of software and hardware. Disclosed functions may correspond to modules that are software, hardware, firmware, or any combination thereof.

[0046] A software implementation may include a program having one or more computer-executable instructions stored on a computer-readable medium, such as memory or other types of storage devices. For example, one or more microprocessors or general-purpose computers with communication processing capability may be programmed with computer-executable instructions and perform the disclosed function(s) or algorithm(s).

[0047] The microprocessors or general-purpose computers may be formed of application-specific integrated circuits (ASICs), programmable logic arrays, and / or one or more digital signal processors (DSPs). Although some of the disclosed implementations are oriented to software installed and executing on computer hardware, alternative implementations implemented as firmware, as hardware, or as a combination of hardware and software are well within the scope of the present disclosure. The computer-readable medium includes, but is not limited to, random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, compact disc read-only memory (CD ROM), magnetic cassettes, magnetic tape, magnetic disk storage, or any other equivalent medium capable of storing computer-executable instructions. The computer-readable medium may be a non-transitory computer-readable medium.

[0048] FIG. 1 is a block diagram illustrating a system 100 having a first electronic device and a second electronic device for encoding and decoding video data, in accordance with one or more example implementations of this disclosure.

[0049] The system 100 includes a first electronic device 110, a second electronic device 120, and a communication medium 130.

[0050] The first electronic device 110 may be a source device including any device configured to encode video data and transmit the encoded video data to the communication medium 130. The second electronic device 120 may be a destination device including any device configured to receive encoded video data via the communication medium 130 and decode the encoded video data.

[0051] The first electronic device 110 may communicate via wire, or wirelessly, with the second electronic device 120 via the communication medium 130. The first electronic device 110 may include a source module 112, an encoder module 114, and a first interface 116, among other components. The second electronic device 120 may include a display module 122, a decoder module 124, and a second interface 126, among other components. The first electronic device 110 may be a video encoder and the second electronic device 120 may be a video decoder.

[0052] The first electronic device 110 and / or the second electronic device 120 may be a mobile phone, a tablet, a desktop, a notebook, or other electronic devices. FIG. 1 illustrates one example of the first electronic device 110 and the second electronic device 120. The first electronic device 110 and second electronic device 120 may include greater or fewer components than illustrated or have a different configuration of the various illustrated components.

[0053] The source module 112 may include a video capture device to capture new video, a video archive to store previously captured video, and / or a video feed interface to receive the video from a video content provider. The source module 112 may generate computer graphics-based data, as the source video, or may generate a combination of live video, archived video, and computer-generated video, as the source video. The video capture device may include a charge-coupled device (CCD) image sensor, a complementary metal-oxide-semiconductor (CMOS) image sensor, or a camera.

[0054] The encoder module 114 and the decoder module 124 may each be implemented as any one of a variety of suitable encoder / decoder circuitry, such as one or more microprocessors, a central processing unit (CPU), a graphics processing unit (GPU), a system-on-a-chip (SoC), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combinations thereof. When implemented partially in software, a device may store the program having computer-executable instructions for the software in a suitable, non-transitory computer-readable medium and execute the stored computer-executable instructions using one or more processors to perform the disclosed methods. Each of the encoder module 114 and the decoder module 124 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (CODEC) in a device.

[0055] The first interface 116 and the second interface 126 may utilize customized protocols or follow existing standards or de facto standards including, but not limited to, Ethernet, IEEE 802.11 or IEEE 802.15 series, wireless USB, or telecommunication standards including, but not limited to, Global System for Mobile Communications (GSM), Code-Division Multiple Access 2000 (CDMA2000), Time Division Synchronous Code Division Multiple Access (TD-SCDMA), Worldwide Interoperability for Microwave Access (WiMAX), Third Generation Partnership Project Long-Term Evolution (3GPP-LTE), or Time-Division LTE (TD-LTE). The first interface 116 and the second interface 126 may each include any device configured to transmit a compliant video bitstream via the communication medium 130 and to receive the compliant video bitstream via the communication medium 130.

[0056] The first interface 116 and the second interface 126 may include a computer system interface that enables a compliant video bitstream to be stored on a storage device or to be received from the storage device. For example, the first interface 116 and the second interface 126 may include a chipset supporting Peripheral Component Interconnect (PCI) and Peripheral Component Interconnect Express (PCIe) bus protocols, proprietary bus protocols, Universal Serial Bus (USB) protocols, Inter-Integrated Circuit (I2C) protocols, or any other logical and physical structure(s) that may be used to interconnect peer devices.

[0057] The display module 122 may include a display using liquid crystal display (LCD) technology, plasma display technology, organic light-emitting diode (OLED) display technology, or light-emitting polymer display (LPD) technology, with other display technologies used in some other implementations. The display module 122 may include a High-Definition display or an Ultra-High-Definition display.

[0058] FIG. 2 is a block diagram illustrating a decoder module 124 of the second electronic device 120 illustrated in FIG. 1, in accordance with one or more example implementations of this disclosure. The decoder module 124 may include an entropy decoder (e.g., an entropy decoding unit 2241), a prediction processor (e.g., a prediction processing unit 2242), an inverse quantization / inverse transform processor (e.g., an inverse quantization / inverse transform unit 2243), a summer (e.g., a summer 2244), a filter (e.g., a filtering unit 2245), and a decoded picture buffer (e.g., a decoded picture buffer 2246). The prediction processing unit 2242 further may include an intra prediction processor (e.g., an intra prediction unit 22421) and an inter prediction processor (e.g., an inter prediction unit 22422). The decoder module 124 receives a bitstream, decodes the bitstream, and outputs a decoded video.

[0059] The entropy decoding unit 2241 may receive the bitstream including multiple syntax elements from the second interface 126, as shown in FIG. 1, and perform a parsing operation on the bitstream to extract syntax elements from the bitstream. As part of the parsing operation, the entropy decoding unit 2241 may entropy decode the bitstream to generate quantized transform coefficients, quantization parameters, transform data, motion vectors, intra modes, partition information, and / or other syntax information.

[0060] The entropy decoding unit 2241 may perform context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding technique to generate the quantized transform coefficients. The entropy decoding unit 2241 may provide the quantized transform coefficients, the quantization parameters, and the transform data to the inverse quantization / inverse transform unit 2243 and provide the motion vectors, the intra modes, the partition information, and other syntax information to the prediction processing unit 2242.

[0061] The prediction processing unit 2242 may receive syntax elements, such as motion vectors, intra modes, partition information, and other syntax information, from the entropy decoding unit 2241. The prediction processing unit 2242 may receive the syntax elements including the partition information and divide image frames according to the partition information.

[0062] Each of the image frames may be divided into at least one image block according to the partition information. The at least one image block may include a luminance block for reconstructing multiple luminance samples and at least one chrominance block for reconstructing multiple chrominance samples. The luminance block and the at least one chrominance block may be further divided to generate macroblocks, coding tree units (CTUs), coding blocks (CBs), sub-divisions thereof, and / or other equivalent coding units.

[0063] During the decoding process, the prediction processing unit 2242 may receive predicted data including the intra mode or the motion vector for a current image block of a specific one of the image frames. The current image block may be the luminance block or one of the chrominance blocks in the specific image frame.

[0064] The intra prediction unit 22421 may perform intra-predictive coding of a current block unit relative to one or more neighboring blocks in the same frame, as the current block unit, based on syntax elements related to the intra mode in order to generate a predicted block. The intra mode may specify the location of reference samples selected from the neighboring blocks within the current frame. The intra prediction unit 22421 may reconstruct multiple chroma components of the current block unit based on multiple luma components of the current block unit when the multiple chroma components is reconstructed by the prediction processing unit 2242.

[0065] The intra prediction unit 22421 may reconstruct multiple chroma components of the current block unit based on the multiple luma components of the current block unit when the multiple luma components of the current block unit is reconstructed by the prediction processing unit 2242.

[0066] The inter prediction unit 22422 may perform inter-predictive coding of the current block unit relative to one or more blocks in one or more reference image blocks based on syntax elements related to the motion vector in order to generate the predicted block. The motion vector may indicate a displacement of the current block unit within the current image block relative to a reference block unit within the reference image block. The reference block unit may be a block determined to closely match the current block unit. The inter prediction unit 22422 may receive the reference image block stored in the decoded picture buffer 2246 and reconstruct the current block unit based on the received reference image blocks.

[0067] The inverse quantization / inverse transform unit 2243 may apply inverse quantization and inverse transformation to reconstruct the residual block in the pixel domain. The inverse quantization / inverse transform unit 2243 may apply inverse quantization to the residual quantized transform coefficient to generate a residual transform coefficient and then apply inverse transformation to the residual transform coefficient to generate the residual block in the pixel domain.

[0068] The inverse transformation may be inversely applied by the transformation process, such as a discrete cosine transform (DCT), a discrete sine transform (DST), an adaptive multiple transform (AMT), a mode-dependent non-separable secondary transform (MDNSST), a Hypercube-Givens transform (HyGT), a signal-dependent transform, a Karhunen-Loeve transform (KLT), a wavelet transform, an integer transform, a sub-band transform, or a conceptually similar transform. The inverse transformation may convert the residual information from a transform domain, such as a frequency domain, back to the pixel domain, etc. The degree of inverse quantization may be modified by adjusting a quantization parameter.

[0069] The summer 2244 may add the reconstructed residual block to the predicted block provided by the prediction processing unit 2242 to produce a reconstructed block.

[0070] The filtering unit 2245 may include a deblocking filter, a sample adaptive offset (SAO) filter, a bilateral filter, and / or an adaptive loop filter (ALF) to remove the blocking artifacts from the reconstructed block. Additional filters (in loop or post loop) may also be used in addition to the deblocking filter, the SAO filter, the bilateral filter, and the ALF. Such filters (are not explicitly illustrated for brevity of the description) may filter the output of the summer 2244. The filtering unit 2245 may output the decoded video to the display module 122 or other video receiving units after the filtering unit 2245 performs the filtering process for the reconstructed blocks of the specific image frame.

[0071] The decoded picture buffer 2246 may be a reference picture memory that stores the reference block to be used by the prediction processing unit 2242 in decoding the bitstream (e.g., in inter-coding modes). The decoded picture buffer 2246 may be formed by any one of a variety of memory devices, such as a dynamic random-access memory (DRAM), including synchronous DRAM (SDRAM), magneto-resistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The decoded picture buffer 2246 may be on-chip along with other components of the decoder module 124 or may be off-chip relative to those components.

[0072] FIG. 3 is a flowchart illustrating a method / process 300 for decoding and / or encoding video data by an electronic device, in accordance with one or more example implementations of this disclosure. The method / process 300 is an example implementation, as there may be a variety of mechanisms of decoding the video data.

[0073] The method / process 300 may be performed by an electronic device using the configurations illustrated in FIGS. 1 and / or 2, where various elements of these figures may be referenced to describe the method / process 300. Each block illustrated in FIG. 3 may represent one or more processes, methods, or subroutines performed by an electronic device.

[0074] The order in which the blocks appear in FIG. 3 is for illustration only, and may not be construed to limit the scope of the present disclosure, thus the order may be different from what is illustrated. Additional blocks may be added or fewer blocks may be utilized without departing from the scope of the present disclosure.

[0075] At block 310, the method / process 300 may start by receiving (e.g., via the decoder module 124, as shown in FIG. 2) the video data. The video data received by the decoder module 124 may include a bitstream.

[0076] With reference to FIGS. 1 and 2, the second electronic device 120 may receive the bitstream from an encoder, such as the first electronic device 110 (or other video providers) via the second interface 126.

[0077] At block 320, the decoder module 124 may determine a block unit from a current frame included in the video data.

[0078] With reference to FIGS. 1 and 2, the decoder module 124 may determine the image frames included in the bitstream when the video data received by the decoder module 124 includes the bitstream. The current frame may be one of the image frames determined, according to the bitstream. The decoder module 124 may further divide the current frame to determine the block unit, according to the partition indications in the bitstream. In some implementations, the decoder module 124 may divide the current frame to generate multiple CTUs, and may further divide a current CTU included in the CTUs to generate multiple divided blocks and to determine the block unit from the divided blocks, according to the partition indications (e.g., based on any video coding standard). In some other implementations, the decoder module 124 may divide the current frame to generate multiple slices, and further divide a current slice included in the slices to generate multiple CTUs. In addition, the decoder module 124 may further divide a current CTU included in the CTUs to generate multiple divided blocks and to determine the block unit from the divided blocks, according to the partition indications. The size of the block unit may be Wb×Hb. In some implementations, each of the Wb and Hb may be a positive integer (e.g., 4, 8, etc.) that may be equal to, or different from, each other.

[0079] At block 330, the decoder module 124 may determine, from the current frame, a reference block indicated by a block vector.

[0080] With reference to FIGS. 1 and 2, the decoder module 124 may derive the block vector in a reconstruction-reordered (RR) intra block copy (IBC) (RR-IBC) mode. Thus, the reference block indicated by the block vector may be determined using the RR-IBC mode.

[0081] The decoder module 124 may determine a vector candidate list for predicting the block unit. The vector candidate list may be derived based on multiple neighboring blocks neighboring the block unit. The neighboring blocks may be reconstructed prior to the reconstruction of the block unit. The neighboring blocks may include at least one of a first neighboring block, located above a top-right corner of the block unit, a second neighboring block, located at a left side of a bottom-left corner of the block unit, a third neighboring block, located at a top-right side of the block unit, a fourth neighboring block, located at a bottom-left side of the block unit, and a fifth neighboring block, located at a top-left side of the block unit. When a specific one of the neighboring blocks is reconstructed based on a neighboring block vector, the neighboring block vector of the specific neighboring block may be determined as a neighboring candidate vector of the block unit. The neighboring block vector may be generated for predicting or reconstructing the specific neighboring block using one of the IBC mode and an intra template matching prediction (IntraTMP) mode.

[0082] In some implementations, the decoder module 124 may determine an association type of the block unit from the video data. The association type may be a flip type for specifying which one of multiple flip directions is applied. In some implementations, the flip directions may include a horizontal flip and a vertical flip. When the flip directions only include the horizontal flip and the vertical flip, the video data may include an association flag for specifying which one of the horizontal flip and the vertical flip is applied. In some other implementations, the flip directions may further include other directional flips. When number of the flip directions is greater than two, the video data may include an association index for specifying which one of the flip directions is applied. Since both of the association flag and the association index are determined for the whole of the block unit, the association types, each corresponding to different neighboring candidate vectors of the block unit, may be identical to each other. In the RR-IBC mode, the association flag may be a flip flag.

[0083] In some other implementations, the association type of the block unit, corresponding to the neighboring block vector of the specific neighboring block, may be inherited from the association type of the specific neighboring block. Thus, when the association type of the specific neighboring block specifies the horizontal flip, the association type of the block unit, corresponding to the neighboring block vector of the specific neighboring block, may also be the horizontal flip. In addition, when the association type of the specific neighboring block specifies the vertical flip, the association type of the block unit, corresponding to the neighboring block vector of the specific neighboring block, may also be the vertical flip. Since different neighboring blocks may have different association types, the association types, each corresponding to different neighboring block vectors of the neighboring blocks, may also be different from each other.

[0084] The decoder module 124 may further compare the neighboring candidate vector with the corresponding association type, which may be the association type of the block unit indicated by the video data or inherited from the association type of the specific neighboring block. In some implementations, when a vector direction of the neighboring candidate vector is identical to the flip direction specified by the corresponding association type, the neighboring candidate vector may be determined as a reference candidate vector in the vector candidate list for deriving the block vector. For example, when the neighboring candidate vector is a horizontal vector and the corresponding association type specifies the horizontal flip, the neighboring candidate vector may be determined as the reference candidate vector in the vector candidate list. In addition, when the neighboring candidate vector is a vertical vector and the corresponding association type specifies the vertical flip, the neighboring candidate vector may also be determined as the reference candidate vector in the vector candidate list. In some other implementations, when the vector direction of the neighboring candidate vector is different from the flip direction specified by the corresponding association type, the neighboring candidate vector of the block unit may be excluded from the vector candidate list. In yet some other implementations, when the flip directions only include the horizontal flip and the vertical flip, the neighboring candidate vector, having the vector direction different from the horizontal direction and the vertical direction, may be excluded from the vector candidate list.

[0085] FIGS. 4A and 4B are schematic illustrations of a reference block of the block unit, indicated by a block vector, in accordance with one or more example implementations of this disclosure. FIG. 4A illustrates a reference block 410 of a block unit 400, indicated by a block vector 4012 derived based on a neighboring block vector 4011 of a neighboring block 401. The neighboring block vector 4011 of the neighboring block 401 may be used to indicate a neighboring reference 411 for predicting and / or reconstructing the neighboring block 401. The neighboring block vector 4011 of the neighboring block 401 may be a horizontal vector. Thus, when the association type corresponding to the neighboring block vector 4011 is the horizontal flip, the neighboring block vector 4011 may be determined as the reference candidate vector for deriving the block vector 4012. As shown in FIG. 4A, since the association type corresponding to the neighboring block vector 4011 is the horizontal flip, the block unit 400 and the reference block 410 are symmetric to each other along a vertical symmetry line 4013. Thus, the association type may also indicate an association between the block unit 400 and the reference block 410. As shown in FIG. 4A, the association between the block unit 400 and the reference block 410 may be a left-right symmetrical relationship.

[0086] The coordinates of a center point in the block unit 400 may be (Xc, Yc). When the horizontal coordinates of a center point in the neighboring block 401 is Xn and the neighboring block vector 4011 of the reference block 410 is (BVn, 0), the block vector 4012 of the block unit 400 may be (2Xn - 2Xc+BVn, 0). Thus, the coordinates of a center point in the reference block 410 may be (2Xn - Xc+BVn, Yc).

[0087] FIG. 4B illustrates a reference block 420 of the block unit 400, indicated by a block vector 4022 derived based on a neighboring block vector 4021 of the neighboring block 401. The neighboring block vector 4021 of the neighboring block 401 may be used to indicate a neighboring reference 421 for predicting and / or reconstructing the neighboring block 401. The neighboring block vector 4021 of the neighboring block 401 may be a vertical vector. Thus, when the association type corresponding to the neighboring block vector 4021 is the vertical flip, the neighboring block vector 4021 may be determined as the reference candidate vector for deriving the block vector 4022. As shown in FIG. 4B, since the association type corresponding to the neighboring block vector 4021 is the vertical flip, the block unit 400 and the reference block 420 are symmetric to each other along a horizontal symmetry line 4023. Thus, the association type may also indicate an association between the block unit 400 and the reference block 420. As shown in FIG. 4B, the association between the block unit 400 and the reference block 420 may be an upward-downward symmetrical relationship.

[0088] The coordinates of a center point in the block unit 400 may be (Xc, Yc). When the vertical coordinates of a center point in the neighboring block 401 is Yn and the neighboring block vector 4011 of the reference block 410 is (0, BVn), the block vector 4012 of the block unit 400 may be (0, 2Yn-2Yc+BVn). Thus, the coordinates of a center point in the reference block 410 may be (Xc, 2Yn-Yc+BVn).

[0089] In some implementations, when number of the reference candidate vectors is equal to one, the reference candidate vector may be directly determined as a reference vector for deriving the reference block. In some other implementations, when number of the reference candidate vectors is greater than one, one of the reference candidate vector may be selected as the reference vector by a candidate index determined from the video data.

[0090] Referring back to FIG. 3, at block 340, the decoder module 124 may determine, based on an association type, a corresponding relationship between a block template region, neighboring the block unit, and a reference template region, neighboring the reference block.

[0091] With reference to FIGS. 1 and 2, the decoder module 124 may determine the block template region neighboring the block unit in the current frame. The block template region may include at least one of a first block adjacent region, located at a left side of the block unit, a second block adjacent region, located above the block unit, and a third block adjacent region, located at a top-left side of the block unit. Since the first to the third block adjacent regions may be reconstructed prior to the reconstruction of the block unit, the block template region may be reconstructed prior to the reconstruction of the block unit. Thus, the decoder module 124 may be allowed to directly determine a reconstructed result of the block template region when the block unit is being predicted and / or reconstructed.

[0092] In some implementations, the size of the block template region may be determined based on the size of the block unit. In some other implementations, the size of the block template region may be predefined. For example, the height of the second block adjacent region and the width of the first block adjacent region may be equal to one, two, three, four, or other positive integers.

[0093] The decoder module 124 may determine the reference template region neighboring the reference block in the current frame based on the association type. The reference template region may include at least one of a first reference adjacent region, located at a left side of the reference block, a second reference adjacent region, located above the reference block, a third reference adjacent region, located at a right side of the reference block, a fourth reference adjacent region, located below the reference block, a fifth block adjacent region, located at a top-right side of the block unit, and a sixth block adjacent region, located at a bottom-left side of the block unit. In some implementations, when the association type specifies the horizontal flip, the reference template region may include at least one of the second reference adjacent region, located above the reference block, the third reference adjacent region, located at the right side of the reference block, and the fifth block adjacent region, located at the top-right side of the block unit. In some other implementations, when the association type specifies the vertical flip, the reference template region may include at least one of the first reference adjacent region, located at the left side of the reference block, the fourth reference adjacent region, located below the reference block, and the sixth block adjacent region, located at the bottom-left side of the block unit.

[0094] Since the first to the sixth reference adjacent regions may be reconstructed prior to the reconstruction of the block unit, the reference template region may be reconstructed prior to the reconstruction of the block unit. Thus, the decoder module 124 may be allowed to directly determine a reconstructed result of the reference template region when the block unit is being predicted and / or reconstructed. The size of the reference template region may be identical to the size of the block template region.

[0095] The corresponding relationship between the block template region and the reference template region may be determined based on the association type. In some implementations, when the association type specifies the horizontal flip, the first block adjacent region, located at the left side of the block unit, may correspond to the third reference adjacent region, located at the right side of the reference block. The second block adjacent region, located above the block unit, may correspond to the second reference adjacent region, located above the reference block. In addition, the third block adjacent region, located at the top-left side of the block unit may correspond to the fifth block adjacent region, located at the top-right side of the reference block when the block template region includes the third block adjacent region. In some other implementations, when the association type specifies the vertical flip, the first block adjacent region, located at the left side of the block unit, may correspond to the first reference adjacent region, located at the left side of the reference block. The second block adjacent region, located above the block unit may correspond to the fourth reference adjacent region, located below the reference block. In addition, the third block adjacent region, located at the top-left side of the block unit may correspond to the sixth block adjacent region, located at the bottom-left side of the reference block when the block template region includes the third block adjacent region.

[0096] FIGS. 5A and 5B are schematic illustrations of the corresponding relationship between the block template region and the reference template region, in accordance with one or more example implementations of this disclosure. FIG. 5A illustrates the corresponding relationship between a first block adjacent region 403 and a second block adjacent region 404 in the block template region and a third reference adjacent region 413 and a second reference adjacent region 414 in the reference template region. The first block adjacent region 403, located at the left side of the block unit 400, may correspond to the third reference adjacent region 413, located at the right side of the reference block 410. The second block adjacent region 404, located above the block unit 400, may correspond to the second reference adjacent region 414, located above the reference block 410.

[0097] Since the association type specifies the horizontal flip, as shown in FIG. 5A, the block template region and the block unit 400 may be symmetric to the reference template region and the reference block 410 along the vertical symmetry line 4013. In some implementations, a first block template sample 4031 in the first block adjacent region 403 may correspond to a first reference template sample 4131 in the third reference adjacent region 413. A second block template sample 4032 in the first block adjacent region 403 may correspond to a second reference template sample 4132 in the third reference adjacent region 413. In addition, the other block template samples in the first block adjacent region 403 may correspond to the other reference template samples in the third reference adjacent region 413 along the vertical symmetry line 4013. In some other implementations, a first block template sample 4041 in the second block adjacent region 404 may correspond to a first reference template sample 4141 in the second reference adjacent region 414. A second block template sample 4042 in the second block adjacent region 404 may correspond to a second reference template sample 4142 in the second reference adjacent region 414. In addition, the other block template samples in the second block adjacent region 404 may correspond to the other reference template samples in the second reference adjacent region 414 along the vertical symmetry line 4013.

[0098] The width of the first block adjacent region 403 and the third reference adjacent region 413 may be Wt, and the height of the second block adjacent region 404 and the second reference adjacent region 414 may be Ht. In some implementations, each of the Wt and Ht may be a positive integer (e.g., 2, 3, 4, etc.) that may be equal to, or different from, each other.

[0099] The coordinates of the first block template sample 4031, located at the top-left side of the first block adjacent region 403, may be set as T1(0,0). The coordinates of the second reference template sample 4132, located at the top-left side of the third reference adjacent region 413, may be set as T1’(0,0). Thus, the coordinates of the first reference template sample 4131 in the third reference adjacent region 413, corresponding to the first block template sample 4031 in the first block adjacent region 403, may be set as T1’(Wt-1, 0). In addition, the coordinates of the second block template sample 4032 in the first block adjacent region 403, corresponding to the second reference template sample 4132 in the third reference adjacent region 413, may be set as T1(Wt-1, 0). Furthermore, the coordinates of the block template samples in the bottom of the first block adjacent region 403 may be set as T1(0, Hb-1), …, T1(Wt-1, Hb-1) and the coordinates of the reference template samples in the bottom of the third reference adjacent region 413 may be set as T1’(0, Hb-1), …, T1’(Wt-1, Hb-1). The block template sample, located at T1(0, Hb-1), may correspond to the reference template sample, located at T1’(Wt-1, Hb-1), and the block template sample, located at T1(Wt-1, Hb-1), may correspond to the reference template sample, located at T1’(0, Hb-1).

[0100] The coordinates of the first block template sample 4041, located at the top-left side of the second block adjacent region 404 may be set as T2(0,0). The coordinates of the reference template sample, located at the top-left side of the second reference adjacent region 414 may be set as T2’(0,0). Thus, the coordinates of the first reference template sample 4141 in the second reference adjacent region 414, corresponding to the first block template sample 4041 in the second block adjacent region 404, may be set as T2’(Wb-1, 0). Furthermore, the coordinates of the block template samples, located at the right side of the second block adjacent region 404, may be set as T2(Wb-1,0), …, T2(Wb-1,Ht-1). The coordinates of the reference template samples, located at the left side of the second reference adjacent region 414, may be set as T2’(0,0), …, T2’(0, Ht-1). The block template sample, located at T2(Wb-1,0), may correspond to the reference template sample, located at T2’(0,0). The block template sample, located at T2(Wb-1,Ht-1), may correspond to the reference template sample, located at T2’(0, Ht-1).

[0101] FIG. 5B illustrates the corresponding relationship between the first block adjacent region 403 and the second block adjacent region 404 in the block template region and a first reference adjacent region 423 and a fourth reference adjacent region 424 in the reference template region. The first block adjacent region 403, located at the left side of the block unit 400, may correspond to the first reference adjacent region 423, located at the left side of the reference block 420. The second block adjacent region 404, located above the block unit 400, may correspond to the fourth reference adjacent region 424, located below the reference block 420.

[0102] Since the association type specifies the vertical flip, as shown in FIG. 5B, the block template region and the block unit 400 may be symmetric to the reference template region and the reference block 420 along the horizontal symmetry line 4023. In some implementations, the first block template sample 4031 in the first block adjacent region 403 may correspond to a first reference template sample 4231 in the first reference adjacent region 423. The second block template sample 4032 in the first block adjacent region 403 may correspond to a second reference template sample 4232 in the first reference adjacent region 423. In addition, the other block template samples in the first block adjacent region 403 may correspond to the other reference template samples in the first reference adjacent region 423 along the horizontal symmetry line 4023. In some other implementations, the first block template sample 4041 in the second block adjacent region 404 may correspond to a first reference template sample 4241 in the fourth reference adjacent region 424. The second block template sample 4042 in the second block adjacent region 404 may correspond to a second reference template sample 4242 in the fourth reference adjacent region 424. In addition, the other block template samples in the second block adjacent region 404 may correspond to the other reference template samples in the fourth reference adjacent region 424 along the horizontal symmetry line 4023.

[0103] The coordinates of the reference template sample located at the top-left side of the first reference adjacent region 423 may be set as T1’’(0,0). Thus, the coordinates of the first reference template sample 4231 in the first reference adjacent region 423, corresponding to the first block template sample 4031 in the first block adjacent region 403, may be set as T1’’(0,Hb-1) . In addition, the coordinates of the second reference template sample 4232 in the first reference adjacent region 423, corresponding to the second block template sample 4032 in the first block adjacent region 403, may be set as T1’’(Wt-1, Hb-1).

[0104] The coordinates of the reference template sample located at the top-left side of the fourth reference adjacent region 424 may be set as T2’’(0,0). Thus, the coordinates of the first reference template sample 4241 in the fourth reference adjacent region 424, corresponding to the first block template sample 4041 in the second block adjacent region 404, may be set as T2’’(0,Ht-1). In addition, the coordinates of the second reference template sample 4232 in the fourth reference adjacent region 424, corresponding to the second block template sample 4042 in the second block adjacent region 404, may be set as T2’’(1,Ht-1).

[0105] Referring back to FIG. 3, at block 350, the decoder module 124 may derive a prediction model of the block unit based on the corresponding relationship between block template region and the reference template region

[0106] The prediction model may be selected from multiple candidate models including at least one of multiple linear candidate models and multiple non-linear candidate models. A linear candidate model may include an Nm-tap linear model. The number Nm, being a positive integer, may be predefined. For example, the number Nm may be equal to 1, 2, 3, 4, 5, 7, or 9. The non-linear candidate model may include a quadratic candidate model, a cubic candidate model, and a quartic candidate model. In some implementations, when a degree of a candidate model is equal to one, the candidate model may be one of the linear candidate models. In some other implementations, when a degree of a candidate model is greater than one, the candidate model may be one of the non-linear candidate models. For example, since the quartic candidate model includes a square term, the quartic candidate model may be included in the non-linear candidate modes. It should be noted that additional candidate models may be added without departing from the scope of the present disclosure.

[0107] In some implementations, the non-linear candidate models may include a quadratic candidate model. The quadratic candidate model including five spatial sample terms, a non-linear term P, and one bias term B may be similar to a convolutional cross-component model (CCCM). The five spatial sample terms may include a center sample C, a north sample N, a south sample S, a west sample W, and an east sample E.

[0108] The center sample C may be located at a sample position T’(i,j) or T’’(i,j) relative to a top-left reference template sample in a corresponding one of the reference adjacent region including the center sample C. The north sample N, located above the center sample C, may be located at a sample position T’(i,j-1) or T’’(i,j-1) relative to the top-left reference template sample in the corresponding reference adjacent region. The south sample S, located below the center sample C, may be located at a sample position T’(i,j+1) or T’’(i,j+1) relative to the top-left reference template sample in the corresponding reference adjacent region. The west sample W, located on the left side of the center sample C, may be located at a sample position T’(i-1,j) or T’’(i-1,j) relative to the top-left reference template sample in the corresponding reference adjacent region. The east sample E, located on the right side of the center sample C, may be located at a sample position T’(i+1,j) or T’’(i+1,j) relative to the top-left reference template sample in the corresponding reference adjacent region. Thus, the quadratic model may be shown in the following functions: 　　　 predLumaVal = C0× C + C1× N + C2× S + C3× E + C4× W + C5× P + C6× B 　　　　　　　　　　　　　　　　　　　　 P = (C× C+midVal) >> bitDepth 　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　B = 2bitDepth-1 where the coefficients C0, C1, C2, C3, C4, C5, and C6are seven prediction coefficients for the spatial sample terms C, N, S, W, and E, the non-linear term P, and the bias term B. The parameter midVal and the bias term B may be set as to a middle luma value, the parameter bitDepth may be a bit depth of the samples in the bitstream. For example, the bias term B may be set as 512 for 10 bits content.

[0109] When the prediction model is derived, the value predLumaVal may be a predicted luma value of a specific block template sample determined based on the corresponding relationship. The specific block template sample may be included in a block adjacent region corresponding to the reference adjacent region including the center sample C, which is currently applied for deriving the prediction model. Thus, the block template sample having the value predLumaVal may be located at a sample position T(m,n) relative to the top-left block template sample of the corresponding block adjacent region. For example, with reference to FIG. 5A, when the first reference template sample 4141, located at T2’(Wb-1, 0), is determined as the center sample C, the first block template sample 4041, located at T2(0, 0), may correspond to the first reference template sample 4141 to derive the prediction model.

[0110] In some other implementations, the linear candidate models may include a one-tap candidate model. The one-tap candidate model may include a spatial sample term and an offset term. The spatial sample term C may include a reference sample in the corresponding reference template region. Thus, the one-tap candidate model may be shown in the following function: 　　　　　　　　　　　　　　　　　　　　　　　 predLumaVal = C0× C + C1 where the coefficients C0and C1are two prediction coefficients. The coefficient C1may also be determined as the offset term. When the prediction model is derived, the value predLumaVal may be a predicted luma value of a block template sample determined based on the corresponding relationship. The block template sample may be included in a block adjacent region corresponding to the reference adjacent region including the center sample C, which is currently applied for deriving the prediction model.

[0111] The prediction coefficients may be derived by comparing previously reconstructed luma values in the block template region with the predicted values predLumaVal in the block template region. In some implementations, the difference minimization of the comparison may be performed by an MSE minimization. In some implementations, the MSE minimization may be performed by calculating an autocorrelation matrix. The autocorrelation matrix may be LDL decomposition and the convolution filter coefficients may be calculated using back-substitution. In some implementations, the decomposition may be Cholesky decomposition. In some implementations, the difference minimization may be performed by a Gaussian elimination. It should be noted that the derivation scheme for the set of prediction coefficients may be changed without departing from the scope of the present disclosure.

[0112] With reference to FIGS. 1 and 2, in order to derive the prediction model, the decoder module 124 may flip the reference template region in advance to correspond to the block template region based on the association type. For example, the association type may specify the horizontal flip. FIGS. 6A-6C are schematic illustrations of a horizontal flipping relationship between the block template region and the reference template region, in accordance with one or more example implementations of this disclosure. FIG. 6A illustrates the corresponding relationship between the first block adjacent region 403 and the second block adjacent region 404 in the block template region and the third reference adjacent region 413 and the second reference adjacent region 414 in the reference template region. The first block adjacent region 403 may be symmetric to the third reference adjacent region 413 along the vertical symmetry line 4013. The second block adjacent region 404 may be symmetric to the second reference adjacent region 414 along the vertical symmetry line 4013. In addition, the block unit 400 may also be symmetric to the reference block 410 along the vertical symmetry line 4013.

[0113] In some implementations, in order to easily derive the prediction model, the decoder module 124 may flip the reference template region in advance along a vertical center line of the reference block 410. FIG. 6B illustrates a horizontal flipping relationship in the decoder module 124 between the first block adjacent region 403 and the second block adjacent region 404 in the block template region and a flipped third reference adjacent region 413’ and a flipped second reference adjacent region 414’ in the reference template region. Thus, the flipped third reference adjacent region 413’ may be located at the left side of a flipped reference block 410’. In addition, multiple flipped reference template samples in the flipped third reference template region 413’ may be symmetric to multiple original reference template samples in the third reference template region 413 along a vertical center line 4014 of the reference block 410. Multiple flipped reference template samples in the flipped second reference template region 414’ may also be symmetric to multiple original reference template samples in the second reference template region 414 along the vertical center line 4014 of the reference block 410. Thus, a location relationship between the flipped reference block 410’ and the flipped reference template region may be identical to a location relationship between the block unit 400 and the block template region for deriving the prediction model. In some other implementations, a location relationship between a flipped reference block and a flipped reference template region, generated by flipped the reference block 420 and the reference template region along a horizontal center line of the reference block 420, may be identical to a location relationship between the block unit 400 and the block template region.

[0114] In some implementations, in order to easily derive the prediction model, the encoder module 114 may flip the block template region in advance along a vertical center line of the block unit 400. FIG. 6C illustrates a horizontal flipping relationship in the encoder module 114 between a flipped first block adjacent region 403’ and a flipped second block adjacent region 404’ in the block template region and the third reference adjacent region 413 and the second reference adjacent region 414 in the reference template region. Thus, the flipped first block adjacent region 403’ may be located at the right side of a flipped block unit 400’. In addition, multiple flipped block template samples in the flipped block template region 403’ may be symmetric to multiple original block template samples in the block template region 403 along a vertical center line 4015 of the block unit 400. Multiple flipped block template samples in the flipped second block template region 404’ may also be symmetric to multiple original block template samples in the second block template region 404 along the vertical center line 4015 of the reference block 410. Thus, a location relationship between the flipped block unit 400’ and the flipped block template region may be identical to a location relationship between the reference unit 410 and the reference template region for deriving the prediction model. In some other implementations, a location relationship between a flipped block unit and a flipped block template region, generated by flipping the block unit 400 and the block template region along a horizontal center line of the block unit 400, may be identical to a location relationship between the reference block 420 and the reference template region.

[0115] Referring back to FIG. 3, at block 360, the decoder module 124 may reconstruct the block unit based on the reference block using the prediction model with the association type.

[0116] With reference to FIGS. 1 and 2, in some implementations, the decoder module 124 may predict the block unit based on the reference block using the prediction model with the association type to generate a predicted block. When a specific reference sample in the reference block is applied to the prediction model as the center sample C, a location of a corresponding block sample, having the predicted luma value predLumaVal generated based on the specific reference sample, may be determined based on the corresponding relationship.

[0117] When the specific reference sample in the reference block is applied to the prediction model as the center sample C, the predicted luma value of the corresponding block sample may be generated. With reference to FIGS. 5A and 5B, the coordinates of a first block sample 4001 located at the top-left side of the block unit 400 may be set as T0(0, 0). The coordinates of a reference sample located at the top-left side of the reference block 410 may be set as T0’(0, 0). The coordinates of a reference sample located at the top-left side of the reference block 420 may be set as T0’’(0, 0). When the specific reference sample in the reference block is applied to the prediction model as the center sample C, the location of the center sample C may be located at a sample position T0’(i, j) or T0’’(i, j) relative to the top-left reference template sample in the corresponding reference block. In addition, the block sample having the value predLumaVal may be located at a sample position T(m, n) relative to the top-left block template sample of the block unit. For example, with reference to FIG. 5A, when the first reference sample 4101, located at T0’(Wb-1, 0) is determined as the center sample C, the value predLumaVal of the first block sample 4001, located at T0(0, 0), may be generated. The location of the first block sample 4001 may be determined based on the corresponding relationship.

[0118] In order to generate the predicted block, the decoder module 124 may flip the reference block in advance to correspond to the block unit based on the association type. For example, the association type may specify the horizontal flip. With reference to FIGS. 6A and 6B, multiple flipped reference samples in the flipped reference block 410’ may be symmetric to multiple original reference samples in the reference block 410 along the vertical center line 4014 of the reference block 410. Thus, a location relationship between the flipped reference samples in the flipped reference block 410’ may be identical to a location relationship between multiple sample values in the block unit 400 for generating the predicted block. In some other implementations, a location relationship between the flipped reference samples in the flipped reference block, generated by flipping the reference block 420 along the horizontal center line, may be identical to the location relationship between the sample values in the block unit 400 for generating the predicted block.

[0119] The decoder module 124 may determine multiple residual components from the bitstream for the block unit and add the residual components into the predicted block to reconstruct the block unit. The decoder module 124 may reconstruct all of the other block units in the image frame for reconstructing the image frame and the video.

[0120] In some implementations, a block-vector-based local illumination compensation (LIC) flag of the block unit may be determined from the video data for indicating whether the reference template region and the block template region are used to derive the prediction model for predicting or reconstructing the block unit. For example, the block-vector-based LIC flag of the block unit may be determined from the video data when the neighboring block vector of the neighboring block, used to determine the reference template region and the reference block, is generated using an IBC merge mode or an IBC advanced motion vector prediction (AMVP) mode. In some other implementations, the block-vector-based LIC flag of the block unit may be inherited from the specific neighboring block when the neighboring block vector of the specific neighboring block, used to determine the reference template region and the reference block, is generated using the IBC merge mode.

[0121] In some implementations, an RR-IBC flag may be determined from the bitstream for indicating whether the reference block indicated by the block vector is determined using the RR-IBC mode. Then, when the RR-IBC flag has been determined as true, an IBC-LIC flag may be further determined from the bitstream for indicating whether the reference template region, generated using the RR-IBC mode, and the block template region are used to derive the prediction model for predicting or reconstructing the block unit. In some other implementations, the IBC-LIC flag may be determined from the bitstream for indicating whether the reference template region and the block template region are used to derive the prediction model for predicting or reconstructing the block unit. Then, when the IBC-LIC flag has been determined as true, the RR-IBC flag may be further determined from the bitstream for indicating whether the block vector, used for determining the reference template region to generate the prediction model, is determined using the RR-IBC mode.

[0122] In some implementations, a template type syntax may be determined from the bitstream for indicating the shape of the block template region and the neighboring template region. For examples, the template type syntax may include a template type index for directly indicating which one or ones of the first to the third block adjacent region is included in the block template region. In addition, the template type syntax may include more than one template type flag to determine the block template region. For example, a first template type flag may be used for determining whether both of the first block adjacent region and the second block adjacent region are included in the block template region. When the first template type flag is false, a second template type flag may be used for determining which one of the first block adjacent region and the second block adjacent region is included in the block template region. When both the RR-IBC flag and the IBC-LIC flag have been determined as true, the template type syntax may be further determined from the bitstream. When at least one of the RR-IBC flag and the IBC-LIC flag have been determined as false, the template type syntax may be excluded from the bitstream.

[0123] The method / process 300 may then end.

[0124] FIG. 7 is a flowchart illustrating a method / process 700 for decoding and / or encoding video data by an electronic device, in accordance with one or more example implementations of this disclosure. The method / process 700 is an example implementation, as there may be a variety of mechanisms of decoding the video data.

[0125] The method / process 700 may be performed by an electronic device using the configurations illustrated in FIGS. 1 and / or 2, where various elements of these figures may be referenced to describe the method / process 700. Each block illustrated in FIG. 7 may represent one or more processes, methods, or subroutines performed by an electronic device.

[0126] The order in which the blocks appear in FIG. 7 is for illustration only, and may not be construed to limit the scope of the present disclosure, thus the order may be different from what is illustrated. Additional blocks may be added or fewer blocks may be utilized without departing from the scope of the present disclosure.

[0127] At block 710, the method / process 700 may start by receiving (e.g., via the decoder module 124, as shown in FIG. 2) the video data. The video data received by the decoder module 124 may include a bitstream.

[0128] With reference to FIGS. 1 and 2, the second electronic device 120 may receive the bitstream from an encoder, such as the first electronic device 110 (or other video providers) via the second interface 126.

[0129] At block 720, the decoder module 124 may determine a block unit from a current frame included in the video data.

[0130] With reference to FIGS. 1 and 2, the decoder module 124 may determine the image frames included in the bitstream when the video data received by the decoder module 124 includes the bitstream. The current frame may be one of the image frames determined, according to the bitstream. The decoder module 124 may further divide the current frame to determine the block unit, according to the partition indications in the bitstream. In some implementations, the decoder module 124 may divide the current frame to generate multiple CTUs, and may further divide a current CTU included in the CTUs to generate multiple divided blocks and to determine the block unit from the divided blocks, according to the partition indications (e.g., based on any video coding standard). In some other implementations, the decoder module 124 may divide the current frame to generate multiple slices, and further divide a current slice included in the slices to generate multiple CTUs. In addition, the decoder module 124 may further divide a current CTU included in the CTUs to generate multiple divided blocks and to determine the block unit from the divided blocks, according to the partition indications. The size of the block unit may be Wb×Hb. In some implementations, each of the Wb and Hb may be a positive integer (e.g., 4, 8, etc.) that may be equal to, or different from, each other.

[0131] At block 730, the decoder module 124 may determine, from the current frame, multiple vector reference blocks of the block unit, indicated by multiple block vector candidates of the block unit. In some implementations, each of the vector reference blocks may be indicated by a corresponding one of the block vector candidates of the block unit.

[0132] With reference to FIGS. 1 and 2, the decoder module 124 may determine a block vector list including the block vector candidates of the block unit for predicting the block unit. In some implementations, the decoder module 124 may determine multiple neighboring blocks neighboring the block unit. The decoder module 124 may further determine whether the neighboring blocks are predicted and / or reconstructed based on multiple neighboring block vectors. The neighboring block vectors may be generated for predicting or reconstructing the neighboring blocks using one of an intra block copy (IBC) mode and an intra template matching prediction (IntraTMP) mode.

[0133] When a specific neighboring block is predicted and / or reconstructed based on a specific neighboring block vector, the specific neighboring block vector may indicate a neighboring reference block in the current frame for reconstructing the specific neighboring block. In other words, each of the neighboring block vectors may indicate a corresponding neighboring reference block in the current frame for reconstructing a corresponding neighboring block. In addition, the neighboring block vectors may be determined as multiple block vector references for determining the block vector candidates in the block vector list.

[0134] The neighboring blocks may be reconstructed prior to the reconstruction of the block unit. The neighboring blocks may include at least one of a first neighboring block, located above a top-right corner of the block unit, a second neighboring block, located at a left side of a bottom-left corner of the block unit, a third neighboring block, located at a top-right side of the block unit, a fourth neighboring block, located at a bottom-left side of the block unit, and a fifth neighboring block, located at a top-left side of the block unit.

[0135] In some other implementations, the block vector references may be generated based on multiple history-based block vectors. For example, multiple previous blocks reconstructed prior to the reconstruction of the block unit may be reconstructed based on multiple reconstruction schemes. When one of the previous blocks is reconstructed using a block vector, the block vector of the previous block may be stored in a block-vector-table on a first-in-first-out (FIFO) basis. The size of the block-vector-table may be equal to Nt. In some implementations, Nt may be a positive integer, such as 6 or 12. In yet some other implementation, the block vector references may include multiple pairwise-average block vectors generated based on two of the other block vector references, and multiple padding block vectors generated using a padding process.

[0136] The decoder module 124 may determine the block vector candidates based on the block vector references including the neighboring block vectors. Then, the decoder module 124 may determine the vector reference blocks indicated by the block vector candidates. In some implementations, when the block unit is allowed to be predicted using the IBC merge mode, the block vector references may be directly determined as the block vector candidates. In some other implementations, when the block unit is allowed to be predicted based on an IBC advanced motion vector prediction (AMVP) mode, the block vector references may be combined with multiple block vector differences, and the combined results may be added into the block vector candidates.

[0137] Referring back to FIG. 7, at block 740, the decoder module 124 may derive multiple filter models of the block unit based on the vector reference blocks. In some implementations, each of the filter models may be derived based on a corresponding one of the vector reference blocks.

[0138] With reference to FIGS. 1 and 2, the decoder module 124 may determine a first block template region neighboring the block unit in the current frame. The first block template region may include at least one of a first block adjacent region, located at a left side of the block unit, a second block adjacent region, located above the block unit, and a third block adjacent region, located at a top-left side of the block unit. Since the first to the third block adjacent regions may be reconstructed prior to the reconstruction of the block unit, the first block template region may be reconstructed prior to the reconstruction of the block unit. Thus, the decoder module 124 may be allowed to directly determine a reconstructed result of the first block template region when the block unit is being predicted and / or reconstructed.

[0139] In some implementations, the size of the first block template region may be determined based on the size of the block unit. In some implementations, the size of the first block template region may be predefined. For example, the height of the second block adjacent region and the width of the first block adjacent region may be equal to one, two, three, four, or other positive integers. In some implementations, the derivation of the size of the first block template region in method / process 700 may be identical to that in a block vector search process of an IBC search method.

[0140] In some other implementations, the size of the first block template region may be determined based on the shapes of the filter shapes used at block 740. For example, when the candidate model for deriving the filter models is the quadratic candidate model including five spatial sample terms C, N, S, E, and W, the size of the first block template region may be equal to, or less than the height of the five spatial sample terms C, N, S, E, and W. Thus, the size of the first block template region may be equal to one, two, or three. In yet some other implementations, the size of the first block template region may be different from the size of other block template regions for different prediction stages. For example, the size of the first block template region used for deriving the filter models to reorder the filter models may be different from the size of the block template region used for deriving the linear filter model used in motion compensation stage of inter prediction.

[0141] The decoder module 124 may determine a first reference template region in the current frame for each of the vector reference blocks. Each of the first reference template regions may neighbor a corresponding one of the vector reference blocks. Each of the first reference template regions may include at least one of a first reference adjacent region, located at a left side of a corresponding vector reference block, a second reference adjacent region, located above the corresponding vector reference block, and a third reference adjacent region, located at a top-left side of the corresponding vector reference block. Since the first to the third reference adjacent regions of the vector reference blocks may be reconstructed prior to the reconstruction of the block unit, the first reference template regions of the vector reference blocks may be reconstructed prior to the reconstruction of the block unit. Thus, the decoder module 124 may be allowed to directly determine multiple reconstructed results of the first reference template regions when the block unit is being predicted and / or reconstructed. The sizes of the first reference template regions may be identical to the size of the first block template region.

[0142] The filter models may be selected from multiple candidate models including at least one of multiple linear candidate models and multiple non-linear candidate models. A linear candidate model may include an Nm-tap linear model. The number Nm, being a positive integer, may be predefined. For example, the number Nm may be equal to 1, 2, 3, 4, 5, 7, or 9. The non-linear candidate model may include a quadratic candidate model, a cubic candidate model, and a quartic candidate model. In some implementations, when a degree of a candidate model is equal to one, the candidate model may be one of the linear candidate models. In some other implementations, when a degree of a candidate model is greater than one, the candidate model may be one of the non-linear candidate models. For example, since the quartic candidate model includes a square term, the quartic candidate model may be included in the non-linear candidate modes. It should be noted that additional candidate models may be added without departing from the scope of the present disclosure. Each of the filter models may be derived further based on the first block template region and a corresponding one of the first reference template regions.

[0143] In some implementations, the non-linear candidate models may include a quadratic candidate model. The quadratic candidate model including five spatial sample terms, a non-linear term P, and one bias term B may be similar to a convolutional cross-component model (CCCM). The five spatial sample terms may include a center sample C located at a sample position (i, j) relative to the top-left luma sample of a specific vector reference block, a north sample N located at a sample position (i, j-1) relative to the top-left luma sample of the specific vector reference block and located above the center sample C, and a south sample S located at a sample position (i, j+1) relative to the top-left luma sample of the specific vector reference block and located below the center sample C. In addition, the five spatial sample terms may further include a west sample W located at a sample position (i-1, j) relative to the top-left luma sample of the specific vector reference block and located on the left side of the center sample C, and an east sample E located at a sample position (i+1, j) relative to the top-left luma sample of the specific vector reference block and located on the right side of the center sample C. Thus, the 5-tap linear model may be shown in the following function: 　　　　predLumaVal = C0× C + C1× N + C2× S + C3× E + C4× W + C5× P + C6× B 　　　　　　　　　　　　　　　　　　　　　　 P = (C× C+midVal) >> bitDepth 　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　B = 2bitDepth-1 where the coefficients C0, C1, C2, C3, C4, C5, and C6are seven filter coefficients for the spatial sample terms C, N, S, W, and E, the non-linear term P, and the bias term B, and predLumaVal may be a predicted luma sample located at a sample position (i, j) relative to the top-left luma sample of the block unit. In addition, the parameter midVal and the bias term B may be set as to a middle luma value, the parameter bitDepth may be a bit depth of the samples in the bitstream. For example, the bias term B may be set as 512 for 10 bits content.

[0144] When the decoder module 124 derives the filter models of the block unit, the decoder module 124 may determine whether the spatial sample terms are included in the first reference template regions or not. Thus, when a specific one of the spatial sample terms is excluded from a specific first reference template region, the specific spatial sample term may be padded by copying a sample adjacent to the specific spatial sample term, included in the specific first reference template region, to replace the excluded term. For example, when the center sample C is located at a top line of a specific first reference template region, the north sample N may be excluded from the specific first reference template region. Since the center sample C adjacent to the north sample N is included in the specific first reference template region, the north sample N may be padded by copying the center sample C to replace the excluded north sample N.

[0145] In some other implementations, the linear candidate models may include a one-tap candidate model. The one-tap candidate model may include a spatial sample term and an offset term. The spatial sample term C may include a reference sample in the corresponding first reference template region. Thus, the one-tap candidate model may be shown in the following function: 　　　　　　　　　　　　　　　　　　　　　　　 predLumaVal = C0× C + C1 where the coefficients C0and C1are two prediction coefficients, and predLumaVal may be a predicted luma sample located at a sample position (i, j) relative to the top-left luma sample of the block unit. The coefficient C1may also be determined as the offset term.

[0146] FIGS. 8A and 8B are schematic illustrations of the relative location between the block unit and the first block template region and the relative location between the reference block and the first reference template region, in accordance with one or more example implementations of this disclosure. FIG. 8A illustrates a first block adjacent region 801 and a second block adjacent region 802 in the first block template region of the block unit 800 and a first reference adjacent region 811 and a second reference adjacent region 812 in the first reference template region of the reference block 810. The first block adjacent region 801 and the second block adjacent region 802 may be directly adjacent to the block unit 800, and the first reference adjacent region 811 and the second reference adjacent region 812 may be directly adjacent to the reference block 810.

[0147] In some implementations, the decoder module 124 may pad the south sample S by copying the center sample C during deriving the filter models of the block unit when the south sample S is included in the reference block 810 and the center sample C is adjacent to the reference block 810. In addition, the decoder module 124 may pad the east sample E by copying the center sample C during deriving the filter models of the block unit when the east sample E is included in the reference block 810 and the center sample C is adjacent to the reference block 810. In some other implementations, the decoder module 124 may directly use the reconstructed value of the south sample S without padding when the south sample S is included in the reference block 810. In addition, the decoder module 124 may directly use the reconstructed value of the east sample E without padding when the east sample E is included in the reference block 810.

[0148] FIG. 8B illustrates a first block neighboring region 803 and a second block neighboring region 804 of the block unit 800 and a first reference neighboring region 813 and a second reference neighboring region 814 of the reference block 810. The first block neighboring region 803 and the second block neighboring region 804 may be separated from and neighbor the block unit 800. In addition, the first reference neighboring region 813 and the second reference neighboring region 814 may be separated from and neighbor the reference block 810. In order to derive the filter models, the first block neighboring region 803 and the second block neighboring region 804 may be set as the first block adjacent region and the second block adjacent region of the first block template region. In addition, the first reference neighboring region 813 and the second reference neighboring region 814 may be set as the first reference adjacent region and the second reference adjacent region of the first reference template region. Thus, the south sample S may not be included in the reference block 810 even if the center sample C is located at the bottom line of the second reference neighboring region 814. In addition, the east sample E may not be included in the reference block 810 even if the east sample E is located at the right line of the first reference neighboring region 813.

[0149] The prediction coefficients may be derived by comparing previously reconstructed luma values in the first block template region with the predicted values predLumaVal in the first block template region. In some implementations, the difference minimization of the comparison may be performed by an MSE minimization. In some implementations, the MSE minimization may be performed by calculating an autocorrelation matrix. The autocorrelation matrix may be LDL decomposition and the convolution filter coefficients may be calculated using back-substitution. In some implementations, the decomposition may be Cholesky decomposition. In some implementations, the difference minimization may be performed by a Gaussian elimination. It should be noted that the derivation scheme for the set of prediction coefficients may be changed without departing from the scope of the present disclosure.

[0150] In yet some other implementations, the decoder module 124 may derive the filter models of the block unit using the method / process 300. In other words, the first block template region may correspond to multiple flipped first reference template regions. Thus, the decoder module 124 may flip the first block template region or flip the first reference template regions to derive the filter models for the vector reference blocks. In some implementations, when the number of the vector reference block is equal to one, the decoder module 124 may flip the first reference template region of the vector reference block in advance to correspond to the first block template region based on an association type in the method / process 300. In some other implementations, when the number of the vector reference blocks is greater than one, the decoder module 124 may flip the first block template region of the block unit in advance to simultaneously correspond to the first reference template regions based on the association type in the method / process 300. Although the first reference template regions in the derivation process including the flipping process are different from those in the derivation process without the flipping process, the derivation of the prediction coefficients in the derivation process including the flipping process may be identical to that in the derivation process without the flipping process.

[0151] The decoder module 124 may determine a block-vector-based candidate list including multiple block-vector-based prediction candidates based on the filter models for predicting the block unit. The block-vector-based candidate list may include at least one of multiple first block-vector-based candidates, multiple second block-vector-based candidates, multiple third block-vector-based candidates, or multiple fourth block-vector-based candidates. Each of the first block-vector-based candidates may correspond to a corresponding one of the block vector candidates and a corresponding one of the filter models. Each of the second block-vector-based candidates may only correspond to a corresponding one of the block vector candidates without using the filter models. Each of the third block-vector-based candidates may correspond to a corresponding one of the block vector candidates and a corresponding one of the filter models derived using the method / process 300. Each of the fourth block-vector-based candidates may only correspond to a corresponding one of the block vector candidates without using the filter models. In some implementations, the fourth block-vector-based candidates may be generated by flipping the reference template regions, and the second block-vector-based candidates may be generated without flipping the reference template regions. In some implementations, the block-vector-based candidate list may only include the first block-vector-based candidates, such that each of the block-vector-based prediction candidates may correspond to a corresponding block vector candidate and a corresponding filter model.

[0152] Referring back to FIG. 7, at block 750, the decoder module 124 may determine multiple template matching costs calculated using the filter models of the block unit. In some implementations, the template matching costs may include multiple first template costs, and each of the first template costs may be calculated using a corresponding one of the filter models of the block unit.

[0153] With reference to FIGS. 1 and 2, the decoder module 124 may determine a second block template region neighboring the block unit in the current frame. The second block template region may include at least one of a fourth block adjacent region, located at a left side of the block unit, a fifth block adjacent region, located above the block unit, and a sixth block adjacent region, located at a top-left side of the block unit. Since the fourth to the sixth block adjacent regions may be reconstructed prior to the reconstruction of the block unit, the second block template region may be reconstructed prior to the reconstruction of the block unit. Thus, the decoder module 124 may be allowed to directly determine a reconstructed result of the second block template region when the block unit is being predicted and / or reconstructed.

[0154] In some implementations, the size of the second block template region may be determined based on the size of the block unit. In some other implementations, the size of the second block template region may be predefined. For example, the height of the fifth block adjacent region and the width of the fourth block adjacent region may be equal to one, two, three, four, or other positive integers.

[0155] In yet some other implementations, the size of the second block template region may be determined based on the size of the first block template region. For example, the second block template region may be included in the first block template region. In other words, the second block template region may not exceed the first block template region. In some implementations, the second block template region may be totally identical to the first block template region. Thus, the size of the second block template region may be also identical to that of the first block template region. For example, when the size of the first block template region is equal to four, the size of the second block template region may also be equal to four. In addition, as shown in FIGS. 8A and 8B, the second block template region may be separated from the block unit 800. In some other implementations, the second block template region may be different from the first block template region. For example, when the size of the first block template region is equal to four, the size of the second block template region may be equal to one, two, or three, less than the size of the first block template region.

[0156] The decoder module 124 may determine a second reference template region in the current frame for each of the vector reference blocks. Each of the second reference template regions may neighbor a corresponding one of the vector reference blocks. Each of the second reference template regions may include at least one of a fourth reference adjacent region, located at a left side of a corresponding vector reference block, a fifth reference adjacent region, located above the corresponding vector reference block, and a sixth reference adjacent region, located at a top-left side of the corresponding vector reference block. Since the fourth to the sixth reference adjacent regions of the vector reference blocks may be reconstructed prior to the reconstruction of the block unit, the second reference template regions of the vector reference blocks may be reconstructed prior to the reconstruction of the block unit. Thus, the decoder module 124 may be allowed to directly determine multiple reconstructed results of the second reference template regions when the block unit is being predicted and / or reconstructed.

[0157] In some implementations, the sizes of the second reference template regions may be identical to the size of the second block template region. Thus, when the size of the second block template region is identical to the size of the first block template region, the sizes of the second reference template regions may be identical to the sizes of the first reference template regions. In other words, each of the first reference template regions may also be identical to a corresponding second reference template region. Thus, as shown in FIGS. 8A and 8B, the second reference template region may also be directly adjacent to or separated from the reference block 810.

[0158] The decoder module 124 may determine a third reference template region in the current frame for each of the vector reference blocks. Each of the third reference template regions may also neighbor a corresponding one of the vector reference blocks. Each of the third reference template regions may include at least one of the fourth reference adjacent region, located at a left side of a corresponding vector reference block, the fifth reference adjacent region, located above the corresponding vector reference block, a seventh reference adjacent region, located at a right side of the corresponding vector reference block, an eighth reference adjacent region, located below the corresponding vector reference block, a ninth block adjacent region, located at a top-right side of the corresponding vector reference block, and a tenth block adjacent region, located at a bottom-left side of the corresponding vector reference block. Since the fourth, the fifth, and the seventh to the tenth reference adjacent regions of the vector reference blocks may also be reconstructed prior to the reconstruction of the block unit, the third reference template regions of the vector reference blocks may be reconstructed prior to the reconstruction of the block unit. Thus, the decoder module 124 may also be allowed to directly determine multiple reconstructed results of the third reference template regions when the block unit is being predicted and / or reconstructed. The sizes of the third reference template regions may also be identical to the size of the second block template region. Each of the third reference template regions in the current frame may be determined based on an association type of the corresponding block vector candidate in the method / process 300. In addition, as shown in FIGS. 8A and 8B, the third reference template region may also be directly adjacent to or separated from the reference block 810.

[0159] In some implementations, the decoder module 124 may predict the second block template region based on each of the reconstructed results in the second reference template regions, using a corresponding filter model to generate one of multiple first predicted results of the second block template region. Each of the second reference template regions may be determined by a corresponding one of the block vector candidates. In other words, each of the first block-vector-based candidates may be used to determine one of the first predicted results of the second block template region. In some implementations, the decoder module 124 may directly set the reconstructed results in the second reference template regions as multiple second predicted results of the second block template region. In other words, each of the second block-vector-based candidates may be used to determine one of the second predicted results of the second block template region.

[0160] In some other implementations, the decoder module 124 may flip the reconstructed results in the third reference template regions based on the association type in the method / process 300. Each of the third reference template regions may be determined by a corresponding one of the block vector candidates. Then, the decoder module 124 may predict the second block template region based on each of the flipped reconstructed results in the third reference template regions using a corresponding filter model, derived using the method / process 300, to generate one of multiple third predicted results of the second block template region. In other words, each of the third block-vector-based candidates may be used to determine one of the third predicted results of the second block template region. In yet some other implementations, the decoder module 124 may directly set the flipped reconstructed results in the third reference template regions as multiple fourth predicted results of the second block template region. In other words, each of the fourth block-vector-based candidates may be used to determine one of the fourth predicted results of the second block template region.

[0161] When the decoder module 124 determines the first and the second predicted results of the block unit, the decoder module 124 may determine whether the spatial sample terms are included in the second reference template regions or not. Thus, when a specific one of the spatial sample terms is excluded from a specific second reference template region, the specific spatial sample term may be padded by copying a sample adjacent to the specific spatial sample term, included in the specific second reference template region, to replace the excluded term. In addition, when the decoder module 124 determines the third and the fourth predicted results of the block unit, the decoder module 124 may determine whether the spatial sample terms are included in the third reference template regions or not. Thus, when a specific one of the spatial sample terms is excluded from a specific third reference template region, the specific spatial sample term may be padded by copying a sample adjacent to the specific spatial sample term, included in the specific third reference template region, to replace the excluded term.

[0162] The decoder module 114 may compare multiple prediction results in the second block template region with the reconstructed result in the second block template region by using a cost function to determine the template matching costs of the block-vector-based prediction candidates. The prediction results may include at least one of the first predicted results, the second predicted results, the third predicted results, and the fourth predicted results. The cost function may include, but not be limited to, Sum of Absolute Difference (SAD), Sum of Absolute Transformed Difference (SATD), Mean Absolute Difference (MAD), Mean Squared Difference (MSD), and Structural SIMilarity (SSIM). It should be noted that any cost function may be used without departing from this disclosure.

[0163] In some implementations, when the block-vector-based prediction candidates include the first block-vector-based candidates, the template matching costs may include the first template costs. Each of the first template costs may be calculated based on the first block-vector-based candidates corresponding to a corresponding block vector candidate and a corresponding filter model. In addition, each of the first template costs may be calculated, using the corresponding filter model, based on the second block template region and a corresponding one of the second reference template regions. For example, when the block-vector-based prediction candidates only include the first block-vector-based candidates, each of the template matching costs may be calculated using a corresponding one of the filter models of the block unit.

[0164] In some implementations, when the block-vector-based prediction candidates include the second block-vector-based candidates, the template matching costs may include multiple second template costs. Each of the second template costs may be calculated directly based on a corresponding block vector candidate without using the filter models of the block unit. In addition, each of the second template costs may be calculated based on the second block template region and a corresponding one of the second reference template regions without using the filter models of the block unit.

[0165] In some other implementations, when the block-vector-based prediction candidates include the third block-vector-based candidates, the template matching costs may include multiple third template costs. Each of the third template costs may be calculated based on the third block-vector-based candidates corresponding to a corresponding block vector candidate and a corresponding filter model generated in method / process 300. In addition, each of the third template costs may be calculated based on the second block template region and a corresponding one of the third reference template regions using the corresponding filter model.

[0166] In yet some other implementations, when the block-vector-based prediction candidates include the fourth block-vector-based candidates, the template matching costs may include multiple fourth template costs. Each of the fourth template costs may be calculated directly based on a corresponding block vector candidate without using the filter models of the block unit. In addition, each of the fourth template costs may be calculated based on the second block template region and a corresponding one of the third reference template regions without using the filter models of the block unit.

[0167] Referring back to FIG. 7, at block 760, the decoder module 124 may determine an arrangement of the filter models based on the template matching costs.

[0168] With reference to FIG. 1 and FIG. 2, the decoder module 124 may determine the arrangement of the block-vector-based prediction candidates based on the template matching costs and reorder the block-vector-based prediction candidates based on the arrangement. In some implementations, the block-vector-based prediction candidates may be reordered in an ascending order or a descending order of the template matching costs. In addition, the arrangement may be used to adjust an original order of the block-vector-based prediction candidates included in the block-vector-based candidate list to generate an adjusted candidate list having the at least one of the block-vector-based prediction candidates.

[0169] Before the arrangement is determined based on the template matching costs, the block-vector-based prediction candidates may be ordered based on arbitrary rules. Then, the block-vector-based prediction candidates may be reordered in the ascending order of the template matching costs. Thus, when the template matching cost of a specific block-vector-based prediction candidate is less than the template matching costs of the other block-vector-based prediction candidates, the specific block-vector-based prediction candidate may be moved forward to be a first block-vector-based prediction candidate based on the arrangement. In other words, the specific block-vector-based prediction candidate may be moved to be the first block-vector-based prediction candidate when the template matching cost of the specific block-vector-based prediction candidate is the minimum of the template matching costs of the block-vector-based prediction candidates.

[0170] The decoder module 124 may select K block-vector-based prediction candidates having the least template matching costs from the block-vector-based prediction candidates and add the selected K block-vector-based prediction candidates into the adjusted candidate list. The number K, being a positive integer equal to, or greater than, one, may be equal to the number of the block-vector-based prediction candidates in the adjusted candidate list. The number K may be equal to, or less than, the total quantity of the block-vector-based prediction candidates. In other words, the decoder module 124 may select the first to the K-th block-vector-based prediction candidates ordered based on the arrangement when the block-vector-based prediction candidates are reordered in the ascending order of the template matching costs to generate the arrangement.

[0171] In some implementations, the block-vector-based prediction candidates may include at least one of the first block-vector-based candidates, the second block-vector-based candidates, the third block-vector-based candidates, or the fourth block-vector-based candidates. Thus, the template matching costs may include at least one of the first template costs, the second template costs, the third template costs, or the fourth template costs. In other words, the arrangement of the block-vector-based prediction candidates may be determined based on the at least one of the first template costs, the second template costs, the third template costs, or the fourth template costs.

[0172] When the block-vector-based prediction candidates include the first block-vector-based candidates, the arrangement may include a first arrangement of the filter models determined based on the first template costs. Thus, the block-vector-based prediction candidates in the block-vector-based candidate list may be ordered based on the first arrangement of the filter models. In some implementations, the block-vector-based prediction candidates may only include the first block-vector-based candidates. Thus, the arrangement may only include the first arrangement of the filter models determined based on the first template costs. The block-vector-based prediction candidates in the block-vector-based candidate list may also be ordered only based on the first arrangement of the filter models. In some other implementations, the block-vector-based prediction candidates may include the first block-vector-based candidates and at least one of the second to the fourth block-vector-based candidates. The arrangement may include the first arrangement of the filter models and at least one of the second to the fourth arrangements. Thus, the block-vector-based prediction candidates may be ordered based on the first arrangement of the filter models and the at least one of the second to the fourth arrangements.

[0173] When the block-vector-based prediction candidates include the first block-vector-based candidates and the second block-vector-based candidates, the arrangement may include a second arrangement of the block-vector-based prediction candidates determined based on the first template costs and the second template costs. Thus, the block-vector-based prediction candidates in the block-vector-based candidate list may be ordered based on the second arrangement of the block-vector-based prediction candidates.

[0174] Referring back to FIG. 7, at block 770, the decoder module 124 may reconstruct the block unit based on the arrangement of the filter models.

[0175] With reference to FIG. 1 and FIG. 2, the decoder module 124 may reconstruct the block unit based on the block-vector-based prediction candidates ordered in the block-vector-based candidate list. In some implementations, when the block-vector-based prediction candidates include the first block-vector-based candidates, the decoder module 124 may reconstruct the block unit based on the block-vector-based prediction candidates, ordered based on the first arrangement of the filter models. In some other implementations, when the block-vector-based prediction candidates include the first block-vector-based candidates and the second block-vector-based candidates, the decoder module 124 may reconstruct the block unit based on the block-vector-based prediction candidates, ordered based on the second arrangement of the block-vector-based prediction candidates.

[0176] In some implementations, each of the block-vector-based prediction candidates in the adjusted candidate list may have an index value. Thus, the index value for the adjusted candidate list may be within an index range of 0 to K-1 since the number of the block-vector-based prediction candidates in the adjusted candidate list is equal to K. The block-vector-based prediction candidates arranged after the K-th block-vector-based prediction candidate ordered by the arrangement may be excluded from the adjusted candidate list since the index value of the prediction index may not be greater than K-1.

[0177] The decoder module 124 may select, based on a prediction index, one of the block-vector-based prediction candidates, ordered based on the arrangement, from the adjusted candidate list. The prediction index of the block unit may indicate the index value of the selected one of the block-vector-based prediction candidates, ordered based on the arrangement in the adjusted candidate list. The prediction index may be included in the video data, such that the decoder module may parse the video data to determine the prediction index

[0178] In some other implementations, the first one of the block-vector-based prediction candidates, ordered based on the arrangement in the adjusted candidate list, may have the minimum of the template matching costs of the block-vector-based prediction candidates. Thus, the decoder module 124 may directly select the first one of the block-vector-based prediction candidates in the adjusted candidate list to predict the block unit without parsing the prediction index from the video data.

[0179] After the selected block-vector-based prediction candidate is determined, the decoder module 124 may predict the block unit using the selected block-vector-based prediction candidate to determine a predicted block of the block unit. The decoder module 124 may further add a plurality of residual components into the predicted block to reconstruct the block unit. The residual components may be determined from the bitstream. The decoder module 124 may reconstruct all of the other block units in the current frame for reconstructing the current frame and the video data.

[0180] In some implementations, a filter flag may be determined from the video data for indicating whether to derive the filter models for determining the first arrangement of the filter models based on the first template matching costs. When the filter flag is equal to zero, the decoder module 124 may bypass deriving the filter models. Thus, the first block-vector-based candidates may be excluded from the block-vector-based prediction candidates. In addition, when the filter flag is equal to one, the decoder module 124 may derive the filter models and calculate the first template matching costs for determining the first arrangement of the filter models.

[0181] The method / process 700 may then end.

[0182] FIG. 9 is a block diagram illustrating an encoder module 114 of the first electronic device 110 illustrated in FIG. 1, in accordance with one or more example implementations of this disclosure. The encoder module 114 may include a prediction processor (e.g., a prediction processing unit 9141), at least a first summer (e.g., a first summer 9142) and a second summer (e.g., a second summer 9145), a transform / quantization processor (e.g., a transform / quantization unit 9143), an inverse quantization / inverse transform processor (e.g., an inverse quantization / inverse transform unit 9144), a filter (e.g., a filtering unit 9146), a decoded picture buffer (e.g., a decoded picture buffer 9147), and an entropy encoder (e.g., an entropy encoding unit 9148). The prediction processing unit 9141 of the encoder module 114 may further include a partition processor (e.g., a partition unit 91411), an intra prediction processor (e.g., an intra prediction unit 91412), and an inter prediction processor (e.g., an inter prediction unit 91413). The encoder module 114 may receive the source video and encode the source video to output a bitstream.

[0183] The encoder module 114 may receive source video including multiple image frames and then divide the image frames according to a coding structure. Each of the image frames may be divided into at least one image block.

[0184] The at least one image block may include a luminance block having multiple luminance samples and at least one chrominance block having multiple chrominance samples. The luminance block and the at least one chrominance block may be further divided to generate macroblocks, CTUs, CBs, sub-divisions thereof, and / or other equivalent coding units.

[0185] The encoder module 114 may perform additional sub-divisions of the source video. It should be noted that the disclosed implementations are generally applicable to video coding regardless of how the source video is partitioned prior to and / or during the encoding.

[0186] During the encoding process, the prediction processing unit 9141 may receive a current image block of a specific one of the image frames. The current image block may be the luminance block or one of the chrominance blocks in the specific image frame.

[0187] The partition unit 91411 may divide the current image block into multiple block units. The intra prediction unit 91412 may perform intra-predictive coding of a current block unit relative to one or more neighboring blocks in the same frame, as the current block unit, in order to provide spatial prediction. The inter prediction unit 91413 may perform inter-predictive coding of the current block unit relative to one or more blocks in one or more reference image blocks to provide temporal prediction.

[0188] The prediction processing unit 9141 may select one of the coding results generated by the intra prediction unit 91412 and the inter prediction unit 91413 based on a mode selection method, such as a cost function. The mode selection method may be a rate-distortion optimization (RDO) process.

[0189] The prediction processing unit 9141 may determine the selected coding result and provide a predicted block corresponding to the selected coding result to the first summer 9142 for generating a residual block and to the second summer 9145 for reconstructing the encoded block unit. The prediction processing unit 9141 may further provide syntax elements, such as motion vectors, intra-mode indicators, partition information, and / or other syntax information, to the entropy encoding unit 9148.

[0190] The intra prediction unit 91412 may intra-predict the current block unit. The intra prediction unit 91412 may determine an intra prediction mode directed toward a reconstructed sample neighboring the current block unit in order to encode the current block unit.

[0191] The intra prediction unit 91412 may encode the current block unit using various intra prediction modes. The intra prediction unit 91412 of the prediction processing unit 9141 may select an appropriate intra prediction mode from the selected modes. The intra prediction unit 91412 may encode the current block unit using a cross-component prediction mode to predict one of the two chroma components of the current block unit based on the luma components of the current block unit. The intra prediction unit 91412 may predict a first one of the two chroma components of the current block unit based on the second of the two chroma components of the current block unit.

[0192] The inter prediction unit 91413 may inter-predict the current block unit as an alternative to the intra prediction performed by the intra prediction unit 91412. The inter prediction unit 91413 may perform motion estimation to estimate motion of the current block unit for generating a motion vector.

[0193] The motion vector may indicate a displacement of the current block unit within the current image block relative to a reference block unit within a reference image block. The inter prediction unit 91413 may receive at least one reference image block stored in the decoded picture buffer 9147 and estimate the motion based on the received reference image blocks to generate the motion vector.

[0194] The first summer 9142 may generate the residual block by subtracting the prediction block determined by the prediction processing unit 9141 from the original current block unit. The first summer 9142 may represent the component or components that perform this subtraction.

[0195] The transform / quantization unit 9143 may apply a transform to the residual block in order to generate a residual transform coefficient and then quantize the residual transform coefficients to further reduce the bit rate. The transform may be one of a DCT, DST, AMT, MDNSST, HyGT, signal-dependent transform, KLT, wavelet transform, integer transform, sub-band transform, and a conceptually similar transform.

[0196] The transform may convert the residual information from a pixel value domain to a transform domain, such as a frequency domain. The degree of quantization may be modified by adjusting a quantization parameter.

[0197] The transform / quantization unit 9143 may perform a scan of the matrix including the quantized transform coefficients. Alternatively, the entropy encoding unit 9148 may perform the scan.

[0198] The entropy encoding unit 9148 may receive multiple syntax elements from the prediction processing unit 9141 and the transform / quantization unit 9143, including a quantization parameter, transform data, motion vectors, intra modes, partition information, and / or other syntax information. The entropy encoding unit 9148 may encode the syntax elements into the bitstream.

[0199] The entropy encoding unit 9148 may entropy encode the quantized transform coefficients by performing CAVLC, CABAC, SBAC, PIPE coding, or another entropy coding technique to generate an encoded bitstream. The encoded bitstream may be transmitted to another device (e.g., the second electronic device 120, as shown in FIG. 1) or archived for later transmission or retrieval.

[0200] The inverse quantization / inverse transform unit 9144 may apply inverse quantization and inverse transformation to reconstruct the residual block in the pixel domain for later use as a reference block. The second summer 9145 may add the reconstructed residual block to the prediction block provided by the prediction processing unit 9141 in order to produce a reconstructed block for storage in the decoded picture buffer 9147.

[0201] The filtering unit 9146 may include a deblocking filter, an SAO filter, a bilateral filter, and / or an ALF to remove blocking artifacts from the reconstructed block. Other filters (in loop or post loop) may be used in addition to the deblocking filter, the SAO filter, the bilateral filter, and the ALF. Such filters are not illustrated for brevity and may filter the output of the second summer 9145.

[0202] The decoded picture buffer 9147 may be a reference picture memory that stores the reference block to be used by the encoder module 914 to encode video, such as in intra-coding or inter-coding modes. The decoded picture buffer 9147 may include a variety of memory devices, such as DRAM (e.g., including SDRAM), MRAM, RRAM, or other types of memory devices. The decoded picture buffer 9147 may be on-chip with other components of the encoder module 114 or off-chip relative to those components.

[0203] The method / process 300 for decoding and / or encoding video data may be performed by the first electronic device 110. With reference to FIGS. 1 and 9, at block 310, the method / process 300 may start by the encoder module 114 receiving the video data. The video data received by the encoder module 114 may be a video. At block 320, the encoder module 114 may determine a block unit from a current frame included in the video data. In some implementations, the encoder module 114 may divide the current frame to generate multiple CTUs, and further divide a current CTU included in the CTUs to generate multiple divided blocks and to determine the block unit from the divided blocks, according to the partition indications (e.g., based on any video coding standard). In some other implementations, the encoder module 114 may divide the current frame to generate multiple slices, and further divide a current slice included in the slices to generate multiple CTUs. In addition, the encoder module 114 may further divide a current CTU included in the CTUs to generate multiple divided blocks and to determine the block unit from the divided blocks, according to the partition indications.

[0204] At block 330, the encoder module 114 may determine, from the current frame, a reference block indicated by a block vector. With reference to FIGS. 1, 3, and 9, the determination process of the decoder module 124 at block 330 may also be performed by the encoder module 114 at block 330. The encoder module 114 may derive the block vector in a reconstruction-reordered (RR) intra block copy (IBC) (RR-IBC) mode. Thus, the reference block indicated by the block vector may be determined using the RR-IBC mode.

[0205] The neighboring candidate vectors determined by the encoder module 114 may be identical to those determined by the decoder module 114. In addition, the flip directions determined by the encoder module 114 may also be identical to those determined by the decoder modules. In some implementations, when a vector direction of a specific one of the neighboring candidate vectors is identical to a specific one of the flip directions, the specific neighboring candidate vector may be determined as a reference candidate vector in the vector candidate list for deriving the block vector. In addition, the association type corresponding to the specific neighboring candidate vector may specify the specific flip direction. In some other implementations, the association type of the block unit, corresponding to the specific neighboring candidate vector of a specific neighboring block, may be inherited from the specific neighboring block. Thus, when a vector direction of the specific neighboring candidate vector of the specific neighboring block is identical to the flip direction specified by the association type of the specific neighboring block, the specific neighboring candidate vector may be determined as the reference candidate vector in the vector candidate list for deriving the block vector.

[0206] In some implementations, when number of the reference candidate vector is equal to one, the reference candidate vector may be determined as a reference vector for deriving the block vector to indicate the reference block. In some other implementations, when number of the reference candidate vectors is greater than one, the reference candidate vectors may be determined as multiple reference vectors, each for deriving a corresponding one of the block vectors to indicate a corresponding one of the reference blocks. The determination method, for determining the block vectors based on the reference vectors, performed by the encoder module 114 may be identical to that performed by the decoder module 124. In addition, the determination method, for determining the reference blocks based on the block vectors, performed by the encoder module 114 may be identical to that performed by the decoder module 124. Thus, number of the reference blocks may be equal to number of the reference candidate vectors in the vector candidate list.

[0207] At block 340, the encoder module 114 may determine, based on an association type, a corresponding relationship between a block template region, neighboring the block unit, and a reference template region, neighboring the reference block. With reference to FIGS. 1, 3, and 9, the determination process of the decoder module 124 at block 340 may also be performed by the encoder module 114 at block 340.

[0208] The block template region determined by the encoder module 114 may be identical to that determined by the decoder module 114. In addition, the determination method, for determining the reference template regions, performed by the encoder module 114 may be identical to the determination method, for determining the reference template region, performed by the decoder module 124. Thus, the reference template regions, each respectively neighboring a corresponding one of the reference blocks, may be determined using the determination method of the decoder module 124. In other words, number of the reference template regions may be equal to number of the reference blocks.

[0209] The corresponding relationship between the block template region and the reference template region may be determined based on the association type. In addition, the determination method, for determining each of the corresponding relationships between the block template region and the corresponding reference template region, performed by the encoder module 114 may be identical to that, for determining the corresponding relationship between the block template region and the reference template region, performed by the decoder module 124. Thus, number of the corresponding relationships may be equal to number of the reference blocks and also equal to number of the reference template regions.

[0210] At block 350, the encoder module 124 may derive a prediction model of the block unit based on the corresponding relationship between block template region and the reference template region. With reference to FIGS. 1, 3, and 9, the derivation process of the decoder module 124 at block 350 may also be performed by the encoder module 114 at block 350.

[0211] The derivation method, for deriving each of the prediction models based on a corresponding one of the corresponding relationships, performed by the encoder module 114 may be identical to the derivation method, for deriving the prediction model based on the corresponding relationship, performed by the decoder module 124. In some implementations, number of the prediction models may be equal to number of the corresponding relationships and also equal to number of the reference blocks. In some other implementations, when the encoder module 114 derives multiple prediction models using multiple candidate models, each corresponding relationship may used to derive more than one prediction models. For example, when number of the corresponding relationship is equal to Ncr and number of the candidate models is equal to Ncm, number of the prediction models Npm may be equal to, or less than, Ncr×Ncm.

[0212] In some implementations, in order to easily derive the prediction model, the encoder module 114 may flip the block template region in advance along a center line of the block unit 400. For example, as shown in FIG. 6C, the encoder module 114 may flip the block template region in advance along a vertical center line 4015 of the block unit 400. Thus, the flipped first block adjacent region 403’ may be located at the right side of a flipped block unit 400’. In addition, as shown in FIGS. 6A and 6C, multiple flipped block template samples in the flipped block template region 403’ may be symmetric to multiple original block template samples in the block template region 403 along the vertical center line 4015 of the block unit 400. Multiple flipped block template samples in the flipped second block template region 404’ may also be symmetric to multiple original block template samples in the second block template region 404 along the vertical center line 4015 of the reference block 410. Thus, a location relationship between the flipped block unit 400’ and the flipped block template region may be identical to a location relationship between the reference unit 410 and the reference template region for deriving a corresponding prediction model. In some other implementations, as shown in FIG. 4B, a location relationship between a flipped block unit and a flipped block template region, generated by flipping the block unit 400 and the block template region along a horizontal center line of the block unit 400, may be identical to a location relationship between the reference block 420 and the reference template region.

[0213] Referring back to FIG. 3, at block 360, the encoder module 124 may reconstruct the block unit based on the reference block using the prediction model with the association type. With reference to FIGS. 1, 3, and 9, the reconstruction process of the decoder module 124 at block 360 may also be performed by the encoder module 114 at block 360.

[0214] The encoder module 114 may predict the block unit based on each of the reference block using a corresponding one of the prediction models to generate a corresponding one of multiple predicted blocks. In addition, the encoder module 114 may predict the block unit based on other prediction modes to generate multiple prediction blocks. The encoder module 114 may select one of the predicted blocks and the prediction blocks based on a mode selection method, such as a cost function. The mode selection method may be an RDO process, a Sum of Absolute Difference (SAD) process, a Sum of Absolute Transformed Difference (SATD) process, a Mean Absolute Difference (MAD) process, a Mean Squared Difference (MSD) process, and a Structural SIMilarity (SSIM) process. The encoder module 114 may provide the selected coding result to the first summer 9142 for generating a residual block and to the second summer 9145 for reconstructing the encoded block unit. The reconstruction of the block unit by the encoder module 114 may be identical to the reconstruction of the block unit by the decoder module 124.

[0215] In some implementations, when one of the predicted blocks corresponding to one of the prediction models is selected to predict and reconstruct the block unit, the encoder module 114 may further provide syntax elements, such as candidate index, included in the bitstream for transmitting to the decoder module 124. In some implementations, the candidate index of the block unit may be used to determine one of the reference candidate vectors corresponding to the selected one of the prediction models.

[0216] In some implementations, the block-vector-based local illumination compensation (LIC) flag of the block unit, the RR-IBC flag, and the template type syntax received by the decoder module 124 in the method / process 300 may be provided by the encoder module 114

[0217] The encoder module 114 may predict and reconstruct all of the other block units in the image frame for reconstructing the image frame and the video. The method / process 300 for the encoder module 114 may then end.

[0218] The method / process 700 for decoding and / or encoding video data may be performed by the first electronic device 110. With reference to FIGS. 1 and 9, at block 710, the method / process 700 may start by the encoder module 114 receiving the video data. The video data received by the encoder module 114 may be a video. At block 720, the encoder module 114 may determine a block unit from a current frame included in the video data. In some implementations, the encoder module 114 may divide the current frame to generate multiple CTUs, and further divide a current CTU included in the CTUs to generate multiple divided blocks and to determine the block unit from the divided blocks, according to the partition indications (e.g., based on any video coding standard). In some other implementations, the encoder module 114 may divide the current frame to generate multiple slices, and further divide a current slice included in the slices to generate multiple CTUs. In addition, the encoder module 114 may further divide a current CTU included in the CTUs to generate multiple divided blocks and to determine the block unit from the divided blocks, according to the partition indications.

[0219] At block 730, the encoder module 114 may determine, from the current frame, multiple vector reference blocks of the block unit, indicated by multiple block vector candidates of the block unit. In some implementations, each of the vector reference blocks may be indicated by a corresponding one of the block vector candidates of the block unit. With reference to FIGS. 1, 3, and 9, the determination process of the decoder module 124 at block 730 may also be performed by the encoder module 114 at block 730.

[0220] At block 740, the encoder module 114 may derive multiple filter models of the block unit based on the vector reference blocks. In some implementations, each of the filter models may be derived based on a corresponding one of the vector reference blocks. With reference to FIGS. 1, 3, and 9, the derivation process of the decoder module 124 at block 740 may also be performed by the encoder module 114 at block 740.

[0221] At block 750, the encoder module 114 may determine multiple template matching costs calculated using the filter models of the block unit. In some implementations, the template matching costs may include multiple first template costs, and each of the first template costs may be calculated using a corresponding one of the filter models of the block unit. With reference to FIGS. 1, 3, and 9, the determination process of the decoder module 124 at block 750 may also be performed by the encoder module 114 at block 750.

[0222] At block 760, the encoder module 114 may determine an arrangement of the filter models based on the template matching costs. With reference to FIGS. 1, 3, and 9, the determination process of the decoder module 124 at block 760 may also be performed by the encoder module 114 at block 760.

[0223] At block 770, the encoder module 114 may reconstruct the block unit based on the arrangement of the filter models. With reference to FIGS. 1, 3, and 9, the reconstruction process of the decoder module 124 at block 770 may also be performed by the encoder module 114 at block 770.

[0224] The encoder module 114 may predict the block unit based on each of the reference block using a corresponding one of the filter models to generate a corresponding one of multiple predicted blocks. In addition, the encoder module 114 may predict the block unit based on other prediction modes to generate multiple prediction blocks. The encoder module 114 may select one of the predicted blocks and the prediction blocks based on a mode selection method, such as a cost function. The mode selection method may be an RDO process, an SAD process, an SATD process, an MAD process, an MSD process, and an SSIM process. The encoder module 114 may provide the selected coding result to the first summer 9142 for generating a residual block and to the second summer 9145 for reconstructing the encoded block unit. The reconstruction of the block unit by the encoder module 114 may be identical to the reconstruction of the block unit by the decoder module 124.

[0225] In some implementations, when one of the predicted blocks corresponding to one of the filter models is selected to predict and reconstruct the block unit, the encoder module 114 may further provide syntax elements, such as a prediction index, included in the bitstream for transmitting to the decoder module 124. Each of the block-vector-based prediction candidates in the adjusted candidate list may have an index value. Thus, the index value for the adjusted candidate list may be within an index range of 0 to K-1 since the number of the block-vector-based prediction candidates in the adjusted candidate list is equal to K. The block-vector-based prediction candidates arranged after the K-th block-vector-based prediction candidate ordered by the arrangement may be excluded from the adjusted candidate list since the index value of the prediction index may not be greater than K-1. In some implementations, the prediction index of the block unit may be equal to one of the index values for determining one of the block-vector-based prediction candidates corresponding to the selected one of the predicted blocks.

[0226] In some other implementations, when the selected one of the predicted blocks is generated based on a first one of the block-vector-based prediction candidates in the adjusted candidate list, the encoder module 114 may not provide the prediction index to the decoder module 124. In other words, the decoder module 124 may directly select the first one of the block-vector-based prediction candidates in the adjusted candidate list to predict the block unit without parsing the prediction index from the video data.

[0227] In some implementations, a filter flag may be encoded into the video data for indicating whether to derive the filter models for determining the first arrangement of the filter models based on the first template matching costs. When the encoder module 114 bypasses deriving the filter models, the filter flag provided by the encoder module 114 may be equal to zero. Thus, the first block-vector-based candidates may be excluded from the block-vector-based prediction candidates. In addition, when the encoder module 114 derives the filter models and calculates the first template matching costs for determining the first arrangement of the filter models, the filter flag provided by the encoder module 114 may be equal to one.

[0228] The encoder module 114 may predict and reconstruct all of the other block units in the image frame for reconstructing the image frame and the video. The method / process 700 for the encoder module 114 may then end.

[0229] The disclosed implementations are to be considered in all respects as illustrative and not restrictive. It should also be understood that the present disclosure is not limited to the specific disclosed implementations, but that many rearrangements, modifications, and substitutions are possible without departing from the scope of the present disclosure.

Claims

1. An electronic device for decoding video data, the electronic device comprising: 　　at least one processor; and 　　one or more non-transitory computer-readable media coupled to the at least one processor and storing one or more computer-executable instructions that, when executed by the at least one processor, cause the electronic device to: 　　　　receive the video data; 　　　　determine a block unit from a current frame included in the video data; 　　　　determine, from the current frame, a plurality of vector reference blocks of the block unit, each of the plurality of vector reference blocks indicated by a corresponding one of a plurality of block vector candidates of the block unit; 　　　　derive a plurality of filter models of the block unit, each of the plurality of filter models derived based on a corresponding one of the plurality of vector reference blocks; 　　　　determine a plurality of first template matching costs, each of the plurality of first template matching costs calculated using a corresponding one of the plurality of filter models of the block unit; 　　　　determine a first arrangement of the plurality of filter models based on the plurality of first template matching costs; and 　　　　reconstruct the block unit based on the first arrangement of the plurality of filter models.

2. The electronic device according to claim 1, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: 　　determine a plurality of neighboring blocks neighboring the block unit; 　　determine a plurality of neighboring block vectors, each of the plurality of neighboring block vectors indicating a corresponding one of a plurality of neighboring reference blocks in the current frame for reconstructing a corresponding one of the plurality of neighboring blocks; and 　　determine the plurality of block vector candidates based on the plurality of neighboring block vectors.

3. The electronic device according to claim 1, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: 　　determine a first block template region, neighboring the block unit, and a plurality of first reference template regions, each of the plurality of first reference template regions neighboring a corresponding one of the plurality of vector reference blocks; and 　　determine a second block template region, neighboring the block unit, and a plurality of second reference template regions, each of the plurality of second reference template regions neighboring a corresponding one of the plurality of vector reference blocks, wherein: 　　　　each of the plurality of filter models is derived further based on the first block template region and a corresponding one of the plurality of first reference template regions, and 　　　　each of the plurality of first template matching costs is calculated further based on the second block template region and a corresponding one of the plurality of second reference template regions using the corresponding one of the plurality of filter models.

4. The electronic device according to claim 3, wherein: 　　the first block template region is identical to the second block template, and 　　each of the plurality of first reference template regions is identical to a corresponding one of the plurality of second template regions.

5. The electronic device according to claim 1, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: 　　determine a block-vector-based candidate list including a plurality of block-vector-based prediction candidates, wherein: 　　　　each of the block-vector-based prediction candidates corresponds to a corresponding one of the plurality of block vector candidates and a corresponding one of the plurality of filter models, 　　　　the plurality of block-vector-based prediction candidates in the block-vector-based candidate list is ordered based on the first arrangement of the plurality of filter models, and 　　　　reconstructing the block unit is further based on the plurality of block-vector-based prediction candidates ordered in the block-vector-based candidate list.

6. The electronic device according to claim 1, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: 　　determine a plurality of second template matching costs, each of the plurality of second template matching costs calculated directly based on a corresponding one of the plurality of block vector candidates without using the plurality of filter models of the block unit; and 　　determine a second arrangement of a plurality of block-vector-based prediction candidates based on the plurality of first template matching costs and the plurality of second template matching costs, wherein reconstructing the block unit is further based on the second arrangement of the plurality of block-vector-based prediction candidates.

7. The electronic device according to claim 6, wherein: 　　the plurality of block-vector-based prediction candidates includes at least one of a plurality of first block-vector-based candidates or a plurality of second block-vector-based candidates, 　　each of the plurality of first block-vector-based candidates corresponds to a corresponding one of the plurality of block vector candidates and a corresponding one of the plurality of filter models, and 　　each of the plurality of second block-vector-based candidates only corresponds to a corresponding one of the plurality of block vector candidates without using the plurality of filter models.

8. The electronic device according to claim 1, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: 　　determine a filter flag indicating whether to derive the plurality of filter models for determining the first arrangement of the plurality of filter models based on the plurality of first template matching costs.

9. A non-transitory machine-readable medium of an electronic device storing one or more computer-executable instructions for decoding video data, the one or more computer-executable instructions, when executed by at least one processor of the electronic device, causing the electronic device to: 　　receive the video data; 　　determine a block unit from a current frame included in the video data; 　　determine, from the current frame, a plurality of vector reference blocks of the block unit, each of the plurality of vector reference blocks indicated by a corresponding one of a plurality of block vector candidates of the block unit; 　　derive a plurality of filter models of the block unit, each of the plurality of filter models derived based on a corresponding one of the plurality of vector reference blocks; 　　determine a plurality of first template matching costs, each of the plurality of first template matching costs calculated using a corresponding one of the plurality of filter models of the block unit; 　　determine a first arrangement of the plurality of filter models based on the plurality of first template matching costs; and 　　reconstruct the block unit based on the first arrangement of the plurality of filter models.

10. The non-transitory machine-readable medium according to claim 9, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: 　　determine a first block template region, neighboring the block unit, and a plurality of first reference template regions, each of the plurality of first reference template regions neighboring a corresponding one of the plurality of vector reference blocks; and 　　determine a second block template region, neighboring the block unit, and a plurality of second reference template regions, each of the plurality of second reference template regions neighboring a corresponding one of the plurality of vector reference blocks, wherein: 　　　　each of the plurality of filter models is derived further based on the first block template region and a corresponding one of the plurality of first reference template regions, and 　　　　each of the plurality of first template matching costs is calculated further based on the second block template region and a corresponding one of the plurality of second reference template regions using the corresponding one of the plurality of filter models.

11. The non-transitory machine-readable medium according to claim 10, wherein: 　　the first block template region is identical to the second block template, and 　　each of the plurality of first reference template regions is identical to a corresponding one of the plurality of second template regions.

12. The non-transitory machine-readable medium according to claim 9, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: 　　determine a block-vector-based candidate list including a plurality of block-vector-based prediction candidates, wherein: 　　　　each of the block-vector-based prediction candidates corresponds to a corresponding one of the plurality of block vector candidates and a corresponding one of the plurality of filter models, 　　　　the plurality of block-vector-based prediction candidates in the block-vector-based candidate list is ordered based on the first arrangement of the plurality of filter models, and 　　　　reconstructing the block unit is further based on the plurality of block-vector-based prediction candidates ordered in the block-vector-based candidate list.

13. The non-transitory machine-readable medium according to claim 9, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: 　　determine a plurality of second template matching costs, each of the plurality of second template matching costs calculated directly based on a corresponding one of the plurality of block vector candidates without using the plurality of filter models of the block unit; and 　　determine a second arrangement of a plurality of block-vector-based prediction candidates based on the plurality of first template matching costs and the plurality of second template matching costs, wherein: 　　　　reconstructing the block unit is further based on the second arrangement of the plurality of block-vector-based prediction candidates, 　　　　the plurality of block-vector-based prediction candidates includes at least one of a plurality of first block-vector-based candidates or a plurality of second block-vector-based candidates, 　　　　each of the plurality of first block-vector-based candidates corresponds to a corresponding one of the plurality of block vector candidates and a corresponding one of the plurality of filter models, and 　　　　each of the plurality of second block-vector-based candidates only corresponds to a corresponding one of the plurality of block vector candidates without using the plurality of filter models.

14. The non-transitory machine-readable medium according to claim 9, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: 　　determine a filter flag indicating whether to derive the plurality of filter models for determining the first arrangement of the plurality of filter models based on the plurality of first template matching costs.

15. An electronic device for encoding video data, the electronic device comprising: 　　at least one processor; and 　　one or more non-transitory computer-readable media coupled to the at least one processor and storing one or more computer-executable instructions that, when executed by the at least one processor, cause the electronic device to: 　　　　receive the video data; 　　　　determine a block unit from a current frame included in the video data; 　　　　determine, from the current frame, a plurality of vector reference blocks of the block unit, each of the plurality of vector reference blocks indicated by a corresponding one of a plurality of block vector candidates of the block unit; 　　　　derive a plurality of filter models of the block unit, each of the plurality of filter models derived based on a corresponding one of the plurality of vector reference blocks; 　　　　determine a plurality of first template matching costs, each of the plurality of first template matching costs calculated using a corresponding one of the plurality of filter models of the block unit; 　　　　determine a first arrangement of the plurality of filter models based on the plurality of first template matching costs; and 　　　　reconstruct the block unit based on the first arrangement of the plurality of filter models.

16. The electronic device according to claim 15, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: 　　determine a first block template region, neighboring the block unit, and a plurality of first reference template regions, each of the plurality of first reference template regions neighboring a corresponding one of the plurality of vector reference blocks; and 　　determine a second block template region, neighboring the block unit, and a plurality of second reference template regions, each of the plurality of second reference template regions neighboring a corresponding one of the plurality of vector reference blocks, wherein: 　　　　each of the plurality of filter models is derived further based on the first block template region and a corresponding one of the plurality of first reference template regions, and 　　　　each of the plurality of first template matching costs is calculated further based on the second block template region and a corresponding one of the plurality of second reference template regions using the corresponding one of the plurality of filter models.

17. The electronic device according to claim 16, wherein: 　　the first block template region is identical to the second block template, and 　　each of the plurality of first reference template regions is identical to a corresponding one of the plurality of second template regions.

18. The electronic device according to claim 15, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: 　　determine a block-vector-based candidate list including a plurality of block-vector-based prediction candidates, wherein: 　　　　each of the block-vector-based prediction candidates corresponds to a corresponding one of the plurality of block vector candidates and a corresponding one of the plurality of filter models, 　　　　the plurality of block-vector-based prediction candidates in the block-vector-based candidate list is ordered based on the first arrangement of the plurality of filter models, and 　　　　reconstructing the block unit is further based on the plurality of block-vector-based prediction candidates ordered in the block-vector-based candidate list.

19. The electronic device according to claim 15, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: 　　determine a plurality of second template matching costs, each of the plurality of second template matching costs calculated directly based on a corresponding one of the plurality of block vector candidates without using the plurality of filter models of the block unit; and 　　determine a second arrangement of a plurality of block-vector-based prediction candidates based on the plurality of first template matching costs and the plurality of second template matching costs, wherein: 　　　　reconstructing the block unit is further based on the second arrangement of the plurality of block-vector-based prediction candidates, 　　　　the plurality of block-vector-based prediction candidates includes at least one of a plurality of first block-vector-based candidates or a plurality of second block-vector-based candidates, 　　　　each of the plurality of first block-vector-based candidates corresponds to a corresponding one of the plurality of block vector candidates and a corresponding one of the plurality of filter models, and 　　　　each of the plurality of second block-vector-based candidates only corresponds to a corresponding one of the plurality of block vector candidates without using the plurality of filter models.

20. The electronic device according to claim 15, wherein the one or more computer-executable instructions, when executed by the at least one processor, further cause the electronic device to: 　　determine a filter flag indicating whether to derive the plurality of filter models for determining the first arrangement of the plurality of filter models based on the plurality of first template matching costs.

Images