Reconstruction reordering in intra template matching prediction merge mode with intra block copy processing

Abstract

Video coding can be performed using intra template matching prediction (ITMP) merge mode with reconstruction reorder intra block copy (RRIBC) processing. A device (e.g., a decoder) can determine whether a neighboring coding unit (CU) is coded in a RRIBC mode. The device can add an ITMP merge candidate associated with the neighboring CU to an ITMP merge candidate list, for example, based on determining that the neighboring CU is coded in the RRIBC mode. The device can flip a template and / or block associated with the added ITMP merge candidate. The template and / or block can be flipped, for example, using a flipping direction. The device can select an ITMP candidate from the ITMP candidate list, for example. The device can decode a current block using the selected ITMP candidate.

Description

Cross-reference to related applications

[0001] This application claims the benefit of European Patent Application No. 24305522.5, filed on 4 April 2024, the contents of which are incorporated herein by reference in their entirety. Background Technology

[0002] Video coding systems can be used to compress digital video signals, for example, to reduce the storage and / or transmission bandwidth required for such signals. Video coding systems can include, for example, block-based, wavelet-based, and / or object-based systems. Summary of the Invention

[0003] Systems, methods, and means for performing video encoding and decoding using Reconstruction Reordering Intra-Block Copying (RRIBC) processing in Intra-Frame Template Matching Prediction (ITMP) merging mode are disclosed.

[0004] In the example, a device including a processor (e.g., a decoder) can be configured to implement a method that may include one or more of the following: The video codec device can determine whether a neighboring coding unit (CU) is encoded in Reconstruction Reordered Intra-Block Copy (RRIBC) mode. The device can, for example, add an Intra-Template Match Prediction (ITMP) merge candidate associated with a neighboring CU to an ITMP merge candidate list based on the determination that the neighboring CU is encoded in RRIBC mode. The device can flip the template and / or block associated with the added ITMP merge candidate. The block can be a current block or a reference block. The template can be associated with either the current block or the reference block.

[0005] In the example, a flip direction can be used to flip the template and / or block. In the example, the device can apply the flip direction to a refined block in RRIBC mode. The device can generate a second ITMP candidate list from a first ITMP merge candidate list and a sparse candidate list. The device can further refine the second ITMP candidate list to generate a third ITMP candidate list from the refined second ITMP candidate list.

[0006] The device can, for example, select the ITMP candidate with the lowest template matching cost from a third list of ITMP merge candidates. The device can then use the selected ITMP candidate to decode the current block.

[0007] In the example, determining whether a neighboring CU is encoded in RRIBC mode can include determining this based on an indicator associated with the neighboring CU. Based on the determination that a neighboring CU is encoded in RRIBC mode, the device can exclude the block vector of the neighboring CU from being added to the ITMP merge candidate list.

[0008] In the example, the device can flip templates and / or blocks during template matching. In the example, the device can rank a list of ITMP merge candidates, including both flipped and unflipped merge candidates, to generate an ordered list of ITMP merge candidates. Attached Figure Description

[0009] Figure 1A This is a system diagram illustrating an example communication system in which one or more of the disclosed embodiments may be implemented.

[0010] Figure 1B This illustrates that, according to an embodiment, it is possible to Figure 1A The diagram shows a system diagram of an example wireless transmit / receive unit (WTRU) used in the communication system.

[0011] Figure 1C This illustrates that, according to an embodiment, it is possible to Figure 1A The diagram shows an example radio access network (RAN) and an example core network (CN) used in the communication system.

[0012] Figure 1D This illustrates that, according to an embodiment, it is possible to Figure 1A The system diagram shows another example RAN and another example CN used in the communication system shown.

[0013] Figure 2 An example video encoder is shown.

[0014] Figure 3 An example video decoder is shown.

[0015] Figure 4 Example systems are shown that can implement various aspects and examples.

[0016] Figure 5 An example of an intra-frame template matching search region is shown.

[0017] Figure 6 Examples of sparse search and refined search that can be part of the ITMP process are shown.

[0018] Figure 7 An example of a vertical flip is shown.

[0019] Figure 8 An example of a horizontally flipped case is shown.

[0020] Figure 9 Examples showing a comparison between the regular template and the current block template in vertical and horizontal flipping scenarios are provided.

[0021] Figure 10 Examples of using a left-side-only template and a top-only template are shown when the appropriate flipping is applied to the template.

[0022] Figure 11 An example is shown where a reference block is flipped to create a prediction.

[0023] Figure 12 This shows an example of using a modified template in both vertical and horizontal flip scenarios. The modified template can be taken from neighboring pixels of a reference block.

[0024] Figure 13 An example of a modified template using vertical and horizontal flipping is shown.

[0025] Figure 14 Examples of using modified templates and regular templates are shown when flipping the current block side.

[0026] Figure 15 This shows an example of template matching when the current template is flipped and a regular template can be used on the reference block side.

[0027] Figure 16 This shows an example of template matching when the current template is flipped and a modified template is used on the reference block side. Detailed Implementation

[0028] A more detailed understanding can be obtained by referring to the following description, which is given by way of example in conjunction with the accompanying drawings.

[0029] Figure 1A This diagram illustrates an example communication system 100 that can implement one or more of the disclosed embodiments. The communication system 100 can be a multiple access system that provides content (e.g., voice, data, video, messaging, broadcasting, etc.) to multiple wireless users. The communication system 100 enables multiple wireless users to access such content by sharing system resources (including wireless bandwidth). For example, the communication system 100 can employ one or more channel access methods, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal FDMA (OFDMA), Single Carrier FDMA (SC-FDMA), Zero-Tail Unique Word DFT Extended OFDM (ZT UW DTS-s OFDM), Unique Word OFDM (UW-OFDM), Resource Block Filtered OFDM, Filter Bank Multicarrier (FBMC), etc.

[0030] like Figure 1AAs shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, Public Switched Telephone Network (PSTN) 108, Internet 110, and other networks 112. However, it should be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, 102d can be any type of device configured to operate and / or communicate in a wireless environment. As an example, WTRUs 102a, 102b, 102c, and 102d (any of which may be referred to as a “station” and / or “STA”) may be configured to transmit and / or receive wireless signals and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscription-based units, pagers, cellular phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated processing chain environments), consumer electronics devices, devices operating on commercial and / or industrial wireless networks, etc. Any of WTRUs 102a, 102b, 102c, and 102d may be interchangeably referred to as a UE.

[0031] The communication system 100 may also include base station 114a and / or base station 114b. Each of base stations 114a and 114b can be any type of device configured to have a wireless interface with at least one of WTRUs 102a, 102b, 102c, and 102d to facilitate access to one or more communication networks, such as CN 106 / 115, Internet 110, and / or other networks 112. As an example, base stations 114a and 114b may be base transceiver stations (BTS), Node-B, eNode B, home Node B, home eNode B, gNB, NR Node B, site controller, access point (AP), wireless router, etc. Although base stations 114a and 114b are each depicted as a single element, it should be understood that base stations 114a and 114b may include any number of interconnected base stations and / or network elements.

[0032] Base station 114a may be part of RAN 104 / 113, which may also include other base stations and / or network elements (not shown), such as base station controllers (BSCs), radio network controllers (RNCs), relay nodes, etc. Base station 114a and / or base station 114b may be configured to transmit and / or receive radio signals on one or more carrier frequencies, which may be referred to as cells (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide radio service coverage for a specific geographic area, which may be relatively fixed or may vary over time. A cell may also be divided into cell sectors. For example, the cell associated with base station 114a may be divided into three sectors. Therefore, in one embodiment, base station 114a may include three transceivers, i.e., one transceiver for each sector of the cell. In embodiments, base station 114a may employ multiple-input multiple-output (MIMO) technology and may use multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in a desired spatial direction.

[0033] Base stations 114a and 114b can communicate with one or more of WTRUs 102a, 102b, 102c, and 102d via air interface 116. Air interface 116 can be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). Air interface 116 can be established using any suitable radio access technology (RAT).

[0034] More specifically, as described above, the communication system 100 can be a multiple access system and can employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, base stations 114a and WTRUs 102a, 102b, and 102c in RAN 104 / 113 can implement radio technologies such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which can establish air interfaces 115 / 116 / 117 using Wideband CDMA (WCDMA). WCDMA can include communication protocols such as High-Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA can include High-Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High-Speed ​​UL Packet Access (HSUPA).

[0035] In the embodiment, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which can establish air interface 116 using Long Term Evolution (LTE) and / or LTE-Advanced (LTE-A) and / or LTE-Advanced Pro (LTE-A Pro).

[0036] In the embodiment, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as NR radio access, which can establish air interface 116 using new radio (NR).

[0037] In this embodiment, base station 114a and WTRUs 102a, 102b, and 102c can implement multiple radio access technologies. For example, base station 114a and WTRUs 102a, 102b, and 102c can simultaneously implement LTE radio access and NR radio access, for example, using the dual connectivity (DC) principle. Therefore, the air interface used by WTRUs 102a, 102b, and 102c can be characterized by multiple types of radio access technologies and / or transmissions sent to and / or received from multiple types of base stations (e.g., eNB and gNB).

[0038] In other embodiments, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as IEEE 802.11 (i.e., Wi-Fi), IEEE 802.16 (i.e., Global Microwave Access Interoperability (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Provisional Standard 2000 (IS-2000), Provisional Standard 95 (IS-95), Provisional Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data Rate GSM Evolution (EDGE), GSM EDGE (GERAN), etc.

[0039] Figure 1ABase station 114b can be, for example, a wireless router, a home Node B, a home eNode B, or an access point, and can utilize any suitable RAT to facilitate wireless connectivity in a local area (e.g., business premises, homes, vehicles, campuses, industrial facilities, air corridors (e.g., for drone use), roads, etc.). In one embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.11 to establish a wireless local area network (WLAN). In another embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, base station 114b and WTRUs 102c, 102d can utilize cellular-based RATs (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish picocells or femtocells. Figure 1A As shown, base station 114b can have a direct connection to Internet 110. Therefore, base station 114b can access Internet 110 without needing to go through CN106 / 115.

[0040] RAN 104 / 113 can communicate with CN 106 / 115, which can be any type of network configured to provide voice, data, application, and / or Voice over Internet Protocol (VoIP) services to one or more of WTRU 102a, 102b, 102c, and 102d. Data can have different Quality of Service (QoS) requirements, such as different throughput requirements, latency requirements, fault tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. CN 106 / 115 can provide call control, billing services, mobile location services, prepaid calling, internet connectivity, video distribution, etc., and / or perform advanced security functions such as user authentication. Although Figure 1A Although not shown, it should be understood that RAN 104 / 113 and / or CN 106 / 115 can communicate directly or indirectly with other RANs that use the same RAT as or a different RAT than RAN 104 / 113. For example, in addition to being connected to RAN 104 / 113, which may utilize NR radio technology, CN 106 / 115 can also communicate with another RAN (not shown) that uses GSM, UMTS, CDMA2000, WiMAX, E-UTRA, or WiFi radio technology.

[0041] CN 106 / 115 can also serve as a gateway for WTRU 102a, 102b, 102c, 102d to access PSTN 108, the Internet 110, and / or other networks 112. PSTN 108 may include a circuit-switched telephone network providing Common Old-Style Telephone Service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices using common communication protocols such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and / or Internet Protocol (IP) from the TCP / IP Internet Protocol suite. Network 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, network 112 may include another CN connected to one or more RANs that may use the same RAT as or a different RAT than RAN 104 / 113.

[0042] Some or all of the WTRUs 102a, 102b, 102c, and 102d in the communication system 100 may include multi-mode capability (e.g., WTRUs 102a, 102b, 102c, and 102d may include multiple transceivers for communicating with different wireless networks via different wireless links). For example, Figure 1A The WTRU 102c shown can be configured to communicate with base station 114a, which can use cellular-based radio technology, and with base station 114b, which can use IEEE 802 radio technology.

[0043] Figure 1B This is a system diagram illustrating example WTRU 102. (See diagram below.) Figure 1B As shown, WTRU 102 may include a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keyboard 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power supply 134, a global positioning system (GPS) chipset 136, and / or other peripheral devices 138, etc. It should be understood that, consistent with the embodiments, WTRU 102 may include any sub-combination of the foregoing elements.

[0044] Processor 118 can be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. Processor 118 can perform signal encoding, data processing, power control, input / output processing, and / or any other functions that enable WTRU 102 to operate in a wireless environment. Processor 118 can be coupled to transceiver 120, and transceiver 120 can be coupled to transmitting / receiving element 122. Although Figure 1B While the processor 118 and transceiver 120 are depicted as separate components, it should be understood that the processor 118 and transceiver 120 may be integrated together in an electronic package or chip.

[0045] Transmitting / receiving element 122 can be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) via air interface 116. For example, in one embodiment, transmitting / receiving element 122 can be an antenna configured to transmit and / or receive RF signals. In another embodiment, transmitting / receiving element 122 can be a transmitter / detector, for example, configured to transmit and / or receive IR, UV, or visible light signals. In yet another embodiment, transmitting / receiving element 122 can be configured to transmit and / or receive both RF and optical signals. It should be understood that transmitting / receiving element 122 can be configured to transmit and / or receive any combination of wireless signals.

[0046] Although Figure 1B While the transmitting / receiving element 122 is depicted as a single element, the WTRU 102 may include any number of transmitting / receiving elements 122. More specifically, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmitting / receiving elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals via the air interface 116.

[0047] Transceiver 120 can be configured to modulate signals transmitted by transmitting / receiving element 122 and demodulate signals received by transmitting / receiving element 122. As described above, WTRU 102 can have multimode capability. Therefore, transceiver 120 can include multiple transceivers to enable WTRU 102 to communicate via various RATs (e.g., NR and IEEE 802.11).

[0048] The processor 118 of WTRU 102 can be coupled to a speaker / microphone 124, a keyboard 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) unit or an organic light-emitting diode (OLED) display unit) and can receive user input data therefrom. The processor 118 can also output user data to the speaker / microphone 124, keyboard 126, and / or display / touchpad 128. Furthermore, the processor 118 can access and store information from any type of suitable memory (e.g., non-removable memory 130 and / or removable memory 132). Non-removable memory 130 may include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. Removable memory 132 may include a Subscriber Identity Module (SIM) card, a Memory Stick, a Secure Digital (SD) memory card, etc. In other embodiments, the processor 118 can access and store information from memory that is not physically located on WTRU 102 (e.g., a server or home computer (not shown)).

[0049] The processor 118 can receive power from the power supply 134 and can be configured to distribute power to and / or control other components in the WTRU 102. The power supply 134 can be any suitable device for powering the WTRU 102. For example, the power supply 134 may include one or more dry cell batteries (such as nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0050] The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) about the current location of the WTRU 102. As a supplement to or alternative to the information from the GPS chipset 136, the WTRU 102 may receive location information from base stations (e.g., base stations 114a, 114b) via air interface 116, and / or determine its location based on the timing of signals received from two or more nearby base stations. It should be understood that, consistent with the embodiments, the WTRU 102 may acquire location information using any suitable location determination method.

[0051] The processor 118 can also be coupled to other peripheral devices 138, which may include one or more software and / or hardware modules that provide additional features, functions, and / or wired or wireless connectivity. For example, peripheral devices 138 may include accelerometers, electronic compasses, satellite transceivers, digital cameras (for photos and / or video), Universal Serial Bus (USB) ports, vibration devices, television transceivers, hands-free headsets, Bluetooth® modules, FM radio units, digital music players, media players, video game player modules, internet browsers, virtual reality and / or augmented reality (VR / AR) devices, activity trackers, etc. Peripheral devices 138 may include one or more sensors, which may be gyroscopes, accelerometers, Hall effect sensors, magnetometers, orientation sensors, proximity sensors, temperature sensors, time sensors; geolocation sensors; altimeters, light sensors, touch sensors, magnetometers, barometers, gesture sensors, biometric sensors, and / or humidity sensors.

[0052] WTRU 102 may include a full-duplex radio for which the transmission and reception of some or all signals (e.g., associated with specific subframes for both UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous. The full-duplex radio may include an interference management unit to reduce and / or substantially eliminate self-interference through hardware (e.g., chokes) or through signal processing via a processor (e.g., a separate processor (not shown) or via processor 118). In embodiments, WTRU 102 may include a half-duplex radio for which the transmission and reception of some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) or downlink (e.g., for reception) may be concurrent and / or simultaneous.

[0053] Figure 1C This is a system diagram illustrating RAN 104 and CN 106 according to an embodiment. As described above, RAN 104 can communicate with WTRUs 102a, 102b, and 102c via air interface 116 using E-UTRA radio technology. RAN 104 can also communicate with CN 106.

[0054] RAN 104 may include eNode-B 160a, 160b, and 160c, but it should be understood that, consistent with the embodiments, RAN 104 may include any number of eNode-Bs. eNode-B 160a, 160b, and 160c may each include one or more transceivers for communicating with WTRU 102a, 102b, and 102c via air interface 116. In one embodiment, eNode-B 160a, 160b, and 160c may implement MIMO technology. Therefore, for example, eNode-B 160a may use multiple antennas to transmit and / or receive radio signals from WTRU 102a.

[0055] Each of the eNode-B 160a, 160b, and 160c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, and user scheduling in the UL and / or DL, etc. Figure 1C As shown, eNode-B 160a, 160b, and 160c can communicate with each other via the X2 interface.

[0056] Figure 1C The CN 106 shown may include a Mobility Management Entity (MME) 162, a Serving Gateway (SGW) 164, and a Packet Data Network (PDN) Gateway (or PGW) 166. While each of the foregoing elements is depicted as part of the CN 106, it should be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0057] The MME 162 can connect to each eNode-B 162a, 162b, 162c in RAN 104 via the S1 interface and can be used as a control node. For example, the MME 162 can be responsible for authenticating users of WTRUs 102a, 102b, 102c, bearer activation / deactivation, selecting specific service gateways, etc., during the initial attachment of WTRUs 102a, 102b, 102c. The MME 162 can provide control plane functions for handover between RAN 104 and other RANs (not shown) employing other radio technologies (such as GSM and / or WCDMA).

[0058] The SGW 164 can connect to each eNode B 160a, 160b, and 160c in RAN 104 via the S1 interface. The SGW 164 typically routes and forwards user data packets to / from WTRUs 102a, 102b, and 102c. The SGW 164 can perform other functions, such as anchoring the user plane during eNode B handover, triggering paging when DL data is available for WTRUs 102a, 102b, and 102c, and managing and storing the context of WTRUs 102a, 102b, and 102c.

[0059] The SGW 164 can be connected to the PGW 166, which can provide WTRU 102a, 102b, and 102c with access to packet-switched networks (such as the Internet 110) to facilitate communication between WTRU 102a, 102b, and 102c and IP-enabled devices.

[0060] CN 106 can facilitate communication with other networks. For example, CN 106 can provide WTRUs 102a, 102b, and 102c with access to a circuit-switched network (e.g., PSTN 108) to facilitate communication between WTRUs 102a, 102b, and 102c and traditional landline communication equipment. For example, CN 106 may include or be able to communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server), which serves as an interface between CN 106 and PSTN 108. Furthermore, CN 106 can provide WTRUs 102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0061] Although Figures 1A to 1D The WTRU is described as a wireless terminal, but in some representative embodiments, it is envisioned that such a terminal may use (e.g., temporarily or permanently) a wired communication interface with a communication network.

[0062] In a representative embodiment, the other network 112 may be a WLAN.

[0063] A WLAN in Infrastructure Basic Services Set (BSS) mode can have an access point (AP) for the BSS and one or more stations (STAs) associated with that AP. The AP may have access or an interface to a distribution system (DS) or another type of wired / wireless network that carries traffic entering and / or leaving the BSS. Traffic originating outside the BSS destined for a STA can reach and be delivered to the STA via the AP. Traffic originating from a STA destined for a destination outside the BSS can be sent to the AP for delivery to the appropriate destination. Traffic between STAs within the BSS can be sent via the AP; for example, a source STA can send traffic to the AP, and the AP can deliver the traffic to the destination STA. Traffic between STAs within the BSS can be considered and / or referred to as peer-to-peer traffic. Peer-to-peer traffic can be sent between a source STA and a destination STA (e.g., directly between them) using Direct Link Establishment (DLS). In some representative embodiments, the DLS may use 802.11e DLS or 802.11z Tunneled DLS (TDLS). WLANs using the Standalone BSS (IBSS) mode can function without an access point (AP), and STAs within the IBSS or using the IBSS (e.g., all STAs) can communicate directly with each other. The IBSS communication mode may sometimes be referred to as a "self-organizing" communication mode in this document.

[0064] When operating in 802.11ac infrastructure mode or a similar mode, the AP can transmit beacons on a fixed channel (e.g., the primary channel). The primary channel can have a fixed width (e.g., a 20 MHz bandwidth) or a width dynamically set via signaling. The primary channel can be the operating channel of the BSS and can be used by STAs to establish connections with the AP. In some representative embodiments, Carrier Sense Multiple Access / Collision Avoidance (CSMA / CA) can be implemented, for example, in an 802.11 system. For CSMA / CA, STAs including the AP (e.g., each STA) can listen on the primary channel. If the primary channel is listened to / detected and / or determined to be busy by a particular STA, that particular STA can back off. In a given BSS, a single STA (e.g., only one station) can transmit at any given time.

[0065] High-throughput (HT) STAs can communicate using a 40 MHz wide channel, for example, by combining a primary 20 MHz channel with adjacent or non-adjacent 20 MHz channels to form a 40 MHz wide channel.

[0066] Very High Throughput (VHT) STAs can support 20 MHz, 40 MHz, 80 MHz, and / or 160 MHz wide channels. 40 MHz and / or 80 MHz channels can be formed by combining consecutive 20 MHz channels. A 160 MHz channel can be formed by combining eight consecutive 20 MHz channels, or by combining two non-consecutive 80 MHz channels (a configuration known as 80+80). For the 80+80 configuration, after channel coding, data is divided into two streams by a segmented parser. Each stream can be processed separately using Inverse Fast Fourier Transform (IFFT) and time-domain processing. The streams can be mapped onto the two 80 MHz channels, and the data can be transmitted by the transmitting STA. At the receiver of the receiving STA, the above operations for the 80+80 configuration can be reversed, and the combined data can be sent to the Media Access Control (MAC).

[0067] 802.11af and 802.11ah support sub-1 GHz operating modes. Compared to the channel operating bandwidth and carrier used in 802.11n and 802.11ac, 802.11af and 802.11ah reduce the channel operating bandwidth and carrier. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV Blank Band (TVWS) spectrum, while 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to representative embodiments, 802.11ah can support instrument-type control / machine-type communication, such as MTC devices in macro coverage areas. MTC devices may have certain capabilities, such as limited capabilities, including support (e.g., only support) certain and / or limited bandwidths. MTC devices may include batteries with a battery life exceeding a threshold (e.g., to maintain a very long battery life).

[0068] WLAN systems that can support multiple channels and channel bandwidths (e.g., 802.11n, 802.11ac, 802.11af, and 802.11ah) include channels that can be designated as the primary channel. The bandwidth of the primary channel can be equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel can be set and / or limited by the STA that supports the minimum bandwidth operating mode among all STAs operating in the BSS. In the example of 802.11ah, for STAs that support (e.g., only support) the 1 MHz mode (e.g., MTC type devices), the primary channel can be 1 MHz wide, even if the AP and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and / or other channel bandwidth operating modes. Carrier Sense and / or Network Allocation Vector (NAV) settings can depend on the status of the primary channel. If the primary channel is busy, for example because an STA (which only supports the 1 MHz operating mode) is transmitting to the AP, the entire available band can be considered busy even if most of the band remains idle and available.

[0069] In the United States, the available frequency band for 802.11ah is 902 MHz to 928 MHz. In South Korea, the available frequency band is 917.5 MHz to 923.5 MHz. In Japan, the available frequency band is 916.5 MHz to 927.5 MHz. Depending on the country code, the total available bandwidth for 802.11ah ranges from 6 MHz to 26 MHz.

[0070] Figure 1D This is a system diagram illustrating RAN 113 and CN 115 according to an embodiment. As described above, RAN 113 can communicate with WTRUs 102a, 102b, and 102c via air interface 116 using NR radio technology. RAN 113 can also communicate with CN 115.

[0071] RAN 113 may include gNBs 180a, 180b, and 180c, but it should be understood that RAN 113 may include any number of gNBs, consistent with the embodiments. gNBs 180a, 180b, and 180c may each include one or more transceivers for communicating with WTRUs 102a, 102b, and 102c via air interface 116. In one embodiment, gNBs 180a, 180b, and 180c may implement MIMO technology. For example, gNBs 180a and 180b may utilize beamforming to transmit signals to and / or receive signals from gNBs 180a, 180b, and 180c. Therefore, for example, gNB 180a may use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 102a. In an embodiment, gNBs 180a, 180b, and 180c may implement carrier aggregation technology. For example, gNB 180a can transmit multiple component carriers to WTRU 102a (not shown). A subset of these component carriers may be on unlicensed spectrum, while the remaining component carriers may be on licensed spectrum. In embodiments, gNBs 180a, 180b, and 180c may implement Coordinated Multipoint (CoMP) technology. For example, WTRU 102a may receive coordinated transmissions from gNBs 180a and 180b (and / or gNB 180c).

[0072] WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using transmissions associated with a scalable set of parameters. For example, OFDM symbol spacing and / or OFDM subcarrier spacing can vary for different transmissions, different cells, and / or different portions of the radio transmission spectrum. WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using multiple or scalable length subframes or transmission time intervals (TTIs) (e.g., containing different numbers of OFDM symbols and / or absolute times of varying durations).

[0073] gNBs 180a, 180b, and 180c can be configured to communicate with WTRUs 102a, 102b, and 102c in standalone and / or non-standalone configurations. In standalone configuration, WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c without simultaneously accessing other RANs (e.g., eNode-Bs 160a, 160b, and 160c). In standalone configuration, WTRUs 102a, 102b, and 102c can utilize one or more of gNBs 180a, 180b, and 180c as mobility anchors. In standalone configuration, WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using signals in unlicensed frequency bands. In a non-standalone configuration, WTRUs 102a, 102b, and 102c can communicate / connect with another RAN (e.g., eNode-B 160a, 160b, and 160c) while simultaneously communicating / connecting with gNBs 180a, 180b, and 180c. For example, WTRUs 102a, 102b, and 102c can implement DC principles to communicate substantially simultaneously with one or more gNBs 180a, 180b, and 180c and one or more eNode-Bs 160a, 160b, and 160c. In a non-standalone configuration, eNode-Bs 160a, 160b, and 160c can be used as mobility anchors for WTRUs 102a, 102b, and 102c, and gNBs 180a, 180b, and 180c can provide additional coverage and / or throughput to serve WTRUs 102a, 102b, and 102c.

[0074] Each of gNBs 180a, 180b, and 180c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, network slicing support, dual connectivity, interoperability between NR and E-UTRA, routing of user plane data to User Plane Functions (UPF) 184a and 184b, and routing of control plane information to Access and Mobility Management Functions (AMF) 182a and 182b, etc. Figure 1D As shown, gNB 180a, 180b, and 180c can communicate with each other via the Xn interface.

[0075] Figure 1DThe CN 115 shown may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one Session Management Function (SMF) 183a, 183b, and possibly a Data Network (DN) 185a, 185b. While each of the foregoing elements is depicted as part of the CN 115, it should be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0076] AMF 182a and 182b can connect to one or more gNBs 180a, 180b, and 180c in RAN 113 via the N2 interface and can be used as control nodes. For example, AMF 182a and 182b can be responsible for authenticating users of WTRU 102a, 102b, and 102c, supporting network slicing (e.g., handling different PDU sessions with different requirements), selecting specific SMF 183a and 183b, managing registration areas, terminating NAS signaling, mobility management, etc. AMF 182a and 182b can use network slicing to customize CN support for WTRU 102a, 102b, and 102c based on the service types they are using. For example, different network slices can be established for different use cases, such as services relying on Ultra Reliable Low Latency (URLLC) access, services relying on Enhanced Massive Mobile Broadband (eMBB) access, and services relying on Machine Type Communication (MTC) access. The AMF162 can provide control plane functions for switching between RAN 113 and other RANs (not shown) that employ other radio technologies, such as LTE, LTE-A, LTE-A Pro and / or non-3GPP access technologies such as WiFi.

[0077] SMFs 183a and 183b can connect to AMFs 182a and 182b via the N11 interface in CN 115. SMFs 183a and 183b can also connect to UPFs 184a and 184b via the N4 interface in CN 115. SMFs 183a and 183b can select and control UPFs 184a and 184b, and configure service routes through UPFs 184a and 184b. SMFs 183a and 183b can perform other functions, such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notifications. PDU session types can be IP-based, non-IP-based, Ethernet-based, etc.

[0078] UPF 184a and 184b can be connected to one or more gNBs 180a, 180b, and 180c in RAN 113 via the N3 interface. This provides WTRU 102a, 102b, and 102c with access to a packet-switched network (e.g., Internet 110), facilitating communication between WTRU 102a, 102b, and 102c and IP-enabled devices. UPF 184 and 184b can perform other functions, such as routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering downlink packets, and providing mobility anchoring.

[0079] CN 115 can facilitate communication with other networks. For example, CN 115 may include or be able to communicate with an IP gateway (such as an IP Multimedia Subsystem (IMS) server), which serves as an interface between CN 115 and PSTN 108. Furthermore, CN 115 can provide WTRUs 102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, WTRUs 102a, 102b, and 102c can be connected to local data networks (DNs) 185a and 185b via the N3 interface with UPFs 184a and 184b and the N6 interface between UPFs 184a and 184b and DNs 185a and 185b.

[0080] Given Figures 1A to 1D And to Figures 1A to 1D The corresponding descriptions herein indicate that one or more of the functions described herein, including one or more of the following, can be performed by one or more emulation devices (not shown): WTRU102a to 102d, base stations 114a and 114b, eNode-B 160a to 160c, MME 162, SGW 164, PGW 166, gNB 180a to 180c, AMF 182a and 182b, UPF 184a and 184b, SMF 183a and 183b, DN 185a and 185b, and / or any other devices described herein. An emulation device can be one or more devices configured to emulate one or more of the functions described herein. For example, an emulation device can be used to test other devices and / or emulate network and / or WTRU functions.

[0081] Simulation devices can be designed to perform one or more tests on other devices in laboratory and / or carrier network environments. For example, one or more simulation devices can perform one or more functions, fully or partially implemented and / or deployed as part of a wired and / or wireless communication network, to test other devices within the communication network. One or more simulation devices can perform one or more functions, temporarily implemented / deployed as part of a wired and / or wireless communication network. Simulation devices can be directly coupled to another device for testing purposes and / or can perform tests using over-the-air wireless communication.

[0082] One or more simulation devices can perform one or more functions without being implemented / deployed as part of a wired and / or wireless communication network. For example, simulation devices can be used in test scenarios within test laboratories and / or non-deployed (e.g., test-oriented) wired and / or wireless communication networks to perform testing on one or more components. One or more simulation devices can be test equipment. Simulation devices can transmit and / or receive data using direct RF coupling and / or wireless communication via RF circuitry (e.g., which may include one or more antennas).

[0083] This application describes various aspects, including tools, features, examples, models, schemes, etc. Many of these aspects are described in detail, and often in a manner that may sound restrictive, at least to illustrate their respective features. However, this is for the purpose of clarity and does not limit the scope of this application or these aspects. In fact, all the different aspects can be combined and interchanged to provide other aspects. Furthermore, these aspects can be combined and interchanged with aspects described in prior applications.

[0084] The aspects described and envisioned in this application can be implemented in many different forms. Figures 5 to 16 Some examples can be provided, but other examples are envisioned. Figures 5 to 16 The discussion does not limit the breadth of implementations. At least one aspect generally relates to video encoding and decoding, and at least another aspect generally relates to the transmission of a generated or encoded bitstream. These and other aspects can be implemented as methods, apparatus, computer-readable storage media storing instructions for encoding or decoding video data according to any of the described methods, and / or computer-readable storage media storing bitstreams generated according to any of the described methods.

[0085] In this application, the terms “reconstruction” and “decoding” are used interchangeably, the terms “pixel” and “sample” are used interchangeably, and the terms “image”, “picture” and “frame” are used interchangeably.

[0086] This document describes various methods, each including one or more steps or actions for implementing the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined. Furthermore, terms such as "first" and "second" can be used in various examples to modify elements, components, steps, operations, etc., e.g., "first decoding" and "second decoding." The use of such terms does not imply a sequence of operations unless specifically required. Therefore, in this example, the first decoding does not need to be performed before the second decoding and can occur, for example, before, during, or within an overlapping period of the second decoding.

[0087] The various methods and other aspects described in this application can be used to modify the module, for example... Figure 2 and Figure 3 The decoding modules of the video encoder 200 and decoder 300 are shown. Furthermore, the subject matter disclosed herein can be applied to, for example, any type, format, or version of video encoding and decoding, whether described in standards or recommendations, whether pre-existing or future-developed, and any extensions to such standards and recommendations. Unless otherwise instructed or technically impractical, the aspects described in this application may be used individually or in combination.

[0088] The examples described in this application use various numerical values, such as chroma format (e.g., 4:2:0, 4:2:2, 4:4:4), block size or coding unit size (e.g., height, width), partition size, number of search regions, number of sub-regions or areas, search step size, number of reconstructed blocks, number of entries in the list, precision, number of directions, number of patterns, number of candidates, number of samples, number of references, bit value, sub-block size, number of parameters, flag value, multiplier value, constant value, range value, minimum value, maximum value, threshold, index value, refinement size, etc. These and other specific values ​​are used to describe the examples, and the aspects described are not limited to these specific values.

[0089] Figure 2 This is a diagram illustrating an example video encoder. Variations of the example encoder 200 are envisioned, but for clarity, encoder 200 is described below without describing all anticipated variations.

[0090] Before encoding, the video sequence can undergo pre-coding (201), for example, by applying a color transformation to the input color image (e.g., from RGB 4:4:4 to YCbCr 4:2:0), or by performing remapping of the input image components to obtain a signal distribution that is more robust to compression (e.g., using histogram equalization on one of the color components). Metadata can be associated with pre-processing and appended to the bitstream.

[0091] In encoder 200, the image is encoded by encoder elements as described below. The image to be encoded is divided (202) and processed in units, for example, coding units (CUs). Each unit is encoded using, for example, an intra-frame mode or an inter-frame mode. When a unit is encoded in intra-frame mode, it performs intra-frame prediction (260). In inter-frame mode, motion estimation (275) and compensation (270) are performed. The encoder determines (205) which of the intra-frame or inter-frame modes to use for encoding the unit and indicates the intra-frame / inter-frame decision by, for example, a prediction mode flag. The prediction residual is calculated, for example, by subtracting (210) the prediction block from the original image block.

[0092] The predicted residual is then transformed (225) and quantized (230). The quantized transform coefficients, along with motion vectors and other syntax elements (e.g., image segmentation information), are entropy encoded (245) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., the residual is directly encoded without applying either the transform or quantization process.

[0093] The encoder decodes the coded block to provide a reference for further prediction. The quantization transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (255) to reconstruct the image block. A loop filter (265) is applied to the reconstructed image to perform, for example, deblocking / SAO (Sample Adaptive Shift) / ALF (Adaptive Loop Filtering) filtering to reduce coding artifacts. The filtered image is stored in a reference image buffer (280).

[0094] Figure 3 This is a diagram illustrating an example video decoder. In the example decoder 300, the bitstream is decoded by decoder elements as described below. The video decoder 300 typically performs the same operations as... Figure 2 The encoding process is the reverse of the decoding process. Encoder 200 typically also performs video decoding as part of the video data encoding.

[0095] Specifically, the input to the decoder includes a video bitstream, which can be generated by the video encoder 200. The bitstream is first entropy decoded (330) to obtain transform coefficients, prediction modes, motion vectors, and other encoded information. Picture segmentation information indicates how the picture is segmented. The decoder can therefore segment (335) the picture based on the decoded picture segmentation information. The transform coefficients are dequantized (340) and inverse transformed (350) to decode the prediction residuals. The decoded prediction residuals and prediction blocks are combined (355) to reconstruct the image blocks. The prediction blocks can be obtained from intra-frame prediction (360) or motion-compensated prediction (i.e., inter-frame prediction) (375) (370). A loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (380). In some examples (e.g., for a given picture), the contents of the reference picture buffer 380 on the decoder 300 side can be the same as the contents of the reference picture buffer 280 on the encoder 200 side (e.g., for the same picture).

[0096] The decoded image can be further processed by post-decoding (385), such as inverse color transformation (e.g., from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping of the inverse of the remapping process performed in pre-encoding (201). Post-decoding can utilize metadata derived in pre-encoding and signaled in the bitstream. In the example, the decoded image (e.g., after applying a loop filter (365) and / or after post-decoding (385) if post-decoding is used) can be sent to a display device for rendering to the user.

[0097] Figure 4 This is a diagram illustrating system examples that can implement the various aspects and examples described herein. System 400 can be embodied as a device including the various components described below and configured to perform one or more aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, networked home appliances, and servers. Elements of system 400 can be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one example, the processing and encoder / decoder elements of system 400 are distributed across multiple ICs and / or discrete components. In many examples, system 400 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In many examples, system 400 is configured to implement one or more aspects described in this document.

[0098] System 400 includes at least one processor 410 configured to execute instructions loaded thereon to implement various aspects, such as those described in this document. Processor 410 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 400 includes at least one memory 420 (e.g., a volatile memory device and / or a non-volatile memory device). System 400 includes a storage device 440, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 440 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0099] System 400 includes an encoder / decoder module 430 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 430 may include its own processor and memory. The encoder / decoder module 430 represents a module that can be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both encoding and decoding modules. Furthermore, the encoder / decoder module 430 may be implemented as a separate element of system 400, or may be incorporated into processor 410 as a combination of hardware and software, as is known to those skilled in the art.

[0100] Program code to be loaded onto processor 410 or encoder / decoder 430 to execute the various aspects described in this document may be stored in storage device 440 and subsequently loaded onto memory 420 for execution by processor 410. According to various examples, one or more of processor 410, memory 420, storage device 440, and encoder / decoder module 430 may store one or more items of various kinds during the execution of the processes described in this document. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results of processing equations, formulas, operations, and operational logic.

[0101] In some examples, the memory within processor 410 and / or encoder / decoder module 430 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other examples, external memory (e.g., the processing device may be processor 410 or encoder / decoder module 430) is used for one or more of these functions. External memory may be memory 420 and / or storage device 440, such as volatile memory and / or non-volatile flash memory. In several examples, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one example, fast external volatile memory (e.g., RAM) is used as working memory for video encoding and decoding operations.

[0102] The inputs of the system 400 components can be provided by a variety of input devices indicated in block 445. Such input devices include, but are not limited to: (i) a radio frequency (RF) section that receives, for example, RF signals transmitted over the air by a broadcaster; (ii) component (COMP) input terminals (or sets of COMP input terminals); (iii) universal serial bus (USB) input terminals; and / or (iv) high-definition multimedia interface (HDMI) input terminals. Figure 4 Other examples not shown include composite video.

[0103] In various examples, the input device of block 445 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or band-limiting a signal to a frequency band); (ii) down-converting the selected signal; (iii) band-limiting it again to a narrower frequency band to select, for example, a signal band that may be referred to as a channel in some examples; (iv) demodulating the down-converted and band-limited signal; (v) performing error correction; and / or (vi) demultiplexing to select a desired data packet stream. The RF section in various examples includes one or more elements performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners performing a variety of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one set-top box example, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to the desired frequency band. Various examples rearrange the order of the above (and other) components, remove some of them, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as inserting amplifiers and analog-to-digital converters. In many examples, the RF section includes an antenna.

[0104] USB and / or HDMI terminals may include corresponding interface processors for connecting system 400 to other electronic devices across USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, within a separate input processing IC or processor 410 as needed. Similarly, aspects of USB or HDMI interface processing may be implemented, as needed, within a separate interface IC or processor 410. Demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, processor 410 and encoder / decoder 430, which operate in conjunction with memory and storage elements to process the data streams as needed for presentation on the output device.

[0105] Various components of system 400 can be housed within an integrated housing. Within the integrated housing, various components can be interconnected and data can be transferred between them using suitable connection means 425 (e.g., internal buses known in the art, including inter-integrated circuit (I2C) buses, wiring, and printed circuit boards).

[0106] System 400 includes a communication interface 450 that enables communication with other devices via a communication channel 460. The communication interface 450 may include, but is not limited to, a transceiver configured to transmit and receive data on the communication channel 460. The communication interface 450 may include, but is not limited to, a modem or network card, and the communication channel 460 may be implemented, for example, within a wired and / or wireless medium.

[0107] In several examples, data is provided to system 400 via streaming or otherwise using a wireless network, such as a Wi-Fi network (e.g., IEEE 802.11, where IEEE stands for Institute of Electrical and Electronics Engineers). In these examples, the Wi-Fi signal is received via a communication channel 460 and a communication interface 450 adapted for Wi-Fi communication. The communication channel 460 in these examples is typically connected to an access point or router, providing access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other examples use a set-top box to provide streaming data to system 400, delivering data via an HDMI connection to input block 445. Still other examples use an RF connection to input block 445 to provide streaming data to system 400. As mentioned above, several examples provide data in a non-streaming manner. Furthermore, several examples use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth® networks.

[0108] System 400 can provide output signals to a variety of output devices, including a display 475, a speaker 485, and other peripheral devices 495. Display 475 includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. Display 475 can be used in televisions, tablet computers, laptop computers, cellular phones (mobile phones), or other devices. Display 475 can also be integrated with other components (e.g., as in a smartphone) or separate (e.g., as an external monitor for a laptop computer). In various examples, other peripheral devices 495 include one or more of a standalone digital video disc (or digital multifunction disc) (DVD, for both terms), a disk player, a stereo system, and / or a lighting system. Various examples utilize one or more peripheral devices 495 that provide functionality based on the output of system 400. For example, a disk player performs the function of playing the output of system 400.

[0109] In various examples, control signals communicate between system 400 and display 475, speaker 485, or other peripheral devices 495 using signaling protocols such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control (with or without user intervention). Output devices can be communicatively coupled to system 400 via dedicated connections through corresponding interfaces 470, 480, and 490. Alternatively, output devices can be connected to system 400 via communication interface 450 using communication channel 460. Display 475 and speaker 485 can be integrated into a single unit within an electronic device (e.g., a television) along with other components of system 400. In various examples, display interface 470 includes a display driver, such as a timing controller (TCon) chip.

[0110] Display 475 and speaker 485 may alternatively be separated from one or more other components, for example, if the RF section of input 445 is part of a separate set-top box. In various examples where display 475 and speaker 485 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0111] The example can be executed by computer software implemented by processor 410, or by hardware, or by a combination of hardware and software. As a non-limiting example, the example can be implemented by one or more integrated circuits. Memory 420 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as a non-limiting example. Processor 410 can be of any type suitable for the technical environment and can include one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures, as a non-limiting example.

[0112] Various implementations involve decoding. As used herein, “decoding” can encompass, for example, all or part of the process performed on a received coded sequence to produce a final output suitable for display. In various examples, such processes include one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various examples, such processes also or alternatively include processes performed by a decoder of various implementations described herein, such as determining whether a neighboring coding unit (CU) is encoded in Reconstruction Reordered Intra-Block Copy (RRIBC) mode; adding intra-template matching prediction (ITMP) merge candidates associated with the neighboring CU to an ITMP merge candidate list based on the determination that the neighboring CU is encoded in RRIBC mode; flipping the template and / or block associated with the added ITMP merge candidate (e.g., where the flip direction is used to flip the template and / or block); performing the ITMP process; selecting an ITMP candidate from a refined ITMP candidate list; decoding the current block using the selected ITMP candidate, etc.

[0113] As another example, in one example "decoding" refers only to entropy decoding, in another example "decoding" refers only to differential decoding, and in yet another example "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to specifically refer to a subset of operations or generally to a broader decoding process will be clear based on the specific context of the description and is thought to be fully understood by those skilled in the art.

[0114] Various implementations involve encoding. In a manner similar to the above discussion of “decoding,” the term “encoding,” as used herein, can encompass, for example, all or part of the process performed on an input video sequence to produce an encoded bitstream. In various examples, such processes include one or more processes typically performed by an encoder, such as partitioning, differential coding, transform, quantization, and entropy coding. In various examples, such processes also or alternatively include processes performed by an encoder according to various embodiments described herein. In various examples, such processes also or alternatively include processes performed by an encoder according to various embodiments described herein, such as determining whether a neighboring coding unit (CU) is encoded in Reconstruction Reordered Intra-Block Copy (RRIBC) mode; based on determining that the neighboring CU is encoded in RRIBC mode, adding intra-template matching prediction (ITMP) merge candidates associated with the neighboring CU to an ITMP merge candidate list; flipping the template and / or block associated with the added ITMP merge candidate (e.g., where a flip direction is used to flip the template and / or block); performing the ITMP process; selecting an ITMP candidate from a refined ITMP candidate list; encoding the current block using the selected ITMP candidate, etc.

[0115] As another example, in one example "encoding" refers only to entropy encoding, in another example "encoding" refers only to differential encoding, and in yet another example "encoding" refers to a combination of differential and entropy encoding. Whether the phrase "encoding process" is intended to specifically refer to a subset of operations or generally to a broader encoding process will be clear based on the specific context of the description and is thought to be fully understood by those skilled in the art.

[0116] Note that the syntax elements used in this article, such as SearchRange_w, SearchRange_h, intra_tmp_flag, intra_tmp_idx, intra_tmp_fusion_flag, intra_tmp_fusion_idx, intra_tmp_lic_flag, intra_tmp_filter_flag, intra_tmp_sub_pel_precision_idx, intra_tmp_sub_pel_direction_idx, itmp_idx, oneNullComp flag, bvOneNullComp flag, bvUseFlip flag, and bvNullCompDir, are descriptive terms. Therefore, their use does not preclude the use of other syntax element names.

[0117] When a diagram is presented as a flowchart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flowchart of the corresponding method / process.

[0118] The implementations and aspects described herein can be implemented in, for example, methods or processes, apparatuses, software programs, data streams, or signals. Even if discussed only in the context of a single implementation (e.g., discussed only as a method), implementations of the discussed features can be implemented in other forms (e.g., apparatuses or programs). Apparatuses can be implemented in, for example, suitable hardware, software, and firmware. Methods can be implemented in, for example, a processor, where processor refers generally to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as computers, cellular phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0119] References to “an example” or “an example” or “an implementation” or “an implementation” and their variations mean that a particular feature, structure, characteristic, etc., described in connection with that example is included in at least one example. Therefore, the phrases “in an example” or “in a sample” or “in an implementation” or “in an implementation” and any other variations appearing in multiple places in this application do not necessarily refer to the same example.

[0120] Furthermore, this application may refer to "determining" various types of information. Determining information may include, for example, one or more of estimation information, calculation information, prediction information, or information retrieved from memory. Obtaining may include receiving, retrieving, constructing, generating, and / or determining.

[0121] Furthermore, this application may refer to "accessing" various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or one or more of these.

[0122] Furthermore, this application may refer to "receiving" various types of information. Like "accessing," "receiving" is intended to be a broad term. Receiving information may include, for example, accessing information or retrieving information (e.g., from memory) from one or more sources. Moreover, "receiving" generally involves, in some way, operations such as, storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0123] It should be understood that the use of any of the following “ / ”, “and / or”, and “…at least one of…”, such as in the cases of “A / B”, “A and / or B”, and “at least one of A and B”, is intended to cover selecting only the first listed option (A), selecting only the second listed option (B), or selecting both options (A and B). As another example, in the cases of “A, B, and / or C” and “at least one of A, B, and C”, such wording is intended to cover selecting only the first listed option (A), selecting only the second listed option (B), selecting only the third listed option (C), selecting only the first and second listed options (A and B), selecting only the first and third listed options (A and C), selecting only the second and third listed options (B and C), or selecting all three options (A, B, and C). It will be apparent to those skilled in the art and related fields that this can be extended to as many items as possible listed.

[0124] Furthermore, as used herein, the term "signaling" specifically refers to instructing the corresponding decoder to do something. Encoder signals may include, for example, residual signals, metadata, encoded or motion representation patterns and related signals (e.g., ITMP-related flags, LIC-related flags), partitioning patterns, reference images, motion vector predictors (MVP), motion vector differences (MVD), indices, candidate lists, indications of RRIBC patterns, flip types, flip directions, etc. In this way, in the example, the same parameters are used on both the encoder and decoder sides. Therefore, for example, the encoder can send (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. Conversely, if the decoder already has specific parameters as well as other parameters, signaling can be used without transmission (implicit signaling) to allow only the decoder to know and select specific parameters. Bit savings are achieved in many examples by avoiding the transmission of any actual functionality. It should be understood that signaling can be done in many ways. For example, in many examples, information is signaled to the corresponding decoder using one or more syntax elements, flags, etc. Although the foregoing content involves the verb form of the word "signal," the word "signal" can also be used as a noun in this article.

[0125] As will be apparent to those skilled in the art, implementations can generate signals of various formats to carry information that can, for example, be stored or transmitted. The information may include, for example, instructions for performing a method or data generated by one of the described implementations. For example, the signal may be formatted to carry a bit stream of the described example. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. The signal may be transmitted via a variety of different wired or wireless links known in the art. The signal may be stored on, or accessed or received from, a processor-readable medium.

[0126] This document describes numerous examples. Features of the examples may be provided individually or in any combination across various claim classes and types. Furthermore, examples may include one or more of the features, devices, or aspects described herein, individually or in any combination across various claim classes and types. For example, features described herein may be implemented in a bitstream or signal including information generated as described herein. This information may allow a decoder to decode a bitstream, encoder, bitstream, and / or decoder according to any of the embodiments described. For example, features described herein may be implemented by creating and / or transmitting and / or receiving and / or decoding a bitstream or signal. For example, features described herein may be implemented as a method, process, apparatus, medium storing instructions, medium storing data, or signal. For example, features described herein may be implemented by a TV, set-top box, cellular phone, tablet computer, or other electronic device performing decoding. The TV, set-top box, cellular phone, tablet computer, or other electronic device may display (e.g., using a monitor, screen, or other type of display) a resulting image (e.g., an image reconstructed from the residual of a video bitstream). The TV, set-top box, cellular phone, tablet computer, or other electronic device may receive a signal including an encoded image and perform decoding.

[0127] Intra-frame template matching prediction (ITMP) can be an intra-frame prediction mode. ITMP can involve copying (e.g., best) a prediction block from the reconstructed portion of the current frame, whose L-shaped template can match the current template. For a predefined search range, the encoder can search the reconstructed portion of the current frame for a template similar to (e.g., most similar) the current template and can use the corresponding block as a prediction block. The encoder can signal the use of ITMP to the decoder. The same prediction operation can be performed on the decoder side.

[0128] Predictive signals can be generated by matching the L-shaped, top-only, or left-only causal neighbor blocks of the current block with another block in a predefined search area, for example... Figure 5 As shown. Figure 5This shows an example of an intra-frame template matching search region. There can be multiple (e.g., 6) predefined search regions, for example... Figure 5 R1 to R6, as shown, can include reconstruction samples from the top and left CTUs, as well as partial reconstruction samples within the current CTU that can be located above, to the left, below the left, and above the right of the current block. The sum of absolute differences (SAD) can be used as the cost function for ITMP.

[0129] Given the search order of the multiple (e.g., 6) regions used, such as R4, R5, R6, R1, R2, and R3, the decoder can construct a candidate list of template-matching block vectors (e.g., up to 19 such vectors), which can be ranked (e.g., in ascending order) based on template cost (e.g., calculated based on SAD). One or more of the following modes can be supported for ITMP.

[0130] It supports a single predictor mode, where a single predictor can be selected from a candidate list. It supports a multi-predictor fusion mode, where multiple predictors can be blended to derive a final prediction block. The blending weights can be computed based on the template matching cost of each predictor, or using a suitable weight derivation method (e.g., a Wiener filter-based weight derivation method). It supports a sub-pixel precision mode, where, if a single predictor is used, sub-pixel precision, 1 / 2 pixel precision, 1 / 4 pixel precision, and / or 3 / 4 pixel precision can be employed, each with multiple (e.g., 8) possible orientations. It supports a linear filter model mode, where a linear filter can be learned between a reference template and the current template. The linear filter can be applied to the reference block. The linear filter model mode can be used for a single predictor, for example, when sub-pixel precision is not used.

[0131] The size of one or more (e.g., all) regions can be defined by SearchRange_w and SearchRange_h, which can be set to be proportional to the block size (BlkW, BlkH) to have a fixed number of SAD comparisons per pixel, for example based on the following: SearchRange_w = min(64,a BlkW) SearchRange_h = min(64,a BlkH) 'a' can be a constant that controls the tradeoff between gain and complexity (e.g., 'a' can be equal to 5).

[0132] The complexity of ITMP can be proportional to the template matching process, where the difference between the templates of the current block and the predicted block can be computed for each position within the available search range. The search range of one or more (e.g., all) search regions can be subsampled by a factor (e.g., factor 3) to accelerate the template matching process (this technique could be called sparse search). For example, during the search, every third position in the horizontal and / or vertical directions can be considered in the first step of the search. A refinement process can be performed after the first step is completed. During refinement, positions within the sparse search (e.g., all) can be examined and their template costs computed. Refinement can be performed via a second template matching search around the best match (e.g., or around several best candidates from the sparse search), for example, using a reduced range.

[0133] ITMP can be enabled for one or more (e.g., certain) CUs (e.g., CUs with width and / or height dimensions less than or equal to 64 pixels). The CU size used for intra-frame template matching (e.g., the maximum CU size) can be configurable.

[0134] The use of ITMP can be signaled at the CU level, for example by a dedicated flag and / or if decoder-side intra-mode derivation (DIMD) is not used in the current CU. For example, it can be inferred that if ITMP is used, other intra-modes are not used.

[0135] ITMP can be used for natural content and / or screen content. A multi-candidate ITMP can be performed, where a candidate list can be constructed, with candidate block vectors (BVs) ranked in ascending order based on their template matching cost. The index of the selected candidate can be signaled in the bitstream.

[0136] Additional ITMP patterns can be executed, including, for example, left-side template patterns and / or top-side template patterns (e.g., in addition to the L-shaped patterns described herein). Left-side and / or top-side template patterns can derive template matching candidates using (e.g., only) the left or top of the current prediction unit (PU). Different templates can be used during the matching process.

[0137] In some examples, ITMP can construct a candidate list where candidate BVs are ranked in ascending order of their template matching cost. The index of the selected candidate can be signaled in the bitstream.

[0138] In some examples, ITMP can select (e.g., only one) the BV with the minimum template matching cost, and may do so without additional signaling (e.g., the same process can be performed on both the encoder and decoder sides, so additional signaling to indicate the final predictor is unnecessary). In some examples (e.g., for camera-captured content), the decoder may struggle to find a perfect match, for example, when multiple blocks are similar to the current block and their respective template matching costs are close. The BV candidate with the minimum template matching cost may not be the optimal predictor.

[0139] In some examples, ITMP can use multiple candidates (e.g., to address the problem described above). A candidate list can be constructed. Candidate BVs on the list can be ranked in ascending order based on their template matching cost. The index can be signaled in the bitstream to indicate which candidate BV to use for the current block. This approach can leverage template matching to select a short list of promising candidates from a large number of possible BVs. The encoder (which can refer to the original current block) can make the final decision about which BV to use. A simplified syntax example associated with this approach can be shown in Table 1: Table 1 – Simplified Syntax Examples

[0140] If the current block uses ITMP, then intra_tmp_flag can be equal to one (1). intra_tmp_idx can indicate which BV in the candidate BV list can be used to identify the predicted block.

[0141] Candidate lists can be constructed based on sparse search and / or refined search, for example, as shown below. Figure 6 Execute as shown. Figure 6 Examples of sparse search and refinement search in the ITMP process are shown. In the example of sparse search, a subsampling factor can be set. For example, based on factor three (3), every third position in the horizontal and vertical directions can be tested, for example, in the first step (e.g., sparse search). Multiple (e.g., 30) optimal BVs can be maintained based on template matching costs. In the second step (e.g., refinement search), 3×3 blocks (e.g., each) around one or more (e.g., each of the 30) optimal BVs can be tested. 3×3 refinement can adhere to region boundaries. Refinement candidates of candidates selected within a region can (e.g., must) remain within that region. Multiple (e.g., 19) optimal BVs can be selected to form a candidate list. One of the candidates can be selected (e.g., by the encoder) as the final predictor. The index used for the final predictor can be signaled (e.g., by the encoder to the decoder).

[0142] ITMP syntax can be encoded or decoded, for example, if the DIMD flag is set to false and / or if the ITMP sps flag is set to true, as shown in the examples in Table 2: Table 2 – ITMP Syntax Examples

[0143] `intra_tmp_flag` indicates whether the intra-prediction type for the current block is IntraTMP. `intra_tmp_fusion_flag` indicates whether fusion is used for the current block. `intra_tmp_fusion_idx` specifies the candidate set used for IntraTMP fusion (e.g., `intra_tmp_fusion_idx` can range from 0 to 2). `intra_tmp_fusion_idx` can be used to indicate one of multiple (e.g., three) candidate sets: {BV0 to BV4}, {BV5 to BV9}, and {BV10 to BV14}. `intra_tmp_lic_flag` indicates whether ITMP combined with LIC is used. `intra_tmp_idx` specifies the index of the BV in the candidate list used for the current block (e.g., `intra_tmp_idx` can range from 0 to 18, and / or for candidates from L-shaped templates, the top and left templates can be included in the same candidate list). `intra_tmp_sub_pel_precision_idx` can specify the precision index used for the current block (e.g., `intra_tmp_sub_pel_precision_idx` can range from 0 to 3, which can be used to indicate integer pixel precision, 1 / 2 pixel precision, 1 / 4 pixel precision, and 3 / 4 pixel precision, respectively). `intra_tmp_sub_pel_direction_idx` can specify the subpixel direction index used for the current block (e.g., `intra_tmp_sub_pel_phase_idx` can range from 0 to 7).

[0144] ITMP can be improved by adding more candidates to the ITMP search process (e.g., by utilizing the block vectors of one or more surrounding blocks of the current block).

[0145] In ITMP mode, a template matching search process can be performed within a predefined search area to obtain multiple (e.g., up to 19) candidate block vectors. In some examples, the ITMP template matching search may disregard block vector information from one or more neighboring PUs of the current block. The ITMP template matching search process can be enhanced with additional (e.g., new) candidates, such as using one or more block vectors associated with neighboring PUs that can be encoded in IBC (Intra-Block Copy) mode or ITMP mode. These additional candidates may be referred to herein as ITMP merge candidates. Additional candidates can improve ITMP mode by providing the ITMP search process with more possible candidates. For example, some other implementations of ITMP mode may not explore block vectors from neighboring blocks, even if they may be more relevant to the properties of the current block. The ITMP candidate list can be expanded to include additional candidate block vectors (e.g., ITMP merge candidates).

[0146] In some examples (e.g., if the neighboring CU is ITMP encoded), a signaling flag can be used to instruct the current CU to copy the same ITMP information from the neighboring CU.

[0147] In the example, including ITMP merge candidates could involve testing BV (e.g., from a typical ITMP search range) Figure 5 Within the search range shown) and / or (e.g., typical) BV outside the ITMP search range (e.g. Figure 5 (Outside the search range shown). This scheme can be similar to the IBC merge pattern, where the merge index can be encoded to indicate which neighboring PU block vector to select. A PU encoded in the IBC merge pattern can utilize the selected block vectors of one or more neighboring PUs (e.g., a PU encoded in IBC or ITMP can be considered), which can be better correlated with the current PU. In some implementations of the IBC merge pattern, the ITMP block vector can be used as an IBC merge candidate, but not vice versa, which may impose constraints on the ITMP process. Therefore, using information from one or more neighboring PUs when performing template matching search for ITMP can lead to improved performance. A template matching cost process can be used to rank additional (e.g., new) candidates among already computed candidates (e.g., candidates from a sparse search).

[0148] In the example, the same merge candidate list used for IBC can be considered for ITMP. This list can include local and non-local neighbor candidates (e.g., up to 25 local and non-local PUs can be considered when selecting merge candidates). The processing order of neighboring CUs can be the same as the IBC merge pattern. Block vectors found within the local or non-local neighborhood of the current PU can be added to the current ITMP candidate list. For example, block vectors can be added after sparse candidates identified during the first step of the candidate selection process (e.g., sparse search) (e.g., block vectors can be concatenated to the end of the sparse search candidate list). A predefined number (e.g., up to 25) of merge candidates can be added. The corresponding template cost associated with the merge candidates can be obtained, for example, in the same way as for sparse candidates. Merge candidates can be ranked together with other candidates (e.g., sparse candidates). A predefined number of optimal (e.g., 30) candidates can be selected for further refinement. Therefore, by using ITMP merge candidates, the optimal (e.g., 30) candidates can include some candidates that do not belong to the sparse search.

[0149] The optimal (e.g., 30) candidates can be refined using a second template matching step (e.g., a refinement search process). In the example, the refinement search process for candidates from the merged candidate list can differ from the refinement process for candidates from the sparse search. For example, if the candidates come from the merged list, 11×11 refinement can be used around the selected point instead of 3×3 refinement.

[0150] Candidates tested during the refinement process (e.g., all candidates) can be ranked. A list of candidates (e.g., a final 19) can be selected. A candidate (e.g., one) can be selected from the list (e.g., 19) by the encoder (e.g., during the rate-distortion process). The index of the selected candidate (e.g., the index of the candidate in the list) can be signaled (e.g., to the decoder).

[0151] Other aspects of the encoding / decoding process for the ITMP CU can be performed, for example, after obtaining a candidate set of ITMP predictors for the ITMP CU using the techniques described herein. For example, a predictor indicated by an ITMP index can be selected. Residual blocks of the CU can be encoded / decoded based on the selected predictor (e.g., residual blocks can be added to the predictor blocks to produce a reconstructed CU on the decoder side).

[0152] The techniques described in this paper can expand the ITMP candidate list with additional (e.g., new) block values ​​(BVs) that may not have been previously checked during the ITMP process. Additional BVs can include IBC block vectors, and their search area can be larger than the ITMP BV. Additional BVs can include candidates that might not have been selected in the first pass of template matching (e.g., during a sparse search of a subsampled region). Additional BVs can be better correlated with the current block, for example, because they may have been previously selected for predicting nearby blocks.

[0153] By adding more merge candidates that can be inherited from neighboring PUs, the compression efficiency of the video encoder and decoder (codec) with respect to ITMP mode video block encoding and decoding can be improved. In some examples (e.g., if the neighboring PU is encoded in double-predict IBC mode), BVs (e.g., two BVs) can be included in the ITMP merge list. Otherwise (e.g., if the neighboring PU is not encoded in double-predict IBC mode), single-predict IBC BVs can be included in the ITMP merge list.

[0154] In some examples (e.g., if a neighboring PU is encoded in ITMP mode and intra_tmp_idx > 0), the BV selected for that PU and the cost-optimal ITMP candidate can be included in the ITMP merge list. The selected and cost-optimal BV can be the same, for example, if intra_tmp_idx equals 0, in which case one or the other candidate can be included in the ITMP list.

[0155] In some examples, N out of M available merge candidates can be used (e.g., only N out of M available merge candidates), for example, if M > N. Merge candidates outside the ITMP search scope can be given priority. In some examples, M ITMP merge candidates can be selected (e.g., M can be up to 50). In some examples, N merge candidates can ultimately be used (e.g., N=10) for testing and comparison with other candidates.

[0156] Increasing the sampling factor (e.g., from 3 to 4) can (e.g., slightly) change the refinement process (e.g., instead of 3×3 refinement, the refinement process can use 2 rows on the left, 2 rows on the top, 1 row on the right, and 1 row on the bottom).

[0157] In some examples, the BV refinement range can be reduced compared to some implementations (e.g., in the case of LIC), and / or fewer BV candidates from sparse search can be retained in LIC. Additional merged candidates can be utilized in a manner that maintains reasonable complexity for encoding and / or decoding operations.

[0158] The implementation of ITMP using merge candidates can be referred to as the ITMP merge pattern in this paper. The ITMP search process can be improved or optimized, and its complexity can be reduced. One or more bipredictive IBC candidates can be included in the ITMP merge list. One or more cost-optimal ITMP candidates can be included in the ITMP merge list. A subset of available merge candidates (e.g., N out of M available merge candidates) can be used (e.g., if M>N). IBC candidates can be prioritized. Merge candidates outside the ITMP search range can be prioritized. Refinement (e.g., 3×3 refinement) can be performed, for example, if a merge candidate is within the ITMP search region. The cost and / or location of tested template matching candidates can be stored (e.g., cached in memory) to skip duplicate template matching cost calculations.

[0159] Optimization refinements can be performed, such as if merge candidates are outside the ITMP search area. In the example, multiple (e.g., 25) candidates associated with neighboring PUs of the current CU can be added to the ITMP merge list. Candidates can be added in a similar (e.g., identical) manner and / or order to candidates used for the IBC merge mode (e.g., candidates could be selected BVs associated with already processed PUs). Such an ITMP merge list can be further expanded, for example, by adding more candidates from blocks encoded in IBC or ITMP modes.

[0160] In the example (e.g., if a neighboring PU is encoded in a double-predictive IBC mode), multiple (e.g., two) BVs from the double-predictive IBC mode (e.g., instead of one BV) can be used and added to the ITMP merge list (e.g., two BVs can be associated with a PU encoded in a double-predictive IBC mode).

[0161] In the example, the dual-prediction merge candidate may not include multiple (e.g., two) distinct ITMP prediction candidates associated with the PU. The ITMP prediction may include dual-prediction candidates. The ordering of dual-prediction candidates may be performed, for example, by calculating the template matching (TM) cost of the dual-prediction candidates as the average TM cost of the combined (e.g., two) single-prediction BV.

[0162] In some examples (e.g., if neighboring PUs are encoded in ITMP mode), the BV selected for a neighboring PU can be added to the ITMP merge list. The BV selected for a PU can be added to the list. The BV can be indicated, for example, by an ITMP index (e.g., intra_tmp_idx) signaled in the bitstream. As described herein, the BV may or may not be the BV with the minimum template cost. It can include the BV with the minimum template cost (e.g., this could be the first candidate on the final candidate list, e.g., intra_tmp_idx equals 0), for example, if intra_tmp_idx is greater than 0.

[0163] One or more of the candidate selection techniques described in this paper can be combined. For example, one or more bipredictive IBC candidates and the best ITMP candidate by template cost can be added to the ITMP candidate merging list (e.g., in addition to the candidates already defined in this paper).

[0164] The ITMP merge list can be expanded (e.g., to include up to 50 candidates), for example, by using one or more of the techniques described herein. The expanded candidate list can be used in the same way as the shorter list (e.g., to include up to 25 neighboring candidates).

[0165] The operations associated with constructing the ITMP merge candidate list can be optimized. In the example, up to 50 candidates can be added as ITMP merge candidates. Increasing the number of candidates can lead to improved coding performance, but may increase computational cost. Pre-selection of ITMP merge candidates can be applied, for example, to keep computational cost within reasonable limits. Pre-selection can be applied before template cost calculation (e.g., some candidates can be discarded before template matching cost calculation to reduce complexity). Pre-selection can be applied in a way that achieves reduced complexity without reducing coding gain compared to using an expanded candidate list (e.g., all 50 candidates).

[0166] In the example, a subset of the available merge candidates can be used (e.g., N out of M) (e.g., M > N). For example, there may be a total of M ITMP merge candidates available for the current CU (e.g., M can be less than or equal to 50). Instead of using all M candidates, which might increase the computational complexity of the encoder and / or decoder, the number of merge candidates can be limited to N (e.g., N = 5). The value of N can be preconfigured. If M > N, the ITMP merge candidate list can be shortened to N (e.g., the top N candidates from the original list can be kept on the ITMP merge list, and other candidates can be discarded). If M <= N, all M available candidates can be used.

[0167] In the example, the ITMP merge list can be populated in such a way that multiple (e.g., all) available neighboring IBC merge candidates can be added to the list first (e.g., including single-predictive BV and bidirectional BV, if available), followed by one or more ITMP candidates (e.g., if available). The candidate list can (e.g., then) be shortened, for example as described herein (e.g., by selecting the top N candidates on the list). Therefore, IBC candidates can be prioritized when constructing and pre-selecting ITMP candidates (e.g., N candidates).

[0168] In the example, it is located in the ITMP search area (e.g., as shown in the example). Figure 5 ITMP merge candidates other than those shown can be given priority when constructing the ITMP merge list, for example, regardless of whether the candidate comes from a neighboring block encoded by IBC or ITMP. The ITMP merge candidate list can be populated in this manner. The list can be (e.g., then) shortened (e.g., as described herein).

[0169] In the example, when the list needs to be shortened (e.g., to N candidates), early termination of the ITMP merge list construction can be applied (e.g., in all cases). For example, when N candidates have already been added to the list using one or more techniques described herein, the construction process can terminate without acquiring other ITMP merge candidates, and the N candidates can be used (e.g., as described herein).

[0170] In the example, the refinement operation applied to candidates derived from the sparse search can differ from the refinement operation applied to candidates derived from the ITMP merge list. For example, 3×3 refinement can be applied to candidates derived from the sparse search (e.g., as the first step in the search process), and 11×11 dense refinement can be applied to candidates selected from the ITMP merge candidates during the first step (e.g., dense refinement could mean that every candidate within an 11×11 window can be tested). Given the computational power of some encoders / decoders, 11×11 dense refinement may be overly complex.

[0171] In the example, if the candidate is located in the ITMP search area (e.g.) Figure 5 If the candidate is within one of the regions shown, a 3×3 refinement can be performed on the ITMP merge candidate, while if the candidate is outside the ITMP search region, an 11×11 refinement can be performed (e.g., regardless of whether the candidate comes from an IBC block or an ITMP block).

[0172] In the example, ITMP merge candidates associated with IBC blocks can use an 11×11 refinement process, while ITMP merge candidates associated with ITMP neighboring blocks can use a 3×3 refinement process (e.g., regardless of the block's position relative to the ITMP search area).

[0173] The search window sizes described herein (e.g., 3×3 and 11×11 search window sizes) are examples. They can generally refer to small and large refinement windows, one of which can add an acceptable computational complexity to the refinement process, while the other can add an undesirable computational complexity (e.g., acceptable and undesirable complexity levels can be based on software and / or hardware capabilities). Different search window sizes may be used in various implementations.

[0174] The Reconstruction Reordered Intra-Block Copy (RRIBC) mode can be extended to block vectors (e.g., any block vector with a component equal to zero). The RRIBC mode can be used for IBC-coded blocks. When RRIBC is applied, samples in the reconstructed block can be flipped according to the current block's flip type (e.g., in the flip direction). On the encoder side, the original block can be flipped, for example, before motion search and residual computation. The prediction block can be derived without flipping. On the decoder side, the reconstructed block can be flipped back to recover the original block. The use of the RRIBC mode implies the performance of flipping.

[0175] Supported flipping methods for RRIBC coded blocks can include horizontal flipping and vertical flipping. Syntax flags can be signaled for IBC AMVP coded blocks. Syntax flags can be used to indicate whether a reconstruction is flipped (e.g., indicating whether RRIBC mode is used). If a reconstruction is flipped, another flag can be signaled to specify the flipping type. For IBC merging, the flipping type can be inherited from neighboring blocks, e.g., no syntax signaling. Considering horizontal or vertical symmetry, the current block and the reference block can be horizontally or vertically aligned. In the example, if / when a horizontal flip is applied, the vertical component of the BV may be unsigned and / or inferred to be equal to zero. In the example, if / when a vertical flip is applied, the horizontal component of the BV may be unsigned and / or inferred to be equal to zero.

[0176] The use of RRIBC mode can be indicated by a signal, accompanied by (for example, followed by) an indication of the flip direction (e.g., horizontal or vertical). The direction with zero components (e.g., the horizontal or vertical component is zero) can be derived, for example, from the flip direction indicator indicated by the signal.

[0177] Alternatively (e.g., instead of the usual IBC AMVP list), when using RRIBC, multiple (e.g., two) block vector predictor (BVP) candidates and / or the symbols for non-zero BV components can be derived on the decoder side. For example, AMVP BVP0 can be set to the nearest valid location to the current block (e.g., -cbWidth or -cbHeight), such that the non-zero block vector difference (BVD) can always be negative, pointing to the left for a BV with zero vertical component, or pointing upwards for a BV with zero horizontal component. AMVP BVP1 can be set to the furthest location from the current block in the valid reference region, such as the left or top boundary of the IBC search region. Therefore, if BVP1 is selected, BVD can always be positive, pointing to the right for a BV with zero vertical component, or pointing downwards for a BV with zero horizontal component.

[0178] The IBC AMVP index can be signaled (e.g., optimally), which allows the symbols of non-zero BVD components to be obtained (e.g., derived) on the decoder side. The absolute magnitude of the non-zero BVD components can be signaled (e.g., further).

[0179] RRIBC patterns can be generalized to block vectors (e.g., any block vector with a vector component equal to zero). The systems and methods described herein can be extended to block vectors (BVs) with (e.g., one) null components, regardless of whether the BV uses an RRIBC pattern (e.g., regardless of whether flipping is used). In other words, a BV with (e.g., one) null component can be signaled to the decoder via a dedicated flag, even if block flipping is unfavorable. A BV with (e.g., one) null component (e.g., including an RRIBC block (an RRIBC block is a block with a flipping pattern applied)) can be signaled to the decoder, for example, via the bvOneNullComp flag. Instead of invoking the AMVP IBC list construction, BVP candidates can be determined in the same manner as RRIBC patterns, which can be adjusted to the boundaries of the valid IBC search region based on a horizontal or vertical direction (e.g., as indicated by the bvNullCompDir flag). The use of flipping can be signaled, for example, via the bvUseFlip flag. The flipping direction (e.g., horizontal or vertical) can be derived from the direction of the non-zero component (e.g., the bvNullCompDir flag). This scheme represents extending RRIBC to any block vector that has (e.g., one) a component equal to zero (0).

[0180] The syntax can be summarized as follows. If the `oneNullComp` flag indicates (e.g., one) that a BV component is zero, then another flag (e.g., `bvNullCompDir`) can indicate the direction of the zero component. If flipping is possible, then another flag (e.g., `bvUseFlip`) can indicate whether flipping is performed in the direction of the non-zero component. The zero component of a BV can be deduced from the `oneNullComp` and `bvNullCompDir` flags. Therefore, the zero component of a BV may not require encoding or decoding syntax elements associated with vector components.

[0181] The use of RRIBC modes and / or their generalization to (e.g., any) BV with (e.g., one) empty components can be controlled by higher-level syntactic elements, such as the Sequence Parameter Set (SPS) flag. In some examples, the use of inversion can be controlled by the SPS flag. Table 3 shows syntactic examples that can assume the use of inversion is enabled (e.g., in both cases).

[0182] Table 3 – Syntax Examples

[0183] RRIBC mode can refer to native RRIBC mode (as described in this article) and (as also) the promotion of RRIBC when useFlip is enabled at the SPS level (as described in this article).

[0184] For example, compression efficiency of state-of-the-art video codecs can be improved for blocks encoded in ITMP merge mode, such as when an ITMP merge list is used. ITMP merge performance can be improved, such as when a neighboring block is encoded in RRIBC mode, for example, when flipping can be used. Information about flipping (e.g., the BV of neighboring CUs used only during ITMP merge list construction, such as when merging candidates from neighboring blocks) can be considered during ITMP merge list construction. For example, one of the neighboring blocks may be RRIBC encoded and flipping may be applied. Knowledge of RRIBC encoding and flipping application can be useful (e.g., when creating the ITMP merge list, when creating ITMP predictions, and / or when calculating template matching (TM) costs), for example, to handle the encoding / decoding process in a more consistent manner when constructing ITMP blocks using ITMP merge mode.

[0185] Examples regarding ITMP merge candidates and coexistence with RRIBC flip mode are provided. ITMP merge candidates obtained from neighboring merge candidate blocks encoded in RRIBC can be handled in a more consistent manner in ITMP merge mode. As described herein, merge candidates can be discarded (e.g., to avoid adding merge candidates to the ITMP merge list), for example, if neighboring CUs are encoded in RRIBC mode.

[0186] ITMP merge candidates can be added to the list, for example, if neighboring CUs are encoded in RRIBC mode. A flip can be performed during the template matching process, where one or more of the following can occur, for example: the reference template can be flipped; the reference block can be flipped; and / or (e.g., the same) the flip process can be applied to (e.g., all) refined blocks in RRIBC mode.

[0187] ITMP merge candidates can be added to the list, for example, if neighboring CUs are encoded in RRIBC mode. A flip can be performed during the template matching process, where one or more of the following can occur, for example: the template of the current block can be flipped; the current block can be flipped; and / or (e.g., the same) the flip process can be applied to (e.g., all) refined blocks in RRIBC mode.

[0188] It can test both flipped and non-flipped versions of ITMP merge candidates, for example, regardless of whether neighboring merge blocks are encoded in RRIBC mode.

[0189] Figure 7 An example of vertical flipping is shown. Several schemes can be derived, depending on whether the flip is applied to a reference template or the current template. Flipping an image patch can be applied to the reference side (e.g., ...). Figure 7 (as shown in the left side) or the current block side (such as...) Figure 7 (As shown on the right). A regular template or a modified template can be used on the reference side. Template flipping can be applied to either the reference side or the current block side.

[0190] Figure 8 An example of horizontal flipping is shown. Flipping of image patches can be applied to the reference side (e.g., ...). Figure 8 (as shown in the left side) or the current block side (such as...) Figure 8 (As shown on the right). A regular template or a modified template can be used on the reference side. Template flipping can be applied to either the reference side or the current block side.

[0191] The example can assume that ITMP merge candidates are obtained from neighboring blocks encoded in RRIBC mode. Information about the RRIBC mode and flip direction of the merge candidate block (e.g., neighboring blocks) can be stored for use in the ITMP merge process. The current ITMP candidate can inherit the same RRIBC mode and flip direction as the merge candidate block. Information (e.g., about the RRIBC mode and flip direction) can be used, for example, during the ITMP process of the current block, such as when creating predictions and calculating the template matching cost of the ITMP merge candidate.

[0192] In some examples, merge candidates can be discarded. In some examples, it can be determined whether an ITMP merge candidate to be considered can be obtained from a neighboring block encoded using RRIBC flip mode. ITMP merge candidates can be discarded, for example, not added to the ITMP merge list. In some examples, ITMP merge candidates can be removed from the ITMP merge list (e.g., if they have already been added to the ITMP merge list).

[0193] In some examples, flipping can be applied to the reference block side; for example, templates and / or reference image blocks can be flipped horizontally or vertically, depending on the RRIBC flip direction of the merge candidate blocks. The flip direction can be indicated, for example, using syntax elements that can be obtained / inherited from the ITMP merge candidate blocks.

[0194] In some examples, reference blocks using a standard template can be used during the ITMP search process. Figure 9 Examples comparing the regular template to the current block template in both vertical and horizontal flip scenarios are shown. A non-flipped version of the reference template may not match the current template. In some examples, (e.g., only) the left-side template can be used in a vertical flip scenario. In some examples, (e.g., only) the top template can be used in a horizontal flip scenario.

[0195] In some examples, such as when using only the left or top template, appropriate template flipping can be applied. Figure 10 Examples are shown using only the left-side template and only the top-side template with appropriate flipping applied to the templates. In the examples, the top template can be flipped horizontally (e.g., in the case of using horizontal flipping) and / or the left-side template can be flipped vertically (e.g., in the case of using vertical flipping) before calculating the template matching cost, which can lead to Figure 10 The scenario shown involves flipped left-side and top-side templates. Figure 9 The flipped version of the template shown.

[0196] In the example, after an appropriate flip is applied to the left or top template, the left or top template can match the current template (e.g., a more accurate match). The template matching process can achieve better cost and can perform a more accurate template matching process, for example, compared to using a non-flipped and / or full template.

[0197] In some examples, using the full template may be inappropriate (e.g., even if the full template is flipped according to the flipped RRIBC direction), for example because the left template (e.g., in the case of a horizontal flip) or the top template (e.g., in the case of a vertical flip) may include structural features different from those included in the current block template. The result could be higher costs and / or the possibility of discarding the current ITMP merge candidate during ITMP competition. A candidate may be discarded or not selected, even if it might give the best possible prediction when the appropriate flip is applied.

[0198] Figure 11 This shows an example of flipping a reference block to create a prediction. (Example:) Figure 11 As shown, during the creation of the prediction block, the reference block can be flipped (for example, if necessary). The flipping of the reference block can be in the same direction as the flipping of the template, which can be indicated using the merge / inheritance syntax element.

[0199] The management of the ITMP process may include using appropriate template cost calculations and reference block flipping, as described herein. Other encoder / decoder processes can continue as usual.

[0200] In some examples, template modification mechanisms and the flipping of reference blocks or templates can be utilized. Template modification and flipping can be implemented. In some examples, reference blocks with modified templates can be used during the ITMP search process.

[0201] Figure 12 This shows an example of using a modified template in both vertical and horizontal flip scenarios. The modified template can be taken from neighboring pixels of a reference block. For example... Figure 12 As shown, a modified but non-flipped version of the reference template may not match the current template, but they may be structurally similar. Therefore, in some examples, the flipping of the reference template can be performed, for example, before the template cost calculation.

[0202] like Figure 12 As shown, a flipped version of the reference template can structurally match the current template more closely. However, additional template reorganization can be performed before calculating the template cost. For example (e.g., for a precise process), a modified vertical template can be flipped vertically, and a horizontal template can be flipped horizontally.

[0203] Figure 13 This shows an example of a modified template that uses vertical and horizontal flipping. For example... Figure 13 As shown, templates created and managed in this way can structurally match the current template.

[0204] Template portions (e.g., all template portions) can be used during template matching (e.g., left-side template, top-left template, and top template can be used). Template matching can be improved by modifying and flipping templates, for example. Template matching can support more accurate template matching, for example, compared to regular or non-flipped reference templates.

[0205] In some examples, during the creation of the prediction block, a flip may (e.g., it may be necessary) be performed on the reference block. The same flip direction can be applied during the template matching process. The flip direction can be indicated, for example, by a syntax element obtained / inherited from the ITMP merge candidate block.

[0206] In some examples, flipping can be applied to the current block side. For example, the template and / or the current image block can be flipped horizontally or vertically, depending on the RRIBC flip direction of the merge candidate block, for example. The flip direction can be indicated, for example, by a syntax element obtained / inherited from the ITMP merge candidate block.

[0207] Figure 14 Examples of using modified templates and regular templates are shown when flipping the current block side. For example... Figure 14 As shown, a regular template or a modified template can be used on the reference block side, for example, without flipping (as mentioned earlier, flipping can be done on the current block side).

[0208] In some examples, a regular template can be used on the reference side. Figure 15 This illustrates an example of template matching when the current template is flipped and a regular template can be used on the reference block side. After flipping, either the left template (e.g., only) or the top template (e.g., only) can be used in the template matching process and template matching cost calculation. Other template sections can be omitted, for example, because they may involve different structural parts of the image.

[0209] Figure 16 This shows an example of template matching when the current template is flipped and a modified template is used on the reference block side. For example... Figure 16 As shown, a modified template can be used on the reference block side, which allows for the use (e.g., all) of the template portion in the template matching process and cost calculation. Figure 16 As can be observed, the flipped template from the current block and the modified template from the reference block are highly matched. Therefore, using appropriate flipping can lead to a better ITMP search process. As shown and described in this paper, ITMP candidates obtained from RRIBC blocks can be processed in a more accurate manner.

[0210] In some examples, the current image block can be flipped. The reference block can be used as is, and the current block can be flipped depending on the flip direction, such as before creating the residual signal. Flipping the current block can provide computational savings, for example, compared to flipping the reference block. For instance, flipping the current block can be performed only once and can be used for all reference blocks, while flipping the reference image block may result in (e.g., all) reference blocks being flipped individually.

[0211] In some examples, the flip process can be applied to the refinement stage. For instance, if a merge candidate is selected for the next stage (e.g., refinement is applied to that candidate), the flip process can be applied to (e.g., all) refinement candidates in the same way as the initial candidate, such as if the initial candidate is an RRIBC merge candidate. Each refinement candidate can be tagged with RRIBC syntax in the same way as the initial candidate; for example, a refinement candidate can inherit RRIBC syntax from the initial candidate. In some examples, the flip process can be applied to every refinement candidate.

[0212] In some examples, it may not be necessary to refine the ITMP merge candidates obtained from merge blocks encoded in flipped mode (e.g., RRIBC mode).

[0213] Template costs can be normalized. In some examples, template costs can be normalized before being compared with other candidates, such as if / when using a left-only template or a top-only template (e.g., as explained in this paper). In some examples of normalization, the average cost per pixel can be calculated and then scaled by the total number of pixels in the full template. For example, the number of pixels in a partial template (e.g., top or left, depending on the flip direction) can be represented as M. The total number of pixels in the full template can be represented as N. The partial template cost can be represented as J. The cost per pixel can be determined as follows: J p =J / M The normalization cost can be determined as follows: J N =J p N Normalization cost J N This can then be used as an approximate cost for the complete template. The normalized cost J can be used in the ITMP candidate ranking process. N .

[0214] In some examples, weighted normalization cost can be used. For example, J N This can be represented as partial template cost J, w1 J and w2 The sum of J, where w1 and w2 can be adjusted to compensate for unused template portions. For example, the total template cost can be determined as follows: JN =J+1 / 4 J+1 / 4 J The total template cost can be simplified as follows: J N =J+( w1+w2) J In some examples, ITMP merge candidates can be tested with and without flipping applied, for example, regardless of whether the ITMP merge block candidate uses an RRIBC mode. This might mean that it is not necessary to inherit the RRIBC mode from neighboring blocks, for example, because all variants are tested. For example, additional template matching tests can be applied to (e.g., one) ITMP merge candidate, such as performing vertical flipping tests and horizontal flipping tests. Flipping can be implemented in any way, such as as described herein. Candidates can (e.g., then) be ranked among each other in a conventional manner, such as by comparing template costs. For each ITMP merge candidate, multiple associated costs can be provided, such as a standard cost, a horizontally flipped cost, and a vertically flipped cost. If flipping is tested, flipping can be applied to the template during the template matching process and applied to the image block (e.g., the current image block or a reference image block, which may depend on the implementation, such as different schemes for flipping image blocks as explained herein). Syntax elements can be used to signal to the decoder the flipping mode used, such as no flipping, vertical flipping, or horizontal flipping mode.

[0215] Although the features and elements have been described above in specific combinations, those skilled in the art will understand that each feature or element can be used alone or in any combination with other features and elements. Furthermore, the methods described herein can be implemented in computer programs, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (e.g., internal hard disks and removable disks), magneto-optical media, and optical media (e.g., CD-ROM discs and digital multifunction optical discs (DVDs)). The processor associated with the software can be used to implement a radio frequency transceiver for a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. A video decoder, comprising: Processor, configured to at least: Determine whether the first neighboring coding unit (CU) is encoded in Reconstructed Reordered Intra-Block Copy (RRIBC) mode; Based on determining that the neighboring CU is encoded in the RRIBC mode, the intra-template matching prediction (ITMP) merging candidate associated with the first neighboring CU is added to the first ITMP merging candidate list; Flip at least one of the templates and blocks associated with the added ITMP merge candidate, wherein a flip direction is used to flip at least one of the templates and blocks; A second ITMP candidate list is generated from the first merged ITMP candidate list and the sparse candidate list. Refine the second ITMP candidate list; A third ITMP candidate list is generated from the refined second ITMP candidate list; Select an ITMP candidate from the third ITMP candidate list; as well as Decode the current block using the selected ITMP candidate.

2. The video decoder of claim 1, wherein the processor is configured to determine whether the first neighboring CU is coded with the RRI BC mode comprises: The processor is configured to determine whether the first neighboring CU is encoded in the RRIBC mode based on an indicator associated with the first neighboring CU.

3. The video decoder of claim 1, wherein, based on determining that the second neighboring CU is encoded in the RRIBC mode, the processor is further configured to exclude the second neighboring CU from being added to the ITMP merge candidate list.

4. The video decoder of claim 1, wherein the processor is further configured to: Receive the indicator associated with the ITMP candidate; and wherein the processor is configured to select the ITMP candidate from the third ITMP candidate list comprises: The processor is configured to select the ITMP candidate from the third candidate ITMP list based on the received indicator.

5. The video decoder of claim 1, wherein the processor is further configured to apply the flip direction to the thinning block in the RRIBC mode.

6. The video decoder of claim 1, wherein the processor is configured to flip at least one of the template and the block during template matching.

7. The video decoder of claim 1, wherein the processor is configured to rank the third ITMP candidate list, which includes at least one of one or more flipped merge candidates and one or more unflipped merge candidates.

8. A video decoding method, comprising: Determine whether the first neighboring coding unit (CU) is encoded in Reconstructed Reordered Intra-Block Copy (RRIBC) mode; Based on determining that the neighboring CU is encoded in the RRIBC mode, the intra-template matching prediction (ITMP) merging candidate associated with the first neighboring CU is added to the first ITMP merging candidate list; Flip at least one of the templates and blocks associated with the added ITMP merge candidate, wherein a flip direction is used to flip at least one of the templates and blocks; A second ITMP candidate list is generated from the first merged ITMP candidate list and the sparse candidate list. Refine the second ITMP candidate list; A third ITMP candidate list is generated from the refined second ITMP candidate list; Select an ITMP candidate from the third ITMP candidate list; as well as Decode the current block using the selected ITMP candidate.

9. The method of claim 8, further comprising determining whether the first neighboring CU is encoded in the RRIBC mode based on an indicator associated with the first neighboring CU.

10. The method of claim 8, wherein based on determining that the second neighboring CU is encoded in the RRIBC mode, the method further includes excluding the second neighboring CU from being added to the ITMP merge candidate list.

11. The method of claim 8, further comprising: Receive the indicator associated with the ITMP candidate; as well as The ITMP candidate is selected from the third candidate ITMP list based on the received indicator.

12. The method of claim 8, further comprising applying the flip direction to the refinement block in the RRIBC mode.

13. The method of claim 8, further comprising flipping at least one of the template and the block during template matching.

14. The method of claim 8, further comprising ranking the third ITMP candidate list, which includes at least one of one or more flipped merge candidates and one or more unflipped merge candidates.

15. The video decoder of claim 1 or the method of claim 8, wherein the block is a current block or a reference block, and wherein the template is associated with the current block or the reference block.

Images