Template matching prediction using subsampling

Abstract

Devices and techniques for template matching prediction (TMP) using subsampling. An exemplary device for video decoding may determine that the subsampled TMP is valid for the current block. The device may determine a search area for performing the TMP. The device may determine a subsampled search area for performing the subsampled TMP based on the search area. The device may perform the subsampled TMP on the subsampled search area to determine a candidate prediction block for the current block. The device may decode the current block based on the candidate prediction block.

Description

Technical Field

[0001] (Cross - reference to related applications) This application is filed on July 1, 2022, and claims priority to European Patent Application No. 22305971.8 entitled "TEMPLATE MATCHING", which is incorporated by reference in its entirety as if fully set forth herein.

Background Art

[0002] Video encoding and decoding may involve searching for a predicted block of the current block. Therefore, improving the efficiency and / or speed of these searches can lead to an improvement in coding performance.

Summary of the Invention

[0003] Disclosed herein are systems, methods, and means associated with template matching (TMP), such as intra template matching (IntraTMP) or inter template matching (interTMP) using subsampling. An exemplary video decoding device may determine that subsampled template matching prediction (TMP) is valid (e.g., for a current block). The device may determine a search area for performing TMP. The device may determine a subsampled search area for performing subsampled TMP based on the search area. The subsampled search area may be determined based on a subsampling factor. The subsampling factor may be preset. The subsampling factor may be indicated in the video data. For example, the device may receive an indication of the subsampling factor and determine the subsampled search area based on the indicated subsampling factor. For example, the subsampling factor may have a value of N, and the subsampled search area may include every Nth pixel of the search area. The device may perform subsampled TMP on the subsampled search area to determine a candidate prediction block for the current block. The device may decode the current block based on the candidate prediction block.

[0004] The video decoding device may identify a refined search area smaller than the search area for performing refined TMP around the candidate prediction block. The device may perform refined TMP on the refined search area to determine a refined prediction block for the current block. Decoding the current block based on the candidate prediction block may involve decoding the current block based on the refined prediction block.

[0005] The device may identify a refined search area for performing refined TMP based on candidate prediction blocks for the current block. The device may determine a refined subsampling coefficient associated with the refined TMP. The refined subsampling coefficient may be smaller than the subsampling coefficient. The device may determine a subsampled refined search area from the refined search area based on the refined subsampling coefficient. The refined subsampling coefficient may be preset. The subsampling coefficient may be indicated in the video data. The device may receive a refined subsampling coefficient indication and determine a subsampled refined search area based on the refined subsampling coefficient indication. For example, the refined subsampling coefficient may be N / 2. The subsampled refined search area may include every N / 2th pixel of the refined search area. The device may perform refined TMP on the subsampled refined search area to determine a refined prediction block for the current block.

[0006] For example, the device may receive a subsampling indication configured to indicate whether subsampled TMP is effective. The indication may be signaled at the sequence level (e.g., in the sequence parameter set), at the slice level, and / or at the coding unit level. The device may determine that subsampled TMP is effective for the current block based on the subsampling TMP indication.

[0007] The device may receive a refined TMP indication configured to indicate whether refined TMP is effective for the current block. The device may identify a refined search area smaller than the search area around the candidate prediction block based on the refined TMP indication indicating that refined TMP is effective for the current block. The device may perform refined TMP on the refined search area to determine a refined prediction block for the current block.

[0008] The subsampled TMP may be performed on the subsampled search area based on one or more set parameters. The set parameters may include a template size associated with the TMP, a search range factor associated with the TMP, a minimum block size to which the TMP may be applied, a maximum block size to which the TMP may be applied, a search area associated with the TMP, a slice type to which the subsampled TMP is applied, a subsampling factor, wherein determining the subsampled search area includes determining the subsampled search area based on the subsampling factor, the subsampling factor, a first indication of whether to refine the subsampled TMP, or a second indication of whether to perform a hierarchical template matching search. The subsampled TMP may be associated with IntraTMP.

[0009] An exemplary video encoding device may determine to enable the subsampled TMP. The device may determine a search area for performing the TMP. The device may determine a subsampled search area for performing the subsampled TMP based on the search area. The device may perform the subsampled TMP on the subsampled search area to determine a candidate prediction block for the current block. The device may encode the current block based on the candidate prediction block.

[0010] A video encoding device can determine a subsampling coefficient and can determine a subsampled search area for performing subsampled TMP based on the subsampling coefficient. The device may include an indication of the subsampling coefficient in the video data. The device may determine a refined search area smaller than the search area for performing refined TMP around a candidate prediction block. The device may perform refined TMP on the refined search area to determine a refined prediction block for the current block. The current block may be encoded based on the refined prediction block.

[0011] A video encoding device may determine a subsampling coefficient having a value of N. Determining a subsampled search area for performing subsampled TMP may involve determining the subsampled search area for performing subsampled TMP based on the subsampling coefficient. The subsampled search area may include every Nth pixel of the search area.

[0012] Determining the subsampled search area may involve determining the subsampled search area based on the subsampling coefficient. A video encoding device may identify a refined search area for performing refined TMP based on a candidate prediction block for the current block. The video encoding device may determine a refined subsampling coefficient associated with the refined TMP. The refined subsampling coefficient may be smaller than the subsampling coefficient. The video encoding device may determine a subsampled refined search area from the refined search area based on the refined subsampling coefficient. The video encoding device may perform refined TMP on the subsampled refined search area to determine a refined prediction block for the current block.

Brief Description of the Drawings

[0013]

Figure 1A

Figure 1B

Figure 1C

Figure 1D

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

[0014] A more detailed understanding can be obtained from the following description, given by way of example in conjunction with the accompanying drawings.

[0015] FIG. 1A is a diagram illustrating an exemplary communication system 100 in which one or more of the disclosed embodiments may be implemented. The communication system 100 may be a multiple access system that provides content such as voice, data, video, messaging, broadcast, etc. to a plurality of wireless users. The communication system 100 may enable a plurality of wireless users to access such content through sharing of system resources including wireless bandwidth. For example, the communication system 100 may employ one or more channel access methods such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), zero-tail unique-word DFT-Spread OFDM (ZT UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block filtered OFDM, filter bank multicarrier (FBMC).

[0016] As shown in Figure 1A, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, public switched telephone network (PSTN) 108, Internet 110, and other networks 112, although it will be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, 102d can be any type of device configured to operate and / or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, 102d, any of which may be referred to as a "station" and / or "STA", can be configured to transmit and / or receive wireless signals and can be user equipment (UE), mobile station, fixed subscriber unit or mobile subscriber unit, subscriber-based unit, pager, cellular phone, personal digital assistant (PDA), smartphone, laptop, netbook, personal computer, wireless sensor, hotspot or Mi-Fi device, Internet of Things (IoT) device, watch or other wearable device, head-mounted display (HMD), vehicle, drone, medical device and application (e.g., for remote surgery), industrial device and application (e.g., robots and / or other wireless devices operating in an industrial and / or automated processing chain context), home appliance device, device operating in a commercial wireless network and / or industrial wireless network, etc. Any of the WTRUs 102a, 102b, 102c, and 102d can also be interchangeably referred to as a UE.

[0017] The communication system 100 may also include base station 114a and / or base station 114b. Each of base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks, such as CN106 / 115, the Internet 110, and / or other network 112. By way of example, base stations 114a, 114b may be a base transceiver station (BTS), Node B, eNode B, Home Node B, Home eNode B, gNB, NR Node B, site controller, access point (AP), wireless router, etc. Although base stations 114a, 114b are each shown as a single element, it will be understood that base stations 114a, 114b may include any number of interconnected base stations and / or network elements.

[0018] Base station 114a can be part of RAN104 / 113, which may also include other base stations and / or network elements (not shown) such as a base station controller (BSC), a radio network controller (RNC), a relay node, etc. Base station 114a and / or base station 114b can be configured to transmit and / or receive radio signals at one or more carrier frequencies, which may be referred to as a cell (not shown). These frequencies can be a licensed spectrum, an unlicensed spectrum, or a combination of a licensed spectrum and an unlicensed spectrum. The cell can provide wireless service coverage to a specific geographic area that can be relatively fixed or can change over time. The cell can be further divided into cell sectors. For example, the cell associated with base station 114a can be divided into three sectors. Thus, in one embodiment, base station 114a can include three transceivers, i.e., one transceiver for each sector of the cell. In one embodiment, base station 114a can employ multiple-input multiple-output (MIMO) technology and utilize multiple transceivers for each sector of the cell. For example, beamforming can be used to transmit and / or receive signals in a desired spatial direction.

[0019] Base stations 114a, 114b can communicate with one or more of WTRUs 102a, 102b, 102c, 102d via air interface 116, which can be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, millimeter wave, infrared (IR), ultraviolet (UV), visible light, etc.). Air interface 116 can be established using any suitable radio access technology (RAT).

[0020] More specifically, as described above, the communication system 100 can be a multiple access system and can adopt one or more channel access methods such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, the base stations 114a within RAN104 / 113, and the WTRUs 102a, 102b, 102c can implement radio technologies such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which can use wideband CDMA (WCDMA) to establish the air interfaces 115 / 116 / 117. WCDMA can include communication protocols such as High-Speed Packet Access (HSPA) and / or evolved HSPA (HSPA+). HSPA can include High-Speed Downlink Packet Access (HSDPA) and / or High-Speed UL Packet Access (HSUPA).

[0021] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c can implement radio technologies such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which can use Long Term Evolution (LTE) and / or LTE-Advanced (LTE-A) and / or LTE-Advanced Pro (LTE-A Pro) to establish the air interface 116.

[0022] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c can implement radio technologies such as NR radio access, which can use New Radio (NR) to establish the air interface 116.

[0023] In one embodiment, base station 114a and WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, base station 114a and WTRUs 102a, 102b, 102c may implement both LTE radio access and NR radio access using, for example, the dual connectivity (DC) principle. Accordingly, the air interface utilized by WTRUs 102a, 102b, 102c may be characterized by transmissions sent to / from multiple types of radio access technologies and / or multiple types of base stations (e.g., eNBs and gNBs).

[0024] In other embodiments, base station 114a and WTRUs 102a, 102b, 102c may implement wireless technologies such as IEEE 802.11 (i.e., Wireless Fidelity (WiFi)), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), etc.

[0025] The base station 114b in FIG. 1A can be, for example, a wireless router, a home node B, a home eNode B, or an access point, and can utilize any suitable RAT to facilitate wireless connection in a local area such as an office, a home, a vehicle, a campus, an industrial facility, an aerial corridor (for use by drones, for example), a road, etc. In one embodiment, the base station 114b and the WTRUs 102c, 102d can implement a wireless technology such as IEEE 802.11 to establish a wireless local area network (WLAN). In one embodiment, the base station 114b and the WTRUs 102c, 102d can implement a wireless technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, the base station 114b and the WTRUs 102c, 102d can utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish a pico cell or a femto cell. As shown in FIG. 1A, the base station 114b can have a direct connection to the Internet 110. Thus, the base station 114b may not need to access the Internet 110 via the CN 106 / 115.

[0026] RAN 104 / 113 may communicate with CN 106 / 115, which may be any type of network configured to provide voice, data, applications, and / or voice over internet protocol (VoIP) services to one or more of WTRUs 102a, 102b, 102c, 102d. The data may have various quality of service (QoS) requirements, such as different throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. CN 106 / 115 may provide call control, billing services, mobile location-based services, prepaid calls, internet connectivity, video distribution, etc., and / or may implement high-level security functions such as user authentication. Although not shown in Figure 1A, it will be understood that RAN 104 / 113 and / or CN 106 / 115 may communicate directly or indirectly with other RANs that employ the same or a different radio access technology (RAT) as RAN 104 / 113. For example, in addition to being connected to RAN 104 / 113 which may utilize NR radio technology, CN 106 / 115 may communicate with another RAN (not shown) that employs GSM, UMTS, CDMA2000, WiMAX, E-UTRA, or WiFi radio technology.

[0027] CN106 / 115 can also function as a gateway for WTRU102a, 102b, 102c, 102d to access the PSTN108, the Internet 110, and / or other networks 112. The PSTN 108 can include a circuit-switched telephone network that provides a plain old telephone service (POTS). The Internet 110 can include a global system of interconnected computer networks and devices, and these networks and devices use a common communication protocol such as the transmission control protocol (TCP), the user datagram protocol (UDP), and / or the internet protocol (IP) of the TCP / IP internet protocol suite. The network 112 can include a wired communication network and / or a wireless communication network that is owned and / or operated by another service provider. For example, the network 112 can include another CN connected to one or more RANs that can employ the same RAT or a different RAT as the RAN 104 / 113.

[0028] Some or all of the WTRU102a, 102b, 102c, 102d in the communication system 100 can include a multi-mode function (for example, the WTRU102a, 102b, 102c, 102d can include multiple transceivers for communicating with different wireless networks via different wireless links). For example, the WTRU102c shown in Figure 1A can be configured to communicate with a base station 114a that can employ a cellular-based wireless technology and a base station 114b that can employ IEEE802 wireless technology.

[0029] Figure 1B is a system diagram showing an exemplary WTRU 102. As shown in Figure 1B, the WTRU 102 may include, among other things, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, a non-removable memory 130, a removable memory 132, a power supply 134, a global positioning system (GPS) chipset 136, and / or other peripheral devices 138. It will be understood that the WTRU 102 may include any partial combination of the foregoing elements while remaining consistent with one embodiment.

[0030] The processor 118 may be a general-purpose processor, a dedicated processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor 118 may perform signal coding, data processing, power control, input / output processing, and / or any other function that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, and the transceiver 120 may be coupled to the transmit / receive element 122. Although Figure 1B shows the processor 118 and the transceiver 120 as separate components, it will be understood that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.

[0031] The transmit / receive element 122 may be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) via the air interface 116. For example, in one embodiment, the transmit / receive element 122 may be an antenna configured to transmit and / or receive RF signals. In one embodiment, the transmit / receive element 122 may be an emitter / detector configured to transmit and / or receive, for example, IR signals, UV signals, or visible light signals. In yet another embodiment, the transmit / receive element 122 may be configured to transmit and / or receive both RF signals and optical signals. It will be understood that the transmit / receive element 122 may be configured to transmit and / or receive any combination of wireless signals.

[0032] Although the transmit / receive element 122 is shown in FIG. 1B as a single element, the WTRU 102 may include any number of transmit / receive elements 122. More specifically, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals via the air interface 116.

[0033] The transceiver 120 may be configured to modulate signals transmitted by the transmit / receive element 122 and demodulate signals received by the transmit / receive element 122. As noted above, the WTRU 102 may have a multimode function. Thus, the transceiver 120 may include multiple transceivers to enable the WTRU 102 to communicate via multiple RATs such as, for example, NR and IEEE 802.11.

[0034] The processor 118 of the WTRU 102 may be coupled to the speaker / microphone 124, keypad 126, and / or the display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light-emitting diode (OLED) display unit) and may receive data input by a user from these. The processor 118 may also output user data to the speaker / microphone 124, keypad 126, and / or the display / touchpad 128. In addition, the processor 118 may access information from and store data in any suitable type of memory, such as the non-removable memory 130 and / or the removable memory 132. The non-removable memory 130 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from and store data in a memory that is not physically located on the WTRU 102, such as on a server or a home computer (not shown).

[0035] Processor 118 may receive power from power supply 134 and may be configured to distribute and / or control power to other components in WTRU 102. Power supply 134 may be any suitable device for supplying power to WTRU 102. For example, power supply 134 may include one or more dry batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, and the like.

[0036] Processor 118 may also be coupled to GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of WTRU 102. In addition to, or instead of, information from GPS chipset 136, WTRU 102 may receive location information from a base station (e.g., base stations 114a, 114b) via air interface 116 and / or may determine its location based on the timing of signals received from two or more neighboring base stations. It will be understood that WTRU 102 may obtain location information by any suitable location determination method while remaining consistent with one embodiment.

[0037] Processor 118 may be further coupled to other peripheral devices 138, which may include one or more software and / or hardware modules that provide additional features, functionality, and / or wired or wireless connections. For example, the peripheral devices 138 may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (for photos and / or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, a virtual reality and / or augmented reality (VR / AR) device, an activity tracker, etc. The peripheral devices 138 may include one or more sensors, which may be one or more of a gyroscope, an accelerometer, a Hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor, a geolocation sensor, an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, and / or a humidity sensor.

[0038] The WTRU 102 may include a full-duplex radio in which some or all of the transmission and reception of signals associated with a particular subframe (e.g., for both UL (e.g., for transmission) and downlink (e.g., for reception)) may be parallel and / or simultaneous. The full-duplex radio may include an interference management unit for reducing and / or substantially eliminating self-interference either via hardware (e.g., choke) or via signal processing through a processor (e.g., a separate processor (not shown) or via processor 118). In one embodiment, the WTRU 102 may include a half-duplex radio for some or all of the transmission and reception of signals associated with a particular subframe (e.g., for either UL (e.g., for transmission) or downlink (e.g., for reception)).

[0039] Figure 1C is a system diagram showing RAN 104 and CN 106 according to one embodiment. As described above, RAN 104 may employ E-UTRA radio technology to communicate with WTRUs 102a, 102b, 102c via air interface 116. RAN 104 may also communicate with CN 106.

[0040] RAN 104 may include eNodeBs 160a, 160b, 160c, although it will be understood that RAN 104 may include any number of eNodeBs while remaining consistent with one embodiment. Each of eNodeBs 160a, 160b, 160c may include one or more transceivers for communicating with WTRUs 102a, 102b, 102c via air interface 116. In one embodiment, eNodeBs 160a, 160b, 160c may implement MIMO technology. Thus, eNodeB 160a, for example, may transmit wireless signals to and / or receive wireless signals from WTRU 102a using multiple antennas.

[0041] Each of eNodeBs 160a, 160b, 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, etc. As shown in Figure 1C, eNodeBs 160a, 160b, 160c may communicate with each other via the X2 interface.

[0042] CN 106 shown in Figure 1C may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (or PGW) 166. Although each of the foregoing elements is shown as part of CN 106, it will be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0043] The MME 162 can be connected to each of the eNodeBs 162a, 162b, 162c in the RAN 104 via an S1 interface and can function as a control node. For example, the MME 162 can play roles such as authenticating users of the WTRUs 102a, 102b, 102c, activating / deactivating bearers, selecting a specific serving gateway during the initial connection of the WTRUs 102a, 102b, 102c, etc. The MME 162 can provide control plane functions for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies such as GSM and / or WCDMA.

[0044] The SGW 164 can be connected to each of the eNodeBs 160a, 160b, 160c in the RAN 104 via an S1 interface. The SGW 164 can generally route and transfer user data packets to / from the WTRUs 102a, 102b, 102c. The SGW 164 can perform other functions such as the function of anchoring the user plane during handover between eNodeBs, the function of triggering paging when DL data is available to the WTRUs 102a, 102b, 102c, and the function of managing and storing the context of the WTRUs 102a, 102b, 102c.

[0045] The SGW 164 can be connected to the PGW 166, and the PGW 166 can provide the WTRUs 102a, 102b, 102c with access to a packet switched network such as the Internet 110 to facilitate communication between the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0046] CN106 can facilitate communication with other networks. For example, CN106 can provide access to a circuit-switched network such as the PSTN 108 to the WTRUs 102a, 102b, 102c to facilitate communication between the WTRUs 102a, 102b, 102c and a conventional landline communication device. For example, CN106 can include or communicate with an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that functions as an interface between CN106 and the PSTN 108. In addition, CN106 can provide the WTRUs 102a, 102b, 102c with access to another network 112, which can include other wired and / or wireless networks owned and / or operated by other service providers.

[0047] The WTRU is described as a wireless terminal in FIGS. 1A - 1D, but in certain representative embodiments, it is contemplated that such a terminal can (e.g., temporarily or permanently) use a wired communication interface with a communication network.

[0048] In a representative embodiment, the other network 112 can be a WLAN.

[0049] A WLAN in Infrastructure Basic Service Set (BSS) mode can have an access point (AP) of the BSS and one or more stations (STAs) associated with the AP. The AP can have access to or an interface to a Distribution System (DS) or another type of wired / wireless network that carries traffic within and / or outside the BSS. Traffic sent to an STA from outside the BSS can reach and be delivered to the STA through the AP. Traffic sent from an STA to a destination outside the BSS can be sent to the AP to be delivered to their respective destinations. Traffic between STAs within the BSS can be sent through the AP. For example, a source STA can send traffic to the AP, and the AP can deliver the traffic to the destination STA. Traffic between STAs within the BSS can be considered and / or referred to as peer-to-peer traffic. Peer-to-peer traffic can be sent between a source STA and a destination STA (e.g., directly between them) using direct link setup (DLS). In certain representative embodiments, DLS can use 802.11e DLS or 802.11z tunneled DLS (TDLS). A WLAN using Independent BSS (IBSS) mode may not have an AP, and STAs within the IBSS or STAs using the IBSS (e.g., all of the STAs) can communicate directly with each other. The IBSS mode of communication may also be referred to herein as the "ad hoc" communication mode.

[0050] When using the 802.11ac infrastructure operation mode or a similar operation mode, the AP can transmit beacons on a fixed channel such as the primary channel. The primary channel can be of a fixed width (e.g., a 20 MHz bandwidth) or a width dynamically set via signaling. The primary channel can be the operating channel of the BSS, but can also be used by the STA to establish a connection with the AP. In certain representative embodiments, for example, in an 802.11 system, Carrier Sense Multiple Access with Collision Avoidance (CSMA / CA) can be implemented. In the case of CSMA / CA, STAs including the AP (e.g., all STAs) can sense the primary channel. If the primary channel is sensed / detected and / or determined to be busy by a particular STA, the particular STA can back off. Only one STA (e.g., only one station) can transmit at any given time in a given BSS.

[0051] A High Throughput (HT) STA can use a 40 MHz wide channel for communication, and this 40 MHz wide channel can be formed, for example, through a combination of a primary 20 MHz channel and an adjacent or non - adjacent 20 MHz channel.

[0052] A Very High Throughput (VHT) STA may support channels with widths of 20 MHz, 40 MHz, 80 MHz, and / or 160 MHz. A 40 MHz and / or 80 MHz channel may be formed by combining consecutive 20 MHz channels. A 160 MHz channel may be formed by combining eight consecutive 20 MHz channels or by combining two non - consecutive 80 MHz channels, which may be referred to as an 80+80 configuration. In the case of the 80+80 configuration, after channel encoding, the data may pass through a segment parser that may divide the data into two streams. Inverse Fast Fourier Transform (IFFT) processing and time - domain processing may be performed separately for each stream. The streams may be mapped to two 80 MHz channels and the data may be transmitted by the transmitting STA. At the receiver of the receiving STA, the operations described above for the 80+80 configuration may be reversed and the combined data may be sent to the Medium Access Control (MAC).

[0053] The sub-1 GHz operating modes are supported by 802.11af and 802.11ah. The channel operating bandwidth and carriers are reduced in 802.11af and 802.11ah compared to those used in 802.11n and 802.11ac. 802.11af supports bandwidths of 5 MHz, 10 MHz, and 20 MHz in the TV White Space (TVWS) spectrum, and 802.11ah supports bandwidths of 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz using non-TVWS spectrum. According to an exemplary embodiment, 802.11ah may support meter type control / machine type communication such as MTC devices within a macro coverage area. The MTC device may have limited capabilities, including certain capabilities, e.g., support for a certain and / or limited bandwidth (e.g., support only for these). The MTC device may include a battery having a battery life above a threshold (e.g., to maintain a very long battery life).

[0054] A WLAN system that supports multiple channels and channel bandwidths such as 802.11n, 802.11ac, 802.11af, and 802.11ah includes channels that can be designated as primary channels. The primary channel may have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel can be set and / or restricted by an STA from among all STAs operating in a BSS that supports a minimum bandwidth operation mode. In an example of 802.11ah, the primary channel is 1 MHz wide for an STA (e.g., an MTC type device) that supports the 1 MHz mode (e.g., supports only this) even when the AP and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and / or other channel bandwidth operation modes. Carrier sensing and / or Network Allocation Vector (NAV) setting may depend on the status of the primary channel. For example, due to an STA transmitting to an AP (supporting only the 1 MHz operation mode), if the primary channel is busy, the entire available frequency band may be considered busy even though most of the frequency band remains idle and available.

[0055] In the United States, the available frequency band that can be used by 802.11ah is 902 MHz to 928 MHz. In South Korea, the available frequency band is 917.5 MHz to 923.5 MHz. In Japan, the available frequency band is 916.5 MHz to 927.5 MHz. The total bandwidth available for 802.11ah is 6 MHz to 26 MHz depending on the country code.

[0056] FIG. 1D is a system diagram showing RAN 113 and CN 115 according to one embodiment. As described above, RAN 113 may employ NR radio technology to communicate with WTRUs 102a, 102b, 102c via air interface 116. RAN 113 may also communicate with CN 115.

[0057] RAN 113 may include gNBs 180a, 180b, and 180c, but it will be understood that RAN 113 may include any number of gNBs while maintaining consistency with one embodiment. Each of gNBs 180a, 180b, and 180c may include one or more transceivers for communicating with WTRUs 102a, 102b, and 102c via air interface 116. In one embodiment, gNBs 180a, 180b, and 180c may implement MIMO technology. For example, gNBs 180a and 108b may utilize beamforming to transmit signals to and / or receive signals from gNBs 180a, 180b, and 180c. Thus, gNB 180a may transmit and / or receive radio signals from WTRU 102a using, for example, multiple antennas. In one embodiment, gNBs 180a, 180b, and 180c may implement carrier aggregation technology. For example, gNB 180a may transmit multiple component carriers to WTRU 102a (not shown). A subset of these component carriers may be on unlicensed spectrum, while the remaining component carriers may be on licensed spectrum. In one embodiment, gNBs 180a, 180b, and 180c may implement Coordinated Multi-Point (CoMP) technology. For example, WTRU 102a may receive coordinated transmissions from gNB 180a and gNB 180b (and / or gNB 180c).

[0058] WTRU102a, 102b, and 102c can communicate with gNB180a, 180b, and 180c using transmissions associated with scalable numerology. For example, the OFDM symbol interval and / or the OFDM sub-carrier interval can vary for different transmissions, different cells, and / or different portions of the radio transmission spectrum. WTRU102a, 102b, and 102c can communicate with gNB180a, 180b, and 180c using sub-frames or transmission time intervals (TTIs) of various or scalable lengths (e.g., including various numbers of OFDM symbols and / or having absolute times of various lengths).

[0059] gNBs 180a, 180b, and 180c may be configured to communicate with WTRUs 102a, 102b, and 102c in a stand-alone configuration and / or a non-stand-alone configuration. In a stand-alone configuration, WTRUs 102a, 102b, and 102c may communicate with gNBs 180a, 180b, and 180c without accessing other RANs (e.g., eNodeBs 160a, 160b, and 160c, etc.). In a stand-alone configuration, WTRUs 102a, 102b, and 102c may utilize one or more of gNBs 180a, 180b, and 180c as a mobility anchor point. In a stand-alone configuration, WTRUs 102a, 102b, and 102c may communicate with gNBs 180a, 180b, and 180c using signals in an unlicensed band. In a non-stand-alone configuration, WTRUs 102a, 102b, and 102c may communicate with and connect to gNBs 180a, 180b, and 180c while also communicating with and connecting to another RAN such as eNodeBs 160a, 160b, and 160c. For example, WTRUs 102a, 102b, and 102c may implement a DC principle for communicating with one or more gNBs 180a, 180b, and 180c and one or more eNodeBs 160a, 160b, and 160c substantially simultaneously. In a non-stand-alone configuration, eNodeBs 160a, 160b, and 160c may function as a mobility anchor for WTRUs 102a, 102b, and 102c, and gNBs 180a, 180b, and 180c may provide additional coverage and / or throughput for serving WTRUs 102a, 102b, and 102c.

[0060] Each of gNBs 180a, 180b, and 180c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, support for network slicing, dual connectivity, interworking between NR and E-UTRA, routing of user plane data to user plane functions (UPFs) 184a, 184b, routing of control plane information to access and mobility management functions (AMFs) 182a, 182b, etc. As shown in FIG. 1D, gNBs 180a, 180b, and 180c can communicate with each other via the Xn interface.

[0061] CN 115 shown in FIG. 1D can include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one session management function (SMF) 183a, 183b, and optionally data networks (DNs) 185a, 185b. Although each of the foregoing elements is shown as part of CN 115, it will be understood that any of these elements can be owned and / or operated by entities other than the CN operator.

[0062] AMF 182a and 182b may be connected to one or more of gNBs 180a, 180b, and 180c in RAN 113 via the N2 interface and may function as control nodes. For example, AMF 182a and 182b may be responsible for user authentication of WTRUs 102a, 102b, and 102c, support for network slicing (e.g., handling different PDU sessions with different requirements), selection of specific SMFs 183a and 183b, registration area management, termination of NAS signaling, mobility management, etc. The network slice may be used by AMF 182a and 182b to customize the CN support for WTRUs 102a, 102b, and 102c based on the type of service utilized by WTRUs 102a, 102b, and 102c. For example, different network slices may be established for different use cases such as services that rely on ultra-reliable low latency (URLLC) access, services that rely on enhanced massive mobile broadband (eMBB) access, services for machine type communication (MTC) access, etc. AMF 162 may provide control plane functions for switching between RAN 113 and other RANs (not shown) that employ other radio technologies such as non-3GPP access technologies like LTE, LTE-A, LTE-A Pro, and / or WiFi.

[0063] SMF183a and 183b can be connected to AMF182a and 182b in CN115 via the N11 interface. SMF183a and 183b can also be connected to UPF184a and 184b in CN115 via the N4 interface. SMF183a and 183b can select and control UPF184a and 184b, and configure the routing of traffic passing through UPF184a and 184b. SMF183a and 183b can perform other functions such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notifications. The PDU session type can be IP-based, non-IP-based, Ethernet-based, etc.

[0064] UPF184a and 184b may be connected to one or more of gNB180a, 180b, and 180c in RAN113 via the N3 interface, thereby providing access to a packet-switched network such as the Internet 110 to WTRU102a, 102b, and 102c to facilitate communication between WTRU102a, 102b, and 102c and IP-compatible devices. UPF184 and 184b can perform other functions such as routing and forwarding packets, enforcing user plane policies, supporting multi-home PDU sessions, handling user plane QoS, buffering downlink packets, and providing mobility anchoring.

[0065] CN115 may facilitate communication with other networks. For example, CN115 may include or communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that functions as an interface between CN115 and the PSTN 108. Additionally, CN115 may provide access to other network 112 for the WTRUs 102a, 102b, 102c, and other network 112 may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, the WTRUs 102a, 102b, 102c may be connected to the local data networks (DNs) 185a, 185b through the UPFs 184a, 184b via an N3 interface to the UPFs 184a, 184b and an N6 interface between the UPFs 184a, 184b and the DNs 185a, 185b.

[0066] Referring to FIGS. 1A-1D and the corresponding descriptions thereof, one or more of the functions described herein with respect to one or more of the WTRUs 102a-d, base stations 114a and 114b, eNodeBs 160a-c, MME 162, SGW 164, PGW 166, gNBs 180a-c, AMFs 182a and 182b, UPFs 184a and 184b, SMFs 183a and 183b, DNs 185a and 185b, and / or any other device described herein may be implemented by one or more emulation devices (not shown). An emulation device may be one or more devices configured to emulate one or more of the functions described herein. For example, emulation devices may be used to test other devices and / or simulate network and / or WTRU functionality.

[0067] An emulation device can be designed to implement one or more tests of other devices in a laboratory environment and / or an enterprise network environment. For example, one or more emulation devices can perform one or more or all functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices within the communication network. One or more emulation devices can perform one or more or all functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. An emulation device can be directly coupled to another device for testing purposes and / or can perform tests using over-the-air wireless communication.

[0068] One or more emulation devices can perform one or more functions including all while not being implemented / deployed as part of a wired and / or wireless communication network. For example, an emulation device can be utilized in a test scenario in a test laboratory and / or in a wired and / or wireless communication network that is not deployed (e.g., for testing) to implement tests of one or more components. One or more emulation devices can be test equipment. Direct RF coupling and / or wireless communication via an RF circuit (which can include one or more antennas) can be used by an emulation device to transmit and / or receive data.

[0069] This application describes various aspects including tools, features, examples, models, techniques, etc. Many of these aspects are specifically described and often described in a limiting way to at least show individual characteristics. However, this is for the purpose of clarifying the description and is not intended to limit the application or scope of those aspects. In fact, all of the different aspects can be combined and exchanged to provide further aspects. Moreover, these aspects can similarly be combined and exchanged with aspects described in prior applications.

[0070] The aspects described and contemplated in this application can be implemented in many different forms. The drawings described herein may provide some examples, but other examples are also contemplated. Consideration of the drawings does not limit the scope of the implementation forms. At least one of the aspects generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects can be implemented as a method, an apparatus, a computer-readable medium (storage medium) including instructions (e.g., stored) for encoding or decoding video data according to any of the methods described, and / or a computer-readable storage medium storing a bitstream generated according to any of the methods described. As referred to herein, a bitstream may refer to transmitted data in some cases, but may also refer to data (e.g., non-temporary data) that is stored, generated, and / or accessed without being transmitted.

[0071] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image", "picture", and "frame" may be used interchangeably.

[0072] Various methods are described herein, and each of the methods includes one or more steps or actions for achieving the described method. The order and / or use of specific steps and / or actions may be changed or combined, provided that a particular order of steps or actions is not required for the proper operation of the method. Additionally, terms such as "first", "second", etc. may be used in various embodiments to modify elements, components, steps, actions, etc., such as, for example, "first decoding" and "second decoding". The use of such terms does not imply an ordering of the modified actions unless specifically required. Thus, in this embodiment, the first decoding need not be performed before the second decoding, and may be performed, for example, before, during, or overlapping with the second decoding.

[0073] As shown in FIGS. 2 and 3, the various methods and other aspects described in this application can be used to modify modules of the video encoder 200 and decoder 300, such as the decoding module. Moreover, the subject matter disclosed herein can be applied to any type, format, or version of video coding, as well as any extension of any such standard and recommendation, whether existing or to be developed in the future, regardless of whether described in a standard or recommendation. Unless otherwise indicated or technically impossible, the aspects described in this application can be used individually or in combination.

[0074] In the examples described in this application, various numerical values are used. These and other specific values are for illustrative purposes and the aspects described are not limited to these specific values.

[0075] FIG. 2 is a diagram showing an exemplary video encoder. Although variations of the exemplary encoder 200 are contemplated, the encoder 200 is described below for purposes of clarity without describing all the expected variations.

[0076] The video sequence may pass through the pre-encoding processing 201 before being encoded. For example, a color conversion may be applied to the input color picture (e.g., conversion from RGB4:4:4 to YCbCr4:2:0), or a remapping of the input picture components may be performed to obtain a more resilient signal distribution for compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and attached to the bitstream.

[0077] In encoder 200, as described below, a picture is encoded by encoder elements. The picture to be encoded is divided 202 and processed, for example, in units of coding units (CUs). Each unit is encoded using, for example, either an intra mode or an inter mode. When a unit is encoded in the intra mode, this unit performs an intra prediction 260. In the inter mode, motion estimation 275 and motion compensation 270 are performed. The encoder determines 205 which of the intra mode or the inter mode should be used to encode a unit and indicates the intra / inter decision, for example, by a prediction mode flag. For example, a prediction residual is calculated by subtracting 210 a prediction block from an original image block.

[0078] Next, this prediction residual is transformed 225 and quantized 230. The quantized transform coefficients, along with motion vectors and other syntax elements are entropy coded 245 to output a bitstream. The encoder may skip the transform and apply quantization directly to the untransformed residual signal. The encoder may bypass both the transform and quantization, i.e., the residual is directly coded without applying the transform process or the quantization process.

[0079] The encoder decodes the encoded block to provide a reference for further prediction. The quantized transform coefficients are inverse quantized 240 and inverse transformed 250 to decode the prediction residual. The decoded prediction residual and the prediction block are combined 255 to reconstruct an image block. For example, an in-loop filter 265 is applied to the reconstructed picture to perform deblocking / SAO (sample adaptive offset) filtering to reduce encoding artifacts. The filtered image is stored in a reference picture buffer (280).

[0080] FIG. 3 is a diagram showing an example of a video decoder. In an exemplary decoder 300, a bitstream is decoded by decoder elements as described below. The video decoder 300 generally performs a decoding path that is the reverse of the encoding path described in FIG. 2. Further, the encoder 200 generally performs video decoding as part of the encoding of video data.

[0081] In particular, the input to the decoder includes a video bitstream that can be generated by the video encoder 200. First, the bitstream is entropy decoded 330 to obtain transform coefficients, motion vectors, and other coded information. The picture partitioning information indicates how the picture is partitioned. Thus, the decoder can partition 335 the picture according to the decoded picture partitioning information. The transform coefficients are inverse quantized 340 and inverse transformed 350 to decode the prediction residual. The decoded prediction residual and the predicted block are combined 355 to reconstruct the image block. The predicted block can be obtained 370 from intra prediction 360 or motion compensated prediction (i.e., inter prediction) 375. A loop filter 365 is applied to the reconstructed image. The filtered image is stored in the reference picture buffer 380.

[0082] The decoded picture may further undergo post-decoding processing 385, such as an inverse color conversion (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4), or an inverse remapping that performs the reverse of the remapping process performed in the pre-encoding processing 201. In the post-decoding processing, metadata derived in the pre-encoding processing and signaled in the bitstream may be used. In one embodiment, the decoded image (e.g., after the application of the loop filter 365 and / or after the post-decoding processing 385 if post-decoding processing is used) can be sent to a display device for rendering to the user.

[0083] FIG. 4 is a diagram illustrating an example of a system in which the various aspects and embodiments described herein may be implemented. System 400 may be embodied as a device that includes various components described below and is configured to implement one or more of the aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of system 400 may be embodied alone or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 400 are distributed across multiple ICs and / or discrete components. In various embodiments, system 400 is communicatively coupled to one or more other systems or other electronic devices, for example, via a communication bus or through dedicated input ports and / or output ports. In various embodiments, system 400 is configured to implement one or more of the aspects described herein.

[0084] System 400 includes at least one processor 410 configured to execute instructions loaded therein to implement various aspects described herein, for example. Processor 410 may include built-in memory, an input / output interface, and various other circuits known in the art. System 400 includes at least one memory 420 (e.g., a volatile memory device and / or a non-volatile memory device). System 400 includes a storage device 440, which may include non-volatile memory and / or volatile memory, such as erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash, magnetic disk drive, and / or optical disk drive, but is not limited thereto. Storage device 440 may include, by way of non-limiting example, an internal storage device, an attached storage device (including removable and non-removable storage devices), and / or a network-accessible storage device.

[0085] System 400 includes, for example, an encoder / decoder module 430 configured to process data to provide encoded or decoded video, and the encoder / decoder module 430 may include its own processor and memory. The encoder / decoder module 430 represents a module that may be included within a device to perform encoding and / or decoding functions. As is known, the device may include one or both of an encoding module and a decoding module. Additionally, the encoder / decoder module 430 may be implemented as a separate element of the system 400 or may be incorporated within the processor 410 as a combination of hardware and software known to those skilled in the art.

[0086] The program code loaded into the processor 410 or the encoder / decoder 430 to implement the various aspects described herein may be stored in the storage device 440 and then loaded into the memory 420 for execution by the processor 410. According to various embodiments, one or more of the processor 410, the memory 420, the storage device 440, and the encoder / decoder module 430 may store one or more of the various items during the implementation of the processes described herein. Such stored items may include, but are not limited to, input video, decoded video or a portion of the decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, expressions, operations, and operation logic.

[0087] In some embodiments, the memory internal to the processor 410 and / or the encoder / decoder module 430 is used to store instructions and to provide a working memory for the processing required during encoding or decoding. However, in other embodiments, an external memory of the processing device (e.g., the processing device can be either the processor 410 or the encoder / decoder module 430) is used for one or more of these functions. The external memory can be the memory 420 and / or the storage device 440, such as, for example, dynamic volatile memory and / or non-volatile flash memory. In some embodiments, an external non-volatile flash memory is used, for example, to store the operating system of a television. In at least one embodiment, a high-speed external dynamic volatile memory such as RAM is used as the working memory for video encoding and decoding operations.

[0088] Inputs to the elements of the system 400 can be provided through various input devices as shown by block 445. Such input devices include, but are not limited to, (i) a radio frequency (RF) portion that receives, for example, an RF signal transmitted over the air by a broadcast station, (ii) a component (COMP) input terminal (or a set of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high definition multimedia interface (HDMI) input terminal. Other embodiments not shown in FIG. 4 include composite video.

[0089] In various embodiments, the input device of block 445 has respective input processing elements known in the art. For example, the RF portion may be associated with appropriate elements to (i) select a desired frequency (also referred to as selecting a signal or band-limiting a signal to a frequency band), (ii) down-convert the selected signal, (iii) in certain embodiments, band-limit to a narrower frequency band again to select a signal frequency band, which may be referred to as a channel for example, (iv) demodulate the down-converted and band-limited signal, (v) perform error correction, and / or (vi) demultiplex to select a desired stream of data packets. The RF portion of various embodiments includes one or more elements for performing these functions, such as a frequency selector, signal selector, band limiter, channel selector, filter, down-converter, demodulator, error corrector, and demultiplexer. The RF portion may include a tuner for performing these various functions, including for example down-converting a received signal to a lower frequency (e.g., an intermediate frequency or near baseband frequency) or to baseband. In one embodiment of a set-top box, the RF portion and its associated input processing elements perform frequency selection by receiving, filtering, down-converting, and filtering again to a desired frequency band an RF signal transmitted over a wired (e.g., cable) medium. Various embodiments may reorder the (and other) elements described above, remove some of these elements, and / or add other elements performing similar or different functions. Adding elements may include inserting elements between existing elements, such as inserting an amplifier and an analog-to-digital converter for example. In various embodiments, the RF portion includes an antenna.

[0090] The USB terminal and / or the HDMI terminal may each include an interface processor for connecting the system 400 to other electronic devices via a USB connection and / or an HDMI connection. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented, for example, in a separate input processing IC or within the processor 410 as needed. Similarly, aspects of USB or HDMI interface processing may be implemented, as needed, in a separate interface IC or within the processor 410. Demodulation, error correction, and multiplexed and separated streams are provided to various processing elements, including, for example, the processor 410 and an encoder / decoder 430 that operates in combination with memory and storage elements to process data streams as needed for presentation to an output device.

[0091] The various elements of the system 400 may be provided within an integrated housing. Within the integrated housing, the various elements are interconnected and may transmit data between them using a suitable connection configuration 425, such as an Inter-IC (I2C) bus, wiring, and an internal bus known in the art that includes a printed circuit board.

[0092] The system 400 includes a communication interface 450 that enables communication with other devices via a communication channel 460. The communication interface 450 may include, but is not limited to, a transceiver configured to transmit and receive data via the communication channel 460. The communication interface 450 may include, but is not limited to, a modem or a network card, and the communication channel 460 may be implemented, for example, within a wired medium and / or a wireless medium.

[0093] In various embodiments, data is streamed to system 400 or otherwise provided using a Wi-Fi network, such as a wireless network like IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi-Fi signals in these embodiments are received via a communication channel 460 and a communication interface 450 that are adapted for Wi-Fi communication. The communication channel 460 in these embodiments is typically connected to an access point or router that provides access to an external network, including the Internet, to enable streaming applications and other over-the-top communications. Other embodiments provide streamed data to system 400 using a set-top box that distributes data via an HDMI connection of input block 445. Still other embodiments provide streamed data to system 400 using an RF connection of input block 445. As noted above, various embodiments provide data in a non-streaming fashion. Additionally, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth® networks.

[0094] System 400 can provide output signals to various output devices including display 475, speaker 485, and other peripheral devices 495. The display 475 in various embodiments includes, for example, one or more of a touch screen display, an organic light emitting diode (OLED) display, a curved display, and / or a foldable display. The display 475 can be for a television, a tablet, a laptop, a mobile phone, or other devices. The display 475 may also be integrated with other components (e.g., as in a smartphone), or may be separate (e.g., an external monitor for a laptop). The other peripheral devices 495 include, in various embodiments, one or more of a stand-alone digital video disc (or digital versatile disc) (DVD for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 495 that provide functions based on the output of the system 400. For example, a disc player implements the function of playing back the output of the system 400.

[0095] In various embodiments, the control signal uses signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable control between devices regardless of the presence or absence of user intervention to communicate between system 400 and display 475, speaker 485, or other peripheral device 495. The output devices may be communicatively coupled to system 400 via dedicated connections through respective interfaces 470, 480, and 490. Alternatively, the output devices may be connected to system 400 via communication interface 450 using communication channel 460. Display 475 and speaker 485 may be integrated into a single unit with other components of system 400 within an electronic device such as, for example, a television. In various embodiments, display interface 470 includes a display driver such as, for example, a timing controller (TCon) chip.

[0096] Display 475 and speaker 485 may alternatively be separate from one or more of the other components, for example, if the RF portion of input 445 is part of a separate set-top box. In various embodiments where display 475 and speaker 485 are external components, the output signal may be provided via a dedicated output connection including, for example, an HDMI port, a USB port, or a COMP output.

[0097] These embodiments can be performed by computer software implemented by the processor 410, or by hardware, or by a combination of hardware and software. As a non-limiting example, these embodiments can be implemented by one or more integrated circuits. The memory 420 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology, such as, by way of non-limiting example, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 410 can be of any type suitable for the technical environment and can include, by way of non-limiting example, one or more of a microprocessor, a general-purpose computer, a dedicated computer, and a processor based on a multi-core architecture.

[0098] Various implementations include decoding. As used in this application, "decoding" can include all or part of a process performed on a received encoded sequence, for example, to generate a final output suitable for a display. In various embodiments, such a process typically includes one or more of the processes performed by a decoder, such as entropy decoding, inverse quantization, inverse transformation, and differential decoding. In various embodiments, such a process further or alternatively includes the processes performed by the decoders of the various implementations described in this application, such as determining that a subsampled template matching prediction (TMP) is valid for the current block, determining a search area for performing the TMP, determining a subsampled search area for performing the subsampled TMP based on the search area, and performing the subsampled TMP on the subsampled search area to determine a predicted block for the current block.

[0099] As a further example, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to specifically refer to a subset of operations or to a more extensive decoding process as a whole will become apparent based on the context of the specific description and is considered to be fully understood by those skilled in the art.

[0100] Various implementations include encoding. Similar to the above considerations regarding "decoding", "encoding" as used in this application may include all or a part of the process performed on an input video sequence, for example, to generate an encoded bitstream. In various embodiments, such a process typically includes one or more of the processes performed by an encoder, such as splitting, differential encoding, transformation, quantization, and entropy encoding. In various embodiments, such a process may further or alternatively include the processes performed by the encoders of the various implementations described in this application, for example, determining to enable subsampled template matching prediction (TMP) for the current block, determining a search area for performing TMP, determining a subsampled search area for performing subsampled TMP based on the search area, performing subsampled TMP on the subsampled search area to determine a predicted block for the current block, and encoding the current block based on the predicted block.

[0101] As a further example, in one embodiment, "encoding" refers only to entropy encoding; in another embodiment, "encoding" refers only to differential encoding; and in another embodiment, "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding process" is intended to specifically refer to a subset of operations or to a more general encoding process as a whole will become apparent based on the context of the specific description and is considered to be fully understood by those skilled in the art.

[0102] Note that the syntactic elements as used herein are terms of description. Thus, they do not preclude the use of other syntactic element names.

[0103] When a figure is presented as a flowchart, it should be understood that the figure also provides a block diagram of the corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that the figure also provides a flowchart of the corresponding method / process.

[0104] The implementations and aspects described herein can be implemented, for example, in a method or process, an apparatus, a software program, a data stream, or a signal. Even if discussed only in the context of a single form of implementation (e.g., only discussed as a method), the implementation of the discussed features can also be implemented in other forms (e.g., an apparatus or a program). An apparatus can be implemented, for example, with appropriate hardware, software, and firmware. A method can be implemented, for example, with a processor, which generally refers to a processing device and includes, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes, for example, a communication device such as a computer, a mobile phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate the communication of information between end users.

[0105] References to "one example" or "an example" or "one implementation" or "an implementation", and variations thereof, mean that the particular features, structures, characteristics, etc. described in connection with the example are included in at least one example. Thus, the phrases "in one example" or "in an example" or "in one implementation" or "in an implementation", and any other variations thereof, that appear in various places throughout this application are not necessarily all referring to the same example.

[0106] In addition, this application may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from memory. Obtaining may include receiving, retrieving, constructing, generating, and / or determining.

[0107] Furthermore, this application may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0108] In addition, this application may refer to "receiving" various information. Receiving is intended to be a broad term, similar to "accessing". Receiving information can include, for example, accessing the information or retrieving the information (e.g., from memory), among one or more of these. Further, "receiving" typically involves, in some manner, during operations such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0109] For example, in the cases of "A / B", "A and / or B", and "at least one of A and B", it should be understood that the use of any of the following, " / ", "and / or", and "at least one of", is intended to include the selection of only the first-listed option (A), or only the second-listed option (B), or the selection of both options (A and B). As a further example, in the cases of "A, B, and / or C" and "at least one of A, B, and C", such expressions are intended to include the selection of only the first-listed option (A), or only the second-listed option (B), or only the third-listed option (C), or the selection of only the first and second-listed options (A and B), or the selection of only the first and third-listed options (A and C), or the selection of only the second and third-listed options (B and C), or the selection of all three options (A and B and C). As will be apparent to those of ordinary skill in the relevant art, this can be extended for as many listed items as there are.

[0110] Also, as used herein, the term "signal" specifically means to indicate something to the corresponding decoder. In this way, in one example, the same parameters are used on both the encoder side and the decoder side. Thus, for example, the encoder may send (explicitly signal) specific parameters to the decoder so that the decoder can use the same specific parameters. Conversely, if the decoder already has specific parameters as well as other parameters, signaling may be used without sending (implicit signaling) in order to enable the decoder to easily recognize and select the specific parameters. By avoiding the transmission of any actual functionality, bit savings are achieved in various embodiments. It should be understood that signaling can be achieved in various ways. For example, in various embodiments, one or more syntax elements, flags, etc. are used to signal information to the corresponding decoder. The foregoing relates to the verb form of the word "signal", but the word "signal" can also be used as a noun in this specification.

[0111] As will be apparent to those skilled in the art, implementations can generate various signals formatted to carry information that can be stored or transmitted, for example. The information can include, for example, instructions for implementing a method or data generated by one of the described implementations. For example, a signal can be formatted to carry a bitstream of the described embodiment. Such a signal can be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting can include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal can be, for example, analog information or digital information. The signal can be transmitted by various different wired or wireless links, as is known. The signal can be stored on a processor-readable medium or accessed or received from a processor-readable medium.

[0112] Numerous embodiments are described herein. The features of the embodiments can be provided singly or in any combination across various claim categories and types. Further, an embodiment can include one or more of the features, devices, or aspects described herein, singly or in any combination, across various claim categories and types. For example, the features described herein can be implemented in a bitstream or signal that includes information generated as described herein. This information can enable a decoder to decode a bitstream, encoder, bitstream, and / or decoder according to any of the described embodiments. For example, the features described herein can be implemented by generating, transmitting, receiving, and / or decoding a bitstream or signal. For example, the features described herein can be implemented as a method, process, apparatus, medium storing instructions (e.g., computer-readable medium), medium storing data, or signal. For example, the features described herein can be implemented by a TV, set-top box, mobile phone, tablet, or other electronic device that performs decoding. A TV, set-top box, mobile phone, tablet, or other electronic device can display a resulting image (e.g., an image from residual reconstruction of a video bitstream) (e.g., using a monitor, screen, or other type of display). A TV, set-top box, mobile phone, tablet, or other electronic device can receive a signal including an encoded image and perform decoding.

[0113] Template matching can include intra-template matching (e.g., intra-template matching prediction) and inter-template matching (e.g., inter-template matching prediction). Embodiments of the disclosure can be described by way of example using intra-template matching prediction (IntraTMP), but one of ordinary skill in the art will understand that at least some of the disclosed techniques can also be applied to inter-template matching prediction.

[0114] IntraTMP can be used to search for predicted blocks (e.g., within the reconstructed area) where the template can match the template of the current block. Then, the matching block can be used for prediction. IntraTMP can be activated for some content, such as screen content (e.g., Class F content and / or text and graphics with motion (TGM) content), to achieve a good compromise between coding gain and coding time. IntraTMP can be deactivated for some content, such as natural sequences, as these content may be associated with longer coding times. The IntraTMP operation can be accelerated (e.g., for an encoder and / or decoder) based on the content being processed (e.g., the IntraTMP acceleration may be content-specific).

[0115] IntraTMP (which can be regarded as an intra prediction mode or technique, for example) may involve copying a predicted block (e.g., the best predicted block) from the reconstructed part of the current frame, where the template of the reconstructed part of the current frame (e.g., an L-shaped template) can match the current template (e.g., the template of the current block). For example, for a predefined search range, the encoder may search for a template in the reconstructed part of the current frame that is similar (e.g., most similar) to the current template. The encoder may use the corresponding block (e.g., the block associated with the matching template) as the predicted block. The encoder may signal (e.g., in the bitstream) the use of IntraTMP so that the same (e.g., similar) prediction operation can be performed on the decoder side (e.g., in response to receiving signaling).

[0116] FIG. 5 shows an example of generating a prediction signal by matching the L-shaped neighborhood (e.g., accidental neighborhood) of the current block with another block within a predefined search area. As shown, the predefined search area may include R1 (e.g., which may be the current coding tree unit (CTU)), R2 (e.g., which may be the upper left CTU), R3 (e.g., which may be the upper CTU), and R4 (e.g., which may be the left CTU). In an embodiment, the sum of absolute difference (SAD) may be used as the cost function for the search. Within the area (e.g., each area), the decoder (or encoder) may search for the template having the minimum SAD with respect to the current template. The decoder (or encoder) may use the corresponding block as the prediction block.

[0117] One or more dimensions of the search area (e.g., represented by (SearchRange_w, SearchRange_h)) may be set to be proportional to, as shown by the following equation (e.g., represented by the block dimensions (BlkW, BlkH)), such that there may be a fixed number of SAD comparisons per pixel: SearchRange_w = a * BlkW SearchRange_h = a * BlkH Here, "a" may be a constant that controls the trade-off between coding gain and coding complexity (e.g., "a" may be 5).

[0118] In an embodiment, IntraTMP may be enabled for a coding unit (CU) having a specific size. The size of the CU for which IntraTMP is enabled may be configurable (e.g., the width and / or height are 64 or less).

[0119] The complexity of IntraTMP may be related to one or more factors (e.g., configurable factors) such as, for example, template size, search range, block size, picture type, etc. Regarding the template size and / or search range, the smaller the template, the faster the coding can be (e.g., because fewer comparisons may be performed). Regarding the block size, if IntraTMP is allowed for a subset of the block sizes, the coding can be faster (e.g., because fewer rate distortion (RD) searches may be performed). Regarding the picture type, if IntraTMP is allowed only for a specific slice type (e.g., only for I slices), the coding can also be faster.

[0120] In an embodiment, one or more of the factors (or parameters) described herein may not be set for a particular content type. (For example, for each) content type, syntax elements (e.g., high-level syntax elements) may be used (e.g., signaled) to control IntraTMP so that a suitable (e.g., best) setting can be set. The search for matching templates can be reduced.

[0121] The template size associated with IntraTMP may be fixed (e.g., 4 pixels above and / or to the left of the current block). Such a template size may be reduced, for example, to reduce the coding time. For example, a higher-level parameter or flag may be signaled (e.g., in video data) to indicate the size of the template. The parameter or flag may be signaled at the sequence parameter set (SPS) level and / or gated (e.g., conditioned) by an SPS parameter or flag associated with the activation of IntraTMP.

[0122] Signaling (e.g., including various parameters or flags) may be described herein as being performed at the SPS level, but the signaling may be performed at the picture parameter set (PPS) level without affecting the applicability and / or scope of the present disclosure.

[0123] In some examples (e.g., in the case of simplified coding), the template size may be reduced to 1 pixel. The reduction of the template size may lead to a reduction in the number of comparisons by 1 / 3. In some examples (e.g., in the case of normal coding), the template size may be maintained at 4 pixels.

[0124] Table 1 below shows an exemplary syntax associated with IntraTMP. The syntax associated with IntraTMP may include a template prediction template size indication (e.g., sps_tmp_four_minus_template_size) that may indicate the template size associated with IntraTMP. For example, the variable may be given an integer value from 0 to 3.

[0125] [Table 1]

[0126] The search range associated with IntraTMP can be controlled by a search range doubling factor. The value of the search range doubling factor may be signaled at the SPS level and / or gated by an IntraTMP SPS flag (e.g., to enable / disable IntraTMP). A smaller value for the search range doubling factor may be considered (e.g., to simplify coding). Table 2 below shows an exemplary syntax that may include a search range doubling factor indication (e.g., sps_tmp_search_range_factor). The search range doubling factor indication may indicate the search range doubling factor. The search range doubling factor may be used to determine the dimensions (e.g., range) of the search area for the current block (e.g., the current frame). The search range may be proportional to the block size. The value range of the search range doubling factor parameter may be, for example, from 1 to 5.

[0127]

Table 2

[0128] SPS parameters or indications (e.g., two SPS flags) may be used (e.g., signaled) to indicate the minimum block size and the maximum block size of IntraTMP. For non-screen content, IntraTMP may be disabled for large blocks. For screen content where the pattern can be repeated, IntraTMP may be applied to large blocks.

[0129] Since small blocks can consume energy for template matching (e.g., a higher search range may be allowed for small blocks compared to large blocks), the maximum block size and / or the minimum block size may be signaled at the SPS level. The maximum and / or minimum block size may be gated by an SPS activation flag for IntraTMP.

[0130] Table 3 below shows an exemplary syntax that may include parameters configured to indicate a minimum block size and a maximum block size (e.g., sps_log2_min_tmp_block_size_diff and sps_log2_max_tmp_block_size_diff, respectively). For example, the minimum TMP block size may be Min_size = 1<<(log2_min_block_size + sps_log2_min_tmp_block_size_diff), where sps_log2_min_tmp_block_size_diff is set in the range of 0 to N, and N = log2(maximum block size) - 2. The maximum TMP block size may be Max_size = 1<<(log2_max_block_size - sps_log2_max_tmp_block_size_diff), where sps_log2_max_tmp_block_size_diff is set in the range of 0 to N, and N = log2(maximum block size) - log2(Min_size).

[0131] [Table 3]

[0132] As described herein, IntraTMP search may be performed in different regions (e.g., inside the current CTU, above the current CTU, to the left of the current CTU, etc.). In some examples (e.g., in the case of simplified coding), a portion of these regions (e.g., a subset of the search region) may be considered for template matching (e.g., only considering the area inside the current CTU). For example, a match may be found in a nearby area in the case of a natural sequence.

[0133] SPS parameters or flags can be signaled to indicate which one or more regions can be used for template matching. The SPS parameters or flags can be signaled to indicate that the area inside the current CTU can be used for template matching. In some examples, based on the limited availability of the search area, IntraTMP can be presumed to be invalid (e.g., it can be presumed that the IntraTMP enable flag has a value of 0). This can accommodate situations where the first coding block is within the CTU and the decoded CTU area is smaller than the current block.

[0134] Table 4 shows an exemplary syntax that may include a template match prediction search area indication (e.g., sps_tmp_search_area) for indicating the search area. For example, 0 may indicate to search the current CTU + above + left, 1 may indicate to search the current CTU, 2 may indicate to search the current CTU + above, and / or 3 may indicate to search the current CTU + left.

[0135] [Table 4]

[0136] SPS parameters or flags can be used to indicate whether IntraTMP is enabled for a particular slice type, e.g., whether IntraTMP is enabled for non-I slices. Such parameters or flags can be used when IntraTMP provides a coding gain for intracoding (e.g., a greater gain for slices intracoded than for non-I slices). For example, in a coding (e.g., decoding) pipeline, inter blocks may first be coded (e.g., decoded), and then intra blocks may be coded (e.g., decoded). Template matching may be allowed for inter slices (e.g., by using only the inter block area during search, e.g., only for inter slices). In these examples, when searching for the motion vector of the current lock in the search area, if the template or the block itself overlaps with an intracoded block, the motion vector may be considered invalid.

[0137] The subsampled TMP (e.g., IntraTMP) can be enabled for the current block. A search area for performing TMP can be determined. The subsampled search area for performing the subsampled TMP can be determined based on the search area. For example, the search area for performing TMP (e.g., the search range associated with IntraTMP) can be subsampled by a coefficient "a". The subsampled search area can be determined based on subsampling the search area by a subsampling coefficient. For example, when "a" is 2, every other pixel within the search range can be skipped when searching for a match (e.g., the best match). This can reduce the number of comparisons to one quarter of the number of comparisons. The value of "a" can be increased to further reduce the complexity of the search. The subsampling coefficient "a" can have a value range of 1 to N. For example, N can be equal to 2, 4, or 8. When the value of the subsampling coefficient is N, the subsampled search area can include every Nth pixel of the search area (e.g., every other pixel when N = 2, every fourth pixel when N = 4, etc.). The subsampled TMP can be performed on the subsampled search area to determine a candidate prediction block for the current block. The current block can be encoded based on the candidate prediction block. The template of the current block can be similarly subsampled based on the subsampling coefficient.

[0138] In an embodiment, the sub-sampled TMP search can be refined. The refined TMP can be performed after the sub-sampling search. For example, the template matching search may be performed at a coarse scale (e.g., due to sub-sampling), and a further search (e.g., full pixel search) may be performed in a refined search area (e.g., a small area) around the best match found during the sub-sampling search. For example, the refined search area can be identified based on a candidate prediction block for the current block identified based on the initial sub-sampled TMP search. The refined search area may be smaller than the search area used in the initial search. The refined TMP can be performed using a sub-sampling factor (e.g., a refined sub-sampling factor) proportional to the block size (e.g., the smaller the block, the smaller the sub-sampling factor can be). The refined sub-sampling factor may be used to determine a refined search area sub-sampled from the refined search area as described herein. This can reduce the coding time while still generating an accurate estimate of the best match. The refined TMP may be used to determine a refined prediction block for the current block. The current block can be decoded and / or encoded based on the refined prediction block.

[0139] The sub-sampling factor can be predefined (e.g., pre-set). The sub-sampling factor can be determined based on a TMP sub-sampling indication in the video data. For example, the TMP sub-sampling indication (e.g., sps_tmp_subsampling_factor_minus1) can be an SPS level parameter set to indicate the value of the sub-sampling factor.

[0140] Whether to perform refined TMP can be determined based on a TMP subsampling refinement indication (e.g., sps_tmp_subsampling_refinement) in the video data. The TMP subsampling refinement indication can be an SPS-level parameter or flag configured to indicate the performance of the refinement (e.g., whether refined TMP is effective), and / or the subsampling factor used for the refinement (e.g., the refined subsampling factor can be indicated by a refined subsampling factor indication). Table 5 shows an exemplary syntax that may include one or more of these parameters or flags. In some embodiments, the indication can be signaled at the coding unit level to enable or disable TMP subsampling refinement for the current block.

[0141] [[ID=…]]

Table 5

[0142] The subsampling factor for refinement (e.g., the refined subsampling factor) can be predefined (e.g., preset) in the decoder and / or encoder. In some examples, the subsampling factor for refinement can be determined based on a TMP refinement indication in the video data. For example, the TMP refinement indication (e.g., sps_tmp_subsampling_refinement_factor_minus1) can be an SPS-level parameter or flag set to indicate the value of the refined subsampling factor to be used for the refinement (e.g., the refined subsampling factor can be indicated by a refined subsampling factor indication). If the indicated value is greater than 1, the decoder can determine (e.g., infer) to perform the refinement. Table 6 shows an exemplary syntax that may include this parameter or flag.

[0143]

Table 6

[0144] The decoder can determine whether to perform refined TMP based on the TMP subsampling refinement indication (e.g., sps_tmp_subsampling_refinement_flag) in the video data. The TMP subsampling refinement indication may be an SPS level parameter or a flag set to indicate the performance of the refinement (e.g., whether the refined TMP is enabled for the current block). For example, when the flag indicates a value of 1, the decoder may perform the refinement. Table 7 shows an exemplary syntax that may include this parameter or flag.

[0145] [Table 7]

[0146] Subsampling and refinement operations may be performed hierarchically. The search range (e.g., refinement range) can be reduced as the subsampling factor decreases during hierarchical search for a prediction block. For example, the search (e.g., initial search) can start with a subsampling factor of N (e.g., N = 8). The search can be performed over a subsampled search area (e.g., determined based on the subsampling factor) to determine a candidate decision block for the current block. The refinement search may be performed with a refinement subsampling factor of N / 2 (e.g., the refinement subsampling factor may be smaller than the subsampling factor). The refinement search can be performed over a refinement search area (e.g., determined at least in part based on the refinement subsampling factor). The refinement search area may be an area smaller than the initial search area. The refinement search area can be an area around the candidate prediction block determined during the initial search. When the refinement subsampling factor is N / 2, the subsampled refinement search area may include every N / 2th pixel of the refinement search area. The process can be repeated, for example, until the value of the subsampling factor reaches 1 (e.g., N / N). For example, the subsampling factor for hierarchical search can be 8 for the initial search, 4 for the first refinement search, 2 for the second refinement search, and 1 for the third refinement search. For example, the subsampling factor for hierarchical search can be 8 for the initial search, 4 for the first refinement search, and 1 for the second refinement search. This technique can refine the search at multiple scales to reduce the coding time (e.g., encoding time and / or decoding time).

[0147] The TMP configuration indication can be used to indicate whether to perform a simplified IntraTMP and / or how to perform it. One or more of the techniques described herein (e.g., combinations) can be applied in these examples. For example, a fast (e.g., simplified) version of TMP can be used, and a fast intra-template matching prediction indication (e.g., sps_fast_intra_tmp_flag) can be used to indicate this. In an example, such a flag can be set to 1 to indicate that all or some of the following are true. The template size is reduced, the search range is reduced, the minimum and / or maximum block sizes are set, IntraTMP is restricted within the CTU region, IntraTMP is applied to the I slice, IntraTMP search range subsampling is used with refinement, hierarchical search is applied, etc. The fast intra-template matching prediction indication can be signaled at the SPS level, at the picture level, and / or at the coding unit level.

[0148] FIG. 6 shows exemplary operations that may be performed by a video decoding device according to one or more embodiments of the present disclosure. As shown, the video decoding device may receive video data at 602 and determine at 604 that IntraTMP is valid (e.g., based on parameters or flags included in the video data). The video decoding device may obtain at 606 one or more setting parameters associated with IntraTMP (e.g., from the video data), where the one or more parameters may include at least one of a template size, a search range factor, a minimum block size, a maximum block size, a search area, a slice type to which IntraTMP is applied, a subsampling factor, an indication of whether to refine template matching, or an indication of whether to perform hierarchical search for template matching. The video decoding device may perform IntraTMP on the video data at 608 based on at least one or more parameters (e.g., the video decoding device may perform hierarchical search for matching blocks and may refine the search based on a subsampling factor proportional to the block size).

[0149] FIG. 7 shows exemplary operations that may be performed by a video encoding device according to one or more embodiments of the present disclosure. As shown, at 702, the video encoding device may determine that IntraTMP is valid (e.g., based on the video content being encoded). At 704, the video encoding device may determine one or more configuration parameters associated with IntraTMP. The one or more parameters may include at least one of a template size (e.g., the template size associated with the TMP), a search range factor (e.g., the search range factor associated with the TMP), a minimum block size (e.g., the minimum block size to which the TMP may be applied), a maximum block size (e.g., the maximum block size to which the TMP may be applied), a search area (e.g., the search area associated with the TMP), the slice type to which the subsampled TMP (e.g., IntraTMP) is applied, a subsampling factor, an indication of whether to refine the template matching (e.g., whether to refine the subsampled TMP), or an indication of whether to perform a hierarchical template matching search. At 706, the video encoding device may indicate the one or more configuration parameters to a receiving device (e.g., a video decoding device). Performing the subsampled TMP may be based on at least one of the configuration parameters.

[0150] FIG. 8 shows exemplary operations that may be performed by a video decoding device in accordance with one or more embodiments of the present disclosure. As illustrated, the video decoding device may determine at 802 that subsampled template matching prediction (TMP) is valid for the current block. The video decoding device may determine at 804 a search area for performing the TMP. At 806, the video decoding device may determine a subsampled search area for performing the subsampled TMP based on the search area. The video decoding device may perform the subsampled TMP on the subsampled search area at 808 to determine a candidate prediction block for the current block. At 810, the video decoding device may decode the current block based on the candidate prediction block.

[0151] FIG. 9 shows exemplary operations that may be performed by a video encoding device in accordance with one or more embodiments of the present disclosure. As illustrated, the video encoding device may determine at 902 to enable subsampled TMP for the current block. The video encoding device may determine at 904 a search area for performing the TMP. At 906, the video encoding device may determine a subsampled search area for performing the subsampled TMP based on the search area. The video encoding device may perform the subsampled TMP on the subsampled search area at 908 to determine a candidate prediction block for the current block. At 910, the video encoding device may encode the current block based on the candidate prediction block.

[0152] FIG. 10 is a block diagram showing a refined TMP search. As shown, template matching search (TM search or TMP search) can be performed based on a subsampled search range. A candidate prediction block (e.g., best match) can be obtained from the template matching search. If the number of iterations of the search performed is less than N (e.g., subsampling factor), a refined search range can be used to perform the template matching search. A refined prediction block (e.g., new best match) can be obtained from the template matching search using the refined search range.

[0153] The examples provided herein may assume that media content is streamed to a display device, but there are no specific restrictions on the type of display device that can benefit from the exemplary techniques described herein. For example, the display device may be a television, projector, mobile phone, tablet, etc. Further, the exemplary techniques described herein can be applied not only to streaming use cases but also to remote conferencing settings. In addition, the decoder and display described herein may be separate devices or may be part of the same device. For example, a set-top box may decode an incoming video stream and provide the decoded stream to a display device (e.g., via HDMI) (e.g., thereafter), and information regarding viewing conditions such as viewing distance may be transmitted from the display device to the set-top box (e.g., via HDMI).

[0154] The features and elements are described above in specific combinations, but one of ordinary skill in the art will understand that each feature or element can be used alone or in any combination with other features and elements. Additionally, the methods described herein can be implemented in a computer program, software, or firmware incorporated into a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, magnetic media such as read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, internal hard disks, and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs). A processor associated with the software can be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. A method for decrypting video data, comprising: determining that subsampled template matching prediction (TMP) is valid for a current block; determining a search area for performing TMP; determining a subsampled search area for performing the subsampled TMP based on the search area; performing the subsampled TMP on the subsampled search area to determine a candidate prediction block for the current block; decrypting the current block based on the candidate prediction block; A method comprising the above steps.

2. The method further comprises: identifying a refined search area smaller than the search area around the candidate prediction block for performing refined TMP; performing the refined TMP on the refined search area to determine a refined prediction block for the current block; wherein decrypting the current block based on the candidate prediction block includes decrypting the current block based on the refined prediction block. The method according to claim 1. The method according to claim 1.

3. Determining the subsampled search area includes determining the subsampled search area based on a subsampling factor having a value of N, and the subsampled search area includes every Nth pixel of the search area. The method according to claim 1 or 2.

4. Determining the subsampled search area includes determining the subsampled search area based on a subsampling factor, and the method further comprises: identifying a refined search area for performing refined TMP based on the candidate prediction block for the current block; determining a refined subsampling factor associated with the refined TMP that is smaller than the subsampling factor; determining a subsampled refined search area from the refined search area based on the refined subsampling factor; performing the refined TMP on the subsampled refined search area to determine a refined prediction block for the current block; further comprising decrypting the current block based on the candidate prediction block, which includes decrypting the current block based on the refined prediction block. The method according to claim 1 or 3. **Claim 5** Determining that the subsampled TMP is valid for the current block, Receiving a subsampling instruction configured to indicate whether the subsampled TMP is valid for the current block, Based on the subsampling instruction, determining that the subsampled TMP is valid for the current block, The method according to any one of claims 1 to 4, comprising: **Claim 6** The method further comprising receiving an indication of a subsampling coefficient, wherein determining the subsampled search area includes determining the subsampled search area based on the subsampling coefficient. The method according to any one of claims 1 to 5. **Claim 7** The method Receiving a refined TMP instruction configured to indicate whether the refined TMP is valid for the current block, Based on the refined TMP instruction indicating that the refined TMP is valid for the current block, identifying a refined search area smaller than the search area around the candidate prediction block, Performing the refined TMP on the refined search area to determine a refined prediction block for the current block, further comprising decrypting the current block based on the candidate prediction block, which includes decrypting the current block based on the refined prediction block. The method according to any one of claims 1 to 3 or 5 or 6. **Claim 8** The method Receiving a refined TMP instruction configured to indicate whether the refined TMP is valid for the current block, Receiving a refined subsampling coefficient instruction, Based on the refined TMP instruction indicating that the refined TMP is valid for the current block, identifying a refined search area smaller than the search area around the candidate prediction block, Determining a subsampled refined search area from the refined search area based on the refined subsampling coefficient instruction. Performing the refined TMP on the subsampled refined search area to determine a refined prediction block for the current block; Further including decoding the current block based on the candidate prediction block, wherein decoding the current block based on the candidate prediction block includes decoding the current block based on the refined prediction block; The method according to any one of claims 1 or 3 to 6.

9. Determining the subsampled search area includes determining the subsampled search area based on a subsampling factor N, the subsampled search area including every Nth pixel of the search area, and the method includes: Identifying a refined search area for performing the refined TMP based on the candidate prediction block for the current block; Determining a subsampled refined search area including every N / 2th pixel of the refined search area from the refined search area based on a refined subsampling factor of N / 2; Performing the refined TMP on the subsampled refined search area to determine a refined prediction block for the current block; Further including decoding the current block based on the candidate prediction block, wherein decoding the current block based on the candidate prediction block includes decoding the current block based on the refined prediction block; The method according to any one of claims 1 or 3 to 6 or 8.

10. Performing the subsampled TMP on the subsampled search area is A set parameter, The template size associated with the TMP, The search range factor associated with the TMP, The minimum block size to which the TMP can be applied, The maximum block size to which the TMP can be applied, The search area associated with the TMP, The slice type to which the subsampled TMP is applied, A subsampling factor, wherein determining the subsampled search area includes determining the subsampled search area based on the subsampling factor, A first indication of whether to refine the subsampled TMP, or A second indication of whether to perform a hierarchical template matching search. Receiving one or more of the set parameters; Performing the subsampled TMP on the subsampled search area based at least on the set parameters; The method according to any one of claims 1 to 9, comprising:

11. The method according to any one of claims 1 to 10, wherein the subsampled TMP is associated with intra-template matching prediction (IntraTMP).

12. A method for video encoding, comprising: Determining to enable subsampled template matching prediction (TMP) for a current block; Determining a search area for performing TMP; Determining a subsampled search area for performing the subsampled TMP based on the search area; Performing the subsampled TMP on the subsampled search area to determine a candidate prediction block for the current block; Encoding the current block based on the candidate prediction block; A method comprising:

13. The method is Determining a subsampling coefficient, wherein determining the subsampled search area for performing the subsampled TMP includes determining the subsampled search area for performing the subsampled TMP based on the subsampling coefficient; Including an indication of the subsampling coefficient in the video data; The method according to claim 12, further comprising:

14. The method is Determining a refined search area smaller than the search area for performing refined TMP around the candidate prediction block; Performing the refined TMP on the refined search area to determine a refined prediction block for the current block; Further comprising, encoding the current block based on the candidate prediction block includes encoding the current block based on the refined prediction block; The method according to claim 12 or 13.

15. The method is Further including determining a subsampling coefficient having a value of N, determining the subsampled search area for performing the subsampled TMP, determining the subsampled search area for performing the subsampled TMP based on the subsampling coefficient, the subsampled search area including every Nth pixel of the search area, the method according to any one of claims 12 to 14.

16. Determining the subsampled search area includes determining the subsampled search area based on a subsampling coefficient, the method including identifying a refined search area for performing a refined TMP based on the candidate prediction block for the current block; determining a refined subsampling coefficient associated with the refined TMP that is smaller than the subsampling coefficient; determining a subsampled refined search area from the refined search area based on the refined subsampling coefficient; performing the refined TMP on the subsampled refined search area to determine a refined prediction block for the current block; Further including, encoding the current block based on the candidate prediction block includes encoding the current block based on the refined prediction block, The method according to any one of claims 12, 13, or 15.

17. The method according to any one of claims 12 to 16, wherein the subsampled TMP includes Intra Template Matching Prediction (IntraTMP).

18. A video decoding device comprising a processor, wherein the processor is configured to implement the steps of the method according to any one of claims 1 to 11.

19. A video encoding device comprising a processor, wherein the processor is configured to implement the steps of the method according to any one of claims 12 to 17.

20. A computer program product stored on a non-transitory computer-readable medium, the computer program product including program code instructions for implementing the steps of the method according to any one of claims 1 to 17 when executed by a processor.

21. Video data including information representing the encoded current block generated according to one of the methods according to any one of claims 12 to 17.

Images