Decoder-side motion vector refinement and sub-block motion derivation for merge mode

Abstract

The invention discloses decoder-side motion vector refinement and sub-block motion derivation for merge mode. Systems, methods, and instrumentalities are disclosed for sub-block motion derivation and motion vector refinement for merge mode. Video data may be coded (e.g., encoded and / or decoded). A co-located picture of a current slice of the video data may be identified. The current slice may include one or more coding units (CUs). One or more neighboring CUs may be identified for a current CU. Adjacent CUs (e.g., each adjacent CU) may correspond to a reference picture. For example, a neighboring CU (e.g., one) may be selected as a candidate neighboring CU based on the reference picture and the co-located picture. An MV (e.g., a co-located MV) may be identified from the co-located picture based on a motion vector (MV) (e.g., a reference MV) of the candidate neighboring CUs. The current CU may be coded (e.g., encoded and / or decoded) using the co-located MV.

Description

This application is a divisional application of Chinese patent application No. 201880080912.0, filed on November 1, 2018, entitled “Decoder-side motion vector refinement and sub-block motion derivation for merge mode”, the contents of which are incorporated herein by reference. CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Application Serial No. 62 / 580,184 filed on November 1, 2017, U.S. Provisional Application Serial No. 62 / 623,001 filed on January 29, 2018, U.S. Provisional Application Serial No. 62 / 678,576 filed on May 31, 2018, and U.S. Provisional Application Serial No. 62 / 690,661 filed on June 27, 2018, the contents of which are incorporated herein by reference. Background Art

[0002] Video coding systems can be used to compress digital video signals, for example, to reduce the storage and / or transmission bandwidth required for these signals. Video coding systems can include block-based, wavelet-based, and / or object-based systems. A block-based hybrid video coding system can be deployed. Summary of the invention

[0003] Systems, methods, and means for sub-block motion derivation and motion vector refinement for merge mode may be disclosed herein. Video data may be coded (e.g., encoded and / or decoded). A collocated picture of a current slice of the video data may be identified. The collocated picture may be identified based on, for example, a collocated picture indication in a slice header. The current slice may include one or more coding units CUs. One or more neighboring CUs may be identified for the current CU. Neighboring CUs (e.g., each neighboring CU) may correspond to a reference picture. (e.g., one) neighboring CU may be selected as a candidate neighboring CU based on a reference picture of the neighboring CU and the collocated picture. An MV (e.g., a collocated MV) may be identified from the collocated picture based on a motion vector (MV) (e.g., a reference MV) of the candidate neighboring CU. The collocated MV may be a temporal MV, and the reference MV may be a spatial MV. The current CU may be coded (e.g., encoded and / or decoded) using the collocated MV.

[0004] A neighboring CU may be selected as the candidate neighboring CU based on a corresponding temporal difference between the reference picture of the neighboring CU and the co-located picture. For example, a reference picture (e.g., each reference picture) may be associated with a picture order count (POC), and the neighboring CU having the lowest POC difference with the co-located picture may be selected. The selected neighboring CU may have the same reference picture as the co-located picture. The neighboring CU having the same reference picture as the co-located picture may be selected without further consideration of other neighboring CUs.

[0005] For example, if the reference picture of the candidate neighboring CU is different from the co-located picture, then the reference MV may be temporally scaled. For example, the reference MV may be multiplied by a scaling factor. The scaling factor may be based on the temporal difference between the reference picture of the candidate neighboring CU and the co-located picture.

[0006] The co-located picture may include one or more co-located blocks. One or more of the co-located blocks may be valid co-located blocks. The valid co-located blocks may be continuous and may form a valid co-located block region. For example, the region may be identified based on the current slice. The co-located MV may be associated with a first co-located block that may be valid or invalid. If the first co-located block is invalid, a valid second co-located block may be selected. The co-located MV from the first co-located block may be replaced with a second co-located MV associated with the second co-located block. The second co-located MV may be used to decode (e.g., encode and / or decode) the current CU. The second co-located block may be selected, for example, based on the second co-located block having the lowest distance from the first co-located block. For example, the second co-located block may be a valid block closest to the first co-located block.

[0007] The current CU may be subdivided into one or more sub-blocks. A sub-block (e.g., each sub-block) may correspond to a reference MV. A co-located MV of a sub-block (e.g., each sub-block) may be identified from the co-located picture based on the reference MV of the sub-block (e.g., each sub-block). The size of the sub-block may be determined based on the temporal layer of the current CU. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1A is a system diagram illustrating an exemplary communication system;

[0009] Figure 1B It shows that it can be Figure 1A A system diagram of an exemplary wireless transmit / receive unit (WTRU) used within a communication system is shown;

[0010] Figure 1C It shows that it can be Figure 1AA system diagram of an exemplary radio access network (RAN) and an exemplary core network (CN) used within the communication system shown; and

[0011] Figure 1D It shows that it can be Figure 1A A system diagram of another exemplary RAN and another exemplary CN used within the communication system shown;

[0012] Figure 2 An example schematic diagram of a block-based video encoder is shown.

[0013] Figure 3 An example block diagram of a video decoder is shown.

[0014] Figure 4 Example spatial merge candidates are shown.

[0015] Figure 5 An example of advanced temporal motion vector prediction is shown.

[0016] Figure 6 An example of spatio-temporal motion vector prediction is shown.

[0017] Figure 7 An example decoder-side motion vector refinement (DMVR) for regular merge mode is shown.

[0018] Figure 8 An example refresh of a picture is shown where the Advanced Temporal Motion Vector Prediction (ATMVP) / Spatio-Temporal Motion Vector Prediction (STMVP) block size statistics may be reset to zero when the Advanced Temporal Motion Vector Prediction (ATMVP) / (STMVP) derivation granularity is adaptively determined.

[0019] Fig. 9 A flow chart of an example motion compensation for merge mode when DMVR early termination is applied is shown.

[0020] Fig.10 An example bi-prediction is shown, where the average of two prediction signals is calculated at an intermediate bit depth.

[0021] Fig.11 An example of co-located block derivation for ATMVP is shown.

[0022] Fig.12 Unconstrained access to co-located blocks of ATMVP is shown.

[0023] Fig.13 The constraint regions used to derive the co-located blocks of the ATMVP decoding unit are shown.

[0024] Fig.14An example of co-located block derivation for ATMVP is shown.

[0025] Fig.15 An example of using a co-located picture to derive the MV of the current block is shown. DETAILED DESCRIPTION

[0026] A more detailed understanding may be obtained from the following description given by way of example in conjunction with the accompanying drawings, in which:

[0027] Figure 1A 1 is a schematic diagram showing an exemplary communication system 100 in which one or more disclosed examples may be implemented. The communication system 100 may be a multiple access system that provides content such as voice, data, video, messaging, broadcast, etc. to multiple wireless users. The communication system 100 may enable multiple wireless users to access such content by sharing system resources including wireless bandwidth. For example, the communication system 100 may use one or more channel access methods such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single carrier FDMA (SC-FDMA), zero tail unique word DFT-spread OFDM (ZT UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block filtered OFDM, and filter bank multi-carrier (FBMC), etc.

[0028] like Figure 1AAs shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, public switched telephone network (PSTN) 108, Internet 110, and other networks 112, although it should be appreciated that the disclosed examples contemplate any number of WTRUs, base stations, networks, and / or network components. Each of the WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. By way of example, any of the WTRUs 102a, 102b, 102c, 102d may be referred to as a “station” and / or “STA”, which may be configured to transmit and / or receive wireless signals, and may include a user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a subscription-based unit, a pager, a cellular phone, a personal digital assistant (PDA), a smart phone, a laptop, a netbook, a personal computer, a wireless sensor, a hotspot or Mi-Fi device, an Internet of Things (IoT) device, a watch or other wearable device, a head-mounted display (HMD), a vehicle, a drone, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices working in an industrial and / or automated process chain environment), consumer electronic devices, and devices working on a commercial and / or industrial wireless network, etc. Any of the WTRUs 102a, 102b, 102c, 102d may be interchangeably referred to as a UE.

[0029] The communication system 100 may also include a base station 114a and / or a base station 114b. Each of the base stations 114a, 114b may be any type of device configured to facilitate access to one or more communication networks (e.g., CN 106 / 115, Internet 110, and / or other networks 112) by wirelessly interfacing with at least one of the WTRUs 102a, 102b, 102c, 102d. For example, the base stations 114a, 114b may be a base transceiver station (BTS), a Node B, an eNode B, a Home Node B, a Home eNode B, a gNB, a New Radio (NR) Node B, a site controller, an access point (AP), a wireless router, and the like. Although each of the base stations 114a, 114b is depicted as a single component, it should be appreciated that the base stations 114a, 114b may include any number of interconnected base stations and / or network components.

[0030] The base station 114a may be part of the RAN 104 / 113, and the RAN may also include other base stations and / or network components (not shown), such as a base station controller (BSC), a radio network controller (RNC), a relay node, and the like. The base station 114a and / or the base station 114b may be configured to transmit and / or receive wireless signals on one or more carrier frequencies called cells (not shown). These frequencies may be in a licensed spectrum, an unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide wireless service coverage for a specific geographic area that is relatively fixed or may change over time. A cell may be further divided into cell sectors. For example, a cell associated with the base station 114a may be divided into three sectors. Thus, in an example, the base station 114a may include three transceivers, i.e., each transceiver corresponds to a sector of the cell. In an example, the base station 114a may use multiple-input multiple-output (MIMO) technology, and may use multiple transceivers for each sector of the cell. For example, by using beamforming, signals may be transmitted and / or received in a desired spatial direction.

[0031] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, millimeter wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0032] More specifically, as described above, the communication system 100 may be a multiple access system and may use one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, and SC-FDMA, among others. For example, the base station 114a in the RAN 104 / 113 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may use Wideband CDMA (WCDMA) to establish the air interface 115 / 116 / 117. WCDMA may include communication protocols such as High Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High Speed ​​UL Packet Access (HSUPA).

[0033] In an example, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 116 using Long Term Evolution (LTE) and / or LTE-Advanced (LTE-A) and / or LTE-Advanced Pro (LTE-A Pro).

[0034] In an example, the base station 114a and the WTRUs 102a, 102b, 102c may implement some radio technology that may establish the air interface 116 using new radio (NR), such as NR radio access.

[0035] In an example, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may jointly implement LTE radio access and NR radio access (e.g., using dual connectivity (DC) principles). Thus, the air interface used by the WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations (e.g., eNBs and gNBs).

[0036] In an example embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement radio technologies such as IEEE 802.11 (i.e., Wireless Fidelity (WiFi)), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data Rates for GSM Evolution (EDGE), and GSM EDGE (GERAN), among others.

[0037] Figure 1AThe base station 114b in the example can be, for example, a wireless router, a Home Node B, a Home eNode B, or an access point, and can use any appropriate RAT to facilitate wireless connectivity in a local area, such as a business place, a residence, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), and a road, etc. In an example, the base station 114b and the WTRUs 102c, 102d can establish a wireless local area network (WLAN) by implementing a radio technology such as IEEE 802.11. In an example, the base station 114b and the WTRUs 102c, 102d can establish a wireless personal area network (WPAN) by implementing a radio technology such as IEEE 802.15. In an example, the base station 114b and the WTRUs 102c, 102d can establish a picocell or a femtocell by using a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.). Figure 1A As shown, the base station 114b may be directly connected to the Internet 110. Thus, the base station 114b does not need to access the Internet 110 via the CN 106 / 115.

[0038] The RAN 104 / 113 may be in communication with the CN 106 / 115, which may be any type of network configured to provide voice, data, applications, and / or Voice over Internet Protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have different quality of service (QoS) requirements, such as different throughput requirements, latency requirements, fault tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. The CN 106 / 115 may provide call control, billing services, mobile location-based services, prepaid calls, Internet connectivity, video distribution, etc., and / or may perform advanced security functions such as user authentication. Although in Figure 1A Although not shown, it will be appreciated that the RAN 104 / 113 and / or the CN 106 / 115 may be in direct or indirect communication with other RANs that employ the same RAT or a different RAT as the RAN 104 / 113. For example, in addition to being connected to the RAN 104 / 113 employing NR radio technology, the CN 106 / 115 may also be in communication with another RAN (not shown) employing GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technology.

[0039] The CN 106 / 115 may also act as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and / or other networks 112. The PSTN 108 may include a circuit-switched telephone network that provides plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer network devices that use common communication protocols, such as TCP, User Datagram Protocol (UDP), and / or IP in the Transmission Control Protocol / Internet Protocol (TCP / IP) Internet protocol suite. The networks 112 may include wired or wireless communication networks owned and / or operated by other service providers. For example, the networks 112 may include another CN connected to one or more RANs, where the one or more RANs may use the same RAT or a different RAT as the RAN 104 / 113.

[0040] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communication system 100 may include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks over different wireless links). Figure 1A The illustrated WTRU 102c may be configured to communicate with the base station 114a, which may employ a cellular-based radio technology, and with the base station 114b, which may employ an IEEE 802 radio technology.

[0041] Figure 1B is a system diagram illustrating an exemplary WTRU 102. Figure 1B As shown, the WTRU 102 may include a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a numeric keypad 126, a display / touchpad 128, a non-removable memory 130, a removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and / or peripherals 138. It should be appreciated that the WTRU 102 may also include any sub-combination of the foregoing components.

[0042] The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, and the like. The processor 118 may perform signal decoding, data processing, power control, input / output processing, and / or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit / receive element 122. Although Figure 1B The processor 118 and the transceiver 120 are depicted as separate components, however it should be appreciated that the processor 118 and the transceiver 120 may also be integrated together in an electronic package or chip.

[0043] The transmit / receive component 122 may be configured to transmit or receive signals to or from a base station (e.g., base station 114a) via the air interface 116. For example, in an example, the transmit / receive component 122 may be an antenna configured to transmit and / or receive RF signals. As an example, in an example, the transmit / receive component 122 may be an emitter / detector configured to transmit and / or receive IR, UV, or visible light signals. In an example, the transmit / receive component 122 may be configured to transmit and / or receive RF and light signals. It should be appreciated that the transmit / receive component 122 may be configured to transmit and / or receive any combination of wireless signals.

[0044] Although in Figure 1B 102 as a single component, but the WTRU 102 may include any number of transmit / receive components 122. More specifically, the WTRU 102 may employ MIMO technology. Thus, in an example, the WTRU 102 may include two or more transmit / receive components 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.

[0045] The transceiver 120 may be configured to modulate signals to be transmitted by the transmit / receive component 122 and to demodulate signals received by the transmit / receive component 122. As described above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers that allow the WTRU 102 to communicate via multiple RATs, such as NR and IEEE 802.11.

[0046] The processor 118 of the WTRU 102 may be coupled to a speaker / microphone 124, a numeric keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light emitting diode (OLED) display unit), and may receive user input data from these components. The processor 118 may also output user data to the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128. In addition, the processor 118 may access information from and store information in any suitable memory such as a non-removable memory 130 and / or a removable memory 132. The non-removable memory 130 may include a random access memory (RAM), a read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In an example, the processor 118 may access information from and store data in memory that is not physically located in the WTRU 102, such as, for example, a server or a home computer (not shown).

[0047] The processor 118 may receive power from the power source 134, and may be configured to distribute and / or control power for use by other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (Ni-Cd), nickel-zinc (Ni-Zn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, and the like.

[0048] The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) related to the current location of the WTRU 102. In addition to or in lieu of the information from the GPS chipset 136, the WTRU 102 may receive location information from a base station (e.g., base stations 114a, 114b) via the air interface 116 and / or determine its location based on the timing of signals received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information using any suitable positioning method.

[0049] The processor 118 may also be coupled to other peripheral devices 138, wherein the peripheral devices may include one or more software and / or hardware modules that provide additional features, functions, and / or wired or wireless connectivity. For example, the peripheral devices 138 may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (for photos and / or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, Modules, frequency modulation (FM) radio units, digital music players, media players, video game console modules, Internet browsers, virtual reality and / or augmented reality (VR / AR) devices, and activity trackers, etc. The peripheral device 138 may include one or more sensors, which may be one or more of the following: gyroscopes, accelerometers, Hall effect sensors, magnetometers, orientation sensors, proximity sensors, temperature sensors, time sensors, geographic location sensors, altimeters, light sensors, touch sensors, magnetometers, barometers, gesture sensors, biometric sensors, and / or humidity sensors, etc.

[0050] The WTRU 102 may include a full-duplex radio where the reception or transmission of some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous for the radio. The full-duplex radio may include an interference management unit 139 that reduces and / or substantially eliminates self-interference by means of hardware (e.g., chokes) or by signal processing by a processor (e.g., a separate processor (not shown) or by the processor 118). In an example, the WTRU 102 may include a half-duplex radio that transmits and receives some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) or downlink (e.g., for reception)).

[0051] Figure 1C 1 is a system diagram showing an example RAN 104 and CN 106. As described above, the RAN 104 may communicate with the WTRUs 102a, 102b, 102c using an E-UTRA radio technology over the air interface 116. The RAN 104 may also be in communication with the CN 106.

[0052] The RAN 104 may include eNode-Bs 160a, 160b, 160c, however it will be appreciated that the RAN 104 may include any number of eNode-Bs. The eNode-Bs 160a, 160b, 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In an example, the eNode-Bs 160a, 160b, 160c may implement MIMO technology. Thus, for example, the eNode-B 160a may use multiple antennas to transmit wireless signals to, and / or receive wireless signals from, the WTRU 102a.

[0053] Each of the eNodeBs 160a, 160b, 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, user scheduling in the UL and / or DL, etc. Figure 1C As shown, the eNode-Bs 160a, 160b, 160c may communicate with each other via an X2 interface.

[0054] Figure 1C The illustrated CN 106 may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (or PGW) 166. Although each of the aforementioned components is described as being part of the CN 106, it should be appreciated that any of these components may be owned and / or operated by an entity other than the CN operator.

[0055] The MME 162 may be connected to each of the eNode-Bs 162a, 162b, 162c in the RAN 104 via an S1 interface and may serve as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, performing bearer activation / deactivation processing, selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, 102c, and the like. The MME 162 may provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM and / or WCDMA.

[0056] The SGW 164 may be connected to each of the eNode-Bs 160a, 160b, 160c in the RAN 104 via an S1 interface. The SGW 164 may generally route and forward user data packets to / from the WTRUs 102a, 102b, 102c. The SGW 164 may also perform other functions, such as anchoring the user plane during an inter-eNB handover, triggering paging processing when DL data is available for the WTRUs 102a, 102b, 102c, and managing and storing the contexts of the WTRUs 102a, 102b, 102c, and the like.

[0057] The SGW 164 may be connected to the PGW 146, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0058] The CN 106 may facilitate communications with other networks. For example, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the CN 106 may include or communicate with an IP gateway, such as an IP Multimedia Subsystem (IMS) server, and the IP gateway may serve as an interface between the CN 106 and the PSTN 108. In addition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0059] Although in Figures 1A-1D The WTRU is described as a wireless terminal, however it should be appreciated that in some examples such a terminal may use a (eg, temporary or permanent) wired communication interface with a communication network.

[0060] In an example, the other network 112 may be a WLAN.

[0061] A WLAN using an infrastructure basic service set (BSS) mode may have an access point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP may access or be connected to a distributed system (DS) or other types of wired / wireless networks that send traffic into and / or out of the BSS. Traffic originating from outside the BSS and destined for the STA may arrive through the AP and be delivered to the STA. Traffic originating from the STA and destined for a destination outside the BSS may be sent to the AP for delivery to the corresponding destination. Traffic between STAs within the BSS may be sent through the AP, for example, when the source STA may send traffic to the AP and the AP may deliver traffic to the destination STA. Traffic between STAs within the BSS may be considered and / or referred to as point-to-point traffic. The point-to-point traffic may be sent using a direct link establishment (DLS) between the source and destination STAs (e.g., directly therebetween). In an example, the DLS may use 802.11e DLS or 802.11z channelized DLS (TDLS)). For example, a WLAN using an independent BSS (IBSS) mode does not have an AP, and STAs (eg, all STAs) within or using the IBSS can communicate directly with each other. Here, the IBSS communication mode may sometimes be referred to as an "ad-hoc" communication mode.

[0062] When using the 802.11ac infrastructure mode of operation or a similar mode of operation, the AP may transmit a beacon on a fixed channel (e.g., a primary channel). The primary channel may have a fixed width (e.g., a bandwidth of 20 MHz) or a width dynamically set via signaling. The primary channel may be an operating channel of the BSS and may be used by the STA to establish a connection with the AP. In an example, carrier sense multiple access (CSMA / CA) with collision avoidance (e.g., in an 802.11 system) may be implemented. For CSMA / CA, STAs including the AP (e.g., each STA) may sense the primary channel. If a particular STA senses / detects and / or determines that the primary channel is busy, the particular STA may back off. In a specified BSS, there is one STA (e.g., only one station) transmitting at any given time.

[0063] A high throughput (HT) STA may communicate using a 40 MHz wide channel (eg, by combining a 20 MHz wide primary channel with a 20 MHz wide adjacent or non-adjacent channel to form a 40 MHz wide channel).

[0064] Very high throughput (VHT) STA can support channels with widths of 20MHz, 40MHz, 80MHz and / or 160MHz. 40MHz and / or 80MHz channels can be formed by combining continuous 20MHz channels. A 160MHz channel can be formed by combining 8 continuous 20MHz channels or by combining two discontinuous 80MHz channels (this combination can be referred to as an 80+80 configuration). For the 80+80 configuration, after channel coding, the data can be transmitted and passed through a segment parser, which can separate the data into two streams. Inverse Fast Fourier Transform (IFFT) processing and time domain processing can be performed separately on each stream. The stream can be mapped on two 80MHz channels, and the data can be transmitted by the STA performing the transmission. At the receiver of the STA performing the reception, the above operations for the 80+80 configuration can be reversed, and the combined data can be sent to the medium access control (MAC).

[0065] 802.11af and 802.11ah support working modes below 1 GHz. Compared with 802.11n and 802.11ac, the channel working bandwidth and carrier used in 802.11af and 802.11ah are reduced. 802.11af supports 5MHz, 10MHz and 20MHz bandwidths in the TV White Space (TVWS) spectrum, and 802.11ah supports 1MHz, 2MHz, 4MHz, 8MHz and 16MHz bandwidths using non-TVWS spectrum. According to an example, 802.11ah can support meter type control / machine type communication (MTC) (e.g., MTC devices in macro coverage areas). MTC devices can have certain capabilities, such as limited capabilities including support (e.g., only support) certain and / or limited bandwidths. MTC devices can include a battery, and the battery life of the battery is higher than a threshold (e.g., for maintaining a very long battery life).

[0066] For WLAN systems that can support multiple channels and channel bandwidths (e.g., 802.11n, 802.11ac, 802.11af, and 802.11ah), these systems include channels that can be designated as primary channels. The bandwidth of the primary channel can be equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel can be set and / or limited by a certain STA, where the STA originates from all STAs operating in the BSS that supports the minimum bandwidth operating mode. In the example of 802.11ah, even if the AP and other STAs in the BSS support 2MHz, 4MHz, 8MHz, 16MHz, and / or other channel bandwidth operating modes, for STAs (e.g., MTC-type devices) that support (e.g., only support) 1MHz mode, the width of the primary channel can be 1MHz. Carrier sensing and / or network allocation vector (NAV) settings can depend on the status of the primary channel. If the primary channel is busy (e.g., because a STA (which only supports the 1MHz operating mode) transmits to the AP), then even if most of the available frequency bands remain idle and available for use, the entire available frequency band can be considered busy.

[0067] In the United States, the available frequency band for 802.11ah is 902MHz to 928MHz. In South Korea, the available frequency band is 917.5MHz to 923.5MHz. In Japan, the available frequency band is 916.5MHz to 927.5MHz. Depending on the country code, the total bandwidth available for 802.11ah is 6MHz to 26MHz.

[0068] Figure 1D1 is a system diagram showing an example RAN 113 and CN 115. As described above, the RAN 113 may communicate with the WTRUs 102a, 102b, 102c using NR radio technology over the air interface 116. The RAN 113 may also be in communication with the CN 115.

[0069] The RAN 113 may include gNBs 180a, 180b, 180c, though it will be appreciated that the RAN 113 may include any number of gNBs. The gNBs 180a, 180b, 180c may each include one or more transceivers to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. In an example, the gNBs 180a, 180b, 180c may implement MIMO technology. For example, the gNBs 180a, 180b may use beamforming processing to transmit and / or receive signals to and / or from the gNBs 180a, 180b, 180c. Thus, for example, the gNB 180a may use multiple antennas to transmit wireless signals to the WTRU 102a and receive wireless signals from the WTRU 102a. In an example, the gNBs 180a, 180b, 180c may implement carrier aggregation technology. For example, the gNB 180a may transmit a plurality of component carriers to the WTRU 102a (not shown). A subset of these component carriers may be on an unlicensed spectrum, while the remaining component carriers may be on a licensed spectrum. In an example, the gNBs 180a, 180b, 180c may implement coordinated multi-point (CoMP) technology. For example, the WTRU 102a may receive coordinated transmissions from the gNB 180a and the gNB 180b (and / or the gNB 180c).

[0070] The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using transmissions associated with a scalable digital configuration. For example, the OFDM symbol spacing and / or OFDM subcarrier spacing may be different for different transmissions, different cells, and / or different portions of the radio transmission spectrum. The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using subframes or transmission time intervals (TTIs) of different or scalable lengths (e.g., containing different numbers of OFDM symbols and / or lasting different absolute time lengths).

[0071] The gNBs 180a, 180b, 180c may be configured to communicate with the WTRUs 102a, 102b, 102c in a standalone configuration and / or a non-standalone configuration. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c without accessing other RANs (e.g., the eNodeBs 160a, 160b, 160c). In a standalone configuration, the WTRUs 102a, 102b, 102c may use one or more of the gNBs 180a, 180b, 180c as mobility anchors. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using signals in an unlicensed band. In a non-standalone configuration, the WTRU 102a, 102b, 102c may communicate / connect to the gNB 180a, 180b, 180c while communicating / connecting to another RAN (e.g., the eNode-B 160a, 160b, 160c). For example, the WTRU 102a, 102b, 102c may communicate with one or more gNBs 180a, 180b, 180c and one or more eNode-Bs 160a, 160b, 160c in a substantially simultaneous manner by implementing the DC principle. In a non-standalone configuration, the eNode-B 160a, 160b, 160c may act as a mobility anchor for the WTRU 102a, 102b, 102c, and the gNB 180a, 180b, 180c may provide additional coverage and / or throughput to serve the WTRU 102a, 102b, 102c.

[0072] Each of the gNBs 180a, 180b, 180c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, support network slicing, dual connectivity, implement interworking between NR and E-UTRA, route user plane data to a user plane function (UPF) 184a, 184b, and route control plane information to an access and mobility management function (AMF) 182a, 182b, and the like. Figure 1D As shown, gNBs 180a, 180b, and 180c can communicate with each other through the Xn interface.

[0073] Figure 1DThe CN 115 shown may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one session management function (SMF) 183a, 183b, and may include a data network (DN) 185a, 185b. Although each of the aforementioned components is described as part of the CN 115, it should be understood that any of these components may be owned and / or operated by an entity other than the CN operator.

[0074] The AMF 182a, 182b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via the N2 interface and may act as a control node. For example, the AMF 182a, 182b may be responsible for authenticating users of the WTRU 102a, 102b, 102c, supporting network slicing (e.g., handling different PDU sessions with different requirements), selecting a specific SMF 183a, 183b, managing registration areas, terminating NAS signaling, and mobility management, etc. The AMF 182a, 182b may use network slicing processing to customize the CN support provided to the WTRU 102a, 102b, 102c based on the type of service used by the WTRU 102a, 102b, 102c. As an example, different network slices may be established for different use cases, such as services relying on Ultra Reliable Low Latency (URLLC) access, services relying on enhanced massive mobile broadband (eMBB) access, and / or services for machine type communication (MTC) access, etc. The AMF 182 may provide a control plane function for switching between the RAN 113 and other RANs (not shown) using other radio technologies (e.g., LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies such as WiFi).

[0075] The SMF 183a, 183b may be connected to the AMF 182a, 182b in the CN 115 via the N11 interface. The SMF 183a, 183b may also be connected to the UPF 184a, 184b in the CN 115 via the N4 interface. The SMF 183a, 183b may select and control the UPF 184a, 184b, and may configure traffic routing through the UPF 184a, 184b. The SMF 183a, 183b may perform other functions such as managing and allocating WTRU or UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notifications, etc. The PDU session type may be IP-based, non-IP-based, Ethernet-based, etc.

[0076] The UPF 184a, 184b can be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via the N3 interface, which can provide the WTRUs 102a, 102b, 102c with access to packet-switched networks (e.g., the Internet 110) to facilitate communication between the WTRUs 102a, 102b, 102c and IP-enabled devices. The UPF 184, 184b can perform other functions, such as routing and forwarding packets, implementing user plane policies, supporting multi-host PDU sessions, processing user plane QoS, buffering downlink packets, and providing mobility anchor processing, etc.

[0077] The CN 115 may facilitate communications with other networks. For example, the CN 115 may include or may communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that acts as an interface between the CN 115 and the PSTN 108. In addition, the CN 115 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In an example, the WTRUs 102a, 102b, 102c may be connected to the local data networks (DNs) 185a, 185b via an N3 interface connected to the UPFs 184a, 184b and an N6 interface between the UPFs 184a, 184b and the DNs 185a, 185b and through the UPFs 184a, 184b.

[0078] In view of Figures 1A-1D and about Figures 1A-1D , one or more or all of the functions described herein with reference to one or more of the following may be performed by one or more simulation devices (not shown): WTRU 102a-d, base station 114a-b, eNodeB 160a-c, MME 162, SGW 164, PGW 166, gNB 180a-c, AMF 182a-b, UPF 184a-b, SMF 183a-b, DN 185a-b, and / or one or more other devices described herein. These simulation devices may be one or more devices configured to simulate one or more or all of the functions described herein. For example, these simulation devices may be used to test other devices and / or simulate network and / or WTRU functions.

[0079] The simulation device can be designed to implement one or more tests about other devices in a laboratory environment and / or an operator network environment. For example, the one or more simulation devices can perform one or more or all functions while being implemented and / or deployed as a part of a wired and / or wireless communication network in whole or in part to test other devices inside the communication network. The one or more simulation devices can perform one or more or all functions while being temporarily implemented or deployed as a part of a wired and / or wireless communication network. The simulation device can be directly coupled to other devices to perform the test, and / or can use over-the-air wireless communication to perform the test.

[0080] One or more simulation devices can perform one or more functions, including all functions, while not being implemented or deployed as part of a wired and / or wireless communication network. For example, the simulation device can be used in a test lab and / or a test scenario of a wired and / or wireless communication network that is not deployed (e.g., testing) to implement tests on one or more components. The one or more simulation devices can be test devices. The simulation device can transmit and / or receive data using direct RF coupling and / or wireless communication with the aid of RF circuits (e.g., which can include one or more antennas).

[0081] A block-based hybrid video coding framework may be provided. Figure 2A block diagram of an exemplary block-based hybrid video coding framework is given. An input video signal 2 may be processed block by block. Block sizes (e.g., extended block sizes, such as coding units (CUs)) may compress high-resolution (e.g., 1080p and above) video signals. For example, a CU may contain 64×64 pixels. A CU may be partitioned into multiple prediction units (PUs), for which multiple separate predictions may be used. For (e.g., each) input video block (e.g., MB and / or CU), spatial prediction 60 and / or temporal prediction 62 may be performed. Spatial prediction (e.g., intra-frame prediction) may use pixels from samples (e.g., reference samples) of coded neighboring blocks in a video picture / slice to predict a current video block. Spatial prediction may reduce spatial redundancy that may be inherent in, for example, a video signal. Temporal prediction (inter-frame prediction and / or motion compensated prediction) may use reconstructed pixels (e.g., ) from a coded video picture to predict a current video block. Temporal prediction may reduce temporal redundancy that may be inherent in, for example, a video signal. The temporal prediction signal for a video block may be signaled by one or more motion vectors (MVs). The MV may indicate the amount and / or direction of motion between a current block and a reference block for the current block. If multiple reference pictures for a (e.g., each) video block are supported, a reference picture index for the video block may be sent. The reference index may be used to identify which reference picture in the reference picture store 64 the temporal prediction signal was derived from. After spatial and / or temporal prediction, a mode decision block 80 in the encoder may determine a prediction mode (e.g., an optimal prediction mode), for example, based on rate-distortion optimization. The prediction block 16 may be subtracted from the current video block. The prediction residual may be decorrelated using a transform 4 and / or quantization 6. The quantized residual coefficients may be inverse quantized 10 and / or inverse transformed 12 to, for example, form a reconstructed residual. For example, the reconstructed residual may be added to the prediction block 26 to form a reconstructed video block. In-loop filtering (e.g., a deblocking filter and / or an adaptive loop filter) may be applied 66 to the reconstructed video block before the reconstructed video block is placed in the reference picture store 64 and / or used to decode a video block (e.g., a future video block). To form the output video bitstream 20, the coding mode (e.g., inter or intra), prediction mode information, motion information, and / or quantized residual coefficients may be sent (e.g., may all be sent) to the entropy coding unit 8, e.g., to be compressed and / or packed to form the bitstream.

[0082] Figure 3A block diagram of an exemplary block-based video decoder is shown. The video bitstream 202 may be unpacked (e.g., first unpacked) and / or entropy decoded at the entropy decoding unit 208. The coding mode and prediction information may be sent to the spatial prediction unit 260 (e.g., if intra-coded) and / or to the temporal prediction unit 262 (e.g., if inter-coded) to form the prediction block. The residual transform coefficients may be sent to the inverse quantization unit 210 and / or the inverse transform unit 212, for example, to reconstruct the residual block. The prediction block may be added to the residual block at 226. For example, the reconstructed block may be in-loop filtered before being stored in the reference picture store 264. The reconstructed video in the reference picture store may be sent to drive a display device and / or used to predict a video block (e.g., a future video block).

[0083] In motion compensated prediction, for (e.g., each) inter-coded block, motion information (e.g., motion vector (MV) and reference picture index) can be used to track the corresponding matching block in the corresponding reference picture, for example, which can be synchronized by the encoder and / or decoder. The motion information of the inter-frame block can be decoded using two modes (e.g., merge mode and non-merge mode). If a block is decoded by non-merge mode, the MV can be decoded (e.g., differentially decoded) using an MV predictor. The difference between the MV and the MV predictor can be sent to the decoder. For (e.g., each) block decoded by merge mode, the motion information of the block can be derived from spatial and / or temporal neighboring blocks. For example, a competition-based scheme can be applied to select neighboring blocks (e.g., the best neighboring blocks) from available candidates. The index (e.g., only the index) of the candidate (e.g., the best candidate) can be sent for reconstructing the motion information (e.g., the same motion information) at the decoder.

[0084] A merge mode may be performed. A candidate (eg, a candidate set) in the merge mode may consist of one or more spatial neighboring candidates (eg, a temporal neighboring candidate and one or more generated candidates). Figure 4Example positions of spatial candidates are shown. In order to construct a list of merge candidates, the spatial candidates may be checked and / or added to the list, for example, in the order of A1, B1, B0, A0, and B2. If a block at a spatial position is intra-coded and / or outside the boundary of the current slice, the block may be unavailable. In order to reduce the redundancy of the spatial candidates, redundant entries (e.g., entries where the candidate has the same motion information as an existing candidate) may be excluded from the list. After including valid spatial candidates, temporal candidates may be generated from the motion information of the co-located blocks in the co-located reference picture by temporal motion vector prediction (TMVP). The size of the merge candidate list (e.g., N) may be set. For example, N may be 5. If the number of merge candidates (e.g., including spatial candidates and temporal candidates) is greater than N, the first N-1 spatial candidates and / or temporal candidates may be kept in the list. For example, (e.g., only) the first N-1 spatial candidates and temporal candidates may be kept in the list. If the number of merge candidates is less than N, one or more candidates (eg, combined candidates and zero candidates) may be added to the candidate list, eg, until the number of merge candidates reaches N.

[0085] One or more candidates may be included in the merge candidate list. Figure 4 The five spatial candidates and the TMVP candidate shown in may be included in the merge candidate list. One or more aspects of motion derivation for the merge mode may be modified, including, for example, sub-block based motion derivation and decoder-side motion vector refinement.

[0086] Sub-block based motion derivation for merge mode may be performed. (Eg., each) merge block may include a set of motion parameters (e.g., motion vector and reference picture index) for (e.g., each) prediction direction. One or more (e.g., two) merge candidates that may enable derivation of motion information at the sub-block level may be included in the merge candidate list. Including merge candidates with sub-block level motion information in the merge candidate list may be achieved by increasing the maximum size (e.g., N) of the merge candidate list from 5 to 7. When selecting one or more of the candidates, the encoder / decoder may divide a CU (e.g., a current CU) into 4×4 sub-blocks and may derive motion information of one (e.g., each) sub-block. Advanced temporal motion vector prediction (ATMVP) may be used. For example, ATMVP may divide a CU into multiple sub-blocks (e.g., 4×4 sub-blocks). ATMVP may be built on TMVP and may allow a CU to obtain motion information of a sub-block of the CU from multiple small blocks from a temporally neighboring picture (e.g., a co-located reference picture) of a current picture. In spatio-temporal motion vector prediction (STMVP), the motion parameters of the sub-block may be derived (eg, recursively) by averaging the motion vectors of temporal neighbors with the motion vectors of spatial neighbors.

[0087] Advanced temporal motion vector prediction may be performed. In ATMVP, a block may be allowed to derive multiple motion information (e.g., motion vectors and / or reference indexes) of multiple sub-blocks in the block from multiple smaller blocks of a temporally neighboring picture of the current picture. For example, ATMVP may derive motion information of a sub-block of a block as follows. A corresponding block (e.g., a co-located block) of the current block may be identified in a temporal reference picture. The selected temporal reference picture may be a co-located picture. The current block may be divided into a plurality of sub-blocks, and motion information of (e.g., each) sub-block may be derived from a corresponding small block in the co-located picture, such as Figure 5 shown.

[0088] The co-located block and / or the co-located picture may be identified by motion information of the spatial neighboring block of the current block. An available candidate (eg, a first available candidate) in a merge candidate list may be considered. Figure 5 An example of available candidates in a merge candidate list being considered is shown. For example, based on the scanning order of the merge candidate list, block A can be identified as an available (e.g., first available) merge candidate for a block (e.g., a current block). The motion vector (e.g., a corresponding motion vector) (e.g., MVA) of block A and / or a reference index of block A can be used to identify the co-located picture and / or the co-located block. The position of the co-located block in the co-located picture can be determined by adding the motion vector (e.g., MVA) of block A to the coordinates of the current block.

[0089] For (eg, each) sub-block in a block (eg, a current block), motion information of a corresponding small block of the sub-block in the same block (eg, Figure 5 ) may be used to derive motion information for the sub-block. For example, after identifying the motion information of (e.g., each) small block in the co-located block, the motion information of the small block may be converted to a motion vector and / or reference index of a corresponding sub-block in the current block, e.g., in the same manner as temporal motion vector prediction (TMVP), where temporal motion vector scaling may be applied.

[0090] Spatio-temporal motion vector prediction (STVMP) may be performed. In STVMP, motion information of multiple sub-blocks in (eg, one) coding block may be derived in a recursive manner. Figure 6 An example of STVMP is shown. For example, the current block may include one or more (e.g., four) sub-blocks, such as A, B, C, and D. The neighboring small blocks that are spatially adjacent to the current block may be labeled as a, b, c, and d, respectively. The motion derivation of sub-block A may start by identifying the spatial neighbors (e.g., two spatial neighbors) of block A. A neighbor (e.g., the first neighbor) of sub-block A may be the upper neighbor c. If the small block c is not available or is not intra-coded, then the neighboring small blocks above the current block may be checked in order (e.g., from left to right), for example. If the small block b is not available or is not intra-coded, then the neighboring small blocks to the left of the current block may be checked in order (e.g., from top to bottom), for example. After the motion information of the spatial neighbors is extracted, the motion information of the temporal neighbors of sub-block A may be obtained in a manner similar to (e.g., the same as) in TMVP. The motion information (e.g., all motion information) of the available spatial and / or temporal neighbors (e.g., up to 3) may be averaged and / or used as the motion information of sub-block A. Based on the raster scan order, STMVP may be repeated, for example, to derive motion information for sub-blocks (eg, all other sub-blocks) in the current video block.

[0091] Decoder-side motion vector refinement for conventional merge candidates may be performed. For merge mode, when a selected merge candidate is bi-predicted, a prediction signal for the current CU may be formed, for example, by averaging the two prediction blocks using two MVs associated with the candidate's reference lists L0 and L1. The motion parameters of spatial / temporal neighbors may be inaccurate and may not represent the true motion of the current CU. Decoder-side motion vector refinement (DMVR) may be applied to refine the MV of a bi-directionally predicted conventional merge candidate. For example, when a traditional merge candidate (e.g., a spatial merge candidate and / or a TMVP merge candidate) is selected, a bi-prediction template may be generated (e.g., first generated) as an average value, for example, based on motion vectors from reference lists L0 and L1, respectively. For example, when weighted prediction is enabled, the average value may be a weighted average value. As described herein, template matching-based local motion refinement may be performed by DMVR using a bi-prediction template around an initial MV.

[0092] Figure 7 An example motion refinement that can be applied in DMVR is shown. DMVR can refine the MV of a conventional merge candidate, for example, as follows. Figure 7 As shown in , a bidirectional prediction template can be generated by averaging prediction blocks (e.g., two prediction blocks) using initial MVs (e.g., MV0 and MV1) in L0 and L1 of the merge candidate. For (e.g., each) reference list (e.g., L0 or L1), a template matching-based motion search can be performed in a local area around the initial MV. For (e.g., each) motion vector (e.g., MV0 or MV1) of the corresponding reference list around the initial MV in the list, a cost value (e.g., the sum of absolute differences (SAD)) between the corresponding prediction block using the bidirectional prediction template and the motion vector can be measured. For the prediction direction, the MV that minimizes the template cost in the prediction direction can be regarded as the final MV in the reference list of the conventional merge candidate. For the prediction direction, one or more (e.g., eight) neighboring MVs (e.g., which have an integer sample offset) around the initial MV can be considered during motion refinement. Refined MVs (e.g., two refined MVs, such as MV0' and MV1', such as Figure 7 As shown) can be used to generate the final bidirectional prediction signal of the current CU.

[0093] As described herein, sub-block based motion derivation (eg, ATMVP and STMVP) and / or DMVR may enhance the efficiency of the merge mode, for example, by improving the granularity and / or precision of the derived motion vectors.

[0094] For ATMVP and / or STMVP, the motion parameters of the current CU may be derived, for example, based on a granularity of 4×4 blocks. The motion derivation may be repeated, for example, to generate motion information (e.g., all motion information) of the current CU. Reference samples may be obtained from a temporal reference picture. The encoder / decoder may switch memory access to one or more (e.g., different) regions within the reference picture.

[0095] Granularity (e.g., 4×4 block size) may be applied in ATMVP and / or STMVP and may be used to derive motion parameters of a CU coded by ATMVP / STMVP in one or more pictures. The motion of video blocks in different pictures may display different characteristics. For example, based on the correlation between the current picture and the reference picture of the current picture, video blocks in one or more pictures (e.g., pictures at a high temporal layer of a random access configuration) may display stable motion. The motion of video blocks in one or more pictures (e.g., pictures at a low temporal layer of a random access configuration) may be unstable. For example, the level of granularity used to derive the motion parameters of the CU coded by ATMVP / STMVP may be adjusted according to different pictures.

[0096] DMVR can be used to compensate for motion inaccuracies caused, for example, by using the motion of the spatial / temporal neighbors of the current CU. DMVR can be enabled for CUs coded by conventional merge mode. When the motion parameters provided by conventional merge candidates are accurate, the improvement achieved by DMVR can be ignored. For example, DMVR can be skipped.

[0097] Signaling may support picture / slice level variations in the granularity (e.g., sub-block size) of the derivation of motion parameters for calculating ATMVP-coded CUs and / or STMVP-coded CUs. The optimal granularity for motion derivation of ATMVP and / or STMVP for the current picture may be determined.

[0098] Early termination may be performed on motion derivation of a DMVR-based merge mode. DMVR may be applied in a regular merge mode. Two or more prediction signals (e.g., two prediction signals) may be generated from a regular merge candidate. For example, the similarity between the prediction signals may be measured to determine whether to skip DMVR.

[0099] Local motion refinement at intermediate bit depths may be performed. Motion refinement for DMVR may be performed at the input bit depth. Some bit shifting and rounding operations (eg, unnecessary bit shifting and rounding operations) may be removed from DMVR.

[0100] Sub-block based motion derivation based on ATMVP and STMVP may be performed. For ATMVP and / or STMVP, motion derivation may be performed at a fixed granularity. This granularity may be signaled in a sequence parameter set (SPS) as a syntax element syntax log2_sub_pu_tmvp_size. The same derivation granularity may be applied to ATMVP and STMVP and may be used to calculate motion parameters of ATMVP / STMVP coded CUs in pictures in a sequence.

[0101] The motion fields generated by ATMVP and / or STMVP can provide different characteristics. As described herein, the motion parameters of the sub-blocks of a CU decoded by STMVP can be recursively derived by averaging the motion information of the spatial and / or temporal neighbors of (e.g., each) sub-block within the CU based on the raster scan order. The motion parameters of a CU decoded by ATMVP can be derived from the temporal neighbors of the sub-blocks within the CU. STMVP can result in stable motion, and the motion parameters of the sub-blocks within the CU can be consistent. Different granularities can be used to derive the motion parameters of ATMVP and STMVP.

[0102] ATMVP and / or STMVP may use motion parameters of temporal neighbors in a reference picture, for example, to calculate motion parameters of a current block. ATMVP and / or STMVP may provide motion estimation (e.g., reliable motion estimation) when there is small motion between the current block and the co-located block of the current block in the reference picture. For blocks with small motion between the current block and the co-located block (e.g., blocks in the highest temporal layer of a random access (RA) configuration), the sub-block motion parameters generated by ATMVP and / or STMVP may be similar. For video blocks that exhibit large motion from the co-located block (e.g., blocks in the lowest temporal layer of a RA configuration), the motion parameters calculated by ATMVP and / or STMVP for a (e.g., each) sub-block may deviate from the motion parameters of the spatially neighboring sub-blocks of the sub-block. The motion derivation may be performed on small sub-blocks. For example, a current coding unit (CU) may be subdivided into one or more sub-blocks, wherein a sub-block (e.g., each sub-block) corresponds to an MV (e.g., a reference MV). The MV (e.g., co-located MV) of a sub-block (e.g., each sub-block) may be identified from the co-located picture based on a reference MV of the sub-block. The motion parameters may be derived from one or more (e.g., different) pictures at one or more (e.g., different) granularities for a CU coded by ATMVP / STMVP. For example, at the picture / slice level, the derived granularity (e.g., the size of the sub-block) may be selected (e.g., adaptively selected) for ATMVP and / or STMVP. For example, the size of the sub-block may be determined based on the temporal layer of the current CU.

[0103] Signaling of ATMVP / STMVP derived granularity adaptively selected at the picture / slice level may be performed. One or more (e.g., two) granularity flags may be signaled in the SPS. For example, slice_atmvp_granularity_enabled_flag and / or slice_stmvp_granularity_enabled_flag may be signaled in the SPS to indicate whether the derived granularity of ATMVP and / or STMVP may be adjusted at the slice level, respectively. A value (e.g., 1) may indicate that the corresponding ATMVP- / STMVP-based derived granularity is signaled at the slice level. A value (e.g., 0) may indicate that the corresponding ATMVP / STMVP derived granularity is not signaled at the slice level and the syntax element sps_log2_subblk_atmvp_size or sps_log2_subblk_stmvp_size) is signaled in the SPS, for example to specify the corresponding ATMVP- / STMVP-based derived granularity, which may be used to reference the slice of the current SPS. Table 1 shows example syntax elements that may be signaled in the SPS. The syntax elements in Table 1 may be used in other high-level syntax structures, such as video parameter sets (VPS) and / or picture parameter sets (PPS). Table 1: Example syntax elements in SPS where ATMVP and STMVP are controlled separately

[0104] The parameter slice_atmvp_granularity_enabled_flag may specify the presence or absence of the syntax element slice_log2_subblk_atmvp_size in the slice segment header of the slice referencing the SPS. For example, a value of 1 may indicate that the syntax element slice_log2_subblk_atmvp_size is present, and a value of 0 may indicate that the syntax element slice_log2_subblk_atmvp_size is not present in the slice segment header of the slice referencing the SPS.

[0105] The parameter sps_log2_subblk_atmvp_size may specify the value of the sub-block size that may be used to derive motion parameters for advanced temporal motion vector prediction for slices that reference the SPS.

[0106] The parameter slice_stmvp_granularity_enabled_flag may specify the presence or absence of the syntax element slice_log2_subblk_stmvp_size in the slice segment header that references the SPS. For example, a value of 1 may indicate that the syntax element slice_log2_subblk_stmvp_size is present, and a value of 0 may indicate that the syntax element slice_log2_subblk_stmvp_size is not present in the slice segment header that references the SPS.

[0107] The parameter sps_log2_subblk_stmvp_size may specify the value of the sub-block size that may be used to derive motion parameters for spatio-temporal motion vector prediction of slices that reference the SPS.

[0108] In Table 1, the syntax elements sps_log2_subblk_atmvp_size and sps_log2_subblk_stmvp_size may be specified (e.g., specified once) and may be applied to a video sequence. One or more (e.g., different) values ​​of sps_log2_subblk_atmvp_size and sps_log2_subblk_stmvp_size may be specified for pictures at one or more (e.g., different) temporal levels. For a current picture that references the SPS, the values ​​of sps_log2_subblk_atmvp_size and sps_log2_subblk_stmvp_size may be determined and / or applied depending on the temporal level to which the current picture belongs. The syntax elements sps_log2_subblk_atmvp_size and sps_log2_subblk_stmvp_size may be applied to a sub-block unit of a square shape. The sub-block unit may be rectangular. For example, if the sub-block unit is rectangular, the sub-block width and height of ATMVP and / or STMVP may be specified.

[0109] Slice-level adaptation of the ATMVP / STMVP-based derivation granularity may be achieved. For example, slice_atmvp_granularity_enabled_flag and / or slice_stmvp_granularity_enabled_flag in the SPS may be set to a value (e.g., 1) indicating the presence of a syntax element. Syntax elements may be signaled in a slice slice segment header of (e.g., each) slice that references the SPS to specify a corresponding level of granularity for the ATMVP / STMVP-based motion derivation of the slice. For example, Table 2 shows example syntax elements that may be signaled in a slice segment header. Table 2: Example syntax elements in a slice segment header, where ATMVP and STMVP are controlled separately

[0110] The parameter slice_log2_subblk_atmvp_size may specify the value of the sub-block size that may be used to derive motion parameters for advanced temporal motion vector prediction for the current slice.

[0111] The parameter slice_log2_subblk_stmvp_size may specify the value of the sub-block size that may be used to derive motion parameters for spatio-temporal motion vector prediction for the current slice.

[0112] As shown in Tables 1 and 2, one or more groups (e.g., two groups) of syntax elements may be used to (e.g., individually) control the sub-block granularity derived from ATMVP and / or STMVP based motion. For example, when the characteristics (e.g., motion regularity) of motion parameters derived from ATMVP and / or STMVP are different, the sub-block granularity derived from ATMVP and / or STMVP based motion may be controlled separately. For example, a set of syntax elements slice_atmvp_stmvp_granularity_enabled_flag and sps_log2_subblk_atmvp_stmvp_size in the SPS and slice_log2_subblk_atmvp_stmvp_size in the slice segment header may be signaled to (e.g., jointly) control the derived granularity of ATMVP and / or STMVP at the sequence level and slice level. Tables 3 and 4 show exemplary syntax changes in the SPS and slice segment header, for example, when the sub-block granularity derived from ATMVP and STMVP based motion is (e.g., jointly) adjusted. Table 3: Example syntax elements in SPS where ATMVP and STMVP are jointly controlled

[0113] The parameter slice_atmvp_stmvp_granularity_enabled_flag may specify the presence or absence of the syntax element slice_log2_subblk_atmvp_stmvp_size in the slice segment header that references the SPS. For example, a value of 1 may indicate that the syntax element slice_log2_subblk_atmvp_stmvp_size is present, and a value of 0 may indicate that the syntax element slice_log2_subblk_atmvp_stmvp_size is not present in the slice segment header that references the SPS.

[0114] The parameter sps_log2_subblk_atmvp_stmvp_size may specify a value of a sub-block size that may be used to derive motion parameters for ATMVP and / or STMVP for slices that reference the SPS. Table 4: Example syntax elements in a slice segment header where ATMVP and STMVP are jointly controlled

[0115] The parameter slice_log2_subblk_atmvp_stmvp_size may specify a value for a sub-block size that may be used to derive motion parameters for advanced temporal motion vector prediction and / or spatio-temporal motion vector prediction for the current slice.

[0116] The sub-block granularity of ATMVP / STMVP based motion derivation at picture / slice level may be determined.

[0117] The granularity level of ATMVP / STMVP-based motion derivation for a picture / slice may be determined based on, for example, the temporal layer of the picture / slice. As described herein, given the correlation between pictures in the same video sequence, the ATMVP / STMVP-derived granularity may be similar to the derived granularity of the picture / slice in, for example, adjacent pictures in the same temporal layer. For example, for pictures in the highest temporal layer of the RA, motion estimation based on ATMVP / STMVP may result in large block partitions. The granularity value may be adjusted (e.g., to a larger value). For pictures in the lowest temporal layer of the RA, motion parameters derived by ATMVP and / or STMVP may be less accurate. The average size of CUs decoded by ATMVP and / or STMVP from previously decoded pictures may be used in the same temporal layer, for example, to calculate sub-block granularity that can be used for ATMVP / STMVP-based motion derivation in the current picture. For example, the current picture may be the i-th picture in the k-th temporal layer. There may be M CUs in the current picture that can be decoded by ATMVP and / or STMVP. If the sizes of the M CUs are s0, s1, …, s M-1 , then the average size of the CU decoded by ATMVP- / STMVP- in the current picture can be calculated as:

[0118] Based on equation (1), when decoding the (i+1)th picture in the kth temporal layer, the corresponding sub-block size derived based on the ATMVP / STMVP motion is This can be determined by:

[0119] For the RA configuration, parallel encoding can be supported. When parallel encoding is enabled, the full-length sequence can be split into multiple independent random access segments (RAS), for example, these segments can span a short duration of sequence playback (for example, about one second) and (for example, each) video segment can be encoded separately. Adjacent RAS can be independent. The result of sequential decoding (for example, encoding the entire sequence frame by frame) can be the same as parallel encoding. When adaptive sub-block granularity derivation is applied (for example, in addition to parallel encoding), a picture can be avoided, for example, using ATMVP / STMVP block size information from a picture of a previous RAS. For example, when encoding the first inter-frame picture in the RAS, the value σ can be set for one or more (for example, all) temporal layers (for example, using 4×4 sub-blocks). k Reset to 0. Figure 8 An example is shown (eg, when the intra period is equal to 8), which indicates the location of the refresh picture, where the ATMVP / STMVP block size statistics can be reset to 0 when parallel coding is enabled. Figure 8 In , blocks surrounded by dashed and / or solid lines can represent intra pictures and inter pictures, respectively, and pattern blocks can represent refresh pictures. Once calculated, The log2() of the value may be sent in the bitstream in the slice header, for example, according to the syntax in Table 1 and Table 2.

[0120] In one example, the sub-block size for the ATMVP / STMVP-based motion derivation can be determined at the encoder and sent to the decoder. In one example, the sub-block size for the ATMVP / STMVP-based motion derivation can be determined at the decoder. Adaptive determination of sub-block granularity can be used as a decoder-side technique. For example, when the encoder and decoder determine the corresponding sub-block size of the ATMVP / STMVP-based motion derivation of a picture / slice, for example using equations (1) and (2), the ATMVP / STMVP block size statistics (e.g., as indicated in equation (1)) can be maintained during encoding and / or decoding and can be used to synchronize the encoder with the decoder. For example, if the ATMVP / STMVP block size statistics are maintained during encoding and decoding, then it may not be sent. The value of .

[0121] Early termination of DMVR based on similarity of prediction blocks may be performed. DMVR may be performed for a CU coded using conventional merge candidates (e.g., spatial candidates and / or TMVP candidates). When the motion parameters provided by the conventional merge candidates are accurate, DMVR may be skipped without coding loss. To determine whether the conventional merge candidate can provide accurate motion estimation for the current CU, the average difference between the two prediction blocks may be calculated, e.g., Among them I (0) (x,y) and I (1) (x, y) may be the sample value at the coordinate (x, y) of the L0 and L1 motion compensation blocks generated using the motion information of the merge candidate; B and N may be the number of samples and the set of sample coordinates as defined in the current CU, respectively; D may be a distortion measure (e.g., sum of squared errors (SSE), sum of absolute differences (SAD), and / or sum of absolute transform differences (SATD)). Given equation (3), for example, if the difference measure between the two prediction signals is not greater than a predefined threshold (e.g., Diff≤D thres ), DMVR may be skipped. If the difference measure between the two prediction signals is greater than a predefined threshold, the prediction signal generated by the merge candidate may be less relevant and / or DMVR may be applied. Fig. 9 Example motion compensation after applying early termination to DMVR is shown.

[0122] Sub-CU level motion derivation may be disabled (e.g., adaptively disabled), for example, based on prediction similarity. Prior to performing DMVR, one or more (e.g., two) prediction signals using the MV of the merge candidate may be available. The prediction signal may be used to determine whether DMVR should be disabled.

[0123] High-precision prediction for DMVR can be performed. For example, the prediction signal for a bidirectional predictive block can be generated by averaging the prediction signals from L0 and L1 with the precision of the input bit depth. If the MV points to a fractional sample position, the prediction signal can be obtained, for example, using interpolation with an intermediate precision (e.g., which may be higher than the input bit depth due to an interpolation operation). The intermediate prediction signal can be rounded to the input bit depth before the averaging operation. For example, the input signal of the averaging operation can be shifted to a lower precision, which may introduce rounding errors into the generated bidirectional prediction signal. For example, if a fractional MV is used for a block, the two prediction signals at the input bit depth can be averaged with the intermediate precision. If the MV corresponds to a fractional sample position, the interpolation filter may not round the intermediate value to the input bit depth, and the intermediate value may be kept at a high precision (e.g., the intermediate bit depth). In the case where one of the two MVs is an integer motion (e.g., the corresponding prediction is generated without applying interpolation), the precision of the corresponding prediction can be increased to the intermediate bit depth before applying the averaging. For example, the accuracy of the two prediction signals may be the same. Fig.10 shows an example bidirectional prediction when averaging two intermediate prediction signals of high precision, where and may refer to two prediction signals at an intermediate bit depth (eg, 14 bits) obtained from lists L0 and L1, and BitDepth may indicate the bit depth of the input video.

[0124] In DMVR, a bidirectional prediction signal (e.g., I (0) (x,y) and I (1) (x, y)). The input signal bit depth may be, for example, 8 bits (e.g., if the input signal is 8 bits) or 10 bits (e.g., if the input signal is 10 bits). For example, the prediction signal may be converted to low precision before motion refinement. Rounding errors may be introduced when measuring the distortion cost. The conversion of the prediction signal from the intermediate bit depth to the input bit depth may involve one or more rounding and / or clipping operations. The prediction signal may be generated at a high bit depth (e.g., at Fig.10 The intermediate bit depth in and ) to perform DMVR-based motion refinement. The corresponding distortion between the two prediction blocks in equation (3) can be calculated with high accuracy as follows: in and It can be the high-precision samples at the coordinates (x, y) of the prediction blocks generated from L0 and L1 respectively. h The corresponding distortion measure calculated at the intermediate bit depth may be represented. Due to the increased bit depth, a threshold (e.g., a distortion measure threshold) that may be used to terminate DMVR early may be adjusted such that the threshold is defined at the same bit depth as the prediction signal. If an L1 norm distortion (e.g., SAD) is used, the distortion threshold may be adjusted from the input bit depth to the intermediate bit depth using the following equation:

[0125] Advanced Temporal Motion Vector Prediction (ATMVP) may be derived. Co-located pictures and co-located blocks may be selected for ATMVP. Motion vectors from spatially neighboring CUs may be added to a candidate list (e.g., a list of potential candidate neighboring CUs). For example, if a neighboring CU is available and the MV of the neighboring CU is different from one or more MVs in the existing candidate list, the motion vector from the spatially neighboring CU may be added. For example, Figure 4 As shown, MVs from neighboring blocks may be added in the order of A1, B1, B0, and A0. The number of available spatial candidates may be denoted as N0. ATMVP may be derived using N0 MVs.

[0126] N0 may be greater than 0. If N0 is greater than 0, then the MV (eg, the first available MV) may be used to determine the co-located picture and / or to extract the offset of the motion. Figure 5 As shown, the first available MV may be from the adjacent CU A. The co-located picture for ATMVP may be a reference picture associated with the MV from CU A. The offset for extracting the motion field may be derived from the MV. N0 may be equal to 0, if N0 is equal to 0, the co-located picture may be set to the co-located picture signaled in the slice header, and the offset for extracting the motion field may be set to 0.

[0127] For example, if multiple reference pictures are used, the co-located pictures derived by ATMVP for different CUs may be different. For example, if multiple reference pictures are used, the co-located pictures derived by ATMVP for different CUs may be different because the determination of the co-located pictures may depend on their neighboring CUs. For the current picture decoding, the co-located picture used for ATMVP may not be fixed, and ATMVP may refer to the motion field of multiple reference pictures. The co-located picture for the current picture decoding may be set to (e.g., one) reference picture, which may be signaled at the slice header, for example. The co-located picture may be identified. The reference picture of the neighboring CU A may be different from the co-located picture. The reference picture of CU A may be represented as R A , the same position picture can be expressed as R col , and the current picture can be represented as P. POC(x) can be used to indicate the POC of picture x. The MV of CU A can be obtained from picture R A Scaling to the co-located picture, as calculated in equation (6), for example, to obtain a prediction for an offset position. MV col =MV(A)*(POC(R col )-POC(P)) / (POC(R A )-POC(P) (6)

[0128] Zoomed MV col can be used as an offset, for example, to extract the co-located image R colThe motion field in equation (6). The scaling in equation (6) may be based on the picture temporal distance. For example, the first available MV from the spatial CU may be selected for scaling. An MV that minimizes the scaling error may be selected. For example, the MV to be scaled (e.g., the best MV) may be selected from one or more (e.g., two) directions (e.g., list L0, list L1) of the N0 MVs to minimize the scaling error. For example, (e.g., each) of the adjacent blocks (e.g., adjacent CU) may have a corresponding reference picture. The adjacent block may be selected as a candidate adjacent block (e.g., a candidate adjacent CU) based on the difference between the reference picture of the adjacent block and the co-located picture. The adjacent block selected as the candidate adjacent block may have the minimum temporal distance between its reference picture and the co-located picture. For example, the reference picture (e.g., each reference picture) and the co-located picture may have a picture order count (POC), and the candidate adjacent block may be selected based on the difference between the POC of the reference picture and the POC of the co-located picture (e.g., having the lowest difference). An MV (e.g., a co-located MV) may be identified from a co-located picture based on an MV (e.g., a reference MV) from a reference picture. The neighboring CU may have the same reference picture as the co-located picture, and the neighboring CU may be selected immediately after determining that the reference picture is the same as the co-located picture (e.g., without considering other neighboring CUs). The MV from the reference picture may be a spatial MV, and the MV from the co-located picture may be a temporal MV.

[0129] For example, if the reference picture is different from the co-located picture, the MV from the reference picture may be scaled. For example, the MV from the reference picture may be multiplied by a scaling factor. The scaling factor may be based on the temporal difference between the reference picture and the co-located picture. The scaling factor may be defined as ((POC(R col )-POC(P)) / (POC(R A )-POC(P)). The scaling factor may have a value indicating no scaling (eg, 1). The scaling factor having a value indicating no scaling may indicate R A and R col is the same picture. The scaling error may be measured in one or more of the following ways. For example, the scaling error may be measured as provided in equation (7) and / or as provided in equation (8). For example, the absolute difference between the scaling factor of a given MV and the scaling factor value representing no scaling may be measured as provided in equation (7). ScaleError=abs((POC(R col )-POC(R A )) / (POC(R A )-POC(P)) (7)

[0130] The absolute difference between a reference picture and a co-located picture for a given MV can be measured, as provided in Equation (8). ScaleError = abs((POC(R col ) - POC(R A ))) (8)

[0131] The best MV search can be terminated. For example, if the ScaleError for a particular MV equals 0 during the search for MV candidates to be scaled, the search can be terminated (e.g., terminated early).

[0132] Neighboring MVs can be used to match the motion field of the co-located picture for ATMVP derivation. For example, the neighboring MVs can be selected by minimizing the accuracy loss caused by MV scaling. This can improve the accuracy of the scaled MV. The presence of valid motion information in the reference block can be indicated by the selected neighboring MVs in the co-located picture. For example, when the reference block is an intra block (e.g., one), the ATMVP can be considered unavailable, e.g., because there is no motion information associated with the reference block.

[0133] The best neighboring MV can be selected from the neighboring MVs pointing to the corresponding inter-coded block(s) in the co-located picture. The MV scaling error can be minimized. Constraints can be imposed, e.g., to ensure that the selected neighboring MV points to an inter-coded block (e.g., one) in the co-located picture when determining the best neighboring MV (e.g., as shown in Equations (7) and (8)). The selection of the best neighboring MV can be formulated as provided in Equation (9). (x, y) can be the center position of the current CU. ColPic(x, y) can indicate the block at position (x, y) within the co-located picture. inter() can represent an indicator function that can indicate whether the input block is an inter block. S can indicate a set of available spatial neighboring blocks, e.g., S = {A1, B1, B0, A0}. ScaleError can indicate the MV scaling error calculated as in Equations (7) and (8). can be the scaled motion vector of neighboring block N. N* can represent the selected spatial neighbor whose MV is used to derive the motion field of the current CU, e.g., based on ATMVP. In one example, the current block can have 3 spatial neighbors A, B, and C, with increasing scaling error (e.g., ScaleError(A) < ScaleError(B) < ScaleError(C)). If is false (e.g., after scaling, the motion of block A identifies an intra-coded reference block in the co-located picture), then the motion of block B (e.g., whose scaling error is the second smallest) can be used to identify the reference block in the co-located picture. If the scaled motion of B identifies an intra-coded reference block, then the motion of block C can be used to identify the reference block in the co-located picture.

[0134] The MV scaling error (e.g., as shown in equation (7)) can be used as a criterion for identifying the best neighboring block, such as shown in equation (9). The motion of the best neighboring block can be used to select the co-located block from the co-located picture. The calculation of the MV scaling error (e.g., as shown in equation (7)) can include, for example, two subtractions, one division, and one absolute operation. The division can be implemented by multiplication and / or right shift (e.g., based on a LUT). The best spatial neighboring block of the current CU can be selected to determine the co-located block for ATMVP. The co-located picture can be signaled at the slice and / or picture level. The co-located picture can be used for ATMVP derivation. It can be in order (e.g., as Figure 4 A1, B1, B0 and A0 shown in the figure check the MV of the existing merge candidate. The MV of the first merge candidate associated with the co-located picture and identifying (e.g., one) block decoded by inter-frame prediction can be selected to obtain the co-located block from the co-located picture. For example, if there is no such candidate, zero motion can be selected. For example, if the selected MV points to (e.g., one) co-located block decoded by inter-frame, ATMVP can be enabled. For example, if the selected MV points to (e.g., one) co-located block decoded by intra-frame, ATMVP can be disabled. Fig.11 An example flow chart illustrating co-located block derivation as described herein is shown.

[0135] The MVs of existing merge candidates can be checked in order. For example, Figure 4 , the order can be A1, B1, B0 and A0. The MV of the first merge candidate associated with the co-located picture can be selected to extract the co-located block from the co-located picture. ATMVP can be enabled based on the decoding mode of the co-located block. For example, if the co-located block is intra-coded, ATMVP can be disabled, for example because the co-located block may not provide motion information. If, for example, no merge candidate is associated with the co-located picture, ATMVP can be disabled. Early termination of the check may be performed. For example, once the first merge candidate associated with the co-located picture is found, the check can be terminated. Fig.14 A flow chart illustrating the use of examination of merge candidates to derive co-located blocks of ATMVP is shown.

[0136] Zero motion can be used to extract the same block in the same position picture. The block at the same position of the current CU in the same position picture can be checked to determine whether to enable ATMVP. For example, if the block is inter-coded, ATMVP can be enabled. For example, if the block is intra-coded, ATMVP can be disabled.

[0137] The region of the co-located block for extracting ATMVP may be constrained. The co-located picture derived for ATMVP for different ATMVP blocks may be constrained to (e.g., one) reference picture. The corresponding co-located block may be indicated by the MV of the selected merge candidate of the adjacent block. The corresponding co-located blocks may be far away from each other. The encoder or decoder may (e.g., frequently) switch between motions (e.g., MVs and / or reference picture indexes) accessing different regions in the co-located picture. Fig.12 An example of unconstrained access to the same block of ATMVP is shown. Fig.12 As shown, there may be one or more (eg, three) ATMVP CUs in the current CTU, wherein the CUs (eg, each CU) use motion offsets (eg, different motion offsets), such as Fig.12 The offset can be used to locate the corresponding co-located block in the co-located picture, for example, Fig.12 The co-located blocks may be located in different areas of the co-located picture, for example, due to motion offsets having different values.

[0138] The co-located blocks of an ATMV CU (eg, each ATMVCP CU) may be derived within (eg, one) restricted range. Fig.13 An example of applying constrained regions to derive co-located blocks of ATMVP is shown. Fig.13 It can be shown that Fig.12 Same co-location image and CTU. Fig.13 As shown in , given the position of the current CU, a constrained area (e.g., region) in the co-located picture can be determined. For example, the constrained area can be determined based on the current slice. The position of the co-located block derived by ATMVP for the current CU can be within the region. The co-located block within the region can be a valid co-located block. The MV from a neighboring block (e.g., a candidate neighboring block) that is not within the region can be replaced by an MV from a valid co-located block. For example, Fig.13As shown, the original co-located blocks of B1 and B2 (e.g., ColB1 and ColB2) may be within the constrained region. The original co-located blocks ColB1 and ColB2 may be used for ATMVP derivation of B1 and B2. Because the original co-located block of B0 (e.g., ColB0) is outside the constrained region, (e.g., one) co-located block (e.g., ColB0') may be generated by clipping the position of ColB0 toward the nearest boundary of the constrained region. For example, ColB0' may be the valid block closest to ColB0. The position of ColB0' may be set to a block located at the same position as the current block in the co-located picture (e.g., the motion of B0 may be set to zero). The MV from ColB0' (e.g., replacing the MV from ColB0) may be used for decoding (e.g., encoding and / or decoding) the CU. For example, when ColB0 is out of bounds, ATMVP may be disabled.

[0139] The size of the constrained region for the derivation of the co-located blocks for ATMVP may be determined. For example, (e.g., one) fixed region may be applied to CUs (e.g., all CUs) decoded by ATMVP in (e.g., one) video sequence. The derivation of the co-located blocks of (e.g., one) ATMVP CU of the current CTU (e.g., the CTU containing the current CU) may be constrained to be within the same region of the co-located CTU in the co-located picture. For example, (e.g., only) the co-located blocks within the CTU in the co-located picture that is located at the same position as the current CTU may be derived. The derivation of the co-located blocks for the TMVP process of (e.g., one) CU may be constrained to be within the current CTU and the column of (e.g., one) 4×4 block. The CTU may include WxH (e.g., width times height) samples. The region for deriving the co-located blocks of the TMVP may be (W+4)xH. The same constrained region size (e.g., the current CTU plus the column (e.g., one column) of the 4×4 block) may be used to derive the co-located blocks for both ATMVP and TMVP. The size of the constraint region may be selected and signaled from the encoder to the decoder. Syntax elements may be added at the sequence and / or picture or slice level. Different profiles and / or levels may be defined for various application requirements. For example, syntax elements may be signaled in a sequence parameter set SPS and / or a picture parameter set PPS, or may be signaled in a slice header.

[0140] The selection of co-located blocks for ATMVP and the constraints on the regions for co-located block derivation can be combined. Fig.14 As shown, the co-located blocks may be selected based on the characteristics of one or more merge candidates. Figure 4A1, B1, B0 and A0 shown in the figure check the MV of the existing merge candidate. The MV of the first merge candidate associated with the co-located picture can be selected to extract the co-located block from the co-located picture. For example, if the co-located block is intra-coded or if no merge candidate is associated with the co-located picture, ATMVP can be disabled. A valid merge candidate can be found. For example, the valid merge candidate may be A1. The co-located block corresponding to A1 in the co-located picture may be represented as ColA1. ColA1 may be outside the limits of the constraint range. If ColA1 is outside the limits of the constraint range, ColA1 can be cropped back to the closest boundary of the constraint area, set to a block in the co-located picture located at the same position as the current block (for example, the motion of A1 can be set to zero), and / or ATMVP can be marked as disabled.

[0141] Fig.15 An example of using a co-located picture to derive an MV of a current block is shown. The co-located picture for the current picture may be signaled, for example, in a slice header. The co-located picture may be used to perform ATMVP on the current picture. For example, a reference picture of a neighboring block of the current block within the current picture may be compared with the co-located picture. The neighboring block may be selected based on the reference picture of the neighboring block having the lowest temporal distance to the co-located picture. The temporal distance may be a POC difference. The reference picture of the neighboring block may be the same as the co-located picture. The MV from the reference picture of the selected neighboring block may be used to determine the MV from the co-located picture. The MV from the co-located picture may be used to decode the current block. For example, if the reference picture of the selected neighboring block is different from the co-located picture, the MV from the co-located picture may be scaled.

[0142] Although the features and elements are described above in specific combinations, it will be understood by those skilled in the art that each feature or element may be used alone or in any combination with other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware embedded in a computer-readable medium and executed by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via a wired or wireless connection) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, buffer memory, semiconductor storage devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM discs and digital versatile discs (DVDs). A processor associated with the software may be used to implement a radio frequency transceiver used in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. A method for decoding a video, the method comprising: Identifying a co-located picture of a current slice of the video, wherein the current slice includes a current block and a neighboring block of the current block, the neighboring block being associated with a reference picture; determining a picture order count (POC) difference between the reference picture and the co-located picture of the neighboring block; determining a temporal motion vector (MV) to obtain a co-located sub-block in the co-located picture based on the POC difference between the reference picture and the co-located picture of the neighboring block, wherein the temporal MV is a zero value based on a case where the POC difference indicates that the reference picture is different from the co-located picture; Based on the time MV, obtaining the co-located sub-block in the co-located picture; and The current block is decoded based on the co-located sub-block in the co-located picture. 2 . The method of claim 1 , wherein the temporal MV is determined as the MV of the neighboring block based on a case where the POC difference indicates that the reference picture is identical to the co-located picture. 3 . The method of claim 1 , wherein the POC difference indicates a temporal distance between the reference picture and the co-located picture of the neighboring block.

4. The method of claim 1 , wherein the co-located picture is identified based on a co-located picture indication in a slice header, and wherein the neighboring block is one of a plurality of neighboring blocks of the current block, and the neighboring block is selected based on a corresponding POC difference between corresponding reference pictures of the plurality of neighboring blocks and the co-located picture.

5. The method of claim 1, wherein the neighboring block is one of a plurality of neighboring blocks of the current block, and the neighboring block is selected without considering other neighboring blocks based on determining that the reference picture of the neighboring block is the same as the co-located picture. The method of claim 1 , wherein the neighboring blocks are associated with a spatial MV. 7 . The method of claim 1 , wherein the neighboring block is associated with a reference MV, and the method further comprises performing temporal scaling on the reference MV based on a condition that the reference picture of the neighboring block is different from the co-located picture.

8. The method of claim 7, wherein performing time scaling comprises: The reference MV is multiplied by a scaling factor, wherein the scaling factor is based on the POC difference between the reference picture and the co-located picture of the neighboring block.

9. The method according to claim 1, wherein the method further comprises: Determining a constrained area in the co-located picture based on the position of the current block; as well as Based on the constraint region, a co-located block associated with the current block in the co-located picture is obtained.

10. The method according to claim 9, wherein the size of the constraint area is larger than the size of the current block, and wherein the method further comprises: Based on the constrained region in the co-located picture, a position of the co-located block associated with the current block is determined, and wherein the co-located block is obtained based on the position of the co-located block.

Images