Adaptive Motion Vector Resolution (AMVR) using Depth or Motion Maps

Abstract

The device may obtain an indication of motion associated with the video block. The device may determine an accuracy level of a motion vector differential associated with the video block based on the indication of motion associated with the video block. The device may decode the motion vector differential associated with the video block using the accuracy level, e.g., if the device is a decoder. The indication of motion may include one or more of a motion vector predictor (MVP) value, motion information, e.g., motion strength, or depth information indicating a distance associated with the video block from a reference point.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to European Provisional Patent Application No. 21306882.8, filed December 21, 2021, and entitled "Adaptive Motion Vector Resolution Using Depth Map or Motion Map", the entirety of which is incorporated by reference as if fully set forth herein. [Background technology]

[0002] Video encoding and / or decoding may be performed using block-based, wavelet-based, and / or object-based systems. A hybrid block-based video coding system may be deployed. Summary of the Invention

[0003] A system, method and means for determining an accuracy level of a motion vector based on an indication of motion of a video block is disclosed.

[0004] The device may obtain an indication of motion associated with the video block. The device may determine an accuracy level of motion vector information (e.g., motion vector difference (MVD)) associated with the video block based on the motion indication associated with the video block. The device may use the accuracy level to decode the motion vector information associated with the video block, e.g., if the device is a decoder. The device may use the accuracy level to encode the motion vector information associated with the video block, e.g., if the device is an encoder.

[0005] The device may receive motion vector information in video data signaled to the device, for example if the device is a decoder. The video data may not include an accuracy level of the motion vector information associated with the video block. The device may determine the accuracy level based on an indication of motion associated with the video block.

[0006] The device may determine an adaptive motion vector resolution (AMVR) index for the video block based on a motion indication associated with the video block. AMVR may be enabled for the video block. The device may implement AMVR by determining an AMVR index for the video block based on the motion indication associated with the video block. The AMVR index may indicate an accuracy level of an MVD associated with the video block, and the accuracy level of the MVD may be determined based on the AMVR index.

[0007] The indication of motion may include one or more of motion information, depth information, or a motion vector predictor (MVP) value. The MVP associated with a video block may indicate motion associated with neighboring blocks of the video block. The motion information associated with a video block may indicate a motion intensity associated with the video block. The depth information associated with a video block may indicate a distance associated with the video block from a reference point (e.g., a distance from a virtual camera).

[0008] In an example, a device may receive an MVD associated with a video block in video data signaled to the device. The video data may indicate a set of accuracy levels, and the device may determine an accuracy level of the MVD based on the set of accuracy levels.

[0009] The device may modify or reorder the AMVR set based on a motion indication associated with the video block. The motion indication may include one or more of motion information associated with the video block, depth information associated with the video block, or a motion vector predictor (MVP) value associated with the video block.

[0010] The device may determine the AMVR index from a set of AMVR indexes for the video block. The device may implement AMVR by obtaining a set of AMVR indexes and selecting an AMVR index for the video block from the set of AMVR indexes based on a motion indication associated with the video block.

[0011] The device may select an AMVR set for the video block based on a motion indication associated with the video block. The device may implement AMVR by obtaining multiple AMVR sets and selecting an AMVR set for the video block from the multiple AMVR sets based on a motion indication associated with the video block. An AMVR index of the selected AMVR set may indicate an accuracy level of the MVD associated with the video block. [Brief description of the drawings]

[0012] [Figure 1A] FIG. 1A is a system diagram illustrating an example communication system in which one or more disclosed embodiments may be implemented. [Figure 1B] FIG. 1B is a system diagram illustrating an example wireless transmit / receive unit (WTRU) that may be used within the communications system shown in FIG. 1A, according to one embodiment. [Figure 1C]FIG. 1C is a system diagram illustrating an example radio access network (RAN) and an example core network (CN) that may be used within the communications system illustrated in FIG. 1A, according to one embodiment. [Figure 1D] FIG. 1D is a system diagram illustrating a further exemplary RAN and a further exemplary CN that may be used within the communications system illustrated in FIG. 1A, according to one embodiment. [Diagram 2] FIG. 2 illustrates an exemplary video encoder. [Diagram 3] FIG. 3 illustrates an exemplary video decoder. [Figure 4] FIG. 4 illustrates an example of a system in which various aspects and embodiments may be implemented. [Diagram 5] FIG. 5 illustrates an example texture frame of a video game along with corresponding depth map, horizontal motion data, and vertical motion data that may be extracted from the game engine. [Figure 6] FIG. 6 illustrates an example architecture of a cloud gaming system in which the game engine may run on a cloud server. [Figure 7] FIG. 7 illustrates an example game scene having a foreground and a background. [Figure 8] FIG. 8 illustrates an exemplary PMVR scheme. [Figure 9] FIG. 9 illustrates one example 900 for determining an AMVR index (eg, a best AMVR index). [Figure 10] FIG. 10 illustrates one example 1000 for determining an AMVR set (eg, a best AMVR set). [Figure 11] FIG. 11 illustrates the speedup of an exemplary encoder using, for example, depth, motion information, etc. [Figure 12] FIG. 12 illustrates an example AMVR index derivation at the encoder side and the decoder side (eg, based on a depth map, a motion map, etc.). [Figure 13]FIG. 13 illustrates one example for deriving a particular AMVR set (eg, a best AMVR set) at the encoder side and decoder side based on, eg, a depth map, a motion map, and the like. [Figure 14] FIG. 14 illustrates an example of a method for obtaining an AMVR parameter (eg, an AMVR parameter associated with at least one of a depth map of a video block, a motion map of a video block, or an average motion vector predictor size). [Figure 15] FIG. 15 illustrates one example of a method for sending acquired AMVR parameters or an indication of acquired AMVR parameters. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] A more detailed understanding may be had from the following description, given by way of example in conjunction with the accompanying drawings, in which:

[0014] 1A is a diagram illustrating an example communications system 100 in which one or more disclosed embodiments may be implemented. Communications system 100 may be a multiple access system that provides content, such as voice, data, video, messaging, broadcasts, etc., to multiple wireless users. Communications system 100 may enable multiple wireless users to access such content through sharing of system resources, including wireless bandwidth. For example, the communications system 100 may use one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), zero-tail unique-word DFT-Spread OFDM (ZT UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block filtered OFDM, filter bank multicarrier (FBMC), and the like.

[0015] 1A, communications system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, public switched telephone network (PSTN) 108, Internet 110, and other networks 112, although it will be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, 102d, any of which may be referred to as a “station” and / or “STA,” may be configured to transmit and / or receive wireless signals, and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscription-based units, paging, mobile phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (loT) devices, watches or other wearable, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., for remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in an industrial and / or automated processing chain context), consumer electronics devices, devices operating in commercial and / or industrial wireless networks, and the like. Any of the WTRUs 102a, 102b, 102c, and 102d may be referred to interchangeably as a UE.

[0016] The communication system 100 may also include a base station 114a and / or a base station 114b. Each of the base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks, such as the CN 106 / 115, the Internet 110, and / or other networks 112. By way of example, the base stations 114a, 114b may be a base station transceiver station (BTS), a Node B, an eNode B, a Home Node B, a Home eNode B, a gNB, an NR Node B, a site controller, an access point (AP), a wireless router, etc. Although the base stations 114a, 114b are each illustrated as a single element, it will be understood that the base stations 114a, 114b may include any number of interconnected base stations and / or network elements.

[0017] The base station 114a may be part of the RAN 104 / 113, which may also include other base stations and / or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), a relay node, etc. The base station 114a and / or the base station 114b may be configured to transmit and / or receive wireless signals on one or more carrier frequencies, which may be referred to as cells (not shown). These frequencies may be licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide wireless service coverage for a particular geographic area, which may be relatively fixed or may change over time. A cell may be further divided into cell sectors. For example, the cell associated with the base station 114a may be divided into three sectors. Thus, in one embodiment, the base station 114a may include three transceivers, i.e., one transceiver for each sector of the cell. In one embodiment, the base station 114a may employ multiple-input multiple output (MIMO) technology and may utilize multiple transceivers for each sector of the cell, for example, using beamforming to transmit and / or receive signals in a desired spatial direction.

[0018] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d via an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0019] More specifically, as noted above, the communications system 100 may be a multiple access system, but may use one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, the base stations 114a and the WTRUs 102a, 102b, 102c in the RAN 104 / 113 may implement a radio technology, such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 115 / 116 / 117 using wideband CDMA (WCDMA). WCDMA may include communications protocols such as High-Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High-Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High-Speed ​​Uplink Packet Access (HSUPA).

[0020] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 116 using Long Term Evolution (LTE) and / or LTE-Advanced (LTE-A) and / or LTE-Advanced Pro (LTE-A Pro).

[0021] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as NR radio access, which may establish the air interface 116 using New Radio (NR).

[0022] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may implement LTE radio access and NR radio access together, e.g., using dual connectivity (DC) principles. Thus, the air interface utilized by the WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations (e.g., eNBs and gNBs).

[0023] In other embodiments, the base station 114a and the WTRUs 102a, 102b, 102c may implement a wireless technology such as IEEE 802.11 (i.e., Wireless Fidelity, WiFi), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access, WiMAX), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), or the like.

[0024] 1A may be, for example, a wireless router, a Home NodeB, a Home eNodeB, or an access point, and may utilize any suitable RAT to facilitate wireless connectivity in a localized area, such as an office, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a road, etc. In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology, such as IEEE 802.11, to establish a wireless local area network (WLAN). In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology, such as IEEE 802.15, to establish a wireless personal area network (WPAN). In yet another embodiment, the base station 114b and the WTRUs 102c, 102d may establish a picocell or a femtocell using a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.). As shown in FIG. 1A, the base station 114b may have a direct connection to the Internet 110. Thus, the base station 114b may not need to access the Internet 110 via the CN 106 / 115.

[0025] The RAN 104 / 113 may communicate with the CN 106 / 115, which may be any type of network configured to provide voice, data, application, and / or voice over internet protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have various quality of service (QoS) requirements, such as, for example, different throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. The CN 106 / 115 may provide call control, billing services, mobile location-based services, prepaid calling, Internet connectivity, video distribution, etc., and / or perform high level security functions such as user authentication. Although not shown in FIG. 1A, it will be appreciated that the RAN 104 / 113 and / or the CN 106 / 115 may communicate directly or indirectly with other RANs that use the same RAT as the RAN 104 / 113 or a different RAT. For example, in addition to being connected to the RAN 104 / 113, which may utilize NR radio technology, the CN 106 / 115 may also communicate with another RAN (not shown) using GSM, UMTS, CDMA2000, WiMAX, E-UTRA, or WiFi radio technology.

[0026] The CN 106 / 115 may also act as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and / or other networks 112. The PSTN 108 may include a circuit-switched telephone network providing plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols, such as transmission control protocol (TCP), user datagram protocol (UDP), and / or internet protocol (IP) of the TCP / IP Internet protocol suite. The networks 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, the networks 112 may include another CN connected to one or more RANs, which may use the same RAT as the RANs 104 / 113 or a different RAT.

[0027] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communications system 100 may include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks over different wireless links). For example, the WTRU 102c shown in FIG. 1A may be configured to communicate with a base station 114a, which may employ a cellular-based wireless technology, and a base station 114b, which may employ an IEEE 802 wireless technology.

[0028] Figure 1B is a system diagram illustrating an example WTRU 102. As shown in Figure 1B, the WTRU 102 may include, among other things, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and / or other peripherals 138. It will be understood that the WTRU 102 may include any sub-combination of the foregoing elements while remaining consistent with an embodiment.

[0029] The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor 118 may perform signal coding, data processing, power control, input / output processing, and / or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit / receive element 122. Although FIG. 1B illustrates the processor 118 and the transceiver 120 as separate components, it will be understood that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.

[0030] The transmit / receive element 122 may be configured to transmit or receive signals to or from a base station (e.g., base station 114a) over the air interface 116. For example, in one embodiment, the transmit / receive element 122 may be an antenna configured to transmit and / or receive RF signals. In one embodiment, the transmit / receive element 122 may be an emitter / detector configured to transmit and / or receive IR, UV, or visible light signals, for example. In yet another embodiment, the transmit / receive element 122 may be configured to transmit and / or receive both RF and light signals. It will be appreciated that the transmit / receive element 122 may be configured to transmit and / or receive any combination of wireless signals.

[0031] 1B as a single element, the WTRU 102 may include any number of transmit / receive elements 122. More specifically, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.

[0032] The transceiver 120 may be configured to modulate signals transmitted by the transmit / receive element 122 and demodulate signals received by the transmit / receive element 122. As mentioned above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers to enable the WTRU 102 to communicate via multiple RATs, such as, for example, NR and IEEE 802.11.

[0033] The processor 118 of the WTRU 102 may be coupled to and may receive user-entered data from a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light-emitting diode (OLED) display unit). The processor 118 may also output user data to the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128. In addition, the processor 118 may access information from and store data in any type of suitable memory, such as non-removable memory 130 and / or removable memory 132. The non-removable memory 130 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, etc. In other embodiments, the processor 118 may access information from and store data in memory that is not physically located on the WTRU 102, such as on a server or home computer (not shown).

[0034] The processor 118 may receive power from the power source 134 and may be configured to distribute and / or control the power to other components in the WTRU 102. The power source 134 may be any suitable device for providing power to the WTRU 102. For example, the power source 134 may include one or more dry batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0035] The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to, or in lieu of, information from the GPS chipset 136, the WTRU 102 may receive location information from a base station (e.g., base stations 114a, 114b) over the air interface 116 and / or determine its location based on the timing of signals being received from two or more nearby base stations. It will be appreciated that the WTRU 102 may obtain location information by way of any suitable location determination method while remaining consistent with an embodiment.

[0036] The processor 118 may further be coupled to other peripherals 138, which may include one or more software and / or hardware modules that provide additional features, functionality, and / or wired or wireless connectivity. For example, the peripherals 138 may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (for photos and / or videos), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, a virtual reality and / or augmented reality (VR / AR) device, an activity tracker, and the like. The peripherals 138 may include one or more sensors, which may be one or more of a gyroscope, an accelerometer, a Hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor, a geolocation sensor, an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, and / or a humidity sensor.

[0037] The WTRU 102 may include a full-duplex radio where transmission and reception of some or all of the signals associated with a particular subframe (e.g., for both the UL (e.g., for transmission) and the downlink (e.g., for reception) may be parallel and / or simultaneous. The full-duplex radio may include an interference management unit to reduce and or substantially eliminate self-interference via either hardware (e.g., a choke) or signal processing via a processor (e.g., via a separate processor (not shown) or via the processor 118). In one embodiment, the WTRU 102 may include a half-duplex radio for transmission and reception of some or all of the signals (e.g., associated with a particular subframe for either the UL (e.g., for transmission) or the downlink (e.g., for reception)).

[0038] 1C is a system diagram illustrating the RAN 104 and the CN 106, according to one embodiment. As mentioned above, the RAN 104 may employ E-UTRA radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 104 may also communicate with the CN 106.

[0039] The RAN 104 may include eNodeBs 160a, 160b, 160c, although it will be understood that the RAN 104 may include any number of eNodeBs while remaining consistent with an embodiment. The eNodeBs 160a, 160b, 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In an embodiment, the eNodeBs 160a, 160b, 160c may implement MIMO technology. Thus, the eNodeB 160a may, for example, use multiple antennas to transmit wireless signals to and / or receive wireless signals from the WTRU 102a.

[0040] Each of the eNodeBs 160a, 160b, 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and / or DL, etc. As shown in FIG 1C, the eNodeBs 160a, 160b, 160c may communicate with one another via an X2 interface.

[0041] 1C may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (or PGW) 166. Although each of the foregoing elements is illustrated as part of the CN 106, it will be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0042] The MME 162 may be connected to each of the eNodeBs 162a, 162b, 162c in the RAN 104 via an S1 interface and may function as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, activating / deactivating bearers, selecting a particular serving gateway during initial attachment of the WTRUs 102a, 102b, 102c, etc. The MME 162 may provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM and / or WCDMA.

[0043] The SGW 164 may be connected to each of the eNodeBs 160a, 160b, 160c in the RAN 104 via an S1 interface. The SGW 164 may generally route and forward user data packets to and from the WTRUs 102a, 102b, 102c. The SGW 164 may perform other functions, such as anchoring the user plane during inter-eNodeB handover, triggering paging when DL data is available to the WTRUs 102a, 102b, 102c, and managing and storing the context of the WTRUs 102a, 102b, 102c.

[0044] The SGW 164 may be connected to a PGW 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0045] The CN 106 may facilitate communications with other networks. For example, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the CN 106 may include or communicate with an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that serves as an interface between the CN 106 and the PSTN 108. In addition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0046] Although the WTRU is depicted in FIGS. 1A-1D as a wireless terminal, in certain representative embodiments, it is contemplated that such a terminal may use a wired communications interface (e.g., temporarily or permanently) with the communications network.

[0047] In an exemplary embodiment, the other network 112 may be a WLAN.

[0048] A WLAN in infrastructure Basic Service Set (BSS) mode may have an Access Point (AP) of the BSS and one or more stations (STAs) associated with the AP. The AP may have access or interface to a Distribution System (DS) or another type of wired / wireless network that carries traffic into and / or out of the BSS. Traffic originating from outside the BSS to the STA may arrive through the AP and be delivered to the STA. Traffic originating from the STA to a destination outside the BSS may be sent to the AP to be delivered to the respective destination. Traffic between STAs within the BSS may be sent, for example, through the AP, where the source STA may send traffic to the AP, which may deliver the traffic to the destination STA. Traffic between STAs within the BSS may be considered and / or referred to as peer-to-peer traffic. Peer-to-peer traffic may be sent between (e.g., directly between) the source STA and the destination STA using a direct link setup (DLS). In certain representative embodiments, the DLS may use 802.11e DLS or 802.11z tunneled DLS (TDLS). A WLAN using an Independent BSS (IBSS) mode may not have an AP, and STAs (e.g., all of the STAs) within or using the IBSS may communicate directly with each other. The IBSS mode of communication may be referred to herein as an "ad-hoc" communication mode.

[0049] When using an 802.11ac infrastructure mode of operation or a similar mode of operation, an AP may transmit a beacon on a fixed channel, such as a primary channel. The primary channel may be a fixed width (e.g., a 20 MHz wide bandwidth) or a width that is dynamically set via signaling. The primary channel may be the operating channel of the BSS, but may be used by STAs to establish a connection with the AP. In certain representative embodiments, for example, in an 802.11 system, Carrier Sense Multiple Access with Collision Avoidance (CSMA / CA) may be implemented. With CSMA / CA, STAs (e.g., all STAs), including the AP, may sense the primary channel. If the primary channel is sensed / detected and / or determined to be busy by a particular STA, the particular STA may back off. One STA (e.g., only one station) may transmit at any given time in a given BSS.

[0050] A High Throughput (HT) STA may use a 40 MHz wide channel for communication, which may be formed, for example, through a combination of a primary 20 MHz channel and adjacent or non-adjacent 20 MHz channels.

[0051] A Very High Throughput (VHT) STA may support 20 MHz, 40 MHz, 80 MHz, and / or 160 MHz wide channels. A 40 MHz and / or 80 MHz channel may be formed by combining multiple contiguous 20 MHz channels. A 160 MHz channel may be formed by combining eight contiguous 20 MHz channels or by combining two non-contiguous 80 MHz channels, which may be referred to as an 80+80 configuration. For the 80+80 configuration, after channel encoding, the data may pass through a segment parser that may split the data into two streams. Inverse Fast Fourier Transform (IFFT) processing and time domain processing may be performed separately on each stream. The streams may be mapped to two 80 MHz channels and the data may be transmitted by the transmitting STA. At the receiver of the receiving STA, the operations described above for the 80+80 configuration may be reversed and the combined data may be sent to the Medium Access Control (MAC).

[0052] Sub-1 GHz operating modes are supported by 802.11af and 802.11ah. Channel operating bandwidths and carriers are reduced in 802.11af and 802.11ah compared to those used in 802.11n and 802.11ac. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in TV White Space (TVWS) spectrum, and 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11ah may support meter-type control / machine-type communication, such as MTC devices in macro coverage areas. MTC devices may have limited capabilities, including certain capabilities, for example, support for (e.g., support only for) certain and / or limited bandwidths. MTC devices may include batteries with above-threshold battery life (e.g., to maintain very long battery life).

[0053] WLAN systems that may support multiple channels and channel bandwidths, such as 802.11n, 802.11ac, 802.11af, and 802.11ah, include a channel that may be designated as a primary channel. The primary channel may have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel may be configured and / or limited by the STAs among all STAs operating in the BSS that support the smallest bandwidth operating mode. In an 802.11ah embodiment, the primary channel may be 1 MHz wide for STAs (e.g., MTC-type devices) that support (e.g., only) the 1 MHz mode, even if the AP and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and / or other channel bandwidth operating modes. Carrier sensing and / or Network Allocation Vector (NAV) configuration may depend on the status of the primary channel. For example, if the primary channel is active due to a STA (that only supports the 1 MHz mode of operation) transmitting to the AP, the entire available frequency band may be considered active even though most of the frequency band may remain dormant and available.

[0054] In the United States, the available frequency bands that can be used by 802.11ah are 902MHz to 928MHz. In South Korea, the available frequency bands are 917.5MHz to 923.5MHz. In Japan, the available frequency bands are 916.5MHz to 927.5MHz. The total bandwidth available for 802.11ah is 6MHz to 26MHz depending on the country code.

[0055] 1D is a system diagram illustrating the RAN 113 and the CN 115, according to one embodiment. As mentioned above, the RAN 113 may employ NR radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 113 may also communicate with the CN 115.

[0056] The RAN 113 may include gNBs 180a, 180b, 180c, although it will be understood that the RAN 113 may include any number of gNBs while remaining consistent with an embodiment. The gNBs 180a, 180b, 180c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In an embodiment, the gNBs 180a, 180b, 180c may implement MIMO technology. For example, the gNBs 180a, 108b may utilize beamforming to transmit signals to and / or receive signals from the gNBs 180a, 180b, 180c. Thus, the gNB 180a may transmit and / or receive wireless signals to and from the WTRU 102a, for example, using multiple antennas. In one embodiment, the gNBs 180a, 180b, 180c may implement carrier aggregation technology. For example, the gNB 180a may transmit multiple component carriers to the WTRU 102a (not shown). A subset of these component carriers may be on an unlicensed spectrum, while the remaining component carriers may be on a licensed spectrum. In one embodiment, the gNBs 180a, 180b, 180c may implement Coordinated Multi-Point (CoMP) technology. For example, the WTRU 102a may receive coordinated transmissions from the gNBs 180a and 180b (and / or gNB 180c).

[0057] The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using transmissions associated with scalable numerology. For example, the OFDM symbol spacing and / or OFDM subcarrier spacing may vary for different transmissions, different cells, and / or different portions of the wireless transmission spectrum. The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using subframes or transmission time intervals (TTIs) of different or scalable lengths (e.g., including different numbers of OFDM symbols and / or lasting different lengths of absolute time).

[0058] The gNBs 180a, 180b, 180c may be configured to communicate with the WTRUs 102a, 102b, 102c in a standalone configuration and / or a non-standalone configuration. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c without accessing another RAN (e.g., eNodeBs 160a, 160b, 160c, etc.). In a standalone configuration, the WTRUs 102a, 102b, 102c may utilize one or more of the gNBs 180a, 180b, 180c as mobility anchor points. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using signals in unlicensed bands. In a non-standalone configuration, the WTRUs 102a, 102b, 102c may communicate with and connect to a gNB 180a, 180b, 180c while also communicating with and connecting to another RAN, such as an eNodeB 160a, 160b, 160c. For example, the WTRUs 102a, 102b, 102c may implement DC principles to communicate with one or more gNBs 180a, 180b, 180c and one or more eNodeBs 160a, 160b, 160c substantially simultaneously. In a non-standalone configuration, the eNodeBs 160a, 160b, 160c may act as mobility anchors for the WTRUs 102a, 102b, 102c, and the gNBs 180a, 180b, 180c may provide additional coverage and / or throughput for serving the WTRUs 102a, 102b, 102c.

[0059] Each of the gNBs 180a, 180b, 180c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and / or DL, support for network slicing, dual connectivity, interworking between NR and E-UTRA, routing of user plane data to User Plane Functions (UPFs) 184a, 184b, routing of control plane information to Access and Mobility Management Functions (AMFs) 182a, 182b, etc. As shown in FIG. 1D, the gNBs 180a, 180b, 180c may communicate with each other via an Xn interface.

[0060] 1D may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one Session Management Function (SMF) 183a, 183b, and possibly a Data Network (DN) 185a, 185b. Although each of the foregoing elements is illustrated as part of the CN 115, it will be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0061] The AMF 182a, 182b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via an N2 interface and may function as a control node. For example, the AMF 182a, 182b may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, supporting network slicing (e.g., handling different PDU sessions with different requirements), selecting a particular SMF 183a, 183b, managing registration areas, terminating NAS signaling, mobility management, etc. The network slicing may be used by the AMF 182a, 182b to customize the CN support of the WTRUs 102a, 102b, 102c based on the type of service utilizing the WTRUs 102a, 102b, 102c. For example, different network slices may be established for different use cases such as services relying on ultra-reliable low latency (URLLC) access, services relying on enhanced massive mobile broadband (eMBB) access, services for machine type communication (MTC) access, etc. The AMF 162 may provide a control plane function for switching between the RAN 113 and other RANs (not shown) that employ other radio technologies such as LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies such as WiFi.

[0062] The SMFs 183a, 183b may be connected to the AMFs 182a, 182b in the CN 115 via an N11 interface. The SMFs 183a, 183b may also be connected to the UPFs 184a, 184b in the CN 115 via an N4 interface. The SMFs 183a, 183b may select and control the UPFs 184a, 184b and configure the routing of traffic through the UPFs 184a, 184b. The SMFs 183a, 183b may perform other functions such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, providing downlink data notification, etc. The PDU session type may be IP-based, non-IP-based, Ethernet-based, etc.

[0063] The UPFs 184a, 184b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via an N3 interface, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The UPFs 184, 184b may perform other functions, such as routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering downlink packets, providing mobility anchoring, etc.

[0064] The CN 115 may facilitate communication with other networks. For example, the CN 115 may include or communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between the CN 115 and the PSTN 108. In addition, the CN 115 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, the WTRUs 102a, 102b, 102c may be connected to local data networks (DNs) 185a, 185b through the UPFs 184a, 184b via an N3 interface to the UPFs 184a, 184b and an N6 interface between the UPFs 184a, 184b and the DNs 185a, 185b.

[0065] 1A-1D and the corresponding description thereof, one or more or all of the functions described herein with respect to one or more of the WTRUs 102a-d, base stations 114a-b, eNodeBs 160a-c, MME 162, SGW 164, PGW 166, gNBs 180a-c, AMFs 182a-b, UPFs 184a-b, SMFs 183a-b, DNs 185a-b, and / or any other devices described herein may be performed by one or more emulation devices (not shown). The emulation devices may be one or more devices configured to emulate one or more or all of the functions described herein. For example, the emulation devices may be used to test other devices and / or to simulate network and / or WTRU functions.

[0066] The emulation device may be designed to implement one or more tests of other devices in a lab environment and / or an operator network environment. For example, one or more emulation devices may perform one or more or all functions while fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices in the communication network. One or more emulation devices may perform one or more or all functions while temporarily implemented / deployed as part of a wired and / or wireless communication network. The emulation device may be directly coupled to another device for testing purposes and / or may use terrestrial wireless communication to implement the tests.

[0067] The one or more emulation devices may perform one or more functions, inclusive, while not being implemented / deployed as part of a wired and / or wireless communication network. For example, the emulation devices may be utilized in test scenarios in a test lab and / or in an undeployed (e.g., test) wired and / or wireless communication network to implement testing of one or more components. The one or more emulation devices may be test equipment. Direct RF coupling and / or wireless communication via RF circuitry (which may include, for example, one or more antennas) may be used by the emulation devices to transmit and / or receive data.

[0068] This application describes various aspects, including tools, features, examples, models, techniques, and the like. Many of these aspects are described with specificity and often in a potentially limiting manner to at least illustrate their individual characteristics. However, this is for purposes of clarity of description and does not limit the applicability or scope of the aspects. In fact, all of the different aspects may be combined and interchanged to provide further aspects. Moreover, these aspects may likewise be combined and interchanged with aspects described in prior applications.

[0069] Aspects described and contemplated in this application may be implemented in many different forms. While Figures 5-15 described herein may provide some examples, other examples are contemplated. Discussion of Figures 5-15 is not intended to limit the scope of implementations. At least one of the aspects relates generally to video encoding and decoding, and at least one other aspect relates generally to transmitting a generated or encoded bitstream. These and other aspects may be implemented as a method, an apparatus, a computer-readable medium including (e.g., stored thereon) instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium having stored thereon a bitstream generated according to any of the described methods.

[0070] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image", "picture", and "frame" may be used interchangeably.

[0071] Various methods are described herein, each of which includes one or more steps or actions for achieving the described method. Unless a specific order of steps or actions is required for proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined. Additionally, terms such as "first", "second", etc. may be used in various embodiments to modify elements, components, steps, operations, etc., such as, for example, "first decode" and "second decode". The use of such terms does not imply any ordering to the modified operations, unless specifically required. Thus, in this embodiment, the first decode need not be performed before the second decode, but may occur, for example, before, during, or during an overlapping time with the second decode.

[0072] Various methods and other aspects described herein may be used to modify modules, such as the decoding modules, of the video encoder 200 and decoder 300, as shown in Figures 2 and 3. Moreover, the subject matter disclosed herein may apply to any type, format, or version of video coding, whether described in a standard or recommendation, whether existing or developed in the future, and to extensions of any such standards and recommendations, for example. Unless otherwise indicated or technically impossible, aspects described herein may be used individually or in combination.

[0073] In the embodiments described in this application, various numerical values ​​are used, such as ¼ pixel, ½ pixel, 1 pixel, etc. These and other specific values ​​are provided for purposes of illustrating the embodiments, and the described aspects are not limited to these specific values.

[0074] 2 illustrates an exemplary video encoder. Although variations of the exemplary encoder 200 are contemplated, the encoder 200 is described below for purposes of clarity without describing all possible variations.

[0075] Before being encoded, a video sequence undergoes pre-coding processing at 201, for example by applying a color transformation to the input color picture (e.g., converting from RGB 4:4:4 to YCbCr 4:2:0) or by performing a re-mapping of the input picture components to obtain a signal distribution that is more resistant to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with that pre-processing and attached to the bitstream.

[0076] In the encoder 200, a picture is coded by the encoder elements as described below. The picture to be coded is divided 202 and processed, for example, in units of coding units (CUs). Each unit is coded, for example, using either intra mode or inter mode. When a unit is coded in intra mode, it performs 260 intra prediction. In inter mode, it performs motion estimation 275 and motion compensation 270. The encoder decides 205 whether to use intra mode or inter mode to code the unit, and indicates the intra / inter decision, for example, by a prediction mode flag. It calculates a prediction residual, for example, by subtracting 210 the prediction block from the original image block.

[0077] This prediction residual is then transformed 225 and quantized 230. The quantized transform coefficients, as well as the motion vectors and other syntax elements, are entropy coded 245 to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., the residual is coded directly without applying a transform or quantization process.

[0078] The encoder decodes the coded blocks to provide references for further prediction, dequantizes 240 and inverse transforms 250 the quantized transform coefficients, and decodes the prediction residual. The encoder combines 255 the decoded prediction residual with the prediction block to reconstruct an image block. An in-loop filter 265 is applied to the reconstructed picture to perform, for example, deblocking / sample adaptive offset (SAO) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (280).

[0079] 3 is a diagram illustrating an example of a video decoder. In the exemplary decoder 300, the bitstream is decoded by a decoder element as described below. The video decoder 300 generally performs a decoding path that is the inverse of the encoding path described in FIG. 2. The encoder 200 also generally performs video decoding as part of encoding the video data.

[0080] In particular, the decoder input includes a video bitstream, which may be generated by the video encoder 200. First, the bitstream is entropy decoded 330 to obtain transform coefficients, motion vectors, and other coded information. The picture partition information indicates how the picture is partitioned. Thus, the decoder can partition 335 the picture according to the decoded picture partition information. The transform coefficients are dequantized 340 and inverse transformed 350, and the prediction residual is decoded. The decoded prediction residual is combined 355 with a prediction block to reconstruct an image block. The predicted block may be obtained 370 from intra prediction 360 or from motion compensated prediction (i.e., inter prediction) 375. An in-loop filter 365 is applied to the reconstructed image. The filtered image is stored in a reference picture buffer 380.

[0081] The decoded picture may further undergo post-decoding processing 385, such as an inverse color conversion (e.g., YCbCr 4:2:0 to RGB 4:4:4 conversion) or inverse remapping that performs the inverse of the remapping process performed in pre-encoding processing 201. Post-decoding processing may use metadata derived in pre-encoding processing and signaled in the bitstream. In one embodiment, the decoded image (e.g., after application of the in-loop filter 365 and / or after post-decoding processing 385 if post-decoding processing is used) may be sent to a display device for rendering to a user.

[0082] FIG. 4 illustrates an example of a system in which various aspects and embodiments described herein may be implemented. System 400 may be embodied as a device including various components described below and configured to implement one or more of the aspects described herein. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected appliances, and servers. The elements of system 400, singly or in combination, may be embodied in a single integrated circuit (IC), multiple ICs, and / or separate components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 400 are distributed across multiple ICs and / or separate components. In various embodiments, system 400 is communicatively coupled to one or more other systems or other electronic devices, for example, via a communication bus or through dedicated input and / or output ports. In various embodiments, system 400 is configured to implement one or more of the aspects described herein.

[0083] The system 400 includes at least one processor 410 configured to execute instructions loaded therein, for example, to implement various aspects described herein. The processor 410 may include embedded memory, input / output interfaces, and various other circuits known in the art. The system 400 includes at least one memory 420 (e.g., a volatile memory device and / or a non-volatile memory device). The system 400 includes a storage device 440, which may include non-volatile memory and / or volatile memory, including, but not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Flash, magnetic disk drives, and / or optical disk drives. Storage devices 440 may include, by way of non-limiting example, internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0084] The system 400 includes an encoder / decoder module 430 configured to process data to provide, for example, encoded or decoded video, which may include its own processor and memory. The encoder / decoder module 430 represents a module that may be included within a device to perform encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Additionally, the encoder / decoder module 430 may be implemented as a separate element of the system 400 or may be incorporated within the processor 410 as a combination of hardware and software, as is known to those skilled in the art.

[0085] Program code to be loaded into the processor 410 or the encoder / decoder 430 to implement various aspects described herein may be stored in the storage device 440 and then loaded into the memory 420 for execution by the processor 410. According to various embodiments, one or more of the processor 410, the memory 420, the storage device 440, and the encoder / decoder module 430 may store one or more of various items during performance of the processes described herein. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from processing of equations, expressions, operations, and operational logic.

[0086] In some embodiments, memory internal to the processor 410 and / or the encoder / decoder module 430 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device can be either the processor 410 or the encoder / decoder module 430) is used for one or more of these functions. The external memory can be the memory 420 and / or the storage device 440, e.g., dynamic volatile memory and / or non-volatile flash memory. In some embodiments, an external non-volatile flash memory is used, for example, to store the television's operating system. In at least one embodiment, a high-speed external dynamic volatile memory, such as a RAM, is used as working memory for video encoding and decoding operations.

[0087] Inputs to the elements of system 400 may be provided through a variety of input devices, as shown in block 445. Such input devices may include, but are not limited to, (i) a radio frequency (RF) section that receives, for example, RF signals transmitted over the air by a broadcast station, (ii) a component (COMP) input terminal (or set of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a High Definition Multimedia Interface (HDMI) input terminal. Other examples include composite video, although not shown in FIG. 4.

[0088] In various embodiments, the input devices of block 445 have associated respective input processing elements as known in the art. For example, the RF section may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or band-limiting a signal to a band of frequencies), (ii) down-converting the selected signal, (iii) band-limiting again to a narrower frequency band to select a signal frequency band, which in certain embodiments may be referred to as a channel (for example), (iv) demodulating the down-converted band-limited signal, (v) performing error correction, and / or (vi) demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements for performing these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner to perform various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or to baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted over a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and filtering again to a desired frequency band. Various embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements to perform similar or different functions. Adding elements can include inserting elements between existing elements, such as, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0089] The USB and / or HDMI terminals may include respective interface processors for connecting the system 400 to other electronic devices via USB and / or HDMI connections. It should be appreciated that various aspects of the input processing, e.g., Reed-Solomon error correction, may be implemented, for example, in a separate input processing IC or within the processor 410, as desired. Similarly, aspects of the USB or HDMI interface processing may be implemented, for example, in a separate interface IC or within the processor 410, as desired. The demodulated, error corrected, and demultiplexed streams are provided to various processing elements, including, for example, the processor 410 and an encoder / decoder 430 operating in combination with memory and storage elements that process the data streams as desired for presentation on an output device.

[0090] The various elements of system 400 may be provided within an integrated housing in which the various elements may be interconnected and transmit data between them using a suitable connection arrangement 425, such as an internal bus known in the art, including an Inter-IC (I2C) bus, hardwired, and printed circuit boards.

[0091] System 400 includes a communication interface 450 that enables communication with other devices over a communication channel 460. Communication interface 450 may include, but is not limited to, a transceiver configured to transmit and receive data over communication channel 460. Communication interface 450 may include, but is not limited to, a modem or a network card, and communication channel 460 may be implemented in a wired and / or wireless medium, for example.

[0092] Data is streamed or otherwise provided to system 400 in various embodiments using a wireless network such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi-Fi signal in these embodiments is received via communication channel 460 and communication interface 450 adapted for Wi-Fi communication. Communication channel 460 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to enable streaming applications and other over-the-top communications. Other embodiments provide streamed data to system 400 using a set-top box that delivers data via an HDMI connection in input block 445. Still other embodiments provide streamed data to system 400 using an RF connection in input block 445. As noted above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, e.g., cellular networks or Bluetooth networks.

[0093] The system 400 can provide output signals to various output devices, including a display 475, speakers 485, and other peripheral devices 495. The display 475 in various embodiments includes, for example, one or more of a touch screen display, an organic light emitting diode (OLED) display, a curved display, and / or a foldable display. The display 475 can be for a television, a tablet, a laptop, a mobile phone, or other device. The display 475 can also be integrated with other components (e.g., as in a smart phone) or can be separate (e.g., an external monitor for a laptop). The other peripheral devices 495 in various embodiments include one or more of a stand-alone digital video disc (or digital versatile disc) (DVD for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 495 that provide functionality based on the output of the system 400. For example, a disc player performs the functionality of playing the output of the system 400.

[0094] In various embodiments, control signals are communicated between system 400 and display 475, speaker 485, or other peripheral devices 495 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that allow control between devices with or without user intervention. Output devices may be communicatively coupled to system 400 via dedicated connections through respective interfaces 470, 480, and 490. Alternatively, output devices may be connected to system 400 via communication interface 450 using communication channel 460. Display 475 and speaker 485 may be integrated into a single unit with other components of system 400 in an electronic device such as, for example, a television. In various embodiments, display interface 470 includes a display driver, such as, for example, a timing controller (TCon) chip.

[0095] Display 475 and speakers 485 may alternatively be separate from one or more of the other components, for example, if the RF portion of input 445 is part of a separate set-top box. In various embodiments in which display 475 and speakers 485 are external components, output signals may be provided via dedicated output connections including, for example, an HDMI port, a USB port, or a COMP output.

[0096] These embodiments may be performed by the processor 410, or by computer software implemented by hardware, or by a combination of hardware and software. As a non-limiting example, these embodiments may be implemented by one or more integrated circuits. The memory 420 may be of any type suitable for the technology environment, and may be implemented using any suitable data storage technology, such as, as non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memories, and removable memories. The processor 410 may be of any type suitable for the technology environment, and may include, as non-limiting examples, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0097] Various implementations include decoding. As used in this application, "decoding" can encompass all or part of the processes performed on a received encoded sequence to produce a final output suitable for, for example, a display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding.In various embodiments, such a process may also or alternatively be performed by a decoder of various implementations described herein, such as, for example, obtaining an adaptive motion vector resolution (AMVR) parameter associated with at least one of a depth map of the video block, a motion map of the video block, or an average motion vector predictor size; processing the video block based on the obtained AMVR parameter associated with at least one of the depth map of the video block, the motion map of the video block, or the average motion vector predictor size; obtaining the AMVR parameter based on at least one of the depth map of the video block, the motion map of the video block, or the average motion vector predictor size; receiving an indication of the AMVR parameter, e.g., at a slice level via a bitstream; The method includes obtaining an R parameter; selecting an AMVR index from a plurality of AMVR indices based on at least one of a depth map of the video block, a motion map of the video block, or an average motion vector predictor size; selecting an AMVR set from a plurality of AMVR sets based on at least one of the depth map of the video block, the motion map of the video block, or an average motion vector predictor size; receiving an indication of AMVR enablement, e.g., at a picture level or a slice level via a bitstream, and processing the video block using the indication of AMVR enablement; determining an accuracy level of motion vector information (e.g., motion vector differential (MVD)) associated with the video block based on an indication of motion associated with the video block; and decoding the motion vector information associated with the video block using the accuracy level.

[0098] As a further example, in one embodiment, "decoding" refers to entropy decoding only, while in another embodiment, "decoding" refers to differential decoding only, while in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or to the broader decoding process as a whole will be clear based on the context of the specific description and will be well understood by one of ordinary skill in the art.

[0099] Various implementations include encoding. Similar to the above discussion regarding “decoding,” “encoding” as used herein can encompass all or a portion of the processes performed on an input video sequence to produce, for example, an encoded bitstream. In various embodiments, such processes include one or more of processes typically performed by an encoder, such as partitioning, differential encoding, transforming, quantizing, and entropy encoding. In various embodiments, such processes may also or alternatively include processes performed by an encoder of various implementations described herein, such as obtaining an AMVR parameter based on at least one of a depth map of the video block, a motion map of the video block, or an average motion vector predictor size, sending the obtained AMVR parameter or an indication of the obtained AMVR parameter to a device, for example, in a bitstream, sending the indication of the AMVR parameter at a slice level, determining whether to enable AMVR for the video block, and encoding the video block in the bitstream, for example, at a picture level or a slice level. sending an indication of whether to enable AMVR for the video block; selecting an AMVR index from a plurality of AMVR indices based on at least one of a depth map of the video block, a motion map of the video block, or an average motion vector predictor size; selecting an AMVR set from a plurality of AMVR sets based on at least one of the depth map of the video block, the motion map of the video block, or an average motion vector predictor size; determining an accuracy level of motion vector information (e.g., motion vector differential (MVD)) associated with the video block based on an indication of motion associated with the video block; and encoding the motion vector information associated with the video block using the accuracy level.

[0100] As a further example, in one embodiment, "encoding" refers to only entropy encoding, while in another embodiment, "encoding" refers to only differential encoding, while in another embodiment, "encoding" refers to a combination of differential and entropy encoding. Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or to the broader encoding process as a whole will be clear based on the context of the specific description and will be well understood by one of ordinary skill in the art.

[0101] It should be noted that the syntax elements used in this specification, such as sh_amvr_enabled_flag, ph_amvr_enabled_flag, sh_signal_amvr_set_flag, sh_amvr_set_idc, sh_num_amvr_set, or amvr_precision_set[i], are descriptive terms and therefore do not exclude the use of other syntax element names.

[0102] Where a figure is presented as a flow diagram, it should be understood that the figure also provides a block diagram of the corresponding apparatus. Similarly, where a figure is presented as a block diagram, it should be understood that the figure also provides a flow diagram of the corresponding method / process.

[0103] The implementations and aspects described herein may be implemented, for example, in a method or process, an apparatus, a software program, a data stream, or a signal. Even if discussed in the context of only a single implementation (e.g., discussed only as a method), the implementation of the discussed features may also be implemented in other forms (e.g., an apparatus or a program). An apparatus may be implemented, for example, in appropriate hardware, software, and firmware. A method may be implemented, for example, in a processor, which generally refers to a processing device and includes, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include, for example, communication devices such as computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end users.

[0104] References to "one example" or "an example" or "one implementation" or "an implementation," as well as other variations thereof, mean that a particular feature, structure, characteristic, etc. described in connection with an embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one example" or "in an example" or "in one implementation" or "in an implementation" in various places throughout this application, as well as any other variations, are not necessarily all referring to the same embodiment.

[0105] Additionally, the application may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory. Obtaining may include receiving, retrieving, constructing, generating, and / or determining.

[0106] Additionally, the application may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0107] Additionally, the application may refer to "receiving" various information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" generally involves in some manner an operation, such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0108] For example, in the case of "A / B," "A and / or B," and "at least one of A and B," it should be understood that the use of any of the following " / ," "and / or," and "at least one of" is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), or the selection of only the first and third listed alternatives (A and C), or the selection of only the second and third listed alternatives (B and C), or the selection of all three alternatives (A and B and C). This may be expanded as many times as the number of items listed, as would be apparent to one of ordinary skill in the art and related arts.

[0109] Also, as used herein, the term "signaling" means, among other things, to indicate something to a corresponding decoder. The encoder signal may include, for example, an indication of a depth map and / or a motion map, a coding function on the depth map and / or the motion map, etc. In this way, in one embodiment, the same parameters are used at both the encoder and decoder sides. Thus, for example, the encoder may transmit a specific parameter to the decoder so that the decoder can use the same specific parameter (explicit signaling). Conversely, if the decoder already has the specific parameter as well as other parameters, it may use signaling without transmitting, simply to allow the decoder to know and select the specific parameter (implicit signaling). By avoiding transmitting any actual function, bit savings are realized in various embodiments. It should be understood that signaling may be achieved in various ways. For example, in various embodiments, one or more syntax elements, flags, etc. are used to signal information to a corresponding decoder. The above relates to the verb form of the word "signal", which may also be used as a noun in this specification.

[0110] As will be apparent to one of ordinary skill in the art, implementations may generate a variety of signals formatted to carry information that may be, for example, stored or transmitted. Information may include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal may be formatted to carry a bit stream of the described embodiments. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using a radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The signal that the signal carries may be, for example, analog or digital information. The signal may be transmitted by a variety of different wired or wireless links, as is known. The signal may be stored in, accessed from, or received from a processor-readable medium.

[0111] Many examples are described herein. Features of the examples may be provided alone or in any combination across various claim categories and types. Furthermore, examples may include one or more of the features, devices, or aspects described herein, alone or in any combination across various claim categories and types. For example, features described herein may be implemented in a bitstream or signal including information generated as described herein. The information may enable a decoder to decode the bitstream, encoder, bitstream, and / or decoder according to any of the described embodiments. For example, features described herein may be implemented by generating and / or transmitting and / or receiving and / or decoding a bitstream or signal. For example, features described herein may be implemented as a method, a process, an apparatus, a medium (e.g., a computer-readable medium) that stores instructions, a medium that stores data, or a signal. For example, features described herein may be implemented by a TV, a set-top box, a mobile phone, a tablet, or other electronic device that performs decoding. A TV, set-top box, mobile phone, tablet, or other electronic device may display (e.g., using a monitor, screen, or other type of display) the resulting image (e.g., an image from the residual reconstruction of the video bitstream). A TV, set-top box, mobile phone, tablet, or other electronic device may receive a signal that includes the encoded image and perform decoding.

[0112] A device may obtain an adaptive motion vector resolution (AMVR) parameter. The AMVR parameter may be associated with at least one of a depth map of the video block, a motion map of the video block, or an average motion vector predictor size. The device may process the video block based on the obtained AMVR parameter associated with at least one of the depth map of the video block, the motion map of the video block, or the average motion vector predictor size. The device may receive an indication of the AMVR parameter, e.g., at a slice level via the bitstream. The device may obtain the AMVR parameter according to the indication of the AMVR parameter. The obtained AMVR parameter may include an AMVR index. The device may select an AMVR index from multiple AMVR indexes based on at least one of the depth map of the video block, the motion map of the video block, or the average motion vector predictor size. The obtained AMVR parameter may include an AMVR set. The device may receive multiple AMVR indexes via the bitstream, e.g., at a picture level (e.g., in a picture header) or a slice level (e.g., in a slice header).

[0113] The device may select an accuracy level of the MVD based on a set of accuracy levels. The accuracy level of the MVD may be determined based on the AMVR index. In an embodiment, the device may select an AMVR set from the multiple AMVR sets based on at least one of a depth map of the video block, a motion map of the video block, or an average motion vector predictor size. The device may receive an indication of AMVR enablement at a picture level or a slice level, e.g., via a bitstream, and the device may process the video block using the indication of AMVR enablement. The device may receive multiple AMVR sets at a picture level (e.g., in a picture header) or a slice level (e.g., in a slice header) via a bitstream, for example.

[0114] The device may obtain an AMVR parameter based on at least one of a depth map of the video block, a motion map of the video block, or an average motion vector predictor size. The device may send the obtained AMVR parameter or an indication of the obtained AMVR parameter. The device may send the obtained AMVR parameter or an indication of the obtained AMVR parameter to a device, e.g., in a bitstream. The device may send an indication of the AMVR parameter (e.g., multiple AMVR indices or multiple AMVR sets) at a slice level. The device may determine whether to enable AMVR for a video block and send an indication of whether to enable AMVR for a video block in a bitstream, e.g., at a picture level or slice level. The AMVR parameter may include an AMVR index. The device may select an AMVR index from multiple AMVR indices based on at least one of a depth map of the video block, a motion map of the video block, or an average motion vector predictor size. The AMVR parameter may include an AMVR set. The device may select an AMVR set from the multiple AMVR sets based on at least one of a depth map of the video block, a motion map of the video block, or an average motion vector predictor size.

[0115] A system, method and means for determining an accuracy level of a motion vector based on an indication of motion of a video block is disclosed.

[0116] The device may obtain an indication of motion associated with the video block. The device may determine an accuracy level of motion vector information (e.g., motion vector differential (MVD)) associated with the video block based on the motion indication associated with the video block. The device may use the accuracy level to decode the motion vector information associated with the video block, e.g., if the device is a decoder. The device may use the accuracy level to encode the motion vector information associated with the video block, e.g., if the device is an encoder.

[0117] The device may receive motion vector information in video data signaled to the device, for example if the device is a decoder. The video data may not include an accuracy level of the motion vector information associated with the video block. The device may determine the accuracy level based on an indication of motion associated with the video block.

[0118] The device may determine an adaptive motion vector resolution (AMVR) index for the video block based on a motion indication associated with the video block. AMVR may be enabled for the video block. The device may implement AMVR by determining an AMVR index for the video block based on the motion indication associated with the video block. The AMVR index may indicate a precision level of an MVD associated with the video block, and the precision level of the MVD may be determined based on the AMVR index.

[0119] The indication of motion may include one or more of motion information, depth information, or a motion vector predictor (MVP) value. The MVP associated with a video block may indicate motion associated with neighboring blocks of the video block. The motion information associated with a video block may indicate a motion intensity associated with the video block. The depth information associated with a video block may indicate a distance associated with the video block from a reference point (e.g., a distance from a virtual camera).

[0120] In an embodiment, a device may receive an MVD associated with a video block in video data signaled to the device, the video data may indicate a set of accuracy levels, and the device may determine an accuracy level of the MVD based on the set of accuracy levels.

[0121] The device may modify or reorder the AMVR set based on a motion indication associated with the video block. The motion indication may include one or more of motion information associated with the video block, depth information associated with the video block, or a motion vector predictor (MVP) value associated with the video block.

[0122] The device may determine the AMVR index from a set of AMVR indexes for the video block. The device may implement AMVR by obtaining a set of AMVR indexes and selecting an AMVR index for the video block from the set of AMVR indexes based on a motion indication associated with the video block.

[0123] The device may select an AMVR set for the video block based on a motion indication associated with the video block. The device may implement AMVR by obtaining multiple AMVR sets and selecting an AMVR set for the video block from the multiple AMVR sets based on a motion indication associated with the video block. An AMVR index of the selected AMVR set may indicate an accuracy level of the MVD associated with the video block.

[0124] Content may be sent, received, or coded (e.g., encoded and / or decoded) if at least one of depth information, motion information, and / or average motion vector predictor size is available as information along with texture information (e.g., information consisting of luminance and / or chroma information). Compression efficiency may be improved. Bit rate may be reduced while maintaining quality. Quality may be improved while maintaining bit rate. Depth information, motion information, and / or average motion vector predictor size may be coded for (e.g., all) blocks of a picture or for (e.g., only for) a portion of the blocks of a picture. When depth information, motion information, and / or average motion vector predictor size are coded within an area of ​​a picture, they may be used to adaptively determine the accuracy level (e.g., motion vector resolution) of the motion vector for the block. In some embodiments, depth information, motion information, and / or average motion vector predictor size may be used to adapt a list of allowed motion vector resolutions to the test (e.g., candidate motion vector resolutions).

[0125] Joint coding of luma and depth may be implemented, for example, in "multi-view plus depth" (MVD) versions of 3D video processing. A depth map (e.g., information representing the fundamental geometry of a captured video scene) may be available for (e.g., each) texture picture of the video content, for example as a dense monochrome picture of the same resolution as the luma picture.

[0126] A video coding system, such as a cloud gaming server or a device with light detection and ranging (LiDAR) capabilities, may receive an input video frame (e.g., a texture frame) along with depth information (e.g., a depth map) and / or motion information that may be correlated. FIG. 5 illustrates an example texture frame 502 of a video game with a corresponding depth map 504, horizontal motion data 506, and vertical motion data 508 that may be extracted (e.g., directly) from a game engine rendering the game scene. The depth map 504 may be represented by a gray-level image indicating the distance between the camera and real-world objects. The depth map 504 may represent the basic geometry of the captured video scene. The depth map 504 may correspond to a texture picture of the video content and may include a dense monochrome picture with the same resolution as the luma picture. The depth map 504 may have the same resolution as the luma picture or a different resolution.

[0127] FIG. 6 illustrates an example architecture of a cloud gaming system in which a game engine may run on a cloud server. The game system may render game scenes based on player actions. The rendered game scenes may be encoded into a bitstream, for example, using a video encoder. The bitstream may be encapsulated by a transport protocol and sent as a transport stream to a player's device. The player's device may decapsulate and decode the transport stream and present the decoded game scenes to the player.

[0128] As shown in FIG. 6, additional information including one or more of a depth map, motion information, object ID, occlusion mask, camera parameters, etc. may be obtained from the game engine (e.g., as an output of the game engine) and made available to the server device (e.g., an encoder of the server device) as prior information.

[0129] Inter-predicted blocks may be enhanced by improving the representation of motion vectors, for example, by using adaptive motion vector resolution (AMVR).

[0130] A block-based AMVR scheme may be used for video processing. A motion vector differential (MVD) between a first motion vector (e.g., a motion vector of a CU) and a second motion vector (e.g., a predicted motion vector of a CU) may be signaled in units of quarter-luminance samples when an indication (e.g., use_integer_mv_flag) is equal to a certain value (e.g., zero) in a slice header. The value may indicate whether the MVD is signaled. The value may indicate whether the MVD is signaled with integer-sample accuracy. An AMVR (e.g., a CU-level AMVR) may be used to process (e.g., encode and / or decode) a CU. Using the AMVR may enable one or more MVDs of a CU to be coded with different accuracy. Depending on the mode (e.g., normal AMVP mode or affine AMVP mode) of the current CU, the MVD of the current CU and / or one or more accuracy levels of the MVD may be adaptively selected. For example, a first MVD accuracy (e.g., ¼ luma samples, half luma samples, integer luma samples, or 4 luma samples) may be selected for the current CU in normal AMVP mode, and a second MVD accuracy (e.g., ¼ luma samples, integer luma samples, or 1 / 16 luma samples) may be selected for the current CU in affine AMVP mode.

[0131] In some embodiments, a CU-level MVD resolution indicator may be signaled, for example, if the current CU has one or more non-zero MVD components. If the MVD components (e.g., horizontal and vertical MVD for reference lists L0 and L1) are zero, the MVD accuracy (e.g., MVD resolution) may be determined (e.g., inferred) to be, for example, ¼ luma sample.

[0132] For a CU with one or more non-zero MVD components, an indicator (e.g., a first flag) may be signaled to indicate whether the MVD accuracy of the quarter-luminance samples is used for the CU. If the indicator (e.g., the first flag) has a value (e.g., 0) indicating that the MVD accuracy of the quarter-luminance samples is used for the CU, the MVD accuracy of the quarter-luminance samples may be used for the CU and further signaling may be skipped. If the indicator (e.g., the first flag) has a value (e.g., 1) indicating that the MVD accuracy of the quarter-luminance samples is not used for the CU, an indicator (e.g., a second flag) may be signaled to indicate whether the MVD accuracy of the half-luminance samples is used for the AMVP CU. If the indicator (e.g., the second flag) indicates that the MVD accuracy of the half-luminance samples is used for the AMVP CU, a 6-tap interpolation filter may be used for the half-luminance sample positions (e.g., instead of an 8-tap interpolation filter, which may be the default). If an indicator (e.g., a second flag) indicates that half-luminance sample MVD accuracy is not used for the AMVP CU, then an indicator (e.g., a third flag) may be signaled to indicate whether 4-luminance sample MVD accuracy or integer-luminance sample MVD accuracy is used for the AMVP CU.

[0133] For an affine AMVP CU, an indicator (e.g., a different flag) may be used to indicate whether the MVD accuracy of integer luma samples or the MVD accuracy of 1 / 16 luma samples is used. One or more motion vector predictors for the CU may be rounded to the same accuracy as that of the MVD (e.g., before being added to the MVD), e.g., so that the reconstructed MV has the intended accuracy (e.g., 1 / 4 luma samples, half luma samples, integer luma samples, or 4 luma samples). The motion vector predictors may be rounded towards zero (e.g., negative motion vectors may be rounded towards positive infinity and positive motion vector predictors may be rounded towards negative infinity).

[0134] The video processing device may determine a motion vector accuracy (e.g., motion vector resolution) for a current CU, for example, using an RD check. The video processing device may include an encoder. The RD check for one or more MVD accuracies (e.g., other than the 1 / 4 luma sample MVD accuracy) may be skipped in some embodiments and may be activated (e.g., conditionally) in other embodiments. In the case of a normal AMVP mode, an RD cost for the 1 / 4 luma sample MVD accuracy and / or the MV accuracy of the integer luma sample may be obtained (e.g., calculated). The RD cost of the integer luma sample MVD accuracy may be compared to the RD cost of the 1 / 4 luma sample MVD accuracy, for example, to determine whether to further check the RD cost of the 4 luma sample MVD accuracy. If the RD cost of the 1 / 4 luma sample MVD accuracy is smaller (e.g., by a certain value) than the RD cost of the MVD accuracy of the integer luma sample, the RD check of the 4 luma sample MVD accuracy may be skipped. For example, RD testing of the MVD accuracy of the 4-luminance samples may be skipped if the ratio of the RD cost of the MVD prediction of the mid-luminance samples to the RD cost of the MVD accuracy of the quarter-luminance samples is in the range of about 1.04 to 1.1 (e.g., 1.06). Testing of the MVD accuracy of the half-luminance samples may be skipped if the RD cost of the MVD accuracy of the integer luminance samples is greater (e.g., significantly) than the best RD cost of the previously tested MVD accuracy. For example, testing of the MVD accuracy of the half-luminance samples may be skipped if the ratio of the RD cost of the integer luminance samples to the best RD cost is in the range of about 1.2 to 1.3 (e.g., 1.25). For affine AMVP mode, if affine inter mode is not selected after examining the rate-distortion cost of affine merge / skip mode, merge / skip mode, 1 / 4 luma sample MVD precision AMVP mode and / or 1 / 4 luma sample MVD precision affine AMVP mode, then 1 / 16 luma sample MV precision and / or 1 pixel MV precision affine inter mode may not be examined.One or more affine parameters obtained in the 1 / 4 luma sampled MV precision affine inter mode may be used as a starting search point for the 1 / 16 luma sampled and / or 1 / 4 luma sampled MV precision affine inter mode.

[0135] Increasing the resolution of the motion vectors may improve prediction accuracy and / or reduce motion vector differential signaling costs. Higher resolution motion vectors may require more bits to encode and / or decode. In some embodiments, a tradeoff between prediction accuracy and motion vector resolution may be made.

[0136] In some embodiments, integer pixel motion vector resolution may be employed. Half pixel motion vector resolution may be used in some embodiments. Quarter pixel motion vector resolution may be employed to achieve better compression gain, for example, compared to integer pixel motion vector resolution and / or half pixel motion vector resolution. 1 / 8 pixel resolution may be used. In some embodiments, increasing the resolution to 1 / 8 pixel may provide little coding gain due to increased signaling cost of the higher resolution MVD.

[0137] In embodiments, not all blocks in a video sequence require a higher resolution to represent motion vectors. The prediction error may be linearly correlated to the texture complexity in the scene in some embodiments. For the same level of mismatch between the original and predicted blocks, in some embodiments, the sum of absolute difference (SAD) score for the complex textured block may be higher compared to the SAD score for the simple textured block.

[0138] AMVR may be used, for example, to reduce MVD signaling costs. AMVR may be applied to (e.g., only) normal AMVP and / or affine AMVP schemes. Multiple modes (e.g., normal AMVP and normal affine AMVP) may require signaling of MVD, which is obtained after subtracting the motion vector (MV) from the motion vector predictor (MVP), as shown in Equations 1.1 and 1.2.

[0139]

number

[0140] Half-pixel, one-pixel, and four-pixel precision may be allowed at the CU level (e.g., instead of allowing only quarter-pixel precision for signaling MVD). The precision level of the MVD (e.g., MVD precision) may be adaptively selected per CU, and the index may be signaled in the bitstream. The MVD may be encoded and / or decoded using the MVD precision. If the MVD is 0, the AMVR index may not be signaled, and the MV resolution may be inferred as quarter-pixel. During the construction of the predictor (e.g., for AMVP), the predictor may be rounded to the same precision as the MVD. Table 1 shows an example of AMVR index signaling. In one or more embodiments herein, the MVD precision and the MVD resolution may be used interchangeably. Table 1 shows an example of AMVR index signaling.

[0141] [Table 1]

[0142] AMVR modes with different accuracy can be tested in the RD scheme with some encoder speedup in AMVP mode and affine AMVP mode. The tested resolutions are illustrated in Table 2. For example, in affine AMVP, MV accuracy of 1 / 16 pixel can be tested instead of 1 / 2 pixel and 4 pixel.

[0143] The AMVR scheme can be extended to an intra block copy (IBC) scheme in which the MV is replaced by a displacement vector. For IBC, a displacement vector resolution of 1 pixel or 4 pixels can be tested. Table 2 shows the MV accuracy used to code vectors in different modes.

[0144] [Table 2]

[0145] A game scene may include a background scene and some foreground objects, which may be further classified as moving objects or stationary objects. For example, a background scene is usually static and may be subjectively less important. The background may include heavy textures (e.g., rocks, mountains, grass, etc.). Heavy textures may be difficult to encode and / or decode. Figure 7 illustrates an example game scene with a foreground and a background.

[0146] Adaptive motion resolution (e.g., frame-level adaptation of resolution, block-level adaptation, etc.) may be used to encode / decode game scenes. In some embodiments, frame-level adaptation of resolution may be less expensive in terms of signaling, but may or may not adapt to local spatiotemporal characteristic variations within a frame.

[0147] A progressive MV resolution (PMVR) scheme may be used, where finer resolution may be adopted near the MVP and coarser resolution may be used as the MV search moves away from the MVP. In an embodiment, the MV resolution may be progressively reduced as the MV moves away from the MV predictor based on a pre-determined threshold. FIG. 8 illustrates an exemplary PMVR scheme. In FIG. 8, a 1 / 4 pixel threshold THe=2 and a 1 / 8 pixel threshold THq=4 may define the MVP region. In FIG. 8, region A refers to 1 / 8 pixels, region B refers to 1 / 4 pixels, and region C refers to 1 / 2 pixels.

[0148] In an exemplary PMVR scheme, the threshold (e.g., the one used in the original PMVR) may be adapted based on the PU size. For example, a larger PU size may use a finer resolution and a smaller PU size may use a coarser resolution. The threshold may further be adapted based on the spatial complexity of the PU. For example, a finer MV resolution may be used for a PU with a higher complexity and a coarser MV resolution may be used for a PU with a lower complexity. In some embodiments, the current PU may not be causal and / or may not be available at the decoder side. This complexity may be derived, for example, by computing gradients on a block obtained from a reference frame.

[0149] In an embodiment, the PMVR scheme may provide good coding gain. In some embodiments, the PMVR scheme may lead to higher encoder and decoder complexity. For example, the PMVR scheme may be used to achieve a coding gain of 1.2% (e.g., on top of HEVC) with an additional 17% complexity at the encoder and decoder sides. In an embodiment, much of the complexity may come from the computation of gradients using Sobel filters. In some embodiments, the motion estimation algorithm may be improved by performing motion search along MVPs (e.g., both of the two MVPs available in AMVP mode), which may further lead to increased complexity with marginal coding gain.

[0150] In some approaches, the distance or depth of an object in a scene from the camera may be taken into account, for example, based on motion parallax. Based on motion parallax, for the same degree of camera movement, an object located far from the camera may move less compared to an object located close to the camera. In an embodiment, a finer precision may be used for areas with slow movement. A coarser precision may be used for areas with fast movement.

[0151] In an embodiment, motion information (e.g., motion data from a game engine) may be used to determine a level of activity between different frames and / or between different regions (e.g., CU or CTU) within a given frame. Based on the level of activity, a motion vector resolution may be adaptively determined (e.g., adaptively chosen). For higher levels of activity, a lower resolution may be used (e.g., a higher level of activity may require a lower resolution). For lower levels of activity, a higher resolution may be used (e.g., a lower level of activity may require a higher resolution). In an embodiment, quality loss around areas where fast motion is occurring may be less perceptible than quality loss around areas where less motion is occurring. In regions with high motion, the predictor may be less accurate on average (e.g., this may cause the MVP and MV to differ), e.g., a coarser resolution (e.g., a coarser resolution than that used by the PMVR) may be used. In regions with slow or no motion (eg, zero motion), the MVP and MV may be close to each other and, for example, a finer resolution may be used.

[0152] When depth information (e.g., a depth map) and / or motion information (e.g., as shown in FIG. 5) are available, the depth information and / or motion information may be used, for example, by an encoder or by a decoder to determine the MV resolution. The information (e.g., information signals such as depth information and / or motion information) may indicate or provide the distance of a block from a camera and / or the motion characteristics of an object. The information may be employed, for example, for encoder improvement and / or coding gain improvement.

[0153] Encoder and / or decoder improvements may be achieved using depth information (eg, a depth map) and / or motion information.

[0154] In some embodiments, the depth information may be used to switch (e.g., adaptively switch) from a finer resolution to a coarser resolution, for example, using a threshold-based method. The AMVR index may be signaled in one or more of these embodiments. The encoder run time may be reduced by having fewer RD checks performed to obtain a particular AMVR index (e.g., the best AMVR index).

[0155] In some embodiments, the motion information may be used to select (e.g., adaptively select) an MV resolution based on a threshold. The resolution may be adapted to a level of activity in (e.g., each) CU. In embodiments, an accuracy level of the motion vector information may be determined based on the motion information. The motion information may, for example, indicate a level of activity associated with a video block.

[0156] In some implementations, motion information obtained from a game engine may be used to improve the PMVR scheme: the run-time of the encoder may be reduced, for example, by skipping computationally intensive MV searches (e.g., associated with motion estimation).

[0157] In one or more embodiments herein, coding gain improvements (eg, at the decoder side) may be achieved using depth maps and / or motion information.

[0158] In an embodiment, signaling information related to the AMVR (e.g., an AMVR index) may be skipped, and the information related to the AMVR may be inferred at the decoder side. For example, the decoder may analyze the (e.g., signaled) depth information and / or motion information to infer an AMVR index for (e.g., each) block (e.g., a CU). The AMVR index may indicate an accuracy level of the motion vector information (e.g., MVD). The motion vector information may be coded and / or decoded using the accuracy level.

[0159] In some embodiments, the depth information and / or the motion information may be used to sort the AMVR index table. Signaling may be improved.

[0160] In one or more embodiments herein, depth information (e.g., a depth map) and / or motion information (e.g., a motion map) may be used to improve the AMVR scheme. MV resolution may be inferred based on the depth information and / or motion information, which may be used to accelerate encoding time and / or increase coding gain, for example, by reducing signaling overhead.

[0161] Determining an accuracy level (e.g., an AMVR index, such as a best AMVR index) by using one or more of the depth map, motion map, and / or MVP (e.g., MVP size), e.g., to speed up video processing (e.g., by the encoder or only) and / or to reduce signaling; and determining an accuracy level (e.g., an AMVR index, such as a best AMVR index) by using one or more of the depth map, motion map, and / or MVP (e.g., MVP size), e.g., to speed up video processing (e.g., by the encoder) and / or to reduce signaling costs (e.g., by pruning the AMVR set per CU), and / or to reduce signaling (e.g., by reordering the AMVR set per CU). Methods, processes, apparatus, medium storing instructions, medium storing data, or signals may be described herein, including but not limited to, one or more of: determining a set of accuracy levels (e.g., an AMVR set, such as a best AMVR set) by using one or more of a depth map and / or an MVP (e.g., an MVP size); signaling the AMVR set at the slice level (e.g., based on a depth map and / or a motion map); signaling an AMVR enable flag in a slice header and / or picture header (e.g., based on a depth map and / or a motion map); and speeding up the PMVR scheme using a motion map.

[0162] The depth information (e.g., a depth map) may indicate the distance of an object from the camera. The motion information may be used to determine the motion characteristics of a given object or region in a scene. Using the depth information and / or motion information may lead to easier detection of the best accuracy of the motion vector information (e.g., the accuracy level of the MV) for a given coding unit (CU), for example, to perform MV search and / or encoding of the MVD.

[0163] The depth information may include a depth map, which may be represented by a gray-level image indicating the distance between the camera and the real object. The depth map may represent the basic geometry of the captured video scene. The depth map may correspond to a texture picture of the video content and may include a dense monochrome picture with the same resolution as the luma picture. The depth map may have the same or a different resolution as the luma picture. The depth information may be of a limited scope, such as foreground, midground, and background. This information may be made available on the client (e.g., decoder) side, for example, if the information is encoded in the bitstream, or if additional information (e.g., flags, etc.) is derived and coded together with the picture (e.g., block-wise). The depth information may be inferred from the motion information (e.g., if not otherwise available), for example, if the intrinsic and / or extrinsic parameters of the camera are known. This inference may be performed, for example, based on epipolar geometry. The motion information may be inferred from previously reconstructed blocks in the current frame or from previously reconstructed (e.g., reference) frames (e.g., using temporal motion vector prediction or decoder-side motion estimation). In the examples provided herein, it is assumed that additional information about the picture, such as picture depth and / or motion information, may be available at the picture decoding stage, or reduced information may be derived block by block.

[0164] The MVP list may be constructed canonically, for example. The MVP list may be constructed from neighboring blocks of the same frame (e.g., the current frame including the current block) or from co-located blocks in a reference frame. In some embodiments, MVs (e.g., previously used MVs) may be stored in a first in first out (FIFO) buffer (e.g., used as a history-based motion vector predictor (HMVP)) and / or may be used when constructing the MVP list. The MVPs may indicate the motion of neighboring blocks.

[0165] An MVP list may include multiple MVPs (e.g., for normal and affine AMVP modes, an MVP list may include two predictors). The average MVP size may be determined based on (e.g., some or all) of the MVPs in the MVP list, for example, using the absolute grades of the MVPs in the MVP list. As an example, for an MVP list with two predictors, the average MVP size may refer to the average absolute grade of both MVPs in the list, as shown in Equations 3.1-3.3, where N is the average absolute grade of both MVPs in the list when the MVP list includes two MVPs. P =2.

[0166]

number

[0167] Information related to MVPs (e.g., MVP size or average MVP size) may be available at the encoder side and / or decoder side. A device (e.g., a decoder) may construct an MVP list based on various normative rules. MVPs or MVP lists may be constructed for various inter-prediction modes. A device may use the MVP list to determine the average MVP size (MVP avg) and determine the accuracy level of the MV information (e.g., AMVR accuracy) based on the average MVP size. The device may determine the accuracy level of the MV information based on the variance between the MVPs (e.g., two or more MVPs in an MVP list) and / or the size of the base MV in the MVP list.

[0168] In some embodiments, a rate distortion (RD) search may be performed using possible MV resolutions (e.g., all MV resolution candidates). The resolution with the smallest RD cost (e.g., the best resolution) may be selected, and an index for the selected resolution may be signaled in the bitstream, e.g., as in Table 1. Features from depth information (e.g., depth maps) and / or motion information (e.g., motion maps) may be used to determine a particular AMVR index (e.g., directly determine the best AMVR index) and / or to determine a particular AMVR set (e.g., directly determine a new reordered or modified (e.g., pruned) AMVR set). Encoder speed and / or bitrate savings may be improved. The MV resolution may indicate the accuracy level of the MV information (e.g., MV, MVD, etc.).

[0169] The AMVR index (e.g., best AMVR index) may be determined, for example, based on at least one of depth information (e.g., depth map), motion information, and / or MVP (e.g., average MVP size). In one embodiment, the depth map, motion map, and / or average MVP size may be used to determine the best MV resolution for a given CU block. FIG. 9 illustrates one example 900 for determining an AMVR index (e.g., best AMVR index).

[0170] The motion indication may include depth information. The depth information may indicate, for example, a distance from a virtual camera. In an embodiment, the distance (e.g., average distance) from a reference point (e.g., a camera such as a virtual camera) may be determined based on, for example, one or more of a bit depth associated with the block, a number of pixels in the block, and a depth value at (e.g., each) position in the block. For example, in 902 of FIG. 9, a depth map corresponding to the current block may be used to calculate the average distance d using Equation 4.1.

[0171]

number

[0172] where bd is defined as the bit depth and N B is the number of pixels in block B, and D(x,y) is the depth value at location (x,y) in block B.

[0173] The motion indication may include motion information. The motion information may indicate a motion intensity. The motion intensity may be determined, for example, based on a motion map. In an embodiment, the motion intensity (e.g., average motion intensity) of a block may be determined, for example, based on one or more of the number of pixels in the block, the horizontal motion vector of the block (e.g., the average horizontal motion vector of the block), and the vertical motion vector of the block (e.g., the average vertical motion vector of the block). For example, in 904 of FIG. 9, the motion map corresponding to the current block may be used to calculate the average motion intensity MI using Equation 4.2.

[0174]

number

[0175] where N is the number of pixels in the block,

[0176]

number

[0177] The motion indication may include an MVP. The MVP may be a motion vector or a motion field associated with a block adjacent to the current block. The MVP may indicate a motion associated with a block adjacent to the current block. The motion associated with a block adjacent to the current block may indicate a motion associated with the current block. In an embodiment, the MVP (e.g., an average of the MVPs) may be used to determine an accuracy level of the MV information of the block. For example, in 906 of FIG. 9, the average size of the MVP (MVP size ) may be used as input features when determining the best MV resolution for a given block.

[0178] An indication of movement (e.g., as indicated by a level of activity) may be determined based on, for example, one or more of distance, movement intensity, or MVP. Respective weights may be given to distance, movement intensity, or MVP. For example, the above features may be combined using a weighted average, as shown in Equation 4.3.

[0179]

number

[0180] where one or more of the weights α, β, and γ may be determined (e.g., empirically) based on a training set (e.g., a large training set). Here, DMI is referred to as a depth motion activity index (e.g., DMI may be normalized to a value between 0 and 1). For example, an algorithm (e.g., a machine learning algorithm) may be used to determine one or more of the weights. In an embodiment, the training data for the algorithm may include a training set of N coding blocks with associated depth d, motion intensity MI, MVP list, and best AMVR index chosen by the encoder in the encoding stage. An algorithm may be used or configured that minimizes the cost of predicting the correct AMVR index by setting various values ​​of the weights. The output of this algorithm may be a set of optimized weights.

[0181] The indication of motion may be compared to one or more thresholds. For example, in 908 of FIG. 9, the DMI value may be compared to a set of thresholds representing the best MV resolution. The thresholds may be calculated (e.g., empirically) from training data (e.g., a large set of training data). For example, an algorithm (e.g., a machine learning algorithm) may be used to determine one or more of the thresholds. For example, the training data for the algorithm may include a training set of N coding blocks with associated depth d, motion intensity MI, MVP list, and the best AMVR index chosen by the encoder in the encoding stage. An algorithm may be used or configured that minimizes the cost of predicting the correct AMVR index by setting various values ​​of the thresholds. The output of this algorithm may be a set of optimized thresholds. In another embodiment, an algorithm that jointly optimizes the weights α, β, and γ and the set of thresholds may be used or configured.

[0182] The device can modify or reorder a set of accuracy levels based on, for example, one or more of depth information, motion information, or MVP. In some embodiments, the device can use a threshold to modify the set of accuracy levels. In an exemplary case, if DMI < T, where T is the threshold, the AMVR set can be modified as follows. The modified AMVR set A1 = {1 / 4, 1 / 2} pixels. If DMI > T, the AMVR set can be modified as follows. The modified AMVR set A2 = {1, 4} pixels.

[0183] Figure 10 illustrates an example 1000 for determining an AMVR set (e.g., the best AMVR set). One or more exemplary techniques herein can be used to determine the AMVR set. In 1002 of Figure 10, the depth map corresponding to the current block can be used to calculate the average distance d, for example, using Equation 4.1. In 1004 of Figure 10, the motion map corresponding to the current block can be used to calculate the average motion intensity MI, for example, using Equation 4.2. In 1006 of Figure 10, the average size of the MVP (MVP size ) can be used as an input feature when determining the best MV resolution for a given block. The above features can be combined using a weighted average, for example, as shown in Equation 4.3. In 1008 of Figure 10, the DMI value can be compared with a set of thresholds representing the best MV resolution.

[0184] The motion indication can include depth information, motion information, or MVP. One or more embodiments herein can use the values of the weighting factors described herein. In some embodiments, only the motion map can be used to determine the best MV resolution (e.g., when α = 0 and γ = 0). In some embodiments, only the depth map can be used to determine the best MV resolution (e.g., when β = 0 and γ = 0). In some embodiments, only the MVP (e.g., from the MVP list) can be used to determine the MV resolution (e.g., when α = 0 and β = 0).

[0185] An AMVR set (e.g., a best AMVR set) may be determined. FIG. 10 illustrates an example for determining an AMVR set (e.g., a best AMVR set). As shown in FIG. 10, one or more of 1002-1010 may be used to determine the AMVR set (e.g., instead of a single index for a given coding unit (CU)). This AMVR set may be modified, for example, this AMVR set may be defined by a set of reduced motion vector resolutions adapted to a given block, or may be a reordered version of the originally used AMVR set adapted to a given block. An example of a reduced AMVR set and a reordered AMVR set may be shown in Tables 3 and 4, respectively.

[0186] [Table 3]

[0187] [Table 4]

[0188] Video processing (e.g., by an encoder) may be accelerated. The embodiment presented in FIG. 9 may be used to reduce RD search (e.g., encoder RD search) and / or reduce overall encoding time. In the embodiment shown in FIG. 11, depth information (e.g., depth map) and motion information (e.g., motion map) may be available to a device (e.g., encoder) along with blocks of texture information (e.g., texture blocks). The device may use the input depth information and / or motion information to determine a particular AMVR index (e.g., best AMVR index). The device may avoid testing all possible AMVR modes (e.g., four AMVR modes in a particular codec such as VVC). The device may use the AMVR index to encode or decode texture information and include the encoded or decoded information in the video data. The embodiment of FIG. 11 may enable speedup of an encoder, for example, using depth information, motion information, etc. Depth information (e.g., depth map) and / or motion information (e.g., motion map) may be available to the encoder and decoder. Signaling may be reduced.

[0189] FIG. 12 illustrates an example of AMVR index derivation at the transmitting side and the receiving side (e.g., based on one or more of a depth map, a motion map, an MVP, etc.). The embodiment shown in FIG. 12 may be implemented in a normative manner. The embodiment shown in FIG. 12 may be applicable to an encoder and a decoder. In the embodiment shown in FIG. 12, the transmitter may include an encoder. The transmitting side 1208 (e.g., the encoder side) may have depth information and / or motion information available. The receiver may include a decoder. The receiving side 1212 (e.g., the decoder side) may have depth information and / or motion information available. The transmitting side 1208 may receive texture information (e.g., an input texture). The texture information may be encoded and included in a bitstream. The receiving side 1212 may receive the bitstream and decode the texture information to generate a decoded texture. One or more of the depth information (e.g., depth map), motion information (e.g., motion map), or values ​​determined from the MVP (e.g., MVP list) may be used at the transmitting side 1208 to infer the best MV resolution index, e.g., if the depth map or motion map is available to both the encoder and the decoder. The depth information (e.g., depth map) or motion information (e.g., motion map) may be used at the receiving side 1212 to infer the best MV resolution index, e.g., if the depth map or motion map is available to both the encoder and the decoder. In an embodiment, the encoder and the decoder may perform the same process to find the best MV resolution for a given block. The AMVR index may not need to be signaled to the decoder, for example. The device (e.g., decoder) may infer the AMVR index based on the depth map and / or motion map analysis (e.g., in the same manner as the embodiment shown in FIG. 9). Signaling overhead may be reduced, thus providing bitrate savings.

[0190] The AMVR set can be adapted or used flexibly. FIG. 13 illustrates an example for deriving a certain AMVR set (e.g., best AMVR set) at the transmitting side and the receiving side (e.g., based on one or more of a depth map, a motion map, an MVP, etc.). The embodiment shown in FIG. 13 may be applicable to an encoder and a decoder. In the embodiment shown in FIG. 13, the transmitter may include an encoder. The transmitting side 1308 (e.g., the encoder side) may have depth information and / or motion information available. The receiver may include a decoder. The receiving side 1312 (e.g., the decoder side) may have depth information and / or motion information available. The transmitting side 1308 may receive texture information (e.g., input texture). The texture information may be encoded and included in the bitstream. The receiving side 1312 may receive the bitstream and decode the texture information to generate a decoded texture. The embodiment of FIG. 13 may be considered as an extension of the AMVR mode in which different AMVR sets are allowed for different CUs (e.g., AMVR sets beyond the normal set shown in Table 2). The best AMVR set may be chosen, for example, from among a set of dictionaries already defined at the encoder side and / or the decoder side, depending on one or more of depth information (e.g., depth map), motion information, or MVP (e.g., average motion vector predictor size). In FIG. 13 (similar to the embodiment shown in FIG. 10), depth information and / or motion information may be available at the encoder side and / or the decoder side. In the embodiment, the same process of finding the best AMVR set may be implemented at the encoder side and the decoder side. An index for determining the best AMVR set may not need to be coded (e.g., encoded or decoded). The AMVR index may be signaled and / or determined using RD search (e.g., the RD search method adopted in VVC).

[0191] In some implementations where the AMVR set is a pruned version of the original AMVR set, the implementation shown in FIG. 13 may help speed up the encoder while reducing the overall indexing cost.

[0192] In some embodiments where the AMVR set is a reordered version of the original AMVR set, the embodiment shown in Figure 13 may lead to bitrate savings, for example because when reordered, the most likely AMVR index is placed first. Using variable length coding to signal the AMVR may lead to cost savings.

[0193] A set of accuracy levels (e.g., an AMVR set) may be signaled at the slice level (e.g., in a slice header). For example, a syntax may be used to signal an AMVR set at the slice level. The AMVR set may be determined using the embodiment shown in FIG. 10, which utilizes, for example, one or more of depth information (e.g., a depth map), motion information, and / or MVPs (e.g., an MVP size such as an average MVP size). In some embodiments, the AMVR set may be determined using previously decoded frames or may be calculated offline using a training set (e.g., a large training set). The AMVR set may be determined using an algorithm (e.g., a machine learning algorithm). Once the AMVR set is determined, in one embodiment (e.g., Table 5), an index to the chosen AMVR set may be signaled to a device (e.g., a decoder). The device may be configured with (e.g., knows about) a predefined dictionary of AMVR sets. In another embodiment (e.g., Table 6), one or more AMVR sets may be signaled (e.g., explicitly) in the slice header. The device may, for example, decode the AMVR set from the bitstream.

[0194] Table 5 shows an example syntax by which an index into a pre-defined dictionary of an AMVR set (eg, a dictionary that exists on both the encoder and decoder sides) may be signaled.

[0195] [Table 5] Table 6 shows an example syntax where the AMVR set may be adaptively determined at the encoder side (eg, the AMVR set may not be previously known to the decoder).

[0196] [Table 6]

[0197] An AMVR enablement indication (e.g., an AMVR enable flag) may be signaled at the slice level. As shown in the embodiments herein, an AMVR set may be signaled at the slice level. In these embodiments, an indication (e.g., a flag) may be coded at the slice level to enable or disable the AMVR tools. In some embodiments, an AMVR enablement indication (e.g., an AMVR tools enable / disable flag) may be added at the picture level.

[0198] Depth information (e.g., depth features) and / or motion information (e.g., motion features) may be used when determining the indication. For example, AMVR tools may be disabled for frames with global motion above a certain value (e.g., a certain threshold). The use of this indication is not limited to the availability of depth information (e.g., depth data) and / or motion information (e.g., motion data). This indication may be used when encoding and / or decoding mixed content, where a frame or set of frames (e.g., a sequence) may include screen content, game content, and / or natural content. In case of screen content (e.g., when MV sub-pixel precision does not achieve gain), this indication may be disabled. Table 7 shows an example syntax in a picture header. Table 8 shows an example syntax in a slice header.

[0199] [Table 7]

[0200] [Table 8]

[0201] Gradual AMVR encoder acceleration may be achieved in one or more embodiments of the present disclosure. actual refers to the actual MV of the block derived using the block motion estimation algorithm (e.g., diamond search) employed in the encoder. MV predicted refers to one or more MVPs obtained from adjacent blocks in the current frame or from co-located blocks in the previous frame. MV inferred can be derived from the block's motion field (MF) obtained from motion information (e.g., input motion data, typically from a game engine) as shown in equations 4.4, 4.5, and 4.6.

[0202]

number

[0203] This embodiment using equations 4.4, 4.5, and 4.6 can be applied to devices (e.g., only the encoder side) to speed up, for example, PMVR methods. MV actual The computationally intensive process of deriving M may be skipped. The actual motion may be replaced by motion information inferred from the input motion map. Then, the predictor (e.g., each predictor) M predicted This is the speculated movement MV inferred It can be directly compared to MV predicted and M.V. inferred If MV is close (e.g., based on a decision to use a certain value), a finer MV resolution may be chosen. predicted and M.V. inferred If ,are not close, a coarser MV resolution may be chosen.,In an embodiment, the value (threshold) used for the comparison may be,empirically derived.

[0204] In some embodiments, early termination may be performed based on an occlusion map. PMVR speedup (in one or more embodiments herein) may be terminated when occlusion is detected in the current block. The encoder may have access to additional information about occlusion through the occlusion map.

[0205] In some embodiments, early termination may be performed based on motion block variance. Early termination may be based on the variance of the motion field (MF) within the CU. The variance may be calculated in the horizontal and / or vertical directions. The sum of the variances may be calculated and / or may be scaled to a value (e.g., a predefined threshold TH mf ) When the variance is higher than that value, PMVR speedup (in one or more embodiments herein) may be terminated (e.g., may be terminated).

[0206] A device may implement the method 1400 as described in FIG. 14. FIG. 14 illustrates one example of a method for obtaining an AMVR parameter associated with at least one of a depth map of a video block, a motion map of the video block, or an average motion vector predictor size. For example, the device may obtain an AMVR parameter associated with at least one of a depth map of a video block, a motion map of the video block, or an average motion vector predictor size, at 1404, and process the video block based on the obtained AMVR parameter, at 1406. The device may be a decoder (e.g., the decoder shown in FIG. 3) or an encoder (e.g., the encoder shown in FIG. 2).

[0207] A device may implement method 1500 as described in FIG. 15. FIG. 15 illustrates one example of a method for sending a derived AMVR parameter or an indication of the derived AMVR parameter. For example, the device may obtain an AMVR parameter based on at least one of a depth map of the video block, a motion map of the video block, or an average motion vector predictor size, at 1504, and send the derived AMVR parameter or an indication of the derived AMVR parameter, at 1506. The device may be a decoder (e.g., the decoder shown in FIG. 3) or an encoder (e.g., the encoder shown in FIG. 2).

[0208] 5 at a decoder. These decoding tools and techniques may include one or more of entropy decoding, inverse quantization, inverse transform, and differential decoding. These decoding tools and techniques may include obtaining an adaptive motion vector resolution (AMVR) parameter associated with at least one of a depth map of the video block, a motion map of the video block, or an average motion vector predictor size, processing the video block based on the obtained AMVR parameter associated with at least one of the depth map of the video block, the motion map of the video block, or an average motion vector predictor size, obtaining the AMVR parameter based on at least one of the depth map of the video block, the motion map of the video block, or the average motion vector predictor size, receiving an indication of the AMVR parameter, e.g., at a slice level via a bitstream, obtaining the AMVR parameter according to the indication of the AMVR parameter, and processing the video block based on the obtained AMVR parameter. The method includes selecting an AMVR index from a plurality of AMVR indices based on at least one of a depth map, a motion map of the video block, or an average motion vector predictor size; selecting an AMVR set from a plurality of AMVR sets based on at least one of a depth map, a motion map of the video block, or an average motion vector predictor size of the video block; receiving an indication of AMVR enablement, e.g., at a picture level or a slice level via a bitstream, and processing the video block using the indication of AMVR enablement; determining an accuracy level of motion vector information (e.g., MVD) associated with the video block based on an indication of motion associated with the video block; and decoding the motion vector information associated with the video block using the accuracy level.

[0209] Encoding tools and techniques including one or more of quantization, entropy coding, dequantization, inverse transform, and differential encoding may be used to enable the methods described herein in an encoder. These encoding tools and techniques may include obtaining an AMVR parameter based on at least one of a depth map of the video block, a motion map of the video block, or an average motion vector predictor size, sending the obtained AMVR parameter or an indication of the obtained AMVR parameter, sending the obtained AMVR parameter or an indication of the obtained AMVR parameter to a device, e.g., at a bitstream, sending an indication of the AMVR parameter at a slice level, determining whether to enable AMVR for the video block and sending an indication of whether to enable AMVR for the video block in the bitstream, e.g., at a picture level or slice level, The frame rate may be used to enable one or more of: selecting an AMVR index from a plurality of AMVR indices based on at least one of a depth map of the block, a motion map of the video block, or an average motion vector predictor size; selecting an AMVR set from a plurality of AMVR sets based on at least one of a depth map of the video block, a motion map of the video block, or an average motion vector predictor size; determining an accuracy level of motion vector information (e.g., MVD) associated with the video block based on an indication of motion associated with the video block; and encoding the motion vector information associated with the video block using the accuracy level.

[0210] The syntax elements may be inserted into the signaling, for example, to enable the decoder to identify an instruction associated with performing the method described in Figure 14 or Figure 15, or the method to be used. For example, the syntax elements may include one or more of sh_amvr_enabled_flag, ph_amvr_enabled_flag, sh_signal_amvr_set_flag, sh_amvr_set_idc, sh_num_amvr_set, or amvr_precision_set[i]. As an example, the decoder may decide whether to enable AMVR at the slice level based on the instruction sh_amvr_enabled_flag.

[0211] Although the examples provided herein may assume that media content is streamed to a display device, there is no particular limitation on the type of display device that can benefit from the exemplary techniques described herein. For example, the display device may be a television, a projector, a mobile phone, a tablet, etc. Furthermore, the exemplary techniques described herein may be applied to remote conferencing settings as well as streaming use cases. In addition, the decoder and display described herein may be separate devices or may be part of the same device. For example, a set-top box may decode an incoming video stream and (e.g., subsequently) provide the decoded stream to a display device (e.g., via HDMI), and may transmit information about viewing conditions, such as viewing distance, from the display device to the set-top box (e.g., via HDMI).

[0212] Although the features and elements are described above in certain combinations, one skilled in the art will understand that each feature or element may be used alone or in any combination with the other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware embodied in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted over wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random-access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs). A processor in association with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. 1. A device for decoding video content, comprising: a processor, the processor comprising: Obtaining a motion instruction associated with the video block; determining an accuracy level of a motion vector differential (MVD) associated with the video block based on the indication of motion associated with the video block; The device is configured to decode the MVD associated with the video block using the accuracy level.

2. 1. A device for encoding video content, comprising: a processor, the processor comprising: Obtaining a motion instruction associated with the video block; determining an accuracy level of a motion vector differential (MVD) associated with the video block based on the indication of motion associated with the video block; A device configured to encode the MVD associated with the video block using the accuracy level.

3. 2. The device of claim 1, wherein adaptive motion vector resolution (AMVR) is enabled for the video block, and the processor is configured to perform AMVR by determining an AMVR index for the video block based on the indication of motion associated with the video block, the AMVR index indicating the accuracy level of the MVD associated with the video block, and the accuracy level of the MVD is determined based on the AMVR index.

4. 2. The device of claim 1, wherein adaptive motion vector resolution (AMVR) is enabled for the video block, and the processor is configured to modify or reorder an AMVR set based on the indication of motion associated with the video block.

5. 5. The device of claim 1 , wherein the indication of motion associated with the video block includes a motion vector predictor (MVP) value, the MVP indicating motion associated with blocks neighboring the video block.

6. 5. The device of claim 1 , wherein the indication of motion associated with the video block includes motion information associated with the video block, the motion information associated with the video block indicating a motion intensity associated with the video block.

7. 5. A device as claimed in claim 1, wherein the indication of movement associated with the video block includes depth information associated with the video block, the depth information associated with the video block indicating a distance associated with the video block from a reference point associated with a virtual camera.

8. The processor, receiving the MVD associated with the video block in video data indicative of a set of accuracy levels signaled to the device; The device of claim 1 , configured to determine the accuracy level of the MVD associated with the video block based on the set of accuracy levels.

9. Adaptive motion vector resolution (AMVR) is enabled for the video block, and the processor: Obtain a set of AMVR indices; 2. The device of claim 1, configured to perform AMVR by selecting an AMVR index for the video block from the set of AMVR indexes based on the indication of motion associated with the video block, the AMVR index indicating the accuracy level of the MVD associated with the video block, and the accuracy level of the MVD determined based on the AMVR index.

10. Adaptive motion vector resolution (AMVR) is enabled for the video block, and the processor: Obtaining multiple AMVR sets; 2. The device of claim 1, configured to perform AMVR by selecting an AMVR set for the video block from the plurality of AMVR sets based on the indication of motion associated with the video block, an AMVR index of the selected AMVR set indicating the accuracy level of the MVD associated with the video block, and the accuracy level of the MVD determined based on the AMVR index.

11. 2. The device of claim 1, wherein the processor is configured to receive the MVD associated with the video block in video data signaled to the device, the video data not indicating the accuracy level of the MVD associated with the video block.

12. 1. A method for decoding video content, comprising: Obtaining a motion indication associated with the video block; determining an accuracy level of a motion vector differential (MVD) associated with the video block based on the indication of motion associated with the video block; and decoding the MVD associated with the video block using the accuracy level.

13. 1. A method for encoding video content, comprising: Obtaining a motion indication associated with the video block; determining an accuracy level of a motion vector differential (MVD) associated with the video block based on the indication of motion associated with the video block; and encoding the MVD associated with the video block using the accuracy level.

14. 13. The method of claim 12, wherein adaptive motion vector resolution (AMVR) is enabled for the video block, the method including performing AMVR by determining an AMVR index for the video block based on the indication of motion associated with the video block, the AMVR index indicating the level of accuracy of the MVD associated with the video block, the accuracy level of the MVD being determined based on the AMVR index.

15. 13. The method of claim 12, wherein adaptive motion vector resolution (AMVR) is enabled for the video block, the method comprising modifying or reordering an AMVR set based on the indication of motion associated with the video block.

16. 16. The method of claim 12, wherein the indication of motion associated with the video block includes a motion vector predictor (MVP) value, the MVP indicating motion associated with blocks neighboring the video block.

17. 16. The method of claim 12, wherein the indication of motion associated with the video block includes motion information associated with the video block, the motion information associated with the video block indicating a motion intensity associated with the video block.

18. 16. The method of claim 12, wherein the indication of movement associated with the video block includes depth information associated with the video block, the depth information associated with the video block indicating a distance associated with the video block from a reference point associated with a virtual camera.

19. A non-transitory computer readable medium comprising data content generated according to the method of any one of claims 12 to 18.

20. A computer readable medium comprising instructions for causing one or more processors to perform the method of any one of claims 12-18.

21. A computer program product comprising instructions for performing the method of any one of claims 12 to 18 when executed by one or more processors.

22. A signal generated based on the determined accuracy level according to the method of any one of claims 13 and 16 to 18.

23. An apparatus comprising: A device according to any one of claims 1 to 11, 1. An apparatus comprising: (i) an antenna configured to receive a signal, the signal including data representing an image; (ii) a band limiter configured to limit the received signal to a frequency band including the data representing the image; or (iii) a display configured to display the image.

24. A device according to any one of claims 1 to 11, comprising a TV, a mobile phone, a tablet or a set-top box (STB).

25. An apparatus comprising: an access unit configured to access data including residuals generated based on the determined accuracy level by the device according to any one of claims 2 and 5 to 7; a transmitter configured to transmit the data including the residual.

Images