Method for video decoding, method for video encoding, and apparatus therefor

Abstract

A method for video decoding, a method for video encoding, and an apparatus therefor. The apparatus may determine a direction in which directional BDOF refinement is performed. The apparatus may calculate a vertical direction gradient difference and a horizontal direction gradient difference for the CU. The vertical gradient difference may indicate a difference between a vertical gradient for a first reference picture and a vertical gradient for a second reference picture. The horizontal gradient difference may indicate a difference between a horizontal gradient for a first reference picture and a horizontal gradient for a second reference picture. The video coding device may determine a direction on which directional BDOF refinement is performed based on the vertical direction gradient difference and the horizontal direction gradient difference. The video coding device may perform directional BDOF refinement in the determined direction.

Description

[0001] This application is a divisional application of the Chinese invention patent application with application date of April 5, 2019, application number 201980034247.6, and invention name “Bidirectional optical flow method using simplified gradient derivation”. Technical Field

[0002] The invention relates to the technical field of video encoding and decoding, and mainly to a bidirectional optical flow method using simplified gradient derivation.

[0003] Cross-references

[0004] This application claims the benefit of U.S. Provisional Application No. 62 / 653,674, filed on April 6, 2018, the contents of which are incorporated herein by reference. Background Art

[0005] Video coding systems are widely used to compress digital video signals to reduce the storage requirements and / or transmission bandwidth of such signals. Among various types of video coding systems, such as block-based, wavelet-based, and object-based systems, block-based hybrid video coding systems are widely used and deployed. Examples of block-based video coding systems may include international video coding standards such as MPEG1 / 2 / 4 Part 2, H.264 / MPEG-4 Part 10, AVC, VC-1, and High Efficiency Video Coding (HEVC). Summary of the invention

[0006] A video decoding device may be configured to perform directional bidirectional optical flow (BDOF) refinement on a decoding unit (CU). The video decoding device may be or may include an encoder and / or a decoder. Video decoding may refer to encoding and / or decoding. The device may determine the direction on which the directional BDOF refinement is performed. The device may calculate a vertical gradient difference and a horizontal gradient difference for the CU. The vertical gradient difference may indicate the difference between the vertical gradient in the first reference picture and the vertical gradient for the second reference picture. The horizontal gradient difference may indicate the difference between the horizontal gradient in the first reference picture and the horizontal gradient for the second reference picture. The video decoding device may determine the direction on which the directional BDOF refinement is performed based on the vertical gradient difference and the horizontal gradient difference. The video decoding device may perform directional BDOF refinement in the determined direction. For example, directional BDOF refinement may include performing BDOF refinement in the vertical direction or performing directional BDOF refinement in the horizontal direction.

[0007] A video decoding device may be configured to determine a direction in which to perform directional BDOF refinement. As described herein, the direction in which the directional BDOF refinement is performed may be based on a vertical gradient difference associated with a CU and a horizontal gradient difference associated with the CU. The video decoding device may compare the gradient difference with a threshold (e.g., a first threshold and / or a second threshold). When the vertical gradient difference associated with the CU is greater than the first threshold, the video decoding device may perform directional BDOF refinement in the vertical direction. When the horizontal gradient difference associated with the CU is greater than the second threshold, the video decoding device may perform directional BDOF refinement in the horizontal direction. The first threshold and the second threshold may be variable, static (e.g., pre-configured), or semi-static. For example, the first threshold and the second threshold may be variable, and the first threshold may be or may include a horizontal gradient difference, and the second threshold may be or may include a vertical gradient difference.

[0008] A video decoding device may be configured to determine whether to perform BDOF refinement on a CU, which may include: performing directional BDOF refinement in the vertical direction, performing directional BDOF refinement in the horizontal direction, and performing directional BDOF refinement in the vertical and horizontal directions. The video decoding device may be configured to determine whether to perform BDOF refinement on the CU based on one or more characteristics associated with the CU. The video decoding device may determine whether to perform BDOF refinement based on an inter-coding mode associated with the CU and / or a size associated with the CU. For example, if the inter-coding mode associated with the CU supports sub-CU prediction, the video decoding device may determine to skip BDOF refinement.

[0009] A video decoding device may be configured to perform BDOF refinement on a CU. The video decoding device may identify a motion vector associated with a reference CU of the CU. The motion vector may include one or more motion components (e.g., a first motion component and a second motion component). The motion component may include an integer motion component or a non-integer (e.g., fractional) motion component. For example, the motion component may include a non-integer motion component. The device may calculate a directional gradient associated with the CU, for example, by applying a gradient filter to a reference sample at an integer position of the reference CU. For example, if the motion component includes a non-integer motion component, the video decoding device may identify an integer position corresponding to the non-integer motion component. The video decoding device may apply a gradient filter to the reference sample at the integer position of the reference CU in a first direction. For example, one or more reference samples (one or more) at integer positions associated with the reference CU may approximate their corresponding samples (one or more) at fractional positions in the reference CU. The directional gradient associated with the CU may be used to calculate the vertical gradient difference for the CU or the horizontal gradient difference associated with the CU. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1A is a system diagram illustrating an example communication system in which one or more disclosed embodiments may be implemented.

[0011] Figure 1B is a diagram showing that according to an embodiment, Figure 1A A system diagram of an exemplary wireless transmit / receive unit (WTRU) for use in a communication system is shown.

[0012] Figure 1C is a diagram showing that according to an embodiment, Figure 1A A system diagram of an example radio access network (RAN) and an example core network (CN) for use within a communication system is shown in FIG.

[0013] Figure 1D is a diagram showing that according to an embodiment, Figure 1A System diagram of another example RAN and another example CN used within the communication system shown in .

[0014] Figure 2 A block diagram of a video encoder is shown.

[0015] Figure 3 A block diagram of a video decoder is shown.

[0016] Figure 4 Examples associated with bidirectional optical flow (BDOF) are shown.

[0017] Figure 5A and 5BAn example of gradient derivation in BDOF with a motion accuracy of 1 / 16 pixel is shown.

[0018] Fig. 6A An example showing memory access for BDOF without block extension constraints

[0019] Figure 6B An example of memory access of BDOF with block extension constraint is shown.

[0020] Figure 7 An example of Advanced Temporal Motion Vector Prediction (ATMVP) is shown.

[0021] Figure 8 An example of spatio-temporal motion vector prediction (STMVP) is shown.

[0022] Fig.9A An example of frame rate up-conversion (FRUC) with template matching is shown.

[0023] Fig. 9B A plot of FRUC with bilateral matching is shown.

[0024] Fig. 10A Diagrams showing the affine pattern and the simplified affine model.

[0025] Fig. 10B Diagram showing affine mode and sub-block level motion derivation for affine blocks.

[0026] Fig.11 A diagram showing the BDOF refinement of a prediction unit (PU).

[0027] Fig.12 A diagram showing directional BDOF refinement. DETAILED DESCRIPTION

[0028] Figure 1A 1 is a diagram illustrating an example communication system 100 in which one or more disclosed embodiments may be implemented. The communication system 100 may be a multiple access system that provides content such as voice, data, video, messaging, broadcast, etc. to multiple wireless users. The communication system 100 may enable multiple wireless users to access such content by sharing system resources including wireless bandwidth. For example, the communication system 100 may employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single carrier FDMA (SC-FDMA), zero tail unique word DFT spread OFDM (ZT-UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block filtered OFDM, filter bank multi-carrier (FBMC), etc.

[0029] like Figure 1A As shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, public switched telephone network (PSTN) 108, Internet 110, and other networks 112, but it should be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each WTRU 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, 102d (any of which may be referred to as a “station” and / or “STA”) may be configured to transmit and / or receive wireless signals and may include user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a subscription-based unit, a pager, a cellular phone, a personal digital assistant (PDA), a smart phone, a laptop, a netbook, a personal computer, a wireless sensor, a hotspot or MiFi device, an Internet of Things (IoT) device, a watch or other wearable device, a head-mounted display (HMD), a vehicle, a drone, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in an industrial and / or automated process chain environment), a consumer electronic device, a device operating on a commercial and / or industrial wireless network, etc. Any of the WTRUs 102a, 102b, 102c, and 102d may be interchangeably referred to as a UE.

[0030] The communication system 100 may also include a base station 114a and / or a base station 114b. Each of the base stations 114a, 114b may be any type of device that is configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks (e.g., CN 106 / 115, Internet 110, and / or other networks 112). By way of example, the base stations 114a, 114b may be a base transceiver station (BTS), a Node B, an eNode B, a Home Node B, a Home eNode B, a gNB, an NR Node B, a site controller, an access point (AP), a wireless router, and the like. Although the base stations 114a, 114b are each depicted as a single element, it will be appreciated that the base stations 114a, 114b may include any number of interconnected base stations and / or network elements.

[0031] The base station 114a may be part of the RAN 104 / 113, which may also include other base stations and / or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), a relay node, etc. The base station 114a and / or the base station 114b may be configured to send and / or receive wireless signals on one or more carrier frequencies (which may be referred to as cells (not shown)). These frequencies may be in a licensed spectrum, an unlicensed spectrum, or a combination of licensed and unlicensed spectrums. A cell may provide coverage of wireless services to a specific geographic area, which may be relatively fixed or may change over time. The cell may be further divided into cell sectors. For example, a cell associated with the base station 114a may be divided into three sectors. Therefore, in one embodiment, the base station 114a may include three transceivers, i.e., one transceiver for each sector of the cell. In an embodiment, the base station 114a may employ multiple-input multiple-output (MIMO) technology, and may use multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in a desired spatial direction.

[0032] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d via an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0033] More specifically, as described above, the communication system 100 may be a multiple-access system and may employ one or more channel access schemes such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, the base station 114a in the RAN 104 / 113 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may use Wideband CDMA (WCDMA) to establish the air interface 115 / 116 / 117. WCDMA may include communication protocols such as High Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High Speed ​​UL Packet Access (HSUPA).

[0034] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 116 using Long Term Evolution (LTE) and / or LTE-Advanced (LTE-A) and / or LTE-Advanced Pro (LTE-A Pro).

[0035] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as NR radio access, which may establish the air interface 116 using new radio (NR).

[0036] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may implement LTE radio access and NR radio access together, for example using dual connectivity (DC) principles. Thus, the air interface used by the WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions to / from multiple types of base stations, such as eNBs and gNBs.

[0037] In other embodiments, the base station 114a and the WTRUs 102a, 102b, 102c may implement wireless technologies such as IEEE 802.11 (i.e., Wireless Fidelity (WiFi), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Internet Standard 2000 (IS-2000), Internet Standard 95 (IS-95), Internet Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data Rates for GSM Evolution (EDGE), GSM EDGE (GERAN), etc.

[0038] Figure 1AThe base station 114b in the example may be, for example, a wireless router, a Home NodeB, a Home eNodeB, or an access point, and may utilize any suitable RAT to facilitate wireless connectivity in a local area, such as a business location, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a road, and the like. In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement radio technologies such as IEEE 802.11 to establish a wireless local area network (WLAN). In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement radio technologies such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, the base station 114b and the WTRUs 102c, 102d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish a picocell or a femtocell. Figure 1A As shown, the base station 114b may have a direct connection to the Internet 110. Therefore, the base station 114b may not need to access the Internet 110 via the CN 106 / 115.

[0039] The RAN 104 / 113 may be in communication with the CN 106 / 115, which may be any type of network configured to provide voice, data, applications and / or Voice over Internet Protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have varying quality of service (QoS) requirements, such as different throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. The CN 106 / 115 may provide call control, billing services, mobile location-based services, prepaid calls, Internet connectivity, video distribution, etc., and / or perform advanced security functions (e.g., user authentication). Although in Figure 1A Although not shown, it will be appreciated that the RAN 104 / 113 and / or the CN 106 / 115 may be in direct or indirect communication with other RANs that employ the same RAT as the RAN 104 / 113 or a different RAT. For example, in addition to being connected to the RAN 104 / 113 that may utilize NR radio technology, the CN 106 / 115 may also communicate with another RAN (not shown) that employs GSM, UMTS, CDMA2000, WiMAX, E-UTRA, or WiFi radio technology.

[0040] The CN 106 / 115 may also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and / or other networks 112. The PSTN 108 may include a circuit-switched telephone network that provides plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the Transmission Control Protocol (TCP), the User Datagram Protocol (UDP), and / or the Internet Protocol (IP) in the TCP / IP Internet protocol suite. The networks 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, the networks 112 may include another CN connected to one or more RANs, which may use the same RAT as the RAN 104 / 113 or a different RAT.

[0041] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communication system 100 may include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers to communicate with different wireless networks via different wireless links). Figure 1A The WTRU 102c shown may be configured to communicate with the base station 114a, which may employ a cellular-based radio technology, and with the base station 114b, which may employ an IEEE 802 radio technology.

[0042] Figure 1B is a system diagram illustrating an example WTRU 102. Figure 1B As shown, the WTRU 102 may include, among other things, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keyboard 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and / or other peripherals 138. It will be appreciated that the WTRU 102 may include any sub-combination of the foregoing elements while remaining consistent with an embodiment.

[0043] Figure 1B is a system diagram illustrating an example WTRU 102. Figure 1B As shown, the WTRU 102 may include, among other things, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keyboard 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and / or other peripherals 138. It will be appreciated that the WTRU 102 may include any sub-combination of the foregoing elements while remaining consistent with an embodiment.

[0044] The transmit / receive element 122 may be configured to transmit or receive signals to or from a base station (e.g., base station 114a) via the air interface 116. For example, in one embodiment, the transmit / receive element 122 may be an antenna configured to transmit and / or receive RF signals. In one embodiment, the transmit / receive element 122 may be a transmitter / detector configured to transmit and / or receive, for example, IR, UV, or visible light signals. In yet another embodiment, the transmit / receive element 122 may be configured to transmit and / or receive both RF and light signals. It should be appreciated that the transmit / receive element 122 may be configured to transmit and / or receive any combination of wireless signals.

[0045] Although the transmit / receive element 122 is Figure 1B Although depicted as a single element in the embodiment, the WTRU 102 may include any number of transmit / receive elements 122. More specifically, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.

[0046] The transceiver 120 may be configured to modulate signals to be transmitted by the transmit / receive element 122 and to demodulate signals received by the transmit / receive element 122. As described above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers to enable the WTRU 102 to communicate via multiple RATs (e.g., NR and IEEE 802.11).

[0047] The processor 118 of the WTRU 102 may be connected to and may receive user input data from a speaker / microphone 124, a keyboard 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light emitting diode (OLED) display unit). The processor 118 may also output user data to the speaker / microphone 124, the keyboard 126, and / or the display / touchpad 128. In addition, the processor 118 may access information from and store data in any type of suitable memory, such as a non-removable memory 130 and / or a removable memory 132. The non-removable memory 130 may include a random access memory (RAM), a read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from and store data in a memory that is not physically located on the WTRU 102, such as on a server or a home computer (not shown).

[0048] The processor 118 may receive power from the power source 134, and may be configured to distribute and / or control power to other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium, nickel-zinc, nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0049] The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to or in lieu of the information from the GPS chipset 136, the WTRU 102 may receive location information from a base station (e.g., base stations 114a, 114b) over the air interface 116 and / or determine its location based on the timing of signals received from two or more neighboring base stations. It will be appreciated that the WTRU 102 may acquire location information by any suitable location-determination method while remaining consistent with an embodiment.

[0050] The processor 118 may also be coupled to other peripherals 138, which may include one or more software and / or hardware modules that provide additional features, functionality, and / or wired or wireless connectivity. For example, the peripherals 138 may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (for photos and / or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, module, a frequency modulation (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, a virtual reality and / or augmented reality (VR / AR) device, an activity tracker, etc. The peripheral device 138 may include one or more sensors, which may be one or more of a gyroscope, an accelerometer, a Hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor; a geolocation sensor; an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a posture sensor, a biometric sensor, and / or a humidity sensor.

[0051] The WTRU 102 may include a full-duplex radio for which transmission and reception of some or all signals (e.g., signals associated with particular subframes for UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous. A full-duplex radio may include an interference management unit that reduces and / or substantially eliminates self-interference by means of hardware (e.g., chokes) or by signal processing by a processor (e.g., a separate processor (not shown) or by the processor 118). In an embodiment, the WTRU 102 may include a half-duplex radio that transmits and receives some or all signals (e.g., signals associated with particular subframes for UL (e.g., for transmission) or downlink (e.g., for reception)).

[0052] Figure 1C 1 is a system diagram illustrating the RAN 104 and the CN 106 in accordance with an embodiment. As described above, the RAN 104 may employ an E-UTRA radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 104 may also be in communication with the CN 106.

[0053] The RAN 104 may include eNode-Bs 160a, 160b, 160c, though it will be appreciated that the RAN 104 may include any number of eNode-Bs while remaining consistent with an embodiment. The eNode-Bs 160a, 160b, 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the eNode-Bs 160a, 160b, 160c may implement MIMO technology. Thus, for example, the eNode-B 160a may use multiple antennas to transmit wireless signals to and / or receive wireless signals from the WTRU 102a.

[0054] Each of the eNode-Bs 160a, 160b, 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, user scheduling in the UL and / or DL, etc. Figure 1CAs shown in FIG. , the eNode-Bs 160a, 160b, 160c may communicate with one another via an X2 interface.

[0055] Figure 1C The CN 106 shown in FIG. 1 may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (or PGW) 166. Although each of the foregoing elements is depicted as part of the CN 106, it will be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0056] The MME 162 may be connected to each of the eNode-Bs 162a, 162b, 162c in the RAN 104 via an S1 interface and may serve as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, bearer activation / deactivation, selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, 102c, and the like. The MME 162 may provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM and / or WCDMA.

[0057] The SGW 164 may be connected to each of the eNode-Bs 160a, 160b, 160c in the RAN 104 via an S1 interface. The SGW 164 may generally route and forward user data packets to / from the WTRUs 102a, 102b, 102c. The SGW 164 may perform other functions, such as anchoring the user plane during inter-eNode-B handovers, triggering paging when DL data is available for the WTRUs 102a, 102b, 102c, managing and storing the context of the WTRUs 102a, 102B, 102c, and the like.

[0058] The SGW 164 may be connected to the PGW 166 , which may provide the WTRUs 102a , 102b , 102c with access to packet-switched networks, such as the Internet 110 , to facilitate communications between the WTRUs 102a , 102b , 102c and IP-enabled devices.

[0059] The CN 106 may facilitate communications with other networks. For example, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks (e.g., the PSTN 108) to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the CN 106 may include, or may communicate with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between the CN 106 and the PSTN 108. In addition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0060] Although the WTRU Figures 1A-1D Although described as a wireless terminal, it is contemplated that in certain representative embodiments, such a terminal may use a (eg, temporary or permanent) wired communication interface with a communication network.

[0061] In a representative embodiment, other network 112 may be a WLAN.

[0062] A WLAN in a service set (BSS) mode may have an access point (AP) for a BSS and one or more stations (STAs) associated with the AP. The AP may access or dock to a distribution system (DS) or another type of wired / wireless network that carries traffic entering and / or leaving the BSS. Traffic originating from outside the BSS and destined for the STA may be reached through the AP and may be delivered to the STA. Traffic originating from the STA and destined for a destination outside the BSS may be sent to the AP to be delivered to the corresponding destination. Traffic between STAs within the BSS may be sent through the AP, for example, where the source STA may send traffic to the AP, and the AP may deliver traffic to the destination STA. Traffic between STAs within the BSS may be considered and / or referred to as peer-to-peer traffic. Peer-to-peer traffic may be sent between the source and destination STAs (e.g., directly between the source and destination STAs) using a direct link setup (DLS). In certain representative embodiments, the DLS may use 802.11e DLS or 802.11z tunnel DLS (TDLS). A WLAN using an independent BSS (IBSS) mode may not have an AP, and STAs (eg, all STAs) within or using the IBSS may communicate directly with each other. The IBSS communication mode may sometimes be referred to herein as an "ad-hoc" communication mode.

[0063] When using the 802.11ac infrastructure operating mode or a similar operating mode, the AP can transmit beacons on a fixed channel (e.g., a primary channel). The primary channel can be a fixed width (e.g., a 20MHz wide bandwidth) or a width dynamically set via signaling. The primary channel can be a working channel of the BSS and can be used by the STA to establish a connection with the AP. In certain representative embodiments, carrier sensing multiple access (CSMA / CA) with collision avoidance can be implemented, for example, in an 802.11 system. For CSMA / CA, a STA (e.g., each STA) including the AP can sense the main channel. If the main channel is sensed / detected and / or determined to be busy by a specific STA, the specific STA can back off. One STA (e.g., only one station) can transmit at any given time in a given BSS.

[0064] A high throughput (HT) STA may communicate using a 40 MHz wide channel, for example, by combining a primary 20 MHz channel with an adjacent or non-adjacent 20 MHz channel to form a 40 MHz wide channel.

[0065] Very high throughput (VHT) STA can support 20MHz, 40MHz, 80MHz and / or 160MHz wide channels. 40MHz and / or 80MHz channels can be formed by combining adjacent 20MHz channels. 160MHz channels can be formed by combining 8 consecutive 20MHz channels or by combining two discontinuous 80MHz channels, which can be referred to as 80+80 configuration. For 80+80 configuration, after channel coding, data can be passed and passed through a segment parser, which can divide the data into two streams. Each stream can be processed by inverse fast Fourier transform (IFFT) and time domain processing respectively. The stream can be mapped to two 80MHz channels, and the data can be transmitted by the transmitting STA. At the receiver of the receiving STA, the operation of the above 80+80 configuration can be reversed, and the combined data can be sent to the medium access control (MAC).

[0066] The operating mode below 1 GHz is supported by 802.11af and 802.11ah. The channel operating bandwidth and carrier are reduced in 802.11af and 802.11ah relative to the channel operating bandwidth and carrier used in 802.11n and 802.11ac. 802.11af supports 5MHz, 10MHz and 20MHz bandwidths in the TV white space (TVWS) spectrum, while 802.11ah supports 1MHz, 2MHz, 4MHz, 8MHz and 16MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11ah can support instrument type control / machine type communication, such as MTC devices in macro coverage areas. MTC devices may have certain capabilities, for example, including limited capabilities to support (e.g., only support) certain and / or limited bandwidths. MTC devices may include batteries with battery life above a threshold (e.g., to maintain a very long battery life).

[0067] A WLAN system that can support multiple channels and channel bandwidths, such as 802.11n, 802.11ac, 802.11af, and 802.11ah, includes a channel that can be designated as a primary channel. The primary channel may have a bandwidth equal to the maximum operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel may be set and / or limited by the STA among all STAs operating in the BSS (which supports the minimum bandwidth operating mode). In the example of 802.11ah, for STAs (e.g., MTC-type devices) that support (e.g., only support) 1MHz mode, the primary channel may be 1MHz wide, even if the AP and other STAs in the BSS support 2MHz, 4MHz, 8MHz, 16MHz, and / or other channel bandwidth operating modes. Carrier sensing and / or network allocation vector (NAV) settings may depend on the state of the primary channel. If the primary channel is busy (e.g., due to a STA (which only supports the 1MHz operating mode) transmitting to the AP), the entire available band may be considered busy even if most of the band remains idle and available.

[0068] In the United States, the available frequency band that 802.11ah can use is from 902MHz to 928MHz. In South Korea, the available frequency band is from 917.5MHz to 923.5MHz. In Japan, the available frequency band is from 916.5MHz to 927.5MHz. Depending on the country code, the total bandwidth available for 802.11ah is 6MHz to 26MHz.

[0069] Figure 1D1 is a system diagram illustrating the RAN 113 and the CN 115 according to an embodiment. As described above, the RAN 113 may communicate with the WTRUs 102a, 102b, 102c over the air interface 116 using NR radio technology. The RAN 113 may also communicate with the CN 115.

[0070] RAN113 may include gNB180a, 180b, 180c, but it should be understood that RAN113 may include any number of gNBs while remaining consistent with the embodiment. Each of gNB180a, 180b, 180c includes one or more transceivers for communicating with WTRU102a, 102b, 102c via the air interface 116. In one embodiment, gNB180a, 180b, 180c may implement MIMO technology. For example, gNB180a, 108b may utilize beamforming to transmit signals to and / or receive signals from gNB180a, 180b, 180c. Therefore, gNB180a may, for example, use multiple antennas to transmit wireless signals to and / or receive wireless signals from WTRU102a. In one embodiment, gNB180a, 180b, 180c may implement carrier aggregation technology. For example, gNB180a may transmit multiple component carriers (not shown) to WTRU102a. A subset of these component carriers may be on unlicensed spectrum, while the remaining component carriers may be on licensed spectrum. In one embodiment, the gNBs 180a, 180b, 180c may implement coordinated multi-point (CoMP) technology. For example, the WTRU 102a may receive coordinated transmissions from gNB 180a and gNB 180b (and / or gNB 180c).

[0071] The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using transmissions associated with scalable parameter configurations (numerology). For example, OFDM symbol spacing and / or OFDM subcarrier spacing may vary for different transmissions, different cells, and / or different portions of the wireless transmission spectrum. The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using subframes or transmission time intervals (TTIs) of various or scalable lengths (e.g., containing different numbers of OFDM symbols and / or continuously varying absolute time lengths).

[0072] The gNBs 180a, 180b, 180c may be configured to communicate with the WTRUs 102a, 102b, 102c in a standalone configuration and / or a non-standalone configuration. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c without also accessing other RANs (e.g., eNodeBs 160a, 160b, 160c). In a standalone configuration, the WTRUs 102a, 102b, 102c may utilize one or more of the gNBs 180a, 180b, 180c as mobility anchors. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using signals in unlicensed bands. In a non-standalone configuration, the WTRUs 102a, 102b, 102c may communicate / connect with the gNBs 180a, 180b, 180c while also communicating / connecting with another RAN, such as the eNode-Bs 160a, 160b, 160c. For example, the WTRUs 102a, 102b, 102c may implement the DC principle to communicate with one or more of the gNBs 180a, 180b, 180c and one or more of the eNode-Bs 160a, 160b, 160c substantially simultaneously. In a non-standalone configuration, the eNode-Bs 160a, 160b, 160c may serve as mobility anchors for the WTRUs 102a, 102b, 102c, and the gNBs 180a, 180b, 180c may provide additional coverage and / or throughput for serving the WTRUs 102a, 102b, 102c.

[0073] Each of the gNBs 180a, 180b, 180c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, support network slicing, dual connectivity, interworking between NR and E-UTRA, route user plane data to a user plane function (UPF) 184a, 184b, route control plane information to an access and mobility management function (AMF) 182a, 182b, etc. Figure 1D As shown, gNB180a, 180b, and 180c can communicate with each other through the Xn interface.

[0074] Figure 1DThe CN 115 shown in the figure may include at least one of the AMF 182a, 182b, at least one of the UPF 184a, 184b, at least one of the session management function (SMF) 183a, 183b, and possibly a data network (DN) 185a, 185b. Although each of the aforementioned elements is depicted as part of the CN 115, it will be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0075] AMF182a, 182b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via the N2 interface and may act as a control node. For example, AMF182a, 182b may be responsible for authenticating users of WTRU102a, 102b, 102c, supporting network slicing (e.g., handling different PDU sessions with different requirements), selecting specific SMF183a, 183b, managing registration areas, terminating NAS signaling, mobility management, and the like. AMF182a, 182b may use slices to customize CN support for WTRU102a, 102b, 102c based on the type of service used by WTRU102a, 102b, 102c. For example, different network slices may be established for different use cases, such as services that rely on ultra-reliable low latency (URLLC) access, services that rely on enhanced massive mobile broadband (eMBB) access, services for machine type communication (MTC) access, and the like. The AMF 162 may provide a control plane function for switching between the RAN 113 and other RANs (not shown) that employ other radio technologies (e.g., LTE, LTE-A, LTE-APro, and / or non-3GPP access technologies such as WiFi).

[0076] SMF183a, 183b can be connected to AMF182a, 182b in CN115 via N11 interface. SMF183a, 183b can also be connected to UPF184a, 184b in CN115 via N4 interface. SMF183a, 183b can select and control UPF184a, 184b, and configure service routing through UPF184a, 184b. SMF183a, 183b can perform other functions, such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy implementation and QoS, providing downlink data notification, etc. The PDU session type can be IP-based, non-IP-based, Ethernet-based, etc.

[0077] The UPF 184a, 184b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via the N3 interface, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The UPF 184, 184b may perform other functions, such as routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering downlink packets, providing mobility anchoring, and the like.

[0078] The CN 115 may facilitate communications with other networks. For example, the CN 115 may include an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) or may communicate with an IP gateway that serves as an interface between the CN 115 and the PSTN 108. In addition, the CN 115 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, the WTRUs 102a, 102b, 102c may be connected to the local data networks (DNs) 185a, 185b via an N3 interface to the UPFs 184a, 184b and an N6 interface between the UPFs 184a, 184b and the DNs 185a, 185b, and through the UPFs 184a, 184b.

[0079] Given that Figures 1A-1D and Figures 1A-1D , one or more or all of the functions described herein with respect to one or more of the following items may be performed by one or more simulation devices (not shown): WTRU 102a-d, base station 114a-b, eNode-B 160a-c, MME 162, SGW 164, PGW 166, gNB 180a-c, AMF 182a-b, UPF 184a-b, SMF 183a-b, DN 185a-b, and / or any other device (one or more) described herein. These simulation devices may be one or more devices configured to simulate one or more or all of the functions described herein. For example, the simulation device may be used to test other devices and / or simulate network and / or WTRU functions.

[0080] The simulation device can be designed to implement one or more tests of other devices in a laboratory environment and / or an operator network environment. For example, one or more simulation devices can perform one or more or all functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network in order to test other devices within the communication network. One or more simulation devices can perform one or more or all functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. The simulation device can be directly coupled to another device for testing purposes and / or can use over-the-air wireless communications to perform testing.

[0081] One or more simulation devices can perform one or more functions (including all functions) without being implemented / deployed as part of a wired and / or wireless communication network. For example, the simulation device can be used in a test lab and / or in a test scenario in which a wired and / or wireless communication network is not deployed (e.g., testing) to implement testing of one or more components. One or more simulation devices can be test devices. The simulation device can transmit and / or receive data using direct RF coupling and / or via RF circuits (e.g., which can include one or more antennas) and / or wireless communications.

[0082] For example, bidirectional optical flow (BDOF) may be performed to enhance the efficiency of bi-predictive prediction. BDOF may increase the granularity and / or precision of motion vectors used during motion compensation. BDOF may improve decoding performance. BDOF may increase the complexity at a decoding device (e.g., an encoder and / or a decoder).

[0083] Video decoding systems can be used to compress digital video signals to reduce the storage requirements and / or transmission bandwidth of such signals. There may be various types of video decoding systems, such as block-based, wavelet-based, object-based, etc. For example, a block-based hybrid video decoding system may be widely used and deployed. Examples of block-based video decoding systems may conform to one or more international video decoding standards, such as MPEG1 / 2 / 4 Part 2, H.264 / MPEG-4 Part 10 AVC, VC-1. Examples of block-based video decoding systems may also include the video decoding standard High Efficiency Video Coding (HEVC) developed by ITU-T / SG16 / Q.6 / VCEG and ISO / IEC / MPEG's JCT-VC (Joint Collaboration Team on Video Coding).

[0084] Figure 2 Diagram illustrating a block-based video encoder that may be constructed based on a block-based hybrid video coding framework. Figure 2A block diagram of a block-based video coding system, which may be a block-based hybrid video coding system, is shown. An input video signal 202 may be processed block by block. Extended block sizes (which may be referred to as coding units or CUs) may be used to efficiently compress high-resolution (e.g., 1080p and above) video signals. A CU may be at most 64×64 pixels. A CU may be further partitioned into prediction units (PUs), to which separate prediction methods may be applied. For an input video block (MB or CU), such as one or more input video blocks, spatial prediction (260) and / or temporal prediction (262) may be performed. Spatial prediction or intra-frame prediction may use pixels from samples (which may be referred to as reference samples) of coded neighboring blocks in the same video picture / slice to predict the current video block. Spatial prediction may reduce spatial redundancy inherent in a video signal. Temporal prediction (which may be referred to as inter-frame prediction or motion compensated prediction) may use reconstructed pixels from a coded video picture to predict the current video block. Temporal prediction may reduce temporal redundancy that may be inherent in a video signal. A temporal prediction signal for a given video block may be signaled by one or more motion vectors, which may indicate the amount and direction of motion between the current block and its reference blocks. If multiple reference pictures are supported, a reference picture index may be sent for a video block (e.g., for each video block). The reference picture index may be used to identify which reference picture in a reference picture store (264) the temporal prediction signal comes from. After spatial and / or temporal prediction, a mode decision (280) in the encoder may select a prediction mode, for example, based on a rate-distortion optimization method. The selected prediction mode may be the best prediction mode for the situation. The prediction block (216) may be subtracted from the current video block, and the prediction residual may be decorrelated using a transform block (204) and a quantization block (206). The quantized residual coefficients may be inverse quantized (210) and inverse transformed (212) to form a reconstructed residual, which may be added back to the prediction block (226) to form a reconstructed video block. For example, in-loop filtering (266) such as a deblocking filter and an adaptive loop filter may be further applied to the reconstructed video block before the reconstructed video block is placed in a reference picture storage device (264) and used for decoding future video blocks. To form the output video bitstream 220, the decoding mode (e.g., inter-frame or intra-frame), prediction mode information, motion information, and quantized residual coefficients may be sent to the entropy decoding unit (208) to be further compressed and packaged to form a bitstream.

[0085] Figure 3A block diagram of a video decoder is shown, which may be a block-based video decoder. A video bitstream 302 may be unpacked and entropy decoded at entropy decoding 308. Coding mode and prediction information may be sent to spatial prediction 360 (e.g., intra-frame decoding) or motion compensated prediction 362 (e.g., inter-frame decoding) to form a prediction block. Residual transform coefficients may be sent to inverse quantization 310 and inverse transform 312 to reconstruct the residual block. The prediction block may be added to the residual block at 336. The reconstructed block may be further subjected to in-loop filtering before being stored in reference picture storage 354. The reconstructed video in the reference picture storage may then be sent out to drive a display device and may be used to predict future video blocks.

[0086] like Figure 2 and 3 As shown, the encoding / decoding workflow can be based on, for example, one or more of: spatial prediction (e.g., intra-frame prediction); temporal prediction (e.g., inter-frame prediction); transform; quantization; entropy coding; loop filter; and / or the like.

[0087] Bidirectional optical flow (BDOF) can be used for motion vector derivation. For example, bi-prediction can be based on an optical flow model. Bi-prediction in video decoding may include combining two temporal prediction blocks that can be obtained from a reconstructed reference picture. Due to the limitations of block-based motion compensation (MC), there may be residual small motion observable between the two prediction blocks, which may reduce the efficiency of motion compensated prediction. BDOF can, for example, be applied to compensate for small motion within samples within a block. BDOF may include applying sample-by-sample motion refinement. For example, when bi-prediction is used, sample-by-sample motion refinement can be performed on top of block-based motion compensated prediction. Motion vectors (e.g., refined motion vectors) can be derived for samples in a block. Motion vector derivation can be based on an optical flow model (e.g., a classic optical flow model). For example, let I (k) (x, y) is the sample value at coordinate (x, y) of the prediction block derived from reference picture list k (k = 0, 1), where and The horizontal gradient and vertical gradient of the sample can be included respectively. The optical flow model can be used to derive the motion refinement parameters (v x ,v y ):

[0088]

[0089] Using a combination of the optical flow equation (1) and interpolation of prediction blocks along the motion trajectory (e.g., Figure 4 As shown), the BDOF prediction can be obtained as:

[0090]

[0091] Refer to (2), τ 0 With τ 1 Can include I (0) with I (1) Associated reference image Ref 0 and Ref 1 The time distance to the current image CurPic, for example,

[0092]

[0093] Figure 4 An example application of BDOF is shown. Figure 4 In (MV x0 , MV y0 ) and (MV x1 , MV y1 ) can indicate that two prediction blocks can be generated. (0) with I (1) The block-level motion vector can be obtained by minimizing the difference Δ between the sample values ​​after motion refinement compensation (e.g., Figure 4 A and B in the figure are used to calculate the motion refinement parameters (v) at the sample position (x, y). x ,v y ), as shown below:

[0094]

[0095] The regularity of the derived motion refinement can be maintained. The motion refinement can be consistent within a local surrounding area centered at (x, y). In a BDOF design, its value (v) can be derived by minimizing Δ within a 5x5Ω window around the current sample at (x, y). x ,v y ),as follows:

[0096]

[0097] BDOF can be applied to bi-predicted blocks. Bi-predicted coded blocks can be predicted by two or more reference blocks from temporally neighboring pictures. For example, BDOF can be enabled without sending additional information from the decoder to the decoder. For example, BDOF can be applied to a block with forward and backward prediction signals (e.g., τ 0 ·τ 1 >0). If the two prediction blocks of the current block come from the same direction (forward or backward, such as τ 0 ·τ 1 >0), then when two prediction blocks are associated with non-zero motion, such as abs(MV x0 )+abs(MV y0)≠0and abs(MV x1 )+abs(MV y1 )≠0, BDOF is applied, and the two motion vectors may be proportional to the temporal distance between the current picture and the reference picture, e.g.,

[0098]

[0099] If the two prediction blocks of the current block come from the same reference picture (e.g., τ 0 =τ 1 ), BDOF can be disabled. BDOF can be disabled when local illumination compensation (LIC) is used for the current block.

[0100] A gradient derivation process may be performed. As shown in (2) and (4), for example, in addition to the block-level MC, a motion compensation block (e.g., I (0) with I (1) ), which can be used to derive local motion refinement and generate a prediction at the sample location. In one example, when the prediction signal is generated based on a filtering process, two prediction blocks (e.g., and ). For example, a 2D separable finite impulse response (FIR) filter may be used to perform motion compensated interpolation. Horizontal gradients and vertical gradients may be generated simultaneously. The input to the gradient derivation process may include reference samples for motion compensation. The input to the gradient derivation process may include input motion (MV x0 / x1 ,MV y0 / y1 ) fractional components (fracX, fracY). To derive the gradient value at the sample position, two different filters (e.g., interpolation filter h L and the gradient filter h G ). For example, filters may be applied individually in different orders according to the direction in which the gradient may be calculated. One or more of the following may be applied. When deriving a horizontal gradient (e.g., and ), the interpolation filter h L can be applied vertically to samples within the prediction block to derive the sample value at the vertical fractional position at fracY. The gradient filter h G can be applied horizontally to the generated vertical fractional samples to calculate the horizontal gradient value based on the value of fracX. For the vertical gradient ( and ), the gradient filter h G can be applied vertically, for example, on the prediction samples to compute an intermediate vertical gradient corresponding to fracY, and can be followed by using an interpolation filter h according to the value of fracXL The length of the gradient filter and the interpolation filter can be 6 taps. An 8-tap filter can be used for motion compensation. Tables 1 and 2 illustrate the precision of the block-level motion vector (for example, which can be as high as 1 / 16 pixel) that can be used for h G and h L The filter coefficients of .

[0101] Table 1: Filters that can be used for gradient calculation in BDOF

[0102] Score Position <![CDATA[Gradient filter (h G )]]> 0 {8，-39，-3，46，-17，5} 1 / 16 {8，-32，-13，50，-18，5} 1 / 8 {7，-27，-20，54，-19，5} 3 / 16 {6，-21，-29，57，-18，5} 1 / 4 {4，-17，-36，60，-15，4} 5 / 16 {3，-9，-44，61，-15，4} 3 / 8 {1，-4，-48，61，-13，3} 7 / 16 {0，1，-54，60，-9，2} 1 / 2 {-1，4，-57，57，-4，1}

[0103] Table 2: Interpolation filters that can be used for prediction signal generation in BDOF

[0104] Score Position <![CDATA[Interpolation filter (h L ) <!-- 13 -->]]> 0 {0，0，64，0，0，0} 1 / 16 {1，-3，64，4，-2，0} 1 / 8 {1，-6，62，9，-3，1} 3 / 16 {2，-8，60，14，-5，1} 1 / 4 {2，-9，57，19，-7，2} 5 / 16 {3，-10，53，24，-8，2} 3 / 8 {3，-11，50，29，-9，2} 7 / 16 {3，-11，44，35，-10，3} 1 / 2 {3，-10，35，44，-11，3}

[0105] Figure 5A and 5B An example of gradient derivation in BDOF with a motion accuracy of 1 / 16 pixel is shown. Figure 5A and 5B As shown in Figure 1, the gradient derivation process can be applied in BDOF, where the sample values ​​at integer sample positions can be shown with patterned squares and the sample values ​​at fractional sample positions can be shown with blank squares. Because the motion vector accuracy can be increased to 1 / 16 pixel, Figure 5A and 5B There can be a total of 255 fractional sample positions defined within the region of integer samples. The subscripted coordinates (x, y) can represent the corresponding horizontal and vertical fractional positions of the sample (e.g., coordinates (0, 0) can correspond to samples at integer positions). 1,1 ) Calculate the horizontal gradient value and the vertical gradient value. Figure 5A and 5B For the horizontal gradient derivation, the interpolation filter h can be applied in the vertical direction. L To derive the fractional sample f 0,1 ,e 0,1 ,a 0,1 ,b 0,1 ,c 0,1 and d 0,1 ,For example:

[0106]

[0107] Referring to (7): B may include the bit depth of the input signal, and OffSet 0 A rounding offset may be included, which may be equal to:

[0108]

[0109] f 0,1 ,e 0,1 ,a 0,1 ,b 0,1 ,c 0,1 and d 0,1 The precision of the gradient filter h can be 14 bits. G Applied horizontally to the derived fractional sample to compute a 1,1 The horizontal gradient of . One or more of the following may be applied. The non-rounded gradient value at the middle 20 bits may be calculated as follows:

[0110] gH_a′ 1,1 =8·f 0,1 -32·e 0,1 -13·a 0,1 +50 b 0,1 -18·c 0,1 +5·d 0,1 (9)

[0111] The horizontal gradient can be calculated by shifting the intermediate gradient values ​​into the output precision as follows:

[0112] g 1,1 =sign(gH_a′ 1,1 )·((abs(gH_a′ 1,1 )+OffSet 1 )>>(18-B)) (10) Referring to (10): sign(·) and abs(·) may include functions that return the sign and absolute value of the input signal, respectively; and OffSet 1 A rounding offset may be included, which may be calculated as 2 17-B .

[0113] Now referring to the derived vertical gradient value at (1, 1), the intermediate vertical gradient value at the fractional position (0, 1) can be derived, for example:

[0114]

[0115] Intermediate gradient values ​​can be adjusted by shifting to 14-bit values ​​as follows:

[0116]

[0117] This can be done by applying an interpolation filter h on top of the intermediate gradient values ​​at fractional positions (0, 1) LTo obtain the vertical gradient value at the fractional position (1, 1). One or more of the following may be applied. A 20-bit non-rounded gradient value may be calculated and may be adjusted to the output bit depth by a shift operation, as shown below:

[0118]

[0119] Vx 1,1 =sign(gV_a′ 1,1 )·((abs(gV_a′ 1,1 )+OffSet 1 )>>(18-B)) (14)

[0120] The memory bandwidth consumption of BDOF can be provided. As shown in (5), in order to derive the local motion refinement (v x ,v y ), sample values ​​and gradient values ​​may be calculated for one or more samples (e.g., all samples) in a window Ω around the sample. The window size may include (2M+1)x(2M+1), where M=2. Due to the interpolation filters and gradient filters, the gradient derivation may access one or more (e.g., additional) reference samples, which may be found in an extended area of ​​the current block. The length T of the interpolation filters and gradient filters may be 6, and / or the corresponding extended block size may be equal to T-1=5. For example, considering a W x H block, the memory access used by BDOF (e.g., total memory access) may be (W+T-1+2M)x(H+T-1+2M)=(W+9)x(H+9), which may be greater than the memory access (W+7)x(H+7) used by other techniques. The memory access of BDOF may not be greater than the memory access of other techniques. A block extension constraint may be provided. When the block extension constraint is applied, neighboring samples within the current block may be used to calculate a local motion refinement (v) at a position (e.g., each position within the block). x ,v y ). Fig. 6A -B shows a diagram of memory access of BDOF, where Fig. 6A shows memory access without block extension constraints, and where Figure 6B Memory access with block extension constraint is shown. Fig. 6A and 6B The sizes of the memory access regions for BDOF before and after applying the block extension constraint may be compared.

[0121] Motion compensation based on sub-blocks (e.g., sub-CUs) may be performed. A coding block may have a motion vector for a prediction direction. Many sub-block level inter-frame prediction techniques may be used. Sub-block level inter-frame prediction techniques may include advanced temporal motion vector prediction (ATMVP), spatial-temporal motion vector prediction (STMVP), frame rate up-conversion (FRUC) mode, affine mode, and / or similar modes. One or more of the following may be applied. A video block may be partitioned into multiple small sub-blocks and may be used to derive motion information for a sub-block (e.g., deriving each sub-block separately). Motion information for a sub-block may be used, for example, to generate a prediction signal for a block in a motion compensation stage. A sub-block coding mode may be provided.

[0122] As described herein, ATMVP may be performed. One or more of the following may be applied. Temporal motion vector prediction may provide a block to derive motion information. The derived motion information may include motion vectors and reference indices for sub-blocks within the block (e.g., one or more smaller blocks from temporally adjacent pictures of the current picture). ATMVP may derive motion information for sub-blocks of a block: may identify corresponding blocks (which may be referred to as collocated blocks) for the current block (e.g., within a temporal reference picture); may partition the current block into sub-blocks; and / or may derive motion information for sub-blocks from corresponding small blocks in collocated pictures, such as Figure 7 As shown, the selected temporal reference picture may be referred to as a collocated picture.

[0123] Figure 7 An example diagram of ATMVP is shown. For example, the collocated blocks and collocated pictures can be identified by the motion information of the spatial neighboring blocks of the current block. Figure 7 For example, referring to Figure 7 , block A may be identified as an available (eg, first available) merge candidate for the current block based on the existing scanning order of the merge candidate list. The corresponding motion vector (eg, MV A ) and its reference index can be used to identify the collocated picture and the collocated block. The position of the collocated block in the collocated picture can be obtained by converting the motion vector (MV A ) is added to the coordinates of the current block to determine.

[0124] For a sub-block (e.g., each sub-block) in the current block, its corresponding small block (e.g., Figure 7The motion information of the small block in the collocated block (indicated by the arrow in ) can be used to derive the motion information of the corresponding sub-block in the current block. For example, the motion information of the small block in the collocated block can be converted into a motion vector and a reference index of the corresponding sub-block in the current block (e.g., after identifying the motion information of each small block in the collocated block). The motion information of the sub-block in the current block can be derived in a manner similar to other derivation techniques (e.g., where temporal motion vector scaling can be applied).

[0125] As described herein, STMVP may be performed. Figure 8 An exemplary diagram of STMVP is shown. One or more of the following may apply. In STMVP, motion information of sub-blocks in a coded block may be derived (eg, recursively derived). Figure 8 is an embodiment of recursively deriving motion information of sub-blocks in a decoding block. Figure 8 , the current block may include four sub-blocks A, B, C, and D. Neighboring small blocks that may be spatial neighbors of the current block may be labeled as a, b, c, and d, respectively. Motion derivation for sub-block A may include identifying one or more spatial neighbors (e.g., two spatial neighbors of sub-block A). A neighbor of sub-block A (e.g., a first neighbor) may be neighbor c above. If small block c is not available or intra-coded, the subsequent neighboring small blocks above the current block may be checked in order (e.g., from left to right). Another neighbor of sub-block A (e.g., a second neighbor) may be neighbor b on the left. If small block b is not available or intra-coded, the subsequent neighboring small blocks may be checked, for example, in order to the left of the current block (from top to bottom). After obtaining the motion information of the spatially adjacent pixels, the motion information of the temporal neighbors of sub-block A may be obtained using a process similar to TMVP. The motion information of the available spatial neighbors and one or more temporal neighbors (e.g., up to 3 neighbors) may be averaged and used as the motion information of sub-block A. Based on the raster scan order, the above STMVP process may be repeated to derive motion information for other sub-blocks (eg, all other sub-blocks) in the current video block.

[0126] Frame rate up conversion (FRUC) may be performed. FRUC may support inter-coded blocks, which may reduce motion information signaling. When FRUC mode is enabled, motion information of the coded block, which may include motion vectors and reference indexes, may not be signaled. Motion information may be derived (e.g., at a decoder) using, for example, template matching and / or bilateral matching. For example, during the motion derivation process at the decoder, a merge candidate list for the block and a set of preliminary motion vectors generated from the motion vectors of the temporally collocated blocks of the current block may be checked. The candidate that results in the minimum sum of absolute differences (SAD) may be selected as a starting point. A local search based on template matching or bilateral matching around the starting point may be performed. The MV that produces the minimum SAD may be used as the MV for the block. Motion information may be refined at the sub-block level, which may provide efficient motion compensation.

[0127] Fig.9A and 9B Examples related to FRUC are described. For example, Fig.9A -B shows an exemplary diagram for frame rate up-conversion, where Fig.9A shows template matching, while Fig. 9B shows a bilateral match. Fig.9A As shown, template matching can be used to derive motion information of a current block by finding a match (e.g., a best match) between a template in the current picture and a block in a reference picture (e.g., which may be the same size as the template). The template may include the top and / or left neighboring blocks of the current block. Fig. 9B In the embodiment of the present invention, bilateral matching can be used to derive motion information of the current block. The derivation of the motion information of the current block may include finding a match (e.g., a best match) between two blocks along the motion trajectory of the current block in different reference pictures (e.g., two different reference pictures). The motion search process of bilateral matching can be based on the motion trajectory. For example, the motion search process of bilateral matching can be based on the motion vector MV 0 and MV 1 . Motion Vector MV 0 and MV 1 Can point to a reference block. Motion vector MV 0 and MV 1 can be compared with the current image and two reference images (e.g., T 0 and T 1 ) is proportional to the time distance between them.

[0128] Affine models can be used to provide motion information. Fig. 10A and 10B An exemplary diagram showing an affine pattern, where Fig. 10A shows a simplified affine model, and Fig. 10BThe sub-block level motion derivation for affine blocks is shown. The translation motion model can be applied to motion compensation prediction. The motion can include one or more of the following: zoom in / out, rotation, perspective motion, irregular motion, etc. An example affine transform motion compensation prediction can be applied. Fig. 10A As shown in , the affine motion field of a block can be described by one or more (e.g., two) control point motion vectors. Based on the control point motion, the motion field of an affine block can be described as:

[0129]

[0130] Refer to (15), (v 0x ,v 0y ) can be the motion vector of the upper left control point, and (v 1x ,v 1y ) can be the motion vector of the upper right control point, such as Fig. 10A When decoding a video block using an affine mode, the motion field of the video block can be derived based on the granularity of 4x 4 blocks. Fig. 10B As shown, in order to derive the motion vector of a 4x 4 block (e.g., each 4x 4), the motion vector of the center sample of the sub-block (e.g., each sub-block) can be calculated according to (15) and rounded to 1 / 16 pixel accuracy. The derived motion vector can be used in the motion compensation stage to generate a prediction signal for the sub-block within the current block.

[0131] BDOF can be simplified. One or more of the following can be applied. The number of BDOF processes for a dual-predicted CU can be reduced. BDOF can be performed with parallel processing in mind. For a dual-predicted CU, the BDOF process can be skipped at the CU level or sub-CU level based on the similarity of two prediction signals from one or more (e.g., two) reference lists. For example, if the prediction signals are similar, the BDOF adjustment can be skipped. For example, the number of BDOF adjustments called can be reduced to maintain decoding performance. If the motion vector in a list (e.g., each list) is not an integer, the gradient derivation of the BDOF for the list may include sample interpolation and gradient calculation. The order of horizontal and vertical filtering can be changed to be implemented with parallel processing.

[0132] As described herein, integer motion components may be provided. For example, if k bits are used as a representation of the fractional portion of a motion vector component, and the component value of the motion vector is (2 k ), then the motion vector component may include an integer motion vector component. Similarly, if the component value of the motion vector is not ((2 k ), the motion vector component may include a non-integer (e.g., fractional) part and an integer part.

[0133] As described herein, BDOF may be performed taking into account bi-predictive prediction. The granularity and precision of motion vectors used in the motion compensation stage may be increased.

[0134] In BDOF, horizontal gradients and vertical gradients may be generated for reference picture lists (e.g., L0 and L1). For example, BDOF refinement may include invoking one or more filtering processes (e.g., horizontal and / or vertical filtering processes) on horizontal gradients and vertical gradients within a reference picture list (e.g., each reference picture list). As discussed herein, given a motion vector MV (MV x , MV y ), the horizontal gradient derivation may include applying a sample interpolation filter in the vertical direction. For example, if MV y If (e.g., the vertical component of the motion vector MV) includes a non-integer (e.g., fractional) motion vector component, a sample interpolation filter may be used to derive sample values ​​at fractional positions. Gradient filtering may be performed in the horizontal direction using the sample values ​​interpolated vertically at fractional positions. If MV y If an integer motion component is included, a gradient filter (eg, a single gradient filter) may be applied in the horizontal direction, for example, using sample values ​​at integer positions. A similar process may be performed for the derivation of the vertical gradient.

[0135] A bi-predicted CU may be associated with one or more (e.g., two) motion vectors. One or more filtering processes may be performed on the CU. For example, if the horizontal and vertical components of the motion vector associated with the CU include non-integer (e.g., fractional) motion components, multiple (e.g., 8) filtering processes may be performed. The filtering process, when executed (e.g., called), may include a memory access request, multiple multiplications, and multiple additions. The length of the filters used during BDOF (e.g., 8-tap sample interpolation filters and 6-tap gradient filters) may affect the coding complexity.

[0136] BDOF refinement may be conditionally skipped. For example, BDOF refinement may be conditionally skipped based on similarity between two or more prediction signals. One or more other similarity determinations may be performed to determine whether to skip BDOF.

[0137] BDOF refinement of a CU (eg, as described in Equation (2)) may include motion refinement in both horizontal and vertical directions. The motion refinement parameter (v x ,v y ). For example, the motion refinement parameters (v ) can be derived using a least squares technique using prediction. x ,v y ). For example, local gradient information from two or more reference pictures can be used for dual prediction. Motion refinement parameters can be derived.

[0138] A simplified gradient derivation method may be used for BDOF. As described herein, if the horizontal and vertical components of a motion vector associated with a CU include non-integer (e.g., fractional) motion components, then BDOF may include eight filtering processes. The number of filtering processes may be reduced to, for example, four filtering processes. The number of memory accesses, multiplications, and / or additions may be reduced.

[0139] For example, whether to perform BDOF refinement may be determined before performing BDOF refinement.The determination of whether to skip BDOF may be based on one or more characteristics associated with the CU (eg, coding mode and / or block size).

[0140] Directed BDOF refinement may be performed on a CU. For example, directed BDOF refinement may include performing BDOF refinement in a certain direction (e.g., a vertical direction or a horizontal direction). Gradient-based analysis (e.g., gradient calculation) may be performed, for example, for a dual-predicted CU. As described herein, gradient calculation may be performed at one or more levels, for example, including: CU level, sub-CU level, block level, sub-block level, etc. Based on the gradient calculation, one or more of the following may be applied: BDOF refinement may be skipped, directed BDOF refinement may be performed in the horizontal direction, directed BDOF refinement may be performed in the vertical direction, and BDOF refinement may be performed in both the horizontal and vertical directions.

[0141] As described herein, BDOF refinement may include the performance of gradient calculation. The gradient calculation process may include, for example, using a motion vector associated with a reference picture of the CU to derive a horizontal and / or vertical gradient for the reference picture. The motion vector associated with the reference picture may include a motion component. If, for example, the motion vector component in another direction (e.g., the vertical direction) is a non-integer (e.g., fractional) motion component, the gradient calculation in a certain direction may include applying a sample interpolation filter. For example, a gradient filter may be applied after the sample interpolation filter. The sample interpolation filter may include a low-pass filter, and the gradient filter may include a high-pass filter, which may be used for gradient derivation.

[0142] Gradients may be calculated using sample values ​​at integer positions. Sample values ​​at integer positions of a reference CU may be identified. For example, if a motion vector component in that direction includes a non-integer (e.g., fractional) motion component, sample values ​​at integer positions may be used to approximate sample values ​​at fractional positions. Gradient calculations may include applying a gradient filter in a particular direction. Sample interpolation filters may not be applied. A motion vector may be referred to as an MV (MV x , MV y), and may be associated with a reference picture R. k may be the number of bits represented by the fractional value for the motion vector component. The number of gradient filters defined for the fractional position may be represented as n (e.g., 16 as illustrated in Table 1). n may be no greater than (2 k ). The position of a CU (eg, a current bi-predicted CU in a current picture) may be represented as P(P x , P y ).

[0143] A horizontal gradient of a CU (e.g., a reference CU) associated with a motion vector MV may be derived. One or more of the following may apply. An integer position of the reference CU may be identified in a reference picture associated with the CU. The integer position of the reference CU may include P'(P'x, P'y), where P'x=Px+(MVx>>k) and P'y=Py+(MV y >>k). The phase of the gradient filter can be identified (eg, horizontal fractional position) (eg, phase = ((MV x )&(2 k -1)) / (2 k / n)). The phase may be used to identify a gradient filter that may be defined in Table 1. To compute a horizontal gradient, a gradient filter may be applied (eg, in a horizontal direction) to a sample position in a reference CU located at P' (P'x, P'y) in a reference picture R.

[0144] A vertical gradient of a CU (e.g., a reference CU) associated with a motion vector MV may be derived. One or more of the following may apply. The integer position of the reference CU may be identified in a reference picture associated with the CU. The integer position of the reference CU may include P'(P'x, P'y), where P'x=Px+(MVx>>k) and P'y=Py+(MVy>>k). A phase of a gradient filter (e.g., vertical fractional position) may be identified (e.g., phase=((MVy)&(2 k -1)) / (2 k / n)). The phase may be used to identify a gradient filter, which may be defined in Table 1. To compute a vertical gradient, the gradient filter may be applied (eg, in a vertical direction) to a sample position in the reference CU located at P' (P'x, P'y) in the reference picture R.

[0145] The techniques described herein may be applied to derive other sets of horizontal and vertical gradients (e.g., derive the sets of horizontal and vertical gradients from reference picture lists L0 and / or L1). For example, the techniques described herein may be applied to other motion vectors associated with a bi-predicted CU (e.g., the second motion vector) to derive other sets of horizontal and vertical gradients. If the motion vectors associated with a CU (e.g., both motion vectors associated with the bi-predicted CU) include non-integer (e.g., fractional) motion components, the number of filters applied to samples during BDOF may be reduced (e.g., 4 gradient filters may be applied to each sample).

[0146] As described herein, sample interpolation filters may be skipped, for example, during gradient derivation. A gradient filter (e.g., one of the 16 filters defined in Table 1) may be applied, for example, to integer positions associated with a reference CU during gradient derivation. As described herein, a gradient filter is applied to reference samples at integer positions of a reference CU, which may approximate fractional positions of samples in the reference CU (e.g., if a motion vector includes a non-integer motion component). A filter to be used for a CU may be identified based on a motion vector associated with the CU (e.g., each CU). The number of gradient filters (e.g., n) applied during gradient derivation may be reduced. The number of gradient filters may be less than the number of possible fractional positions of motion vector components. For example, a single gradient filter may be used during gradient derivation. A gradient filter may be defined (e.g., predefined), such as a gradient filter defined at position 0 in Table 1. If a single gradient filter is used during gradient derivation, the gradient filter may not be stored and a phase for the gradient filter may not be calculated.

[0147] The BDOF process may be conditionally performed based on one or more characteristics associated with the CU (e.g., coding mode and / or size). For example, BDOF may or may not be performed based on one or more characteristics associated with the CU (e.g., coding mode and / or size). A CU may be associated with one or more inter-coding modes. For example, certain coding modes may support sub-CU (e.g., or sub-block) prediction, such as: ATMVP / STMVP, FRUC bilateral, FRUC template matching, affine, etc. A sub-CU with separate motion information (e.g., a sub-block within a CU) may be referred to as a PU. For example, a sub-CU may be used in BDOF to improve inter-prediction efficiency. For example, a sub-CU may provide a motion field (e.g., a refined motion field) for the sub-CU. Inter-coding modes supporting sub-CU prediction may have different characteristics, respectively. One or more of the following may be applied. ATMVP / STMVP may derive the motion field of the current CU from spatial neighboring blocks, temporal neighboring blocks, and / or spatial and temporal neighboring blocks. FRUC bilateral and template matching may derive motion information in a manner similar to motion search. For example, FRUC bilateral and template matching can derive motion information at the decoder.

[0148] BDOF (e.g., BDOF refinement) may be conditionally performed based on a coding mode associated with a CU. For example, BDOF may be skipped based on an inter-coding mode associated with a CU (e.g., a sub-CU and / or PU). An inter-coding mode associated with a CU may be identified. The inter-coding mode may support sub-CU prediction. If the inter-coding mode associated with a CU supports sub-CU prediction, BDOF refinement may be skipped (e.g., disabled for the CU).

[0149] As described herein, BDOF may or may not be performed based on a size associated with a CU (e.g., a sub-CU and / or a PU). For sub-CU inter mode, if the PU size is large (e.g., larger than a threshold, such as 32×32), the coding region may be in a specific motion region (e.g., a smooth motion region). If the PU size is small (e.g., smaller than a threshold, such as 32×64), the motion in the coding region may be complex. If a CU (e.g., of a current bi-predicted CU) is coded using a sub-block inter mode and the PU size is large (e.g., larger than a threshold that may be predefined), the BDOF process (e.g., BDOF refinement) may be skipped. If the PU size is small (e.g., not larger than a threshold that may be predefined), the BDOF process may be applied. Thresholds for sub-CU modes (e.g., each sub-CU mode) may be signaled in a sequence parameter set, a picture parameter set, a slice header, or the like.

[0150] For example, the coding mode and / or PU size may be identified to determine whether to perform BDOF. The similarity between two or more prediction signals may not be determined, such as SAD calculations. The techniques described herein (for example) regarding determining whether to perform BDOF (e.g., based on the coding mode and / or PU size) may be combined with methods based on other signal characteristics. For example, the techniques may include determining whether to request a BDOF process for the current CU. The device may determine whether to perform BDOF (e.g., using one or more of the techniques described herein). For example, BDOF may be conditionally performed based on PU size and / or coding mode.

[0151] Directed BDOF refinement may be performed. For example, BDOF refinement may be performed in a specific direction (e.g., horizontal or vertical) based on gradient difference calculations. BDOF refinement may be performed for: samples (e.g., per sample), blocks, CUs, and / or sub-CUs (e.g., sub-blocks, such as 4x4). Fig.11 1 shows a diagram of BDOF refinement for a CU. For example, the horizontal gradient and vertical gradient for a CU may be calculated using the techniques described herein. One or more BDOF refinement parameters (v x ,v y ). The derived BDOF refinement parameters may be used, for example, by applying equation (2) to generate a BDOF-refined prediction for the sub-CU. The characteristics of the sub-CUs may be different. If the horizontal gradient of the reference picture in list 0 (e.g., ) is similar to the horizontal gradient of the reference picture in Listing 1 (e.g., ), then BDOF refinement may not provide improvement in the horizontal direction. BDOF refinement may be performed in one or more directions or not.

[0152] The determination of whether to perform BDOF (e.g., directional BDOF) may be based on gradient analysis (e.g., gradient differences) performed in multiple (e.g., two) directions. For example, dGx (e.g., horizontal gradient differences) and dGy (e.g., vertical gradient differences) may be defined for a CU (e.g., or a sub-CU) using equations (16) and (17), respectively. dGx and dGy may be used to determine whether gradients in reference pictures associated with the CU (e.g., or a sub-CU) are similar, and / or to determine the direction in which to perform directional BDOF (e.g., horizontal or vertical).

[0153]

[0154] dGx and dGy may be compared to a threshold (e.g., each may be compared to a threshold), for example, to determine whether to perform directional BDOF. The threshold may be static and / or variable. If dGx is less than or equal to a threshold (e.g., a static threshold (e.g., a predefined threshold)), then the horizontal gradients of the CU (e.g., or a sub-CU) in a reference picture associated with the CU may be similar. If dGy is less than or equal to a threshold (e.g., a static threshold (e.g., a predefined threshold)), then the vertical gradients of the CU (e.g., or a sub-CU) in a reference picture associated with the CU may be similar. One or more of the following may apply. If the gradients in the horizontal and vertical directions are similar, then BDOF refinement may be skipped for the CU (e.g., or a sub-CU). Bi-prediction (e.g.) may be applied to generate a prediction for the CU. If the vertical gradients are similar and the horizontal gradients are not similar, then a horizontal BDOF refinement parameter v may be derived, for example, using equation (18) x And directional BDOF can be performed in the horizontal direction. If the vertical gradients are similar and the horizontal gradients are not similar, the vertical BDOF refinement parameter v can be skipped. y Derivation of .

[0155]

[0156] The vertical gradients may be dissimilar, while the horizontal gradients may be similar. The vertical BDOF refinement parameter v may be derived, for example, using equation (19): y , and directional BDOF refinement can be performed in the vertical direction. Horizontal BDOF refinement parameter v can be skipped x Derivation of .

[0157]

[0158] The gradients in the vertical and horizontal directions may not be similar. If the gradients in the vertical and horizontal directions are not similar, BDOF refinement may be performed, for example, in the horizontal and vertical directions.

[0159] As described herein, dGx and dGy may be compared (e.g., may each be compared to a variable threshold value (e.g., a dynamic threshold value), for example, to determine whether to perform directional BDOF. For example, dGx may be compared to a first threshold value, and dGy may be compared to a second threshold value. The first threshold value and the second threshold value may be variable. For example, the first threshold value may be dGy (e.g., a horizontal gradient difference), and the second threshold value may be dGx (e.g., a vertical gradient difference). If dGx is greater than the first threshold value, directional BDOF may be performed in the vertical direction. If dGy is greater than the second threshold value, directional BDOF may be performed in the horizontal direction.

[0160] Fig.12A diagram is shown for directional BDOF refinement, which may be used for a CU (e.g., or a sub-CU). A gradient difference (e.g., dGx and dGy) may be calculated for a CU or a sub-CU (e.g., each CU or sub-CU). Based on the gradient difference, one or more of the following may be applied: BDOF refinement may not be performed (e.g., skipped), BDOF refinement may be performed, directional BDOF may be performed in a vertical direction, or directional BDOF may be performed in a horizontal direction. The threshold may be signaled at a sequence parameter set, a picture parameter set, a slice header, etc. As described herein, the threshold may be variable (e.g., referenced to Fig.12 , TH1 may include dGy and / or TH2 may include dGx).

[0161] The gradient difference may be calculated over the entire CU, for example, using equations (20) and (21) (e.g., at the beginning of the PU process). A determination may be made at the CU, sub-CU, and / or PU level in which direction to perform directional BDOF refinement. One or more of the following may apply: BDOF refinement may be skipped at the CU, sub-CU, and / or PU level; directional BDOF refinement may be performed in the horizontal direction at the CU, sub-CU, and / or PU level, and directional BDOF refinement may be performed in the vertical direction at the CU, sub-CU, and / or PU level; BDOF refinement may be performed at the CU, sub-CU, and / or PU level; and the like. The threshold may be scaled, for example, based on the area of ​​the CU, sub-CU, and / or PU (e.g., because dGx or dGy comparisons may not be performed for sub-blocks of equal size).

[0162]

Claims

1. An apparatus for video decoding, comprising one or more processors, wherein the one or more processors are configured to: obtaining a size of a decoding block, wherein the decoding block comprises sub-blocks; comparing the size of the decoded block with a value; BDOF is performed on the sub-block of the coding block based on similarities associated with the sub-block.

2. A method for video decoding, comprising: obtaining a size of a decoding block, wherein the decoding block comprises sub-blocks; comparing the size of the decoding block with a value; BDOF is performed on the sub-block of the coding block based on similarities associated with the sub-block.

3. An apparatus for video encoding, comprising one or more processors, wherein the one or more processors are configured to: obtaining a size of a decoding block, wherein the decoding block comprises sub-blocks; comparing the size of the decoding block with a value; BDOF is performed on the sub-block of the coding block based on similarities associated with the sub-block.

4. A method for video encoding, comprising: obtaining a size of a decoding block, wherein the decoding block comprises sub-blocks; comparing the size of the decoding block with a value; BDOF is performed on the sub-block of the coding block based on similarities associated with the sub-block.

Images