Affine motion estimation for affine model-based video coding

Abstract

Systems, methods, and instrumentalities are disclosed herein for affine motion estimation for affine model-based video coding. Comprising one or more motion vectors (MVs) may be derived for a first coded block. The MV may be a control point MV (CPMV), and the MV may be derived by performing affine motion estimation (ME) associated with the first coded block. The first set of MVs may be added to a list of recently estimated MVs. Heads of the most recently estimated list of MVs may be set as a first set of MVs. The most recently estimated list of MVs may be empty, or may include one or more previously added sets of MVs.

Description

[0001] This divisional application is a divisional application with application date of September 20, 2019, application number 201980072083.6, and invention name “Affine motion estimation for affine model-based video decoding”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit of U.S. Provisional Application No. 62 / 734,783, filed on September 21, 2018, and U.S. Provisional Application No. 62 / 812,979, filed on March 2, 2019, the contents of which are incorporated herein by reference. Background Art

[0004] Video decoding systems can be used to compress digital video signals, for example, to reduce the storage and / or transmission bandwidth required for these signals. Video decoding systems can include block-based, wavelet-based and / or object-based systems. A block-based hybrid video decoding system can be deployed. Summary of the invention

[0005] Disclosed herein are systems, methods, and means for motion estimation for video coding. A starting point for motion estimation may be selected based at least in part on one or more MV sets in a most recently estimated MV list. The selection of the starting point for motion estimation may be performed by an encoder and / or a decoder. The encoder and / or decoder may be a wireless transmit / receive unit (WTRU). For example, a first motion vector (MV) set including one or more MVs may be derived for a first coding block. The MV may be a control point MV (CPMV), and the MV may be derived by performing affine motion estimation (ME) associated with the first coding block. The first MV set may be added to the most recently estimated MV list. The header of the most recently estimated MV list may be set to the first MV set. The most recently estimated MV list may be empty, or may include one or more previously added MV sets. For example, the most recently estimated MV list may include one or more MV sets for searching for motion estimation, wherein each search MV set in the most recently estimated MV list is an MV set selected for a neighboring block of a second coding block partitioned by a different tree, a sibling block of the second coding block, a parent block of the second coding block, or a child block of a parent block of the second coding block. The most recently estimated MV list may be, for example, a first-in, first-out (FIFO) list having a size (e.g., which may be four). The starting point for motion estimation may be selected for the second coding block based at least in part on one or more MV sets and one or more MV predictors in the most recently estimated MV list. The second coding block may be encoded based on the selected starting point. For example, an ME (e.g., an affine ME) associated with the second coding block is based on the selected starting point.

[0006] The first tree partitioning may partition the parent block of the second coding block into the second coding block and at least one sibling block. The second tree partitioning may partition the parent block of the second coding block into one or more (e.g., two) child blocks. The first coding block may be a neighboring block of the second coding block, a sibling block of the second coding block, a parent block of the second coding block, or a child block of the parent block partitioned by the second tree. If the second coding block is the last block in the first tree partitioning, the head of the MV list may be set to the MV set associated with the parent block.

[0007] The most recently estimated MV list may be associated with a maximum size. For example, after deriving MVs for a coding block (e.g., a first coding block), it may be determined whether the size of the most recently estimated MV list is equal to the maximum size. If the most recently estimated MV list is equal to the maximum size, the least-recently-estimated set of MVs may be removed from the most recently estimated MV list before adding the MVs for the coding block.

[0008] An MV set and / or an MVP set may be determined. The MV set may include, for example, one or more (e.g., three) MVs (e.g., control point MVs). For example, the MV set may be represented as {MV0, MV1, MV2}. The MVP set may include, for example, one or more (e.g., three) MVPs (e.g., affine MVPs). For example, the MV set may be represented as {MVP0, MVP1, MVP2}. It may be determined whether the first MV (e.g., MV1) of the MV set is equal to the first MVP (e.g., MVP1) of the MVP set. If the first MV is not equal to the first MVP, the MV-MVP combination set is generated by replacing the first MV in the MV set with the first MVP. The bit cost (e.g., ME cost) of the MV-MVP combination set may be determined. For example, the bit cost may be determined based on the prediction error and / or the corresponding bit cost for each MV in the MV-MVP combination set. Based on the bit cost of the MV-MVP combination set, the MV set or the MV-MVP combination set may be selected as the best set. The bit cost of the MV-MVP combination set may be compared with the bit cost of the MV set. For example, if the bit cost of the MV-MVP combination set is lower than the bit cost of the MV set, the MV-MVP combination set may be selected. Otherwise, an MV collection may be selected.

[0009] It may be determined whether a second MV (e.g., MV2) of the MV set is equal to a second MVP (e.g., MVP2) of the MVP set. If the second MV is not equal to the second MVP, the second MV may be replaced with the second MVP in the MV-MVP combination set to generate a second MV-MVP combination set (e.g., which may be represented as {MV0, MVP1, MVP2}). A bit cost (e.g., ME cost) of the second MV-MVP combination set may be determined. The bit cost of the second MV-MVP combination set may be compared with the cost of the current best set, and an updated best set may be selected. For example, if the bit cost of the second MV-MVP combination set is lower than the bit cost of the current best set, the second MV-MVP combination set may be selected as the updated best set. Otherwise, the current best set may be selected as the updated best set. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1A is a system diagram illustrating an example communication system.

[0011] Figure 1B It shows that Figure 1A A system diagram of an example wireless transmit / receive unit (WTRU) for use within a communication system is shown.

[0012] Figure 1C It shows that it can be Figure 1A A system diagram of an exemplary radio access network (RAN) and an exemplary core network (CN) for use within a communication system is shown.

[0013] Figure 1D It is a diagram that can be Figure 1A System diagram of another example RAN and another example CN used within the communication system illustrated in FIG.

[0014] Figure 2 Diagram showing an example of a block-based video encoder.

[0015] Figure 3 A sample block diagram showing a video decoder.

[0016] Figure 4 An example block partitioning in a multi-type tree structure is shown.

[0017] Figure 5 An example of a four-parameter affine pattern is shown.

[0018] Figure 6 An example of a six-parameter affine pattern is shown.

[0019] Figure 7 An example of an affine merge candidate is shown.

[0020] Figure 8 An example of motion vector derivation at the control points of an affine motion model is shown.

[0021] Fig. 9 An example affine motion vector predictor construction is shown.

[0022] Fig.10 Example motion model selection for a coding unit is shown.

[0023] Fig.11 (a)-(f) show example affine MV list updates, where the current head of the list is shaded.

[0024] Fig.12 Examining an example additional set of combined MVs after affine motion estimation is shown.

[0025] Fig.13 An example gradient calculation for an 8x8 coding unit (CU) is shown.

[0026] Fig.14 An example gradient calculation for an 8×8 CU is shown. DETAILED DESCRIPTION

[0027] A more detailed understanding may be obtained from the following description given by way of example in conjunction with the accompanying drawings, in which:

[0028] Figure 1A 1 is a diagram illustrating an example communication system 100 in which one or more disclosed embodiments may be implemented. The communication system 100 may be a multiple access system that provides content such as voice, data, video, messaging, broadcast, etc. to multiple wireless users. The communication system 100 may enable multiple wireless users to access such content by sharing system resources including wireless bandwidth. For example, the communication system 100 may employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single carrier FDMA (SC-FDMA), zero tail unique word DFT-spread OFDM (ZT UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block filtered OFDM, filter bank multi-carrier (FBMC), etc.

[0029] like Figure 1AAs shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, public switched telephone network (PSTN) 108, Internet 110 and other networks 112, but it should be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network components. Each of the WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, 102d (any of which may be referred to as a “station” and / or “STA”) may be configured to transmit and / or receive wireless signals and may include user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a subscription-based unit, a pager, a cellular phone, a personal digital assistant (PDA), a smart phone, a laptop, a netbook, a personal computer, a wireless sensor, a hotspot or MiFi device, an Internet of Things (IoT) device, a watch or other wearable device, a head-mounted display (HMD), a vehicle, a drone, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in an industrial and / or automated process chain environment), consumer electronic devices, devices operating on a commercial and / or industrial wireless network, etc. Any of the WTRUs 102a, 102b, 102c, and 102d may be interchangeably referred to as a UE.

[0030] The communication system 100 may also include a base station 114a and / or a base station 114b. Each of the base stations 114a, 114b may be any type of device that is configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks, such as the CN 106 / 115, the Internet 110, and / or other networks 112. By way of example, the base stations 114a, 114b may be a base transceiver station (BTS), a Node B, an eNode B, a Home Node B, a Home eNode B, a gNB, an NR Node B, a site controller, an access point (AP), a wireless router, and the like. While each of the base stations 114a, 114b is depicted as a single component, it will be appreciated that the base stations 114a, 114b may include any number of interconnected base stations and / or network components.

[0031] The base station 114a may be part of the RAN 104 / 113, which may also include other base stations and / or network components (not shown), such as a base station controller (BSC), a radio network controller (RNC), a relay node, etc. The base station 114a and / or the base station 114b may be configured to send and / or receive wireless signals on one or more carrier frequencies (which may be referred to as cells (not shown)). These frequencies may be in a licensed spectrum, an unlicensed spectrum, or a combination of licensed and unlicensed spectrums. A cell may provide coverage of wireless services to a specific geographic area, which may be relatively fixed or may change over time. The cell may be further divided into cell sectors. For example, a cell associated with the base station 114a may be divided into three sectors. Therefore, in one embodiment, the base station 114a may include three transceivers, i.e., one transceiver for each sector of the cell. In an embodiment, the base station 114a may employ multiple-input multiple-output (MIMO) technology, and may use multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in a desired spatial direction.

[0032] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d via an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0033] More specifically, as described above, the communication system 100 may be a multiple access system and may employ one or more channel access schemes such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, the base station 114a in the RAN 104 / 113 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish an air interface 115 / 116 / 117 using Wideband CDMA (WCDMA). WCDMA may include communication protocols such as High Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High Speed ​​UL Packet Access (HSUPA).

[0034] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 116 using Long Term Evolution (LTE) and / or Advanced LTE (LTE-A) and / or Advanced LTE Pro (LTE-A Pro).

[0035] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as NR radio access, which may establish the air interface 116 using new radio (NR).

[0036] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may together implement LTE radio access and NR radio access, for example using dual connectivity (DC) principles. Thus, the air interface used by the WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations, such as eNBs and gNBs.

[0037] In other embodiments, the base station 114a and the wireless transmit / receive units 102a, 102b, and 102c may implement wireless technologies, such as IEEE802.11 (i.e., Wireless Fidelity (WiFi), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-D0, International Standard 2000 (IS-2000), International Standard 95 (IS-95), International Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data Rates for GSM Evolution (EDGE), GSM EDGE (GERAN), etc.

[0038] Figure 1AThe base station 114b in the example may be, for example, a wireless router, a Home NodeB, a Home eNodeB, or an access point, and may utilize any suitable RAT to facilitate wireless connectivity in a local area, such as a business location, a residence, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a road, and the like. In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement radio technologies such as IEEE 802.11 to establish a wireless local area network (WLAN). In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement radio technologies such as IEEE 802.15 to establish a wireless personal area network (WPAN). In another embodiment, the base station 114b and the WTRUs 102c, 102d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE-A Pro, NR, etc.) to establish a picocell or a femtocell. Figure 1A As shown, the base station 114b may be directly connected to the Internet 110. Therefore, the base station 114b may not need to access the Internet 110 via the CN 106 / 115.

[0039] RANAN 104 / 113 may communicate with CN 106 / 115, which may be any type of network that provides voice, data, applications, and / or voice over Internet Protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have different quality of service (QoS) requirements, such as different throughput requirements, latency requirements, fault tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. CN 106 / 115 may provide call control, billing services, mobile location-based services, prepaid calls, Internet connectivity, video distribution, etc., and / or perform advanced security functions (e.g., user authentication). Although Figure 1A Although not shown, it will be appreciated that the RAN 104 / 113 and / or the CN 106 / 115 may communicate directly or indirectly with other RANs that employ the same RAT or a different RAT as the RAN 104 / 113. For example, in addition to being connected to the RAN 104 / 113, which may utilize NR radio technology, the CN 106 / 115 may also communicate with another RAN (not shown) employing GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technology.

[0040] The CN 106 / 115 may also act as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and / or other networks 112. The PSTN 108 may include a circuit-switched telephone network that provides plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the Transmission Control Protocol (TCP), the User Datagram Protocol (UDP), and / or the Internet Protocol (IP) in the TCP / IP Internet protocol suite. The networks 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, the networks 112 may include another CN connected to one or more RANs, which may use the same RAT as the RAN 104 / 113 or a different RAT.

[0041] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communication system 100 may include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks via different wireless links). Figure 1A The illustrated WTRU 102c may be configured to communicate with the base station 114A, which may employ a cellular-based radio technology, and with the base station 114b, which may employ an IEEE 802 radio technology.

[0042] Figure 1B is a system diagram illustrating an example WTRU 102. Figure 1B As shown, the WTRU 102 may include, among other things, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keyboard 126, a display / touchpad 128, non-removable memory 130, a removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and / or other peripherals 138. It will be appreciated that the WTRU 102 may include any sub-combination of the foregoing elements while remaining consistent with an embodiment.

[0043] The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor 118 may perform signal coding, data processing, power control, input / output processing, and / or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit / receive element 122. Although Figure 1B The processor 118 and the transceiver 120 are depicted as separate components, but it will be appreciated that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.

[0044] The transmit / receive component 122 may be configured to transmit or receive signals to or from a base station (e.g., base station 114a) via an air interface 116. For example, in one embodiment, the transmit / receive component 122 may be an antenna configured to transmit and / or receive RF signals. In one embodiment, the transmit / receive component 122 may be an emitter / detector configured to transmit and / or receive, for example, IR, UV, or visible light signals. In yet another embodiment, the transmit / receive component 122 may be configured to transmit and / or receive both RF and optical signals. It should be understood that the transmit / receive component 122 may be configured to transmit and / or receive any combination of wireless signals.

[0045] Although the transmit / receive element 122 is Figure 1B Although depicted as a single component in the embodiment, the WTRU 102 may include any number of transmit / receive components 122. More specifically, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive components 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.

[0046] The transceiver 120 may be configured to modulate signals to be transmitted by the transmit / receive element 122 and to demodulate signals received by the transmit / receive element 122. As described above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers to enable the WTRU 102 to communicate via multiple RATs (e.g., NR and IEEE 802.11).

[0047] The processor 118 of the WTRU 102 may be coupled to and may receive user input data from a speaker / microphone 124, a keyboard 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light emitting diode (OLED) display unit). The processor 118 may also output user data to the speaker / microphone 124, the keyboard 126, and / or the display / touchpad 128. In addition, the processor 118 may access information from and store data in any type of suitable memory, such as a non-removable memory 130 and / or a removable memory 132. The non-removable memory 130 may include a random access memory (RAM), a read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from and store data in a memory that is not physically located on the WTRU 102, such as on a server or a home computer (not shown).

[0048] The processor 118 may receive power from the power source 134, and may be configured to distribute and / or control power to other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0049] The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to or in lieu of the information from the GPS chipset 136, the WTRU 102 may receive location information from a base station (e.g., base stations 114a, 114b) over the air interface 116 and / or determine its location based on the timing of signals received from two or more neighboring base stations. It will be appreciated that the WTRU 102 may acquire location information by any suitable location-determination method while remaining consistent with an embodiment.

[0050] The processor 118 may also be coupled to other peripherals 138, which may include one or more software and / or hardware modules that provide additional features, functionality, and / or wired or wireless connectivity. For example, the peripherals 138 may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (for photos and / or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, module, a frequency modulation (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, a virtual reality and / or augmented reality (VR / AR) device, an activity tracker, etc. The peripheral device 138 may include one or more sensors, which may be one or more of a gyroscope, an accelerometer, a Hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor, a geolocation sensor, an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, and / or a humidity sensor.

[0051] The WTRU 102 may include a full-duplex radio for which transmission and reception of some or all signals (e.g., signals associated with specific subframes for UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous. The full-duplex radio may include an interference management unit 139 that reduces and / or substantially eliminates self-interference by means of hardware (e.g., chokes) or by signal processing by a processor (e.g., a separate processor (not shown) or by the processor 118). In an embodiment, the WTRU 102 may include a half-duplex radio for transmitting and receiving some or all signals (e.g., signals associated with specific subframes for UL (e.g., for transmission) or downlink (e.g., for reception)).

[0052] Figure 1C 1 is a system diagram showing the RAN 104 and the CN 106 in accordance with an embodiment. As described above, the RAN 104 may employ an E-UTRA radio technology and communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 104 may also be in communication with the CN 106.

[0053] The RAN 104 may include eNode-Bs 160a, 160b, 160c, though it will be appreciated that the RAN 104 may include any number of eNode-Bs while remaining consistent with an embodiment. Each of the eNode-Bs 160a, 160b, 160c may include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the eNode-Bs 160a, 160b, 160c may implement MIMO technology. Thus, for example, the eNode-B 160a may use multiple antennas to transmit wireless signals to and / or receive wireless signals from the WTRU 102a.

[0054] Each of the eNodeBs 160a, 160b, 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and / or DL, etc. Figure 1C As shown in FIG. , the eNode-Bs 160a, 160b, 160c may communicate with one another via an X2 interface.

[0055] Figure 1C The CN 106 shown in FIG. 1 may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (or PGW) 166. Although each of the foregoing components is depicted as part of the CN 106, it will be understood that any of these components may be owned and / or operated by an entity other than a CN operator.

[0056] The MME 162 may be connected to each of the eNode-Bs 162a, 162b, 162c in the RAN 104 via an S1 interface and may serve as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, bearer activation / deactivation, selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, 102c, etc. The MME 162 may provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other wireless technologies, such as GSM and / or WCDMA.

[0057] The SGW 164 may be connected to each of the eNode-Bs 160a, 160b, 160c in the RAN 104 via an S1 interface. The SGW 164 may generally route and forward user data packets to / from the WTRUs 102a, 102b, 102c. The SGW 164 may perform other functions, such as anchoring the user plane during inter-eNode-B handovers, triggering paging when DL data is available for the WTRUs 102a, 102b, 102c, managing and storing the contexts of the WTRUs 102a, 102b, 102c, and the like.

[0058] The SGW 164 may be connected to the PGW 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0059] The CN 106 may facilitate communications with other networks. For example, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the CN 106 may include, or may be in communication with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between the CN 106 and the PSTN 108. In addition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0060] Although the WTRU Figures 1A-1D Although described as a wireless terminal, it is contemplated that in certain representative embodiments such a terminal may use (eg, temporarily or permanently) a wired communications interface with a communications network.

[0061] In a representative embodiment, other network 112 may be a WLAN.

[0062] A WLAN in an infrastructure basic service set (BSS) mode may have an access point (AP) for a BSS and one or more stations (STAs) associated with the AP. The AP may access or dock to a distribution system (DS) or send traffic into and / or out of another type of wired / wireless network carrying the BSS. Traffic initiated from outside the BSS and destined for the STA may be reached through the AP and may be delivered to the STA. Traffic initiated from the STA and destined for a destination outside the BSS may be sent to the AP to be delivered to the corresponding destination. Traffic between STAs within the BSS may be sent through the AP, for example, where the source STA may send traffic to the AP, and the AP may deliver traffic to the destination STA. Traffic between STAs within the BSS may be considered and / or referred to as point-to-point traffic. The point-to-point traffic may be sent between the source STA and the destination STA (e.g., directly between the source STA and the destination STA) using a direct link setup (DLS). In certain representative embodiments, the DLS may use 802.11e DLS or 802.11z tunnel DLS (TDLS). A WLAN using an independent BSS (IBSS) mode may not have an AP, and STAs (eg, all STAs) within or using the IBSS may communicate directly with each other. The IBSS communication mode may sometimes be referred to herein as an "ad hoc" communication mode.

[0063] When using the 802.11ac infrastructure operating mode or a similar operating mode, the AP can transmit beacons on a fixed channel, such as a primary channel. The primary channel can be a fixed width (e.g., a 20MHz wide bandwidth) or a width dynamically set via signaling. The primary channel can be a working channel of the BSS and can be used by the STA to establish a connection with the AP. In certain representative embodiments, such as in an 802.11 system, carrier sensing multiple access (CSMA / CA) with collision avoidance can be implemented. For CSMA / CA, STAs including the AP (e.g., each STA) can sense the main channel. If the main channel is sensed / detected and / or determined to be busy by a specific STA, the specific STA can back off. One STA (e.g., only one station) can transmit at any given time in a given BSS.

[0064] A high throughput (HT) STA may communicate using a 40 MHz wide channel, for example, by combining a primary 20 MHz channel with an adjacent or non-adjacent 20 MHz channel to form a 40 MHz wide channel.

[0065] Very high throughput (VHT) STA can support channels with a width of 20MHz, 40MHz, 80MHz and / or 160MHz. 40MHz and / or 80MHz channels can be formed by combining continuous 20MHz channels. A 160MHz channel can be formed by combining 8 continuous 20MHz channels, or by combining two non-continuous 80MHz channels, which can be referred to as an 80+80 configuration. For the 80+80 configuration, after channel coding, the data can pass through a segment parser that can divide the data into two streams. Each stream can be subjected to inverse fast Fourier transform (IFFT) processing and time domain processing, respectively. The stream can be mapped onto two 80MHz channels, and the data can be transmitted by the transmitting STA. At the receiver of the receiving STA, the operation of the above 80+80 configuration can be opposite, and the combined data can be sent to the medium access control (MAC).

[0066] The operating mode below 1 GHz is supported by 802.11af and 802.11ah. The channel operating bandwidth and carrier are reduced in 802.11af and 802.11ah relative to the channel operating bandwidth and carrier used in 802.11n and 802.11ac. 802.11af supports 5MHz, 10MHz and 20MHz bandwidths in the TV White Space (TVWS) spectrum, while 802.11ah supports 1MHz, 2MHz, 4MHz, 8MHz and 16MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11ah can support metering type control / machine type communication, such as MTC devices in macro coverage areas. MTC devices may have certain capabilities, for example, limited capabilities including support for certain and / or limited bandwidths (e.g., only support for certain and / or limited bandwidths). MTC devices may include batteries with battery life above a threshold (e.g., to maintain very long battery life).

[0067] A WLAN system that can support multiple channels and channel bandwidths (e.g., 802.11n, 802.11ac, 802.11af, and 802.11ah) includes a channel that can be designated as a primary channel. The primary channel may have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel may be set and / or limited by the STA among all STAs operating in the BSS, and the STA supports the minimum bandwidth operating mode. In the example of 802.11ah, for STAs (e.g., MTC-type devices) that support (e.g., only support) a 1MHz mode, the primary channel may be 1MHz wide, even if the AP and other STAs in the BSS support 2MHz, 4MHz, 8MHz, 16MHz, and / or other channel bandwidth operating modes. Carrier sensing and / or network allocation vector (NAV) settings may depend on the state of the primary channel. If the primary channel is busy, for example, because a STA (which only supports a 1MHz operating mode) transmits to the AP, the entire available band may be considered busy even if most of the bands in the band remain idle and available.

[0068] In the United States, the available frequency band that 802.11ah can use is from 902MHz to 928MHz. In South Korea, the available frequency band is from 917.5MHz to 923.5MHz. In Japan, the available frequency band is from 916.5MHz to 927.5MHz. Depending on the country code, the total bandwidth available for 802.11ah is 6MHz to 26MHz.

[0069] Figure 1D1 is a system diagram showing the RAN 113 and the CN 115 according to an embodiment. As described above, the RAN 113 may communicate with the WTRUs 102a, 102b, 102c over the air interface 116 using NR radio technology. The RAN 113 may also communicate with the CN 115.

[0070] The RAN 113 may include gNBs 180a, 180b, 180c, but it should be appreciated that the RAN 113 may include any number of gNBs while remaining consistent with the embodiment. Each of the gNBs 180a, 180b, 180c may include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the gNBs 180a, 180b, 180c may implement MIMO technology. For example, the gNBs 180a, 180b may utilize beamforming to transmit signals to and / or receive signals from the gNBs 180a, 180b, 180c. Therefore, the gNB 180a may, for example, use multiple antennas to transmit wireless signals to and / or receive wireless signals from the WTRU 102a. In one embodiment, the gNBs 180a, 180b, 180c may implement carrier aggregation techniques. For example, the gNB 180a may transmit multiple component carriers (not shown) to the WTRU 102a. A subset of these component carriers may be on an unlicensed spectrum, while the remaining component carriers may be on a licensed spectrum. In an embodiment, the gNBs 180a, 180b, 180c may implement coordinated multi-point (CoMP) techniques. For example, the WTRU 102a may receive coordinated transmissions from the gNB 180a and gNB 180b (and / or gNB 180c).

[0071] The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using transmissions associated with scalable parameter configurations (numerology). For example, OFDM symbol spacing and / or OFDM subcarrier spacing may vary for different transmissions, different cells, and / or different portions of the wireless transmission spectrum. The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using subframes or subframes or transmission time intervals (TTIs) of varying or scalable lengths (e.g., containing different numbers of OFDM symbols and / or continuously varying absolute time lengths).

[0072] The gNBs 180a, 180b, 180c may be configured to communicate with the WTRUs 102a, 102b, 102c in a standalone configuration and / or a non-standalone configuration. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c without also accessing other RANs (e.g., the eNode-Bs 160a, 160b, 160c). In a standalone configuration, the WTRUs 102a, 102b, 102c may utilize one or more of the gNBs 180a, 180b, 180c as mobility anchors. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using signals in an unlicensed band. In a non-standalone configuration, the WTRUs 102a, 102b, 102c may communicate / connect with the gNBs 180a, 180b, 180c while also communicating / connecting with another RAN, such as the eNode-Bs 160a, 160b, 160c. For example, the WTRUs 102a, 102b, 102c may implement DC principles to communicate with one or more of the gNBs 180a, 180b, 180c and one or more of the eNode-Bs 160a, 160b, 160c substantially simultaneously. In a non-standalone configuration, the eNode-B 160a, 160b, 160c may serve as mobility anchor points for the WTRUs 102a, 102b, 102c and the gNBs 180a, 180b, 180c may provide additional coverage and / or throughput for serving the WTRUs 102a, 102b, 102c.

[0073] Each of the gNBs 180a, 180b, 180c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, support network slicing, dual connectivity, interworking between NR and E-UTRA, route user plane data to a user plane function (UPF) 184a, 184b, route control plane information to an access and mobility management function (AMF) 182a, 182b, etc. Figure 1D As shown, gNBs 180a, 180b, and 180c may communicate with each other via an Xn interface.

[0074] Figure 1DThe CN 115 shown in the figure may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one session management function (SMF) 183a, 183b, and possible data networks (DN) 185a, 185b. Although each of the aforementioned components is depicted as part of the CN 115, it will be understood that any of these components may be owned and / or operated by an entity other than the CN operator.

[0075] The AMF 182a, 182b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via an N2 interface and may serve as a control node. For example, the AMF 182a, 182b may be responsible for authenticating users of the WTRU 102a, 102b, 102c, supporting network slicing (e.g., handling different PDU sessions with different requirements), selecting a specific SMF 183a, 183b, management of registration areas, termination of NAS signaling, mobility management, and the like. The AMF 182a, 182b may use network slicing to customize the CN support of the WTRU 102a, 102b, 102c according to the type of service used by the WTRU 102a, 102b, 102c. For example, different network slices may be established for different use cases, such as services that rely on ultra-reliable low latency (URLLC) access, services that rely on enhanced massive mobile broadband (eMBB) access, services for machine type communication (MTC) access, and the like. The AMF 162 may provide a control plane function for switching between the RAN 113 and other RANs (not shown) that employ other radio technologies, such as LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies such as WiFi.

[0076] The SMF 183a, 183b may be connected to the AMF 182a, 182b in the CN 115 via the N11 interface. The SMF 183a, 183b may also be connected to the UPF 184a, 184b in the CN 115 via the N4 interface. The SMF 183a, 183b may select and control the UPF 184a, 184b, and configure service routing through the UPF 184a, 184b. The SMF 183a, 183b may perform other functions, such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, providing downlink data notification, etc. The PDU session type may be IP-based, non-IP-based, Ethernet-based, etc.

[0077] The UPF 184a, 184b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via the N3 interface, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The UPF 184, 184b may perform other functions, such as routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering downlink packets, providing mobility anchoring, and the like.

[0078] The CN 115 may facilitate communications with other networks. For example, the CN 115 may include an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) or may communicate with an IP gateway that acts as an interface between the CN 115 and the PSTN 108. In addition, the CN 115 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In an embodiment, the WTRUs 102a, 102b, 102c may be connected to a local data network (DN) 185a, 185b via an N3 interface to the UPF 184a, 184b and an N6 interface between the UPF 184a, 184b and the DN 185a, 185b, and through the UPF 184a, 184b.

[0079] Given that Figures 1A-1D and Figures 1A-1D In accordance with the corresponding description of the present invention, one or more or all of the functions described herein with respect to one or more of the following items may be performed by one or more simulation devices (not shown): WTRU 102a-d, base station 114a-b, eNodeB 160a-c, MME 162, SGW 164, PGW 166, gNB 180a-c, AMF 182a-ab, UPF184a-b, SMF 183a-b, DN185ab and / or any other device (one or more) described herein. The simulation device may be one or more devices configured to simulate one or more or all of the functions described herein. For example, the simulation device may be used to test other devices and / or simulate network and / or WTRU functions.

[0080] The simulation device can be designed to implement one or more tests of other devices in a laboratory environment and / or an operator network environment. For example, one or more simulation devices can perform one or more or all functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network in order to test other devices within the communication network. One or more simulation devices can perform one or more or all functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. The simulation device can be directly coupled to another device for testing purposes and / or can perform tests using over-the-air wireless communications.

[0081] One or more simulation devices can perform one or more functions (including all functions) without being implemented / deployed as part of a wired and / or wireless communication network. For example, the simulation device can be used in a test scenario in a test lab and / or a non-deployed (e.g., testing) wired and / or wireless communication network to enable testing of one or more components. The one or more simulation devices can be test devices. The simulation device can transmit and / or receive data using direct RF coupling and / or wireless communication with the aid of RF circuits (e.g., which can include one or more antennas).

[0082] One or more of the following may be used: one or more (e.g., 65) angular intra-prediction directions, modified coefficient coding, advanced multiple transform (AMT) + 4×4 non-separable secondary transform (NSST), affine motion model, generalized adaptive loop filter (GALF), advanced temporal motion vector prediction (ATMVP), adaptive motion vector precision, decoder-side motion vector refinement (DMVR), and / or linear model (LM) chroma mode.

[0083] A block-based hybrid video coding framework may be provided. Figure 2 A block diagram of an example block-based hybrid video encoder is given. For example, the encoder may be a WTRU. An input video signal 2 may be processed block by block. Block sizes (e.g., extended block sizes, such as coding units (CUs)) may compress high-resolution (e.g., 1080p and above) video signals. For example, a CU may contain 128×128 pixels or more. A coding tree unit CTU may be partitioned into CUs to accommodate varying local characteristics (e.g., based on quad / bin / ternary trees). A CU (e.g., each CU) may be used as a basic unit for prediction and / or transformation (e.g., in the absence of other partitioning). In a multi-type tree structure, a CTU may be partitioned by a quadtree structure. Quadtree leaf nodes (e.g., each quadtree leaf node) may be partitioned by a binary tree and / or ternary tree structure. As Figure 4As shown, there may be one or more (e.g., five) split types. For example, the split types may include quaternary splitting, horizontal binary splitting, vertical binary splitting, horizontal ternary splitting, and / or vertical ternary splitting. For (e.g., each) input video block (e.g., MB and / or CU), spatial prediction 60 and / or temporal prediction 62 may be performed. Spatial prediction (e.g., intra prediction) may use pixels from samples (e.g., reference samples) of coded neighboring blocks in a video picture / slice to predict the current video block. Spatial prediction may reduce spatial redundancy inherent in, for example, a video signal. Temporal prediction (e.g., inter prediction and / or motion compensated prediction) may use reconstructed pixels (e.g., from a coded video picture) to predict the current video block. Temporal prediction may reduce temporal redundancy inherent in, for example, a video signal. A temporal prediction signal for a CU may be signaled by one or more motion vectors (MVs). An MV may indicate the amount and / or direction of motion between a current CU and its temporal reference. If multiple reference pictures for (e.g., each) CU are supported, a reference picture index for the CU may be sent. The reference index may be used to identify which reference picture in the reference picture store 64 the temporal prediction signal may be derived from. After spatial and / or temporal prediction, a mode decision block 80 in the encoder may determine a prediction mode (e.g., an optimal prediction mode), e.g., based on rate-distortion optimization. The prediction block may be subtracted from the current video block 16 and / or the prediction residual may be decorrelated using a transform 4 and / or a quantization 6. The quantized residual coefficients may, for example, be inverse quantized 10 and / or inverse transformed 12 to form a reconstructed residual, which may, for example, be added to the prediction block 26 to form a reconstructed CU. In-loop filtering 66 (e.g., a deblocking filter) may be applied to the reconstructed CU before the reconstructed CU may be placed in the reference picture store 64 and / or used to code the CU (e.g., a future CU). To form the output video bitstream 20, the coding mode (e.g., inter or intra), prediction mode information, motion information, and / or quantized residual coefficients may be sent (e.g., may all be sent) to an entropy coding unit 8, for example, for compression and / or packing to form a bitstream.

[0084] Figure 3A block diagram of an exemplary block-based video decoder is shown. For example, the decoder may be a WTRU. The video bitstream 202 may be unpacked (e.g., first unpacked) and / or entropy decoded at the entropy decoding unit 208. The coding mode and prediction information may be sent to the spatial prediction unit 260 (e.g., if intra-coded) and / or to the temporal prediction unit 262 (e.g., if inter-coded) to form a prediction block. The residual transform coefficients may be sent to the inverse quantization unit 210 and / or the inverse transform unit 212, for example, to reconstruct the residual block. The prediction block and / or the residual block may be added together at 226. For example, the reconstructed block may be in-loop filtered before being stored in the reference picture store 264. The reconstructed video in the reference picture store may be sent to drive a display device and / or used to predict a video block (e.g., a future video block).

[0085] Coding modules (eg, associated with temporal prediction) may be extended and / or enhanced. Affine motion compensation may be used as an inter-coding tool.

[0086] Implementations using affine modes may be described herein. A translational motion model may be applied to motion compensated prediction. There may be one or more motions (e.g., zooming in or out, rotation, perspective motion, and / or other irregular motions). Affine transform motion compensated prediction may be applied. A flag for an inter-coded CU (e.g., each inter-coded CU) may be signaled, for example, to indicate whether a translational motion model or an affine motion model is applied to inter prediction. If an affine motion model is applied, a flag may be signaled to indicate the number of parameters used in the model (e.g., four or six).

[0087] The affine motion model may be a four-parameter model. One or more (e.g., two) parameters may be used for translational movement (e.g., one parameter for each of the horizontal and vertical directions). One or more parameters (e.g., one parameter) may be used for scaling motion. One or more parameters (e.g., one parameter) may be used for rotational motion. The horizontal scaling parameter may be equal to the vertical scaling parameter. The horizontal rotation parameter may be equal to the vertical rotation parameter. The four-parameter motion model may be coded using one or more (e.g., two) motion vectors (MVs) (e.g., as a pair) at one or more (e.g., two) control point locations defined, for example, at the top left and top right corners of the current CU. Figure 5 As shown in , the affine motion field of the block can be described by two control point motion vectors (V0, V1). Based on the control point motion, the motion field (v x , v y ):

[0088]

[0089] Where (v0x , v 0y ) can be the motion vector of the upper left control point, (v 1x , v 1y ) can be the motion vector of the upper right corner control point, such as Figure 5 As shown, w may be the width of the CU.

[0090] The affine motion model may be a six-parameter model. One or more (e.g., two) parameters may be used for translational movement (e.g., one parameter for each of the horizontal and vertical directions). One or more (e.g., two) parameters may be used for scaling movement (e.g., one parameter for each of the horizontal and vertical directions). One or more (e.g., two) parameters may be used for rotational movement (e.g., one parameter for each of the horizontal and vertical directions). The six-parameter motion model may be coded with one or more (e.g., three) MVs at one or more (e.g., three) control points. For example, Figure 6 As shown in , the control points of a CU of six parameter affine decoding can be defined at the upper left corner, upper right corner, and lower left corner of the CU. The motion at the upper left control point can be related to the translation motion. The motion at the upper right control point can be related to the rotation and / or scaling motion in the horizontal direction. The motion at the lower left control point can be related to the rotation and / or scaling motion in the vertical direction. The rotation and scaling motion in the horizontal direction can be different from the motion in the vertical direction. The MV (v) of the sub-block (e.g., each sub-block) x , v y ) can be derived using one or more (e.g., three) MVs at the control points according to the following equation:

[0091]

[0092] Where (v 2x , v 2y ) may be the motion vector of the lower left control point, (x, y) may be the center position of the sub-block, and w and h may be the width and height of the CU, respectively.

[0093] When a block is coded with an affine motion model, its motion field may be derived based on, for example, the granularity of a sub-block. For example, to derive the MV of a sub-block (e.g., each sub-block), the MV of the center sample of the sub-block may be calculated according to equation (1) (e.g., as Figure 5 ), and may be rounded to a certain accuracy (e.g., 1 / 16 pixel accuracy). The derived MV may be used in the motion compensation stage to generate prediction signals for sub-blocks (e.g., each sub-block) within the current block. The sub-block size applied to affine motion compensation may be 4×4. One or more (e.g., four) parameters of the 4-parameter affine model may be estimated iteratively. For example, one or more MV pairs at step k may be represented as The original luminance signal can be represented as I(i, j). The predicted luminance signal can be represented as I' k (i, j). Spatial gradient g x (i, j) and g y (i, j) can be derived using a Sobel filter applied to the prediction signal in the horizontal and vertical directions respectively. The derivative of equation (1) can be expressed as follows:

[0094]

[0095] where (a, b) may be the translation parameter delta, and (c, d) may be the scale and rotation parameter deltas of step k. The MV deltas at the control points may be derived using the coordinates as in equations (5) and (6).

[0096] For example, (0, 0), (w, 0) may be the coordinates of the upper left and upper right control points, respectively.

[0097]

[0098] The relationship between brightness change, spatial gradient and temporal shift can be expressed as follows:

[0099]

[0100] and They can be replaced by their equivalent values ​​in equation (4) to obtain the equations for the parameters (a, b, c, d):

[0101] I′ k (i, j)-I(i, j)=(g x (i, j)*i+g y (i, j)*j)*c+(-g x (i, j)*j+g y (i, j)*i)*d+g x (i, j)*a+g y (i, j)*b. (8)

[0102] The parameter set (a, b, c, d) may be derived using a least squares implementation (e.g., because the samples in the CU satisfy equation (8)). In step (k+1), the control point The MV at can be solved using equations (5) and (6), and they can be rounded to a certain precision (e.g., 1 / 4 pixel). Using iterations, the MV at one or more (e.g., two) control points can be refined until, for example, the parameters (a, b, c, d) are all zero or the number of iterations that have been performed reaches a predetermined limit.

[0103] One or more (e.g., six) parameters of the six-parameter affine model may be estimated. Equation (4) may be changed to the following equation:

[0104]

[0105] Where (a, b) may be a translation parameter increment, (c, d) may be a scaling and rotation parameter increment for a first direction (e.g., horizontal direction), and (e, f) may be a scaling and rotation parameter increment for a second direction (e.g., vertical direction), at step k, equation (8) may be changed to the following equation:

[0106] I′ k (i, j)-I(i, j)=(g x (i, j)*i)*c+(g x (i,j)*j)*d+(g y (i, j)*i)*e+(g y (i, j)*j)*f+g x (i, j)*a+g y (i, j)*b (10)

[0107] The parameter set (a, b, c, d, e, f) can be derived using a least squares implementation, for example by considering samples (e.g., multiple samples) within a CU. It can be calculated using equation (5). and the lower left control point It can be calculated according to the following equation:

[0108]

[0109]

[0110] Affine merge mode can be used to decode a CU. If (e.g., one) CU is decoded in affine mode, one or more (e.g., two or three) MV sets for control points of a reference list (e.g., each reference list) can be signaled with predictive decoding. The difference between the MV and its predictor can be losslessly decoded. The affine merge mode can be applied by considering the local continuity of the motion field. The motion vector at the control point of the current CU can be derived using the affine motion model of its affine merge candidate selected from the neighboring blocks of the current CU. If the affine merge candidate is decoded with a four-parameter affine model, one or more (e.g., two) MVs at one or more (e.g., two) control points of the current CU can be derived. If the affine merge candidate is decoded with a six-parameter affine model, one or more (e.g., three) MVs at three control points of the current CU can be derived.

[0111] If the current CU is coded in affine merge mode, there may be one or more (eg, five) neighboring blocks (eg, Figure 7 ). The neighboring blocks may be checked in order (e.g., from N0 to N4). The first affine-coded neighboring blocks may be used as affine merge candidates. For example, Figure 8 As shown in , the current CU can be decoded into affine merge mode, and the lower left neighboring block (N0) can be selected as its affine merge candidate. The width and height of the CU containing block N0 can be represented as nw and nh, respectively. The width and height of the current CU can be represented as cw and ch, respectively. The MV at position Pi can be represented as (v ix , v iy ). The coordinates of Pi can be expressed as (x i ,y i ). If block N0 is decoded using a four-parameter affine model, one or more (e.g., two) MVs of the control points can be derived. For example, the MV (v 0x , v 0y ) can be derived according to the following equation:

[0112]

[0113] The MV(v 1x , v 1y ) can be derived according to the following equation:

[0114]

[0115] If block N0 is coded with a six-parameter affine model, one or more (eg, three) MVs {P0, P1, P2} of the control points may be derived according to the following equations:

[0116]

[0117] After deriving the MV at the control point {P0, P1} or {P0, P1, P2}, the MV of the sub-block (e.g., each sub-block) within the current CU may be derived as described herein. The derived sub-block MV may be used for sub-block-based motion compensation and picture temporal motion vector prediction for picture coding.

[0118] Affine MVs may be predictively coded. Predictive coding may be used for non-affine merge-coded CUs (e.g., to reduce signaling overhead). Affine MV predictors may be generated from the motion of neighboring coded blocks. Predictors for MV prediction of affine coded CUs may include, but are not limited to, inherited affine motion predictors derived from neighboring blocks coded with an affine motion model; affine motion predictors constructed from information from one or more neighboring blocks; and / or translational motion predictors for MV prediction (e.g., when the number of affine predictors from other options is below a threshold).

[0119] The inherited affine motion predictor may be derived from one or more neighboring affine-coded blocks. The neighboring block candidate list may be constructed in the same manner as in the affine merge mode (e.g., as Figure 7 ). For example, if the candidate block is coded in a bi-prediction mode, the candidates in the candidate list (e.g., each candidate) may be checked for MVs of one or more (e.g., two) lists. If the reference picture from one list of neighboring blocks is the same as the current reference picture, the affine motion of the neighboring blocks may be used to derive the MV at the control point of the current block, and the MV may be added to the affine predictor list (e.g., after a uniqueness check).

[0120] One or more (e.g., three) MV sets may be used to generate a constructed affine motion predictor. Fig. 9 As shown, the set of MVs may be as follows: the first available MV from the neighboring block {A, B, C} of the corner P0 (e.g., set S1); the first available MV from the neighboring block {D, E} of the corner P1 (e.g., set S2); and the first available MV from the neighboring block {F, G} of the corner P2 (e.g., set S3). If the neighboring block has the same reference picture as the current block, the MV from the neighboring block may be considered available. If S1 and S2 are not empty and S3 is empty, and the current block is a six-parameter affine model, the MVs in S1 and S2 may be considered as four-parameter affine models, and the MVs may be derived at P2. If S1 and S3 are not empty and S2 is empty, the MVs in S1 and S3 may be considered as four-parameter affine models, and the MVs may be derived at P1.

[0121] For example, there may be (e.g., one) affine predictor {MVP0, MVP'1, MVP'2} generated as described herein for a 6-parameter affine model. The MVs for three control points may be {MV0, MV1, MV2}. The final predictor may be derived as {MVP0, MVP'1+(MV0-MVP0), MVP'2+(MV0-MVP0)}. The terms "affine motion predictor" and "final predictor" may be used interchangeably herein.

[0122] Affine MV coding may be performed. If a CU is coded in an affine mode, it may be an affine merge mode or an affine non-merge mode. For the affine merge mode described herein, an affine MV at a control point may be derived from an affine MV of an adjacent affine-coded CU. For an affine non-merge mode, the MV at the control point may be coded with differential coding. An MV predictor may be generated using adjacent MVs as described herein. The difference between the current MV and its predictor may be coded. The MV difference to be signaled may be referred to as an MVD. A four-parameter affine model may have one or more (e.g., two) control points (e.g., two MVDs may be signaled for uni-prediction, and four MVDs may be signaled for bi-prediction). A six-parameter affine model may have one or more (e.g., three) control points (e.g., three MVDs may be signaled for uni-prediction, and six MVDs may be signaled for bi-prediction). An MVD may be a two-dimensional vector (e.g., having a horizontal component and a vertical component) and / or may be losslessly coded.

[0123] Affine motion estimation may be performed. Fig.10 An example flow chart for motion model selection for a CU is shown. A motion model may be selected based on the bit cost (e.g., motion estimation (ME) cost) of one or more motion models. The ME cost may be calculated according to equation (23), for example, by considering the distortion measured by the prediction error and the bit cost for decoding one or more (e.g., all) MVs:

[0124]

[0125] "o" can represent the original signal, and "p" can represent the i}. Dist(o, p) represents the distortion between signals o and p. The distortion can be evaluated by the sum of squared errors (SSE), or the sum of absolute differences (SAD), and / or the sum of absolute transform differences (SATD). R(MV i ) can be the i-th motion vector MV i K may be the number of decoded bits. K may be the number of motion vectors. K may be 1 for a translation model, 2 for a four-parameter affine model, and / or 3 for a six-parameter affine model. λ may be a weighting factor between distortion and bit cost.

[0126] The encoder may check the translational motion model (e.g., first) and may calculate the bit cost (e.g., ME cost). If the width and / or height of the block is greater than a threshold (e.g., 8 pixels), affine motion estimation with a four-parameter affine model may be performed. If the ME cost of the four-parameter affine model is not much greater than the ME cost of the translational motion model, the encoder may check the affine motion cost with a six-parameter affine model. For example, if the ME cost is approximately 100%-110% greater than the ME cost of the translational motion model, the encoder may check the affine motion cost with a six-parameter affine model. The encoder may select the motion model with the smallest ME cost from the available motion models it has evaluated. In order to reduce the complexity of the six-parameter affine motion estimation, the encoder may (e.g., only) use the best reference picture selected by the four-parameter affine motion estimation to perform affine motion estimation. For an iteration (e.g., each iteration) of the affine motion estimation, one or more (e.g., 4) actions may be performed. For example, affine motion compensation may be used to generate predictions; spatial gradients may be calculated in one or more (e.g., two) directions (e.g., using Sobel filtering); a correlation matrix may be calculated based on the gradients of the samples and their coordinates; affine model parameters may be calculated using the least squares method, and / or MV increments of control points may be calculated using the affine model parameters. The starting point for the four-parameter affine model motion search may be selected from, for example, an affine MV predictor or an MV from a translational motion model. The one with the smaller prediction error measured between the original signal and the predicted signal may be selected as the starting point for the affine motion estimation. For the six-parameter affine model motion search, the control point MVs of the four-parameter affine model may be considered for starting point selection. There may be a maximum iteration limit.

[0127] Affine motion model parameter estimation may include an iterative search based on the spatial gradient of the prediction signal and the difference between the original signal and the prediction signal. The gradient information may be affected by noise. It may be difficult to control the parameter changes for one step. Affine motion model parameter estimation may include high-dimensional parameter estimation, for example, four dimensions or six dimensions for a four-parameter affine model and a six-parameter affine model, respectively. Motion estimation may include MV searches in one or more (e.g., two) dimensions, and the search may be adjusted during motion estimation. Blocks may be partitioned by different tree partitions, such as quadtrees (QT), binary trees (BT), and / or ternary trees (TT) (e.g., as Figure 4 As shown in FIG. 1 ). Affine motion may be performed for (eg, each) decoding block. There may be a correlation (eg, a strong correlation) between the decoding block and its sub-blocks partitioned by the tree structure. Some correlation motion information may be reused for affine motion estimation, for example, to improve decoding efficiency.

[0128] Affine motion vector (MV) candidates may be used for starting point selection for affine motion estimation. For example, after iterating affine motion estimation, one or more affine motion predictors may be checked. One or more control point MVs may be refined, for example, using a search method. One or more spatial gradients may be calculated for use in affine motion estimation.

[0129] Affine motion estimation as described herein may be applied to one or more types of pictures. A hierarchical prediction structure may be used in video coding, where different reference picture groups may be used to predict the current picture (e.g., depending on the temporal level to which the current picture belongs). In hierarchical prediction, for the current picture at the lowest temporal level, one or more (e.g., all) of its reference pictures may precede the current picture in display order. The current picture may be referred to as a low-latency picture. A current picture having (e.g., any) reference picture following it in display order may be referred to as a non-low-latency picture. Affine motion estimation and / or affine MV check may be applied to the current picture (e.g., depending on its temporal level). For example, in some cases, affine motion estimation and / or affine MV check may be applied (e.g., only) to pictures of a high temporal layer (e.g., if TL>TL threshold). Affine motion estimation and / or affine MV check may be applied (e.g., only) to non-low-latency pictures (e.g., TL>0).

[0130] Multiple MV candidates (eg, affine MV candidates) may be used in starting point selection to perform motion estimation (ME) (eg, affine ME). Although the examples described herein may be related to affine motion estimation, those skilled in the art will appreciate that the techniques may be applied to motion estimation based on other models.

[0131] A circular list (e.g., a first-in, first-out (FIF.) list) containing one or more MVs (e.g., affine control point MVs) may be saved for a current coding block ME (e.g., four-parameter affine motion estimation). The circular list may have a size (e.g., a maximum size) that may be four. The circular list may be referred to as a most recently estimated MV list. The MVs stored in the list may be MVs searched by motion estimation (e.g., translation ME, four-parameter affine ME, and / or six-parameter affine ME). For example, a first MV set (e.g., a first MV, a first CPMV set) may be derived for a first coding block (e.g., by performing an affine ME associated with the first coding block). The first MV set may be added to the most recently estimated MV list (e.g., at the head of the most recently estimated MV list). An entry (e.g., each entry) of the list may indicate a coding block position, a block width, and a block height and / or an MV (e.g., a control point MV) of a four-parameter affine model associated with the coding block of (e.g., each) reference picture. The list may be used for starting point selection. For example, the starting point of the ME for the second decoding block may be selected based on the MV set and / or one or more MV predictors in the most recently estimated MV list. The starting point may be selected based on the corresponding prediction errors of the MV set, MV predictor and / or one or more translation model MVs in the most recently estimated MV list.

[0132] Given the set of control point MVs stored in the list, the control point MVs of the current block can be derived as follows. This derivation can be similar to the MV derivation in the affine merge mode described herein. MV(B) can represent the set of control point MVs of block B of the four-parameter affine model. It can represent the control point MV set for the i-th reference picture of block B. B ,y B} and {x C ,y C} can represent the position of block B and the current block respectively. R ,h B} and {w C ,h C} can represent the width and height of block B and the current block respectively. The MV (v 0x , v 0y ) can be derived as follows:

[0133]

[0134] The MV at the upper right control point (v 1x , v 1y ) can be derived as follows:

[0135]

[0136] If a six-parameter affine search is performed on the current block, the MV at the lower left control point (v 2x , v 2y ) can be derived as follows:

[0137]

[0138] The parent block can be segmented in a variety of ways. For example, the first tree segmentation can segment the parent block of the second coding block into a second coding block and at least one sibling block. The second tree segmentation can segment the parent block of the second coding block into at least two child blocks. The first coding block can be a neighboring block of the second coding block, a sibling block of the second coding block, a parent block of the second coding block, or a child block of the parent block segmented by the second tree. Each searched MV set in the most recently estimated MV list can be a MV set selected for a neighboring block of the second coding block segmented by different trees, a sibling block of the second coding block, a parent block of the second coding block, or a child block of the parent block of the second coding block.

[0139] The derived MV set can be used as an affine MV candidate for starting point selection. One or more (e.g., 3) affine MV candidates can be compared (e.g., before the encoder starts affine ME for a 4-parameter affine model) to select a starting point: one or more affine MV predictors (e.g., derived as described herein); the MV of the translation model; and the MV set derived from the entries (e.g., each entry) of the circular list.

[0140] For example, the list can be updated after the encoder performs ME (e.g., four-parameter affine ME). As described herein, the search results can be added to a list (e.g., a list of recently estimated MVs). If there is an entry in the list with the same block position, block width, and block height as the candidate to be added to the list, the control point MV of the entry can be updated using the MV of the candidate to be added. The list size and the head of the list can remain the same. If there are no repeated blocks in the list, the candidate can be added to the end of the circular list. For example, if the list is full, the oldest entry in the circular list can be overwritten. The size of the list and the head of the list can be changed accordingly.

[0141] Fig.11 (a)-(f) show example list construction and updates. Fig.11The shaded block in (a)-(f) may indicate the current head of the circular list. Given a decoded block B0, the encoder may try non-split mode, horizontal BT, vertical BT, horizontal TT, vertical TT, and / or QT. The list may have a current size and / or a maximum size. For example, the initial list may be empty and the maximum list size may be 4. If the list has reached the maximum size, the least recently estimated MV set may be removed from the list before the derived MV set is added to the list. Fig.11 As shown in (a), for example, after the encoder checks the non-split of block B0, the affine control point MV of block B0 can be added to the list. For example, after the encoder checks the horizontal BT split, it can press Fig.11 (b) The affine control point MVs of blocks B1 and B2 are added to the list in the order shown. For example, after the encoder completes encoding of the partition, the affine control point MVs of the parent block of the partition can be added to the list again (for example, if it is not in the list). For example, if the second decoded block is the last block in the first tree partition, the head of the most recently estimated MV list can be set to the MV set associated with the parent block. The MVs of the parent block can be used for the ME of its child blocks for another tree partitioning scheme. The MVs of the parent block can be (for example, always) saved in the list before the encoder checks the new tree partition. Fig.11 (c)-(f) show the update lists after the encoder checks vertical BT, horizontal TT, vertical TT, and QT.

[0142] Starting point generation using previously searched motion model parameters described herein can be used in a six-parameter affine model. It can be expressed as a set of control point MVs of the i-th reference picture of block B for the six-parameter affine model. The set of control point MVs of the i-th reference picture of the current block for starting point selection can be derived as shown in equations (30)-(35). The MV (v 0x , v 0y ) can be derived as follows:

[0143]

[0144] The MV (v 1x , v 1y ) can be derived as follows:

[0145]

[0146] The MV (v2x, v2y) at the lower left control point of the i-th reference picture can be derived as follows:

[0147]

[0148] The derived MV set can be used as an affine MV candidate for starting point selection for a six-parameter affine search. If the current block performs a four-parameter affine search and the entry in the loop list is a six-parameter affine model, then (e.g., only) the top left and top right MVs can be derived using equations (30)-(33).

[0149] The encoder may consider an affine MV predictor in an affine merge mode. Using (e.g., directly) an MV predictor at (e.g., one) control point may reduce the MVD signaling overhead for the MV of the control point. An additional MV set may be generated by combining one or more found MVs and predictors. For example, the current block ME may be used for a six-parameter affine model. An MV set (e.g., a control point MV set) and / or an MV predictor (MVP) set (e.g., an affine MVP set) may be determined for the current block. After ME (e.g., a six-parameter affine ME), the control point MV set may be {MV0, MV1, MV2} for the current block. The affine MV predictor set for the current block may be {MVP0, MVP1, MVP2} for one or more (e.g., three) control points generated as described herein. It may be determined whether the first MV of the MV set is equal to the first MVP of the MVP set. If the first MV is not equal to the first MVP, the first MV in the MV set is replaced with the first MVP to generate a MV-MVP combination set. The first MV may be, for example, MV0, MV1, or MV2, and the first MVP may be MVP0, MVP1, MVP2. The MV-MVP combination set generated by combining the found MV and predictor may be one or more of the following: {MVP0, MV1, MV2}, {MV0, MVP1, MV2}, and / or {MV0, MV1, MVP2}. The encoder may calculate the cost (e.g., ME cost) for (e.g., each) MV-MVP combination set described herein, and may compare it with the cost of the MV set. For example, the cost of the MV set and / or the MV-MVP combination set may be a ME cost, and may be calculated based on the prediction error and the corresponding bit cost of each MV or MVP in the MV set or the MV-MVP combination set. The best MV set may be selected from the MV-MVP combination set and the MV set based on their respective ME costs. For example, the best MV set may be selected from the MV set and each MV-MVP combination set (e.g., {MVP0, MV1, MV2}, {MV0, MVP1, MV2}, and / or {MV0, MVP1, MV2}) based on their respective ME costs. The best MV set may be the MV set or MV-MVP combination set with the lowest ME cost. Fig.12A flow chart illustrating the check for combined affine MVs is shown.

[0150] The second MV of the MV set may be compared with the second MVP of the MVP set. If the second MV is not equal to the second MVP, the second MV may be replaced with the second MVP to generate a second MV-MVP combination set, which may be {MV0, MVP1, MVP2}. The encoder may calculate the bit cost (e.g., ME cost) of the second MV-MVP combination set. The bit cost of the second MV-MVP combination set may be compared with the bit cost of the current best MV set, and another best MV set may be selected. For example, if the bit cost of the second MV-MVP combination set is lower than the bit cost of the current best MV set, the second MV-MVP combination set may be selected. Otherwise, the current best set may be selected.

[0151] ME can be performed in part based on potential MVD signaling overhead. MVs related to rotation and scaling parameters can be derived (e.g., taking into account MVD signaling overhead and / or prediction error). For example, after affine ME, refinement (e.g., fast refinement) can be added. MVs related to scaling and rotation parameters of a four-parameter affine motion model can be refined (e.g., only MVs related to scaling and rotation parameters). For example, after motion estimation (e.g., affine ME), there can be a set of MVs {MV0, MV1} for the top-left and top-right control points. The encoder can check the closest four neighboring MVs of MV1, while fixing the top-left MV to MV0: (MV 1x -s, MV 1y ), (MV 1x , MV 1y -s), (MV 1x , MV 1y +s), (MV 1x +s, MV 1y ). The parameter may be a step size (e.g., 1 / 4). The encoder may calculate the cost (e.g., ME cost) of (e.g., each) refined MV at the upper right control point and may compare it with the current minimum cost. In the refinement, the encoder may consider the signaling overhead when it calculates the ME cost. The encoder may refine the MV at the upper left control point while fixing the upper right MV. The refinement process may be performed iteratively.

[0152] A spatial gradient calculation of an affine ME may be performed. As described herein, one or more parameters (e.g., affine parameters) may be estimated using, for example, spatial gradients and / or temporal differences between the original signal and the predicted signal. The spatial gradient may be calculated using a Sobel filter. The Sobel filter may be a 3×3 two-dimensional filter. The Sobel filter used for horizontal gradient calculation may be The Sobel filter used for vertical gradient calculation can be For example, gradients may be calculated (eg, only) for internal samples in a CU. The CU size may be WxH, and gradients may be calculated (eg, only) for (W-2)x(H-2) internal samples. Fig.13 An example gradient calculation for an 8×8 coding unit (CU) is shown. Fig.13 As shown, a Sobel filter may be applied to calculate the gradient of 6x6 internal samples, and may be applied to samples on the boundary of a CU (eg, Fig.13 After computing the gradients for the internal 6x6 samples with a Sobel filter, padding may be applied to samples at the CU boundary, for example, by copying the gradients from the nearest neighbor in the 6x6 internal block.

[0153] Filters can be used for gradient calculation. The filter for horizontal gradient calculation can be (-1 01), and the filter for vertical gradient calculation can be The gradient calculation complexity may be reduced. Gradient calculation for CU boundary samples may be performed. Gradient accuracy at CU boundaries may be improved. A CU may be filled by filling rows and columns at CU boundaries (eg, each CU boundary). Fig.14 An example gradient calculation for an 8×8 CU is shown. Fig.14 As shown in , the 8×8 prediction block can be padded to a 10×10 block. Fig.14 The shaded circles shown in may represent padded samples. A 3-tap filter may be applied to calculate gradients for one or more 8×8 CU samples (eg, all 8×8 CU samples) using the padded samples.

[0154] like Fig.14 As shown, the gradient for the 8×8 boundary samples can be calculated using the padding samples and / or the 8×8 prediction samples. The padding samples can be derived in one or more ways as described herein. The padding samples can be derived from the reference picture by interpolation using the motion vector of the CU. Integer samples can be extracted from the reference picture buffer. The padding samples derived from the reference picture by interpolation can be relatively accurate. The padded samples can be derived by copying from the reference samples in the closest integer position based on the motion vector of the CU. The computational complexity can be reduced by copying from the reference samples to derive the padding samples.

[0155] For example, the position to be filled may be (x, y). The CU MV may be (MVx, MVy). RefPic may be a reference picture buffer. RefPic(i, j) may be the sample value at integer position (i, j). The filled sample value of (x, y) may be RefPic(round(x+MVx), round(y+MVy)), where round(x) may be a function that rounds the input variable x to its nearest integer value.

[0156] Although the features and elements are described above in specific combinations, it will be understood by those skilled in the art that each feature or element can be used alone or in any combination with other features and elements. In addition, the methods described herein can be implemented in a computer program, software, or firmware, which is incorporated into a computer-readable medium to be executed by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via a wired or wireless connection) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs). A processor associated with the software can be used to implement a radio frequency transceiver used in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. A method comprising: determining an MV set including one or more motion vectors MV and an MVP set including one or more MV predictors MVP; Determine whether a first MV of the MV set is equal to a first MVP of the MVP set; Under the condition that the first MV is not equal to the first MVP, replacing the first MV in the MV set with the first MVP to generate a first MV-MVP combination set; determining a cost of the first MV-MVP combination set; as well as Based on the cost of the first MV-MVP combination set, one of the MV set or the first MV-MVP combination set is selected as an optimal MV set.

2. The method according to claim 1, wherein selecting one of the MV set or the first MV-MVP combination set as the best MV set comprises: comparing the cost of the first MV-MVP combination set with the cost of the MV set; Under the condition that the cost of the first MV-MVP combination set is lower than the cost of the MV set, selecting the first MV-MVP combination set as the optimal MV set; as well as Under the condition that the cost of the first MV-MVP combination set is greater than or equal to the cost of the MV set, the MV set is selected as the optimal MV set. 3 . The method of claim 1 , wherein the MV set comprises one or more control point MVs, and the MVP set comprises one or more affine MVPs.

4. The method according to claim 1, further comprising: Determining whether a second MV of the MV set is equal to a second MVP of the MVP set; Under the condition that the second MV is not equal to the second MVP, replacing the second MV in the MV set with the second MVP to generate a second MV-MVP combination set; determining a cost of the second MV-MVP combination set; updating the best MV set based on the cost of the second MV-MVP combination set; determining whether a third MV of the MV set is equal to a third MVP of the MVP set; Under the condition that the third MV is not equal to the third MVP, replacing the third MV in the MV set with the third MVP to generate a third MV-MVP combination set; determining a cost for the third MV-MVP combination set; and The best MV set is updated based on the cost of the third MV-MVP combination set.

5. The method according to claim 1, further comprising: Determining whether a second MV of the MV set is equal to a second MVP of the MVP set; When the second MV is not equal to the second MVP, replacing the second MV in the first MV-MVP combination set with the second MVP to generate a fourth MV-MVP combination set; determining a cost of the fourth MV-MVP combination set; as well as The best MV set is updated based on the cost of the fourth MV-MVP combination set. 6 . The method of claim 1 , wherein the cost of the first MV-MVP combination set is a motion estimation (ME) cost, and the ME cost is determined based on a prediction error and a corresponding bit cost for each MV in the first MV-MVP combination set.

7. The method of claim 1, wherein the method further comprises encoding a coding block based on the best MV set.

8. A video encoding device, comprising: The processor is configured as: determining an MV set including one or more motion vectors MV and an MVP set including one or more MV predictors MVP; Determine whether a first MV of the MV set is equal to a first MVP of the MVP set; Under the condition that the first MV is not equal to the first MVP, replacing the first MV in the MV set with the first MVP to generate a first MV-MVP combination set; determining a cost of the first MV-MVP combination set; as well as Based on the cost of the first MV-MVP combination set, one of the MV set or the first MV-MVP combination set is selected as an optimal MV set.

9. The apparatus of claim 8, wherein the processor is configured to select one of the MV set or the first MV-MVP combination set as the best MV set comprises the processor being configured to: comparing the cost of the first MV-MVP combination set with the cost of the MV set; Under the condition that the cost of the first MV-MVP combination set is lower than the cost of the MV set, selecting the first MV-MVP combination set as the optimal MV set; and Under the condition that the cost of the first MV-MVP combination set is greater than or equal to the cost of the MV set, the MV set is selected as the optimal MV set.

10. The apparatus of claim 8, wherein the set of MVs comprises one or more control point MVs, and the set of MVPs comprises one or more affine MVPs.

11. The device of claim 8, wherein the processor is further configured to: Determining whether a second MV of the MV set is equal to a second MVP of the MVP set; Under the condition that the second MV is not equal to the second MVP, replacing the second MV in the MV set with the second MVP to generate a second MV-MVP combination set; determining a cost of the second MV-MVP combination set; updating the best MV set based on the cost of the second MV-MVP combination set; determining whether a third MV of the MV set is equal to a third MVP of the MVP set; Under the condition that the third MV is not equal to the third MVP, replacing the third MV in the MV set with the third MVP to generate a third MV-MVP combination set; determining a cost for the third MV-MVP combination set; and The best MV set is updated based on the cost of the third MV-MVP combination set.

12. The device of claim 8, wherein the processor is further configured to: Determining whether a second MV of the MV set is equal to a second MVP of the MVP set; When the second MV is not equal to the second MVP, replacing the second MV in the first MV-MVP combination set with the second MVP to generate a fourth MV-MVP combination set; determining a cost of the fourth MV-MVP combination set; as well as The best MV set is updated based on the cost of the fourth MV-MVP combination set.

13. The apparatus of claim 8, wherein the cost of the first MV-MVP combination set is a motion estimation (ME) cost, and the ME cost is determined based on a prediction error and a corresponding bit cost for each MV in the first MV-MVP combination set.

14. The apparatus of claim 8, wherein the processor is further configured to encode a coding block based on the best MV set.

15. A method comprising: Determining a set of motion vectors MV based on affine motion estimation, wherein the set of MVs includes a first MV and a second MV; calculating a first motion estimation cost associated with a first neighboring MV, the first neighboring MV being associated with at least one of a scaling parameter or a rotation parameter, wherein the first neighboring MV is adjacent to at least one of the first MV or the second MV; calculating a second motion estimation cost associated with a second neighboring MV, the second neighboring MV being associated with at least one of a scaling parameter or a rotation parameter, wherein the second neighboring MV is adjacent to at least one of the first MV or the second MV; selecting at least one of the first neighboring MV or the second neighboring MV based on the first motion estimation cost and the second motion estimation cost; as well as The current block is encoded based on the selected neighboring MVs.

16. The method according to claim 15, wherein the method further comprises: determining a step size associated with the affine motion estimate based on the signaled accuracy; and The first adjacent MV and the second adjacent MV are determined based on the step size.

17. A video encoding device comprising: The processor is configured as: Determining a set of motion vectors MV based on affine motion estimation, wherein the set of MVs includes a first MV and a second MV; calculating a first motion estimation cost associated with a first neighboring MV, the first neighboring MV being associated with at least one of a scaling parameter or a rotation parameter, wherein the first neighboring MV is adjacent to at least one of the first MV or the second MV; calculating a second motion estimation cost associated with a second neighboring MV, the second neighboring MV being associated with at least one of a scaling parameter or a rotation parameter, wherein the second neighboring MV is adjacent to at least one of the first MV or the second MV; selecting at least one of the first neighboring MV or the second neighboring MV based on the first motion estimation cost and the second motion estimation cost; as well as The current block is encoded based on the selected neighboring MVs.

18. The apparatus according to claim 17, wherein: The processor is also configured to: determining a step size associated with the affine motion estimate based on the signaled accuracy; and The first adjacent MV and the second adjacent MV are determined based on the step size.

Images