Ibc lic model merge mode enhancement

Abstract

Systems, methods, and instrumentalities are disclosed for performing operations associated with intra block copy (IBC) local illumination compensation (LIC) merge mode. An IBC-LIC model merge mode can be performed, which can include creating a model candidate list including model parameters from spatial neighboring neighbor blocks, non-neighboring neighbor blocks, history candidates, default mode, and the like. The model candidate list can be reordered based on template costs. The reordering of the model parameter candidate list can be improved, for example, to reduce computational burden. For example, the LIC model candidates can be reordered based on a determined merge candidate (e.g., a best merge candidate).

Description

Cross-references to related applications

[0001] This application claims the benefit of European patent application No. 23307342.8, filed on 22 December 2023, the contents of which are incorporated herein by reference in their entirety. Background Technology

[0002] Video coding systems can be used to compress digital video signals, for example, to reduce the storage space and / or transmission bandwidth required for such signals. Video coding systems can include, for example, block-based systems, wavelet-based systems, and / or object-based systems. Summary of the Invention

[0003] Systems, methods, and means for performing operations associated with intra-block copy (IBC) local illumination compensation (LIC) merging modes are disclosed. The IBC-LIC model merging mode can be executed, and may include, for example, creating a model candidate list comprising model parameters from spatially adjacent blocks, non-adjacent blocks, historical candidates, default modes, etc. The model candidate list can be reordered based on template cost. The reordering of the model parameter candidate list can be improved, for example, to reduce computational overhead. For example, the LIC model candidates can be reordered based on determined merging candidates (e.g., best merging candidates).

[0004] A device (e.g., a video encoder or video encoder) can reorder a list of model parameter candidates, for example, to reduce computational complexity and improve compression efficiency. A combination of intra-block copying (IBC) and local illumination compensation (LIC) (e.g., as an IBC-LIC mode) can be performed. An IBC-LIC model merging mode can be performed. A LIC model candidate list and / or an IBC candidate list can be determined. An IBC merge candidate can be selected (e.g., from the IBC candidate list). The IBC merge candidate list can be sorted based on template cost. An IBC merge candidate can be selected based on template cost (e.g., the IBC merge candidate with the best template cost). The LIC model candidate list can be reordered, for example, based on the selected IBC merge candidate. For example, a LIC model (e.g., a LIC model candidate) can be selected based on the reordered LIC model candidate list. The selected LIC model can be the first LIC model candidate in the reordered LIC model candidate list. An indication (e.g., included in the video data) can indicate whether the IBC-LIC model merging mode is used and can indicate the selected IBC merge candidate.

[0005] The LIC model candidate list can be divided into multiple candidate lists (e.g., a primary LIC model candidate list and a secondary LIC model candidate list). LIC model candidates to be included in the primary and secondary LIC model candidate lists can be determined (e.g., based on the spatial distance associated with each LIC model candidate). For example, the primary LIC model candidate list may include LIC model candidates associated with spatially adjacent model candidates. The secondary LIC model candidate list may include the remaining LIC model candidates. The primary LIC model candidate list may include LIC model candidates based on the order of LIC model candidates in the LIC model candidate list (e.g., the first number of LIC model candidates in the LIC model candidate list (such as, for example, the first six LIC model candidates in the LIC model candidate list) included in the primary LIC model candidate list). The secondary LIC model candidate list may include LIC model candidates that follow (e.g., those ranked after the first number of LIC model candidates) in the LIC model candidate list (e.g., LIC model candidates that follow the first six LIC model candidates in the LIC model candidate list).

[0006] Spatial Geometric Partitioning (SGPM) can be used in conjunction with IBC-LIC models. The SGPM partitioning pattern to be used in conjunction with the IBC-LIC model can be determined. For example, a primary list of LIC model candidates can be determined based on the determined SGPM partitioning pattern.

[0007] A mixture associated with multiple (e.g., two) LIC models can be performed, for example, to determine a final prediction. For instance, a first LIC model and a second LIC model can be selected based on a reordered list of LIC model candidates. A first prediction can be determined based on applying the first LIC model to a reference sample, and a second prediction can be determined based on applying the second model to the reference sample. A final prediction can be determined based on a mixture of the first and second predictions.

[0008] The systems, methods, and means described herein may relate to a decoder. In some examples, the systems, methods, and means described herein may relate to an encoder. In some examples, the systems, methods, and means described herein may relate to signals (e.g., received from an encoder and / or by a decoder). A computer-readable medium may include instructions for causing one or more processors to perform the methods described herein. A computer program product may include instructions that, when executed by one or more processors, cause one or more processors to perform the methods described herein. Attached Figure Description

[0009] Figure 1A This is a system diagram illustrating an exemplary communication system in which one or more of the disclosed embodiments may be implemented.

[0010] Figure 1B This illustrates that, according to an embodiment, it is possible to Figure 1A The diagram shows an exemplary wireless transmit / receive unit (WTRU) used within a communication system.

[0011] Figure 1C This illustrates that, according to an embodiment, it is possible to Figure 1A The diagram shows an exemplary radio access network (RAN) and an exemplary core network (CN) used within the communication system.

[0012] Figure 1D This illustrates that, according to an embodiment, it is possible to Figure 1A The system diagram shows another exemplary RAN and another exemplary CN used within the communication system shown.

[0013] Figure 2 An example video encoder is shown.

[0014] Figure 3 An example video decoder is shown.

[0015] Figure 4 Examples of systems in which various aspects and examples can be implemented are shown.

[0016] Figure 5 The reference area for intra-block copy mode is shown.

[0017] Figure 6 An example of padding candidates for replacing zero vectors in the intra-block copy list is shown.

[0018] Figure 7 An example of an extended reference region used for intra-frame block copying is shown.

[0019] Figure 8 An example block is shown that encodes a spatial geometry partitioning pattern. Detailed Implementation

[0020] A more detailed understanding can be obtained through the following description, which is given by way of example in conjunction with the accompanying drawings.

[0021] Figure 1AThis is a diagram illustrating an example communication system 100 in which one or more of the disclosed embodiments may be implemented. Communication system 100 may be a multiple access system providing content such as voice, data, video, messaging, and broadcasting to multiple wireless users. Communication system 100 enables multiple wireless users to access such content by sharing system resources (including wireless broadband). For example, communication system 100 may employ one or more channel access methods, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal FDMA (OFDMA), Single Carrier FDMA (SC-FDMA), Zero-Tail Unique Word DFT Extended OFDM (ZT UW DTS-s OFDM), Unique Word OFDM (UW-OFDM), Resource Block Filtered OFDM, Filter Bank Multicarrier (FBMC), etc.

[0022] like Figure 1A As shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, public switched telephone network (PSTN) 108, Internet 110, and other networks 112. However, it will be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, and 102d can be any type of device configured to operate and / or communicate in a wireless environment. For example, WTRUs 102a, 102b, 102c, and 102d (any of which may be referred to as a “station” and / or “STA”) may be configured to transmit and / or receive wireless signals and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscription-based units, pagers, cellular phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in the context of industrial and / or automated processing chains), consumer electronics devices, devices operating on commercial and / or industrial wireless networks, etc. Any of WTRUs 102a, 102b, 102c, and 102d may be interchangeably referred to as a UE.

[0023] The communication system 100 may also include base station 114a and / or base station 114b. Each of base stations 114a and 114b can be any type of device configured to wirelessly connect to at least one of WTRUs 102a, 102b, 102c, and 102d to facilitate access to one or more communication networks, such as CN 106 / 115, the Internet 110, and / or other networks 112. For example, base stations 114a and 114b can be base transceiver stations (BTS), Node-B, eNode B, home Node B, main eNode B, gNB, NR Node B, site controllers, access points (APs), wireless routers, etc. Although base stations 114a and 114b are each depicted as a single element, it will be understood that base stations 114a and 114b can include any number of interconnected base stations and / or network elements.

[0024] Base station 114a may be part of RAN 104 / 113, which may also include other base stations and / or network elements (not shown), such as base station controllers (BSCs), radio network controllers (RNCs), relay nodes, etc. Base station 114a and / or base station 114b may be configured to transmit and / or receive radio signals on one or more carrier frequencies, which may be referred to as cells (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage for a specific geographic area that may be relatively fixed or may change over time. A cell may also be divided into cell sectors. For example, the cell associated with base station 114a may be divided into three sectors. Therefore, in one embodiment, base station 114a may include three transceivers, i.e., one transceiver per sector of the cell. In embodiments, base station 114a may employ multiple-input multiple-output (MIMO) technology and may utilize multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in a desired spatial direction.

[0025] Base stations 114a and 114b can communicate with one or more of WTRUs 102a, 102b, 102c, and 102d via air interface 116, which can be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). Any suitable radio access technology (RAT) can be used to establish air interface 116.

[0026] More specifically, as described above, the communication system 100 can be a multiple access system and can employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, base station 114a in RAN 104 / 113 and WTRUs 102a, 102b, 102c can implement radio technologies such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which can use Wideband CDMA (WCDMA) to establish air interfaces 115 / 116 / 117. WCDMA can include communication protocols such as High-Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA can include High-Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High-Speed ​​UL Packet Access (HSUPA).

[0027] In the embodiment, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which can use Long Term Evolution (LTE) and / or LTE-Advanced (LTE-A) and / or LTE-Advanced Pro (LTE-A Pro) to establish air interface 116.

[0028] In the embodiment, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as NR radio access, which can use New Radio (NR) to establish air interface 116.

[0029] In the embodiments, base station 114a and WTRUs 102a, 102b, and 102c can implement multiple radio access technologies. For example, base station 114a and WTRUs 102a, 102b, and 102c can, for instance, use the dual connectivity (DC) principle to jointly implement LTE radio access and NR radio access. Therefore, the air interface used by WTRUs 102a, 102b, and 102c can be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations (e.g., eNBs and gNBs).

[0030] In other embodiments, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as IEEE 802.11 (i.e., Wi-Fi), IEEE 802.16 (i.e., WiMAX), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Provisional Standard 2000 (IS-2000), Provisional Standard 95 (IS-95), Provisional Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data Rate GSM Evolution (EDGE), GSMEDGE (GERAN), etc.

[0031] Figure 1A Base station 114b can be, for example, a wireless router, a home NodeB, a home eNodeB, or an access point, and can utilize any suitable RAT to facilitate wireless connectivity in localized areas such as commercial locations, homes, vehicles, campuses, industrial facilities, air corridors (e.g., for use by drones), roads, etc. In one embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.11 to establish a wireless local area network (WLAN). In another embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, base station 114b and WTRUs 102c, 102d can utilize cellular-based RATs (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-a, LTE-a Pro, NR, etc.) to establish picocells or femtocells. Figure 1A As shown, base station 114b can be directly connected to Internet 110. Therefore, base station 114b does not need to access Internet 110 via CN 106 / 115.

[0032] RAN 104 / 113 can communicate with CN 106 / 115, which can be any type of network configured to provide voice, data, application, and / or Voice over Internet Protocol (VoIP) services to one or more of WTRUs 102a, 102b, 102c, and 102d. Data can have different Quality of Service (QoS) requirements, such as different throughput requirements, latency requirements, fault tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. CN 106 / 115 can provide call control, billing services, location-based services, prepaid calling, internet connectivity, video distribution, etc., and / or perform advanced security functions (such as user authentication). Although Figure 1AAs not shown, but will be understood, RAN 104 / 113 and / or CN 106 / 115 can communicate directly or indirectly with other RANs using the same RAT as or a different RAT than RAN 104 / 113. For example, in addition to being connected to RAN 104 / 113, which may be utilizing NR radio technology, CN 106 / 115 can also communicate with another RAN (not shown) using GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technology.

[0033] CN 106 / 115 can also serve as a gateway for WTRU 102a, 102b, 102c, 102d to access PSTN 108, the Internet 110, and / or other networks 112. PSTN 108 may include a circuit-switched telephone network providing Common Old-Style Telephone Service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices using common communication protocols such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and / or Internet Protocol (IP) from the TCP / IP Internet Protocol suite. Network 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, network 112 may include another CN connected to one or more RANs, which may use the same RAT as RAN 104 / 113 or a different RAT.

[0034] Some or all of the WTRUs 102a, 102b, 102c, and 102d in communication system 100 may include multi-mode capabilities (e.g., WTRUs 102a, 102b, 102c, and 102d may include multiple transceivers for communicating with different wireless networks via different wireless links). For example, Figure 1A The WTRU 102c shown can be configured to communicate with a base station 114a that can use cellular-based radio technology and with a base station 114b that can use IEEE 802 radio technology.

[0035] Figure 1B This is a system diagram illustrating example WTRU 102. (See diagram below.) Figure 1B As shown, WTRU 102 may include a processor 118, a transceiver 120, a transmitting / receiving element 122, a speaker / microphone 124, a keyboard 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power supply 134, a Global Positioning System (GPS) chipset 136, and / or other peripheral devices 138, etc. It will be understood that, while remaining consistent with the embodiments, WTRU 102 may include any sub-combination of the foregoing elements.

[0036] Processor 118 may be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. Processor 118 may perform signal encoding, data processing, power control, input / output processing, and / or any other functions that enable WTRU 102 to operate in a wireless environment. Processor 118 may be coupled to transceiver 120, which may be coupled to transmitting / receiving element 122. Although Figure 1B The processor 118 and transceiver 120 are depicted as separate components, but it will be understood that the processor 118 and transceiver 120 can be integrated together in an electronic package or chip.

[0037] Transmitting / receiving element 122 can be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) via air interface 116. For example, in one embodiment, transmitting / receiving element 122 can be an antenna configured to transmit and / or receive RF signals. In another embodiment, transmitting / receiving element 122 can be a transmitter / detector configured to transmit and / or receive, for example, IR, UV, or visible light signals. In yet another embodiment, transmitting / receiving element 122 can be configured to transmit and / or receive both RF signals and optical signals. It will be understood that transmitting / receiving element 122 can be configured to transmit and / or receive any combination of wireless signals.

[0038] Although the transmitting / receiving element 122 is in Figure 1B While depicted as a single element, WTRU 102 may include any number of transmit / receive elements 122. More specifically, WTRU 102 may employ MIMO technology. Thus, in one embodiment, WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals via air interface 116.

[0039] Transceiver 120 can be configured to modulate signals transmitted by transmitting / receiving element 122 and demodulate signals received by transmitting / receiving element 122. As described above, WTRU 102 can have multi-mode capability. Therefore, transceiver 120 can include multiple transceivers for enabling WTRU 102 to communicate via various RATs (e.g., such as NR and IEEE 802.11).

[0040] The processor 118 of WTRU 102 can be coupled to and receive user input data from: a speaker / microphone 124, a keyboard 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) unit or an organic light-emitting diode (OLED) display unit). The processor 118 can also output user data to the speaker / microphone 124, keyboard 126, and / or display / touchpad 128. Additionally, the processor 118 can access information and store data from any suitable type of memory, such as non-removable memory 130 and / or removable memory 132. Non-removable memory 130 may include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. Removable memory 132 may include a subscriber identity module (SIM) card, memory stick, secure digital storage (SD) card, etc. In other embodiments, the processor 118 can access information and store data from memory that is not physically located on WTRU 102 (such as on a server or home computer (not shown)).

[0041] The processor 118 may receive power from the power supply 134 and may be configured to distribute power to other components in the WTRU 102 and / or control power to those other components. The power supply 134 may be any suitable device for powering the WTRU 102. For example, the power supply 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0042] The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) about the current location of the WTRU 102. In addition to or instead of information from the GPS chipset 136, the WTRU 102 may receive location information from base stations (e.g., base stations 114a, 114b) via air interface 116 and / or determine its location based on the timing of signals received from two or more nearby base stations. It will be understood that, while remaining consistent with the embodiments, the WTRU 102 may acquire location information using any suitable location determination method.

[0043] The processor 118 may also be coupled to other peripheral devices 138, which may include one or more software and / or hardware modules that provide additional features, functions, and / or wired or wireless connectivity. For example, peripheral devices 138 may include accelerometers, electronic compasses, satellite transceivers, digital cameras (for photos and / or videos), Universal Serial Bus (USB) ports, vibration devices, television transceivers, hands-free headsets, Bluetooth® modules, FM radio units, digital music players, media players, video game player modules, internet browsers, virtual reality and / or augmented reality (VR / AR) devices, activity trackers, etc. Peripheral devices 138 may include one or more sensors, which may be one or more of the following: gyroscopes, accelerometers, Hall effect sensors, magnetometers, orientation sensors, proximity sensors, temperature sensors, time sensors; geolocation sensors; altimeters, light sensors, touch sensors, magnetometers, barometers, gesture sensors, biometric sensors, and / or humidity sensors.

[0044] WTRU 102 may include a full-duplex radio, for which the transmission and reception of some or all of the signals (e.g., signals associated with a specific subframe of both UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous. The full-duplex radio may include an interference management unit to reduce and / or substantially eliminate self-interference through signal processing via hardware (e.g., a choke) or via a processor (e.g., a separate processor (not shown) or via processor 118). In embodiments, WTRU 102 may include a half-duplex radio, for which the transmission and reception of some or all of the signals (e.g., signals associated with a specific subframe of both UL (e.g., for transmission) and downlink (e.g., for reception)) are separate.

[0045] Figure 1C This is a system diagram illustrating RAN 104 and CN 106 according to an embodiment. As described above, RAN 104 can employ E-UTRA radio technology to communicate with WTRUs 102a, 102b, and 102c via air interface 116. RAN 104 can also communicate with CN 106.

[0046] RAN 104 may include eNode-Bs 160a, 160b, and 160c, but it will be understood that RAN 104 may include any number of eNode-Bs while remaining consistent with the embodiments. eNode-Bs 160a, 160b, and 160c may each include one or more transceivers for communicating with WTRUs 102a, 102b, and 102c via air interface 116. In one embodiment, eNode-Bs 160a, 160b, and 160c may implement MIMO technology. Therefore, for example, eNode-B 160a may use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 102a.

[0047] Each of the eNode-B 160a, 160b, and 160c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, etc. Figure 1C As shown, eNode-B 160a, 160b, and 160c can communicate with each other via the X2 interface.

[0048] Figure 1C The CN 106 shown may include a Mobility Management Entity (MME) 162, a Serving Gateway (SGW) 164, and a Packet Data Network (PDN) Gateway (or PGW) 166. While each of the foregoing elements is described as part of CN 106, it will be understood that any of these elements may be owned and / or operated by an entity other than a CN operator.

[0049] The MME 162 can connect to each of the eNode-Bs 162a, 162b, and 162c in RAN 104 via the S1 interface and can act as a control node. For example, the MME 162 can be responsible for authenticating users of WTRUs 102a, 102b, and 102c, activating / deactivating bearers, selecting a specific serving gateway during the initial attachment of WTRUs 102a, 102b, and 102c, etc. The MME 162 can provide control plane functions for handover between RAN 104 and other RANs (not shown) employing other radio technologies such as GSM and / or WCDMA.

[0050] The SGW 164 can connect to each of the eNode Bs 160a, 160b, and 160c in RAN 104 via the S1 interface. The SGW 164 can typically route and forward user data packets to or from WTRUs 102a, 102b, and 102c. The SGW 164 can perform other functions, such as anchoring the user plane during eNode-B handover, triggering paging when DL data is available to WTRUs 102a, 102b, and 102c, and managing and storing the context of WTRUs 102a, 102b, and 102c.

[0051] SGW 164 can be connected to PGW 166, which can provide WTRU 102a, 102b, 102c with access to packet-switched networks (such as Internet 110) to facilitate communication between WTRU 102a, 102b, 102c and IP-enabled devices.

[0052] CN 106 can facilitate communication with other networks. For example, CN 106 can provide WTRUs 102a, 102b, and 102c with access to circuit-switched networks (such as PSTN 108) to facilitate communication between WTRUs 102a, 102b, and 102c and traditional terrestrial line communication equipment. For example, CN 106 may include an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between CN 106 and PSTN 108, or can communicate with it. Additionally, CN 106 can provide WTRUs 102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0053] Despite WTRU in Figures 1A to 1D While described as a wireless terminal, it is conceivable that in some representative embodiments, such a terminal may (e.g., temporarily or permanently) use a wired communication interface with a communication network.

[0054] In a representative embodiment, the other network 112 may be a WLAN.

[0055] A WLAN in Infrastructure Basic Services Set (BSS) mode may have an Access Point (AP) for the BSS and one or more Stations (STAs) associated with the AP. The AP may access or interface with a Distribution System (DS) or another type of wired / wireless network that carries traffic entering and / or leaving the BSS. Traffic originating outside the BSS destined for a STA can be delivered to the AP. Traffic originating from a STA destined for a destination outside the BSS can be sent to the AP for delivery to the appropriate destination. Traffic between STAs within the BSS can be sent via the AP, for example, where a source STA can send traffic to the AP, and the AP can deliver the traffic to the destination STA. Traffic between STAs within the BSS can be considered and / or referred to as peer-to-peer traffic. Peer-to-peer traffic can be sent between a source STA and a destination STA using a Direct Link Setup (DLS) (e.g., directly between them). In some representative embodiments, the DLS may use 802.11e DLS or 802.11z Tunneled DLS (TDLS). A WLAN using the Standalone BSS (IBSS) mode may not have an access point (AP), and STAs within the IBSS or using the IBSS (e.g., all STAs) can communicate directly with each other. The IBSS communication mode may sometimes be referred to as the "ad-hoc" communication mode in this article.

[0056] When operating in 802.11ac infrastructure mode or a similar mode, the AP can transmit beacons on a fixed channel, such as the primary channel. The primary channel can be of fixed width (e.g., a bandwidth of 20 MHz) or dynamically set via signaling. The primary channel can be the operating channel of the BSS and can be used by the STA to establish a connection with the AP. In some representative embodiments, Carrier Sense Multiple Access (CSMA / CA) with collision avoidance can be implemented, for example, in an 802.11 system. For CSMA / CA, the AP STA (e.g., each STA) can sense the primary channel. If a particular STA senses / detects that the primary signal is busy and / or determines that the primary signal is busy, that particular STA can back off. In a given BSS, at any given time, only one STA (e.g., only one station) can transmit.

[0057] High-throughput (HT) STAs can communicate using a 40 MHz wide channel, for example, by combining a primary 20 MHz channel with adjacent or non-adjacent 20 MHz channels.

[0058] Very High Throughput (VHT) STAs can support channels with widths of 20 MHz, 40 MHz, 80 MHz, and / or 160 MHz. 40 MHz and / or 80 MHz channels can be formed by combining consecutive 20 MHz channels. A 160 MHz channel can be formed by combining eight consecutive 20 MHz channels, or by combining two non-consecutive 80 MHz channels, which can be referred to as an 80+80 configuration. In the 80+80 configuration, data, after channel coding, can be passed through a fragment resolver that splits the data into two streams. Inverse Fast Fourier Transform (IFFT) processing and time-domain processing can be performed on each stream separately. The streams can be mapped onto the two 80 MHz channels, and the data can be transmitted by the transmitting STA. At the receiver of the receiving STA, the above operations for the 80+80 configuration can be reversed, and the combined data can be sent to the Media Access Control (MAC).

[0059] 802.11af and 802.11ah support operating modes below 1 GHz. The channel operating bandwidth and carrier in 802.11af and 802.11ah are reduced compared to those used in 802.11n and 802.11ac. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV Blank (TVWS) spectrum, and 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11ah may support instrument-type control / machine-type communication (MTC), such as MTC devices in macro coverage areas. MTC devices may have certain capabilities, including, for example, limited capabilities to support (e.g., only support) certain and / or limited bandwidths. MTC devices may include batteries with a battery life exceeding a threshold (e.g., to maintain a very long battery life).

[0060] WLAN systems that can support multiple channels and channel bandwidths (such as 802.11n, 802.11ac, 802.11af, and 802.11ah) include a channel that can be designated as the primary channel. The primary channel can have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel can be set and / or limited by the STAs operating in the BSS that support the minimum bandwidth operating mode. In the 802.11ah example, for STAs that support (e.g., only support) the 1 MHz mode (e.g., MTC type devices), the primary channel can be 1 MHz wide even if the AP and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and / or other channel bandwidth operating modes. Carrier Sense and / or Network Assignment Vector (NAV) settings can depend on the status of the primary channel. If the primary channel is busy, for example, because an STA (which only supports the 1 MHz operating mode) is transmitting to the AP, the entire available band may be considered busy even if most of the band remains idle and potentially available.

[0061] In the United States, the available frequency band for 802.11ah is 902 MHz to 928 MHz. In South Korea, the available frequency band is 917.5 MHz to 923.5 MHz. In Japan, the available frequency band is 916.5 MHz to 927.5 MHz. The total available bandwidth for 802.11ah is 6 MHz to 26 MHz, depending on the country code.

[0062] Figure 1D This is a system diagram illustrating RAN 113 and CN 115 according to an embodiment. As described above, RAN 113 may employ NR radio technology to communicate with WTRUs 102a, 102b, and 102c via air interface 116. RAN 113 may also communicate with CN 115.

[0063] RAN 113 may include gNBs 180a, 180b, and 180c, but it will be understood that RAN 113 may include any number of gNBs while remaining consistent with the embodiments. gNBs 180a, 180b, and 180c may each include one or more transceivers for communicating with WTRUs 102a, 102b, and 102c via air interface 116. In one embodiment, gNBs 180a, 180b, and 180c may implement MIMO technology. For example, gNBs 180a and 180b may utilize beamforming to transmit signals to and / or receive signals from gNBs 180a, 180b, and 180c. Thus, for example, gNB 180a may use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 102a. In an embodiment, gNBs 180a, 180b, and 180c may implement carrier aggregation technology. For example, gNB 180a can transmit multiple component carriers to WTRU 102a (not shown). A subset of these component carriers may be located on unlicensed spectrum, while the remaining component carriers may be located on licensed spectrum. In embodiments, gNBs 180a, 180b, and 180c can implement Coordinated Multipoint (CoMP) technology. For example, WTRU 102a can receive coordinated transmissions from gNBs 180a and 180b (and / or gNB 180c).

[0064] WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using transmissions associated with a scalable set of parameters. For example, the OFDM symbol spacing and / or OFDM subcarrier spacing can be varied for different transmissions, different cells, and / or different portions of the radio transmission spectrum. WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using subframes or transmission time intervals (TTIs) of various lengths or scalable lengths (e.g., including different numbers of OFDM symbols and / or absolute times of varying durations).

[0065] gNBs 180a, 180b, and 180c can be configured to communicate with WTRUs 102a, 102b, and 102c in standalone and / or non-standalone configurations. In standalone configuration, WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c without accessing other RANs (e.g., eNodeBs 160a, 160b, and 160c). In standalone configuration, WTRUs 102a, 102b, and 102c can use one or more of gNBs 180a, 180b, and 180c as mobile anchors. In standalone configuration, WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using signals in unlicensed frequency bands. In a non-standalone configuration, WTRUs 102a, 102b, and 102c can communicate / connect with gNBs 180a, 180b, and 180c while also communicating / connecting with another RAN (such as eNode-Bs 160a, 160b, and 160c). For example, WTRUs 102a, 102b, and 102c can implement DC principles to communicate substantially simultaneously with one or more gNBs 180a, 180b, and 180c and one or more eNode-Bs 160a, 160b, and 160c. In a non-standalone configuration, eNode-Bs 160a, 160b, and 160c can act as mobile anchors for WTRUs 102a, 102b, and 102c, and gNBs 180a, 180b, and 180c can provide additional coverage and / or throughput to serve WTRUs 102a, 102b, and 102c.

[0066] Each of gNBs 180a, 180b, and 180c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in uplink (UL) and / or downlink (DL), support for network slicing, dual connectivity, interoperability between NR and E-UTRA, routing of user plane data to User Plane Functions (UPF) 184a and 184b, routing of control plane information to Access and Mobility Management Functions (AMF) 182a and 182b, etc. Figure 1D As shown, gNB 180a, 180b, and 180c can communicate with each other via the Xn interface.

[0067] Figure 1DThe CN 115 shown may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one Session Management Function (SMF) 183a, 183b, and possibly a Data Network (DN) 185a, 185b. While each of the foregoing elements is described as part of the CN 115, it will be understood that any of these elements may be owned and / or operated by an entity other than a CN operator.

[0068] AMF 182a and 182b can connect to one or more of the gNBs 180a, 180b, and 180c in RAN 113 via the N2 interface and can act as control nodes. For example, AMF 182a and 182b can be responsible for authenticating users of WTRU 102a, 102b, and 102c, supporting network slicing (e.g., handling different PDU sessions with different requirements), selecting specific SMFs 183a and 183b, managing registration areas, terminating NAS signaling, mobility management, etc. AMF 182a and 182b can use network slicing to customize CN support for WTRU 102a, 102b, and 102c based on the service types being used by WTRU 102a, 102b, and 102c. For example, different network slices can be established for different use cases, such as services relying on Ultra Reliable Low Latency (URLLC) access, services relying on Enhanced Massive Mobile Broadband (eMBB) access, and services for Machine Type Communication (MTC) access. AMF 162 can provide control plane functions for handover between RAN 113 and other RANs (not shown) that employ other radio technologies (such as LTE, LTE-A, LTE-A Pro) and / or non-3GPP access technologies (such as WiFi).

[0069] SMFs 183a and 183b can connect to AMFs 182a and 182b in CN 115 via the N11 interface. SMFs 183a and 183b can also connect to UPFs 184a and 184b in CN 115 via the N4 interface. SMFs 183a and 183b can select and control UPFs 184a and 184b, and configure them to route traffic through UPFs 184a and 184b. SMFs 183a and 183b can perform other functions, such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notifications. PDU session types can be IP-based, non-IP-based, or Ethernet-based.

[0070] UPF 184a and 184b can connect via the N3 interface to one or more of the gNBs 180a, 180b, and 180c in RAN 113. These gNBs can provide WTRU 102a, 102b, and 102c with access to packet-switched networks (such as the Internet 110) to facilitate communication between WTRU 102a, 102b, and 102c and IP-enabled devices. UPF 184 and 184b can perform other functions such as routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering downlink packets, and providing mobility anchoring.

[0071] CN 115 can facilitate communication with other networks. For example, CN 115 may include or be able to communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between CN 115 and PSTN 108. Additionally, CN 115 can provide WTRUs 102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, WTRUs 102a, 102b, and 102c can be connected to DN 185a and 185b via UPF 184a and 184b through the N3 interface to UPF 184a and 184b and the N6 interface between UPF 184a and 184b and local data networks (DNs) 185a and 185b.

[0072] Given Figures 1A to 1D and Figures 1A to 1D The corresponding descriptions can be performed by one or more emulation devices (not shown) that perform one or more of the functions described herein with respect to: WTRU 102a to 102d, base stations 114a to 114b, eNode-B 160a to 160c, MME 162, SGW 164, PGW 166, gNB 180a to 180c, AMF 182a to 182b, UPF 184a to 184b, SMF 183a to 183b, DN 185a to 185b, and / or any other devices described herein. An emulation device can be one or more devices configured to emulate one or more of the functions described herein. For example, an emulation device can be used to test other devices and / or simulate network and / or WTRU functions.

[0073] Simulation devices can be designed to perform one or more tests on other devices in laboratory and / or carrier network environments. For example, one or more simulation devices may perform one or more functions when fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices within the communication network. One or more simulation devices may perform one or more functions when temporarily implemented / deployed as part of a wired and / or wireless communication network. Simulation devices may be directly coupled to another device for testing purposes and / or may use over-the-air wireless communication to perform tests.

[0074] One or more simulation devices may perform one or more functions without being implemented / deployed as part of a wired and / or wireless communication network. For example, a simulation device may be used to test scenarios in a laboratory and / or an undeployed (e.g., tested) wired and / or wireless communication network to perform testing of one or more components. One or more simulation devices may be test devices. Simulation devices may transmit and / or receive data using direct RF coupling and / or wireless communication via an RF circuit system (e.g., which may include one or more antennas).

[0075] This application describes various aspects, including tools, features, examples, models, methods, etc. Many of these aspects are described in a specific manner, and are generally described in a way that may sound restrictive, at least to illustrate the individual features. However, this is for the purpose of clarity of description and does not limit the application or scope of these aspects. In fact, all the different aspects can be combined and interchanged to provide other aspects. Furthermore, these aspects can also be combined and interchanged with those described in previous documents.

[0076] The aspects described and envisioned in this application can be implemented in many different forms. Figures 5 to 8 Some examples can be provided, but other examples are envisioned. Figures 5 to 8 The discussion does not limit the breadth of implementations. At least one aspect generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting the generated or encoded bitstream. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions thereon stored thereon for encoding or decoding video data according to any of the methods, and / or computer-readable storage media having bitstreams generated according to any of the methods stored thereon.

[0077] In this application, the terms “reconstruction” and “decoding” are used interchangeably, the terms “pixel” and “sample” are used interchangeably, and the terms “image”, “picture” and “frame” are used interchangeably.

[0078] This document describes various methods, each of which includes one or more steps or actions to implement the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined. Additionally, in various examples, terms such as "first," "second," etc., may be used to modify elements, components, steps, operations, etc., such as, for example, "first decoding" and "second decoding." Unless specifically required, the use of such terms does not imply a sequence of operations. Therefore, in this example, the first decoding does not need to be performed before the second decoding, but can be performed, for example, before, during, or in a time period overlapping with the second decoding.

[0079] The various methods and other aspects described in this application can be used to modify, for example... Figure 2 and Figure 3 The illustrated modules of the video encoder 200 and decoder 300 include, for example, a decoding module. Furthermore, the subject matter disclosed herein can be applied to, for example, any type, format, or version of video coding, whether described in standards or recommendations, whether pre-existing or future-developed, and any extensions to such standards and recommendations. Unless otherwise stated or technically excluded, these aspects described in this application may be used alone or in combination.

[0080] Various numerical values, such as block size, table value, number of samples, reference region, number of candidates, number of weights, number of patterns, weight values, constants, etc., are used in the examples described in this application. These and other specific values ​​are for illustrative purposes only, and the aspects described are not limited to these specific values.

[0081] Figure 2 This is a diagram illustrating an exemplary video encoder. Figure 2 An example of a block-based hybrid video encoder is shown. Variations of the exemplary encoder 200 are envisioned, but for clarity, encoder 200 is described below without describing all anticipated variations.

[0082] Before being encoded, the video sequence may undergo pre-coding processing 201 (e.g., applying color transformations to the input color picture (e.g., a conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing remapping of the input picture components) to obtain a signal distribution more suited to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and appended to the bitstream.

[0083] In encoder 200, the frame is encoded by encoder elements as described below. The frame to be encoded is partitioned (202) and processed in units such as coding units (CUs). Each unit is encoded using, for example, an intra-frame or inter-frame mode. When a unit is encoded in intra-frame mode, intra-frame prediction (260) is performed. In inter-frame mode, motion estimation (275) and compensation (270) are performed. The encoder determines (205) which mode, intra-frame or inter-frame, to use to encode the unit, and indicates the intra-frame / inter-frame decision by, for example, a prediction mode flag. For example, the prediction residual is calculated by subtracting (210) the prediction block from the original image block.

[0084] The prediction residual is then transformed (225) and quantized (230). The quantized transform coefficients, along with the motion vector and other syntax elements (e.g., screen partition information), are entropy encoded (245) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., directly encode the residual without applying either the transform or quantization process.

[0085] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (255) to reconstruct the image block. An in-loop filter (265) is applied to the reconstructed image to perform, for example, deblocking / SAO (Sample Adaptive Offset) / ALF (Adaptive Loop Filtering) filtering, thereby reducing coding artifacts. The filtered image is stored in a reference image buffer (280).

[0086] Figure 3 This is a diagram illustrating an exemplary video decoder. In the exemplary decoder 300, the bitstream is decoded by decoder elements, as described below. The video decoder 300 typically performs operations similar to... Figure 2 The encoding process described herein is the inverse of the decoding process. Encoder 200 typically also performs video decoding as part of the encoded video data.

[0087] Specifically, the input to the decoder includes a video bitstream, which can be generated by the video encoder 200. First, entropy decoding (330) is performed on the bitstream to obtain transform coefficients, prediction modes, motion vectors, and other encoded information. Picture partitioning information indicates how the picture should be partitioned. Therefore, the decoder can partition (335) the picture based on the decoded picture partitioning information. The transform coefficients are dequantized (340) and inverse transformed (350) to decode the prediction residuals. The decoded prediction residuals and prediction blocks are combined (355) to reconstruct the image blocks. Prediction blocks (370) can be obtained from intra-frame prediction (360) or motion-compensated prediction (i.e., inter-frame prediction) (375). An in-loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (380). In some examples (e.g., for a given picture), the contents of the reference picture buffer 380 on the decoder 300 side can be the same as the contents of the reference picture buffer 280 on the encoder 200 side (e.g., for the same picture).

[0088] The decoded image can also undergo post-decoding processing (385), such as inverse color transformation (e.g., a conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping, which is the inverse of the remapping process performed in the pre-encoding process (201). Post-decoding processing can use metadata derived in the pre-encoding process and signaled in the bitstream. In the example, the decoded image (e.g., after applying an in-loop filter (365) and / or, in the case of post-decoding processing, after post-decoding processing (385)) can be sent to a display device for presentation to the user.

[0089] Figure 4 This is a diagram illustrating examples of systems in which the various aspects and examples described herein can be implemented. System 400 can be implemented as a device including the various components described below and configured to perform one or more aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 400 can be implemented individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one example, the processing elements and encoder / decoder elements of system 400 are distributed across multiple ICs and / or discrete components. In various examples, system 400 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various examples, system 400 is configured to implement one or more aspects described in this document.

[0090] System 400 includes at least one processor 410 configured to execute instructions loaded thereon to implement various aspects described herein, such as those described. Processor 410 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 400 includes at least one memory 420 (e.g., a volatile memory device and / or a non-volatile memory device). System 400 includes a storage device 440 that may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 440 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0091] System 400 includes an encoder / decoder module 430 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 430 may include its own processor and memory. The encoder / decoder module 430 represents a module that can be included in a device to perform encoding and / or decoding functions. It is well known that a device can include one or both of the encoding and decoding modules. Alternatively, the encoder / decoder module 430 may be implemented as a separate element of system 400, or it may be incorporated into processor 410 as a combination of hardware and software known to those skilled in the art.

[0092] Program code to be loaded onto processor 410 or encoder / decoder 430 to execute the various aspects described in this document may be stored in storage device 440 and subsequently loaded onto memory 420 for execution by processor 410. Depending on various examples, one or more of processor 410, memory 420, storage device 440, and encoder / decoder module 430 may store one or more of various items during the execution of the processes described in this document. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from equations, formulas, operations, and operational logic processing.

[0093] In some examples, the memory within processor 410 and / or encoder / decoder module 430 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other examples, external memory (e.g., the processing device could be processor 410 or encoder / decoder module 430) is used for one or more of these functions. External memory could be memory 420 and / or storage device 440, such as volatile memory and / or non-volatile flash memory. In several examples, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one example, fast external volatile memory such as RAM is used as working memory for video encoding and decoding operations.

[0094] Inputs to the components of system 400 can be provided through various input devices, as indicated in box 445. Such input devices include, but are not limited to, (i) a radio frequency (RF) section that receives, for example, RF signals transmitted over the air by a broadcaster, (ii) a component (COMP) input terminal (or a set of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Figure 4 Other examples not shown include composite video.

[0095] In various examples, the input device of block 445 has associated input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting the signal band to a band), (ii) down-converting the selected signal, (iii) further band-limiting to a narrower band to select (e.g.,) a signal band that may be referred to as a channel in some examples), (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and / or (vi) demultiplexing to select the desired data packet stream. The RF section of various examples includes one or more elements performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners performing various functions among these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one set-top box example, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and filtering again to the desired frequency band. Various examples rearrange the above (and other) components, remove some of them, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as inserting amplifiers and analog-to-digital converters. In various examples, the RF section includes an antenna.

[0096] USB and / or HDMI terminals may include corresponding interface processors for connecting system 400 to other electronic devices via USB and / or HDMI connections. It should be understood that aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, within a separate input processing IC or within processor 410 as needed. Similarly, aspects of USB or HDMI interface processing may be implemented, as needed, within a separate interface IC or within processor 410. The demodulated, error-corrected, and demultiplexed stream is provided to various processing elements (including, for example, processor 410 and encoder / decoder 430), which operate in conjunction with memory and storage elements to process the data stream as needed for presentation on the output device.

[0097] Various components of system 400 can be housed within an integrated housing. Within the integrated housing, the various components can be interconnected and transmit data between them using suitable connection devices 425 (e.g., internal buses known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards).

[0098] System 400 includes a communication interface 450 that enables communication with other devices via a communication channel 460. The communication interface 450 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 460. The communication interface 450 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 460 may be implemented, for example, within a wired and / or wireless medium.

[0099] In various examples, wireless networks, such as Wi-Fi networks (e.g., IEEE 802.11, where IEEE refers to the Institute of Electrical and Electronics Engineers), are used to stream or otherwise provide data to system 400. In these examples, the Wi-Fi signal is received via a communication channel 460 and a communication interface 450 adapted for Wi-Fi communication. The communication channel 460 in these examples is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other examples use a set-top box to provide streaming data to system 400, delivering data via an HDMI connection to input block 445. Still other examples use an RF connection to input block 445 to provide streaming data to system 400. As indicated above, various examples provide data in non-streaming modes. Additionally, various examples use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth® networks.

[0100] System 400 can provide output signals to various output devices, including display 475, speaker 485, and other peripheral devices 495. Various examples of display 475 include one or more of, for example, touchscreen displays, organic light-emitting diode (OLED) displays, curved displays, and / or foldable displays. Display 475 can be used in televisions, tablets, laptops, cellular phones (mobile phones), or other devices. Display 475 can also be integrated with other components (e.g., in a smartphone) or separate (e.g., an external monitor for a laptop computer). In various examples, other peripheral devices 495 include one or more of stand-alone digital video discs (or digital multifunction discs) (DVDs, for both terms), disc players, stereo systems, and / or lighting systems. Various examples use one or more peripheral devices 495 that provide functionality based on the output of system 400. For example, a disc player performs the function of playing the output of system 400.

[0101] In various examples, signaling (such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention) is used to transmit control signals between system 400 and display 475, speaker 485, or other peripheral devices 495. Output devices may be communicatively coupled to system 400 via dedicated connections through corresponding interfaces 470, 480, and 490. Alternatively, output devices may be connected to system 400 via communication interface 450 using communication channel 460. Display 475 and speaker 485 may be integrated into a single unit along with another component of system 400 in an electronic device, such as a television. In various examples, display interface 470 includes a display driver, such as a timing controller (TCon) chip.

[0102] For example, if the RF input section 445 is part of a separate set-top box, the display 475 and speaker 485 can alternatively be separate from one or more of the other components. In various examples where the display 475 and speaker 485 are external components, the output signal can be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0103] The example can be executed by processor 410 or by computer software implemented by hardware or a combination of hardware and software. As a non-limiting example, the example can be implemented by one or more integrated circuits. As a non-limiting example, memory 420 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. As a non-limiting example, processor 410 can be of any type suitable for the technical environment and can encompass one or more microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0104] Various implementations involve decoding. As used herein, "decoding" can encompass all or part of a process performed, for example, on a received encoded sequence to produce a final output suitable for display. In various examples, such a process includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various examples, such a process may also include, or alternatively may include, processes performed by the decoder of the various implementations described herein.

[0105] As another example, in one example, "decoding" refers only to entropy decoding; in another example, "decoding" refers only to differential decoding; and in yet another example, "decoding" refers to a combination of entropy decoding and differential decoding. Based on the specific context of the description, it will be clear whether the phrase "decoding process" is intended to specifically refer to a subset of operations or to refer to the broader decoding process, and it is believed that those skilled in the art will understand this well.

[0106] Various implementations involve encoding. Similar to the discussion of "decoding" above, "encoding" as used in this application can include, for example, all or part of a process performed on an input video sequence to produce an encoded bitstream. In various examples, such processes include one or more processes typically performed by an encoder, such as partitioning, differential coding, transform, quantization, and entropy coding. In various examples, such processes can also include, or alternatively can include, processes performed by an encoder of the various implementations described in this application, such as: determining a candidate list of Local Illumination Compensation (LIC) models; determining a candidate list of Intra-Block Copy (IBC) models; selecting an IBC merge candidate from the IBC candidate list; reordering the LIC model candidate list; selecting a LIC model based on the reordered LIC model candidate list; and so on.

[0107] As another example, in one example, "encoding" refers only to entropy encoding; in another example, "encoding" refers only to differential encoding; and in yet another example, "encoding" refers to a combination of entropy encoding and differential encoding. It will be clear from the specific context of the description whether the phrase "encoding process" is intended to refer specifically to a subset of operations or to a broader encoding process, and it is believed that those skilled in the art will understand this well.

[0108] It should be noted that the names of grammatical elements used in this article are descriptive terms only. Therefore, the use of other grammatical element names is not excluded.

[0109] When a diagram is presented as a flowchart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flowchart of the corresponding method / process.

[0110] The implementations and aspects described herein can be implemented in, for example, methods or processes, apparatuses, software programs, data streams, or signals. Even if discussed only in the context of a single implementation (e.g., discussed only as a method), the implementation of the features in question can be implemented in other forms (e.g., apparatuses or programs). Apparatuses can be implemented, for example, with appropriate hardware, software, and firmware. Methods can be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as computers, cellular phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0111] The reference to “an example” or “an example” or “an implementation” or “an implementation”, and their variations, means that the specific features, structures, characteristics, etc., described in connection with the example are included in at least one example. Therefore, the phrases “in an example” or “in the example” or “in an implementation” or “in the implementation”, and any other variations, appearing throughout this application, do not necessarily refer to the same example.

[0112] Additionally, this application may relate to "determining" various types of information. Determining information may include one or more of the following: for example, estimation information, calculation information, prediction information, and information retrieved from memory. Obtaining may include receiving, retrieving, constructing, generating, and / or determining.

[0113] Furthermore, this application may involve "accessing" various types of information. Accessing information may include one or more of the following: for example, receiving information, retrieving information (e.g., retrieving information from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, and estimating information.

[0114] Additionally, this application may relate to "receiving" various types of information. Like "accessing," receiving is a broad term. Receiving information may include one or more of the following: for example, accessing information and retrieving information (e.g., retrieving information from memory). Furthermore, "receiving" is generally referred to in one or more ways during operation, such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0115] It should be understood that the use of any of the following “ / ”, “and / or”, and “…” (e.g., in the cases of “A / B”, “A and / or B”, and “at least one of A and B”) is intended to cover selecting only the first listed option (A), or only the second listed option (B), or selecting both options (A and B). As yet another example, in the cases of “A, B, and / or C” and “at least one of A, B, and C”, this wording is intended to include selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or selecting all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to a large number of listed items.

[0116] Furthermore, as used herein, the term "signaling notification" specifically refers to the corresponding decoder indicating certain information. Encoder signals may include, for example, whether or not IBC is used. Indicators of LIC model merging mode, and whether SGPM mode is compatible with IBC. The LIC pattern combination uses indicators, IBC merging candidates, indices associated with the IBC candidate list, and indices associated with the LIC model candidate list, etc. In this way, in the examples, the same parameters are used on both the encoder and decoder sides. Therefore, for example, the encoder can send (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. Conversely, if the decoder already has specific parameters as well as other parameters, signaling can be used without sending (implicit signaling) to simply allow the decoder to know and select specific parameters. Bit savings are achieved in various examples by avoiding the transmission of any actual functionality. It should be understood that signaling can be implemented in various ways. For example, in various examples, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the verb form of the word "signal" was mentioned above, the word "signal" can also be used as a noun in this paper.

[0117] As will be apparent to those skilled in the art, implementations can generate various signals, which are formatted to carry, for example, information that can be stored or transmitted. The information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream of the described example. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. It is well known that signals can be transmitted via a variety of different wired or wireless links. Signals may be stored on, accessed from, or received from, a processor-readable medium.

[0118] This document describes numerous examples. Features of the examples may be provided individually or in any combination across various claim classes and types. Furthermore, examples may include one or more of the features, devices, or aspects described herein, individually or in any combination across various claim classes and types. For example, the features described herein may be implemented in a bitstream or signal that includes information generated as described herein. This information may allow a decoder to decode the bitstream, and the encoder, bitstream, and / or decoder may be any of the embodiments described. For example, the features described herein may be implemented by creating and / or transmitting and / or receiving and / or decoding a bitstream or signal. For example, the features described herein may be implemented by a method, process, apparatus, medium storing instructions, medium storing data, or signal. For example, the features described herein may be implemented by a TV, set-top box, cellular phone, tablet computer, or other electronic device performing decoding. The TV, set-top box, cellular phone, tablet computer, or other electronic device may (e.g., using a monitor, screen, or other type of display) display the resulting image (e.g., an image reconstructed from the residual of a video bitstream). The TV, set-top box, cellular phone, tablet computer, or other electronic device may receive a signal including the encoded image and perform decoding.

[0119] Systems, methods, and means for performing operations associated with intra-block copy (IBC) local illumination compensation (LIC) merging modes are disclosed. The IBC-LIC model merging mode can be executed, and may include, for example, creating a model candidate list comprising model parameters from spatially adjacent blocks, non-adjacent blocks, historical candidates, default modes, etc. The model candidate list can be reordered based on template cost. The reordering of the model parameter candidate list can be improved, for example, to reduce computational overhead. For example, the LIC model candidates can be reordered based on determined merging candidates (e.g., best merging candidates).

[0120] A device (e.g., a video encoder or video encoder) can reorder a list of model parameter candidates, for example, to reduce computational complexity and improve compression efficiency. A combination of intra-block copying (IBC) and local illumination compensation (LIC) can be performed (e.g., as an IBC-LIC mode). An IBC-LIC model merging mode can be performed. A LIC model candidate list can be determined, and an IBC candidate list can be determined. An IBC merge candidate can be selected (e.g., from the IBC candidate list). The IBC merge candidate list can be sorted based on template cost. An IBC merge candidate can be selected based on template cost (e.g., the IBC merge candidate with the best template cost). The LIC model candidate list can be reordered, for example, based on the selected IBC merge candidate. For example, a LIC model (e.g., a LIC model candidate) can be selected based on the reordered LIC model candidate list. The selected LIC model can be the first LIC model candidate in the reordered LIC model candidate list. An indication (e.g., contained in the video data) can indicate whether the IBC-LIC model merging mode is used, and can indicate the selected IBC merge candidate.

[0121] The LIC model candidate list can be divided into multiple candidate lists (e.g., a primary LIC model candidate list and an auxiliary LIC model candidate list). LIC model candidates to be included in the primary and auxiliary LIC model candidate lists can be determined (e.g., based on the spatial distance associated with each LIC model candidate). For example, the primary LIC model candidate list may include LIC model candidates associated with spatially adjacent model candidates. The auxiliary LIC model candidate list may include the remaining LIC model candidates. The primary LIC model candidate list may include LIC model candidates based on the order of LIC model candidates in the LIC model candidate list (e.g., the first number of LIC model candidates in the LIC model candidate list (such as, for example, the first six LIC model candidates in the LIC model candidate list) included in the primary LIC model candidate list). The auxiliary LIC model candidate list may include LIC model candidates after the first number of LIC model candidates in the LIC model candidate list (e.g., LIC model candidates after the first six LIC model candidates in the LIC model candidate list).

[0122] Spatial Geometric Partitioning (SGPM) can be used in conjunction with the IBC-LIC pattern. An SGPM partitioning pattern for use with the IBC-LIC pattern can be determined. For example, a primary LIC model candidate list can be determined based on the determined SGPM partitioning pattern. For instance, LIC model candidates using IBC-LIC and originating from neighboring spatial candidates on the SGPM partitioning side can be added to the primary candidate list. The remaining LIC model candidates can be added to the auxiliary candidate list.

[0123] A mixture associated with multiple (e.g., two) LIC models can be performed, for example, to determine a final prediction. For instance, a first LIC model and a second LIC model can be selected based on a reordered list of LIC model candidates. A first prediction can be determined based on applying the first LIC model to a reference sample, and a second prediction can be determined based on applying the second model to a reference sample. A final prediction can be determined based on the mixture of the first and second predictions. The mixture can be performed based on mixture weights. The mixture weights can be determined, for example, based on template cost. Alternatively, the mixture can include averaging the predictions (e.g., two predictions), for example, with a mixture weight of 0.5.

[0124] Local illumination compensation (LIC) can be performed. LIC models the local illumination variation between the current block and its predicted block as a function of the local illumination variation between the current block template and the reference block template. The parameters of this function can be represented by scaling α and offset β, which can be formed, for example... A linear equation is used to compensate for illumination variations. Referring to an exemplary linear equation, p[x] can be a reference sample pointed to by the motion vector (MV) at position x on the reference image. For example, if / when surround motion compensation is enabled, surround offset can be considered to limit the MV. The LIC flag can be signaled for AMVP mode to indicate the use of LIC. The LIC parameters α and β can be derived based on the current block template and the reference block template.

[0125] Local illumination compensation can be used for unidirectional prediction of inter-frame CUs. Intra-frame neighbor samples can be used for LIC parameter derivation. LIC can be disabled for blocks with fewer than a threshold number of luminance samples (e.g., 32 luminance samples). For example, LIC parameter derivation can be performed based on template block samples corresponding to the current coding unit (CU), rather than partial template block samples corresponding to a specific unit (e.g., the first 16x16 unit in the top left corner) (e.g., for non-sub-blocks and affine modes). For example, samples of a reference block template can be generated by using block MV for motion compensation (MC) without rounding to integer pixel (pixel element) precision.

[0126] Intra-block copy coding can be performed. For example, intra-block copy (IBC) is a tool that can be used for encoding screen content. IBC can improve the coding efficiency of screen content material. For instance, block matching (BM) can be performed at the encoder to find the (e.g., optimal) block vector or motion vector for (e.g., each) CU, because the IBC mode is implemented as a block-level coding mode. The block vector can indicate the displacement from the current block to a reference block that has been reconstructed within the current frame. The luminance block vector of the IBC-encoded CU can have integer precision. The chrominance block vector can be rounded to integer precision. The IBC mode can switch between precisions (e.g., between 1 pixel and 4 pixel motion vector precision) when combined with Adaptive Motion Vector Resolution (AMVR), for example. The IBC-encoded CU can be considered as another (e.g., a third) prediction mode (e.g., in addition to intra-prediction mode or inter-prediction mode). The IBC mode can be applied to CUs with a width and height of, for example, less than or equal to 64 luminance samples.

[0127] At the CU level, flags can be used to signal the IBC mode. The IBC mode can be signaled as either IBC Advanced Motion Vector Prediction (AMVP) mode or IBC Skip / Merge mode. In an example of IBC Skip / Merge mode, a merge candidate index can be used to indicate which block vectors from a list of neighboring candidate IBC decoded blocks are used to predict the current block. The merge list can include spatial, history-based motion vector predictions (HMVP) and / or paired candidates. In an example of IBC AMVP mode, block vector differences can be encoded in the same way as motion vector differences. The block vector prediction method can use multiple (e.g., two) candidates as predictors, such as one from the left neighbor and one from the upper neighbor (e.g., if IBC encoded). For example, a default block vector can be used as the predictor if / when neighboring blocks are unavailable. Signals can be used to indicate the block vector predictor index (e.g., block vector predictor index indicator).

[0128] The IBC reference region can be identified. The IBC may (e.g., only) allow reconstructed portions of predefined regions (e.g., including at least a portion of the current CTU region and the region of the left CTU), for example, to limit memory consumption and / or decoder complexity. Figure 5 The reference area for the IBC mode is shown. Figure 5 Each of the exemplary blocks shown can represent a 64x64 luminance sample cell. Figure 5 This shows an example of the current CTU processing order and available reference samples in the current and left CTUs.

[0129] The IBC reference area can be identified based on the location of the current coded CU within the current CTU.

[0130] For example, the current block processing can refer to samples that have already been reconstructed in the current CTU. For instance, if the current block falls within the top-left 64x64 block of the current CTU, the current block can, for example, use IBC mode to refer to reference samples in the bottom-right 64x64 block of the left CTU. The current block can, for example, use IBC mode to refer to reference samples in the bottom-left 64x64 block of the left CTU and the top-right 64x64 block of the left CTU.

[0131] For example, current block processing can reference already reconstructed samples in the current CTU. For instance, if the current block falls within the top-right 64x64 block of the current CTU and if the brightness position (0, 64) relative to the current CTU has not yet been reconstructed, the current block processing can, for example, use IBC mode to reference reference samples in the bottom-left and bottom-right 64x64 blocks of the left CTU. Otherwise, if the brightness position (0, 64) relative to the current CTU has been reconstructed, the current block can also reference reference samples in the bottom-right 64x64 block of the left CTU.

[0132] The current block processing can reference already reconstructed samples in the current CTU. For example, if the current block falls within the lower left 64x64 block of the current CTU and if the brightness position (64, 0) relative to the current CTU has not yet been reconstructed, the current block can, for example, use IBC mode to reference reference samples in the upper right and lower right 64x64 blocks of the left CTU. Otherwise, if the brightness position (64, 0) relative to the current CTU has been reconstructed, the current block can also, for example, use IBC mode to reference reference samples in the lower right 64x64 block of the left CTU.

[0133] For example, if the current block falls within the bottom right 64x64 block of the current CTU, then the current block processing (e.g., only) may use IBC mode to reference reconstructed samples in the current CTU.

[0134] Limiting current block processing based on the location of the current encoding CU within the current CTU allows IBC mode to be implemented using local on-chip memory for hardware implementation.

[0135] This allows for the construction of an IBC merging / Advanced Motion Vector Prediction (AMVP) list. For example, IBC merging / AMVP candidates can be inserted into the IBC merging / AMVP candidate list if (e.g., only if) they are valid. Upper-right spatial candidates, lower-left spatial candidates, and upper-left spatial candidates, along with (e.g., a) pairwise average candidates, can be added to the IBC merging / AMVP candidate list. Template-based adaptive reordering of merged candidates (ARMC-TM) can be applied to the IBC merging list.

[0136] The size of the History-Based Motion Vector Predictor (HMVP) table for IBC can be, for example, 25 entries. Merge candidates can be reordered. For example, after obtaining up to 20 IBC merge candidates through full pruning, the merge candidates can be reordered together. Candidates can be selected after reordering. For example, candidates with the lowest template matching cost (e.g., the top 6 candidates) can be selected as the final candidates in the IBC merge list.

[0137] Candidate zero vectors filling the IBC merge / AMVP list can be replaced by a set of block vector prediction (BVP) candidates located in the IBC reference region. Zero vectors can be considered block vectors in the IBC merge mode. Invalid block vectors in the IBC candidate list that are BVPs can be discarded.

[0138] In the example, multiple (e.g., three) candidates may be located at the nearest corner of the reference region, and multiple (e.g., three) additional candidates may be determined at the middle position of three sub-regions (A, B, and C), the coordinates of which may be determined by the width and height of the current block and the parameters ΔX and ΔY, as shown. Figure 6 The example depicted in the text. Figure 6 An example of a filler candidate is shown for replacing a zero vector in the IBC list.

[0139] Figure 7 An example of an IBC reference area is shown. An IBC reference area may include two CTU rows above the CTU processed by the encoder or decoder. Figure 7 The reference region for encoding CTU (m, n) is shown. CTU (m, n) can be encoded based on the reference region including CTUs with indices (m–2, n–2)…(W, n–2), (0, n–1)…(W, n–1), (0, n)…(m, n). W can represent the maximum horizontal index within the current tile, slice, or frame. The per-sample block vector search (or local search) range can be limited, for example, horizontally to [–(C<1), C>2] and vertically to [–C, C>2], to fit, for example, the reference region. C can represent the CTU size.

[0140] Intra-block copying (IBC) can be implemented using template matching (TM). For example, template matching-based motion search and refinement can be applied to IBC.

[0141] The IBC-TM merging mode can be used to implement IBC using template matching. The IBC-TM merging mode may involve a merging candidate list for block vector (BV) prediction, which differs from the candidate list used in the IBC merging mode. Candidates can be selected based on a pruning method with motion distances between candidates (e.g., as in the TM merging mode). Zero-motion candidates can be replaced by (-W, 0), (0, -H), (-W, -H) MV.

[0142] In IBC-TM merge mode, template matching can be used to refine the selected candidates. Indicators such as the TM merge flag can be signaled to indicate the use of template matching merge mode in IBC.

[0143] In an exemplary IBC-TM AMVP pattern, multiple (e.g., up to three (3)) candidates can be selected from the IBC-TM merge list. Each candidate can be refined (e.g., according to a (e.g., regular) template matching method. The refined candidates can be sorted according to their resulting TM cost.

[0144] For example, TM thinning (when used with IBC) can be performed at integer pixel locations. In IBC-TM AMVP mode, TM thinning can be performed, for example, with integer or 4-pixel precision based on the AMVR value. Thinning can be performed within the IBC reference area.

[0145] The interaction between the IBC mode and inter-frame coding tools can be as follows, such as Paired Merge Candidate, HMVP, Combined Intra / Inter-Frame Prediction Mode (CIIP), Merge Mode with Motion Vector Difference (MMVD), Local Illumination Compensation (LIC), and Geometric Partition Mode (GPM).

[0146] IBC can be used with pairwise merge candidates and HMVP. Pairwise IBC merge candidates can be generated by averaging two IBC merge candidates. For HMVP, IBC motion can be inserted into a history buffer for (e.g., future) reference.

[0147] The following inter-frame tools can be avoided when using (e.g., not used): IBC: Affine Motion. IBC can be used in combination with CIIP, MMVD, LIC, and GPM. When using DUAL_TREE partitions, IBC may not be allowed for chroma-coded blocks.

[0148] The current frame may be included in the reference frame list 0 for IBC prediction without being considered one of the reference frames. The derivation of motion vectors for the IBC mode can exclude adjacent blocks in the inter-frame mode (e.g., all adjacent blocks) and vice versa. The following IBC design example can be applied.

[0149] IBC can share a process with MV merging, which includes pairwise merging candidates and HMVP, and may disallow temporal motion vector prediction (TMVP) and zero vector, as TMVP and zero vector may be invalid for IBC mode.

[0150] Separate HMVP buffers (e.g., 5 candidates each) can be used for MV and IBC. Block vector constraints (e.g., in the form of bitstream compliance constraints) can be implemented. Devices (e.g., encoders) can ensure that there are no invalid vectors in the bitstream, and / or avoid using (e.g., not use) merging if a merge candidate is invalid (e.g., out of range or 0). Bitstream compliance constraints can be expressed based on virtual buffers (e.g., as described herein). For deblocking, IBC can be disposed of as inter-frame mode. If the current block is encoded using IBC prediction mode, AMVR may not use quarter pixels or half pixels. AMVR can be signaled to indicate whether MV is inter-frame pixels or 4 integer pixels. The number of IBC merge candidates can be signaled separately from the number of regular merge candidates, sub-block merge candidates, and geometric merge candidates in the slice header.

[0151] It can execute and / or provide an IBC merge mode with block vector difference (IBC-MBVD).

[0152] The merging with motion vector difference (MMVD) that can be used in inter-frame prediction blocks may include one or more of the following features and / or details.

[0153] Affine MMVD and GPM-MMVD can be used as extensions of the regular MMVD mode. The MMVD mode can be extended to the IBC merge mode.

[0154] In IBC-MBVD, the motion vector difference distance set can be {1 pixel, 2 pixels, 4 pixels, 8 pixels, 12 pixels, 16 pixels, 24 pixels, 32 pixels, 40 pixels, 48 ​​pixels, 56 pixels, 64 pixels, 72 pixels, 80 pixels, 88 pixels, 96 pixels, 104 pixels, 112 pixels, 120 pixels, 128 pixels}. The BVD direction can include a horizontal (e.g., two horizontal) direction and a vertical (e.g., two vertical) direction.

[0155] A base candidate can be selected from multiple candidates (e.g., the top five candidates) in the reordered IBC merge list. For example, the possible (e.g., all possible) MBVD refinement positions (20×4) of each base candidate can be reordered based on the SAD cost between the template (e.g., the row above and column to the left of the current block) and / or a reference to each of its refinement positions. For example, for MBVD index encoding, multiple refinement positions with the lowest template SAD cost (e.g., the top 8 refinement positions) can be kept as available positions. The MBVD index can be binarized by (e.g., Rice code with parameter equal to 1).

[0156] IBC can be used in conjunction with Local Illumination Compensation (LIC). IBC and LIC can be used together. IBC can be used with an inter-frame prediction enhancement tool called LIC. LIC is an inter-frame prediction technique used to model the local illumination variation between the current block and its predicted block as a function of the local illumination variation between the current block template and the reference block template. The parameters of this function can be represented by scaling α and offset β. For example... Linear equations are used to compensate for lighting variations, where p[x] can be a reference sample pointed to by the MV at position x on the reference frame. For example, when surround motion compensation is enabled, surround offset can be considered to limit the MV. For example, parameters α and β can be used without signaling overhead because α and β can be derived based on the current block template and the reference block template. The LIC flag can be signaled for AMVP mode to indicate the use of LIC.

[0157] Local illumination compensation can be used for unidirectional prediction of inter-frame CUs. Intra-frame neighbor samples can be used for LIC parameter derivation. LIC can be disabled for blocks with fewer than a threshold number of luminance samples (e.g., 32 luminance samples). For example, LIC parameter derivation can be performed based on template block samples corresponding to the current CU (e.g., rather than partial template block samples corresponding to the top-left first cell (e.g., 16×16 cell)) (e.g., for non-sub-blocks and affine modes). For example, samples of a reference block template can be generated by using block MV for MC without rounding to integer pixel precision.

[0158] Intra-Block Copying with Local Illumination Compensation (IBC-LIC) can compensate for local illumination variations within a frame between an IBC-coded CU and its predicted blocks, for example, using (e.g., linear) equations. The parameters of the (e.g., linear) equations can be derived, for example, in the same or similar manner as for LIC used for inter-frame prediction (e.g., except that block vectors in the IBC-LIC can be used to generate a reference template). IBC-LIC can be applied to IBC AMVP mode and / or IBC merging mode. IBC-LIC indications (e.g., for IBC AMVP mode) can be signaled to indicate the use of IBC-LIC. IBC-LIC indications (e.g., for IBC merging mode) can be inferred from merging candidates.

[0159] For example, additional modes (e.g., three extra modes) can be added to IBC-LIC to further improve encoding performance. These additional IBC-LIC modes (e.g., the first two modes) can be associated with template shape selection. IBC-LIC can be enabled (e.g., allowed) to derive individual model parameters using only top, only left, or L-shaped templates. Additional IBC-LIC modes (e.g., a third mode) can extend MMLM to IBC-LIC, for example, allowing IBC-LIC to have multiple (e.g., two) linear models in a single CU. And in this mode, the template can be L-shaped.

[0160] It can remove the large block size constraints of IBC-LIC.

[0161] Table 1 shows exemplary IBC-LIC signaling, such as IBC AMVP mode.

[0162] Table 1: Exemplary IBC-LIC signaling for IBC AMVP mode

[0163] It can execute, use, and / or provide IBC-LIC model merging modes.

[0164] The IBC-LIC model merging mode can use (e.g., inherit) IBC-LIC model parameters, for example, from previous coding blocks. For example, the LIC model for the IBC-LIC model merging mode can be obtained based on one or more of the following.

[0165] A candidate model list can be constructed. This list can include model parameters from spatially adjacent and non-adjacent blocks, historical candidates, and the default model. The candidate list can be up to twelve sizes.

[0166] Similar to the CCMerge method, LIC models can be collected from previously encoded IBC-LIC models, merging adjacent and non-adjacent locations. A historical IBC-LIC model table (e.g., of size six) can be maintained (e.g., similar to the HMVP table). LIC models from spatially adjacent blocks and the historical IBC-LIC model table can be added to the IBC-LIC model merging candidate list. For example, if the list is not full, default and scaled models can be added. For example, pruning operations can be applied to avoid redundant models.

[0167] The offset of each model candidate can be calculated (e.g.).

[0168] For example, the offset can be calculated from a template (e.g., for a certain inherited IBC-LIC model parameter set (α and β)) according to Equations 1 and 2:

[0169] Rec and Ref can contain pixels from the templates of the current block and the reference block, and N can be the total number of pixels in the template region. β can be modified to β = β + offset.

[0170] An IBC-LIC model can be selected, for example, from a candidate list. The index associated with the IBC-LIC model can be signaled in the video data (e.g., a bitstream).

[0171] A signaling indication (e.g., a flag) can be used to indicate whether the IBC-LIC model merge mode is applied. If this flag is true, then a signaling indication index can be used to indicate which candidate model can be used by the current block. The IBC-LIC mode can be signaled (e.g., explicitly signaled) in both IBC-AMVP and IBC merge modes. In IBC-AMVP mode, the signaling indication (e.g., a flag) can be used if (e.g., when) the IBC-LIC flag is true. In IBC merge mode, the flag can be signaled (e.g., signaled only) if (e.g., when) the current block is not encoded as IBC-CIIP, IBC-GPM, TM merge, or skip mode. Regular inheritance of the IBC-LIC flag can be disabled (e.g., it can be omitted), and the current block can be treated as regular IBC-LIC by other blocks, for example, if the IBC-LIC model merge flag is true.

[0172] The IBC-LIC model merging mode can be modified.

[0173] For example, the β parameter of the LIC model can be inherited and can be avoided from being modified (e.g., it can be left unmodified).

[0174] In the IBC merge mode, during the reordering of the merge list, the selected LIC model can be applied to the reference template. The IBC merge list can then be further ordered by inherited IBC LIC indications (e.g., flags). For example, if (e.g., when) there are adjacent blocks encoded as IBC-LIC or the proposed mode, the proposed mode can be used (e.g., only).

[0175] In the IBC-AMVP mode, the selected LIC model can be applied to the reference template during BVD prediction.

[0176] This article provides the features associated with SGPM.

[0177] SGPM can be considered an intra-frame mode (e.g., an inter-frame coding tool similar to Geometric Partition Mode (GPM), where two prediction parts are generated from the intra-frame prediction process).

[0178] In SGPM, a candidate list can be constructed. The candidate list can have one or more entries. Each entry (e.g., one) can include (e.g., one) partitioning and one or more (e.g., two) intra-frame prediction modes (e.g., such as...). Figure 5 (As shown). 26 partitioned modes and 3 intra-frame prediction modes can be used to form a combination. The length of the candidate list can be (for example, set to equal) 16. The selected candidate index can be signaled.

[0179] Templates can be used (e.g., ... Figure 6 The list is reordered using the template (as shown). The sum of the absolute differences (SAD) between the template's predictions and reconstructions can be used to sort the list. The template size can be (e.g., fixed at) 1.

[0180] For each partition mode, an intra-prediction mode (IPM) list can be derived (e.g., for each partition). The size of the IPM list can be (e.g., predefined) 3. Available IPM candidates include: one or more (e.g., two) derived modes from a fusion of template-based intra-mode derivation (TIMD) with horizontal and vertical orientations; parallel angle modes relative to the SGPM block boundaries (e.g., Figure 7 Parallel mode as shown; a derivation mode (e.g., one) from the decoder-side intra-mode derivation (DIMD); one or more (e.g., five) candidates derived from adjacent blocks; and / or planar mode.

[0181] For adjacency pattern derivation, one or more (e.g., five, or at most five) locations can be used for available adjacent blocks. These locations may be limited by the angle of the SGPM block boundary.

[0182] This article provides the features associated with SGPM using IntraTMP and IBC.

[0183] Block vector-based predictions can be obtained from the intra-TMP and / or IBC modes of neighboring blocks in the SGPM candidate list. For example, the SGPM mode can test the block vectors of neighboring blocks in the IPM list (e.g., in addition to regular intra-prediction modes).

[0184] For example, a block vector of merge candidates (e.g., all available merge candidates) can be obtained. (e.g., based on SAD template cost) An optimal block vector (e.g., the best two block vectors, if available) can be selected. The optimal block vector (e.g., the best two block vectors, if available) can be constructed from a test-side SGPM candidate list.

[0185] Figure 8 An exemplary SGPM design is shown. Intra-predictors (e.g., two intra-predictors) can be generalized (e.g., from regular to regular or based on block vector (BV)). For example, an SGPM candidate (e.g., an additional SGPM candidate) can be (e.g., one) a partitioned pattern and a combination of one or more (e.g., two) regular intra-predictors or BV-based intra-predictors.

[0186] The identifiers intra_pred_0 and intra_pred_1 can correspond to regular prediction or BV-based prediction, respectively. These identifiers can be obtained from adjacent IntraTMP and / or IBC information. For example, one or more (e.g., one or two) available BV-based intra predictors from IntraTMP and / or IBC can be added to the SGPM intra candidate list (e.g., after three regular intra predictors from the IPM list).

[0187] As the number of SGPM intra-candidate entries used for partitioning (e.g., for each partition) increases, the corresponding combination and template analysis can be increased (e.g., more) on both the encoder and decoder sides. The maximum number of additional BV-based predictors can be set to 2. The IBC mode can correspond to the first entry in the merge list. For example, if two BV-based intra-predictors from IntraTMP and / or IBC are available, the number of possible SGPM candidates increases from 156 combinations (e.g., 26 × 3 × 2 = 156 combinations) to 520 combinations (e.g., 26 × 5 × 4 = 520 combinations), as shown in Table 2.

[0188] Table 2 - SGPM combinations of partitioned mode and intra-prediction mode (IPM) candidates and additional BV-based candidates (e.g., in this example, two additional BV-based candidates are shown in gray).

[0189]

[0190] SGPM candidate list generation can be performed. For example, for each list entry in the above combination table (e.g., each list entry), the SAD can be measured between the reconstructed signal and the predicted signal in the template. Optimal candidates (e.g., the top 16 candidates) can be retained. A signal can be used to notify the corresponding SGPM index to indicate the selected combination in the SGPM candidate list.

[0191] For example, the compression efficiency of the IBC model merging mode can be improved by enhancing it. The IBC-LIC model merging mode can use index signaling to indicate the model parameters selected from the model candidate list. For example, the signaling cost of the model index can be reduced by enhancing the reordering step performed on the IBC-LIC model merging mode.

[0192] The IBC-LIC model merging pattern may include creating a list of model candidates consisting of model parameters (e.g., from spatially adjacent and non-adjacent blocks, historical candidates, and the default pattern). For example, this list of model candidates can be reordered for each merge candidate using the template cost between the reference template and the current block template. The size of the IBC-LIC model candidates can be 12, and the size of the merge candidate list can be 28. In the example, the reordering step could involve 12 × 28 template cost calculations, which could increase encoder complexity.

[0193] For example, the reordering of the candidate list of model parameters can be improved, thereby reducing the computational burden on the encoder. The IBC-LIC model indexing signaling process can also be improved.

[0194] In the example, the LIC model candidates (e.g., only LIC model candidates) can be reordered, for example, after the merge candidates (e.g., the best merge candidate) are determined.

[0195] In the example, the model candidate list can be split into a primary list and an auxiliary list. The primary candidate list can be reordered, for example, based on template cost (e.g., only the primary candidate list). For example, the model candidate list can be split in half. For example, the model candidates can be split such that the primary list includes spatially adjacent candidates (e.g., only spatially adjacent candidates). For example, if the primary list is empty (e.g., no spatial candidates), reordering of the primary list can be avoided (e.g., instead, the auxiliary list can be reordered). For example, both the primary and auxiliary lists can be reordered independently. Indicators (e.g., flags) can indicate (e.g., encoded by context) the selection of the primary or auxiliary list. Indexes can be included (e.g., encoded) in the indication to indicate the selected IBC-LIC model.

[0196] In the example, model candidates can be selected based on partitioning patterns in SGPM, for instance.

[0197] In the example, model candidates can be determined, for instance, based on implicit inference.

[0198] In the example, the fusion of two or more IBC-LIC model candidates can be performed.

[0199] The model merging patterns can be reordered to (e.g., determine or select) merge candidates (e.g., only the best merge candidates).

[0200] The IBC-LIC model merging mode can include one or more of the following: constructing a LIC model candidate list using spatial candidates, non-adjacent spatial candidates, historical candidates, and default candidates; reordering the LIC model candidate list (e.g., size 12) and the IBC merging candidate list (e.g., size 28); selecting a model from the candidate list and signaling its index in the video data (e.g., bitstream); and so on. A signaling mode indication (e.g., a flag) can be used to indicate whether the IBC-LIC model merging mode is used. This method can be computed using 12×28 template construction and cost derivation.

[0201] To reduce the overall computational burden, one or more of the following can be performed for the IBC-LIC model merging mode.

[0202] A candidate list for the LIC model can be constructed (e.g., determined) using spatial candidates, non-adjacent spatial candidates, historical candidates, default candidates, etc. A merged candidate list for IBC can be constructed based on template cost. The merged candidate list for IBC can be reordered, for example, using template cost. IBC candidates (e.g., the best IBC candidate) can be selected from the merged candidate list (e.g., the reordered merged candidate list).

[0203] The list of LIC model candidates can be reordered, for example, based on the selected merge candidates. A model can be selected (e.g., the best model). The index of the selected model can be indicated (e.g., signaled) in the video data (e.g., in the bitstream).

[0204] The described IBC-LIC process can be computed in a total of 28 + 12 times.

[0205] The model candidate list can be split into a primary list and an auxiliary list. For example, template cost can be used to reorder model candidates in the primary list (e.g., only the primary list) to reduce the number of reordering operations. Reordering model candidates in the primary list (e.g., only the primary list) can reduce the number of computations performed to calculate the SAD cost. The index of the model (e.g., the best model) can be indicated (e.g., signaled) in the video data (e.g., in the bitstream).

[0206] The rules for splitting candidates into two lists can be determined based on their spatial distance from the current block. For example, the primary list may include (e.g., only) spatially adjacent model candidates. The secondary list may include the remaining candidates. In the example, historical candidates and the default model candidate may be included in the secondary list. In the example, the primary list may include the first number of candidates in the candidate list (e.g., the first 6 candidates). The remaining candidates may be included as a part of the secondary list.

[0207] IBC-LIC merge mode can interact with other encoding modes and / or tools (such as SGPM).

[0208] The SGPM candidate list can include IBC block vectors from neighboring blocks. For example, if (e.g., when) the IBC-LIC model is used in combination with SGPM, LIC can be applied to reference samples before calculating predictions for the current partition of a block. IBC-LIC models can be used to merge modes. Spatial candidates adjacent to partitions predicted using block vectors from IBC can be included in the main list, for example, to reduce the number of candidates to be reordered. For example, candidates going to (e.g., being partitioned into) the main list can depend on the partitioning mode of SGPM. Figure 8 The example block is shown in SGPM encoding, where the right partition is IBC encoded (e.g., inter_pred_0). For this partition, the IBC-LIC model candidate list construction can be modified so that spatial candidates B2, A0, and A1 are not part of the main list, for example, because these candidates are on the other side of the partition line.

[0209] For example, model parameters can be implicitly derived from a list of model candidates.

[0210] The first candidate in the reordered LIC model candidate list can be selected as, for example, the model for IBC-LIC (e.g., the best model), without signaling the model (e.g., the best model candidate).

[0211] In the example, implicit deductions can be applied (e.g., applied only) to the IBC merge pattern. An index indicating the optimal pattern can be provided for the IBC-AMVP pattern (e.g., still encoded).

[0212] In the example, the IBC-AMVP pattern can apply features associated with ranking model merging patterns (e.g., as described in this paper). The IBC merging pattern can use implicitly derived features as described in this paper.

[0213] In the example, this implicit derivation can be applied adaptively based on the block size. For instance, the implicit method can be applied to blocks with fewer than a certain number of samples, such as blocks with fewer than 128 samples.

[0214] Multiple (e.g., two or more) IBC-LIC models can be fused.

[0215] A single model (e.g., the best model) can be selected from the candidate list (e.g., as described herein). This model can be explicitly signaled or inherited. Multiple (e.g., two) IBC-LIC models can be selected (e.g., the two best IBC-LIC models in terms of SAD cost). For example, to compute a prediction, the IBC-LIC models (e.g., each of the selected IBC-LIC models) can be applied to a reference sample. The determined predictions generated using each IBC-LIC model (e.g., the two predictions) can be blended to obtain a final prediction. For each prediction, the blending weight can be set to 0.5. In the example, the blending weight can be computed based on the proximity of the selected best IBC-LIC candidate. IBC-LIC candidates from blocks closer to the current block can be associated with higher weights (e.g., given). In the example, the blending weight can be computed, for example, based on the template cost of the selected best IBC-LIC candidate.

[0216] Although the features and elements have been described above in specific combinations, those skilled in the art will understand that each feature or element can be used alone or in any combination with other features and elements. Furthermore, the methods described herein can be implemented in a computer program, software, or firmware incorporated into a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROMs and digital multifunction discs (DVDs). The processor associated with the software can be used to implement a radio frequency transceiver used in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. A video encoding method, comprising: Determine the candidate list of Local Illumination Compensation (LIC) models; Determine the candidate list for intra-block copy (IBC); IBC merge candidates are selected from the IBC candidate list, wherein the selected IBC merge candidates are associated with the optimal template cost; The LIC model candidate list is reordered based on the selected IBC merging candidates; as well as The LIC model is selected based on the reordered list of LIC model candidates.

2. The video coding method of claim 1, wherein the IBC merge candidate list is sorted based on the corresponding template cost associated with each of the plurality of IBC merge candidates from the IBC merge candidate list.

3. The video encoding method according to claim 1 or 2, wherein the method according to claim 1 or 2 further comprises: The video data includes an indication of whether to use the IBC-LIC model merging mode, the selected IBC merging candidate, and one or more of the LIC model indexes associated with the selected IBC merging candidate.

4. The video coding method according to any one of claims 1 to 3, further comprising: Determine the corresponding spatial distance for each LIC model candidate from the plurality of LIC model candidates in the LIC model candidate list; Based on the corresponding spatial distance of each determined LIC model candidate, a primary LIC model candidate list is determined, wherein the primary LIC model candidate list is associated with spatially adjacent model candidates; as well as Based on the corresponding spatial distance of each LIC model candidate list, a secondary LIC model candidate list is determined, wherein reordering the LIC model candidate list based on the selected IBC merge candidate includes: reordering the LIC model candidates associated with the primary LIC model candidate list.

5. The video coding method according to any one of claims 1 to 4, further comprising: The main LIC model candidate list is determined based on the sorting of the LIC model candidate list, wherein the main LIC model candidate list includes a first number of LIC model candidates from the LIC model candidate list; as well as A secondary LIC model candidate list is determined based on the sorting of the LIC model candidate list, wherein the secondary LIC model candidate list includes a second number of LIC model candidates from the LIC model candidate list that are ranked after the first number of LIC model candidates according to the sorting of the LIC model candidate list, and wherein reordering the LIC model candidate list based on the selected IBC merge candidate includes reordering the LIC model candidates associated with the primary LIC model candidate list.

6. The video coding method according to any one of claims 1 to 5, further comprising: Determine the combination of the Spatial Geometric Partitioning Model (SGPM) and the IBC-LIC model merging model; Determine the SGPM partitioning mode to be used in combination with the IBC-LIC model merging mode; as well as A primary LIC model candidate list is determined at least based on the determined SGPM partitioning pattern, wherein reordering the LIC model candidate list based on the selected IBC merge candidate includes: reordering the LIC model candidates associated with the primary LIC model candidate list.

7. The video coding method according to any one of claims 1 to 6, wherein the selected LIC model is the first LIC model candidate in the reordered LIC model candidate list.

8. The video coding method according to any one of claims 1 to 7, wherein the selected LIC model is a first LIC model, and wherein the method according to any one of claims 1 to 7 further comprises: The second LIC model is selected based on the reordered LIC model candidate list; A first prediction is determined by applying the first LIC model to the reference samples; A second prediction is determined based on applying the second LIC model to the reference sample; as well as The final prediction is determined by combining the first and second predictions.

9. A video decoding method, comprising: Obtain an indication of whether to use the Intra-Block Copy (IBC) Local Illumination Compensation (LIC) model merging mode; Based on the obtained instructions, the IBC-LIC model merging pattern was determined; Obtain the LIC model index associated with the IBC merge candidate, wherein the IBC merge candidate is associated with the optimal template cost, and wherein the LIC model index is associated with a list of LIC model candidates sorted based on the IBC merge candidate; as well as Select the LIC model based on the LIC model index.

10. The video decoding method of claim 9, wherein the IBC merge candidates are obtained from an IBC merge candidate list, the IBC merge candidate list being sorted based on the corresponding template cost associated with each of the plurality of IBC merge candidates from the IBC merge candidate list.

11. The video decoding method of claim 9 or 10, wherein the indication further indicates at least one of the IBC merging candidate and the LIC model index associated with the IBC merging candidate.

12. The video decoding method according to any one of claims 9 to 11, wherein the indication further indicates whether the Spatial Geometric Partitioning (SPGM) mode is used in combination with the IBC-LIC model merging mode, and wherein the method further comprises: Based on the indication, an SGPM partitioning model is determined to be used in combination with the IBC-LIC model merging pattern, wherein the candidate list of LIC models is further sorted based on the determined SGPM partitioning pattern.

13. The video decoding method according to any one of claims 9 to 12, wherein the selected LIC model is the first LIC model candidate in the LIC model candidate list.

14. The video decoding method according to any one of claims 9 to 13, wherein the selected LIC model is a first LIC model, and wherein the method further comprises: Select a second LIC model based on the LIC model candidate list; A first prediction is determined by applying the first LIC model to the reference samples; A second prediction is determined based on applying the second LIC model to the reference sample; as well as The final prediction is determined by combining the first and second predictions.

15. A video encoding device, comprising: The processor is configured as follows: Determine the candidate list of Local Illumination Compensation (LIC) models; Determine the candidate list for intra-block copy (IBC); IBC merge candidates are selected from the IBC candidate list, wherein the selected IBC merge candidates are associated with the optimal template cost; The LIC model candidate list is reordered based on the selected IBC merging candidates; as well as The LIC model is selected based on the reordered list of LIC model candidates.

16. The video encoding apparatus of claim 15, wherein the IBC merge candidate list is sorted based on a corresponding template cost associated with each of the plurality of IBC merge candidates from the IBC merge candidate list.

17. The video encoding apparatus of claim 15 or 16, wherein the processor of claim 15 or 16 is further configured to: The video data includes an indication of whether to use the IBC-LIC model merging mode, the selected IBC merging candidate, and one or more of the LIC model indexes associated with the selected IBC merging candidate.

18. The video encoding apparatus according to any one of claims 15 to 17, wherein the processor according to any one of claims 15 to 17 is further configured to: Determine the corresponding spatial distance for each LIC model candidate from the plurality of LIC model candidates in the LIC model candidate list; Based on the corresponding spatial distance of each determined LIC model candidate, a primary LIC model candidate list is determined, wherein the primary LIC model candidate list is associated with spatially adjacent model candidates; and Based on the corresponding spatial distance of each LIC model candidate list, an auxiliary LIC model candidate list is determined, wherein the reordering of the LIC model candidate list based on the selected IBC merging candidate includes: The LIC model candidates associated with the main LIC model candidate list are reordered.

19. The video encoding apparatus according to any one of claims 15 to 18, wherein the processor according to any one of claims 15 to 18 is further configured to: The primary LIC model candidate list is determined based on the sorting of the LIC model candidate list, wherein the primary LIC model candidate list includes a first number of LIC model candidates from the LIC model candidate list; and A supplementary LIC model candidate list is determined based on the sorting of the LIC model candidate list, wherein the supplementary LIC model candidate list includes a second number of LIC model candidates from the LIC model candidate list that are ranked after the first number of LIC model candidates according to the sorting of the LIC model candidate list, and wherein reordering the LIC model candidate list based on the selected IBC merging candidate includes: The LIC model candidates associated with the main LIC model candidate list are reordered.

20. The video encoding apparatus according to any one of claims 15 to 19, wherein the processor according to any one of claims 15 to 19 is further configured to: The Spatial Geometric Partitioning Model (SGPM) is determined to be used in combination with the IBC-LIC model merging model; Determine the SGPM partitioning mode to be used in conjunction with the IBC-LIC model merging mode; and Determining a primary LIC model candidate list based at least on the determined SGPM partitioning pattern, wherein reordering the LIC model candidate list based on the selected IBC merging candidate includes: The LIC model candidates associated with the main LIC model candidate list are reordered.

21. The video encoding apparatus according to any one of claims 15 to 20, wherein the selected LIC model is the first LIC model candidate in the reordered list of LIC model candidates.

22. The video encoding apparatus according to any one of claims 15 to 21, wherein the selected LIC model is a first LIC model, and wherein the processor according to any one of claims 15 to 21 is further configured to: The second LIC model is selected based on the reordered LIC model candidate list; A first prediction is determined by applying the first LIC model to the reference samples; A second prediction is determined based on applying the second LIC model to the reference sample; as well as The final prediction is determined by combining the first and second predictions.

23. A video decoding device, comprising: The processor is configured as follows: Obtain an indication of whether to use the Intra-Block Copy (IBC) Local Illumination Compensation (LIC) model merging mode; Based on the obtained instructions, the IBC-LIC model merging pattern was determined; Obtain the LIC model index associated with the IBC merge candidate, wherein the IBC merge candidate is associated with the optimal template cost, and wherein the LIC model index is associated with a list of LIC model candidates sorted based on the IBC merge candidate; as well as Select the LIC model based on the LIC model index.

24. The video decoding apparatus of claim 23, wherein the IBC merge candidates are obtained from an IBC merge candidate list, the IBC merge candidate list being sorted based on the corresponding template cost associated with each of the plurality of IBC merge candidates from the IBC merge candidate list.

25. The video decoding apparatus of claim 23 or 24, wherein the indication further indicates at least one of the IBC merging candidate and the LIC model index associated with the IBC merging candidate.

26. The video decoding apparatus according to any one of claims 23 to 25, wherein the indication further indicates whether the Spatial Geometry Partitioning (SPGM) mode is used in combination with the IBC-LIC model merging mode, and wherein the processor according to any one of claims 23 to 25 is further configured to: Based on the indication, an SGPM partitioning model is determined to be used in combination with the IBC-LIC model merging pattern, wherein the candidate list of LIC models is further sorted based on the determined SGPM partitioning pattern.

27. The video decoding apparatus according to any one of claims 23 to 26, wherein the selected LIC model is the first LIC model candidate in the LIC model candidate list.

28. The video decoding apparatus according to any one of claims 23 to 27, wherein the selected LIC model is a first LIC model, and the processor according to any one of claims 23 to 27 is further configured to: Select a second LIC model based on the LIC model candidate list; A first prediction is determined by applying the first LIC model to the reference samples; A second prediction is determined based on applying the second LIC model to the reference sample; as well as The final prediction is determined by combining the first and second predictions.

Images