Method for encoding and decoding video data, medium and codec

Abstract

Methods, media, and codecs for encoding and decoding video data are disclosed. A method of encoding video data includes coding a video block of the video data using a linear model of a template-based video coding technique, the coding including: determining a first parameter of the linear model using a single lookup table, the single lookup table including a value for determining a least significant bit, wherein the determination of the first parameter of the linear model comprises: determining a minimum value and a maximum value derived from luminance samples adjacent to the video block; determining a minimum value and a maximum value derived from chroma samples adjacent to the video block; and calculating the first parameter of the linear model from a reciprocal of a difference between (i) a minimum and a maximum of chroma samples and (ii) a minimum and a maximum of luma samples, where the reciprocal is derived from the single lookup table, and predicting samples of the video block based on the linear model.

Description

[0001] This application is a divisional application of an invention patent application with an application date of December 20, 2019, application number 201980091368.4, and invention name “Methods, architectures, devices and systems for improved linear model estimation for template-based video decoding”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 784,341, filed December 21, 2018, which is incorporated herein by reference. Technical Field

[0004] The present application relates to video decoding technology. Background Art

[0005] Video coding systems can be used to compress digital video signals, e.g., to reduce the storage and / or transmission bandwidth required for such signals. Video coding systems can include block-based, wavelet-based, and / or object-based systems. These systems can employ various template-based coding techniques (e.g., tools), such as cross-component linear models (CCLMs) and local illumination compensation (LICs), which, as currently constructed, have a number of disadvantages, including cumbersome hardware implementations and computational complexity, both individually and in combination. Summary of the invention

[0006] The present disclosure provides a method for encoding video data by an encoder, the method comprising: decoding a video block of the video data using a linear model of a template-based video decoding technology, the decoding comprising: using a single lookup table to determine a first parameter of the linear model, the single lookup table comprising a value for determining a least significant bit, wherein the determination of the first parameter of the linear model comprises: determining a minimum value and a maximum value derived from luma samples adjacent to the video block; determining a minimum value and a maximum value derived from chroma samples adjacent to the video block; and calculating the first parameter of the linear model based on (i) a minimum value and a maximum value of the chroma samples and (ii) an inverse of a difference between a minimum value and a maximum value of the luma samples, wherein the inverse is derived from the single lookup table, and predicting samples of the video block based on the linear model.

[0007] The present disclosure provides an encoder for encoding video data, comprising: at least one processor; and a memory storing instructions, wherein when the instructions are executed by the at least one processor, the encoder executes the above method.

[0008] The present disclosure provides a method for decoding video data by a decoder, the method comprising: decoding a video block of the video data using a linear model of a template-based video coding technology, the decoding comprising: using a single lookup table to determine a first parameter of the linear model, the single lookup table comprising a value for determining a least significant bit, wherein the determination of the first parameter of the linear model comprises: determining a minimum value and a maximum value derived from luma samples adjacent to the video block; determining a minimum value and a maximum value derived from chroma samples adjacent to the video block; and calculating the first parameter of the linear model based on (i) a minimum value and a maximum value of the chroma samples and (ii) an inverse of a difference between a minimum value and a maximum value of the luma samples, wherein the inverse is derived from the single lookup table, and predicting samples of the video block based on the linear model.

[0009] The present disclosure provides a decoder for decoding video data, comprising: at least one processor; and a memory storing instructions, when the instructions are executed by the at least one processor, the decoder executes the above method.

[0010] The present disclosure provides a non-transitory computer-readable medium having instructions stored thereon, and when the instructions are executed by at least one processor, the at least one processor is caused to perform the above method. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] A more detailed understanding can be obtained from the following detailed description given by way of example in conjunction with the accompanying drawings. The drawings in the specification are examples. Similar to the detailed description, the illustrations in the following drawings should not be considered limiting, and other equivalent examples are possible and possible. In addition, the same reference numerals ("ref.") in the figures represent the same elements, wherein:

[0012] Figure 1A is a system diagram illustrating an example communication system;

[0013] Figure 1B It shows that it can be Figure 1A A system diagram of an example wireless transmit / receive unit (WTRU) for use within a communication system is shown;

[0014] Figure 1C It shows that it can be Figure 1A A system diagram of an example radio access network (RAN) and an example core network (CN) used within the communication system shown;

[0015] Figure 1D It shows that it can be Figure 1A A system diagram of another exemplary RAN and another exemplary CN used within the communication system shown;

[0016] Figure 2 is a block diagram illustrating an exemplary block-based video encoder;

[0017] Figure 3 Examples of quadruple partitioning, horizontal binary partitioning, vertical binary partitioning, horizontal ternary partitioning, and vertical ternary partitioning are shown;

[0018] Figure 4 is a block diagram showing an exemplary video decoder;

[0019] Figure 5 is a block diagram of an exemplary block-based video encoder with generalized bidirectional prediction (GBi) support;

[0020] Figure 6 is a block diagram illustrating an example GBi unit for an encoder;

[0021] Figure 7 is a block diagram showing an exemplary block-based video decoder with GBi support;

[0022] Figure 8 is a block diagram showing an example GBi unit for a decoder;

[0023] Fig. 9 is a block diagram illustrating a set of frames / pictures on which local illumination compensation (LIC) may be performed;

[0024] Fig.10 Example locations of left and upper causal samples for a cross-component linear model (CCLM) are shown;

[0025] Fig.11 An exemplary CCLM straight line estimation model for deriving linear model parameters is shown;

[0026] Fig.12 An example LIC straight line estimation model for deriving linear model parameters is shown;

[0027] Fig.13 is a block diagram showing an example linear model mode (LM_A mode) for any CCLM and LIC straight line estimation model based on the above sample;

[0028] Fig.14 is a block diagram showing an example linear model mode (LM_L mode) for any CCLM and LIC straight line estimation model based on left samples;

[0029] Fig.15 An example multiple model parameter estimation method with two LIC min-max based linear models is shown;

[0030] Fig.16 Example point distribution driven selection of multi-model versus single-model LIC is shown;

[0031] Fig.17 is a flowchart illustrating an example process according to one or more embodiments; and

[0032] Fig.18 is a flow diagram illustrating an example process according to one or more embodiments. DETAILED DESCRIPTION

[0033] In the following detailed description, many specific details are set forth to provide a thorough understanding of the embodiments and / or examples disclosed herein. However, it will be understood that such embodiments and examples can be practiced without some or all of the specific details set forth herein. In other cases, known methods, processes, components and circuits are not described in detail to avoid confusing the following description. In addition, the embodiments and examples not specifically described herein can replace the embodiments and other examples provided, disclosed or otherwise explicitly, implicitly and / or inherently (collectively referred to as "providing") described herein or implemented in combination with them. Although various embodiments in which devices, systems, equipment, etc. and / or any of its elements perform operations, processes, algorithms, functions, etc. and / or any part thereof are described and / or claimed herein, it should be understood that any embodiments described and / or claimed herein assume that any device, system, equipment, etc. and / or any of its elements are configured to perform any operations, processes, algorithms, functions, etc. and / or any part thereof.

[0034] Example Communication System

[0035] The methods, devices, and systems provided herein are well suited for communications involving both wired and wireless networks. Wired networks are well known. Figures 1A-1D An overview of various types of wireless devices and infrastructure is provided in which various elements of the network may utilize, perform, be arranged in accordance with the methods, apparatuses, and systems provided herein, and / or be adapted and / or configured for the methods, apparatuses, and systems provided herein.

[0036] Figure 1A1 is a diagram showing an example communication system 100 in which one or more disclosed embodiments may be implemented. The example communication system 100 is provided for illustrative purposes only and is not intended to limit the disclosed embodiments. The communication system 100 may be a multiple access system that provides content such as voice, data, video, messaging, broadcast, etc. to multiple wireless users. The communication system 100 may enable multiple wireless users to access such content by sharing system resources including wireless bandwidth. For example, the communication system 100 may use one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single carrier FDMA (SC-FDMA), zero tail (ZT) unique word (UW) discrete Fourier transform (DFT) spread OFDM (ZT UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block filtered OFDM, and filter bank multi-carrier (FBMC), etc.

[0037] like Figure 1A As shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, a radio access network (RAN) 104 / 113, a core network (CN) 106 / 115, a public switched telephone network (PSTN) 108, the Internet 110, and other networks 112, although it should be appreciated that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network components. Each WTRU 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. For example, any of the WTRUs 102a, 102b, 102c, 102d may be referred to as a “station” and / or “STA”, which may be configured to transmit and / or receive wireless signals, and may include (or be) a user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a subscription-based unit, a pager, a cellular phone, a personal digital assistant (PDA), a smart phone, a laptop, a netbook, a personal computer, a wireless sensor, a hotspot or Mi-Fi device, an Internet of Things (IoT) device, a watch or other wearable device, a head-mounted display (HMD), a vehicle, a drone, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices working in an industrial and / or automated process chain environment), consumer electronic devices, and devices working on a commercial and / or industrial wireless network, etc. Any of the WTRUs 102a, 102b, 102c, 102d may be interchangeably referred to as a UE.

[0038] The communication system 100 may also include a base station 114a and / or a base station 114b. Each base station 114a and / or base station 114b may be any type of device configured to facilitate access to one or more communication networks (e.g., CN 106 / 115, Internet 110, and / or network 112) by wirelessly interfacing with at least one of the WTRUs 102a, 102b, 102c, 102d. For example, the base stations 114a, 114b may be base transceiver stations (BTS), node Bs (NBs), e-node Bs (eNBs), home node Bs (HNBs), home e-node Bs (HeNBs), g-node Bs (gNBs), NR node Bs (NR NBs), site controllers, access points (APs), wireless routers, and the like. Although each base station 114a, 114b is depicted as a single component, it should be appreciated that the base stations 114a, 114b may include any number of interconnected base stations and / or network components.

[0039] The base station 114a may be part of the RAN 104 / 113, and the RAN may also include other base stations and / or network components (not shown), such as a base station controller (BSC), a radio network controller (RNC), a relay node, and the like. The base station 114a and / or the base station 114b may be configured to transmit and / or receive wireless signals on one or more carrier frequencies called cells (not shown). These frequencies may be in a licensed spectrum, an unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide wireless service coverage for a specific geographic area that is relatively fixed or may change over time. A cell may be further divided into cell sectors. For example, a cell associated with the base station 114a may be divided into three sectors. Thus, in one embodiment, the base station 114a may include three transceivers, i.e., each transceiver corresponds to a sector of the cell. In one embodiment, the base station 114a may use multiple-input multiple-output (MIMO) technology, and may use multiple transceivers for each or any sector of the cell. For example, by using beamforming, signals may be transmitted and / or received in a desired spatial direction.

[0040] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, millimeter wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0041] More specifically, as described above, the communication system 100 may be a multiple access system and may use one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, and SC-FDMA, among others. For example, the base station 114a in the RAN 104 / 113 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may use Wideband CDMA (WCDMA) to establish the air interface 115 / 116 / 117. WCDMA may include communication protocols such as High Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High Speed ​​Downlink Packet Access (HSDPA) and / or High Speed ​​Uplink Packet Access (HSUPA).

[0042] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may utilize a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may utilize Long Term Evolution (LTE) and / or LTE-Advanced (LTE-A) and / or LTE-Advanced Pro (LTE-A Pro) to establish the air interface 116.

[0043] In other embodiments, the base station 114a and the WTRUs 102a, 102b, 102c may implement wireless technologies such as IEEE802.16 (i.e., Worldwide Microwave Access Operation (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data Rates for GSM Evolution (EDGE), and GSM EDGE (GERAN), etc.

[0044] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as NR radio access, which may establish the air interface 116 using new radio (NR).

[0045] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may jointly implement LTE radio access and NR radio access (e.g., using dual connectivity (DC) principles). Thus, the air interface used by the WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations (e.g., eNBs and gNBs).

[0046] In other embodiments, the base station 114a and the WTRUs 102a, 102b, 102c may implement radio technologies such as IEEE 802.11 (i.e., Wireless Fidelity (Wi-Fi)), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data Rates for GSM Evolution (EDGE), and GSM EDGE (GERAN), etc.

[0047] Figure 1A The base station 114b in may be a wireless router, a Home NodeB, a Home eNodeB, or an access point, and may use any appropriate RAT to facilitate wireless connectivity in a local area, such as a business location, a residence, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a road, and the like. In one embodiment, the base station 114b and the WTRUs 102c, 102d may establish a wireless local area network (WLAN) by implementing a radio technology such as IEEE 802.11. In one embodiment, the base station 114b and the WTRUs 102c, 102d may establish a wireless personal area network (WPAN) by implementing a radio technology such as IEEE 802.15. In one embodiment, the base station 114b and the WTRUs 102c, 102d may establish any of a small cell, a pico cell, or a femto cell by using a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.). As Figure 1A As shown, the base station 114b may be directly connected to the Internet 110. Thus, the base station 114b does not need to access the Internet 110 via the CN 106 / 115.

[0048] The RAN 104 / 113 may be in communication with the CN 106 / 115, which may be any type of network configured to provide voice, data, applications and / or Voice over Internet Protocol (VoIP) services to one or more WTRUs 102a, 102b, 102c, 102d. The data may have different quality of service (QoS) requirements, such as different throughput requirements, latency requirements, fault tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. The CN 106 / 115 may provide call control, billing services, mobile location-based services, prepaid calls, Internet connectivity, video distribution, etc., and / or may perform advanced security functions such as user authentication. Although in Figure 1A Although not shown, it will be appreciated that the RAN 104 / 113 and / or the CN 106 / 115 may be in direct or indirect communication with other RANs that employ the same RAT or a different RAT as the RAN 104 / 113. For example, in addition to being connected to the RAN 104 / 113 employing NR radio technology, the CN 106 / 115 may also be in communication with another RAN (not shown) employing any of GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or Wi-Fi radio technologies.

[0049] The CN 106 / 115 may also act as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and / or other networks 112. The PSTN 108 may include a circuit-switched telephone network that provides plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer network devices that use common communication protocols (e.g., the Transmission Control Protocol (TCP), the User Datagram Protocol (UDP), and the Internet Protocol (IP) in the TCP / IP Internet Protocol Suite). The network 112 may include a wired or wireless communication network owned and / or operated by other service providers. For example, the network 112 may include another CN connected to one or more RANs, wherein the one or more RANs may use the same RAT or a different RAT as the RAN 104 / 114.

[0050] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communication system 100 may include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks over different wireless links). Figure 1A The illustrated WTRU 102c may be configured to communicate with the base station 114a, which may employ a cellular-based radio technology, and with the base station 114b, which may employ an IEEE 802 radio technology.

[0051] Figure 1B 1 is a system diagram of an example WTRU 102. The example WTRU 102 is provided for illustration purposes only and is not intended to limit the disclosed embodiments. Figure 1B As shown, the WTRU 102 may include a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a numeric keypad 126, a display / touchpad 128, a non-removable memory 130, a removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and other peripherals 138. It should be appreciated that the WTRU 102 may include any sub-combination of the foregoing components while remaining consistent with an embodiment.

[0052] The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, and the like. The processor 118 may perform signal decoding, data processing, power control, input / output processing, and / or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit / receive element 122. Although Figure 1B The processor 118 and the transceiver 120 are depicted as separate components, however it should be appreciated that the processor 118 and the transceiver 120 may also be integrated into one electronic component or chip.

[0053] The transmit / receive component 122 may be configured to transmit or receive signals to or from a base station (e.g., base station 114a) via the air interface 116. For example, in one embodiment, the transmit / receive component 122 may be an antenna configured to transmit and / or receive RF signals. As an example, in one embodiment, the transmit / receive component 122 may be an emitter / detector configured to transmit and receive IR, UV, or visible light signals. In yet another embodiment, the transmit / receive component 122 may be configured to transmit and / or receive RF and light signals. It should be appreciated that the transmit / receive component 122 may be configured to transmit and / or receive any combination of wireless signals.

[0054] In addition, although Figure 1B102 as a single component, but the WTRU 102 may include any number of transmit / receive components 122. For example, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive components 122 (e.g., multiple antennas) for transmitting and receiving radio signals over the air interface 116.

[0055] The transceiver 120 may be configured to modulate signals to be transmitted by the transmit / receive component 122 and to demodulate signals received by the transmit / receive component 122. As described above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers that allow the WTRU 102 to communicate via multiple RATs (e.g., NR and IEEE 802.11).

[0056] The processor 118 of the WTRU 102 may be coupled to a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light emitting diode (OLED) display unit), and may receive user input data from these components. The processor 118 may also output user data to the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128. In addition, the processor 118 may access information from and store information in any suitable memory such as a non-removable memory 130 and / or a removable memory 132. The non-removable memory 130 may include a random access memory (RAM), a read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from, and store data in, memory that is not physically located on the WTRU 102, such as, for example, on a server or a home computer (not shown).

[0057] The processor 118 may receive power from the power source 134, and may be configured to distribute and / or control power for use by other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (Ni-Cd), nickel-zinc (Ni-Zn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, and the like.

[0058] The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) related to the current location of the WTRU 102. In addition to or in lieu of the information from the GPS chipset 136, the WTRU 102 may receive location information from a base station (e.g., base stations 114a, 114b) via the air interface 116 and / or determine its location based on the timing of signals received from two or more nearby base stations. It should be appreciated that the WTRU 102 may acquire location information via any suitable positioning method while remaining consistent with an embodiment.

[0059] The processor 118 may also be coupled to other peripheral devices 138, wherein the peripheral devices may include one or more software and / or hardware modules / units that provide additional features, functionality, and / or wired or wireless connectivity. For example, the peripheral devices 138 may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (e.g., for photos or videos), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, Modules, frequency modulation (FM) radio units, digital music players, media players, video game console modules, Internet browsers, virtual reality and / or augmented reality (VR / AR) devices, and activity trackers, etc. The peripherals 138 may include one or more sensors, which may be one or more of the following: gyroscopes, accelerometers, Hall effect sensors, magnetometers, orientation sensors, proximity sensors, temperature sensors, time sensors, geolocation sensors, altimeters, light sensors, touch sensors, magnetometers, barometers, gesture sensors, biometric sensors, and / or humidity sensors.

[0060] The WTRU 102 may include a full-duplex radio in which reception or transmission of some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous for the radio. A full-duplex radio may include an interference management unit that reduces and / or substantially eliminates self-interference by means of hardware (e.g., chokes) or by signal processing by a processor (e.g., a separate processor (not shown) or by the processor 118). In one embodiment, the WTRU 102 may include a half-duplex radio that transmits or receives some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) or downlink (e.g., for reception)).

[0061] Figure 1C1 is a system diagram of the RAN 104 and the CN 106 according to another embodiment. As described above, the RAN 104 may communicate with the WTRUs 102a, 102b, and 102c using an E-UTRA radio technology over the air interface 116. The RAN 104 may also be in communication with the CN 106.

[0062] The RAN 104 may include eNode-Bs 160a, 160b, 160c, however it will be appreciated that the RAN 104 may include any number of eNode-Bs while remaining consistent with an embodiment. Each of the eNode-Bs 160a, 160b, 160c may include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the eNode-Bs 160a, 160b, 160c may implement MIMO technology. Thus, for example, the eNode-B 140a may use multiple antennas to transmit wireless signals to, and receive wireless signals from, the WTRU 102a.

[0063] Each eNodeB 160a, 160b, and 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, user scheduling in uplink (UL) and / or downlink (DL), etc. Figure 1C As shown, the eNode-Bs 160a, 160b, 160c may communicate with each other via an X2 interface.

[0064] Figure 1C The illustrated core network 106 may include a mobility management gateway (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway 166. Although each of the foregoing components is described as being part of the CN 106, it should be appreciated that any of these components may be owned and / or operated by an entity other than the CN operator.

[0065] The MME 162 may be connected to each of the eNode-Bs 160a, 160b, 160c in the RAN 104 via an S1 interface and may act as a control node. For example, the MME 142 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, performing bearer activation / deactivation processing, and selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, 102c. The MME 162 may also provide a control plane function for switching between the RAN 104 and other RANs (not shown) that use other radio technologies, such as GSM or WCDMA.

[0066] The SGW 164 may be connected to each of the eNode-Bs 160a, 160b, 160c in the RAN 104 via an S1 interface. The SGW 164 may generally route and forward user data packets to / from the WTRUs 102a, 102b, 102c. The SGW 164 may also perform other functions, such as anchoring the user plane during inter-eNode-B handovers, triggering paging processing when DL data is available for the WTRUs 102a, 102b, 102c, and managing and storing the contexts of the WTRUs 102a, 102b, 102c, and the like.

[0067] The SGW 164 may also be connected to a PDN Gateway 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0068] The CN 106 may facilitate communications with other networks. For example, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the CN 106 may include or communicate with an IP gateway, such as an IP Multimedia Subsystem (IMS) server, which may serve as an interface between the CN 106 and the PSTN 108. In addition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks that are owned and / or operated by other service providers.

[0069] Although in Figures 1A-1D While the WTRU is described as a wireless terminal, it should be appreciated that in certain representative embodiments, such a terminal may use a (eg, temporary or permanent) wired communication interface with a communication network.

[0070] In some representative embodiments, other network 112 may be a WLAN.

[0071] A WLAN using an infrastructure basic service set (BSS) mode may have an access point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP may access or be connected to a distributed system (DS) or other types of wired / wireless networks that send traffic into and / or out of the BSS. Traffic originating from outside the BSS and destined for the STA may arrive through the AP and be delivered to the STA. Traffic originating from the STA and destined for a destination outside the BSS may be sent to the AP for delivery to the corresponding destination. Traffic between STAs within the BSS may be sent through the AP, for example, where the source STA may send traffic to the AP and the AP may deliver the traffic to the destination STA. Traffic between STAs within the BSS may be considered and / or referred to as point-to-point traffic. The point-to-point traffic may be sent between the source and destination STAs (e.g., directly therebetween) using a direct link setup (DLS). In certain representative embodiments, the DLS may use 802.11e DLS or 802.11z tunneled DLS (TDLS)). For example, a WLAN using an independent BSS (IBSS) mode may not have an AP, and STAs (eg, all STAs) within or using the IBSS may communicate directly with each other. Here, the IBSS communication mode may sometimes be referred to as an "ad hoc" communication mode.

[0072] When using the 802.11ac infrastructure mode of operation or a similar mode of operation, the AP may transmit a beacon on a fixed channel (e.g., a primary channel). The primary channel may have a fixed width (e.g., a bandwidth of 20 MHz) or a width that is dynamically set by means of signaling. The primary channel may be a working channel of the BSS and may be used by the STA to establish a connection with the AP. In certain representative embodiments, carrier sense multiple access (CSMA / CA) with collision avoidance (e.g., in an 802.11 system) may be implemented. For CSMA / CA, STAs (e.g., each STA) including the AP may sense the primary channel. If a particular STA senses / detects and / or determines that the primary channel is busy, the particular STA may back off. In a specified BSS, one STA (e.g., only one station) transmits at any given time.

[0073] A high throughput (HT) STA may communicate using a 40 MHz wide channel (eg, by combining a 20 MHz wide primary channel with a 20 MHz wide adjacent or non-adjacent channel to form a 40 MHz wide channel).

[0074] Very high throughput (VHT) STA can support channels with widths of 20MHz, 40MHz, 80MHz and / or 160MHz. 40MHz and / or 80MHz channels can be formed by combining continuous 20MHz channels. A 160MHz channel can be formed by combining 8 continuous 20MHz channels or by combining two discontinuous 80MHz channels (this combination can be referred to as an 80+80 configuration). For the 80+80 configuration, after channel coding, the data can be transmitted and passed through a segment parser, which can separate the data into two streams. Inverse fast Fourier transform (IFFT) processing and time domain processing can be performed separately on each stream. The stream can be mapped on two 80MHz channels, and the data can be transmitted by the STA performing the transmission. At the receiver of the STA performing the reception, the above operations for the 80+80 configuration can be reversed, and the combined data can be sent to the medium access control (MAC).

[0075] 802.11af and 802.11ah support sub-1 GHz operating modes. Compared to 802.11n and 802.11ac, the channel operating bandwidth and carrier used in 802.11af and 802.11ah are reduced. 802.11af supports 5MHz, 10MHz and 20MHz bandwidths in the TV White Space (TVWS) spectrum, and 802.11ah supports 1MHz, 2MHz, 4MHz, 8MHz and 16MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11ah can support meter type control / machine type communication (MTC) (e.g., MTC devices in macro coverage areas). MTC can have certain capabilities, such as limited capabilities including support (e.g., only support) certain and / or limited bandwidths. MTC devices can include a battery, and the battery life of the battery is higher than a threshold (e.g., for maintaining a very long battery life).

[0076] For WLAN systems that can support multiple channels and channel bandwidths (e.g., 802.11n, 802.11ac, 802.11af, and 802.11ah), these systems include a channel that can be designated as a primary channel. The bandwidth of the primary channel can be equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel can be set and / or limited by a STA, where the STA is derived from all STAs operating in the BSS that supports the minimum bandwidth operating mode. In the example of 802.11ah, even if the AP and other STAs in the BSS support 2MHz, 4MHz, 8MHz, 16MHz and / or other channel bandwidth operating modes, for STAs (e.g., MTC-type devices) that support (e.g., only support) 1MHz mode, the width of the primary channel can be 1MHz. Carrier sensing and / or network allocation vector (NAV) settings can depend on the state of the primary channel. If the primary channel is busy (eg, because a STA (which only supports 1 MHz operating mode) is transmitting to the AP), then the entire available band may be considered busy even though most of the band remains empty and available for use.

[0077] In the United States, the available frequency band for 802.11ah is 902MHz to 928MHz. In South Korea, the available frequency band is 917.5MHz to 923.5MHz. In Japan, the available frequency band is 916.5MHz to 927.5MHz. Depending on the country code, the total bandwidth available for 802.11ah is 6MHz to 26MHz.

[0078] Figure 1D 1 is a system diagram showing the RAN 113 and the CN 115 according to one embodiment. As described above, the RAN 113 may communicate with the WTRUs 102a, 102b, 102c using NR radio technology over the air interface 116. The RAN 113 may also communicate with the CN 115.

[0079] The RAN 113 may include gNBs 180a, 180b, 180c, though it will be appreciated that the RAN 113 may include any number of gNBs while remaining consistent with the embodiments. Each of the gNBs 180a, 180b, 180c may include one or more transceivers to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the gNBs 180a, 180b, 180c may implement MIMO technology. For example, the gNBs 180a, 180b may use beamforming processing to transmit and / or receive signals to and / or from the gNBs 180a, 180b, 180c. Thus, for example, the gNB 180a may use multiple antennas to transmit wireless signals to and / or receive wireless signals from the WTRU 102a. In one embodiment, the gNBs 180a, 180b, 180c may implement carrier aggregation technology. For example, the gNB 180a may transmit a plurality of component carriers (not shown) to the WTRU 102a. A subset of these component carriers may be on an unlicensed spectrum while the remaining component carriers may be on a licensed spectrum. In one embodiment, the gNBs 180a, 180b, 180c may implement coordinated multi-point (CoMP) technology. For example, the WTRU 102a may receive coordinated transmissions from the gNB 180a and gNB 180b (and / or gNB 180c).

[0080] The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using transmissions associated with the scalable parameter configurations. For example, the OFDM symbol spacing and / or OFDM subcarrier spacing may be different for different transmissions, different cells, and / or different portions of the radio transmission spectrum. The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using subframes or transmission time intervals (TTIs) of different or scalable lengths (e.g., containing different numbers of OFDM symbols and / or lasting different absolute time lengths).

[0081] The gNBs 180a, 180b, 180c may be configured to communicate with the WTRUs 102a, 102b, 102c in a standalone configuration and / or a non-standalone configuration. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c without accessing other RANs (e.g., the eNodeBs 160a, 160b, 160c). In a standalone configuration, the WTRUs 102a, 102b, 102c may use one or more of the gNBs 180a, 180b, 180c as mobility anchors. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using signals in an unlicensed band. In a non-standalone configuration, the WTRU 102a, 102b, 102c may communicate / connect to the gNB 180a, 180b, 180c while communicating / connecting to another RAN (e.g., the eNode-B 160a, 160b, 160c). For example, the WTRU 102a, 102b, 102c may communicate with one or more gNBs 180a, 180b, 180c and one or more eNode-Bs 160a, 160b, 160c in a substantially simultaneous manner by implementing the DC principle. In a non-standalone configuration, the eNode-B 160a, 160b, 160c may act as a mobility anchor for the WTRU 102a, 102b, 102c, and the gNB 180a, 180b, 180c may provide additional coverage and / or throughput to serve the WTRU 102a, 102b, 102c.

[0082] Each gNB 180a, 180b, 180c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, support network slicing, implement dual connectivity, implement interworking between NR and E-UTRA, route user plane data to user plane functions (UPFs) 184a, 184b, and route control plane information to access and mobility management functions (AMFs) 182a, 182b, and the like. Figure 1D As shown, gNBs 180a, 180b, and 180c can communicate with each other through the Xn interface.

[0083] Figure 1DThe CN 115 shown may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one session management function (SMF) 183a, 183b, and may include at least one data network (DN) 185a, 185b. Although each of the aforementioned components is described as part of the CN 115, it should be understood that any of these components may be owned and / or operated by entities other than the CN operator.

[0084] The AMF 182a, 182b may be connected to one or more gNBs 180a, 180b, 180c in the RAN 113 via an N2 interface and may act as a control node. For example, the AMF 182a, 182b may be responsible for authenticating users of the WTRU 102a, 102b, 102c, supporting network slicing (e.g., handling different packet data unit (PDU) sessions with different requirements), selecting a specific SMF 183a, 183b, managing registration areas, terminating NAS signaling, and mobility management, etc. The AMF 182a, 1823b may use network slicing processing to customize the CN support provided to the WTRU 102a, 102b, 102c, for example, based on the type of service used by the WTRU 102a, 102b, 102c. As an example, different network slices may be established for different use cases, such as services relying on ultra-reliable low latency (URLLC) access, services relying on enhanced massive mobile broadband (eMBB) access, and / or services for MTC access, etc. The AMF 162 may provide a control plane function for switching between the RAN 113 and other RANs (not shown) using other radio technologies (e.g., LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies such as Wi-Fi).

[0085] The SMF 183a, 183b may be connected to the AMF 182a, 182b in the CN 115 via the N11 interface. The SMF 183a, 183b may also be connected to the UPF 184a, 184b in the CN 115 via the N4 interface. The SMF 183a, 183b may select and control the UPF 184a, 184b, and may configure traffic routing through the UPF 184a, 184b. The SMF 183a, 183b may perform other functions, such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notifications, etc. The PDU session type may be IP-based, non-IP-based, Ethernet-based, etc.

[0086] The UPF 184a, 184b can be connected to one or more gNBs 180a, 180b, 180c in the RAN 113 via the N3 interface, which can provide the WTRU 102a, 102b, 102c with a packet-switched network (e.g., the Internet 110) connection to facilitate, for example, communication between the WTRU 102a, 102b, 102c and IP-enabled devices. The UPF 184, 184b can perform other functions, such as routing and forwarding packets, implementing user plane policies, supporting multi-host PDU sessions, handling user plane QoS, buffering downlink packets, and providing mobility anchor processing, etc.

[0087] The CN 115 may facilitate communications with other networks. For example, the CN 115 may include or may communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that acts as an interface between the CN 115 and the CN 108. In addition, the CN 115 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, the WTRUs 102a, 102b, 102c may be connected to local data networks (DNs) 185a, 185b via an N3 interface connected to the UPFs 184a, 184b and an N6 interface between the UPFs 184a, 184b and the DNs 185a, 185b and through the UPFs 184a, 184b.

[0088] In view of Figures 1A-1D and about Figures 1A-1D , one or more or all of the functions described herein with reference to one or more of the following may be performed by one or more components / simulation devices (not shown): WTRU 102a-d, base station 114a-b, eNodeB 160a-c, MME 162, SGW 164, PGW 166, gNB 180a-c, AMF 182a-b, UPF 184a-b, SMF 183a-b, DN 185a-b, and / or any other components / devices described herein. These simulation devices may be one or more devices configured to simulate one or more or all of the functions herein. For example, these simulation devices may be used to test other devices and / or simulate network and / or WTRU functions.

[0089] The simulation device can be designed to implement one or more tests about other devices in a laboratory environment and / or an operator network environment. For example, the one or more simulation devices can perform one or more or all functions while being implemented and / or deployed as part of a wired and / or wireless communication network in whole or in part to test other devices inside the communication network. The one or more simulation devices can perform one or more or all functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. The simulation device can be directly coupled to other devices to perform the test, and / or can use over-the-air wireless communication to perform the test.

[0090] One or more simulation devices can perform one or more functions, including all functions, while not being implemented / deployed as part of a wired and / or wireless communication network. For example, the simulation device can be used in a test lab and / or a test scenario of a wired and / or wireless communication network that is not deployed (e.g., tested) to implement tests on one or more components. The one or more simulation devices can be test devices. The simulation device can transmit and / or receive data using direct RF coupling and / or wireless communication with the aid of RF circuits (as an example, the circuits can include one or more antennas).

[0091] introduce

[0092] The video decoding system can be used to compress the digital video signal, which can reduce the storage requirements and / or transmission bandwidth of the video signal. The video decoding system may include a block-based, wavelet-based and / or object-based system. The block-based video decoding system may be based on, use, and comply with one or more standards, such as MPEG-1 / 2 / 4 Part 2, H.264 / MPEG-4 Part 10 AVC, VC-1, High Efficiency Video Coding (HEVC) and / or Versatile Video Coding (VVC), including any reference models associated therewith. The block-based video decoding system may include a block-based hybrid video coding framework.

[0093] Representative architecture / framework

[0094] Figure 2is a block diagram illustrating an exemplary block-based hybrid video decoding framework 200 for an encoder. The encoder and, therefore, the exemplary block-based hybrid video decoding framework 200 may be implemented in a device having a processor communicatively coupled to a memory. The memory may include instructions executable by the processor, including instructions for performing any of the various embodiments disclosed herein (e.g., representative processes). In various embodiments, the device may be or may be configured as and / or have various elements of a wireless transmit and receive unit (WTRU). Example details of the WTRU and its elements are provided herein at Figures 1A-1D and its accompanying disclosures.

[0095] The block-based hybrid video coding framework 200 may include a transform unit 204, a quantization unit 206, an entropy coding unit 208, an inverse quantization unit 210, an inverse transform unit 212, a first adder 216, a second adder 226, a spatial prediction unit 260, a motion / temporal prediction unit 262, a reference picture repository 264, a filter 266, and a mode decision and other encoder control logic unit 280. The block-based hybrid video coding framework 200 (and encoder) may include additional and / or different elements. For the sake of brevity and clarity, these elements are not shown.

[0096] The video signal 202 that undergoes block-based processing of the block-based hybrid video coding framework 200 is performed on a block-by-block basis. Each block that undergoes processing includes NxM pixels; the size of the block ("block size") is defined. Various terms are used in different block-based video coding systems to refer to blocks by their block size. For simplicity of explanation, the term "coding unit" or its abbreviation "CU" is used herein to refer to blocks that undergo processing by the block-based hybrid video coding framework 200 or any other block-based hybrid video coding framework. Each CU can be partitioned into multiple TUs based on the maximum size of the transform kernel. According to the current VVC protocol, a CU may include up to 128×128 pixels. However, it should be recognized that a CU may include more than 128×128 pixels (i.e., N and / or M may be greater than 128), and a CU may be a partition of a larger set / block of pixels. For example, a CU may be a partition of a coding tree unit CTU of a picture (e.g., a slice of a picture). The CTU may be split into multiple CUs based on one or more partitioning mechanisms, for example, to accommodate varying local characteristics. The partitioning mechanism may include any of a quadtree, a binary tree, and a ternary tree structure. Examples of CTU partitioning using quad partitioning, horizontal binary partitioning, vertical binary partitioning, horizontal ternary partitioning, and vertical ternary partitioning are described in Figure 3, respectively as (a), (b), (c), (d) and (e). In one embodiment, a multi-type tree structure may be adopted, in which one type of partition is used to split the CTU, and the same or different types of partitions are used for each partition performed to further partition the partitions one or more times. As an example, a quadtree structure may be used to partition the CTU to form quadtree leaf nodes, and each quadtree leaf node may be further partitioned using any of a binary tree structure and a ternary tree structure.

[0097] See again Figure 2 , a CU may undergo any of spatial prediction and temporal prediction using spatial prediction unit 260 and motion / temporal prediction unit 262, respectively. Spatial prediction (e.g., intra prediction) may use pixels from samples (e.g., reference samples) of coded neighboring blocks in the same video picture / slice to predict the current video block. The spatial prediction may reduce spatial redundancy inherent in, for example, a video signal.

[0098] The temporal prediction (e.g., inter-frame prediction and / or motion compensated prediction) may use reconstructed pixels from a coded video picture to predict a current video block. Temporal prediction may reduce temporal redundancy inherent in, for example, a video signal. The temporal prediction signal for a CU may be signaled by (e.g., using) one or more motion vectors (MVs). These MVs may indicate the amount and / or direction (or directions) of motion between the CU and its temporal reference. If multiple reference pictures for a (e.g., each) video block are supported, a reference picture index for the video block may be sent. The reference picture index may be used to identify which reference picture in the reference picture store 264 the temporal prediction signal may be derived from.

[0099] After spatial and / or temporal prediction, the mode decision unit 280 may determine a prediction mode (e.g., an optimal prediction mode), for example, based on rate-distortion optimization (RDO). The prediction block may be subtracted from the current video block at the adder 216, and / or the prediction residual may be decorrelated using the transform unit 204 and / or the quantization unit 206. The quantized residual coefficients may be inverse quantized and / or inverse transformed using the inverse quantization unit 210 and the inverse transform unit 212, respectively, to form a reconstructed residual. The reconstructed residual may be added back to the prediction block using the adder 226 to form a reconstructed CU. Before the reconstructed CU is stored in the reference picture repository 264, an in-loop filtering (e.g., any of a deblocking filter and an adaptive loop filter, such as a sample adaptive offset SAO and an adaptive in-loop filter ALF) may be applied to the reconstructed CU using the filter unit 266. The reconstructed CU stored in the reference picture repository 264 may be used to decode a video block (e.g., a future video block). Various information including coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients may be passed to the entropy coding unit 208. The entropy coding unit 208 may compress and / or pack such information to form the output video bitstream 220.

[0100] Figure 4 is a block diagram illustrating an exemplary block-based hybrid video decoding framework 300 for a decoder. The decoder and thus the exemplary block-based hybrid video decoding framework 300 may be implemented in a device having a processor communicatively coupled to a memory. The memory may include instructions executable by the processor, including instructions for performing any of the various embodiments disclosed herein (e.g., representative processes). In various embodiments, the device may be configured as and / or configured with various elements of a WTRU.

[0101] The block-based hybrid video coding framework 300 may include an entropy decoding unit 308, an inverse quantization unit 310, an inverse transform unit 312, an adder 326, a spatial prediction unit 360, a motion / temporal prediction unit 362, a reference picture repository 364, and a filter 366. The block-based hybrid video coding framework 300 (and decoder) may include additional and / or different elements. For the sake of brevity and clarity, these elements are not shown.

[0102] Video bitstream 302 (eg, Figure 2The video bitstream 220 of the video bitstream 220 may be unpacked (e.g., first unpacked) and / or entropy decoded at the entropy decoding unit 308. The coding mode and prediction information may be passed to the spatial prediction unit 360 (e.g., if intra-coded) and / or the motion / temporal prediction unit 362 (e.g., if inter-coded and / or temporal coded) to form a prediction block. The residual transform coefficients may be passed to the inverse quantization unit 310 and / or the inverse transform unit 312, for example, to reconstruct the residual block. The prediction block and the residual block may be added together at the adder 326. The reconstructed block may be subjected to in-loop filtering at the filter 366, for example, which may occur before the reconstructed block is stored in the reference picture store 364. The reconstructed video 320 in the reference picture store 364 may be sent to drive a display device and / or used to predict a video block (e.g., a future video block).

[0103] Using bidirectional motion compensated prediction (MCP) in a video codec can remove temporal redundancy by exploiting the temporal correlation between pictures. A bidirectional prediction signal can be formed by combining two unidirectional prediction signals using a weight value (e.g., 0.5). In some videos, illumination characteristics can change rapidly from one reference picture to another. Therefore, prediction techniques can compensate for changes in illumination over time (e.g., fading transitions) by applying global or local weights and offset values ​​to one or more sample values ​​in a reference picture.

[0104] Other coding tools, such as weighted prediction (WP), can compensate for changes in illumination over time when performing motion compensation. For example, for each reference picture in each reference picture list (L0 and L1), a set of multiplicative weights and additive offsets can be signaled at the slice. The weights and offsets can be applied during the MCP stage when the corresponding reference pictures can be used. Thus, the WP coding tool can be employed when illumination changes linearly from picture to picture and / or when changes in illumination are global at the picture / slice level.

[0105] Generalized bidirectional prediction (GBi) can improve the MCP of bidirectional prediction mode. In bidirectional prediction mode, the prediction signal at sample x as given can be calculated by equation ("Equation") (1):

[0106] P[x]=w 0 *P 0 [x+v 0 ]+w 1 *P 1 [x+v 1 ], (1)

[0107] In equation (1), P[x] may represent the resulting prediction signal of sample x at position x in the picture. i [x+vi ] may be the motion vector (MV) v using the i-th list (e.g., list 0, list 1, etc.) i The motion compensation prediction signal of x. 0 and w 1 can be two weight values ​​shared among (eg, all) samples in a block. Based on this equation, the weight value w can be adjusted by 0 and w 1 To obtain a variety of prediction signals. 0 and w 1 Some configurations of may imply the same prediction as unidirectional prediction and bidirectional prediction. For example, (w 0 ,w 1 )=(1,0) can be used for unidirectional prediction using reference list L0. 0 ,w 1 )=(0,1) can be used for unidirectional prediction using reference list L1. 0 ,w 1 ) = (0.5, 0.5) can be used for bidirectional prediction with two reference lists. The weights can be signaled for each CU. To reduce signaling overhead, constraints such as w 0 +w 1 =1, so that a weight can be signaled. Therefore, equation (1) can be further simplified as shown in equation (2):

[0108] P[x]=(1-w 1 )*P 0 [x+v 0 ]+w 1 *P 1 [x+v 1 ]. (2)

[0109] In order to further reduce the weight signaling overhead, w 1 can be discretized (e.g., -2 / 8, 2 / 8, 3 / 8, 4 / 8, 5 / 8, 6 / 8, 10 / 8, etc.) Each weight value can then be indicated by an index value within a (e.g., small) limited range.

[0110] Figure 5 4 is a block diagram showing an exemplary video encoder with GBi support. The encoder may include a mode decision unit 404, a spatial prediction unit 406, a motion prediction unit 408, a transform unit 410, a quantization unit 412, an inverse quantization unit 416, an inverse transform unit 418, a loop filter 420, a reference picture store 422, and an entropy decoding unit 414. Some or all of the units or components of the encoder (e.g., the spatial prediction unit 406) may be combined with Figure 2The units or components described above are the same or similar. In addition, the spatial prediction unit 406 and the motion prediction unit 408 may be pixel domain prediction units. Therefore, the input video bitstream 402 can be processed in a similar manner to the input video bitstream 202, although the motion prediction unit 408 may further include GBi support. Therefore, the motion prediction unit 408 can combine two separate prediction signals in a weighted average manner. In addition, the selected weight index can be signaled in the input video bitstream 402.

[0111] Figure 6 is a block diagram illustrating a GBi estimation unit 500 that may be employed in a motion prediction unit (e.g., motion prediction unit 408) of an encoder. The GBi estimation unit 500 may include a weight value estimation unit 502 and a motion estimation unit 504. In this way, the GBi estimation unit 500 may utilize a two-step process to generate an inter-frame prediction signal, such as a final inter-frame prediction signal. The motion estimation unit 504 may use reference pictures (one or more) received from a reference picture repository 506 and perform motion estimation by searching for two best motion vectors (MVs) pointing to (e.g., two) reference blocks. The weight value estimation unit 502 may search for an optimal weight index to minimize a weighted bidirectional prediction error between a current video block and a bidirectional prediction signal. The prediction signal of the generalized bidirectional prediction may be calculated as a weighted average of the two prediction blocks.

[0112] Figure 7 is a diagram showing the decoder that can be decoded from a GBi-supported encoder (e.g., in combination with Figure 4 A block diagram of an exemplary video decoder for a bit stream of an encoder as described above. Figure 6 As shown, the decoder may include an entropy decoder 604, a spatial prediction unit 606, a motion prediction unit 608, a reference picture repository 610, an inverse quantization unit 612, an inverse transform unit 614, and a loop filter unit 618. Some or all of the units of the decoder may be combined with Figure 2 The motion prediction unit 608 may be configured for GBi support. In this way, the coding mode and prediction information may be used to derive a prediction signal using spatial prediction or MCP with GBi support. For GBi, block motion information and weight values ​​(e.g., in the form of an index indicating the weight value) may be received and decoded to generate the prediction block.

[0113] Figure 8is a block diagram showing a GBi prediction unit 700 that may be employed in a motion prediction unit of a decoder (e.g., the motion prediction unit 608). The GBi prediction unit 700 may include a weighted average unit 702 and a motion compensation unit 704, which may receive one or more reference pictures from a reference picture store 706. The GBi prediction unit 700 may calculate a prediction signal of GBi as a weighted average of (e.g., two) motion compensated prediction blocks using block motion information and weight values.

[0114] There may be various types of motion within a particular video, such as zooming in / out, rotation, perspective motion, and other irregular motions. A translational motion model and / or an affine motion model may be applied to the MCP in conjunction with one or more of the various types of motion. The affine motion model may be four parameters and / or six parameters. A first flag for (e.g., each) inter-coded CU may be signaled to indicate whether the translational motion model or the affine motion model is applied for inter prediction. If the affine motion model is applied, a second flag may be sent to indicate whether the model is four parameters or six parameters.

[0115] The four-parameter affine motion model may include two parameters for translation motion in the horizontal and vertical directions, one parameter for scaling motion in the horizontal and vertical directions, and / or one parameter for rotation motion in the horizontal and vertical directions. The horizontal scaling parameter may be equal to the vertical scaling parameter. The horizontal rotation parameter may be equal to the vertical rotation parameter. The four-parameter affine motion model may be coded (e.g., in a VVC test model (VTM)) using two motion vectors at two control point locations defined at the top-left and top-right corners of a (e.g., current) CU.

[0116] Various template-based decoding techniques (e.g., tools) can be performed by the disclosed block-based hybrid video decoding framework (e.g., framework 200, 300, 400 and / or 600), encoder and / or decoder. Such template-based decoding techniques adopt corresponding models / methods; each technique is characterized by one or more parameters. Currently, there is no unified model / method in template-based decoding techniques for parameter estimation (including template-based decoding techniques that utilize the same or similar types of parameters). As an example, consider the current template-based decoding techniques of cross-component linear model (CCLM) and local illumination compensation (LIC). Although CCLM assumes linear correlation between luma and chroma components in the decoding block, and LIC assumes linear correlation between luma samples of a reference frame and a current frame, they adopt different models / methods for parameter estimation.

[0117] Fig. 99 is a block diagram showing a set of frames / pictures on which LIC may be performed. LIC is an inter-coding tool that may be used to compensate for illumination differences between (i) samples (predictions) from a reference frame and (ii) samples from a current frame. For example, when a camera captures a scene with illumination differences between frames, prediction (e.g., using conventional prediction techniques) may fail. Changes in illumination may occur due to fade-out, fade-in, fade-out scenes (one or more) with illumination differences between the current frame and the reference frame. LIC may access reconstructed samples of neighboring CUs in the current frame and corresponding samples in the reference frame to calculate LIC parameters for an illumination compensation model. The illumination compensation model may be applied to the current motion compensated pixel to compensate for the illumination difference.

[0118] For the current CU, the neighboring reconstructed samples in the top and left rows and columns of both the CU and the reference block of the reference frame are used as input to determine a linear model characterized by a scaling factor α and an offset β. The local illumination compensation parameters are generated using an adaptive downsampling technique of the corresponding reference samples from the neighboring blocks of the current CU block.

[0119] LIC can be adaptively turned on and off for each inter-coded CU. For integer-pixel motion search and fractional-pixel motion search, mean removed sum of absolute differences (MR-SAD) and mean removed sum of absolute Hadamard transform differences (MR-SATD) are obtained instead of SAD and SATD, respectively. When each CU is decoded in 2N×2N merge mode, the LIC flag is copied from the neighboring blocks selected as merge candidates in a manner similar to the motion information copying in merge mode. Otherwise, the LIC flag for the CU is signaled to indicate whether LIC is applicable.

[0120] When LIC is applied, the scaling factor α and the offset β of the linear model are determined using the least squares error method. The goal is to minimize the error between the reconstructed neighboring samples and their corresponding adjusted prediction samples, where the adjusted prediction samples are generated using the linear model applied to the prediction samples, as shown in equation (3).

[0121] P′ L (x, y) = α·P L (x,y)+β (3)

[0122] Among them, P L (x, y) represents the prediction of the unadjusted adjacent luma samples, and P′ L (x, y) represents the adjusted prediction sample. The parameters α and β are derived by minimizing the regression error between the reconstructed neighboring luma samples of the decoded block in the current frame and the adjusted prediction samples.

[0123] When the LIC and GBI tools are used together (e.g., in BMS-2.1rc1 (VTM configuration), the gains of the two tools are cumulative. For a bidirectionally predicted block, LIC parameters (weights and offsets) are estimated for each prediction direction separately. The estimated LIC parameters are applied to each prediction signal separately, and then the two LIC modified prediction signals are combined to form a bidirectional prediction signal, in other words, two separate LIC processes are used for the bidirectionally predicted CU.

[0124] As described above, when LIC is applied to a CU, the least square error method is used to determine the scaling factor α and the offset β. The parameters α and β are derived by minimizing the regression error between the current and reference luma samples as follows.

[0125] Given a set of n points (X i ,Y i ), the goal is to find the best fit line

[0126]

[0127] So that the error Minimize. This requires solving a least squares model, which is a computationally complex operation.

[0128] Under CCLM, the chroma components in a block can be predicted from the luma components using a linear model. An additional rate-distortion (RD) cost check of the chroma components is added to select the chroma intra prediction mode. When using LM mode, the chroma components are predicted from the reconstructed luma samples as:

[0129] Pred c (x, y) = α·Rec′ L (x,y)+β (5)

[0130] Among them, Pred c (x,y) represents the prediction of chrominance samples, and Rec′ L (x, y) represents the reconstructed brightness sample value. Rec′ L (x, y) can be downsampled from the reconstructed luma samples in the decoded block to align the prediction samples from the luma component with the chroma components for non-4:4:4 chroma formats. The parameters α and β determine the model used for prediction. In some embodiments, the optimal parameters are determined using a least squares minimization method. The parameters α and β are derived by minimizing the regression error between the adjacent reconstructed luma and chroma samples around the current block as follows:

[0131]

[0132] Where L(n) represents the downsampled top and left neighboring reconstructed luma samples, C(n) represents the top and left neighboring reconstructed chroma samples, and N is equal to twice the minimum of the width and height of the current chroma coding block. For coding blocks with a square shape, equation (6) is used directly. For non-square coding blocks, the neighboring samples of the longer boundary are first subsampled to have the same number of samples as the shorter boundary. Fig.10 Samples of the current block for CCLM mode are shown, along with example locations of the left and above causal samples.

[0133] The regression error minimization calculation (Eq. (6)) is performed as part of the decoding process to derive the α and β values, rather than just as an encoder search operation. This means that no syntax elements are required to signal the α and β values.

[0134] In one implementation, straight line estimation is used for the CCLM (ie, instead of a least squares (LS) algorithm). Fig.11 An example of a straight-line estimate for deriving the linear model parameters α and β is shown.

[0135] The linear model parameters α and β are obtained using the minimum and maximum luma and chroma sample value pairs within a set of adjacent samples (e.g., as shown) and the following equations:

[0136]

[0137] The value x along the x-axis A and x B is the brightness value at points A and B. The value y along the y-axis A and B are the chromaticity values ​​at points A and B respectively.

[0138] A conventional straight line estimation form of CCLM ("CCLM-SLE") uses two lookup tables (LUTs), equation (8), and an instantiation of pseudocode A (below) to compute the most significant bits (MSBs) and the least significant bits (LSBs).

[0139]

[0140] In the conventional straight line estimation form of CCLM-SLE, the MSB and LSB LUTs both have 512 entries; each of which is 16 bits (thus, the MSB and LSB are calculated using 1024 entries with 16 bits of computational precision (i.e., n=16). The MSB and LSB LUTs are used to calculate the model parameters α and β as follows. The minimum and maximum values ​​of the luma and chroma samples are determined, and these parameters α and β are calculated according to the following pseudo code A:

[0141] shift=(BitDepthC>8)? BitDepthC-9:0

[0142] add=shift? 1<<(shift-1):0

[0143] diff=(MaxLuma-MinLuma+add)>>shift

[0144] k=16

[0145] If(diff>0)

[0146] div=((ChromaForMaxLuma-ChromaForMinLuma)*(LSB)>>16

[0147] α=(((ChromaForMaxLuma-ChromaForMinLuma)*(MSB)+div+add)>>shift)

[0148] Otherwise,

[0149] α=0

[0150] β=ChromaForMinLuma-((a*MinLuma)>>k)

[0151] Pseudocode A - Conventional CCLM-SLE

[0152] Among them, BitDepthC is the chroma bit depth, MaxLuma is the maximum value of the top and left adjacent luma samples downsampled, MinLuma is the minimum value of the top and left adjacent luma samples downsampled, ChromaForMaxLuma is selected from the top and left adjacent reconstructed chroma samples and corresponds to the chroma of the maximum luma value, ChromaForMinLuma is selected from the top and left adjacent reconstructed chroma samples and corresponds to the chroma of the minimum luma sample value.

[0153] According to the above pseudo code, parameter BitDepthC limits the chroma bit depth to 9 bits. The slope α and offset β parameters are calculated with 16-bit precision using the values ​​from the MSB and LSBLUT. The error introduced by using the MSB is mitigated by the LSB calculation, i.e., parameter div is added to account for the error introduced by integer-based operations when calculating the slope and offset metrics.

[0154] CCLM-SLE has two disadvantages. First, the hardware implementation is expensive because 2K bytes of memory must be occupied to store the LUT, which is used to calculate the MSB and LSB during the calculation of CCLM-SLE parameters. Second, errors occur when calculating CCLM-SLE parameters because CCLM-SLE uses a set of minimum and maximum values ​​in a set of adjacent luminance samples, and the minimum and maximum values ​​are prone to errors due to the presence of outliers.

[0155] In addition to the above-mentioned disadvantages of LIC and CCLM, there is an additional burden on hardware implementation because although both LIC and CCLM estimate linear model parameters, the estimation methods are different.

[0156] Overview

[0157] As will be understood by those skilled in the art based on the teachings herein, within the embodiments described herein are included, but are not limited to, processes, methods, architectures, devices, systems, apparatuses, and computer program products for improved linear model estimation for template-based video decoding, including a device (e.g., implementing any of an encoder or a decoder) that can be configured to use a straight line estimation model / method employed by a template-based decoding technique (e.g., tool) (such as CCLM, LIC, etc.) to predict the chrominance (or luminance) component of the current decoding block, wherein a single lookup table ("LUT") is used and the model parameters α (e.g., slope) and β (e.g., intercept) are calculated with no more than 16 bits of precision.

[0158] Included in the process, method, architecture, apparatus, system, device and computer program product is a method that can be implemented in any of an encoder and a decoder and can include determining a minimum and a maximum value of luma samples adjacent to a coding block; determining a minimum and a maximum value of chroma samples adjacent to the coding block, the minimum and the maximum value of the chroma samples corresponding to the minimum and the maximum value of the luma samples, respectively; determining (e.g., calculating) a first linear model parameter (e.g., a slope) of a template-based video coding technique, the first linear model parameter (e.g., a slope) being (i) a single linear model parameter (e.g., a slope) of the template-based video coding technique; The invention relates to a video decoding technique for decoding a chroma block, wherein the first linear model parameter is a function of (i) the minimum value of the chroma sample, the minimum value of the luma sample and the first linear model parameter, and (ii) the precision is no greater than 16 bits; determining (e.g., calculating) a second linear model parameter (e.g., an intercept) of the template-based video decoding technique, wherein the second linear model parameter (e.g., an intercept) is a function of (i) the minimum value of the chroma sample, the minimum value of the luma sample and the first linear model parameter, and (ii) the precision is no greater than 16 bits; and predicting the chroma samples of the decoding block based on the first linear model parameter, the second linear model parameter and the reconstructed luma samples of the decoding block.

[0159] In an embodiment, the single LUT may exclude the value for determining the least significant bit. Alternatively, the single LUT may include the value for determining only the least significant bit (e.g., LSB-only LUT) or the value for determining only the most significant bit (e.g., MSB-only LUT).

[0160] In an embodiment, the minimum and maximum values ​​of the luminance and chrominance may correspond to a corresponding template. In an embodiment, the minimum and maximum values ​​of the luminance and chrominance may correspond to a corresponding template, but may not include all samples of such a template.

[0161] In an embodiment, the method may include any of: trimming one or more samples of a luma template to obtain the luma sample used to determine the minimum and maximum values ​​of the luma sample (e.g., obtained before it); and trimming one or more samples of the luma template to obtain the luma sample used to determine the minimum and maximum values ​​of the luma sample (e.g., obtained before it) (1704). In an embodiment, the trimming of the luma / chroma samples may include removing outliers or otherwise excluding one or more samples of the luma / chroma template based on a statistical method. In an embodiment, the luma and chroma samples belong to a point distribution having a single region or to one of multiple regions of the point distribution.

[0162] In one embodiment, the method may determine the minimum and maximum values ​​of the chroma samples adjacent to the decoding block, and then may determine the minimum and maximum values ​​of the luma samples adjacent to the decoding block, which correspond to the minimum and maximum values ​​of the chroma samples, respectively. In one embodiment, the luma component of the current decoding block may be predicted instead of the chroma component by performing a method in which the terms "chroma" and "luminance" are interchanged.

[0163] Included in the process, method, architecture, apparatus, system, device and computer program product is a method that can be implemented in any of an encoder and a decoder and can include: separating luma and chroma samples adjacent to a coding block into a plurality of sample clusters based on one or more thresholds; and on the condition that the separation of the first and second sample clusters satisfies a separation metric: determining a minimum and a maximum value of luma samples of the first sample cluster; determining a minimum and a maximum value of the chroma samples of the first sample cluster, which correspond to the minimum and the maximum value of the luma samples of the first sample cluster, respectively; determining (e.g., calculating) first linear model parameters of a first template-based video decoding technique, the first linear model parameters (i) being a function of the minimum and the maximum value of the chroma samples of the first sample cluster, and (ii) being a first precision; determining (e.g., calculating) second linear model parameters of the first template-based video decoding technique, the second linear model parameters (i) being a function of the minimum and the maximum value of the chroma samples of the first sample cluster, and (ii) being a first precision; determining a minimum value and a maximum value of luma samples of a second sample cluster; determining a minimum value and a maximum value of the chroma samples of the second sample cluster, the minimum value and the maximum value corresponding to the minimum value and the maximum value of the luma samples of the second sample cluster, respectively; determining (e.g., calculating) a first linear model parameter of a second template-based video coding technique, the first linear model parameter (i) being a function of the minimum value and the maximum value of the chroma samples of the second sample cluster, and (ii) being at the second precision; determining (e.g., calculating) calculating a second linear model parameter of the second template-based video coding technique, the second linear model parameter (i) being a function of the minimum value of the chroma samples of the second sample cluster, the minimum value of the luma samples of the second sample cluster, and the first linear model parameter of the second template-based video coding technique, and (ii) having the second precision; and predicting the chroma samples of the decoding block based on the first and second linear model parameters of the first and second template-based video coding techniques and the reconstructed luma samples of the decoding block.

[0164] In one embodiment, at least one of the thresholds may be based on an average value of the luma samples adjacent to the coding block. In one embodiment, the first and second linear model parameters of each of the first and second template-based video coding techniques are a slope and an intercept.

[0165] In one embodiment, the method may include: under the condition that the separation of any two clusters does not satisfy the separation metric, recombining the two clusters and using the single straight line estimation model / method adopted by the template-based decoding technique to make predictions.

[0166] In an embodiment, the first linear model parameters of the first template-based video decoding technique may be calculated as a function of the minimum and maximum values ​​of the chroma samples of the first sample cluster and one or more first LUTs. In an embodiment, the first linear model parameters of the second template-based video decoding technique are calculated as a function of the minimum and maximum values ​​of the chroma samples of the second sample cluster and one or more LUTs. In an embodiment, any of the LUTs may exclude values ​​used to determine the least significant bit. In an embodiment, any of the LUTs may include values ​​used only to determine the least significant bit. In an embodiment, any of the LUTs may include values ​​used only to determine the most significant bit.

[0167] In an embodiment, the first precision may be based on a statistical description (e.g., distribution) of samples within the first sample cluster. In an embodiment, the second precision may be based on a statistical description (e.g., distribution) of the samples within the second sample cluster. In an embodiment, the first linear model parameters of the first template-based video coding technique may be calculated with a precision of no more than 16 bits. In an embodiment, the second linear model parameters of the first template-based video coding technique may be calculated with a precision of no more than 16 bits. In an embodiment, the first linear model parameters of the second template-based video coding technique may be calculated with a precision of no more than 16 bits. In an embodiment, the second linear model parameters of the first template-based video coding technique may be calculated with a precision of no more than 16 bits.

[0168] In an embodiment, said minimum and maximum values ​​of said luma and chroma may correspond to respective templates. In an embodiment, said minimum and maximum values ​​of said luma and chroma may correspond to respective templates, but may not include all samples of such templates. In an embodiment, the method may include pruning one or more samples of the luma and / or chroma templates prior to separation.

[0169] In an embodiment, the method may include: pruning one or more luma / chroma samples of the first sample cluster before determining the minimum and maximum values ​​of the luma / chroma samples of the first sample cluster, and / or pruning one or more luma / chroma samples of the second sample cluster before determining the minimum and maximum values ​​of the luma / chroma samples of the second sample cluster. In an embodiment, the pruning of the luma / chroma samples may include removing outliers or otherwise excluding one or more samples of the luma / chroma template based on statistical methods, such as based on descriptive statistics (e.g., as described herein).

[0170] In one embodiment, the method may include determining a minimum and a maximum value of the chroma samples of the first (and / or second) sample cluster, and then determining a minimum and a maximum value of the luma samples of the first (and / or second) sample cluster corresponding to the minimum and the maximum value of the chroma samples, respectively. In one embodiment, the luma component of the current coding block may be predicted instead of the chroma component by performing a method in which the terms "chroma" and "luminance" are interchanged.

[0171] Included in a process, method, architecture, apparatus, system, device, and computer program product is a method comprising: applying a unified linear model parameter estimation method to various template-based video decoding techniques such as LIC and CCLM. In one embodiment, LIC parameter calculations may employ a linear model according to a conventional CCLM (e.g., CCLM-SLE). In one embodiment, both LIC and CCLM parameter calculations may employ linear models, which are less memory-intensive than conventional CCLM versions (e.g., CCLM-SLE). In an embodiment, statistical methods and / or measurements of luma and / or chroma samples (e.g., mean and variance of luma and / or chroma samples or other statistical methods of distribution) may be used to calculate linear model parameters (e.g., parameters α and β). In one embodiment, the statistical methods and / or measurements may be based on the following assumptions, and / or the application of the statistical methods and / or measurements may be conditioned on the following assumptions: based on the assumption that the reconstructed distribution and its prediction are correlated. In embodiments, the statistical method and / or measurement may be based on, and / or application of, the statistical method and / or measurement may be conditioned on, a determination that the reconstructed distribution and its prediction are related.

[0172] According to the methods and techniques provided herein, since statistical measurements are used to calculate the scaling factor α and the offset β, the parameter estimation robustness can be improved relative to the traditional solution. In one embodiment, the statistical measurements can be used to employ outlier removal techniques. Removing outliers can increase the robustness over the traditional CCLM.

[0173] In the following description, the method based on the minimum-maximum can be used to calculate the linear model parameters. Those skilled in the art will recognize that the weighted prediction-based method can also be used to calculate these parameters.

[0174] In a method for improved linear model estimation for template-based video decoding, a method may include any of the following steps: generating a model for any of a variety of template-based video decoding techniques, which may be based on parameters estimated or otherwise determined using a unified linear model; and using the generated model to perform template-based video decoding.

[0175] In one embodiment, the various template-based video decoding techniques may include any of CCLM and LIC. In one embodiment, the template-based decoding techniques utilize the same or similar type of model parameters. In one embodiment, the model parameters include any of slope and offset.

[0176] In one embodiment, the method may include estimating model parameters, or otherwise using the unified linear model and template samples to determine model parameters. In one embodiment, the unified linear model may include any of the following: a min-max based method and a weighted prediction based method.

[0177] In an embodiment, the template sample may include any of luma and chroma samples. In an embodiment, the template sample may include samples placed above and / or to the left of the CU.

[0178] In one embodiment, the unified linear model may use statistical methods (e.g., descriptive statistics) about the template samples and / or measurements of the template samples to calculate the model parameters. In one embodiment, the statistical methods and / or measurements may include any of the mean, median, variance, and standard deviation.

[0179] In one embodiment, the method may include removing outliers in the template samples (eg, before calculating the model parameters). In one embodiment, the unified linear model may include any of the following: a min-max based method and a weighted prediction based method.

[0180] Among the processes, methods, architectures, apparatuses, systems, devices, and computer program products is an apparatus that may include any of a processor and a memory configured to perform the method for improved linear model estimation for template-based video decoding provided herein and any of its embodiments. In one embodiment, the apparatus may be an element of a WTRU, may be configured as an element of a WTRU, and / or may be configured with an element of a WTRU.

[0181] Among the processes, methods, architectures, apparatuses, systems, devices, and computer program products is a system that may include any of a processor and a memory configured to perform the methods and embodiments thereof for improved linear model estimation for template-based video decoding provided herein. In addition, among the processes, methods, architectures, apparatuses, systems, devices, and computer program products is a tangible computer-readable storage medium having computer executable instructions stored thereon for performing any of the methods and embodiments thereof for improved linear model estimation for template-based video decoding provided herein.

[0182] Fig.12 An example LIC line estimation model for deriving linear model parameters is shown. The linear model parameters α and β can be obtained using minimum and maximum brightness predictions and brightness template point pairs within a set of adjacent brightness samples (eg, as shown) and the following equation.

[0183]

[0184] where the value x along the x-axis A and x B The templates at points A and B can be used to predict the brightness values, and the value y along the y-axis can be used to predict the brightness values, respectively. A and B They can be the template brightness values ​​at points A and B respectively.

[0185] According to equation (4) and its problem formulation, Can be X i The variance of It can be Y i The variance, σ xy Can be X i and Y i The cross-correlation of

[0186]

[0187] Assuming that E(Y|X) is the best predictor of Y and the best predictor is a linear function, E(Y|X) can be consistent with the best linear predictor of Y. Derivation The derivation process listed below in equations (11)-(13) is for the case where the conditional expectation is linear [equation (10)].

[0188]

[0189] In one embodiment, the unified linear model parameter estimation method may employ a mathematical model to statistically determine the offset and slope based on the input data distribution. Equation (10) may be derived as follows by fitting the data using the expected value of the linear model.

[0190] E(Y)=aE(X)+b

[0191]

[0192] The slope can be calculated as the ratio of the standard deviations of the two distributions given by:

[0193]

[0194] Representative template selection for LIC parameter derivation

[0195] As an alternative (or in addition) to using the upper template and the left template together to calculate the linear model coefficients (similar to CCLM-SLE), the upper template and the left template can be used separately in the LM_A mode and the LM_L mode, respectively. Fig.13 and 14 1 and 1 are block diagrams showing exemplary LM_A mode and LM_L mode, respectively. Fig.13 ), only the upper template can be used to calculate the linear model coefficients, while in LM_L mode ( Fig.14 ), only the left template can be used to calculate the linear model coefficients. Example details that can represent the LIC LM_A mode and LM_L mode can be found in JVET-L0336.

[0196] Representative multi-model extension for parameter derivation

[0197] The unified linear model parameter estimation method for LIC and CCLM (e.g., the minimum-maximum based straight line estimation model disclosed herein) can be extended to a multi-model parameter estimation method. One use case in which the multi-model parameter estimation method may be particularly useful is when a single model cannot be used to effectively model the change in brightness variance (or the change in brightness variance may not be effectively modeled). The multi-model parameter estimation method may use two or more linear models separated based on a threshold. For training purposes, each group may be treated independently to derive a unified linear model parameter estimation method for LIC or CCLM linear model parameters α and β. The multi-model parameter estimation method may be considered as a simplification of the K-means clustering method with n groups. Each of the n groups of linear regression may have enough samples to calculate α and β. Fig.15 An example multi-model parameter estimation method with two LIC min-max based linear models ("multi-model LIC") is shown. Although not shown, according to the accompanying Fig.15 Based on the description of FIG. 1 , a multi-model parameter estimation method with two CCLM min_max based linear models will be obvious to those skilled in the art.

[0198] The linear model based on min-max can perform better than the LS algorithm in smooth areas, which have higher correlation than textured areas where a single linear model may not be able to effectively describe the data distribution. In one embodiment, the multi-model min-max method can model the LS approximation. This multi-model min-max method can better characterize the changes in texture as well as the changes in brightness.

[0199] The multi-model LIC can be mathematically expressed as:

[0200]

[0201] Each group can be treated independently. Alternatively, if the data suggests that the groups can be combined into a single cluster, the groups can be combined, although both classifiers can model the point distribution. As an example, and referring to Fig.16 If the distance between the centroids of two groups is less than or otherwise meets a threshold, the groups can be merged to form a single cluster and treated together (even though two classifiers can model the distribution). The number of data points in each cluster can be used as a metric to ensure that each region has a sufficient number of data points to effectively model the classifier. This approach can be thought of as a piecewise linear model of the data based on the occurrence of the data points.

[0202] Representative process of multi-model LIC classifier

[0203] A threshold may be used to separate the point distribution into two regions. The threshold may be the average of the luminance samples or based on the average of the luminance samples. Alternatively, the threshold may be based on any of a variety of histogram analyses, as described in J. Liu, S. Wu, and JV Zidek, "On segmented multivariate regression," Statist. Sinica, vol. 7, no. 2, pp. 497-525, 1997.

[0204] The distance between the centroids of the two regions can be calculated. The distance can be calculated in various ways. As an example, for the first of the two clusters, the distance (D) between the data point and the centroid in the cluster can be calculated. 1 For the second of the two clusters, the distance (D 2 ). The distance between two centroids can be calculated (D N ). In (D N ≤λ*(D 1 +D 2)), the two regions can be merged into a single region and a single min-max method can be used to estimate / determine α and β. N >λ*(D 1 +D 2 )) under the condition that the multi-model min-max method can be used to estimate / determine α 0 ,β 0 and α 1 ,β 1 .

[0205] The distance (D N ) can be calculated as Euclidean distance or Manhattan distance. λ can be a predefined threshold (e.g., 1.0). N ) is large enough, the encoder can decide to use a single classifier model or a multi-classifier model based on the rate-distortion optimization (RDO) criterion. In one embodiment, a CU-level flag for context decoding can be signaled by the encoder to indicate the choice about model selection. The decoder can use this flag to perform model selection.

[0206] See below Fig.18 .

[0207] Representative LIC interactions with GBi

[0208] As mentioned above, the prediction signal for generalized bidirectional prediction is calculated as a weighted average of the two prediction blocks as follows:

[0209] P[x]=(1-w)×P 0 [x+v 0 ]+w×P 1 [x+v 1 ] (15)

[0210] Where P[x] is the prediction of the current block sample x at position x in the picture. Each P i [x+v i ] is the motion vector (MV) v from the reference picture in the reference list Li. i The motion compensation prediction of x, 1-w and w represent the motion compensation prediction applied to P 0 [x+v 0 ] and P 1 [x+v 1 ]’s weight value.

[0211] For each CU coded with LIC followed by GBi, the prediction in equation (15) can be reformulated as:

[0212] P[x]=(1-w)×(α 0 ×P 0[x+v 0 ]+β 0 )+w×(α 1 ×P 1 [x+v 1 ]+β 1 ) (16)

[0213] Among them, β 0 and β 1 can be used to correct P 0 and P 1 The DC value of the derived offset value, and α 0 and α 1 It can be a slope. If LIC is applied after GBi, then for each CU coded with GBi followed by LIC, the prediction in equation (15) can be reformulated as:

[0214] P′[x]=(α′×P[x]+β′) (17)

[0215] Where β′ may be a derived offset value used to correct the DC value of P′, and α′ may be the LIC slope. Applying LIC after GBi is computationally cheaper than applying LIC followed by GBi, because for the latter, LIC can be performed twice (e.g., for P 0 and P 1 Both), while for the former, LIC can be performed once (e.g., for P′).

[0216] Representative CCLM calculations

[0217] As described above, the calculation of CCLM parameters can use both the MSB and LSB obtained from the MSB LUT and the LSB LUT as input. The filling of the MSB LUT and the LSB LUT can be performed according to equation (8). The memory consumed for instantiating the MSB LUT and the LSB LUT can be, for example, 2K bytes (e.g., 2 LUTs×16-bit precision-2 LUTs×(2 bytes×512 entries)). In one embodiment, the CCLM parameters can be calculated using the MSB instead of the LSB. Not using the LSB in the calculation can eliminate the LSB LUT (e.g., 512 entries, each 16 bits) and the burden of storing it in the memory (e.g., 1K bytes).

[0218] The MSB LUT can be populated using the following

[0219]

[0220] A calculation accuracy of n=16 bits can be used here.

[0221] Calculating the CCLM model parameters α and β using the MSB LUT (rather than the LSB) may be performed as follows. The minimum and maximum values ​​of the luminance and chrominance parameters may be determined. The determined minimum and maximum values ​​of the luminance and chrominance parameters may be used as inputs to a calculation that may be performed according to the following pseudo code ("pseudo code B"):

[0222] shift=(BitDepthC>8)? BitDepthC-9:0

[0223] add=shift? 1<<(shift-1):0

[0224] diff=(MaxLuma-MinLuma+add)>>shift

[0225] k=16

[0226] If(diff>0)

[0227] α=(((ChromaForMaxLuma-ChromaForMinLuma)*(MSB)+add)>>shift)

[0228] Otherwise,

[0229] α=0

[0230] β=ChromaForMinLuma-((a*MinLuma)>>k)

[0231] Pseudocode B – Single LUT Linear Model Parameter Estimation Where BitDepthC can be the chroma bit depth, MaxLuma can be the maximum value of the downsampled top and left neighboring luma samples, MinLuma can be the minimum value of the downsampled top and left neighboring luma samples, ChromaForMaxLuma can be selected from the top and left neighboring reconstructed chroma samples and can correspond to the chroma of the luma maximum value, ChromaForMinLuma can be selected from the top and left neighboring reconstructed chroma samples and can correspond to the chroma of the luma sample minimum value.

[0232] According to the above pseudo code, the parameter BitDepthC can limit the chroma bit depth to 9 bits, and the parameters α (slope) and β (offset) can be calculated with 16-bit precision using the MSBLUT value (and without LSB and LSB LUT). According to the experimental results, no significant difference was found when the model parameters α and β were calculated using both MSB and LSB and when the model parameters α and β were calculated using MSB instead of LSB. As observed, the error generated by calculating with 16-bit precision is negligible. Most of the errors (if any) that can be mitigated by using LSB are due to the reduced precision caused by calculating BitDepthC at a limited bit depth (9 bits).

[0233] In one embodiment, the CCLM parameters may be calculated using both the MSB and the LSB but without the MSB LUT. Not using the MSB LUT (e.g., 512 entries, 16 bits per entry) in the calculation eliminates the burden of storing the MSB LUT in memory (e.g., 1K bytes). In one embodiment, the difference between the minimum and maximum sample values ​​may be calculated using 8-bit precision. The MSB may be calculated based on the average of a floor and a ceiling (e.g., using measurements). The calculation based on the average of the floor and ceiling may mitigate the loss caused by using a floor function instead of a ceiling function. By using the LSB from the LSB LUT, the loss caused by calculating the MSB without the MSB LUT may be mitigated. The LSB may be calculated with 16-bit (or higher) precision, and the LSB LUT may include 512 (or more) entries. Alternatively, the LSB may be calculated using 8-bit precision, and the LSB LUT may include 256 entries without significantly degrading performance. In addition to the reduced complexity, the memory burden of a 256-entry LSB LUT with 8-bit precision is 256 bytes (rather than the 1K bytes required for a 512-entry LSB LUT with 16-bit precision). Combinations of computational precision and LUT sizes other than 8-bit precision and 256 entries may also be used.

[0234] The MSB calculation (without any LUT) can be calculated as follows.

[0235] MSB=(2 n-floorLog2(diff) +2 n-1-floorLog2(diff) )>>1 (19)where foor(Log2(diff)) can be the lower bound of the logarithm base 2 of the input, diff (non-negative) can be the difference between the minimum and maximum sample values, and n can be the calculation precision (e.g., 16 bits, 8 bits, etc.).

[0236] The LSB LUT can be populated using the following equation

[0237]

[0238] Wherein the MSB may be from equation (19), diff (non-negative) may be the difference between the minimum and maximum sample values, and n may be the calculation precision (e.g., 16 bits, 8 bits, etc.). The calculation of the CCLM model parameters α and β is performed as follows, where both the MSB (e.g., the value calculated based on equation (19)) and the LSB LUT (e.g., filled using equation (20)) are used, but the MSB LUT is not used. The minimum and maximum values ​​of the luma and chroma parameters may be determined. The determined minimum and maximum values ​​of the luma and chroma parameters may be used as inputs to the calculations that may be performed according to the following pseudo code ("pseudo code C"):

[0239] shift=(BitDepthC>8)? BitDepthC-9:0

[0240] add=shift? 1<<(shift-1):0

[0241] diff=(MaxLuma-MinLuma+add)>>shift

[0242] k=16,8,etc

[0243] If(diff>0)

[0244] div=((ChromaForMaxLuma-ChromaForMinLuma)*(LSB)>>k

[0245] α=(((ChromaForMaxLuma-ChromaForMinLuma)*(MSB)+div+add)>>shift)

[0246] Otherwise,

[0247] α=0

[0248] β=ChromaForMinLuma-((a*MinLuma)>>k)

[0249] Pseudocode C - Reduced Precision Linear Model Parameter Estimation where BitDepthC may be the chroma bit depth, MaxLuma may be the maximum of the downsampled top and left neighboring luma samples, MinLuma may be the minimum of the downsampled top and left neighboring luma samples, ChromaForMaxLuma may be selected from the top and left neighboring reconstructed chroma samples and may correspond to the chroma of the luma maximum value, and ChromaForMinLuma may be selected from the top and left neighboring reconstructed chroma samples and may correspond to the chroma of the luma sample minimum value.

[0250] According to the above pseudo code, parameter BitDepthC can limit the chroma bit depth to 9 bits, and can use LSB LUT value to calculate parameter α (slope) and β (offset) with any of n-bit precision. MSB calculation can be performed using bitwise shift operation, and does not require MSB LUT. The error tolerance introduced by MSB calculation (if any) can be alleviated by LSB, and this alleviation can overcome the error tolerance introduced by lower calculation accuracy. According to experimental results, CCLM model parameters can be calculated with 8-bit accuracy and using LSB LUT with 64 entries, wherein there is negligible loss for common test condition sequence. Precision other than 8 bits can be used and LSB LUT with more or less than 64 entries can be used to calculate the CCLM model parameters. Various combinations of calculation accuracy and LSB LUT size are used to observe the small gain or negligible loss of common test condition sequence in some cases.

[0251] Representative CCLM robustness techniques

[0252] The straight line approximation may be susceptible to errors due to outliers because it depends on the choice of minimum and maximum values. In one embodiment, the outliers can be removed by discarding, for example, values ​​that are less than the mean value γ of the template samples and greater than δ times the mean value of the template samples. The values ​​of γ and δ can be determined empirically from a common test condition sequence, where the mean, minimum and maximum values ​​are used to determine γ and δ. Additionally and / or alternatively, a median measure can be used instead of an average measure. Additionally and / or alternatively, other statistical measures can be used, such as outlier removal or variance using a K-means clustering method.

[0253] Fig.171700 is a flow chart illustrating an example process 1700 according to one or more embodiments. According to process 1700, a straight line estimation model / method employed by a template-based coding technique (e.g., CCLM, LIC, etc.) may be used to predict chrominance components of a current decoding block, wherein a LUT is used and linear model parameters are calculated with an accuracy of no more than 16 bits (e.g., as a result of LUT configuration). Process 1700 may be performed by any of an apparatus (e.g., device, system, WTRU, etc.) and / or an encoder and a decoder implemented in such an apparatus (collectively referred to as "apparatus" in the description of process 1700 below).

[0254] refer to Fig.17 , the device may determine a minimum and a maximum value of luma samples adjacent to a coding block (1702). The device may determine a minimum and a maximum value of chroma samples adjacent to the coding block and corresponding to the minimum and maximum values ​​of the luma samples, respectively (1704). The device may determine (e.g., calculate) a first linear model parameter (e.g., a slope) of a template-based video coding technique based on the minimum and maximum values ​​of the chroma samples and using a single LUT (1706). For example, the device may calculate the first linear model parameter with an accuracy of no more than 16 bits based on the minimum and maximum values ​​of the chroma samples and using a single LUT. In an embodiment, the single LUT may exclude a value for determining the least significant bit. Alternatively, the single LUT may include a value for determining only the least significant bit (e.g., an LSB-only LUT) or a value for determining only the most significant bit (e.g., an MSB-only LUT).

[0255] The device may determine second linear model parameters of the template-based video decoding technique based on the minimum value of the chrominance and luma samples and the first parameter (1708). For example, the device may calculate the second linear model parameters with an accuracy of no more than 16 bits based on the minimum value of the chrominance and luma samples and the first linear model parameters. In one embodiment, the device may calculate the first and second linear model parameters based on equation (18) or its variant and instantiation of pseudocode B. The device may predict the chrominance samples of the decoding block based on the reconstructed luma samples of the decoding block and the first and second linear model parameters (1710).

[0256] In an embodiment, the minimum and maximum values ​​of the luminance and chrominance may correspond to a corresponding template. In an embodiment, the minimum and maximum values ​​of the luminance and chrominance may correspond to a corresponding template, but may not include all samples of such a template. Fig.17, but the device may trim one or more samples of the luma template to obtain luma samples for determining a minimum and a maximum of the luma samples (e.g., trimming is performed before the determination) (1702), and / or trimming one or more samples of the luma template to obtain luma samples for determining a minimum and a maximum of the luma samples (e.g., trimming is performed before the determination) (1704). In an embodiment, the trimming of the luma / chroma samples may include removing outliers, or otherwise excluding one or more samples of the luma / chroma template based on statistical methods, such as based on descriptive statistics (e.g., as described herein). In an embodiment, the luma and chroma samples belong to a point distribution having a single area (e.g., as described herein). Fig.11 ) or one of the multiple regions belonging to the point distribution (for example, Fig.15 an area above the area shown).

[0257] Although not shown, those skilled in the art will recognize that the apparatus may determine the minimum and maximum values ​​of the chroma samples adjacent to the decoding block (1702), and then may determine the minimum and maximum values ​​of the luma samples adjacent to the decoding block, which correspond to the minimum and maximum values ​​of the chroma samples, respectively (1704). In addition, those skilled in the art will recognize that the luma component of the current decoding block may also be predicted using process 1700 (e.g., by interchanging the terms "chroma" and "luminance").

[0258] Fig.18 1800 is a flow chart illustrating an example process 1800 according to one or more embodiments. According to process 1800, a plurality of straight line estimation models / methods employed by one or more template-based coding techniques such as CCLM, LIC, etc. may be used to predict chrominance components of a current decoding block. Process 1800 may be performed by an apparatus (e.g., device, system, WTRU, etc.) and / or any of an encoder and a decoder implemented in such an apparatus (collectively referred to as "apparatus" in the subsequent description of process 1800).

[0259] refer to Fig.18 The device may separate luma and chroma samples adjacent to a coding block into a plurality of sample clusters based on one or more thresholds (1802). In one embodiment, at least one of the thresholds may be based on an average value of luma samples adjacent to the coding block.

[0260] The apparatus may determine whether the separation of the first sample cluster and the second sample cluster satisfies a separation metric (1804). The separation metric may be, for example, a distance metric as described above under the representative process titled Multi-Model LIC Classifier, and / or based on a separation threshold determined by Otsu's method or based on a linear discriminant analysis (LDA) applied to luma samples. In the event that the separation of any two clusters does not satisfy the separation metric, the apparatus may regroup the two clusters and proceed with prediction using a single straight line estimation model / method employed by the template-based decoding technique, such as in process 1700 ( Fig.17 ), otherwise the device continues to execute process 1800.

[0261] The device may determine a minimum value and a maximum value of luma samples of a first sample cluster (1806). The device may determine a minimum value and a maximum value of chroma samples of the first sample cluster, which correspond to a minimum value and a maximum value of luma samples of the first sample cluster, respectively (1808). The device may determine (e.g., calculate) a first linear model parameter (e.g., a slope) of a first template-based video coding technique (i) based on the minimum value and the maximum value of the chroma samples of the first sample cluster and (ii) with a first precision (1810). The device may determine a second linear model parameter (e.g., an intercept) of the first template-based video coding technique (i) based on the minimum value and the first parameter of the chroma and luma samples of the first sample cluster and (ii) with the first precision (1812).

[0262] The device may determine a minimum value and a maximum value of luma samples of the second sample cluster (1814). The device may determine a minimum value and a maximum value of chroma samples of the second sample cluster that correspond to the minimum value and the maximum value of the luma samples of the second sample cluster, respectively (1816).

[0263] The device may determine (e.g., calculate) first linear model parameters (e.g., slope) of a second template-based video coding technique (i) based on the minimum and maximum values ​​of the chroma samples of the second sample cluster and (ii) with a second precision (1818). The device may determine second linear model parameters (e.g., intercept) of a second template-based video coding technique (i) based on the minimum values ​​of the chroma and luma samples of the second sample cluster and the first parameters of the second template-based video coding technique and (ii) with the second precision (1820). The device may predict chroma samples of the coding block based on the reconstructed luma samples of the coding block and the first and second linear model parameters of each of the first and second template-based video coding techniques (1822).

[0264] In an embodiment, the first linear model parameters of the first template-based video decoding technique may be calculated as a function of the minimum and maximum values ​​of the chroma samples of the first sample cluster and one or more first lookup tables. In an embodiment, the first linear model parameters of the second template-based video decoding technique are calculated as a function of the minimum and maximum values ​​of the chroma samples of the second sample cluster and one or more lookup tables. In an embodiment, any of the lookup tables may exclude values ​​for determining the least significant bit. In an embodiment, any of the lookup tables may include values ​​for determining only the least significant bit. In an embodiment, any of the lookup tables may include values ​​for determining only the most significant bit.

[0265] In an embodiment, the first precision may be based on a statistical description (e.g., distribution) of samples within the first sample cluster. In an embodiment, the second precision may be based on a statistical description (e.g., distribution) of samples within the second sample cluster. In an embodiment, the first linear model parameters of the first template-based video coding technique may be calculated with a precision of no more than 16 bits. In an embodiment, the second linear model parameters of the first template-based video coding technique may be calculated with a precision of no more than 16 bits. In an embodiment, the first linear model parameters of the second template-based video coding technique may be calculated with a precision of no more than 16 bits. In an embodiment, the second linear model parameters of the first template-based video coding technique may be calculated with a precision of no more than 16 bits.

[0266] In an embodiment, the minimum and maximum values ​​of the luminance and chrominance may correspond to a corresponding template. In an embodiment, the minimum and maximum values ​​of the luminance and chrominance may correspond to a corresponding template, but may not include all samples of such a template. Fig.18 Not shown, but the device may trim one or more samples of the luma and / or chroma template prior to separation (1802).

[0267] In an embodiment, the apparatus may prune one or more luma / chroma samples of the first sample cluster before determining the minimum and maximum values ​​of the luma / chroma samples of the first sample cluster, and / or prune one or more luma / chroma samples of the second sample cluster before determining the minimum and maximum values ​​of the luma / chroma samples of the second sample cluster. In an embodiment, the pruning of the luma / chroma samples may include removing outliers or otherwise excluding one or more samples of the luma / chroma template based on statistical methods, such as based on descriptive statistics (e.g., as described herein).

[0268] Although not shown, those skilled in the art will all recognize that the device can determine the minimum and maximum values ​​of the chroma samples of the first (and / or second) sample cluster (1806, 1814), and then can determine the minimum and maximum values ​​of the luma samples of the first (and / or second) sample cluster corresponding to the minimum and maximum values ​​of the chroma samples, respectively (1808, 1816). In addition, those skilled in the art will recognize that the flow 1800 can also be used to predict the luma component of the current decoding block (e.g., by interchanging the terms "chroma" and "luminance").

[0269] in conclusion

[0270] Although features and elements in specific combinations are provided above, it will be appreciated by those skilled in the art that each feature or element may be used alone or in any combination with other features and elements. The present disclosure is not limited in accordance with the embodiments described in the present application, wherein the purpose of the embodiments is to illustrate different aspects. It will be appreciated by those skilled in the art that numerous modifications and variations are possible without departing from the essence and scope. Unless provided in an explicit manner, the elements, behaviors or instructions used in the specification of the present application should not be interpreted as being essential to the present invention. In addition to the methods and devices enumerated here, it will be clear to those skilled in the art from the above description that functionally equivalent methods and devices within the scope of the present disclosure are within the scope of the present disclosure. Such modifications and variations should all fall within the scope of the appended claims. The present disclosure is limited only in accordance with the appended claims and the full range of equivalents that such claims have. It should be understood that the present disclosure is not limited to a specific method or system.

[0271] For simplicity, the foregoing embodiments are discussed with respect to terminology and structure of devices with infrared capabilities (i.e., infrared transmitters and receivers). However, the embodiments discussed are not limited to these systems, but may be applied to other systems using other forms of electromagnetic waves or non-electromagnetic waves such as sound waves.

[0272] It should also be understood that the terminology used herein is for the purpose of describing specific embodiments only and is not intended to be limiting. As used herein, the term "video" or the term "image" may refer to any of a snapshot, a single image, and / or a plurality of images displayed on a time basis. As another example, when referred to herein, the term "user equipment" and its abbreviation "UE", the term "remote" and / or the term "head-mounted display" or its abbreviation "HMD" may refer to or include: (i) a wireless transmit and / or receive unit (WTRU); (ii) any of a number of embodiments of a WTRU; (iii) a wireless and / or wired (e.g., wirelessly communicable) device that is configured with some or all of the structure and functionality of a WTRU, etc.; (iii) a device with wireless capabilities and / or wired capabilities that is configured with less than all of the structure and functionality of a WTRU; or (iv) the like. Reference is made to Attachment 3 herein. Figures 1A-1D Details of an example WTRU are provided, which may represent any WTRU described herein. As another example, various embodiments disclosed herein above and below are described as utilizing a head-mounted display. Those skilled in the art will recognize that devices other than head-mounted displays may be utilized, and some or all of the present disclosure and various disclosed embodiments may be modified accordingly without undue experimentation. Examples of such other devices may include drones or other devices configured to stream information for providing an adapted reality experience.

[0273] In addition, the methods provided herein may be implemented in a computer program, software, or firmware that is introduced into a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (which are transmitted via a wired or wireless connection) and computer-readable storage media. Examples of computer-readable media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, buffer memory, semiconductor storage devices, magnetic media (e.g., internal hard disks and removable disks), magneto-optical media, and optical media (e.g., CD-ROM discs and digital versatile discs (DVDs)). A processor associated with the software may be used to implement a radio frequency transceiver used in a WTRU, UE, terminal, base station, RNC, or any computer host.

[0274] Variations of the methods, devices, and systems provided above are possible without departing from the scope of the present invention. In view of the wide variety of embodiments that may be applied, it should be understood that the illustrated embodiments are merely examples and should not be considered to limit the scope of the appended claims. For example, embodiments provided herein include handheld devices that may include or be used with any suitable voltage source (such as a battery, etc.) that provides any suitable voltage.

[0275] In addition, in the above embodiments, processing platforms, computing systems, controllers, and other devices (including constraint servers and assembly points / servers containing processors) are mentioned. These devices may include at least one central processing unit ("CPU") and memory. According to the practice of technicians in the field of computer programming, references to behaviors or symbolic representations of operations or instructions may be performed by different CPUs and memories. Such behaviors and operations or instructions may be referred to as "running", "computer running", or "CPU running".

[0276] Those of ordinary skill in the art will appreciate that actions and symbolically represented operations or instructions include manipulation of electronic signals by a CPU. The electronic system represents data bits that may cause electronic signals to be transformed or reduced thereby, and stores the data bits in memory locations in a memory system, thereby reconfiguring or otherwise altering the data bits for CPU operations and other signal processing. The memory locations that hold data bits are physical locations that have specific electrical, magnetic, optical, or organic properties that correspond to or represent the data bits. It should be understood that the embodiments herein are not limited to the above-mentioned platforms or CPUs, and other platforms and CPUs may also support the provided methods.

[0277] The data bits may also be stored on a computer-readable medium, including a magnetic disk, an optical disk, and any other volatile (e.g., random access memory ("RAM")) or non-volatile (e.g., read-only memory ("ROM")) mass storage system that can be read by the CPU. The computer-readable medium may include cooperating or interconnected computer-readable media that may exist alone on a processing system or distributed among multiple interconnected processing systems that are local or remote to the processing system. It should be understood that these embodiments are not limited to the above-mentioned memories, and other platforms and memories may also support the provided methods.

[0278] In an illustrative embodiment, any operations, processes, etc. described herein may be implemented as computer-readable instructions stored on a computer-readable medium. The computer-readable instructions may be executed by a processor of a mobile unit, a network component, and / or any other computing device.

[0279] There is almost no difference between hardware and software embodiments of various aspects of the system. Whether to use hardware or software is usually (but not always, because in some contexts, the choice between hardware and software may be important) a design choice that represents a compromise between cost and efficiency. The processes and / or systems and / or other techniques described herein can be implemented by various carriers (e.g., hardware, software, and / or firmware), and the preferred carrier can change with the context in which the processes and / or systems and / or other techniques are deployed. For example, if the implementation determines that speed and accuracy are primary, then the implementer may tend to use primarily hardware and / or firmware carriers. If flexibility is primary, then the implementer may tend to use primarily software embodiments. Alternatively, the implementer may choose some combination of hardware, software, and / or firmware.

[0280] The above specific embodiments have been described in detail using block diagrams, flow charts, and / or examples to describe different embodiments of devices and / or processes. Just as such block diagrams, flow charts, and / or examples contain one or more functions and / or operations, those skilled in the art will understand that each function and / or operation within such block diagrams, flow charts, or examples can be implemented individually and / or collectively by a wide range of hardware, software, firmware, or nearly any combination thereof. In one embodiment, several portions of the subject matter described herein can be implemented by means of an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), and / or other integrated formats. However, those skilled in the art will recognize that some aspects of the embodiments disclosed herein may be implemented in whole or in part in an equivalent manner in an integrated circuit, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as nearly any combination thereof, and circuit design and / or code writing for software and / or firmware are also within the skill of those skilled in the art in accordance with the present disclosure. In addition, those skilled in the art will appreciate that the mechanisms of the subject matter described herein may be distributed in various forms as a program product, and that the illustrative embodiments of the subject matter described herein are applicable regardless of the particular type of signal-bearing medium used to actually perform the distribution. Examples of signal-bearing media include, but are not limited to, the following: recordable media, such as floppy disks, hard drives, CDs, DVDs, digital tapes, computer memories, and the like, and transmission-type media, such as digital and / or analog communication media (e.g., optical cables, waveguides, wired communication links, wireless communication links, and the like).

[0281] Those skilled in the art will recognize that in the art, it is common to describe devices and / or processes in the manner set forth herein, and thereafter use engineering practices to integrate such described devices and / or processes into data processing systems. That is, at least a portion of the devices and / or processes described herein can be integrated into data processing systems via a reasonable number of experiments. Those skilled in the art will recognize that a typical data processing system can generally include one or more of the following: a system unit housing, a video display device, a memory such as a volatile and non-volatile memory, a processor such as a microprocessor and a digital signal processor, a computing entity such as an operating system, a driver, a graphical user interface, and an application, one or more interactive devices such as a touch pad or a touch screen, and / or a control system including a feedback loop and a control motor (e.g., for sensing position and / or velocity feedback, for moving and / or adjusting components and / or amount of control motors). A typical data processing system can be implemented using any suitable commercially available components, such as those components typically found in data computing / communication and / or network computing / communication systems.

[0282] The subject matter described here sometimes shows different components contained in or connected to other different components.It should be understood that the architecture described in this way is only some examples, and other numerous architectures for implementing the same function are actually all implementable.Conceptually, any arrangement of the components that realize the same function is effectively "associated", so that the desired function can be realized.Therefore, any two components that are combined here to realize a specific function can be considered to be "associated" with each other, so that the desired function will be realized, without considering architecture or intermediate components.Similarly, any two components associated in this way can also be considered to be "operably connected" or "operably coupled" to each other, so as to realize the desired function, and any two components that can be associated in this way can also be considered to be "operably coupled" to each other, so as to realize the desired function.About the specific example that can be operably coupled, include but are not limited to components that can be physically paired and / or physically interacted and / or components that can be wirelessly interacted and / or wirelessly interacted and / or components that can be logically interacted and / or components that can be logically interacted.

[0283] As for the use of substantially any plural and / or singular terms herein, those skilled in the art may appropriately convert from the plural to the singular and / or from the singular to the plural, depending on the context and / or application. For clarity, various singular / plural arrangements may be explicitly set forth herein.

[0284] Those skilled in the art will understand that, in general, terms used herein, and particularly in the appended claims (e.g., the bodies of the appended claims), should generally be treated as "open" terms (for example, the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term "comprising" should be interpreted as "including but not limited to," etc.). Those skilled in the art will further understand that if an introduced claim recitation is directed to a specific quantity, such intent should be expressly recited in the claim, and if such recitation is absent, such intent is absent. For example, if only one item is intended, the term "single" or similar language may be used. As an aid to understanding, subsequent appended claims and / or the description herein may include the use of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be construed as a way of introducing claim recitations by the indefinite article "a" or "an" to limit any particular claim containing claim recitations introduced in this manner to embodiments containing only one such recitation, even if the same claim contains the introductory phrases "one or more" or "at least one" and an indefinite article such as "a" or "an" (e.g., "a" and / or "an" should be construed to mean "at least one" or "one or more"). The same is true for the use of definite articles to introduce claim recitations. Furthermore, even if a specific number of claim recitations introduced is explicitly recited, one skilled in the art will recognize that such recitation should be construed to refer to at least the recited number (e.g., an unmodified recitation of "two recitations" without other modifiers means at least two recitations or two or more recitations). Furthermore, in these examples, if a specification similar to “at least one of A, B, and C, etc.” is used, such a structure should generally have the meaning of the specification as understood by those skilled in the art (e.g., “a system having at least one of A, B, and C” would include, but not be limited to, a system having only A, only B, only C, A and B, A and C, B and C, and / or A, B, and C, etc.). In examples where a specification similar to “at least one of A, B, or C, etc. is used, such a structure should generally have the meaning of the specification as understood by those skilled in the art (e.g., “a system having at least one of A, B, or C” would include, but not be limited to, a system having only A, only B, only C, A and B, A and C, B and C, and / or A, B, and C, etc.).Those skilled in the art will further understand that almost any disjunctive words and / or phrases that propose two or more alternatives, whether in the specification, claims or drawings, should be understood to anticipate the possibility of including one, any or both of these items. For example, the phrase "A or B" will be understood to include the possibility of "A" or "B" or "A and B". In addition, the term "any" used herein followed by a series of multiple items and / or multiple item categories is intended to include "any", "any combination", "any multiple" and / or "any combination of multiple" in the items and / or item categories, either alone or in combination with other items and / or other item categories. In addition, the term "set" used herein should include any number of items, including zero. In addition, the term "quantity" used herein is intended to include any number, including zero.

[0285] In addition, where features or aspects of the disclosure are described in terms of Markush groups, those skilled in the art will recognize that the disclosure is thereby described in terms of any individual member or subgroup of members of the Markush group.

[0286] Those skilled in the art will appreciate that, for any and all purposes (e.g., in providing a written description), all ranges disclosed herein also include any and all possible sub-ranges and combinations of sub-ranges thereof. Any range listed can be easily considered to fully describe and enable the same range to be decomposed into at least two equal parts, three equal parts, four equal parts, five equal parts, ten equal parts, etc. As a non-limiting example, each range discussed herein can be easily decomposed into a lower third, a middle third, and an upper third range. Those skilled in the art will appreciate that all languages ​​such as "at most", "at least", "greater than", "less than", etc. include the narrated numbers and refer to ranges that can be subsequently decomposed into sub-ranges as described above. Finally, as those skilled in the art understand, a range includes each individual member. Thus, for example, a group having 1-3 cells refers to a group having 1, 2, or 3 cells. Similarly, a group having 1-5 cells refers to a group having 1, 2, 3, 4, or 5 cells, and so on.

[0287] Furthermore, unless otherwise indicated, the claims should not be misconstrued as limited to the described order or elements. In addition, the term "means for..." used in any claim is intended to be a "means-plus-function" claim format, and any claim without the word "means" is not intended to have this meaning.

Claims

1. A method for encoding video data by an encoder, the method comprising: Decoding the video block of the video data using a linear model of a template-based video decoding technique, the decoding comprising: determining a first parameter of the linear model using a single lookup table, the single lookup table including values ​​for determining a least significant bit, wherein determining the first parameter of the linear model comprises: determining minimum and maximum values ​​derived from luma samples adjacent to the video block; determining minimum and maximum values ​​derived from chroma samples adjacent to the video block; as well as calculating the first parameter of the linear model based on the inverse of the difference between (i) the minimum and maximum values ​​of the chroma samples and (ii) the minimum and maximum values ​​of the luma samples, wherein the inverse is derived from the single lookup table, and Samples of the video block are predicted based on the linear model.

2. The method according to claim 1, further comprising: A second parameter of the linear model is calculated according to a minimum value of the chroma samples, a minimum value of the luma samples, and the first parameter of the linear model. The method of claim 2 , wherein the second parameter is calculated with a precision less than or equal to 16 bits.

4. The method according to any one of claims 1 to 3, wherein the first parameter is calculated with a precision less than or equal to 16 bits. 5 . The method according to claim 1 , wherein the template-based video coding technique is based on a cross-component linear model (CCLM) or a local illumination compensation (LIC).

6. An encoder for encoding video data, comprising: at least one processor; and A memory storing instructions, which, when executed by the at least one processor, cause the encoder to perform the method according to any one of claims 1-5.

7. A non-transitory computer-readable medium having instructions stored thereon, which, when executed by at least one processor, cause the at least one processor to perform the method according to any one of claims 1-5.

8. A method for decoding video data by a decoder, the method comprising: Decoding the video block of the video data using a linear model of a template-based video coding technique, the decoding comprising: determining a first parameter of the linear model using a single lookup table, the single lookup table including values ​​for determining a least significant bit, wherein determining the first parameter of the linear model comprises: determining minimum and maximum values ​​derived from luma samples adjacent to the video block; determining minimum and maximum values ​​derived from chroma samples adjacent to the video block; as well as calculating the first parameter of the linear model based on the inverse of the difference between (i) the minimum and maximum values ​​of the chroma samples and (ii) the minimum and maximum values ​​of the luma samples, wherein the inverse is derived from the single lookup table, and Samples of the video block are predicted based on the linear model.

9. The method according to claim 8, further comprising: A second parameter of the linear model is calculated according to a minimum value of the chroma samples, a minimum value of the luma samples, and the first parameter of the linear model.

10. The method of claim 9, wherein the second parameter is calculated with a precision less than or equal to 16 bits.

11. The method according to any one of claims 8 to 10, wherein the first parameter is calculated with a precision less than or equal to 16 bits.

12. The method according to any one of claims 8 to 10, wherein the template-based video coding technique is based on a cross-component linear model (CCLM) or a local illumination compensation (LIC).

13. A decoder for decoding video data, comprising: at least one processor; and A memory storing instructions which, when executed by the at least one processor, cause the decoder to perform the method according to any one of claims 8 to 12.

14. A non-transitory computer-readable medium having instructions stored thereon, which, when executed by at least one processor, cause the at least one processor to perform the method according to any one of claims 8-12.

Images