Intra CIIP mode

Abstract

Disclosed herein are systems, methods, and instrumentalities for video encoding and / or decoding using intra combined intra-intra prediction modes. In an example, a video decoder / encoder may obtain a first prediction signal and a second intra prediction signal for a block. The first prediction signal may be associated with a first intra prediction mode. The second prediction signal may be associated with a second intra prediction mode. A decoder / encoder may generate prediction samples by determining a first weight associated with a first intra prediction signal and a second weight associated with a second intra prediction signal. The first weight and the second weight may be determined by a first neighboring block prediction mode and a second neighboring block prediction mode. The decoder / encoder may derive residual samples based on the prediction samples. The block may be decoded or encoded based on the derived residual samples.

Description

[0001] Cross-reference to related applications This application claims the benefit of European Provisional Patent Application No. 23305455.0, filed on March 30, 2023, and European Provisional Patent Application No. 23306561.4, filed on September 20, 2023, the contents of which are hereby incorporated by reference. Background Technology

[0002] Video coding systems can be used to compress digital video signals, for example, by reducing the storage and / or transmission bandwidth required for such signals. Video coding systems can include, for example, block-based, wavelet-based, and / or object-based systems. Summary of the Invention

[0003] This document discloses systems, methods, and means for video coding and / or decoding using intra-frame combined intra-frame prediction modes. The prediction process for video coding and / or decoding can combine IntraTMP (Intra-Template Matching) with CIIP (Combined Inter-Frame and Intra-Frame Prediction). This combination can add new modes that, for example, merge predictions from intra-frame prediction and predictions from IntraTMP (e.g., using the merging mechanism in CIIP).

[0004] A device, such as a video decoding device, can perform (e.g., be configured to perform) one or more of the following actions. The device can obtain, for a video block, a first intra-frame prediction signal associated with a first prediction mode and a second intra-frame prediction signal associated with a block vector-based prediction mode (e.g., a second prediction mode). The device can generate prediction samples associated with the video block based on the obtained intra-frame prediction signals (e.g., the first and second intra-frame prediction signals). For example, the device can combine the obtained prediction signals to generate prediction samples. The device can decode the video block based on the prediction samples.

[0005] In the example, the first intra-frame prediction signal can be associated with a list of most probable modes (MPMs) for the video block. In the example, the block vector-based prediction mode can be either Intra-Template Matching Prediction (IntraTMP) or Intra-Block Copy (IBC).

[0006] The device can determine a first weight associated with an intra-prediction signal associated with a first prediction mode and a second weight associated with an intra-prediction signal associated with a block vector-based prediction mode. Prediction samples associated with video blocks can be generated based on the first weight applied to the first intra-prediction signal and the second weight applied to the second intra-prediction signal. The first and second weights can be determined based on whether the IntraTMP is associated with one or more of a first block and a second block of neighboring video blocks.

[0007] In the example, the decoder and / or encoder can determine a first weight and a second weight. For example, based on the fact that either the first or second neighboring block prediction mode is an IntraTMP mode and either the first or second neighboring block prediction mode is a regular intra-frame prediction mode, the decoder and / or encoder can determine that the first weight and the second weight are equally weighted (e.g., each has a weight equal to two). For example, based on the fact that the first and second neighboring block prediction modes are IntraTMP modes, the decoder or encoder can determine that the first weight is 3 and the second weight is 1. For example, based on the fact that the first and second neighboring block prediction modes are regular intra-frame prediction modes, the decoder or encoder can determine that the first weight is 1 and the second weight is 3.

[0008] The device may (e.g., when the first intra-frame prediction signal is associated with a list of most probable modes (MPMs) for a video block) sort multiple MPM modes in the MPM list based on the corresponding cost associated with each MPM mode. The device may determine a first prediction mode based on the sorted multiple MPM modes. For example, the first intra-frame prediction signal may be obtained based on the first prediction mode (e.g., the first prediction mode among the sorted multiple MPM modes). For example, the first prediction mode may be determined as the MPM mode with the lowest corresponding cost. The device may obtain an index associated with the MPM list. The device may determine the first prediction mode based on the index (e.g., using the sorted multiple MPM modes). For example, the first prediction mode may be selected from the sorted MPM list based on the index.

[0009] The device can obtain multiple block vectors. The device can select a block vector from the multiple block vectors. A second intra-frame prediction signal can be obtained based on the selected block vector. The device can obtain an index. The selected block vector can be based on the index. The multiple block vectors can include one or more of multiple block vectors obtained by performing intra-frame template matching prediction (IntraTMP) on video blocks or from at least one block of neighboring video blocks.

[0010] A video decoding method may include obtaining, for a video block, a first intra-frame prediction signal associated with a first prediction mode and a second intra-frame prediction signal associated with a block vector-based prediction mode (e.g., a second prediction mode). For example, the block vector-based prediction mode may be IntraTMP or IBC. The method may include generating prediction samples associated with the video block based on the first and second intra-frame prediction signals. The method may include decoding the video block based on the prediction samples.

[0011] The method may include determining a first weight associated with a first intra-frame prediction signal and a second weight associated with a second intra-frame prediction signal. Prediction samples associated with video blocks may be generated based on the first weight applied to the first intra-frame prediction signal and the second weight applied to the second intra-frame prediction signal. The first and / or second weights may be determined based on whether the IntraTMP is associated with one or more of a first block and a second block of neighboring video blocks.

[0012] A device, such as a video encoding device, can perform (e.g., be configured to perform) one or more of the following actions. The device can obtain, for a video block, a first intra-frame prediction signal associated with a first prediction mode (e.g., a regular prediction mode) and a second intra-frame prediction signal associated with a block vector-based prediction mode (e.g., a second prediction mode). The device can generate prediction samples associated with the video block based on the first and second intra-frame prediction signals. The device can encode the video block based on the prediction samples.

[0013] The first intra-frame prediction signal can be associated with a list of most probable modes (MPMs) for the video block. The prediction mode based on the block vector can be either Intra-Template Matching Prediction (IntraTMP) or Intra-Block Copy (IBC).

[0014] The device can determine a first weight associated with an intra-prediction signal associated with a first prediction mode and a second weight associated with an intra-prediction signal associated with a block vector-based prediction mode. Prediction samples associated with video blocks can be generated based on the first weight applied to the first intra-prediction signal and the second weight applied to the second intra-prediction signal. The first and second weights can be determined based on whether the IntraTMP is associated with one or more of a first block and a second block of neighboring video blocks.

[0015] The device may (e.g., when the first intra-frame prediction signal is associated with a list of most probable modes (MPMs) for a video block) obtain an index associated with the MPM list. The device may sort multiple MPM modes in the MPM list based on the corresponding cost associated with each MPM mode. The device may determine a first prediction mode based on the sorted multiple MPM modes and the index. The first intra-frame prediction signal may be obtained based on the first prediction mode.

[0016] The device can obtain multiple block vectors and / or indices. The device can select a block vector from the multiple block vectors (e.g., based on the index). A second prediction signal can be obtained based on the selected block vector.

[0017] The device can select the transform of the video block based on at least one of a first prediction mode and a second intra-frame prediction mode. For example, the transform used for the video block can be one or more of multiple transform selection (MTS), low-frequency non-separable transform (LFNST), or non-separable primary transform (NSPT).

[0018] A video coding method may include obtaining a first intra-frame prediction signal associated with a first prediction mode and a second intra-frame prediction signal associated with a block vector-based prediction mode (e.g., IntraTMP or IBC) for a video block. The method may include generating prediction samples associated with the video block based on the first and second intra-frame prediction signals. The method may include encoding the video block based on the prediction samples.

[0019] The method may include determining a first weight associated with a first intra-frame prediction signal and a second weight associated with a second intra-frame prediction signal. Prediction samples associated with video blocks may be generated based on the first weight applied to the first intra-frame prediction signal and the second weight applied to the second intra-frame prediction signal. The first and / or second weights may be determined based on whether the IntraTMP is associated with one or more of a first block and a second block of neighboring video blocks.

[0020] The method may include selecting a transform for a video block based on at least one of a first intra-frame prediction mode and a second intra-frame prediction mode, wherein the transform for the video block is at least one of multiple transform selection (MTS), low-frequency non-separable transform (LFNST), or non-separable primary transform (NSPT).

[0021] The systems, methods, and means described herein may relate to a decoder. In some examples, the systems, methods, and means described herein may relate to an encoder. In some examples, the systems, methods, and means described herein may relate to a signal (e.g., from an encoder and / or received by a decoder). Video data (e.g., a video bitstream) may include indications of video blocks / current blocks encoded according to intra-intra-CIIP. A computer-readable medium may include instructions for causing one or more processors to perform the methods described herein. A computer program product may include instructions that, when executed by one or more processors, cause one or more processors to perform the methods described herein. Attached Figure Description

[0022] Figure 1A This is a system diagram illustrating an example communication system in which one or more of the disclosed embodiments may be implemented.

[0023] Figure 1B It is shown that, according to the embodiment, it is possible to Figure 1AThe diagram shows a system diagram of an example wireless transmit / receive unit (WTRU) used in the communication system shown.

[0024] Figure 1C It is shown that, according to the embodiment, it is possible to Figure 1A The system diagram shows an example radio access network (RAN) and an example core network (CN) used in the communication system shown.

[0025] Figure 1D It is shown that, according to the embodiment, it is possible to Figure 1A The system diagram shows another example RAN and another example CN used in the communication system shown.

[0026] Figure 2 An example video encoder is shown.

[0027] Figure 3 An example video decoder is shown.

[0028] Figure 4 Examples of systems in which various aspects and examples can be implemented are shown.

[0029] Figures 5A-5B An example of angular pattern division is shown.

[0030] Figures 6A-6D An example of a geometric partitioning mode (GPM) with inter-frame and intra-frame prediction is shown.

[0031] Figure 7 An example of an intra-template matching prediction (intraTMP) search region is shown.

[0032] Figure 8 An example of combined inter-intra-frame prediction (CIIP) and intra-intra-frame prediction is shown.

[0033] Figure 9 An example of CIIP is shown, where the intra-prediction derivation process is replaced by the intraTMP mode.

[0034] Figure 10 An example of CIIP with intra-intra-prediction is shown. Detailed Implementation

[0035] A more detailed understanding can be obtained through the following description, which is given with reference to the accompanying drawings and examples.

[0036] Figure 1AThis diagram illustrates an example communication system 100, in which one or more of the disclosed embodiments can be implemented. Communication system 100 can be a multiple access system providing content such as voice, data, video, messaging, and broadcasting to multiple wireless users. Communication system 100 enables multiple wireless users to access such content by sharing system resources, including wireless bandwidth. For example, communication system 100 can employ one or more channel access methods, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal FDMA (OFDMA), Single Carrier FDMA (SC-FDMA), Zero-Tail Unique Word DFT Spread Spectrum OFDM (ZT-UW-DFT-S-OFDM), Unique Word OFDM (UW-OFDM), Resource Block Filtered OFDM, Filter Bank Multicarrier (FBMC), etc.

[0037] like Figure 1A As shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, Public Switched Telephone Network (PSTN) 108, Internet 110, and other networks 112. However, it should be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, and 102d may be any type of device configured to operate and / or communicate in a wireless environment. For example, WTRUs 102a, 102b, 102c, and 102d (any of which can be referred to as a "station" and / or "STA") can be configured to transmit and / or receive wireless signals and can include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscription-based units, pagers, cellular phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated processing chain environments), consumer electronics devices, devices operating on commercial and / or industrial wireless networks, etc. Any of WTRUs 102a, 102b, 102c, and 102d can be interchangeably referred to as a UE.

[0038] The communication system 100 may also include base station 114a and / or base station 114b. Each of base stations 114a and 114b may be any type of device configured to wirelessly interface with at least one of WTRUs 102a, 102b, 102c, and 102d to facilitate access to one or more communication networks, such as CN 106 / 115, the Internet 110, and / or other networks 112. For example, base stations 114a and 114b may be base transceiver stations (BTS), node B, eNode-B, home node B, home eNode-B, NB, NR node B, site controller, access point (AP), or wireless router. Although base stations 114a and 114b are depicted as single elements, it should be understood that base stations 114a and 114b may include any number of interconnected base stations and / or network elements.

[0039] Base station 114a may be part of RAN 104, and RAN 104 / 113 may also include other base stations and / or network elements (not shown), such as base station controllers (BSCs), radio network controllers (RNCs), relay nodes, etc. Base station 114a and / or base station 114b may be configured to transmit and / or receive radio signals on one or more carrier frequencies, which may be referred to as cells (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage of a specific geographic area, which may be relatively fixed or may change over time. A cell may also be divided into cell sectors. For example, the cell associated with base station 114a may be divided into three sectors. Thus, in one embodiment, base station 114a may include three transceivers, i.e., one transceiver per sector of the cell. In one embodiment, base station 114a may employ multiple-input multiple-output (MIMO) technology and may utilize multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in a desired spatial direction.

[0040] Base stations 114a and 114b can communicate with one or more of WTRUs 102a, 102b, 102c, and 102d via air interface 116. Air interface 116 can be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). Any suitable radio access technology (RAT) can be used to establish air interface 116.

[0041] More specifically, as described above, the communication system 100 can be a multiple access system and can employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, base stations 114a and WTRUs 102a, 102b, and 102c in RAN 104 / 113 can implement wireless technologies such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which can establish air interfaces 115 / 116 / 117 using Wideband CDMA (WCDMA). WCDMA may include communication protocols such as High-Speed ​​Packet Access (HSPA) and / or evolved HSPA (HSPA+). HSPA may include High-Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High-Speed ​​UL Packet Access (HSUPA).

[0042] In one embodiment, base station 114a and WTRUs 102a, 102b, 102c can implement wireless technologies such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which can use Long Term Evolution (LTE) and / or LTE-A Advanced (LTE-A) and / or LTE-A Pro Advanced (LTE-A Pro) to establish air interface 116.

[0043] In one embodiment, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as NR wireless access, which can establish an air interface 116 using a new radio (NR).

[0044] In this embodiment, base station 114a and WTRUs 102a, 102b, and 102c can implement various radio access technologies. For example, base station 114a and WTRUs 102a, 102b, and 102c can jointly implement LTE radio access and NR radio access, for example, using the dual connectivity (DC) principle. Therefore, the air interface utilized by WTRUs 102a, 102b, and 102c can be characterized by various types of radio access technologies and / or transmissions sent to / from various types of base stations (e.g., eNBs and gNBs).

[0045] In other embodiments, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as IEEE 802.11 (i.e., Wi-Fi), IEEE 802.16 (i.e., WiMAX), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Provisional Standard 2000 (IS-2000), Provisional Standard 95 (IS-95), Provisional Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data Rate GSM Evolution (EDGE), GSMEDGE (GERAN), etc.

[0046] For example, Figure 1A Base station 114b can be a wireless router, home node B, home eNode-B, or access point, and can utilize any suitable RAT to facilitate wireless connectivity in a local area, such as commercial locations, homes, vehicles, campuses, industrial facilities, air corridors (e.g., for drone use), roads, etc. In one embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.11 to establish a wireless local area network (WLAN). In one embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, base station 114b and WTRUs 102c, 102d can utilize cellular-based RATs (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish picocells or femtocells. Figure 1A As shown, base station 114b can have a direct connection to Internet 110. Therefore, it is not required that base station 114b access Internet 110 via CN 106 / 115.

[0047] RAN 104 / 113 can communicate with CN 106 / 115, which can be any type of network configured to provide voice, data, application, and / or Voice over Internet Protocol (VoIP) services to one or more of WTRUs 102a, 102b, 102c, and 102d. Data can have different Quality of Service (QoS) requirements, such as different throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. CN 106 / 115 can provide call control, billing services, location-based services, prepaid calling, internet connectivity, video distribution, and / or perform advanced security functions such as user authentication. Although in Figure 1AAlthough not shown, it should be understood that RAN104 / 113 and / or CN 106 / 115 can communicate directly or indirectly with other RANs using the same RAT as RAN 104 / 113 or a different RAT. For example, in addition to connecting to RAN 104 / 113, which may utilize NR radio technology, CN 106 / 115 can also communicate with another RAN (not shown) using GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technology.

[0048] CN 106 / 115 can also serve as a gateway for WTRU 102a, 102b, 102c, 102d to access PSTN 108, the Internet 110, and / or other networks 112. PSTN 108 may include a circuit-switched telephone network providing Common Old-Style Telephone Service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices using common communication protocols such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and / or Internet Protocol (IP) from the TCP / IP Internet Protocol suite. Network 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, network 112 may include another CN connected to one or more RANs, which may use the same RAT as RAN 104 / 113 or a different RAT.

[0049] Some or all of the WTRUs 102a, 102b, 102c, and 102d in communication system 100 may include multimodal capabilities (e.g., WTRUs 102a, 102b, 102c, and 102d may include multiple transceivers for communicating with different wireless networks via different wireless links). For example, Figure 1A The WTRU 102c shown can be configured to communicate with base station 114a, which can use cellular-based radio technology, and to communicate with base station 114b, which can use IEEE 802 radio technology.

[0050] Figure 1B This shows a system diagram of an example WTRU 102. (See diagram below.) Figure 1B As shown, among other things, WTRU 102 may include a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power supply 134, a Global Positioning System (GPS) chipset 136, and / or other peripherals 138. It should be understood that WTRU 102 may include any sub-combination of the foregoing elements while remaining consistent with the embodiments.

[0051] Processor 118 may be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. As described above, processor 118 may include multiple processors. Processor 118 may perform signal encoding, data processing, power control, input / output processing, and / or any other functions that enable WTRU 102 to operate in a wireless environment. Processor 118 may be coupled to transceiver 120, and transceiver 120 may be coupled to transmitting / receiving element 122. Although Figure 1B The processor 118 and transceiver 120 are depicted as separate components, but it should be understood that the processor 118 and transceiver 120 may be integrated together in an electronic package or chip.

[0052] The transmitting / receiving element 122 can be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) via the air interface 116. For example, in one embodiment, the transmitting / receiving element 122 can be an antenna configured to transmit and / or receive RF signals. In one embodiment, the transmitting / receiving element 122 can be, for example, a transmitter / detector configured to transmit and / or receive IR, UV, or visible light signals. In yet another embodiment, the transmitting / receiving element 122 can be configured to transmit and / or receive both RF and optical signals. It should be understood that the transmitting / receiving element 122 can be configured to transmit and / or receive any combination of wireless signals.

[0053] Although the transmitting / receiving element 122 is in Figure 1B While depicted as a single element, WTRU 102 may include any number of transmit / receive elements 122. More specifically, WTRU 102 may employ MIMO technology. Thus, in one embodiment, WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals via air interface 116.

[0054] Transceiver 120 can be configured to modulate signals transmitted by transmit / receive element 122 and demodulate signals received by transmit / receive element 122. As described above, WTRU 102 can have multimode capability. Therefore, transceiver 120 can include multiple transceivers to enable WTRU 102 to communicate via multiple RATs, such as NR and IEEE 802.11.

[0055] The processor 118 of WTRU 102 can be coupled to a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) unit or an organic light-emitting diode (OLED) display unit) and can receive user input data therefrom. The processor 118 can also output user data to the speaker / microphone 124, keypad 126, and / or display / touchpad 128. Furthermore, the processor 118 can access and store information from any type of suitable memory, such as non-removable memory 130 and / or removable memory 132. Non-removable memory 130 may include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. Removable memory 132 may include a user identity module (SIM) card, a memory stick, a secure digital storage (SD) card, etc. In other embodiments, the processor 118 can access and store information from memory that is not physically located on WTRU 102 (such as on a server or home computer (not shown)).

[0056] The processor 118 may receive power from the power supply 134 and may be configured to distribute and / or control power to other components in the WTRU 102. The power supply 134 may be any suitable device for powering the WTRU 102. For example, the power supply 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0057] The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) about the current location of the WTRU 102. In addition to, or instead of, information from the GPS chipset 136, the WTRU 102 may receive location information from base stations (e.g., base stations 114a, 114b) via the air interface 116, and / or determine its location based on the timing of signals received from two or more nearby base stations. It should be understood that the WTRU 102 may acquire location information using any suitable location determination method while remaining consistent with the embodiments.

[0058] The processor 118 may also be coupled to other peripherals 138, which may include one or more software and / or hardware modules providing additional features, functions, and / or wired or wireless connectivity. For example, peripherals 138 may include accelerometers, electronic compasses, satellite transceivers, digital cameras (for photos and / or videos), Universal Serial Bus (USB) ports, vibration devices, television transceivers, hands-free headsets, Bluetooth® modules, FM radio units, digital music players, media players, video game player modules, internet browsers, virtual reality and / or augmented reality (VR / AR) devices, activity trackers, etc. Peripherals 138 may include one or more sensors, which may be one or more of the following: gyroscopes, accelerometers, Hall effect sensors, magnetometers, orientation sensors, proximity sensors, temperature sensors, time sensors, geolocation sensors; altimeters, light sensors, touch sensors, magnetometers, barometers, attitude sensors, biometric sensors, and / or humidity sensors.

[0059] WTRU 102 may include a full-duplex radio for which the transmission and reception of some or all signals (e.g., associated with a specific subframe of both UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous. The full-duplex radio may include an interference management unit to reduce and / or substantially eliminate self-interference through hardware (e.g., chokes) or through signal processing by a processor (e.g., a separate processor (not shown) or through processor 118). In one embodiment, WTRU 102 may include a half-duplex radio for which the transmission and reception of some or all signals (e.g., associated with a specific subframe of either UL (e.g., for transmission) or downlink (e.g., for reception)) may be concurrent and / or simultaneous.

[0060] Figure 1C This is a system diagram illustrating RAN 104 and CN 106 according to an embodiment. As described above, RAN 104 can communicate with WTRUs 102a, 102b, and 102c via air interface 116 using E-UTRA radio technology. RAN 104 can also communicate with CN 106.

[0061] RAN 104 may include eNode-B 160a, 160b, and 160c; however, it should be understood that RAN 104 may include any number of eNode-Bs while remaining consistent with the embodiments. Each eNode-B 160a, 160b, and 160c may include one or more transceivers for communicating with WTRU 102a, 102b, and 102c via air interface 116. In one embodiment, eNode-B 160a, 160b, and 160c may implement MIMO technology. Therefore, for example, eNode-B 160a may use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 102a.

[0062] Each of the eNode-B 160a, 160b, and 160c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, etc. Figure 1C As shown, eNode-B 160a, 160b, and 160c can communicate with each other via the X2 interface.

[0063] Figure 1C The CN 106 shown may include a Mobility Management Entity (MME) 162, a Serving Gateway (SGW) 164, and a Packet Data Network (PDN) Gateway (or PGW) 166. While each of the foregoing elements is described as part of CN 106, it should be understood that any of these elements may be owned and / or operated by an entity other than a CN operator.

[0064] The MME 162 can connect to each of the eNode-Bs 162a, 162b, and 162c in RAN 104 via the S1 interface and can act as a control node. For example, the MME 162 can be responsible for authenticating users of WTRUs 102a, 102b, and 102c, bearer activation / deactivation, selecting a specific serving gateway during the initial attachment of WTRUs 102a, 102b, and 102c, etc. The MME 162 can provide control plane functions for handover between RAN 104 and other RANs (not shown) employing other radio technologies such as GSM and / or WCDMA.

[0065] The SGW 164 can connect to each of the eNode-Bs 160a, 160b, and 160c in RAN 104 via the S1 interface. The SGW 164 can typically route and forward user data packets to / from WTRUs 102a, 102b, and 102c. The SGW 164 can perform other functions such as anchoring the user plane during inter-eNode-B handover, triggering paging when DL data is available for WTRUs 102a, 102b, and 102c, and managing and storing the context of WTRUs 102a, 102b, and 102c.

[0066] SGW 164 can connect to PGW 166, which can provide WTRU 102a, 102b, 102c with access to packet-switched networks such as Internet 110, so as to facilitate communication between WTRU 102a, 102b, 102c and IP-enabled devices.

[0067] CN 106 can facilitate communication with other networks. For example, CN 106 can provide WTRU 102a, 102b, 102c with access to a circuit-switched network such as PSTN 108, facilitating communication between WTRU 102a, 102b, 102c and traditional landline communication equipment. For example, CN 106 may include, or be able to communicate with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that acts as an interface between CN 106 and PSTN 108. Furthermore, CN 106 can provide WTRU 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0068] Despite WTRU in Figure 1A-1D While described as a wireless terminal, it is conceivable that in some representative embodiments, such a terminal may use (e.g., temporarily or permanently) a wired communication interface with a communication network.

[0069] In a representative embodiment, another network 112 may be a WLAN.

[0070] In Infrastructure Basic Services Set (BSS) mode, a WLAN may have an access point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP may have access or an interface to a distributed system (DS) or another type of wired / wireless network that transmits traffic to and / or out of the BSS. Traffic originating outside the BSS destined for a STA can be delivered to the AP. Traffic from a STA to a destination outside the BSS can be sent to the AP for delivery to the appropriate destination. For example, traffic between STAs within the BSS can be transmitted via the AP, where the source STA can send traffic to the AP, and the AP can deliver traffic to the destination STA. Traffic between STAs within the BSS can be considered and / or referred to as peer-to-peer traffic. Peer-to-peer traffic can be transmitted between source and destination STAs (e.g., directly between them) using Direct Link Establishment (DLS). In some representative embodiments, the DLS may use 802.11e DLS or 802.11z Tunneled DLS (TDLS). A WLAN using Standalone BSS (IBSS) mode may not have an access point (AP), and STAs within the IBSS or using the IBSS (e.g., all STAs) can communicate directly with each other. The IBSS communication mode is sometimes referred to here as an "ad-hoc" communication mode.

[0071] When operating in 802.11ac infrastructure mode or a similar mode, the AP can transmit beacons on a fixed channel, such as the primary channel. The primary channel can be of a fixed width (e.g., a wide bandwidth of 20 MHz) or dynamically set via signaling. The primary channel can be the operating channel of the BSS and can be used by the STA to establish a connection with the AP. In some representative embodiments, Carrier Sense Multiple Access with Collision Avoidance (CSMA / CA) can be implemented, for example in an 802.11 system. For CSMA / CA, the STAs including the AP (e.g., each STA) can sense the primary channel. If a particular STA senses / detects and / or determines that the primary channel is busy, that particular STA can back off. A single STA (e.g., only one station) can transmit at any given time within a given BSS.

[0072] High-throughput (HT) STAs can communicate using a 40 MHz wide channel, for example, by combining a primary 20 MHz channel with adjacent or non-adjacent 20 MHz channels.

[0073] Very High Throughput (VHT) STAs can support 20 MHz, 40 MHz, 80 MHz, and / or 160 MHz wide channels. 40 MHz and / or 80 MHz channels can be formed by combining consecutive 20 MHz channels. A 160 MHz channel can be formed by combining eight consecutive 20 MHz channels, or by combining two non-consecutive 80 MHz channels, which can be referred to as an 80+80 configuration. For the 80+80 configuration, after channel coding, the data can be divided into two streams by a segment parser. Each stream can be processed separately using Inverse Fast Fourier Transform (IFFT) and time-domain processing. The streams can be mapped onto the two 80 MHz channels, and the data can be transmitted by the transmitting STA. At the receiver of the receiving STA, the operation of the 80+80 configuration can be reversed, and the combined data can be sent to the Media Access Control (MAC).

[0074] 802.11af and 802.11ah support sub-1 GHz operating modes. The channel operating bandwidth and carrier are reduced in 802.11af and 802.11ah compared to those used in 802.11n and 802.11ac. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV whitespace (TVWS) spectrum, while 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to representative embodiments, 802.11ah can support metering-type control / machine-type communications, such as MTC devices in macro coverage areas. MTC devices may have certain capabilities, such as limited capabilities, including support (e.g., only support) certain and / or limited bandwidths. MTC devices may include batteries with a battery life exceeding a threshold (e.g., to maintain very long battery life).

[0075] WLAN systems that can support multiple channels and channel bandwidths (such as 802.11n, 802.11ac, 802.11af, and 802.11ah) include a channel that can be designated as the primary channel. The primary channel can have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel can be set and / or limited by the STAs operating in the BSS that support the minimum bandwidth operating mode. In the example of 802.11ah, for STAs that support (e.g., only support) the 1 MHz mode (e.g., MTC type devices), the primary channel can be 1 MHz wide, even if the AP and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and / or other channel bandwidth operating modes. Carrier Sense and / or Network Allocation Vector (NAV) settings may depend on the status of the primary channel. If the primary channel is busy transmitting to the AP, for example due to an STA (only supporting the 1 MHz operating mode), the entire available band can be considered busy, even if most of the band remains idle and can be available.

[0076] In the United States, the available frequency band for 802.11ah is from 902 MHz to 928 MHz. In South Korea, the available frequency band is from 917.5 MHz to 923.5 MHz. In Japan, the available frequency band is from 916.5 MHz to 927.5 MHz. The total available bandwidth for 802.11ah is 6 MHz to 26 MHz, depending on the country code.

[0077] Figure 1D This is a system diagram illustrating RAN 113 and CN 115 according to an embodiment. As described above, RAN 113 can communicate with WTRUs 102a, 102b, and 102c via air interface 116 using NR radio technology. RAN 113 can also communicate with CN 115.

[0078] RAN 113 may include gNBs 180a, 180b, and 180c; however, it should be understood that RAN 113 may include any number of gNBs while remaining consistent with the embodiments. gNBs 180a, 180b, and 180c may each include one or more transceivers for communicating with WTRUs 102a, 102b, and 102c via air interface 116. In one embodiment, gNBs 180a, 180b, and 180c may implement MIMO technology. For example, gNBs 180a and 108b may utilize beamforming to transmit signals to and / or receive signals from gNBs 180a, 180b, and 180c. Therefore, for example, gNB 180a may use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 102a. In one embodiment, gNBs 180a, 180b, and 180c may implement carrier aggregation technology. For example, gNB 180a can transmit multiple component carriers (not shown) to WTRU 102a. A subset of these component carriers may be on unlicensed spectrum, while the remaining component carriers may be on licensed spectrum. In one embodiment, gNBs 180a, 180b, and 180c may implement Coordinated Multipoint (CoMP) technology. For example, WTRU 102a may receive coordinated transmissions from gNBs 180a and 180b (and / or gNB 180c).

[0079] WTRUs 102a, 102b, and 102c can communicate with gNB180a, 180b, and 180c using transmissions associated with scalable digitization. For example, the OFDM symbol spacing and / or OFDM subcarrier spacing can differ for different transmissions, different cells, and / or different portions of the radio transmission spectrum. WTRUs 102a, 102b, and 102c can communicate with gNB180a, 180b, and 180c using subframes or transmission time intervals (TTIs) of various or scalable lengths (e.g., containing a variable number of OFDM symbols and / or a continuously variable absolute time).

[0080] gNB180a, 180b, and 180c can be configured to communicate with WTRU 102a, 102b, and 102c in standalone and / or non-standalone configurations. In standalone configuration, WTRU 102a, 102b, and 102c can communicate with gNB180a, 180b, and 180c without accessing other RANs (e.g., eNode-B 160a, 160b, and 160c). In standalone configuration, WTRU 102a, 102b, and 102c can utilize one or more of gNB180a, 180b, and 180c as mobility anchors. In standalone configuration, WTRU 102a, 102b, and 102c can communicate with gNB180a, 180b, and 180c using signals in unlicensed frequency bands. In a non-standalone configuration, WTRUs 102a, 102b, and 102c can communicate / connect with gNBs 180a, 180b, and 180c, while also communicating / connecting with another RAN such as eNode-Bs 160a, 160b, and 160c. For example, WTRUs 102a, 102b, and 102c can implement DC principles to communicate substantially simultaneously with one or more gNBs 180a, 180b, and 180c, as well as one or more eNode-Bs 160a, 160b, and 160c. In a non-standalone configuration, eNode-Bs 160a, 160b, and 160c can act as mobility anchors for WTRUs 102a, 102b, and 102c, and gNBs 180a, 180b, and 180c can provide additional coverage and / or throughput for serving WTRUs 102a, 102b, and 102c.

[0081] Each of gNB180a, 180b, and 180c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, network slicing support, dual connectivity, interoperability between NR and E-UTRA, routing user plane data to User Plane Functions (UPF) 184a and 184b, and routing control plane information to Access and Mobility Management Functions (AMF) 182a and 182b, etc. Figure 1D As shown, gNB180a, 180b, and 180c can communicate with each other via the Xn interface.

[0082] Figure 1DThe CN 115 shown may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one Session Management Function (SMF) 183a, 183b, and may include a Data Network (DN) 185a, 185b. While each of the foregoing elements is described as part of the CN 115, it should be understood that any of these elements may be owned and / or operated by an entity other than a CN operator.

[0083] AMF 182a and 182b can connect to one or more gNBs 180a, 180b, and 180c in RAN 113 via the N2 interface and can act as control nodes. For example, AMF 182a and 182b can be responsible for authenticating users of WTRU 102a, 102b, and 102c, supporting network slicing (e.g., handling different PDU sessions with different requirements), selecting specific SMF 183a and 183b, managing registration areas, terminating NAS signaling, mobility management, and so on. AMF 182a and 182b can use network slicing to customize CN support for WTRU 102a, 102b, and 102c based on the service types used by WTRU 102a, 102b, and 102c. For example, different network slices can be established for different use cases, such as services relying on Ultra Reliable Low Latency (URLLC) access, services relying on Enhanced Massive Mobile Broadband (eMBB) access, and services for Machine-Type Communication (MTC) access. AMF 162 can provide control plane functions for handover between RAN 113 and other RANs (not shown) employing other radio technologies such as LTE, LTE-A, LTE-A Pro and / or non-3GPP access technologies such as WiFi.

[0084] SMFs 183a and 183b can connect to AMFs 182a and 182b in CN 115 via the N11 interface. SMFs 183a and 183b can also connect to UPFs 184a and 184b in CN 115 via the N4 interface. SMFs 183a and 183b can select and control UPFs 184a and 184b, and configure the routing of services through UPFs 184a and 184b. SMFs 183a and 183b can perform other functions, such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notifications. PDU session types can be IP-based, non-IP-based, Ethernet-based, etc.

[0085] UPF 184a and 184b can be connected to one or more gNB180a, 180b, and 180c in RAN 113 via the N3 interface. This interface provides WTRU 102a, 102b, and 102c with access to packet-switched networks (such as Internet 110) to facilitate communication between WTRU 102a, 102b, 102c and IP-enabled devices. UPF 184 and 184b can perform other functions such as routing and forwarding packets, enforcing user plane policies, supporting multi-destination PDU sessions, handling user plane QoS, buffering downlink packets, and providing mobility anchoring.

[0086] CN 115 can facilitate communication with other networks. For example, CN 115 may include, or be able to communicate with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that acts as an interface between CN 115 and PSTN 108. Furthermore, CN 115 can provide WTRUs 102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, WTRUs 102a, 102b, and 102c may be connected to the local data network (DN) 185a and 185b via the N3 interface to UPFs 184a and 184b and the N6 interface between UPFs 184a and 184b and DNs 185a and 185b.

[0087] Given Figure 1A-1D as well as Figure 1A-1D The corresponding descriptions herein refer to one or more of the following functions: WTRU 102a-d, Base Station 114a-b, eNode-B 160a-c, MME 162, SGW 164, PGW166, gNB 180 ac, AMF 182 ab, UPF 184a-b, SMF 183 ab, DN185a-b, and / or any other device described herein. These functions can be performed by one or more emulation devices (not shown). An emulation device can be one or more devices configured to emulate the functions described herein. For example, an emulation device can be used to test other devices and / or simulate network and / or WTRU functions.

[0088] Simulation devices can be designed to perform tests on one or more other devices in laboratory and / or carrier network environments. For example, one or more simulation devices can perform one or more or all of their functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices within the communication network. One or more simulation devices can perform one or more or all of their functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. For testing purposes, simulation devices can be directly coupled to another device and / or can use over-the-air wireless communication to perform tests.

[0089] One or more simulation devices may perform one or more functions, including all functions, rather than being implemented / deployed as part of a wired and / or wireless communication network. For example, simulation devices may be used to test test scenarios in laboratory and / or non-deployment (e.g., testing) wired and / or wireless communication networks to implement the testing of one or more components. One or more simulation devices may be test devices. Simulation devices may transmit and / or receive data using direct RF coupling and / or wireless communication via RF circuitry (e.g., which may include one or more antennas).

[0090] This application describes multiple aspects, including tools, features, examples, models, methods, etc. Many of these aspects are described in detail, and often in a manner that may sound restrictive, at least to illustrate individual characteristics. However, this is for the purpose of clarity and does not limit the application or scope of those aspects. In fact, all the different aspects can be combined and interchanged to provide other aspects. Furthermore, aspects can also be combined and interchanged with aspects described in earlier applications.

[0091] The aspects described and envisioned in this application can be implemented in many different forms. Figure 5 described herein- Figure 9 Some examples can be provided, but other examples are also expected. Figure 5- Figure 9 The discussion does not limit the scope of implementation. At least one aspect generally relates to video encoding and decoding, and at least one other aspect generally relates to the transmission of generated or encoded bitstreams. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions thereon stored thereon for encoding or decoding video data according to any of the described methods, and / or computer-readable storage media having bitstreams generated according to any of the described methods stored thereon.

[0092] In this application, the terms “reconstruction” and “decoding” are used interchangeably, the terms “pixel” and “sample” are used interchangeably, and the terms “image”, “picture” and “frame” are used interchangeably.

[0093] This document describes various methods, and each method includes one or more steps or actions for implementing the method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined. Additionally, terms such as "first," "second," etc., can be used in various examples to modify elements, components, steps, operations, etc., such as, for example, "first decoding" and "second decoding." Unless specifically required, the use of such terms does not imply a reordering of the modified operations. Thus, in this example, the first decoding does not need to be performed before the second decoding and can occur, for example, before, during, or in a time period overlapping with the second decoding.

[0094] The various methods and other aspects described in this application can be used to modify, for example... Figure 2 and Figure 3 The illustrated video encoder 200 and decoder 300 modules are, for example, decoding modules. Furthermore, the subject matter disclosed herein can be applied to, for example, any type, format, or version of video encoding, whether described in standards or recommendations, whether pre-existing or future-developed, and to extensions applied to any such standards and recommendations. Unless otherwise stated or technically excluded, the aspects described in this application may be used individually or in combination.

[0095] Various numerical values, such as bits, bit depth, etc., are used in the examples described in this application. These and other specific values ​​are for illustrative purposes only, and the aspects described are not limited to these specific values.

[0096] Figure 2 This is a diagram illustrating an example video encoder. Variations of the example encoder 200 are envisioned, but for clarity, encoder 200 is described below without describing all anticipated variations.

[0097] Before being encoded, the video sequence may undergo pre-coding processing (201), such as applying color transformations to the input color image (e.g., a conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing remapping of the input image components to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization with one of the color components). Metadata may be associated with the pre-processing and attached to the bitstream.

[0098] In encoder 200, the image is encoded by encoder elements as described below. The image to be encoded is partitioned (202) and processed in units, for example, coding units (CUs). Each unit is encoded using, for example, intra-frame or inter-frame modes. When a unit is encoded in intra-frame mode, it performs intra-frame prediction (260). In inter-frame mode, motion estimation (275) and compensation (270) are performed. The encoder determines (205) which of the intra-frame or inter-frame modes to use to encode the unit and indicates the intra-frame / inter-frame decision by, for example, a prediction mode flag. For example, the prediction residual is calculated by subtracting (210) the prediction block from the original image block.

[0099] The predicted residual is then transformed (225) and quantized (230). The quantized transform coefficients, along with the motion vector and other syntax elements, are entropy encoded (245) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass the transform and quantization, i.e., the residual is directly encoded without applying the transform or quantization process.

[0100] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (255) to reconstruct the image block. A loop filter (265) is applied to the reconstructed image to perform, for example, deblocking / SAO (sample adaptive offset) filtering, thereby reducing coding artifacts. The filtered image is stored in a reference image buffer (280).

[0101] Figure 3 This is a diagram illustrating an example video decoder. In the example decoder 300, the bitstream is decoded by decoder elements, as described below. The video decoder 300 typically performs the same operations as... Figure 2 The encoding process described herein is the opposite of the decoding process. Encoder 200 typically also performs video decoding as part of the encoded video data.

[0102] Specifically, the decoder's input includes a video bitstream, which can be generated by the video encoder 200. The bitstream is first entropy decoded (330) to obtain transform coefficients, motion vectors, and other encoded information. Picture partitioning information indicates how the picture is partitioned. Therefore, the decoder can partition (335) the picture based on the decoded picture partitioning information. The transform coefficients are dequantized (340) and inverse transformed (350) to decode the prediction residuals. The decoded prediction residuals and prediction blocks are combined (355) to reconstruct the image blocks. The prediction blocks can be obtained (370) from intra-frame prediction (360) or motion-compensated prediction (i.e., inter-frame prediction) (375). A loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (380).

[0103] The decoded image may undergo further post-decoding processing (385), such as inverse color transformation (e.g., a conversion from YCbCr4:2:0 to RGB 4:4:4) or inverse remapping, which is the inverse of the remapping process performed in the pre-encoding process (201). The post-decoding process may use metadata derived in the pre-encoding process and signaled in the bitstream. In the example, the decoded image (e.g., after applying an in-loop filter (365) and / or after post-decoding processing (385), if post-decoding processing is used) may be sent to a display device for presentation to the user.

[0104] Figure 4 This is a block diagram of an example system in which various aspects and examples described herein can be implemented. System 400 can be embodied as a device including the various components described below and configured to perform one or more aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 400 can be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one example, the processing and encoder / decoder elements of system 400 are distributed across multiple ICs and / or discrete components. In various examples, system 400 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various examples, system 400 is configured to implement one or more aspects described in this document.

[0105] System 400 includes at least one processor 410 configured to execute instructions loaded therein for implementing various aspects, such as those described in this application. Processor 410 may include embedded memory, input / output interfaces, and various other circuit systems as known in the art. System 400 includes at least one memory 420 (e.g., a volatile memory device and / or a non-volatile memory device). System 400 includes a storage device 440, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 440 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0106] System 400 includes an encoder / decoder module 430 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 430 may include its own processor and memory. The encoder / decoder module 430 represents one or more modules that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both encoding and decoding modules. Additionally, the encoder / decoder module 430 may be implemented as a separate element of system 400, or it may be incorporated within processor 410 as a combination of hardware and software as known to those skilled in the art.

[0107] Program code to be loaded onto processor 410 or encoder / decoder 430 to execute the various aspects described in this document may be stored in storage device 440 and subsequently loaded onto memory 420 for execution by processor 410. According to various examples, one or more of processor 410, memory 420, storage device 440, and encoder / decoder module 430 may store one or more various items during the execution of the processes described in this document. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0108] In some examples, the memory within processor 410 and / or encoder / decoder module 430 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other examples, external memory (e.g., the processing device may be processor 410 or encoder / decoder module 430) is used for one or more of these functions. External memory may be memory 420 and / or storage device 440, such as volatile memory and / or non-volatile flash memory. In several examples, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one example, fast external volatile memory such as RAM is used as working memory for video encoding and decoding operations.

[0109] As indicated in box 445, inputs to the components of system 400 can be provided through various input devices. Such input devices include, but are not limited to, (i) a radio frequency (RF) section that receives, for example, RF signals transmitted over the air by a broadcaster, (ii) component (COMP) input terminals (or sets of COMP input terminals), (iii) universal serial bus (USB) input terminals, and / or (iv) high-definition multimedia interface (HDMI) input terminals. Figure 4 Other examples not shown include composite video.

[0110] In various examples, the input device of block 405 has associated corresponding input processing elements, as known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting a signal band to a band), (ii) down-converting the selected signal, (iii) further band-limiting to a narrower band to select, for example, a signal band that may be referred to as a channel in some examples, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and / or (vi) demultiplexing to select a desired data packet stream. The RF section of various examples includes one or more elements performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners performing various functions among these functions, such as down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one set-top box example, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to the desired frequency band. Various examples rearrange the order of the aforementioned (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as, for example, inserting amplifiers and analog-to-digital converters. In various examples, the RF section includes an antenna.

[0111] USB and / or HDMI terminals may include corresponding interface processors for connecting system 400 to other electronic devices across USB and / or HDMI connections. It should be understood that various aspects of input processing—such as Reed-Solomon error correction—may be implemented as needed, for example, within a separate input processing IC or within processor 410. Similarly, as needed, various aspects of USB or HDMI interface processing may be implemented within a separate interface IC or within processor 410. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, processor 410 and encoder / decoder 430, which operate in combination with memory and storage elements to process the data streams as needed for presentation on the output device.

[0112] Various components of system 400 can be provided within an integrated housing, in which the various components can be interconnected and transmit data therebetween using suitable connection means 425, such as internal buses as known in the art, including internal IC (I2C) buses, wiring and printed circuit boards.

[0113] System 400 includes a communication interface 450 that enables communication with other devices via a communication channel 460. The communication interface 450 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 460. The communication interface 450 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 460 may be implemented within, for example, wired and / or wireless media.

[0114] In various examples, wireless networks, such as Wi-Fi networks (e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers)), are used to stream or otherwise provide data to system 400. In these examples, the Wi-Fi signal is received via a communication channel 460 and a communication interface 450 adapted for Wi-Fi communication. The communication channel 460 in these examples is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other examples use a set-top box to provide streaming data to system 400, delivering data via an HDMI connection to input block 445. Still other examples use an RF connection to input block 445 to provide streaming data to system 400. As indicated above, various examples provide data in a non-streaming manner. Additionally, various examples use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth® networks.

[0115] System 400 can provide output signals to various output devices, including a display 475, a speaker 485, and other peripheral devices 495. Various examples of the display 475 include one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a flexible display, and / or a foldable display. The display 475 can be used in televisions, tablet devices, laptop computers, telephones (mobile phones), or other devices. The display 475 can also be integrated with other components (e.g., as in a smartphone) or standalone (e.g., an external monitor for a laptop computer). In various examples, other peripheral devices 495 include one or more of a standalone digital video disc (or digital universal disc) (DVD, both terms), a disc player, a stereo system, and / or a lighting system. Various examples utilize one or more peripheral devices 495 that provide functionality based on the output of system 400. For example, a disc player performs the function of playing the output of system 400.

[0116] In various examples, signaling is used to communicate control signals between system 400 and display 475, speaker 485, or other peripheral devices 495. This signaling may include AV.Link, Consumer Electronics Control (CEC), or other communication protocols enabling device-to-device control with or without user intervention. Output devices may be communicatively coupled to system 400 via dedicated connections through corresponding interfaces 470, 480, and 490. Alternatively, output devices may be connected to system 400 via communication interface 450 using communication channel 460. Display 475 and speaker 485 may be integrated into a single unit with other components of system 400 in electronic devices, such as, for example, a television set. In various examples, display interface 470 includes a display driver, such as, for example, a timing controller (TCon) chip.

[0117] For example, if the RF portion of input 445 is part of a separate set-top box, then display 475 and speaker 485 can alternatively be separated from one or more other components. In various examples where display 475 and speaker 485 are external components, the output signal can be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0118] The example can be implemented by computer software implemented by processor 410, or by hardware, or by a combination of hardware and software. As a non-limiting example, the example can be implemented by one or more integrated circuits. Memory 420 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology, such as optical storage devices, magnetic storage devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. As a non-limiting example, processor 410 can be of any type suitable for the technical environment and can comprise one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0119] Various implementations involve decoding. As used herein, “decoding” can include, for example, performing all or part of a process on a received encoded sequence to produce a final output suitable for display. In various examples, such a process includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various examples, such a process may also, or alternatively, include processes performed by a decoder of the various implementations described herein, such as obtaining a first intra-prediction signal associated with a first intra-prediction mode for a block; obtaining a second intra-prediction signal associated with a second intra-prediction mode for the block; generating prediction samples associated with the block by determining a first weight associated with the first intra-prediction signal and a second weight associated with the second intra-prediction signal, wherein the first and second weights are determined by a first neighboring block prediction mode and a second neighboring block prediction mode; and decoding the block based on the prediction samples, etc.

[0120] As another example, in one example, "decoding" refers only to entropy decoding; in another example, "decoding" refers only to differential decoding; and in yet another example, "decoding" refers to a combination of entropy decoding and differential decoding. It will be clear, and is considered well understood by those skilled in the art, whether the phrase "decoding process" is intended to refer specifically to a subset of operations or to a broader decoding process, based on the specific context of the description.

[0121] Various implementations involve encoding. In a manner similar to the discussion above regarding “decoding,” “encoding,” as used herein, can include, for example, all or part of a process performed on an input video sequence to produce an encoded bitstream. In various examples, such a process includes one or more processes typically performed by an encoder, such as partitioning, differential coding, transform, quantization, and entropy coding. In various examples, such a process may also, or alternatively, include processes performed by an encoder of various implementations described herein, such as obtaining a first intra-prediction signal associated with a first intra-prediction mode for a block; obtaining a second intra-prediction signal associated with a second intra-prediction mode for the block; generating prediction samples associated with the block by determining a first weight associated with the first intra-prediction signal and a second weight associated with the second intra-prediction signal, wherein the first and second weights are determined by a first neighboring block prediction mode and a second neighboring block prediction mode; deriving residual samples associated with the block based on the prediction samples; and encoding the block based on the derived residual samples, etc.

[0122] As another example, in one example, "encoding" refers only to entropy encoding; in another example, "encoding" refers only to differential encoding; and in yet another example, "encoding" refers to a combination of differential and entropy encoding. It will be clear, and is considered well understood by those skilled in the art, whether the phrase "encoding process" is intended to refer specifically to a subset of operations or to a broader encoding process, depending on the context of the specific description.

[0123] When a diagram is presented as a flowchart, it should be understood that it also provides a block diagram of the corresponding device. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flowchart of the corresponding method / process.

[0124] The implementations and aspects described herein can be implemented, for example, in methods or processes, apparatuses, software programs, data streams, or signals. Even if discussed only in the context of a single implementation (e.g., discussed only as a method), implementations of the discussed features can be implemented in other forms (e.g., apparatuses or programs). Apparatuses can be implemented, for example, in suitable hardware, software, and firmware. The method can be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as, for example, computers, cellular phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0125] References to “an example” or “an instance” or “an implementation” or “an implementation”, as well as their other variations, imply that the specific features, structures, characteristics, etc., described in connection with that example are included in at least one example. Therefore, the appearance of the phrase “in an example” or “in one example” or “in one implementation” or “in one implementation”, and any other variations appearing in various places throughout the application, do not necessarily all refer to the same example.

[0126] Additionally, this application may refer to "determining" various information pieces. Determining information may include, for example, one or more of estimation information, calculation information, prediction information, or information retrieved from memory. Obtaining may include receiving, retrieving, constructing, generating, and / or determining.

[0127] Furthermore, this application may refer to "accessing" various information chips. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or one or more of these.

[0128] Additionally, this application may refer to "receiving" various pieces of information. Like "accessing," receiving is intended to be a broad term. Receiving information may include, for example, accessing information or retrieving information (e.g., from memory) one or more of the following: Furthermore, "receiving" is generally referred to in one way or another during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0129] It will be understood that, for example, in the cases of “A / B,” “A and / or B,” and “at least one of A and B,” any of the following uses of “ / ,” “and / or,” and “at least one of…” are intended to include selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As another example, in the cases of “A, B, and / or C” and “at least one of A, B, and C,” such wording is intended to include selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to a large number of listed items.

[0130] Furthermore, as used herein, the word “signal” specifically refers, among other things, to instructing the corresponding decoder to do something. Encoder signals may include, for example, encoding functions on the input of a block using a precision factor. Thus, in the example, the same parameters are used on both the encoder and decoder sides. Therefore, for example, the encoder may transmit (explicitly signal) specific parameters to the decoder so that the decoder can use the same specific parameters. Conversely, if the decoder already has specific parameters as well as other parameters, signaling without transmission (implicit signaling) can simply allow the decoder to know and select specific parameters. Bit savings are achieved in various examples by avoiding the transmission of any actual functionality. It will be understood that signaling can be implemented in many ways. For example, in various examples, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the verb form of the word “signal” has been mentioned above, the word “signal” can be used (e.g., also) as a noun in this text.

[0131] As will be apparent to those skilled in the art, the implementation can generate various signals, which are formatted to carry information, for example, that can be stored or transmitted. This information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, the signal may be formatted to carry a bitstream of the described example. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal can be transmitted via a variety of different wired or wireless links. The signal may be stored on, or accessed or received from, a processor-readable medium.

[0132] Numerous examples are described herein. Features of the examples may be provided individually or in any combination across various claim classes and types. Furthermore, examples may include one or more of the features, devices, or aspects described herein, individually or in any combination across various claim classes and types. For example, the features described herein may be implemented in a bitstream or signal including information generated as described herein. This information may allow a decoder to decode the bitstream, an encoder according to any of the described embodiments, a bitstream, and / or a decoder. For example, the features described herein may be implemented by creating and / or transmitting and / or receiving and / or decoding a bitstream or signal. For example, the features described herein may be implemented as a method, process, apparatus, medium storing instructions, medium storing data, or signal. For example, the features described herein may be implemented by a television, set-top box, mobile phone, tablet computer, or other electronic device performing decoding. The television, set-top box, mobile phone, tablet computer, or other electronic device may display (e.g., using a monitor, screen, or other type of display) a resulting image (e.g., an image reconstructed from the residual of a video bitstream). The television, set-top box, mobile phone, tablet computer, or other electronic device may receive a signal including an encoded image and perform decoding.

[0133] These examples can be executed by a device having at least one processor. The device can be an encoder or a decoder. These examples can be executed by a computer program product stored on a non-transitory computer-readable medium and including program code instructions. These examples can be executed by a computer program including program code instructions.

[0134] This section provides examples of intra-frame CIIP. Intra-frame CIIP can use similar or identical mechanisms to CIIP, but instead of inter-frame prediction, it uses intra-template prediction (intraTMP) modes. This allows CIIP to be used in intra-coded blocks and intra-slices, which can provide significant coding gains. Intra-frame CIIP can be performed using hybrid weights determined by intra-prediction modes based on neighboring blocks. Intra-frame CIIP can be performed using transform coding. Intra-frame CIIP can be performed in conjunction with IBC. Intra-frame CIIP can be performed using symbol prediction. Intra-frame CIIP can be performed using various intra-frame modes. Intra-frame CIIP can be performed using various TMP modes. Intra-frame CIIP can be performed using MPM lists.

[0135] Template-based intra-mode derivation (TIMD) can be performed. For intra-prediction modes in the most probable mode (MPM) list (e.g., each intra-prediction mode), the sum of the absolute transform difference (SATD) between the template's prediction and the reconstructed sample can be calculated. The intra-prediction mode with the minimum SATD (e.g., the first two intra-prediction modes) can be selected as the TIMD mode. The TIMD mode (e.g., these two TIMD modes) can be fused with weights, and this weighted intra-prediction can be used to encode the current CU. Location-dependent intra-prediction combination (PDPC) can be included in the derivation of the TIMD mode.

[0136] The costs of two selected modes can be compared to a threshold. In testing, cost factor 2 is applied as follows: costMode2 < 2 * costMode1. If this condition is true, fusion can be applied; otherwise, only mode 1 can be used. The weights of the modes can be calculated based on their SATD costs as follows: weight1 = costMode2 / (costMode1+ costMode2) weight2 = 1 - weight1 This provides an example of decoder-side intra-mode derivation (DIMD). If DIMD is applied, intra-modes (e.g., two intra-modes) can be derived from reconstructed neighboring samples, and those predictors (e.g., two predictors) can be combined with a planar mode predictor, where weights are derived from gradients (Gx, Gy) computed using the reconstructed neighboring samples. Division operations in weight derivation can be performed using a lookup table (LUT) (e.g., the same LUT-based integration scheme used by CCLM modes). In the example, the division operation in orientation computation is as follows: The following LUT-based scheme can be used for calculation: x = Floor( Log2( Gx ) ) normDiff = ( ( Gx<<4 )>>x )&15 x += (3 + (normDiff != 0) 1 : 0 Orient = (Gy* ( DivSigTable[ normDiff ] | 8 ) + ( 1<<( x-1 ) ))>>x in: DivSigTable

[16] = {0, 7, 6, 5,5, 4, 4, 3, 3, 2, 2, 1, 1, 1, 1, 0}.

[0137] The exported intra-frame modes can be included in the main list of intra-frame MPMs, so the DIMD process can be performed before the MPM list is built. The main exported intra-frame modes of a DIMD block can be stored with the block and used for the construction of the MPM list of neighboring blocks.

[0138] This section provides an example of combining CIIP with TIMD and template matching merging. In CIIP, prediction samples can be generated by weighting the inter-frame prediction signals that have been merged using CIIP-TM and the intra-frame prediction signals that have been predicted using intra-frame prediction modes derived using TIMD. This combination can be applied (e.g., it can be applied only to) coded blocks with an area less than or equal to 1024.

[0139] TIMD derivation can be used to derive intra-prediction modes in CIIP. The intra-prediction mode with the smallest SATD value can be selected from the TIMD mode list and mapped to one of 67 regular intra-prediction modes.

[0140] Figures 5A-5B An example of angled pattern partitioning is shown. The weights of the two tests (wintra, winter) can be modified (e.g., if the derived intra-prediction pattern is an angled pattern). Figure 5A Shows vertically divided blocks (e.g., the current block), which can be applied to near-horizontal modes (2 <= angle mode index < 34). Figure 5B Shows horizontally divided blocks (e.g., the current block), which can be applied to near-vertical mode (34 <= angle mode index <= 66).

[0141] The (wIntra, wInter) values ​​for the different sub-blocks are shown in Table 1 below, which lists the weights used for the angular mode modifications. Sub-block index (wIntra, wInter) 0 (6, 2) 1 (5, 3) 2 (3, 5) 3 (2, 6) Table 1.

[0142] In the CIIP template matching example, a list of CIIP template matching merge candidates can be constructed for the CIIP template matching pattern. Merge candidates can be refined using template matching. CIIP template merge candidates can be reordered as regular merge candidates through adaptive reordering (ARMC). The maximum number of CIIP template matching merge candidates can be equal to 2.

[0143] Figures 6A-6C An example of a GPM with inter-frame and intra-frame prediction is shown. In the example of a GPM with inter-frame and intra-frame prediction, the final prediction samples are generated by weighting the inter-frame prediction samples and intra-frame prediction samples of the regions separated by the GPM (e.g., the regions generated by each GPM). The inter-frame prediction samples can be derived from the inter-frame GPM, while the intra-frame prediction samples can be derived from the intra-frame prediction mode (IPM) candidate list and the index signaled from the encoder. The IPM candidate list size can be predefined as 3. The available IPM candidates can be at least one of the following: a parallel angle mode relative to the GPM block boundary (parallel mode), a vertical angle mode relative to the GPM block boundary (vertical mode), or a planar mode, such as... Figures 6A-6C As shown. Figure 6D The GPM with intra-frame and intra-frame prediction shown can be constrained to reduce the signaling overhead of IPM and avoid increasing the size of the intra-frame prediction circuitry on the hardware decoder. Direct motion vectors and IPM storage can be introduced on GPM mixing to improve coding performance.

[0144] In IPM derivation based on DIMD and neighbor patterns, parallel patterns can be registered first. Two IMP candidates (e.g., at most two IMP candidates) can be derived from DIMD, and / or neighboring blocks can be registered if no identical IPM candidates are in the list. For neighbor pattern derivation, there may be five (e.g., at most) locations of available neighboring blocks. These locations may be limited by the GPM block boundary angles, as shown in Table 2 below, which may have been used for GPM with template matching (GPM-TM). As shown in Table 2, the locations of available neighboring blocks for IPM candidates can be derived based on the angles of the GPM block boundaries. A and L represent the top and left sides of the predicted block. GPM perspective 0 2 3 4 5 8 11 12 13 14 First Division A A A A L+A L+A L+A L+A A A Second Division L+A L+A L+A L L L L L+A L+A L+A partition angle 16 18 19 20 21 24 27 28 29 30 First Division A A A A L+A L+A L+A L+A A A Second Division L+A L+A L+A L L L L L+A L+A L+A Table 2.

[0145] GPM-intraframe can be combined with GPMs that have different motion vector combining (GPM-MMVD). TIMD can be used on IPM candidates within GPM-intraframe, which can improve coding performance. Parallel modes can be registered first, followed by TIMD, DIMD, and IPM candidates for neighboring blocks.

[0146] Figure 7 An example of an intra-template matching search region used in intraTMP is shown. IntraTMP is an intra-prediction mode that copies the best prediction block from the reconstructed portion of the current image (e.g., the current frame), with its L-shaped template matching the current template. For a predefined search range, the encoder can search the reconstructed portion of the current image for the template most similar to the current template and use the corresponding block as the prediction block. The encoder can (e.g., then can) signal the use of this mode. The same prediction operation can be performed on the decoder side.

[0147] By connecting the L-shaped causal neighbors of the current block with Figure 7 The prediction signal is generated by matching another block within a predefined search region, which includes: R1: Current CTU R2: Top Left CTU R3: Above CTU R4: Left CTU The sum of absolute differences (SAD) can be used as a cost function. Within a region (e.g., within each region), the decoder can search for the template with the minimum SAD relative to the current template and use its corresponding block as the prediction block. The size of the region (SearchRange_w, SearchRange_h) can be set to be proportional to the block size (BlkW, BlkH) to have a fixed number of SAD comparisons per pixel. That is: SearchRange_w = a * BlkW SearchRange_h = a * BlkH Where "a" can be a constant that controls the gain / complexity tradeoff. For example, "a" can be equal to 5.

[0148] For CUs with a width and height of 64 or less, the intraTMP tool can be enabled. The maximum CU size for intraTMP can be configurable. IntraTMP mode can be signaled at the CU level via a dedicated flag.

[0149] Figure 8-9 An example with intra-intra-predictive CIIP is shown. For example... Figure 8 As shown, in intra-intra-prediction, the inter-frame portion of CIIP can be replaced by the intraTMP mode. The first intra-prediction signal can be associated with the intraTMP mode. For example... Figure 8As shown, in intra-intra-prediction, the intra-frame portion of CIIP can be derived using TIMD patterns. In the example, a regular intra-prediction pattern can be derived via TIMD pattern derivation. The second intra-prediction signal can be associated with the regular intra-prediction pattern (e.g., obtained based on the regular intra-prediction pattern).

[0150] like Figure 9 As shown, in intra-intra-prediction, the intra-frame portion of CIIP can be replaced by the intraTMP mode. The first intra-prediction signal can be associated with the intraTMP mode. For example... Figure 9 As shown, in intra-intra-frame scenarios, the inter-frame portion of CIIP can be merged using template matching. In the example, a CIIP template matching merge candidate list can be constructed for the CIIP template matching pattern. The merge candidates can be refined using template matching. The second intra-frame prediction signal can be associated with the template matching prediction (e.g., obtained based on the template matching prediction).

[0151] Figure 10 An example of CIIP with intra-intra-prediction is shown.

[0152] Here are examples of blending weights associated with the predicted signal. In CIIP, the blending mode (e.g., wIntra and wInter) can depend on the intra-prediction mode. For non-angular modes (e.g., DC or planar) or small blocks (e.g., width less than 4 or height less than 4), the following weights from Table 3 can be used: Table 3.

[0153] As shown in Table 3 above, if two neighboring blocks (left CU and top CU) are both intra-coded (isIntra=true), the intra-prediction weight (wIntra) can be three times the inter-prediction weight (wInter). If two neighboring blocks are both inter-coded (isIntra=false), the inter-prediction weight can be three times the intra-prediction weight. If neighboring block predictions are different (e.g., one is intra-prediction and the other is inter-prediction), a weight of 2 can be assigned to both predictions. These weights can ultimately be normalized by dividing the final prediction by 4.

[0154] In the case of intra-frame CIIP, weights can be calculated based on whether intraTMP is used for neighboring blocks (e.g., rather than whether inter-frame prediction is used). The following weights from Table 4 can be used: Table 4.

[0155] As shown in Table 4 above, if both neighboring blocks (left CU and top CU) are encoded with intraTMP (isIntraTMP = true), the weight of intra-prediction (wIntraTMP) can be three times the weight of regular intra-prediction (wIntra). If neither neighboring block is encoded with intraIMP (isIntraTMP = false) (e.g., they use regular intra-prediction), the weight of regular intra-prediction can be three times the weight of intraTMP prediction. If the neighboring block predictions are different (e.g., one is intra-prediction and the other is intraTMP), then a weight of 2 can be assigned to both predictions. These weights can ultimately be normalized by dividing the final prediction by 4.

[0156] In the example, a video decoder or encoder can obtain a first intra-prediction signal and a second intra-prediction signal for a block. The first intra-prediction signal can be associated with a first intra-prediction mode. The first intra-prediction mode can be an intra-template prediction mode (intraTMP). The second prediction signal can be associated with a second intra-prediction mode. The second intra-prediction mode can be a regular intra-prediction mode. The decoder or encoder can generate prediction samples associated with the block by determining a first weight associated with the first intra-prediction signal and a second weight associated with the second intra-prediction signal. The first and second weights can be determined based on the first and second neighboring block prediction modes. The encoder can derive residual samples associated with the block based on the prediction samples. The block can be decoded or encoded based on the residual samples.

[0157] As described above (e.g., in the example shown in Table 4), if one of the first or second neighboring block prediction modes is an intra-template prediction mode (intraTMP), and one of the first or second neighboring block prediction modes is a regular intra-prediction mode, then the decoder or encoder can determine that the first weight and the second weight are weighted equally (e.g., each has a weight equal to 2). If the first and second neighboring block prediction modes are intra-template prediction (intraTMP) modes, then the decoder or encoder can determine that the first weight is 3 and the second weight is 1. If the first and second neighboring block prediction modes are regular intra-prediction modes, then the decoder or encoder can determine that the first weight is 1 and the second weight is 3.

[0158] In the example, a mixing procedure corresponding to IBC-CIIP can be used. The IBC-CIIP mixing procedure can be defined as follows: Pred(x,y) = (a*PredReg + b*PredIbc + offset)>>shift Where >> represents a downward shift, PredReg corresponds to the prediction obtained through regular intra-frame processing, and PredIbc corresponds to the prediction obtained through IBC processing. Variables a, b, shift, and offset can be calculated as follows: if merging is used: then b = 13, a = 3, shift = 4; otherwise: a = b = 1, shift = 1. The offset can be calculated as 1 << (shift – 1) (e.g., in merging and other cases).

[0159] PredIbc can be replaced by IntraTMP prediction. The following weights can be used: if any neighbor is encoded in block vector (IntraTMP, IBC, or the new CIIP mode): then b=13, a=3, shift=4; otherwise: a=b=1, shift=1.

[0160] This article provides examples of transform coding associated with intra-frame CIIP. Transform coding can have at least one or all three of the following components: Multiple Transform Selection (MTS), Low-Frequency Inseparable Transform (LFNST), or Inseparable Primary Transform (NSPT). MTS (e.g., primary transform) can include trigonometric transforms (e.g., DCT and DST, an alternative to the default DCT2 transform). LFNST (e.g., secondary transform) can be applied in an inseparable manner to the low-frequency portion after the primary transform (on the encoder side). NSPT is a transform that can be applied directly to the residual in an inseparable manner. Due to the high number of multiplications, NSPT may be limited to small blocks.

[0161] The transformations described above may depend on the intra-frame mode used. This is likely because they are trained offline on datasets grouped by intra-frame prediction modes. To adapt intra-frame CIIPs to these modes and transformations, at least one of the following can be used: treat the intra-frame CIIP as a planar mode; use the corresponding intra-frame mode of the intra-frame CIIP; or derive an equivalent mode. To treat intra-frame CIIPs as planar modes, MTS, LFNST, and NSPT can use transform kernels specifically designed for planar modes. For the corresponding intra-frame mode using the intra-frame CIIP, this intra-frame prediction mode can be used for transform kernel selection since the CIIP may already use intra-frame prediction (e.g., it may be a TIMD mode). To derive an equivalent mode, an equivalent mode can be derived for the intra-frame CIIP mode. That is, prediction can be made from the intra-frame CIIP, and a prediction mode similar to the predicted signal can be derived. The prediction mode can be generated from the predicted signal using the DIMD procedure.

[0162] This section provides an example of intra-frame CIIP and IBC interaction. In the example, if intra-frame TMP is used, its block vectors can be used as merge candidates for IBC. There are likely five spatial candidates for IBC merging: top PU, left PU, top-right PU, bottom-left PU, and top-left PU. To generate the merge candidate list, if a PU is encoded using either IBC or intra-frame TMP, they can be treated equally (e.g., because both modes can have usable block vectors). In the example, if any of the five PUs is encoded using intra-frame CIIP, their corresponding block vectors can be used to construct the merge candidate list.

[0163] Intra-frame TMP can be used in chroma direct block vector (chromaDBV) mode. In this mode, the block vectors of IBC and intra-frame TMP (e.g., which can be used only for luma) can be used for chroma parity blocks. The syntax element can be signaled to indicate the use of this mode. For intra-frame CIIP, if the parity luma is intra-frame CIIP encoded, chromaBVD mode can be used, and the corresponding block vectors can be used for chroma.

[0164] This section provides an example of interaction between intra-frame CIIP and the MPM list. The MPM construction process can consider the following five neighboring PUs: top PU, left PU, top-right PU, bottom-left PU, and top-left PU. If these PUs are encoded using regular intra-frame patterns (e.g., plane, DC, or angle), the MPM can be constructed from these PUs using intra-frame patterns. If using intra-frame CIIP, at least one of the following options can be considered: using the intra-frame portion of the intra-frame CIIP pattern; or using an equivalent pattern. To use an equivalent pattern, it can be derived from the predicted signal using DIMD. This allows for improvements to the MPM list structure. In the example, both the intra-frame pattern and the equivalent pattern of the intra-frame CIIP are available for use with the intra-frame CIIP. This can be done by placing either pattern at the end of the MPM list and potentially at the beginning of the secondary MPM list.

[0165] This article provides an example of the interaction between intra-frame CIIP and symbol prediction. The coefficient symbol prediction mode can be deactivated via intra-frame CIIP (e.g., disabled if CIIP is used). This is likely because this mode requires several template analyses / processes. Templates can be used to compute (e.g., may need to compute) intra-frame prediction, TIMP mode derivation, potential equivalent modes for transform kernel selection, and template analysis for symbol prediction. Since this combination may not exhibit a practical gain-complexity tradeoff, symbol prediction can be deactivated if CIIP is used.

[0166] This article provides an example of intra-frame CIIP interacting with multiple intra-frame modes. In the example, the intra-frame modes are derived from the TIMD process. Multiple intra-frame modes can be allowed. For example, N possible intra-frame mode candidates can be constructed. Intra-frame modes can be signaled to the decoder, and prediction can be performed based on the signaled mode. A specific number (e.g., only a specific number) of intra-frame modes can be used, as allowing all modes could be costly. In the example, the MPM mode can be allowed for intra-frame CIIP. In the example, modes can be ordered based on template cost.

[0167] To use MPM modes (e.g., all MPM modes), the same signaling mechanism as the default mode can be used. For example, MPM-based signaling can be derived from a neighboring PU. The encoder can select the optimal mode.

[0168] For sorting patterns based on template cost, some or all patterns can be tested on the reconstructed template, and the patterns can be sorted according to the template cost (e.g., similar to the TIMD process). A blending process or CIIP can be considered. If testing candidate intra-frame patterns, a blending function with intraTMP patterns can be tested on a template weighted between the intra-frame pattern and intraTMP. The tested predictions can produce matching costs (e.g., SSD, SAD, SATD) on the template, and the patterns can be sorted accordingly. In the example, all patterns can be used for sorting. In the example, sorting can be limited to MPM patterns. The number of patterns signaled after sorting can be fixed at N. If N equals 1, this is equivalent to the TIMD process.

[0169] This paper provides an example of intra-frame CIIP interacting with multiple intraTMP candidates. In the example, the intraTMP search process can generate multiple candidates and their associated template distances. The encoder can select the best candidate from N candidates and signal the best candidate to the decoder. In the example, block vectors from neighboring blocks that are available IBC or IntraTMP encoded can be used as candidates for the current IntraTMP mode (e.g., additional candidates). That is, in addition to the current block vector that can be obtained through the IntraTMP process, other block vectors obtained from neighboring blocks can be used. The N IntraTMP candidates can be tested together with M intra-frame modes to generate L ordered modes (e.g., where each mode can include intraTMP candidates and intra-frame mode candidates). The encoder can select the best mode and signal the best mode to the decoder.

[0170] This article provides examples of intra-frame CIIP interacting with horizontal and vertical template selection. In the examples, intraTMP can allow the use of either an upper-only or a left-only template. That is, the encoder can choose to find the best intraTMP candidate from matching left, upper, or upper-left candidates. Intra-frame CIIP can allow one or more of the following: if an upper template is used, a horizontal TIMD can be used for the intra-frame portion; if a left template is used, a vertical TIMD can be used for the intra-frame portion; or the default TIMD can be used. For a special mode of SGPM (Spatial GPM), horizontal and / or vertical TIMD can be performed. In this mode (e.g., in addition to the default TIMD procedure), two additional TIMD candidates can be derived. In the examples, the horizontal TIMD can be derived from the upper template, and the vertical TIMD can be derived from the left template.

[0171] A device (e.g., a video encoding device) may perform (e.g., be configured to perform) one or more actions. For example, the device may obtain a first intra-frame prediction signal associated with a first prediction mode for a video block; obtain a second intra-frame prediction signal associated with a block vector-based prediction mode for the video block; generate prediction samples associated with the video block based on the first and second intra-frame prediction signals; and encode the video block based on the prediction samples.

[0172] In the example, the block vector-based prediction mode can be IntraTMP (Intra-TMP) or IntraBlock Copy (IBC).

[0173] The device can determine a first weight associated with a first intra-frame prediction signal and a second weight associated with a second intra-frame prediction signal. Prediction samples associated with video blocks can be generated based on the first weight applied to the first intra-frame prediction signal and the second weight applied to the second intra-frame prediction signal. For example, prediction samples can be generated based on merging the first and second intra-frame prediction signals. The first and / or second weights can be determined based on whether the IntraTMP is associated with one or more of the first and second blocks of neighboring video blocks. For example, if both neighboring blocks are intra-coded (e.g., the IntraTMP is not associated with neighboring blocks), then the first weight can be 3 and the second weight can be 1.

[0174] The first intra-frame prediction signal can be associated with a list of most probable modes (MPMs) for a video block. The device can obtain an index associated with the MPM list; sort multiple MPM modes in the MPM list based on the corresponding cost associated with each MPM mode; and determine a first prediction mode based on the sorted multiple MPM modes and the index, wherein the first intra-frame prediction signal is obtained based on the first prediction mode.

[0175] The device can obtain multiple block vectors and indices; and select a block vector from the multiple block vectors based on the indices, wherein a second prediction signal is obtained based on the selected block vector.

[0176] The device can be configured to select a transform for a video block based on a first prediction mode and / or a second intra-frame prediction mode. The transform selected for the video block can be one of Multiple Transform Selection (MTS), Low-Frequency Inseparable Transform (LFNST), or Inseparable Primary Transform (NSPT).

[0177] In the example, a video decoder or encoder can obtain a first intra-prediction signal and a second intra-prediction signal for a video block. The first prediction signal may be associated with a first intra-prediction mode (e.g., a non-template-based prediction mode). The second prediction signal may be associated with a second block vector-based intra-prediction mode (e.g., IntraTMP or IBC). The decoder or encoder can generate a combined prediction of the video block by applying a first weight to the first intra-prediction signal and a second weight to the second intra-prediction signal. The first and second weights may be determined based on a first neighboring block prediction mode and a second neighboring block prediction mode. The encoder can derive a residual block associated with the video block based on the combined prediction. The block may be encoded based on the derived residual block. The decoder can obtain the residual block from the video data and / or can reconstruct the block based on the residual block and the generated combined prediction.

[0178] Although the features and elements have been described above in specific combinations, those skilled in the art will understand that each feature or element can be used alone or in any combination with other features and elements. Furthermore, the methods described herein can be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROMs and digital multifunction discs (DVDs). A processor associated with the software can be used to implement a radio frequency transceiver used in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. A video decoding device, comprising: The processor is configured as follows: For a video block, obtain the first intra-frame prediction signal associated with the first prediction mode; For a video block, a second intra-frame prediction signal associated with the block vector-based prediction mode is obtained; Predictive samples associated with video blocks are generated based on the first intra-frame prediction signal and the second intra-frame prediction signal. as well as Video blocks are decoded based on predicted samples.

2. The video decoding device according to claim 1, wherein the block vector-based prediction mode is Intra-Template Matching Prediction (IntraTMP).

3. The video decoding device according to claim 1, wherein the prediction mode based on block vectors is intra-block copy (IBC).

4. The video decoding device according to any one of claims 1-3, wherein the processor is further configured to: A first weight associated with a first intra-frame prediction signal and a second weight associated with a second intra-frame prediction signal are determined, wherein prediction samples associated with video blocks are generated based on the first weight applied to the first intra-frame prediction signal and the second weight applied to the second intra-frame prediction signal.

5. The video decoding device of claim 4, wherein the first weight and the second weight are determined based on whether the IntraTMP is associated with one or more of the first block and the second block of the neighboring video blocks.

6. The video decoding apparatus according to any one of claims 1-3, wherein the first intra-frame prediction signal is associated with a list of most probable modes (MPMs) of a video block, and wherein the processor is further configured to: The MPM patterns in the MPM list are sorted based on the corresponding cost associated with each MPM pattern; and A first prediction mode is determined based on a sorted set of multiple MPM modes, wherein a first intra-frame prediction signal is obtained based on the first prediction mode.

7. The video decoding device according to claim 6, wherein the processor is further configured to: Obtain the index associated with the MPM list, where the first prediction pattern is determined based on the index and the sorted MPM list.

8. The video decoding device according to any one of claims 1-7, wherein the processor is further configured to: Obtain multiple block vectors and indices; and A block vector is selected from multiple block vectors based on an index, where the second intra-frame prediction signal is obtained based on the selected block vector.

9. The video decoding apparatus of claim 8, wherein the plurality of block vectors includes at least one of the following: Multiple block vectors obtained by performing IntraTMP (Intra-Temporal Template Matching) prediction on video blocks; or Multiple block vectors obtained from at least one block of a neighboring video block.

10. A video decoding method, comprising: For a video block, obtain the first intra-frame prediction signal associated with the first prediction mode; For a video block, a second intra-frame prediction signal associated with the block vector-based prediction mode is obtained; Predictive samples associated with video blocks are generated based on the first intra-frame prediction signal and the second intra-frame prediction signal. as well as Video blocks are decoded based on predicted samples.

11. The video decoding method according to claim 10, wherein the block vector-based prediction mode is IntraTMP.

12. The video decoding method according to claim 10, wherein the block vector-based prediction mode is IBC.

13. The video decoding method according to any one of claims 10-12, further comprising: A first weight associated with a first intra-frame prediction signal and a second weight associated with a second intra-frame prediction signal are determined, wherein prediction samples associated with video blocks are generated based on the first weight applied to the first intra-frame prediction signal and the second weight applied to the second intra-frame prediction signal.

14. The video decoding method of claim 13, wherein the first weight and the second weight are determined based on whether the IntraTMP is associated with one or more of the first block and the second block of the neighboring video blocks.

15. A computer program product stored on a non-transitory computer-readable medium and comprising program code instructions, which, when executed by a processor, are used to implement the method of any one of claims 10-14.

Images