Intra subpartitions in video coding

Abstract

An intra subpartition (ISP) may be enabled for the current block, for example, based on an ISP indication. The block may be divided into multiple subpartitions, and the subpartitions may belong to a prediction unit (PU). A subpartition width and a minimum predicted block width for the current block may be obtained. A PU corresponding to the current subpartition may be determined based on the subpartition width and the minimum predicted block width. For example, when the subpartition width is less than the minimum predicted block width, the PU may include multiple subpartitions. In an example, the minimum predicted block width may be four samples. A reference sample may be determined, and the PU may be predicted using the reference sample.

Description

[Technical field]

[0001] The present invention relates to intra sub-partitions, and more particularly to intra sub-partitions in video coding. [Background technology]

[0002] Video coding systems are widely used to compress digital video signals, for example to reduce the storage and / or transmission bandwidth required for such signals. Video coding systems may include block-based, wavelet-based, and / or object-based systems. The systems may utilize video coding techniques, for example, certain prediction techniques. Summary of the Invention

[0003] Intra sub-partitions (ISP) may be an intra-coding tool. Systems, methods, and means may be provided in which prediction of a sub-partition may be partially or completely independent from reconstruction of previous adjacent sub-partitions. A sub-partition may be predicted using prediction samples from adjacent sub-partitions. A sub-partition may be predicted using predicted samples and samples reconstructed from previous adjacent sub-partitions.

[0004] Prediction based on multiple subpartitions may be used when intra prediction of two or more subpartitions may be processed simultaneously from related video blocks (e.g., coding units (CUs) that neighbor reference samples). A prediction unit (PU) that includes multiple subpartitions may be predicted using reference samples that neighbor the PU.

[0005] ISP may be enabled for the current block, for example, based on the ISP indication. A subpartition width may be determined for the current block. The subpartition width may be used to subpartition the current block for intra prediction. The subpartition may belong to a PU. A PU corresponding to the subpartition may be determined based on the subpartition width and a minimum predicted block width. In an example, the minimum predicted block width may be 4 samples. Prediction may be performed based on the PU. For example, a reference sample may be determined, and the PU may be predicted using the reference sample. The reference sample may be adjacent to the PU.

[0006] The minimum predicted block width may be used in conjunction with prediction of a subpartition. One PU may include two or more subpartitions divided vertically or horizontally. The predicted block width may be the subpartition width or a multiple of the subpartition width. The subpartition width and the minimum predicted block width may be used to determine a predicted block. For example, a predicted block having multiple subpartitions may be used for prediction when the subpartition width is less than the minimum predicted block width. For example, when the minimum predicted block width is 4 samples, a predicted block having a width of 4 samples may be used for prediction when the subpartition width is less than 4 samples.

[0007] A PU may include multiple transform block sub-partitions. For example, for CUs with sizes of 4×8, 4×N (e.g., N>8), and 8×N, a PU in each CU may include multiple transform block sub-partitions.

[0008] A PU of a CU may be predicted using neighboring samples of the CU and / or samples from a reconstructed subpartition. For example, for CUs with sizes of 4×8 and 4×N, a video decoder or encoder may predict a PU using neighboring samples of the CU. For a CU with a size of 8×N, a video decoder or encoder may predict a PU using neighboring samples of the CU and samples from a reconstructed subpartition in the CU.

[0009] The method may include determining that ISP is enabled for the current block. The method may include determining a sub-partition width for the current block. The method may include determining a corresponding prediction block based on the sub-partition width and a minimum prediction block width for the sub-partitions in the current block. The method may include determining a reference sample. The method may include predicting the prediction block using the reference sample. The methods herein may be performed by a decoder. In some examples, the methods herein or corresponding methods may be performed by an encoder.

[0010] The computer-readable medium may include instructions for causing one or more processors to perform the method(s) described herein.

[0011] The program is a computer program product comprising instructions that, when executed by one or more processors, may cause the one or more processors to perform the method(s) described herein. [Brief description of the drawings]

[0012] [Figure 1A] FIG. 1 is a system diagram of an example communication system in which one or more disclosed aspects may be implemented. [Figure 1B]FIG. 1B is a system diagram illustrating an example wireless transmit / receive unit (WTRU) that may be used within the communication system illustrated in FIG. 1A according to one aspect. [Figure 1C] FIG. 1B is a system diagram illustrating an example radio access network (RAN) and an example core network (CN) that may be used within the communication system illustrated in FIG. 1A according to one embodiment. [Figure 1D] FIG. 1B is a system diagram illustrating a further exemplary RAN and a further exemplary CN that may be used within the communication system illustrated in FIG. 1A according to one embodiment. [Diagram 2] FIG. 2 is a diagram of an example block-based video encoder. [Diagram 3] FIG. 2 is a diagram of an exemplary video decoder. [Figure 4] 1 illustrates a block diagram of an example system in which various aspects and examples may be implemented. [Diagram 5] 1 illustrates examples of permissible divisions or CU splits for a CU (eg, a 4x8 pixel CU or an 8x4 pixel CU). [Figure 6] 1 illustrates examples of permissible divisions for a CU (eg, a CU larger than 4x8 pixels or 8x4 pixels). [Figure 7] 1 illustrates diagrams (a), (b), and (c) of exemplary predictions in Intra sub-partitions (ISP). [Figure 8] 13 is an example flowchart for most probable mode (MPM) list construction for reference line zero intra prediction. [Figure 9] 13 is an exemplary flowchart for MPM list construction for multiple reference line intra prediction. [Figure 10a]1 illustrates an exemplary flow chart for MPM list configuration for an ISP. [Figure 10b] 1 illustrates an exemplary flow chart for MPM list configuration for an ISP. [Figure 10c] 1 illustrates an exemplary flow chart for MPM list configuration for an ISP. [Figure 11] 13 is an exemplary flowchart for unified MPM list configuration. [Figure 12] 13A and 13B illustrate exemplary CU-based prediction diagrams (a) and (b) where the bottom rows of subpartitions A, B, and C are predicted using top and left reference lines bordering the CU. [Figure 13] Illustrated are diagrams (a) and (b) of exemplary subpartition-based prediction, where the bottom rows of predicted subpartition A in (a) are used when predicting B, and the bottom rows of subpartition B in (b) are used when predicting subpartition C. [Figure 14(a)] FIG. 2 is an example diagram of pipeline stages for an ISP. [Figure 14(b)] FIG. 1 is a diagram of an example of pipeline stages for ISP with subpartition-based prediction. [Figure 14(c)] FIG. 1 is a diagram of an example of pipeline stages for ISP with multiple subpartition-based prediction or multi-subpartition-based prediction. [Figure 15] FIG. 13 is a diagram of an example of hybrid prediction where a subpartition can be predicted using reconstructed or predicted samples from neighboring subpartitions. [Figure 16] 1 illustrates diagrams (a) and (b) of an exemplary hybrid prediction. [Figure 17] 1 illustrates an example of multiple sub-partition based prediction or multi-sub-partition based intra-prediction. [Figure 18]1 illustrates various examples of ISP sub-partitions for horizontally split CUs and vertically split CUs. [Figure 19A] We illustrate an example where 1×N (N≧16) and 2×N (N≧8) sub-partitions as illustrated in FIG. 18 may be removed and replaced with 4×N sub-partitions. [Figure 19B] Illustrates an example where an N×2 (N≧8) sub-partition as illustrated in FIG. 18 may be removed and replaced with an N×4 sub-partition. [Figure 19C] An example is illustrated where a PU may contain 32 samples. [Figure 19D] Illustrates an example of when the examples illustrated in Figures 19A, 19B, and 19C may be applied simultaneously. [Figure 20] 1 illustrates an example of integrated MPM list creation / configuration. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] A detailed description of exemplary embodiments will now be described with reference to various drawings. It should be noted that while the description gives detailed examples of possible implementations, the details are intended to be exemplary and in no way limit the scope of the present application.

[0014] 1A is a diagram illustrating an example communication system 500 in which one or more disclosed aspects may be implemented. The communication system 500 may be a multiple access system that provides content, such as, for example, voice, data, video, messaging, broadcast, etc., to multiple wireless users. The communication system 500 may provide the multiple wireless users access to such content through sharing of system resources, including wireless bandwidth. For example, the communication system 500 may employ one or more channel access methods, such as, for example, code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal frequency division multiple access (OFDMA), single carrier FDMA (SC-FDMA), ZT UW DTS-s zero-tail unique-word DFT-Spread OFDM (OFDM), unique word OFDM (UW-OFDM), resource block-filtered OFDM, filter bank multicarrier (FBMC), etc.

[0015] 1A, the communications system 500 may include wireless transmit / receive units (WTRUs) 502a, 502b, 502c, 502d, a RAN 504 / 513, a CN 506 / 515, a public switched telephone network (PSTN) 508, the Internet 510, and other networks 512, although it will be understood that the disclosed aspects contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 502a, 502b, 502c, 502d may be any type of device configured to operate and / or communicate in a wireless environment. By way of example, the WTRUs 502a, 502b, 502c, 502d may all be referred to as “stations” and / or “STAs” and may be configured to transmit and / or receive wireless signals and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscription-based units, pagers, cellular phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearables, head mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in the context of industrial and / or automated processing chains), consumer electronics devices, devices operating on commercial and / or industrial wireless networks, etc. Any of the WTRUs 502a, 502b, 502c, and 502d may be referred to interchangeably as a UE.

[0016] Additionally, the communication system 500 may include a base station 514a and / or a base station 514b. Each of the base stations 514a, 514b may be any type of device configured to wirelessly interface with at least one of the WTRUs 502a, 502b, 502c, 502d to facilitate access to one or more communication networks, such as the CN 506 / 515, the Internet 510, and / or other networks 512. By way of example, the base stations 514a, 514b may be a BTS (base station transceiver station), a Node-B, an eNode B, a home Node B, a home eNode B, a gNB, a NR NodeB, a site controller, an AP (access point), a wireless router, etc. While the base stations 514a, 514b are each depicted as a single element, it will be understood that the base stations 514a, 514b may include any number of interconnected base stations and / or network elements.

[0017] The base station 514a may be part of the RAN 504 / 513, which may further include other base stations and / or network elements (not shown), such as, for example, a BSC (Base Station Controller), an RNC (Radio Network Controller), relay nodes, and the like. The base station 514a and / or the base station 514b may be configured to transmit and / or receive wireless signals on one or more carrier frequencies, and may be referred to as a cell (not shown). The frequencies just mentioned may be licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage for wireless services for a particular geographic area, which may be relatively fixed or may change over time. Furthermore, a cell may be divided into cell sectors. For example, a cell associated with the base station 514a may be divided into three sectors. Thus, in one aspect, the base station 514a may include three transceivers, i.e., one for each sector of the cell. In an aspect, the base station 514a may employ multiple-input multiple-output (MIMO) technology and may utilize multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in a desired spatial direction.

[0018] The base stations 514a, 514b may communicate with one or more of the WTRUs 502a, 502b, 502c, 502d over an air interface 516, which may be any suitable wireless communications link (e.g., RF (radio frequency), microwave, centimeter wave, micrometer wave, IR (infrared), UV (ultraviolet), visible light, etc.). The air interface 516 may be established using any suitable RAT (radio access technology).

[0019] More specifically, as mentioned above, the communication system 500 may be a multiple access system and may employ one or more channel access schemes such as, for example, CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, the base station 514a in the RAN 504 / 513 and the WTRUs 502a, 502b, 502c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 515 / 516 / 517 using, for example, wideband CDMA (WCDMA). WCDMA may include communication protocols such as, for example, High-Speed ​​Packet Access (HSPA) and / or Evolved HSPA+ (HSPA+). HSPA may include High-Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High-Speed ​​UL Packet Access (HSUPA).

[0020] In an aspect, the base station 514a and the WTRUs 502a, 502b, 502c may establish the air interface 516 using Long Term Evolution (LTE) and / or LTE-Advanced (LTE-A) and / or LTE-Advanced Pro (LTE-A Pro), and may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA).

[0021] In an aspect, the base station 514a and the WTRUs 502a, 502b, 502c may establish an air interface 516 using New Radio (NR) and may implement a radio technology such as NR radio access.

[0022] In an aspect, the base station 514a and the WTRUs 502a, 502b, 502c may implement multiple radio access technologies. For example, the base station 514a and the WTRUs 502a, 502b, 502c may implement both LTE radio access and NR radio access, e.g., using dual connectivity (DC) principles. Thus, the air interface utilized by the WTRUs 502a, 502b, 502c may be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations (e.g., eNBs and gNBs).

[0023] In other aspects, the base station 514a and the WTRUs 502a, 502b, 502c may implement a wireless technology such as, for example, IEEE 802.11 (i.e., Wireless Fidelity (WiFi)), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GERAN (GSM EDGE), etc.

[0024] 1A may be, for example, a wireless router, a Home Node B, a Home eNode B, or an access point, and may utilize any suitable RAT to facilitate wireless connectivity in a localized area, such as, for example, an office, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a roadway, etc. In one aspect, the base station 514b and the WTRUs 502c, 502d may implement a radio technology, such as, for example, IEEE 802.11, to establish a wireless local area network (WLAN). In an aspect, the base station 514b and the WTRUs 502c, 502d may implement a radio technology, such as, for example, IEEE 802.15, to establish a wireless personal area network (WPAN). In yet another aspect, the base station 514b and the WTRUs 502c, 502d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish a picocell or a femtocell. As shown in FIG. 1A, the base station 514b may have a direct connection to the Internet 510. Thus, the base station 514b may not be required to access the Internet 510 via the CN 506 / 515.

[0025] The RAN 504 / 513 may be in communication with the CN 506 / 515 and may be any type of network configured to provide voice, data, application, and / or voice over internet protocol (VoIP) services to one or more of the WTRUs 502a, 502b, 502c, 502d. The data may have various quality of service (QoS) requirements, such as, for example, different throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. The CN 506 / 515 may provide call control, billing services, mobile location-based services, prepaid calling, Internet connectivity, video distribution, etc., and / or perform high level security functions, such as, for example, user authentication. 1A, it will be understood that the RAN 504 / 513 and / or the CN 506 / 515 may be in direct or indirect communication with other RANs employing the same RAT as the RAN 504 / 513 or a different RAT. For example, the CN 506 / 515, in addition to being connected to the RAN 504 / 513, which may utilize NR radio technology, may also be in communication with another RAN (not shown) that employs GSM, UMTS, CDMA2000, WiMAX, E-UTRA, or WiFi radio technology.

[0026] Additionally, the CN 506 / 515 may act as a gateway for the WTRUs 502a, 502b, 502c, 502d to access the PSTN 508, the Internet 510, and / or other networks 512. The PSTN 508 may include a circuit-switched telephone network providing plain old telephone service (POTS). The Internet 510 may include a global system of interconnected computer networks and devices that use common communication protocols, such as transmission control protocol (TCP), user datagram protocol (UDP), and / or IP in the TCP / IP Internet protocol suite. The network 512 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, the network 512 may include another CN connected to one or more RANs that may employ the same RAT as the RAN 504 / 513 or a different RAT.

[0027] Some or all of the WTRUs 502a, 502b, 502c, 502d in the communications system 500 may include multi-mode capabilities (e.g., the WTRUs 502a, 502b, 502c, 502d may include multiple transceivers for communicating with separate wireless networks over separate wireless links). For example, the WTRU 502c shown in FIG. 1A may be configured to communicate with a base station 514a, which may employ a cellular-based radio technology, and with a base station 514b, which may employ an IEEE 802 radio technology.

[0028] 1B is a system diagram illustrating an example WTRU 502. As shown in FIG. 1B, the WTRU 502 may include, among other things, a processor 518, a transceiver 520, a transmit / receive element 522, a speaker / microphone 524, a keypad 526, a display / touchpad 528, non-removable memory 530, removable memory 532, a power source 534, a GPS (Global Positioning System) chipset 536, and / or other peripherals 538. It will be understood that the WTRU 502 may include any sub-combination of the above-mentioned elements without being inconsistent with an aspect.

[0029] The processor 518 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors along with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any of other types of integrated circuits (ICs), a state machine, etc. The processor 518 may perform signal coding, data processing, power control, input / output processing, and / or any other functionality that enables the WTRU 502 to operate in a wireless environment. The processor 518 may be coupled to a transceiver 520, which may be coupled to a transmit / receive element 522. While FIG. 1B depicts the processor 518 and the transceiver 520 as separate components, it will be understood that the processor 518 and the transceiver 520 may be integrated together in an electronic package or chip.

[0030] The transmit / receive element 522 may be configured to transmit or receive signals to a base station (e.g., base station 514a) over the air interface 516. For example, in one aspect, the transmit / receive element 522 may be an antenna configured to transmit and / or receive RF signals. In an aspect, the transmit / receive element 522 may be an emitter / detector configured to transmit and / or receive, for example, IR, UV, or visible light signals. In yet another aspect, the transmit / receive element 522 may be configured to transmit and / or receive both RF and light signals. It will be understood that the transmit / receive element 522 may be configured to transmit and / or receive any combination of wireless signals.

[0031] 1B as a single element, the WTRU 502 may include any number of transmit / receive elements 522. More specifically, the WTRU 502 may employ MIMO techniques. Thus, in one aspect, the WTRU 502 may include two or more transmit / receive elements 522 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 516.

[0032] The transceiver 520 may be configured to modulate signals to be transmitted by the transmit / receive element 522 and to demodulate signals received by the transmit / receive element 522. As mentioned above, the WTRU 502 may have multi-mode capabilities. Thus, for example, the transceiver 520 may include multiple transceivers to enable the WTRU 502 to communicate over multiple RATs, such as, for example, NR and IEEE 802.11.

[0033] The processor 518 of the WTRU 502 may be coupled with and may receive user input data from a speaker / microphone 524, a keypad 526, and / or a display / touchpad 528 (e.g., a liquid crystal display (LCD) display unit or an organic light emitting diode (OLED) display unit). Further, the processor 518 may output user data to the speaker / microphone 524, the keypad 526, and / or the display / touchpad 528. Additionally, the processor 518 may access information and store data in any suitable type of memory, such as, for example, a non-removable memory 530 and / or a removable memory 532. The non-removable memory 530 may include a random-access memory (RAM), a read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 532 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, or the like. In other aspects, the processor 518 may access information and store data in memory that is not physically located in the WTRU 502, such as in a server or host computer (not shown).

[0034] The processor 518 may receive power from a power source 534 and may be configured to distribute and / or control the power to other components in the WTRU 502. The power source 534 may be any device suitable for providing power to the WTRU 502. For example, the power source 534 may include one or more dry batteries (e.g., NiCd (nickel cadmium), NiZn (nickel zinc), NiMH (nickel metal hydride), Li-ion (lithium ion), etc.), solar cells, fuel cells, etc.

[0035] Further, the processor 518 may be coupled to a GPS chipset 536, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 502. In addition to or instead of information from the GPS chipset 536, the WTRU 502 may receive location information from a base station (e.g., base stations 514a, 514b) over the air interface 516 and / or determine its location based on the timing of signals being received from two or more neighboring base stations. It will be understood that the WTRU 502 may obtain location information through any suitable location-determination method without being inconsistent with an aspect.

[0036] Additionally, the processor 518 may be coupled to other peripherals 538, which may include one or more software and / or hardware modules that provide additional features, functionality, and / or wired or wireless connectivity. For example, the peripherals 538 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photos and / or videos), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, a Bluetooth module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, a virtual reality and / or augmented reality (VR / AR) device, an activity tracker, and / or the like. The peripherals 538 may include one or more sensors, which may be one or more of a gyroscope, an accelerometer, a Hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor, a geolocation sensor, an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, and / or a humidity sensor.

[0037] The WTRU 502 may include a full-duplex radio in which the transmission and reception of some or all of the signals (e.g., associated with a particular subframe for both the UL (e.g., for transmission) and the downlink (e.g., for reception)) may be parallel and / or simultaneous. The full-duplex radio may include an interference management unit to reduce and or substantially eliminate self-interference, either by hardware (e.g., a choke) or by signal processing by a processor (e.g., a separate processor (not shown) or by the processor 518). In an aspect, the WRTU 502 may include a half-duplex radio in which the transmission and reception of some or all of the signals (e.g., associated with a particular subframe for either the UL (e.g., for transmission) or the downlink (e.g., for reception)) may be half-duplex.

[0038] 1C is a system diagram illustrating the RAN 504 and the CN 506 according to an aspect. As mentioned above, the RAN 504 may employ E-UTRA radio technology to communicate with the WTRUs 502a, 502b, 502c over the air interface 516. Additionally, the RAN 504 may be in communication with the CN 506.

[0039] The RAN 504 may include eNode-Bs 560a, 560b, 560c, although it will be understood that the RAN 504 may include any number of eNode-Bs without being inconsistent with an aspect. The eNode-Bs 560a, 560b, 560c may each include one or more transceivers for communicating with the WTRUs 502a, 502b, 502c over the air interface 516. In one aspect, the eNode-Bs 560a, 560b, 560c may implement MIMO techniques. Thus, for example, the eNode-B 560a may use multiple antennas to transmit and / or receive wireless signals to the WTRU 502a.

[0040] Each of the eNode-Bs 560a, 560b, 560c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and / or DL, etc. As shown in FIG. 1C, the eNode-Bs 560a, 560b, 560c may communicate with each other over an X2 interface.

[0041] 1C may include a mobility management entity (MME) 562, a serving gateway (SGW) 564, and a packet data network (PDN) gateway (or PGW) 566. While each of the above-mentioned elements is depicted as part of the CN 506, it will be understood that any of the just-mentioned elements may be owned and / or operated by an entity other than the CN operator.

[0042] The MME 562 may be connected to each of the eNode-Bs 562a, 562b, 562c in the RAN 504 via an S1 interface and may act as a control node. For example, the MME 562 may be responsible for authenticating users of the WTRUs 502a, 502b, 502c, activating / deactivating bearers, selecting a particular serving gateway during initial attachment of the WTRUs 502a, 502b, 502c, etc. The MME 562 may provide a control plane function for switching between the RAN 504 and other RANs (not shown) that employ other radio technologies, such as, for example, GSM and / or WCDMA.

[0043] The SGW 564 may be connected to each of the eNode Bs 560a, 560b, 560c in the RAN 504 via an S1 interface. In general, the SGW 564 may route and forward user data packets to the WTRUs 502a, 502b, 502c. The SGW 564 may perform other functions such as, for example, fixing the user plane during inter-eNode B handover, triggering paging when DL data is available to the WTRUs 502a, 502b, 502c, managing and storing the context of the WTRUs 502a, 502b, 502c, etc.

[0044] The SGW 564 may be connected to a PGW 566, which may provide the WTRUs 502a, 502b, 502c with access to a packet-switched network, such as the Internet 510, to facilitate communication between the WTRUs 502a, 502b, 502c and IP-enabled devices.

[0045] The CN 506 may facilitate communications with other networks. For example, the CN 506 may provide the WTRUs 502a, 502b, 502c with access to circuit-switched networks, such as the PSTN 508, to facilitate communications between the WTRUs 502a, 502b, 502c and traditional landline communication devices. For example, the CN 506 may include or communicate with an IP gateway (e.g., an IMS (IP Multimedia Subsystem) server) that serves as an interface between the CN 506 and the PSTN 508. In addition, the CN 506 may provide the WTRUs 502a, 502b, 502c with access to other networks 512, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0046] Although the WTRU is described in Figures 1A-1D as a wireless terminal, it is expected that in certain exemplary aspects such a terminal may use a wired communication interface (e.g., temporarily or permanently) with the communication network.

[0047] In an exemplary embodiment, the other network 512 may be a WLAN.

[0048] A WLAN in infrastructure Basic Service Set (BSS) mode may have an access point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP may have access or interface to a distribution system (DS) or another type of wired / wireless network that carries traffic to and from the BSS. Traffic to the STAs originating from outside the BSS may arrive through the AP and be distributed to the STAs. Traffic originating from the STAs to destinations outside the BSS may be sent to the AP and distributed to the respective destination. Traffic between STAs within a BSS may be routed through the AP, for example, where a source STA may send traffic to the AP, which may distribute the traffic to a destination STA. Traffic between STAs within a BSS may be considered and / or referred to as peer-to-peer traffic. Peer-to-peer traffic may be routed between (e.g., directly between) the source and destination STAs by a direct link setup (DLS). In one exemplary embodiment, the DLS may use 802.11e DLS or 802.11z TDLS (tunneled DLS). A WLAN using an Independent BSS (IBSS) mode may not have an AP, and STAs (e.g., all STAs) in or using an IBSS may communicate directly with each other. The IBSS mode of communication is sometimes referred to herein as an "ad-hoc" mode of communication.

[0049] When using an 802.11ac infrastructure mode of operation or a similar mode of operation, an AP may transmit a beacon on a fixed channel, such as a primary channel. The primary channel may be a fixed width (e.g., a 20 MHz wide bandwidth) or a width that is dynamically set by signaling. The primary channel may be an operating channel of the BSS and may be used by STAs to establish a connection with the AP. In a typical aspect, CSMA / CA (Carrier Sense Multiple Access / Collision Avoidance) may be implemented in an 802.11 system, for example. With CSMA / CA, STAs (e.g., every STA), including the AP, may sense the primary channel. If the primary channel is sensed / detected and / or determined to be busy by a particular STA, the particular STA may back off. One STA (e.g., only one station) may transmit at any given time in a given BSS.

[0050] A HT (high throughput) STA may, for example, use a 40 MHz wide channel for communication by combining a 20 MHz primary channel with adjacent or non-adjacent 20 MHz channels to form a 40 MHz wide channel.

[0051] A VHT (Very High Throughput) STA may support 20 MHz, 40 MHz, 80 MHz, and / or 160 MHz wide channels. A 40 MHz and / or 80 MHz channel may be constructed by combining contiguous 20 MHz. A 160 MHz channel may be constructed by combining eight contiguous 20 MHz channels or by combining two non-contiguous 80 MHz channels, which may be referred to as an 80+80 configuration. For the 80+80 configuration, the data may go to a segment parser, which may split the data into two streams after channel encoding. IFFT (Inverse Fast Fourier Transform) processing and time domain processing may be performed on each stream separately. The streams may be mapped onto two 80 MHz channels, and the data may be transmitted by the transmitting STA. At the receiver of the receiving STA, the operations for the 80+80 configuration described above may be reversed and the combined data may be sent to the MAC (Media Access Control).

[0052] Sub-1 GHz modes of operation are supported by 802.11af and 802.11ah. The operating bandwidth of the channels and carriers are reduced in 802.11af and 802.11ah compared to those used in 802.11n and 802.11ac. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TVWS (TV White Space) spectrum, and 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to an exemplary embodiment, 802.11ah may support Meter Type Control / Machine-Type Communication, such as MTC devices in macro coverage areas. MTC devices may have limited capabilities, including support (e.g., only support) for some and / or limited bandwidths. An MTC device may include a battery that has a battery life that exceeds a threshold (eg, to maintain a very long battery life).

[0053] A WLAN system that may support multiple channels and channel bandwidths, e.g., 802.11n, 802.11ac, 802.11af, and 802.11ah, etc., includes a channel that may be designated as a primary channel. The primary channel may have a bandwidth equal to the largest common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel may be set and / or limited by the STA that supports the smallest bandwidth operating mode among all STAs operating in the BSS. In an 802.11ah example, the primary channel may be 1 MHz wide for STAs (e.g., MTC type devices) that support (e.g., only support) 1 MHz mode, even if the AP and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and / or other channel bandwidth operating modes. Carrier sensing and / or NAV (Network Allocation Vector) setting may depend on the state of the primary channel. If the primary channel is busy, for example due to a STA (that only supports a 1 MHz mode of operation) transmitting to the AP, the entire available frequency band may be considered busy even though most of the frequency band may remain idle and available.

[0054] In the United States, the available frequency bands that may be used by 802.11ah are 902MHz to 928MHz. In South Korea, the available frequency bands are 917.5MHz to 923.5MHz. In Japan, the available frequency bands are 916.5MHz to 927.5MHz. The total bandwidth available for 802.11ah is 6MHz to 26MHz, depending on the country code.

[0055] 1D is a system diagram illustrating a RAN 513 and a CN 515 according to an aspect. As discussed above, the RAN 513 may employ NR radio technology to communicate with the WTRUs 502a, 502b, 502c over the air interface 516. Additionally, the RAN 513 may be in communication with the CN 515.

[0056] The RAN 513 may include gNBs 580a, 580b, 580c, although it will be understood that the RAN 513 may include any number of gNBs without being inconsistent with an aspect. The gNBs 580a, 580b, 580c may each include one or more transceivers to communicate with the WTRUs 502a, 502b, 502c over the air interface 516. In an aspect, the gNBs 580a, 580b, 580c may implement MIMO techniques. For example, the gNBs 580a, 580b may utilize beamforming to transmit and / or receive signals to the gNBs 580a, 580b, 580c. Thus, for example, the gNB 580a may use multiple antennas to transmit and / or receive wireless signals to the WTRU 502a. In an aspect, the gNBs 580a, 580b, 580c may implement carrier aggregation techniques. For example, the gNB 580a may transmit multiple component carriers to the WTRU 502a (not shown). A subset of the just mentioned component carriers may be on a non-licensed spectrum while the remaining component carriers may be on a licensed spectrum. In an aspect, the gNBs 580a, 580b, 580c may implement Coordinated Multi-Point (CoMP) techniques. For example, the WTRU 502a may receive coordinated transmissions from the gNBs 580a and 580b (and / or gNB 580c).

[0057] The WTRUs 502a, 502b, 502c may communicate with the gNBs 580a, 580b, 580c using transmissions associated with scalable numerology. For example, the OFDM symbol spacing and / or OFDM subcarrier spacing may vary for separate transmissions, separate cells, and / or separate portions of the spectrum for wireless transmission. The WTRUs 502a, 502b, 502c may communicate with the gNBs 580a, 580b, 580c using subframes or transmission time intervals (TTIs) of different or scalable lengths (e.g., including different numbers of OFDM symbols and / or different absolute lengths of persistent time).

[0058] The gNBs 580a, 580b, 580c may be configured to communicate with the WTRUs 502a, 502b, 502c in a standalone configuration and / or a non-standalone configuration. In a standalone configuration, the WTRUs 502a, 502b, 502c may communicate with the gNBs 580a, 580b, 580c without accessing another RAN (e.g., eNode-Bs 560a, 560b, 560c, etc.). In a standalone configuration, the WTRUs 502a, 502b, 502c may utilize one or more of the gNBs 580a, 580b, 580c as mobility anchor points. In a standalone configuration, the WTRUs 502a, 502b, 502c may communicate with the gNBs 580a, 580b, 580c using signals in unlicensed bands. In a non-standalone configuration, the WTRUs 502a, 502b, 502c may communicate / connect with a gNB 580a, 580b, 580c while also communicating / connecting with another RAN, such as, for example, an eNode-B 560a, 560b, 560c. For example, the WTRUs 502a, 502b, 502c may implement the DC principle to communicate with one or more gNBs 580a, 580b, 580c and one or more eNode-Bs 560a, 560b, 560c substantially simultaneously. In a non-standalone configuration, the eNode-Bs 560a, 560b, 560c may act as mobility anchors for the WTRUs 502a, 502b, 502c, and the gNBs 580a, 580b, 580c may provide additional coverage and / or throughput in serving the WTRUs 502a, 502b, 502c.

[0059] Each of the gNBs 580a, 580b, 580c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and / or DL, support for network slicing, dual connectivity, interworking between NR and E-UTRA, routing of user plane data to User Plane Functions (UPFs) 584a, 584b, routing of control plane information to Access and Mobility Management Functions (AMFs) 582a, 582b, etc. As shown in FIG. 1D, the gNBs 580a, 580b, 580c may communicate with each other over an Xn interface.

[0060] 1D may include at least one AMF 582a, 582b, at least one UPF 584a, 584b, at least one SMF (Session Management Function) 583a, 583b, and possibly a DN (Data Network) 585a, 585b. While each of the above elements is depicted as part of the CN 515, it will be understood that any of the just-mentioned elements may be owned and / or operated by an entity other than the CN operator.

[0061] The AMF 582a, 582b may be connected to one or more of the gNBs 580a, 580b, 580c in the RAN 513 via an N2 interface and may act as a control node. For example, the AMF 582a, 582b may be responsible for authenticating users of the WTRUs 502a, 502b, 502c, supporting network slicing (e.g., handling sessions of separate PDUs with separate requirements), selecting a particular SMF 583a, 583b, managing registration areas, terminating NAS signaling, mobility management, etc. Network slicing may be used by the AMF 582a, 582b to customize support of the CN for the WTRUs 502a, 502b, 502c based on the type of service being utilized for the WTRUs 502a, 502b, 502c. For example, separate network slices may be established for separate use cases such as, for example, services dependent on ultra-reliable low latency (URLLC) access, services dependent on enhanced massive mobile broadband (eMBB) access, services related to machine type communications (MTC) access, etc. The AMF 562 may provide a control plane function for switching between the RAN 513 and other RANs (not shown) employing other radio technologies such as, for example, LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies such as, for example, WiFi.

[0062] The SMFs 583a, 583b may be connected to the AMFs 582a, 582b in the CN 515 via an N11 interface. Furthermore, the SMFs 583a, 583b may be connected to the UPFs 584a, 584b in the CN 515 via an N4 interface. The SMFs 583a, 583b may select and control the UPFs 584a, 584b and configure the routing of traffic through the UPFs 584a, 584b. The SMFs 583a, 583b may perform other functions such as, for example, managing and assigning UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, providing downlink data notification, etc. The PDU session type may be IP-based, non-IP-based, Ethernet-based, etc.

[0063] The UPFs 584a, 584b may be connected to one or more of the gNBs 580a, 580b, 580c in the RAN 513 via an N3 interface and may provide the WTRUs 502a, 502b, 502c with access to a packet-switched network, such as, for example, the Internet 510, to facilitate communications between the WTRUs 502a, 502b, 502c and IP-enabled devices. The UPFs 584, 584b may perform other functions, such as, for example, routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering downlink packets, providing mobility anchoring, etc.

[0064] The CN 515 may facilitate communication with other networks. For example, the CN 515 may include or communicate with an IP gateway (e.g., an IMS (IP Multimedia Subsystem) server) that serves as an interface between the CN 515 and the PSTN 508. In addition, the CN 515 may provide the WTRUs 502a, 502b, 502c with access to other networks 512, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one aspect, the WTRUs 502a, 502b, 502c may be connected to local DNs 585a, 585b through an N3 interface to the UPFs 584a, 584b, and the UPFs 584a, 584b via an N6 interface between the UPFs 584a, 584b and the DNs (data networks) 585a, 585b.

[0065] 1A-1D and the corresponding description thereof, one or more or all of the functions described herein in association with one or more of the WTRUs 502a-d, base stations 514a-b, eNode-Bs 560a-c, MMEs 562, SGWs 564, PGWs 566, gNBs 580a-c, AMFs 582a-b, UPFs 584a-b, SMFs 583a-b, DNs 585a-b, and / or any other device(s) described herein may be performed by one or more emulation devices (not shown). The emulation devices may be one or more devices configured to emulate one or more or all of the functions described herein. For example, the emulation devices may be used to test other devices and / or to simulate network and / or WTRU functions.

[0066] The emulation device may be designed to implement one or more tests of other devices in a lab environment and / or in an operator's network environment. For example, the emulation device or devices may perform one or more or all functions while fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices in the communication network. The emulation device or devices may perform one or more or all functions while temporarily implemented / deployed as part of a wired and / or wireless communication network. The emulation device may be directly coupled to another device for testing purposes and / or may perform testing using over-the-air (OTA) wireless communications.

[0067] The emulation device or devices may perform one or more functions, including but not limited to, while not implemented / deployed as part of a wired and / or wireless communication network. For example, the emulation device may be utilized in a testing laboratory and / or testing scenario in a non-deployed (e.g., testing) wired and / or wireless communication network to implement testing of one or more components. The emulation device or devices may be test equipment. Direct RF coupling and / or wireless communication via RF circuitry (which may include, e.g., one or more antennas) may be used by the emulation device to transmit and / or receive data.

[0068] This application describes various aspects, including tools, features, examples, models, approaches, and the like. Many of the just described aspects are described with particularity and in a manner that may often seem limiting, at least to illustrate their individual characteristics. However, what is just described is for purposes of clarity in description and does not limit the application or scope of the just described aspects. Indeed, all of the different aspects can be combined and substituted to provide further aspects. Furthermore, aspects can be combined and substituted with aspects described in earlier applications as well.

[0069] The aspects described and contemplated in this application can be implemented in many different ways. While Figures 1-20 described herein may provide some examples, other examples are contemplated, and the discussion of Figures 1-20 does not limit the breadth of implementations. In general, at least one of the aspects relates to video encoding and decoding, and in general, at least one other aspect relates to transmitting the generated or encoded bitstream. The just-described aspects and other aspects can be implemented as a method, an apparatus, a computer-readable medium having instructions stored thereon, and / or a computer-readable medium having a bitstream generated according to any of the described methods, for encoding or decoding video data according to any of the described methods.

[0070] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image", "picture", and "frame" may be used interchangeably. Typically, but not necessarily, "decoded" is used on the decoder side. The term "reconstructed" may be used on the encoder side and / or on the decoder side.

[0071] Various methods are described herein, each of which includes one or more steps or actions to accomplish the method described. Unless a particular order of steps or actions is required for the inherent operation of the method, the order and / or use of specific steps and / or actions may be modified or combined. Additionally, terms such as "first", "second", etc. may be used in various examples of altering elements, components, steps, actions, etc., such as "first decode" and "second decode". The use of such terms does not imply any ordering with respect to the altered operations, unless specifically required. Thus, in the examples just described, the first decode need not occur before the second decode, but may occur, for example, before the second decode, during the second decode, or in a time period that overlaps with the second decode.

[0072] Various methods and other aspects described in the present application can be used to modify modules, such as decode modules, of the video encoder 100 and decoder 200 as shown in Figures 2 and 3. Furthermore, the aspects are not limited to VVC or HEVC, but can be applied to other standards and recommendations, whether pre-existing or developed in the future, as well as extensions of any of the above standards and recommendations (including VVC and HEVC). Unless otherwise indicated or specifically technically prohibited, the aspects described in the present application can be used individually or in combination.

[0073] Various numerical values ​​are used in this application, such as CU sizes of 4×8, 4×N, 8×4, and 8×N, a minimum prediction block width of 4 samples, and a PU containing 32 or more samples. The specific values ​​are for illustrative purposes, and the aspects described are not limited to the specific values ​​just mentioned.

[0074] A block-based video coding system, such as VVC, may include a block-based hybrid video coding framework. FIG. 2 illustrates a block diagram of an exemplary block-based hybrid video encoding system 200. Although variations of the encoder 200 are envisioned, the encoder 200 is described below for the purpose of clarity without describing all expected variations. An input video signal 202 may be processed block by block (e.g., coding unit (CU)). The CU may include a CU size up to 128x128 pixels. A coding tree unit (CTU) may be divided into CUs based on a quad-tree / binary-tree / ternary-tree to adapt to changing local characteristics. The CU may be used as a basic unit for prediction and transformation without further division. In the case of a multi-type tree structure, the CTU may be divided by a quad-tree structure. A leaf node of the quad-tree (e.g., each quad-tree lead node) may be further divided by a binary-tree structure and a ternary-tree structure.

[0075] With reference to FIG. 2, spatial prediction 260 or motion prediction 262 may be performed on an input video block (e.g., a MB (macroblock) or a CU). Spatial prediction (e.g., or intra prediction) may use pixels from neighboring blocks already coded in the same video picture and / or slice that predicts the current video block. Spatial prediction may reduce spatial redundancy inherent in a video signal. Temporal prediction (e.g., referred to as inter prediction, or motion compensated prediction) may use pixels from an already coded video picture that predicts the current video block. Temporal prediction may reduce temporal redundancy inherent in a video signal. A temporal prediction signal for a CU may be signaled by one or more motion vectors (MVs), which may indicate the amount and direction of motion between the current CU and its temporal reference. If multiple reference pictures are supported, a reference picture index may be signaled to the decoder. The reference index may be used to identify which reference picture in the reference picture store 264 the temporal prediction signal may come from.

[0076] After spatial prediction and / or temporal prediction, a mode decision 280 in the encoder may select a prediction mode based, for example, on a rate-distortion optimization mechanism. The prediction block may be subtracted from the current video block at 216. The prediction residual may be decorrelated using the transform module 204 and the quantization module 206 to achieve a target bit rate. The quantized residual coefficients may be inverse quantized at 210 and inverse transformed at 212 to form a reconstructed residual. The reconstructed residual may be added back to the prediction block at 226 to form a reconstructed video block. For example, an in-loop filter, such as a deblocking filter and / or an adaptive loop filter, may be adapted to the reconstructed video block at 266 before being placed in the reference picture store 264. The reference pictures in the reference picture store 264 may be used to encode future video blocks. An output video bitstream 220 may be formed. The coding mode (e.g., inter or intra), prediction mode information, motion information, and / or quantized residual coefficients may be sent to an entropy coding unit 208 to be compressed and packed to form a bitstream 220.

[0077] FIG. 3 illustrates a general block diagram of an exemplary block-based video decoder 300. A video bitstream 302 may be received, unpacked, and / or entropy decoded at an entropy decoding unit 308. Coding mode and / or prediction information may be sent to a spatial prediction unit 360 (e.g., if intra-coded) and / or a temporal prediction unit 362 (e.g., if inter-coded) to form a prediction block. Residual transform coefficients may be sent to an inverse quantization unit 310 and an inverse transform unit 312 to reconstruct a residual block. The prediction block and the residual block may be added at 326. The reconstructed block may go through in-loop filtering 366 and may be stored in a reference picture store 364. The reconstructed video in the reference picture store 364 may be used to drive a display device and / or to predict future video blocks. In an example, the decoder 300 may determine that the ISP is enabled for the current block after receiving, unpacking, and / or entropy decoding the video bitstream 302. The coding mode information (e.g., intra coding mode) may be sent to the spatial prediction unit 360 to form a prediction block. The decoder 300 may determine a sub-partition width for the current block based on the unpacked and / or entropy decoded video bitstream 302. For a sub-partition in the current block, the decoder may determine a corresponding prediction block based on the sub-partition width and a minimum prediction block width. The minimum prediction block width may be a predetermined value for the prediction block width. The decoder 300 may determine reference samples in the spatial prediction unit 360. The decoder 300 may predict the prediction block using the reference samples.

[0078] FIG. 4 illustrates a block diagram of an example system in which various aspects and examples are implemented. System 1000 can be embodied as a device including various components described below and configured to perform one or more aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as, for example, personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television broadcast receivers, personal video recording systems, connected consumer electronics appliances, and servers. The elements of system 1000, singly or in combination, can be embodied in a single integrated circuit (IC), multiple ICs, and / or separate components. For example, in at least one example, the processing and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or separate components. In various examples, system 1000 is communicatively coupled to one or more other systems or other electronic devices, for example, via a communication bus or via dedicated input and / or output ports. In various examples, the system 1000 is configured to implement one or more aspects described in this document.

[0079] The system 1000 includes at least one processor 1010 configured to execute instructions loaded therein to implement various aspects described herein, for example. The processor 1010 can include embedded memory, input / output interfaces, and various other circuits known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 1000 includes a storage device 1040, which can include non-volatile and / or volatile memory, including, but not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Flash, magnetic disk drives, and / or optical disk drives. The storage device 1040 may include, by way of non-limiting example, an internal storage device, an attached storage device (including separable and non-separable storage devices), and / or a network-accessible storage device. In one example, an apparatus including one or more processors may be configured to determine that ISP is enabled for a current block. The one or more processors may be configured to determine a sub-partition width for the current block. The one or more processors may be configured to determine a corresponding predicted block based on the sub-partition width and a minimum predicted block width for the sub-partitions in the current block. The one or more processors may be configured to determine a reference sample.The one or more processors may be configured to predict the predictive block using the reference samples. An apparatus including the one or more processors may be a decoder or an encoder.

[0080] The signal may be sent to enable the device to perform one or more steps herein. For example, the device may include an access unit configured to access data that enables the device to perform one or more steps herein, and a transmitter configured to transmit the data. The method may include accessing data that enables the device to perform one or more steps herein, and transmitting the data. The computer-readable medium may include data that enables the device to perform one or more steps herein.

[0081] The system 1000 includes an encoder / decoder module 1030 configured to process data to provide, for example, encoded or decoded video, which may include its own processor and memory. The encoder / decoder module 1030 represents a module or modules that may be included in a device that performs encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Additionally, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000 or may be incorporated within the processor 1010 as a combination of hardware and software, as is known to those skilled in the art.

[0082] Program code loaded onto the processor 1010 or encoder / decoder 1030 to perform various aspects described herein may be stored in the storage device 1040 and subsequently loaded onto the memory 1020 for execution by the processor 1010. According to various examples, one or more of the processors 1010, memory 1020, storage device 1040, and encoder / decoder module 1030 may store one or more of various items during execution of processes described herein. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from processing of equations, formulas, operations, and operational logic.

[0083] In some examples, memory internal to the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other examples, memory external to the processing device (e.g., the processing device can be either the processor 1010 or the encoder / decoder module 1030) is used for one or more of the just-mentioned functions. The external memory can be the memory 1020 and / or the storage device 1040, for example, a dynamic volatile memory and / or a non-volatile flash memory. In some examples, the external non-volatile flash memory is used to store, for example, the operating system of the television. In at least one example, a high-speed external dynamic volatile memory such as a RAM is used as working memory for video encoding and decoding operations, such as for MPEG-2 (MPEG refers to the Moving Picture Experts Group, which is further referred to as ISO / IEC 13818, which is further known as H.222, and which is further known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, which is further known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a standard being developed by JVET, the Joint Video Experts Team).

[0084] Inputs to the elements of system 1000 may be provided via a variety of input devices, as shown in block 1130. Such input devices include, but are not limited to, (i) an RF section that receives a radio frequency (RF) signal transmitted, for example, over the air by a broadcast station, (ii) a Component (COMP) input terminal (or set of COMP input terminals), (iii) a Universal Serial Bus (USB) input terminal, and / or (iv) a High Definition Multimedia Interface (HDMI) input terminal. Other examples not shown in FIG. 4 include composite video.

[0085] In various examples, the input devices of block 1130 are associated with respective input processing elements known in the art. For example, the RF section can be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or band-limiting a signal to a band of frequencies), (ii) down-converting the selected signal, (iii) band-limiting again to a narrower band of frequencies to select a single frequency band, which in some examples can be referred to as a channel (e.g.), (iv) demodulating the down-converted band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream of data packets. The RF section in various examples includes one or more elements performing the functions just described, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section can include, for example, a tuner performing various functions, including down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency close to baseband) or to baseband. In one set-top box example, the RF section and associated input processing elements receive, filter, downconvert, and perform frequency selection by refiltering RF signals transmitted over a wired medium (e.g., cable) to the desired frequency band. Various examples rearrange the order of the above (and other) elements, remove some of the elements just mentioned, and / or add other elements that perform similar or different functions. Adding elements can include inserting elements between existing elements, such as, for example, inserting amplifiers and analog-to-digital converters. In various examples, the RF section includes an antenna.In an example, the device may include an apparatus herein and at least one of: (i) an antenna configured to receive a signal including data representing an image; (ii) a band limiter configured to limit the received signal to a band of frequencies including the data representing the image; or (iii) a display configured to display the image. A device configured to perform as an apparatus herein may include a television, a mobile phone, a tablet, or a set-top box (STB).

[0086] Additionally, the USB and / or HDMI terminals can include respective interface processors to interface the system 1000 to other electronic devices over USB and / or HDMI. It is understood that various aspects of the input processing, e.g., Reed-Solomon error correction, can be implemented, for example, in a separate input processing IC or within the processor 1010, as appropriate. Similarly, aspects of the USB or HDMI interface processing can be implemented in a separate input processing IC or within the processor 1010, as appropriate. The demodulated, error corrected, and demultiplexed stream is provided to various processing elements, including, for example, the processor 1010 and an encoder / decoder 1030, which operates in cooperation with memory and storage elements to process the data stream for presentation at an output device, as appropriate.

[0087] The various elements of the system 1000 can be provided in an integrated housing in which the various elements can be interconnected and transmit data therebetween using a suitable coupling arrangement 1140, such as an internal bus known in the art, including an I2C (Inter-IC) bus, wiring, and printed circuit boards.

[0088] The system 1000 includes a communication interface 1050 that enables communication with other devices over a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data over the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented within a wired and / or wireless medium, for example.

[0089] Data is streamed or otherwise provided to the system 1000 in various examples using a wireless network such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE citing the Institute of Electrical and Electronics Engineers). The Wi-Fi signal in the example just described is received via a communication channel 1060 and communication interface 1050 that are adapted for Wi-Fi communication. Typically, the communication channel 1060 in the example just described is coupled to an access point or router that provides access to external networks, including the Internet, to enable streaming applications and other over-the-top communications. Another example provides streamed data to the system 1000 using a set-top box that delivers data via an HDMI connection in the input block 1130. Still another example provides streamed data to the system 1000 using an RF connection in the input block 1130. As indicated above, various examples provide data in a non-streaming manner. Additionally, various examples use wireless networks other than Wi-Fi, e.g., a cellular network or a Bluetooth network.

[0090] The system 1000 can provide output signals to various output devices, including a display 1100, speakers 1110, and other peripheral devices 1120. The display 1100 in various examples includes one or more of a touch screen display, an organic light emitting diode (OLED) display, a curved display, and / or a foldable display, for example. The display 1100 can be for a television, a tablet, a laptop, a cellular (mobile) phone, or other device. Additionally, the display 1100 can be integrated with other components (e.g., as in a smartphone) or separate (e.g., an external monitor for a laptop). In various examples of examples, the other peripheral devices 1120 include one or more of a standalone digital video disc (or digital versatile disc) (DVR, for both terms), a disc player, a stereo system, and / or a lighting system. Various examples use one or more peripheral devices 1120 that provide functionality based on the output of the system 1000. For example, a disc player is responsible for playing the output of the system 1000.

[0091] In various examples, control signals are communicated between the system 1000 and the display 1100, speaker 1110, or other peripheral devices 1120 using signaling such as AV.Link, CEC (Consumer Electronics Control), or other communication protocols that allow device-to-device control with or without user intervention. The output devices can be communicatively connected to the system 1000 via dedicated connections through respective interfaces 1070, 1080, and 1090. Alternatively, the output devices can be coupled to the system 1000 via the communication interface 1050 using the communication channel 1060. The display 1100 and speaker 1110 can be integrated in a single unit with other components of the system 1000, such as a television. In various examples, the display interface 1070 includes a display driver, such as a timing controller (T Con) chip.

[0092] Alternatively, the display 1100 and speakers 1110 may be separate from one or more other components, for example if the RF portion of the input 1130 is part of a separate set-top box. In various examples where the display 1100 and speakers 1110 are external components, the output signals may be submitted via dedicated output connections including, for example, an HDMI port, a USB port, or a COMP output.

[0093] The examples can be performed by computer software implemented by the processor 1010, or by hardware, or by a combination of hardware and software. As a non-limiting example, the examples can be implemented by one or more integrated circuits. The memory 1020 can be of any type suitable for the technology environment and can be implemented using any suitable data storage technology, such as, as non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 1010 can be of any type suitable for the technology environment and can include, as non-limiting examples, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0094] Various implementations include decoding. As used herein, "decoding" can include all or part of the processing performed on the received encoding sequence to generate a final output suitable for display, for example. In various examples, the processing includes one or more of the processing typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various examples, the processing includes, in addition to or as an alternative, the processing performed by the decoder of the various implementations described herein, such as determining a corresponding prediction block based on a sub-partition width and a minimum prediction block width for the sub-partitions in the current block.

[0095] As a further example, in one example "decoding" refers only to "entropy decoding," in another example "decoding" refers only to differential decoding, and in another example "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding" is intended to refer specifically to a subset of operations or more generally, it is believed that the decoding process will be clear based on the context of the particular description and will be well understood by those of ordinary skill in the art.

[0096] Various implementations include encoding. In a manner similar to the above discussion of "decoding," "encoding" as used herein can include all or part of the processing performed on an input video sequence, for example, to generate an encoded bitstream. In various examples, the processing includes one or more of the processing typically performed by an encoder, such as partitioning, differential encoding, transforming, quantization, and entropy encoding. In various examples, the processing includes, in addition to or as an alternative, the processing performed by an encoder in various implementations described herein, such as determining a corresponding prediction block based on a sub-partition width and a minimum prediction block width for the sub-partitions in the current block.

[0097] As a further example, in one example "encoding" refers only to "entropy decoding," in another example "encoding" refers only to differential de-encoding, and in another example "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding" is intended to refer specifically to a subset of operations or more generally, it is believed that the encoding process will be clear based on the context of the particular description and will be well understood by those of ordinary skill in the art.

[0098] Note that the syntax elements used herein are descriptive terms; as noted above, they do not preclude the use of other syntax element names.

[0099] It should be understood that when a diagram is provided as a flow diagram, it also provides a block diagram of the corresponding apparatus. Similarly, when a diagram is provided as a block diagram, it should be understood that it also provides a flow diagram of the corresponding method / process.

[0100] Various examples cite rate-distortion optimization. In particular, during the encoding process, a balance or trade-off between rate and distortion is usually considered, often subject to computational complexity constraints. Typically, rate-distortion optimization is formulated as minimizing the rate-distortion function, which is a weighted sum of the rate and the distortion. There are different approaches to solving the rate-distortion optimization problem. For example, an approach may be based on an extensive test of all encoding options, including all considered modes or coding parameter values, together with a full evaluation of the coding cost and associated distortion of the reconstructed signal after encoding and decoding. Furthermore, faster approaches may be used, in particular with a calculation of the approximated distortion based on the predicted or predicted residual signal rather than the reconstructed one, in order to reduce the encoding complexity. A mixture of the two approaches just mentioned may also be used, for example by using the approximated distortion for only some of the possible encoding options and the full distortion for the other encoding options. Other approaches only evaluate a subset of the possible encoding options. More generally, many approaches use any of a variety of techniques to perform an optimization, but the optimization is not necessarily a complete evaluation of both the coding cost and the associated distortion.

[0101] The implementations and aspects described herein may be implemented, for example, in a method or process, an apparatus, a software program, a data stream, or a signal. Even if described only in the context of a single form of implementation (e.g., described only as a method), the implementation of the described features may also be implemented in other forms (e.g., an apparatus or a program). An apparatus may be implemented, for example, in appropriate hardware, software, and firmware. A method may be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Additionally, a processor also includes a communication device, such as, for example, a computer, a mobile phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate communication of information between end users.

[0102] Reference to "one example" or "example" or "one implementation" or "implementation" means that a particular feature, structure, characteristic, etc. described in connection with the example is included in at least one example, as well as other variations. Thus, appearances of the phrases "in one example" or "in an example" or "in one implementation" or "in an implementation" appearing in various places throughout this application are not necessarily all referring to the same example, as well as any other variations.

[0103] Additionally, the application may refer to "determining" various portions of information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory. Obtaining may include receiving, determining, identifying, and / or retrieving.

[0104] Additionally, the application may refer to "accessing" various portions of information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0105] Additionally, the application may refer to "receiving" various portions of information. Receiving, as with "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Additionally, "receiving" is typically included among operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information in some way or another.

[0106] It should be understood that the use of any of the following " / ", "and / or" and "at least one of" is intended to include, for example, in the case of "A / B", "A and / or B" and "at least one of A and B", the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C", the phrase is intended to include the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), or the selection of only the first and third listed alternatives (A and C), or the selection of the second and third listed alternatives (B and C), or the selection of all three alternatives (A and B and C). What has just been said may be expanded beyond those items merely listed, as would be apparent to one of ordinary skill in the art and related arts.

[0107] Furthermore, as used herein, the word "signaling" refers to, among other things, indicating something to a corresponding decoder. For example, in one example, an encoder signals a specific index indicating a selection of samples to be used to predict a sub-partition. In this way, in an example, the same parameters are used on both the encoder side and the decoder side. Thus, for example, an encoder can transmit a specific parameter to a decoder (explicit signaling), and the decoder can use the same specific parameter. Conversely, if the decoder already has a specific parameter as well as others, signaling can be used without transmission (implicit signaling) to simply enable the decoder to know and select the specific parameter. By avoiding the transmission of any actual function, bit savings are realized in various examples. It is understood that signaling can be accomplished in various ways. For example, one or more syntax elements, flags, etc. are used to signal information to a corresponding decoder in various examples. Although the foregoing has concerned the verb form of the word "signaling," the word "signaling" may also be used herein as a noun.

[0108] As would be apparent to one skilled in the art, the implementations can generate a variety of signals formatted to carry information that can be, for example, stored or transmitted. For example, the information can include instructions for performing a method or data generated by one of the described implementations. For example, the signals can be formatted to carry a bit stream of the described examples. For example, the signals can be formatted as electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or as baseband signals. For example, the formatting can include encoding a data stream and modulating a carrier wave with the encoded data stream. For example, the information the signals carry can be analog or digital information. The signals can be transmitted over a variety of separate wired or wireless links, as is known. The signals can be stored in a processor-readable medium.

[0109] We will describe a number of examples. The features of the examples just described may be provided alone or in any combination across the various claim categories and types. Additionally, the examples may include one or more of the following features, devices, or aspects alone or in any combination across the various claim categories and types.

[0110] Although features and elements are described above in certain combinations, one skilled in the art will understand that each feature or element can be used alone or in any combination with other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware embodied in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electrical signals (transmitted over wired or wireless connections) and computer-readable recording media. Examples of computer-readable recording media include, but are not limited to, described ROM, random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, and optical media such as magneto-optical media, e.g., CD-ROM disks and digital versatile disks (DVDs). A processor associated with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

[0111] An intra-coding tool, such as an intra-subpartition (ISP) tool, may be used. The ISP tool may be applied to the luma channel. The ISP tool may divide a CU into multiple subpartitions (e.g., two or four subpartitions). A subpartition may include multiple (e.g., at least 16) samples. The division may be performed horizontally and / or vertically. The CU-split may be performed horizontally and / or vertically. FIG. 5 illustrates an example of a division or CU-split that may be allowed for a CU (e.g., a CU of 4×8 pixels or a CU of 8×4 pixels). As illustrated in FIG. 5, the ISP tool may divide a CU into two subpartitions. The ISP split type may be vertical or horizontal. FIG. 6 illustrates an example of a division that may be allowed for a CU (e.g., a CU larger than 4×8 pixels or 8×4 pixels). As illustrated in Figure 6, for a relatively larger CU (e.g., a CU larger than 4x8 pixels or 8x4 pixels), the ISP tool may divide the CU into four subpartitions. An intra-prediction mode may be used for the subpartitions. For example, the same intra-prediction mode may be used for the subpartitions in a CU.

[0112] The MPM flag may be set (e.g., set to 1), for example, when the intra mode used for the ISP is a member of the most probable mode (MPM) list. The intra reference sample smoothing filter and / or position dependent intra prediction combination (PDPC) may be disabled for the ISP. Reference line zero may be used for the ISP. Multiple reference line intra prediction may be disabled.

[0113] For a horizontal split, the subpartitions may be processed from top to bottom. For a vertical split, the subpartitions may be processed from left to right. A subpartition may be predicted and a residual may be added to the predicted subpartition to generate, for example, a reconstructed subpartition. Samples from the reconstructed subpartition may be used to predict the next subpartition. FIG. 7 shows an example prediction diagram in Intra sub-partitions (ISP). In FIG. 7(a)-(c), a horizontally split subpartition may be processed. FIG. 7(a) illustrates a CU including four subpartitions A, B, C, and D. FIG. 7(b) illustrates that subpartition A may be predicted and reconstructed, and one or more samples in the bottom row of the reconstructed subpartition A may be used to predict subpartition B. FIG. 7(c) illustrates that subpartition B may be predicted and reconstructed, and one or more samples in the bottom row of the reconstructed subpartition B may be used to predict subpartition C. As illustrated in FIG. 7(b), subpartition A (e.g., the topmost subpartition) may be predicted using the above and left reference samples of the CU. Subpartition A may be reconstructed by adding a residual obtained after one or more entropy decoding, inverse quantization, and inverse transform. One or more samples in the bottom row of the reconstructed subpartition A may be used to predict subpartition B, followed by reconstruction of subpartition B. The prediction and reconstruction may be repeated for the next subpartition.

[0114] Respective MPM lists (e.g., separate MPM lists of size 6) may be maintained for reference line zero intra prediction, multiple reference line intra prediction, and ISP. MPM lists may be maintained to indicate (e.g., signal) the intra mode used. MPM list configurations utilized for reference line zero intra prediction, multiple reference line intra prediction, and ISP may be differentiated, for example, as illustrated in FIG. 8, FIG. 9, and FIG. 10a-10c. FIG. 8 is an example flowchart for MPM list configuration for reference line zero intra prediction. FIG. 9 is an example flowchart for MPM list configuration for multiple reference line intra prediction. FIG. 10a-10c illustrate example flowcharts for MPM list configuration for ISP. An MPM list configuration (e.g., as illustrated in Fig. 8, Fig. 9, Fig. 10a-10c) may consider the intra mode of the adjacent left CU (denoted by "A") and / or the intra mode of the upper CU (denoted by "B") during list configuration. Multiple conditions (e.g., four conditions) may be tested, for example, using intra modes "A" and "B". An MPM list may be generated if the conditions are satisfied. For example, each satisfied condition may result in a separate MPM list. Table 1 illustrates an example of four conditions and corresponding logical conditions (for intra modes A and B) that may be used in an MPM list configuration (e.g., three MPM list configurations as illustrated in Fig. 8, Fig. 9, Fig. 10a-10c).

[0115] [Table 1]

[0116] As illustrated in Figures 8, 9, and / or 10a-10c, an MPM list may include an index (e.g., from 0 to 5) and an associated intra mode. An index closer to zero may be closer to the top of the MPM list. For example, as illustrated in Figure 8, a default condition may result in an MPM list containing intra mode "A" as the 0th element. The minimum and maximum of "A" and "B" are indicated by minAB and maxAB, respectively, and "%" indicates modulo operation. As illustrated in Figures 10a-10c, for example in an ISP, the type of split (e.g., horizontal split or vertical split) may be taken into account in the MPM configuration.

[0117] FIG. 11 is an exemplary flowchart for unified MPM list construction. FIG. 11 illustrates unified MPM list construction for multiple intra-coding tools (e.g., reference line zero intra prediction, multiple reference line intra prediction, and ISP). The unified MPM list may be modified for multiple reference line intra prediction and ISP. In the case of multiple reference line intra prediction, DC mode and planar mode may be skipped from the unified MPM list, for example, because DC and planar modes are not used for multiple reference line intra prediction. In the case of ISP, DC mode may be skipped from the unified MPM list, for example, because DC mode is not used for ISP, or may be replaced by angular mode in the unified MPM list.

[0118] In ISP, subpartitions may be predicted (e.g., predicted consecutively) and reconstructed from top to bottom or from left to right based on the ISP split type. For example, subpartitions may be predicted and reconstructed from top to bottom when a horizontal split is used. Subpartitions may be predicted and reconstructed from left to right when a vertical split is used. A subpartition (e.g., a subsequent subpartition) may be predicted, for example, when a previous subpartition is reconstructed. In some examples, prediction of a subpartition may be independent of the reconstruction of adjacent or previous subpartitions. Latency may be reduced.

[0119] The MPM list construction process(es) may correspond to reference line zero intra prediction, multi reference line intra prediction, and ISP. In an example, the respective MPM list construction processes for different intra coding tools (e.g., reference line zero intra prediction, multi reference line intra prediction, and ISP) may be unified. The unified MPM list may be based on multiple conditions. When each condition is satisfied, the MPM list may change. For example, each condition may result in a different MPM list. The unified MPM list may be constructed.

[0120] Systems, methods, and means are disclosed for implementing an ISP, for example, by reducing latency in the ISP and / or simplifying the MPM list construction process. In some examples, a subsequent subpartition may be predicted using predicted samples from a previous adjacent subpartition. In some examples, a subsequent subpartition may be predicted using predicted or reconstructed samples from a previous adjacent subpartition. Furthermore, one or more examples herein relating to horizontal partitioning may also be applied to vertical partitioning. In vertical partitioning, relevant rows may be replaced with columns, and relevant bottom rows may be replaced with right columns. One or more examples herein may reduce latency.

[0121] Prediction samples of a previous subpartition may be used as reference samples for prediction. The ISP may perform prediction and reconstruction of subpartitions consecutively. In some examples, prediction of a subpartition may not depend on reconstructed samples from adjacent subpartitions. The first subpartition may be predicted using a reference row above and a reference column to the left that border the CU. The first subpartition may be the topmost subpartition for a horizontal split. The first subpartition may be the leftmost subpartition for a vertical split. Prediction may be performed on the remaining subpartitions.

[0122] CU-based prediction may be performed to predict, for example, a subpartition, a transform unit (TU), or a prediction unit (PU). For example, if N indicates the number of subpartitions in a CU, in CU-based prediction, prediction may be performed as follows: For each of the first N-1 subpartitions, the bottom row (for example, for a horizontal split) or the right column (for example, for a vertical split) may be predicted. Figure 12, including (a) and (b), illustrates an exemplary CU-based prediction where the bottom row of subpartitions A, B, and C may be predicted using samples at the top and left reference lines bordering the CU. As illustrated in Figure 12, a CU may be horizontally divided into four subpartitions A, B, C, and D. For example, the bottom row of each of the first three subpartitions A, B, and C may be predicted as illustrated in Figures 12(a) and (b). The just-mentioned row (or column in case of vertical split) may be predicted using the upper reference row and the left reference column bordering the CU. The just-mentioned row (or column in case of vertical split) may be predicted by the same prediction mode as the intra mode of the CU. The predicted row (or column) may be used to predict an adjacent subpartition. For example, as illustrated in FIG. 12(b), the predicted bottom row sample of subpartition B may be used to predict subpartition C. Each of the subpartitions (e.g., N-1 subpartitions) of the CU may be predicted.

[0123] Reconstruction may be performed on a subpartition basis, for example, once prediction is complete, or on an entire CU. If reconstruction is performed on a subpartition basis, the residual may be generated by applying inverse quantization and inverse transform on a subpartition basis. If reconstruction processing is performed on a CU (e.g., the entire CU), the residual may be generated for the entire CU. CU-based prediction may reduce latency, for example, for an ISP.

[0124] Sub-partition-based prediction may be performed. Prediction of sub-partitions may be performed successively, for example, using prediction samples from previous sub-partitions. FIG. 13 illustrates an example of sub-partition-based prediction. As illustrated in FIG. 13, a CU may be horizontally divided into four sub-partitions A, B, C, and D. As illustrated in FIG. 13, a first sub-partition may be predicted using the upper reference row and the left reference column bordering the CU. In an example, the first sub-partition may be the topmost sub-partition A. As illustrated in FIG. 13(a), prediction samples from the bottom row of A may be used to predict sub-partition B, together with the left reference column. With reference to FIG. 13(b), prediction samples of the bottom row of sub-partition B (e.g., together with the left reference column) may be used to predict sub-partition C. The process may be repeated until the last sub-partition is predicted. Subpartition-based prediction, as illustrated in FIG. 13, may be performed on subpartitions that are divided vertically.

[0125] Reconstruction may be performed on a subpartition basis, for example, once prediction is complete, or on an entire CU. If reconstruction is performed on a subpartition basis, the residual may be generated by applying inverse quantization and inverse transform on a subpartition basis. If reconstruction processing is performed on a CU (e.g., the entire CU), the residual may be generated for the entire CU. CU-based prediction may reduce latency, for example, for an ISP.

[0126] 14(a)-(c) are diagrams of examples of pipeline stages for (a) ISP, (b) ISP with subpartition-based prediction, and (c) ISP with multiple or multi-subpartition-based prediction. FIG. 14(a) and FIG. 14(b) illustrate a pipeline for an ISP and a pipeline for an ISP that may utilize subpartition-based prediction when considering a CU with subpartitions A and B. FIG. 14(a) and FIG. 14(b) illustrate pipeline stages of prediction, inverse quantization and inverse transform, and reconstruction. FIG. 14(c) illustrates pipeline stages of an ISP (e.g., for the first ISP) based on multi-subpartition intra prediction. Prediction of subpartition B may not depend on reconstruction of subpartition A. Latency reduction may be achieved, for example, as illustrated in FIG. 14(a)-(c).

[0127] Hybrid prediction may be performed, for example, to predict a subpartition, TU, or PU. A subpartition may be predicted using predicted samples from adjacent subpartitions or reconstructed samples from adjacent subpartitions. For predicted samples, CU-based prediction or subpartition-based prediction may be used. FIG. 15 illustrates an example of hybrid prediction where a subpartition may be predicted using reconstructed or predicted samples from adjacent subpartitions. As illustrated in FIG. 15, the bottom row of subpartition A may be predicted, for example, using CU-based prediction as described herein. Residuals of subpartitions A, B, C, and D may be determined, for example, by utilizing inverse quantization and inverse transform operations. Subpartitions A and B may be predicted, for example, after the residuals are obtained. Subpartition A may be predicted using the top reference row and the left reference column bordering the CU. Subpartition B may be predicted using the bottom predicted row of subpartition A and the left reference column of the CU. Subpartition A and subpartition B may be reconstructed. The reconstructed rows at the bottom and reference columns to the left of subpartition B may be used to predict subpartition C. The predicted rows at the bottom and reference columns to the left of subpartition C may be used to predict subpartition D, e.g., using subpartition-based prediction as described herein.

[0128] The last subpartition may be predicted from samples reconstructed from adjacent subpartitions, e.g., as described herein. For example, the last subpartition may be the furthest from one of the reference lines. In a horizontal split, the bottom subpartition may be the furthest from the upper reference line. The accuracy of the prediction of the last subpartition may be improved.

[0129] Hybrid prediction may be used to predict a CU. For example, some subpartition(s) of a CU may be predicted based on reconstructed samples of another subpartition(s), and some subpartition(s) of a CU may be predicted based on predicted samples of other subpartition(s). Figure 16 illustrates an example of hybrid prediction. In Figure 16, subpartition D may be predicted from reconstructed samples of the bottom row of subpartition C. Subpartition-based prediction and / or CU-based prediction may be used. As illustrated in Figure 16(a), subpartition C may be predicted from predicted samples of the bottom row of subpartition B by utilizing subpartition-based prediction. As illustrated in Figure 16(b), subpartition C may be predicted from predicted samples of the bottom row of subpartition B by utilizing CU-based prediction.

[0130] A prediction type (e.g., CU-based prediction, subpartition-based prediction, or hybrid prediction) may be selected based on the CU size. For example, for larger CUs (e.g., CUs of size greater than 16×16 pixels), hybrid prediction may be used. For smaller CUs (e.g., CUs of size up to 16×16 pixels), subpartition-based prediction may be used.

[0131] The prediction type may be selected based on the intra mode: for intra angular modes, sub-partition-based prediction may be used, while for non-angular modes, CU-based prediction may be used.

[0132] One or more prediction types described herein may be used with normal ISP prediction, in which prediction and reconstruction of subpartitions are performed consecutively (e.g., from top to bottom or from left to right). Subpartition-based prediction described herein may be applied to relatively small CUs, e.g., CUs with a size of 8×4 pixels or 4×8 pixels, and normal ISP may be used for larger CUs (e.g., CUs with a size larger than 8×4 pixels or 4×8 pixels). Sometimes, subpartition-based prediction may be applied to different CU sizes (e.g., small CUs and large CUs). In an example, normal ISP may be disabled for relatively small CUs, e.g., CUs with a size of 8×4 pixels and 4×8 pixels, while ISP may be applied to larger CUs (e.g., CUs with a size larger than 8×4 pixels and 4×8 pixels).

[0133] In an example, the samples used to predict the subpartitions may be adaptively selected. For example, the selected samples may be reconstructed samples, samples predicted from CU-based prediction, and samples predicted from subpartition-based prediction. The index may be signaled in the bitstream to the video decoding device. The index may be signaled for each CU. The index may indicate the selection of the sample used by the video encoding device. Table 2 illustrates an example of selections available in an adaptive scheme when considering, for example, a CU with four subpartitions. As illustrated in Table 2, index 2 uses CU-based prediction for subpartition B, subpartition-based prediction for subpartition C, and reconstructed sample-based prediction (e.g., ISP) for subpartition D. As illustrated in Table 2, the adaptive scheme may limit subpartition A to use ISP and subpartition B to use CU-based prediction. As further illustrated in Table 2, the number of selections in the adaptive scheme may be limited to a subset of the listed selections. The signaling overhead for the index may be reduced. For example, the number of selections may be limited to four selections corresponding to indexes 0, 1, 2, and 4, as shown in Table 2. The accuracy of prediction for the last subpartition D may be improved with the limited selection just mentioned.

[0134] [Table 2]

[0135] Intra prediction of a sub-partition may be based on reference samples adjacent to the current block, e.g., adjacent reference samples of a CU. Figure 17 illustrates an example of multi-sub-partition based intra-prediction. Intra prediction of two or more sub-partitions may be performed in parallel. As illustrated in Figure 17, intra prediction of two or more sub-partitions may be performed simultaneously (e.g., all together) from adjacent reference samples of a CU associated with each sub-partition. Reconstruction latency for each of the ISP sub-partitions may be reduced. Reconstruction of multiple consecutive ISP sub-partitions may be performed in parallel. Figure 14(c) illustrates pipeline stages of an ISP based on multi-sub-partition intra-prediction (e.g., for the first ISP).

[0136] An intra-predicted PU may include a transform unit (TU). PUs and TUs may be of the same size. In some examples, one PU may include two or more TUs. A TU may include a subpartition. By merging multiple TUs into a larger size PU (e.g., larger than an individual TU), PU intra prediction for multiple TUs may be performed simultaneously. Decoding / reconstruction of the TUs may be performed in parallel. Decoding and / or processing latency for the TUs may be reduced.

[0137] The device may perform one or more of CU-based intra prediction, subpartition-based intra prediction, and multiple subpartition-based intra prediction. FIG. 18 illustrates various examples of ISP subpartitions for horizontally split CUs and vertically split CUs. As illustrated in FIG. 18, ISP may be applied to CUs of various sizes. FIG. 18 gives examples of various vertically split CUs and horizontally split CUs, e.g., 4×8, 4×N, 8×4, and 8×N. The device may receive an indication of an indication that a subpartition type (e.g., ISP type) is either horizontally split or vertically split. The device may perform prediction based on vertical split or horizontal split, e.g., according to one or more of the examples herein. Based on the indication that the subpartition type is horizontally split, the device may perform subpartition in the horizontal direction. Based on the indication that the subpartition type is vertically split, the device may perform subpartition in the vertical direction. In an example, the device may predict a prediction block based on the type of ISP.

[0138] A PU for a coding block may have a prediction block width. The prediction block width may refer to the number of samples measured from one side of the PU to the other side. The minimum prediction block width may be used in conjunction with prediction of a subpartition. One PU may include two or more subpartitions (e.g., TU subpartitions) that are vertically or horizontally split, as illustrated in Figures 19C and 19D. The prediction block width may be the subpartition width or a multiple of the subpartition width, as illustrated in Figures 19C and 19D, for example. The subpartition width and the minimum prediction block width may be used to determine a prediction block for prediction. For example, when the subpartition type is vertical split, a prediction block having multiple subpartitions may be used for prediction when the subpartition width is less than the minimum prediction block width. As illustrated in Figures 19C and 19D, the minimum prediction block width may be 4 samples. The prediction block width may be 4 samples when the subpartition width is less than 4 samples. When the sub-partition width is greater than or equal to the minimum prediction block width, the prediction block having the sub-partition may be used for prediction. The prediction block width may be the sub-partition width.

[0139] FIG. 19A illustrates examples where 1×N (N≧16) and 2×N (N≧8) subpartitions as illustrated in FIG. 18 may be removed and replaced with 4×N (e.g., N>8) subpartitions. A 1×N subpartition may represent a subpartition that is 1 sample wide and N samples high. A 2×N subpartition may represent a subpartition that is 2 samples wide and N samples high. A 4×N subpartition may represent a subpartition that is 4 samples wide and N samples high.

[0140] For a 4×N subpartition, the subpartition width is 4 samples. For a 4×N subpartition, the subpartition height is N samples. The subpartition width and / or the subpartition height may be used to subpartition the current block for intra prediction. The subpartition width for the current block may be determined based at least in part on the ISP type for the current block being either vertical split or horizontal split, as shown in FIG. 18.

[0141] The minimum prediction block width may be 4 samples. As described herein, a PU may include one or more horizontally or vertically divided subpartitions. A PU may include one or more 4×N subpartitions, for example, since 1×N (N≧16) subpartitions and 2×N (N≧8) subpartitions are replaced with 4×N subpartitions. ISP using multiple subpartition-based intra prediction may be performed in conjunction with the example shown in FIG. 19A. The minimum prediction block width may be used to determine a PU corresponding to each subpartition.

[0142] The prediction of the predicted block may be based on the neighboring samples of the CU and / or the reconstructed subpartition. As illustrated in FIG. 19A, replacing the 1×N (N≧16) subpartition and / or the 2×N (N≧8) subpartition with the 4×N subpartition may result in no vertical split ISP for the 4×8, 4×N (N≧16) CU. The prediction of the subpartition may be based on a prediction unit that is 4 samples or more wide. The prediction of the predicted block in the above 4×8 or 4×N (N≧16) CU may be based on the neighboring samples of the CU and may not be based on the reconstructed subpartition, for example. Replacing the 1×N (N≧16) subpartition and the 2×N (N≧8) subpartition with the 4×N subpartition may result in two 4×N vertical split subpartitions for the 8×N (N≧8) CU, which is highlighted by bold and underlined typeface. Prediction of a predicted block in the above 8×N (N≧8) CU may be based on neighboring samples and reconstructed sub-partitions of the CU.

[0143] A PU may contain multiple TU subpartitions for a certain CU size. A 4×8 CU may represent a CU or coding block that is 4 samples wide and 8 samples high. A 4×N CU may represent a CU or coding block that is 4 samples wide and N samples high. A PU may contain multiple TU subpartitions for CU sizes of, for example, 4×8, 4×N, 8×4, and 8×N.

[0144] In one or more examples herein, CUs and coding blocks may be used interchangeably, PUs and prediction blocks may be used interchangeably, and TUs and transform blocks may be used interchangeably.

[0145] In an example, the device may determine that ISP is enabled for the current block. The device may determine a subpartition width for the current block. For a subpartition in the current block, the device may determine a corresponding prediction block based on the subpartition width and the minimum prediction block width. The device may predict the prediction block using reference samples neighboring the prediction block. For example, the reference samples may include neighboring reference samples of a CU and / or samples from a reconstructed subpartition.

[0146] A reference sample may be adjacent to a CU, PU, ​​or TU. For example, a reference sample adjacent to a PU may include a reference sample that is adjacent to the PU or spatially separated from the PU. A reference sample adjacent to a PU may include a reference sample that is in an adjacent subpartition. A reference sample adjacent to a PU may include a reference sample that is in an adjacent CU, PU, ​​or TU. An adjacent subpartition may be adjacent to the PU or spatially separated from the PU.

[0147] An N×2 (N≧8) subpartition may be removed and replaced with an N×4 subpartition. FIG. 19B illustrates an example in which an N×2 (N≧8) subpartition may be removed and replaced with an N×4 subpartition, as illustrated in FIG. 18, in addition to replacing a 1×N (N≧16) subpartition and a 2×N (N≧8) subpartition with a 4×N subpartition, as illustrated in FIG. 19A. As illustrated in FIG. 19B, for example, in addition to there being no vertical split ISP for 4×8, 4×N (N≧16) CUs, there is no horizontal split ISP that may be applied to an 8×4 CU. Two N×4 horizontal split subpartitions may be applied to an N×8 (N≧8) CU, which are highlighted by bold and underlined typefaces. ISP using multiple subpartition-based intra prediction may be performed in conjunction with the example illustrated in FIG. 19B.

[0148] A PU may include 32 or more samples. FIG. 19C illustrates an example in which a PU may include 32 samples, for example, in addition to replacing 1×N (N≧16) and 2×N (N≧8) subpartitions with 4×N subpartitions as illustrated in FIG. 19A. The change just mentioned from 16 samples per PU to 32 samples per PU may be achieved, for example, by enforcing a minimum PU size for intra prediction. In an example, the minimum PU size may include a 4×8 or 8×4 PU size, which is highlighted by bold and underlined typeface in FIG. 19C. As illustrated in FIG. 19C, a minimum size PU (e.g., a 4×8 PU or an 8×4 PU) may include one TU subpartition of the same PU size, or two TU subpartitions of half PU size (e.g., 4×4). ISP using multiple subpartition-based intra prediction may be performed in conjunction with the example illustrated in FIG. 19C. Multi-sub-partition prediction as described herein may be performed, for example, when a PU may include 32 or more samples.

[0149] Figure 19D illustrates an example where the examples illustrated in Figures 19A, 19B, and 19C may be applied simultaneously. In the example, removal of 1xN, 2xN, and Nx2 sub-partitions may be applied simultaneously, for example, with enforcement of a minimum PU size of 4x8 or 8x4, along with a worst-case throughput of an ISP of 32 samples per PU cycle. ISP using multiple sub-partition-based intra prediction may be performed in conjunction with the example shown in Figure 19D.

[0150] Systems, methods, and means may be provided for MPM list configuration. FIG. 20 illustrates an example of a unified MPM list generation / configuration. In the example of a unified MPM list configuration, multiple conditions (e.g., the four conditions listed in Table 1) may be tested to determine a unified MPM list. For example, conditions 1 and 3 listed in Table 1 may be combined as a single condition. Table 3 illustrates an example of conditions for unified MPM list configuration. As illustrated in Table 3, a single condition, for example, condition (b), may be utilized to check whether either intra mode A or intra mode B is an angular mode.

[0151] [Table 3]

[0152] In an example, the MPM list generated upon satisfaction of a condition listed in Table 3 (e.g., condition (b)) may be the same as the MPM list generated upon satisfaction of a condition listed in Table 1 (e.g., condition 3), as illustrated in Figure 11. An exemplary unified MPM list configuration is illustrated in Figure 20.

Claims

1. determining that an intra-subpartition (ISP) is enabled for the current block; determining a subpartition width for the subpartition obtained via the ISP; determining a predicted block width based on the sub-partition width and a minimum predicted block width, the predicted block width being equal to the minimum predicted block width if the sub-partition width is smaller than the minimum predicted block width, and the predicted block width being equal to the sub-partition width if the sub-partition width is greater than or equal to the minimum predicted block width; predicting the sub-partition based on the predicted block width; Decoding the current block based on the predicted subpartition. Processor configured to A video decoding device comprising:

2. 2. The video decoding device according to claim 1, wherein the minimum prediction block width is four samples.

3. 2. The video decoding device of claim 1, wherein when the current block width is 4 samples and the current block height is 8 samples or more, or when the current block width is 8 samples and the current block height is 8 samples or more, the predicted block for the sub-partition includes a plurality of transform block sub-partitions.

4. 2. The video decoding apparatus of claim 1, wherein the sub-partition is predicted using adjacent samples of the current block and samples from the sub-partition being reconstructed in the current block.

5. 2. The video decoding device of claim 1, wherein when the current block width is 8 samples and the current block height is 8 samples or more, the subpartition is predicted using adjacent samples of the current block and samples from the subpartition being reconstructed in the current block.

6. 2. The video decoding device of claim 1, wherein the prediction block includes a transport block, the transport block includes the sub-partition, and a width of the transport block is the sub-partition width.

7. determining that an intra sub-partition (ISP) is enabled for the current block; determining a subpartition width for a subpartition obtained via an ISP; determining a predicted block width based on the sub-partition width and a minimum predicted block width, the predicted block width being equal to the minimum predicted block width if the sub-partition width is smaller than the minimum predicted block width, and the predicted block width being equal to the sub-partition width if the sub-partition width is greater than or equal to the minimum predicted block width; predicting the sub-partition based on the predicted block width; decoding the current block based on the predicted sub-partition; 13. A video decoding method comprising:

8. 8. The video decoding method of claim 7, wherein the minimum prediction block width is four samples.

9. 8. The video decoding method of claim 7, wherein when the current block width is 4 samples and the current block height is 8 samples or more, or when the current block width is 8 samples and the current block height is 8 samples or more, the prediction block for the sub-partition includes a plurality of transform block sub-partitions.

10. determining that an intra-subpartition (ISP) is enabled for the current block; determining a subpartition width for the subpartition obtained via the ISP; determining a predicted block width based on the sub-partition width and a minimum predicted block width, the predicted block width being equal to the minimum predicted block width if the sub-partition width is smaller than the minimum predicted block width, and the predicted block width being equal to the sub-partition width if the sub-partition width is greater than or equal to the minimum predicted block width; predicting the sub-partition based on the predicted block width; Encoding the current block based on the predicted subpartition. Processor configured to A video encoding device comprising:

11. 11. The video encoding apparatus of claim 10, wherein the minimum prediction block width is four samples.

12. 11. The video encoding apparatus of claim 10, wherein when the current block width is 4 samples and the current block height is 8 samples or more, or when the current block width is 8 samples and the current block height is 8 samples or more, the predicted block for the sub-partition includes a plurality of transform block sub-partitions.

13. determining that an intra sub-partition (ISP) is enabled for the current block; determining a subpartition width for a subpartition obtained via an ISP; determining a predicted block width based on the sub-partition width and a minimum predicted block width, the predicted block width being equal to the minimum predicted block width if the sub-partition width is smaller than the minimum predicted block width, and the predicted block width being equal to the sub-partition width if the sub-partition width is greater than or equal to the minimum predicted block width; predicting the sub-partition based on the predicted block width; encoding the current block based on the predicted subpartition; A video encoding method comprising:

14. 14. The method of claim 13, wherein the sub-partition is predicted using neighboring samples of the current block and samples from the sub-partition being reconstructed in the current block.

Images