Intra sub-partition in video coding

Abstract

The invention relates to an intra sub-partition in video coding. An intra sub-partition (ISP) may be enabled for a current block, e.g., based on an ISP indication. The block may be partitioned into a plurality of sub-partitions, and the sub-partitions may belong to a prediction unit (PU). A sub-partition width and a minimum prediction block width of the current block may be obtained. A PU corresponding to the current sub-partition may be determined based on the sub-partition width and the minimum prediction block width. For example, when a sub-partition width is less than a minimum prediction block width, the PU may include multiple sub-partitions. In an example, the minimum prediction block width may be four samples. A reference sample may be determined, and the PU may be predicted using the reference sample.

Description

[0001] Cross-reference to related applications This application claims the benefit of U.S. Provisional Patent Application Serial No. 62 / 816,548, filed March 11, 2019, and U.S. Provisional Patent Application Serial No. 62 / 860,122, filed June 11, 2019, the contents of which are incorporated herein by reference in their entirety. Background Technology

[0002] Video decoding systems can be used to compress digital video signals, for example, to reduce the storage and / or transmission bandwidth required for such signals. Video decoding systems can include block-based, wavelet-based, and / or object-based systems. The system can employ video decoding techniques, such as certain predictive techniques. Summary of the Invention

[0003] Intra-sub-partition (ISP) can be an intra-frame decoding tool. Systems, methods, and means are provided where sub-partition prediction can be partially or completely independent of the reconstruction of previous neighboring sub-partitions. Sub-partitions can be predicted using samples from neighboring sub-partitions. Sub-partitions can be predicted using both predicted and reconstructed samples from previous neighboring sub-partitions.

[0004] Prediction based on multiple sub-partitions can be used, where intra-frame prediction of two or more sub-partitions can be performed simultaneously from relevant video blocks (e.g., reference samples adjacent to the decoding unit (CU)). Prediction units (PUs) comprising multiple sub-partitions can be predicted using reference samples adjacent to the PU.

[0005] ISP can be enabled for the current block, for example, based on ISP indication. The sub-partition width of the current block can be determined. The sub-partition width can be used to sub-partition the current block for intra-frame prediction. Sub-partitions can belong to PUs. The PU corresponding to a sub-partition can be determined based on the sub-partition width and the minimum prediction block width. In the example, the minimum prediction block width could be four samples. Prediction can be performed based on PUs. For example, a reference sample can be determined, and this reference sample can be used to predict PUs. The reference sample can be adjacent to a PU.

[0006] The minimum prediction block width can be used in conjunction with the predictions of sub-partitions. A PU can include two or more sub-partitions that are vertically or horizontally divided. The prediction block width can be the sub-partition width or a multiple of the sub-partition width. The sub-partition width and the minimum prediction block width can be used to determine the prediction block. For example, when the sub-partition width is less than the minimum prediction block width, a prediction block with multiple sub-partitions can be used for prediction. For instance, when the minimum prediction block width is 4 samples, a prediction block with a width of 4 samples can be used for prediction when the sub-partition width is less than 4 samples.

[0007] A PU can contain multiple transform block sub-partitions. For example, for a CU with sizes of 4×8, 4×N (e.g., N>8), and 8×N, the PU in the corresponding CU can contain multiple transform block sub-partitions.

[0008] The PU of a CU can be predicted using neighboring samples of the CU and / or samples from reconstructed sub-partitions. For example, for CUs of 4×8 and 4×N size, a video decoder or encoder can predict the PU using neighboring samples of the CU. For a CU of 8×N size, a video decoder or encoder can predict the PU using neighboring samples of the CU and samples from reconstructed sub-partitions within the CU.

[0009] A method may include determining that ISP is enabled for a current block. The method may include determining the sub-partition width of the current block. The method may include determining a corresponding prediction block for a sub-partition within the current block based on the sub-partition width and a minimum prediction block width. The method may include determining a reference sample. The method may include using the reference sample to predict the prediction block. The methods described herein can be performed by a decoder. In some examples, the methods described herein, or corresponding methods, can be performed by an encoder.

[0010] A computer-readable medium may include instructions for causing one or more processors to perform one or more of the methods described herein.

[0011] A computer program product includes instructions that, when executed by one or more processors, cause one or more processors to perform one or more of the methods described herein. Attached Figure Description

[0012] Figure 1A This is a system diagram illustrating an example communication system in which one or more of the disclosed embodiments may be implemented.

[0013] Figure 1B It is shown that, according to the embodiment, it is possible to Figure 1A The system diagram shown is of an example wireless transmit / receive unit (WTRU) used in the communication system.

[0014] Figure 1C It is shown that, according to the embodiment, it is possible to Figure 1A The system diagram shows an example radio access network (RAN) and an example core network (CN) used in the communication system shown.

[0015] Figure 1D It is shown that, according to the embodiment, it is possible to Figure 1A The system diagram shows another example RAN and another example CN used in the communication system shown.

[0016] Figure 2 This is a diagram of an example block-based video encoder.

[0017] Figure 3 This is a diagram of an example video decoder.

[0018] Figure 4 A block diagram of an example system in which various aspects and examples are implemented is shown.

[0019] Figure 5 Examples of partitions or CU segmentation that may be allowed for a CU (e.g., a 4×8 pixel CU or an 8×4 pixel CU) are shown.

[0020] Figure 6 An example of a partition that can be allowed for a CU (e.g., a CU larger than 4×8 pixels or 8×4 pixels) is shown.

[0021] Figure 7 The diagrams (a), (b), and (c) show example predictions in the intra-fractional sub-segment (ISP).

[0022] Figure 8 This is an example flowchart for constructing a list of most probable modes (MPMs) for reference line zero-frame intra-prediction.

[0023] Figure 9 This is an example flowchart for constructing an MPM list for intra-frame prediction using multiple reference lines.

[0024] Figures 10a-10c A sample flowchart for constructing an MPM list for an ISP is shown.

[0025] Figure 11 This is an example flowchart for constructing a unified MPM list.

[0026] Figure 12 Examples of CU-based predictions are shown in graphs (a) and (b), where the bottom rows of sub-partitions A, B, and C are predicted using reference lines above and to the left of adjacent CUs.

[0027] Figure 13 The diagrams (a) and (b) show examples of sub-partition-based predictions, where the bottom row of sub-partition A in (a) is used to predict sub-partition B, and the bottom row of sub-partition B in (b) is used to predict sub-partition C.

[0028] Figure 14(a) is a schematic diagram of an example of the pipeline stage for an ISP.

[0029] Figure 14(b) is a schematic diagram of an example of a pipeline stage for an ISP using sub-partition-based prediction.

[0030] Figure 14(c) is a schematic diagram of an example of a pipeline stage for an ISP using predictions based on multiple sub-partitions or predictions based on multiple sub-partitions.

[0031] Figure 15 This is a graph illustrating an example of hybrid prediction, where sub-partitions can be predicted using either reconstructed samples or predicted samples from neighboring sub-partitions.

[0032] Figure 16 Figures (a) and (b) show example mixed predictions.

[0033] Figure 17 Examples of prediction based on multiple sub-partitions or intra-frame prediction based on multiple sub-partitions are shown.

[0034] Figure 18 Examples of ISP sub-partitions for horizontally and vertically partitioned CUs are shown.

[0035] Figure 19A It shows that Figure 18 The example shown is where 1×N (N≥16) and 2×N (N≥8) sub-partitions can be removed and replaced by 4×N sub-partitions.

[0036] Figure 19B It shows that Figure 18 The example shown illustrates how an N×2 (N≥8) subpartition can be removed and replaced by an N×4 subpartition.

[0037] Figure 19C An example is shown where a PU can include 32 samples.

[0038] Figure 19D It shows that they can be applied simultaneously. Figure 19A , 19B And an example of the example shown in 19C.

[0039] Figure 20 An example of generating / constructing a unified MPM list is shown. Detailed Implementation

[0040] A detailed description of illustrative embodiments will now be described with reference to the accompanying drawings. While such description provides detailed examples of possible implementations, it should be noted that these details are intended to be illustrative and in no way intended to limit the scope of this application.

[0041] Figure 1AThis diagram illustrates an example communication system 500 in which one or more of the disclosed embodiments may be implemented. The communication system 500 may be a multi-access system providing content such as voice, data, video, messaging, and broadcasting to multiple wireless users. The communication system 500 enables multiple wireless users to access such content by sharing system resources, including wireless bandwidth. For example, the communication system 500 may employ one or more channel access methods, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal FDMA (OFDMA), Single Carrier FDMA (SC-FDMA), Zero-Tail Unique Word DFT Extended OFDM (ZT-UW DTS-s OFDM), Unique Word OFDM (UW-OFDM), Resource Block Filtered OFDM, Filter Bank Multicarrier (FBMC), etc.

[0042] like Figure 1A As shown, the communication system 500 may include wireless transmit / receive units (WTRUs) 502a, 502b, 502c, 502d, RAN 504 / 513, CN 506 / 515, public switched telephone network (PSTN) 508, Internet 510, and other networks 512. However, it should be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each WTRU 502a, 502b, 502c, 502d may be any type of device configured to operate and / or communicate in a wireless environment. For example, WTRU502a, 502b, 502c, and 502d (any of which may be referred to as a “station” and / or “STA”) may be configured to transmit and / or receive wireless signals and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscription-based units, pagers, cellular phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or MiFi devices, Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated processing chain environments), consumer electronics devices, devices operating on commercial and / or industrial wireless networks, and so on. Any WTRU502a, 502b, 502c, and 502d may be interchangeably referred to as a UE.

[0043] The communication system 500 may also include base station 514a and / or base station 514b. Each of base stations 514a and 514b may be any type of device configured to wirelessly connect to at least one of WTRUs 502a, 502b, 502c, and 502d to facilitate access to one or more communication networks, such as CN506 / 515, the Internet 510, and / or other networks 512. As an example, base stations 514a and 514b may be base transceiver stations (BTS), node B, e-node B, home node B, home e-node B, gNB, NR node B, site controller, access point (AP), wireless router, etc. Although base stations 514a and 514b are each depicted as a single element, it will be understood that base stations 514a and 514b may include any number of interconnected base stations and / or network elements.

[0044] Base station 514a may be part of RAN 504 / 513, and may also include other base stations and / or network elements (not shown), such as base station controllers (BSCs), radio network controllers (RNCs), relay nodes, etc. Base station 514a and / or base station 514b may be configured to transmit and / or receive radio signals on one or more carrier frequencies, which may be referred to as cells (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage of a specific geographic area, which may be relatively fixed or may change over time. A cell may be further divided into cell sectors. For example, the cell associated with base station 514a may be divided into three sectors. Thus, in one embodiment, base station 514a may include three transceivers, i.e., one transceiver per sector of the cell. In embodiments, base station 514a may employ multiple-input multiple-output (MIMO) technology and may use multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in a desired spatial direction.

[0045] Base stations 514a and 514b can communicate with one or more of WTRUs 502a, 502b, 502c, and 502d via air interface 516, which can be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). Air interface 516 can be established using any suitable radio access technology (RAT).

[0046] More specifically, as described above, the communication system 500 can be a multi-access system and can employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, base stations 514a and WTRUs 502a, 502b, and 502c in RAN504 / 513 can implement radio technologies such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which can use Wideband CDMA (WCDMA) to establish air interfaces 515 / 516 / 517. WCDMA can include communication protocols such as High-Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA can include High-Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High-Speed ​​UL Packet Access (HSUPA).

[0047] In the embodiment, base station 514a and WTRUs 502a, 502b, 502c can implement radio technologies such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which can use Long Term Evolution (LTE) and / or LTE-A Advanced (LTE-A) and / or LTE-A Pro Advanced (LTE-A Pro) to establish air interface 516.

[0048] In the embodiments, base station 514a and WTRUs 502a, 502b, 502c can implement radio technologies such as NR radio access, which can use New Radio (NR) to establish air interface 516.

[0049] In the embodiments, base station 514a and WTRUs 502a, 502b, and 502c can implement various radio access technologies. For example, base station 514a and WTRUs 502a, 502b, and 502c can, for example, use a dual connectivity (DC) principle to implement both LTE and NR radio access together. Therefore, the air interface utilized by WTRUs 502a, 502b, and 502c can be characterized by various types of radio access technologies and / or transmissions sent to or from various types of base stations (e.g., eNBs and gNBs).

[0050] In other embodiments, base station 514a and WTRU 502a, 502b, 502c can implement radio technologies such as IEEE 802.11 (i.e., WiFi), IEEE 802.16 (i.e., Global Microwave Access Interoperability (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Provisional Standard 2000 (IS-2000), Provisional Standard 95 (IS-95), Provisional Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data Rate Evolution of GSM (EDGE), GSMEDGE (GERAN), etc.

[0051] Figure 1A Base station 514b can be, for example, a wireless router, a home node B, a home e node B, or an access point, and can utilize any suitable RAT to facilitate wireless connectivity in a local area, such as a business premises, home, vehicle, campus, industrial facility, air corridor (e.g., for use by drones), road, etc. In one embodiment, base station 514b and WTRUs 502c, 502d can implement radio technologies such as IEEE 802.11 to establish a wireless local area network (WLAN). In one embodiment, base station 514b and WTRUs 502c, 502d can implement radio technologies such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, base station 514b and WTRUs 502c, 502d can utilize cellular-based RATs (e.g., WCDMA, CDMA2000, GSM, LTE-A Pro, NR, etc.) to establish picocells or femtocells. Figure 1A As shown, base station 514b can have a direct connection to the Internet 510. Therefore, base station 514b can access the Internet 510 without going through CN506 / 515.

[0052] RAN504 / 513 can communicate with CN506 / 515, which can be any type of network configured to provide voice, data, application, and / or Voice over Internet Protocol (VoIP) services to one or more of WTRU502a, 502b, 502c, and 502d. Data may have varying Quality of Service (QoS) requirements, such as different throughput requirements, latency requirements, fault tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. CN506 / 515 can provide call control, billing services, location-based services, prepaid calling, Internet connectivity, video distribution, etc., and / or perform advanced security functions such as user authentication. Although in Figure 1AAlthough not shown, it should be understood that RAN504 / 513 and / or CN506 / 515 can communicate directly or indirectly with other RANs using the same RAT as RAN504 / 513 or a different RAT. For example, in addition to connecting to RAN504 / 513, which can utilize NR radio technology, CN506 / 515 can also communicate with another RAN (not shown) using GSM, UMTS, CDMA2000, WiMAX, E-UTRA, or WiFi radio technology.

[0053] CN506 / 515 can also serve as a gateway for WTRU502a, 502b, 502c, and 502d to access PSTN508, the Internet 510, and / or other networks 512. PSTN508 may include a circuit-switched telephone network providing Common Old-Style Telephone Service (POTS). The Internet 510 may include a global system of interconnected computer networks and devices using common communication protocols, such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and / or Internet Protocol (IP) from the TCP / IP Internet Protocol suite. Network 512 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, network 512 may include another CN connected to one or more RANs, which may use the same RAT as RAN504 / 513 or a different RAT.

[0054] Some or all of the WTRUs 502a, 502b, 502c, and 502d in the communication system 500 may include multi-mode capabilities (e.g., WTRUs 502a, 502b, 502c, and 502d may include multiple transceivers to communicate with different wireless networks via different wireless links). For example... Figure 1A The WTRU502c shown can be configured to communicate with a base station 51a that can use cellular-based radio technology, and with a base station 514b that can use IEEE 802 radio technology.

[0055] Figure 1B This shows a system diagram of the example WTRU502. (See diagram below.) Figure 1B As shown, WTRU502 may include a processor 518, a transceiver 520, a transmitting / receiving element 522, a speaker / microphone 524, a keyboard 526, a display / touchpad 528, non-removable memory 530, removable memory 532, a power supply 534, a Global Positioning System (GPS) chipset 536, and / or other peripheral devices 538, etc. It is understood that WTRU502 may include any sub-combination of the foregoing elements while remaining consistent with the embodiments.

[0056] Processor 518 can be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. Processor 518 can perform signal decoding, data processing, power control, input / output processing, and / or any other function that enables WTRU502 to operate in a wireless environment. Processor 518 can be coupled to transceiver 520, and transceiver 520 can be coupled to transmitting / receiving element 522. Although Figure 1B While the processor 518 and transceiver 520 are described as separate components, it will be understood that the processor 518 and transceiver 520 may be integrated together in an electronic package or chip.

[0057] Transmitting / receiving element 522 can be configured to transmit signals to or receive signals from a base station (e.g., base station 514a) via air interface 516. For example, in one embodiment, transmitting / receiving element 522 can be an antenna configured to transmit and / or receive RF signals. In one embodiment, transmitting / receiving element 522 can be a transmitter / detector configured to transmit and / or receive, for example, IR, UV, or visible light signals. In yet another embodiment, transmitting / receiving element 522 can be configured to transmit and / or receive both RF and optical signals. It should be understood that transmitting / receiving element 522 can be configured to transmit and / or receive any combination of wireless signals.

[0058] Although the transmitter / receiver unit 522 is in Figure 1B While described as a single element, WTRU502 may include any number of transmit / receive units 522. More specifically, WTRU502 may use MIMO technology. Therefore, in one embodiment, WTRU502 may include two or more transmit / receive elements 522 (e.g., multiple antennas) for transmitting and receiving wireless signals via air interface 516.

[0059] Transceiver 520 can be configured to modulate signals to be transmitted by transmitting / receiving element 522 and to demodulate signals received by transmitting / receiving element 522. As described above, WTRU 502 can have multi-mode capability. Therefore, for example, transceiver 520 may include multiple transceivers for enabling WTRU 502 to communicate via multiple RATs, such as NR and IEEE 802.11.

[0060] The processor 518 of WTRU502 can be coupled to a speaker / microphone 524, a keyboard 526, and / or a display / touchpad 528 (e.g., a liquid crystal display (LCD) unit or an organic light-emitting diode (OLED) display unit), and can receive user input data therefrom. The processor 518 can also output user data to the speaker / microphone 524, keyboard 526, and / or display / touchpad 528. Additionally, the processor 518 can access information from any type of suitable memory and store data in said memory, such as non-removable memory 530 and / or removable memory 532. Non-removable memory 530 may include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. Removable memory 532 may include a user identity module (SIM) card, memory stick, secure digital storage (SD) card, etc. In other embodiments, the processor 518 can access information from memory and store data in memory that is not physically located on WTRU502, such as on a server or home computer (not shown).

[0061] The processor 518 may receive power from the power supply 534 and may be configured to distribute and / or control power to other components in the WTRU 502. The power supply 534 may be any suitable device for powering the WTRU 502. For example, the power supply 534 may include one or more dry cell batteries (e.g., nickel-cadmium, nickel-zinc, nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0062] The processor 518 may also be coupled to a GPS chipset 536, which may be configured to provide location information (e.g., longitude and latitude) about the current location of the WTRU 502. In addition to, or alternatively to, information from the GPS chipset 536, the WTRU 502 may receive location information from base stations (e.g., base stations 514a, 514b) via an air interface 516, and / or determine its location based on the timing of signals received from two or more neighboring base stations. It should be understood that the WTRU 502 may acquire location information using any suitable location determination method while remaining consistent with the embodiments.

[0063] The processor 518 may also be coupled to other peripheral devices 538, which may include one or more software and / or hardware modules providing additional features, functions, and / or wired or wireless connectivity. For example, peripheral devices 538 may include accelerometers, electronic compasses, satellite transceivers, digital cameras (for photos and / or videos), Universal Serial Bus (USB) ports, vibration devices, television transceivers, hands-free headsets, Bluetooth® modules, FM radio units, digital music players, media players, video game player modules, internet browsers, virtual reality and / or augmented reality (VR / AR) devices, activity trackers, etc. Peripheral devices 538 may include one or more sensors, which may be one or more of the following: gyroscopes, accelerometers, Hall effect sensors, magnetometers, orientation sensors, proximity sensors, temperature sensors, time sensors; geolocation sensors; altimeters, light sensors, touch sensors, magnetometers, barometers, gesture sensors, biometric sensors, and / or humidity sensors.

[0064] The WTRU502 may include a full-duplex radio for which the transmission and reception of some or all signals (e.g., signals associated with specific subframes for UL (e.g., for transmission) and downlink (e.g., for reception)) may be simultaneous and / or concurrent.

[0065] Figure 1C This diagram illustrates a system diagram of RAN504 and CN506 according to an embodiment. As described above, RAN504 may employ E-UTRA radio technology to communicate with WTRU502a, 502b, and 502c via air interface 516. RAN504 may also communicate with CN506.

[0066] RAN504 may include e-nodes B560a, 560b, and 560c, but it should be understood that RAN504 may include any number of e-nodes B while remaining consistent with the embodiments. Each of e-nodes B560a, 560b, and 560c may include one or more transceivers for communicating with WTRUs 502a, 502b, and 502c via air interface 516. In one embodiment, e-nodes B560a, 560b, and 560c may implement MIMO technology. Therefore, e-node B560a may, for example, use multiple antennas to transmit and / or receive radio signals from WTRU 502a.

[0067] Each of the e-nodes B560a, 560b, and 560c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, etc. Figure 1CAs shown, nodes B560a, 560b, and 560C can communicate with each other via the X2 interface.

[0068] Figure 1C The CN506 shown may include a Mobility Management Entity (MME) 562, a Serving Gateway (SGW) 564, and a Packet Data Network (PDN) Gateway (or PGW) 566. While each of the foregoing elements is depicted as part of CN506, it will be understood that any of these elements may be owned and / or operated by an entity other than a CN operator.

[0069] The MME562 can connect to each of the e-nodes B562a, 562b, and 562c in RAN504 via the S1 interface and can act as a control node. For example, the MME562 can be responsible for authenticating users of WTRU502a, 502b, and 502c, bearer activation / deactivation, selecting a specific serving gateway during the initial attachment of WTRU502a, 502b, and 502c, etc. The MME562 can provide control plane functions for handover between RAN504 and other RANs (not shown) employing other radio technologies (such as GSM and / or WCDMA).

[0070] The SGW564 can connect to each of the e-nodes B560a, 560b, and 560c in RAN504 via the S1 interface. The SGW564 can typically route and forward user data packets to / from WTRU502a, 502b, and 502c. The SGW564 can perform other functions, such as anchoring the user plane during handover between e-nodes B, triggering paging when DL data is available for WTRU502a, 502B, and 502c, managing and storing the context of WTRU502a, 502b, and 502c, etc.

[0071] The SGW564 can connect to the PGW566, which can provide WTRU502a, 502b, and 502c with access to packet-switched networks such as the Internet 510, to facilitate communication between WTRU502a, 502b, 502c and IP-enabled devices.

[0072] CN506 can facilitate communication with other networks. For example, CN506 can provide WTRU502a, 502b, and 502c with access to a circuit-switched network (e.g., PSTN508) to facilitate communication between WTRU502a, 502b, 502c and traditional landline communication equipment. For example, CN506 may include an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server), or can communicate with an IP gateway that acts as an interface between CN506 and PSTN508. Furthermore, CN506 can provide WTRU502a, 502b, and 502c with access to other networks 512, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0073] Although WTRU is Figure 1A-1D While described as a wireless terminal, it is anticipated that in some representative embodiments, such a terminal may use (e.g., temporarily or permanently) a wired communication interface with a communication network.

[0074] In a representative embodiment, the other network 512 may be a WLAN.

[0075] In an Infrastructure Basic Services Set (BSS) mode, a WLAN may have an Access Point (AP) for the BSS and one or more Stations (STAs) associated with the AP. The AP may have access or an interface to a Distribution System (DS) or another type of wired / wireless network that carries traffic entering and / or leaving the BSS. Traffic originating from a STA outside the BSS can reach and be delivered to the STA via the AP. Traffic originating from a STA to a destination outside the BSS can be sent to the AP for delivery to the appropriate destination. Traffic between STAs within the BSS can be transmitted via the AP, for example, where a source STA can send traffic to the AP, and the AP can deliver traffic to the destination STA. Traffic between STAs within the BSS can be considered and / or referred to as point-to-point traffic. Point-to-point traffic can be transmitted between a source STA and a destination STA (e.g., directly between the source STA and the destination STA) using Direct Link Establishment (DLS). In some representative embodiments, the DLS may use 802.11e DLS or 802.11z Tunneled DLS (TDLS). A WLAN using the Standalone BSS (IBSS) mode may not have an access point (AP), and STAs within the IBSS or using the IBSS (e.g., all STAs) can communicate directly with each other. The IBSS communication mode may sometimes be referred to here as an "ad-hoc" communication mode.

[0076] When using 802.11ac infrastructure operating mode or a similar operating mode, the AP can transmit beacons on a fixed channel, such as the primary channel. The primary channel can be of fixed width (e.g., a bandwidth of 20 MHz) or dynamically set via signaling. The primary channel can be the operating channel of the BSS and can be used by the STA to establish a connection with the AP. In some representative embodiments, such as in an 802.11 system, Carrier Sense Multiple Access (CSMA / CA) with collision avoidance can be implemented. For CSMA / CA, each STA, including the AP, can sense the primary channel. If the primary channel is sensed / detected and / or determined to be busy by a particular STA, that particular STA can back off. A single STA (e.g., only one station) can transmit at any given time within a given BSS.

[0077] High-throughput (HT) STAs can communicate using a 40MHz wide channel, for example, by combining a primary 20MHz channel with adjacent or non-adjacent 20MHz channels to form a 40MHz wide channel.

[0078] Very High Throughput (VHT) STAs can support channels with widths of 20MHz, 40MHz, 80MHz, and / or 160MHz. 40MHz and / or 80MHz channels can be formed by combining adjacent 20MHz channels. A 160MHz channel can be formed by combining eight consecutive 20MHz channels or by combining two non-consecutive 80MHz channels; this is known as an 80+80 configuration. In the 80+80 configuration, after channel coding, the data can pass through a segmented parser that divides the data into two streams. Each stream can be processed separately using Inverse Fast Fourier Transform (IFFT) and time-domain processing. The streams can be mapped onto the two 80MHz channels, and the data can be transmitted by the transmitting STA. At the receiver of the receiving STA, the operation of the 80+80 configuration can be reversed, and the combined data can be sent to the Media Access Control (MAC).

[0079] Operating modes below 1 GHz are supported by 802.11af and 802.11ah. The channel operating bandwidth and carrier are reduced in 802.11af and 802.11ah compared to those used in 802.11n and 802.11ac. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV whitespace (TVWS) spectrum, while 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11ah may support instrument-type control / machine-type communications, such as MTC devices in macro coverage areas. MTC devices may have certain capabilities, such as limited capabilities including support for certain and / or limited bandwidths (e.g., only support). MTC devices may include batteries with a battery life exceeding a threshold (e.g., to maintain a very long battery life).

[0080] WLAN systems that can support multiple channels and channel bandwidths, such as 802.11n, 802.11ac, 802.11af, and 802.11ah, include channels that can be designated as the primary channel. The primary channel can have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel can be set and / or limited by one of the STAs operating in the BSS that supports the minimum bandwidth operating mode. In the 802.11ah example, for a STA that supports (e.g., only supports) the 1MHz mode (e.g., an MTC-type device), the primary channel can be 1MHz wide, even if the AP and other STAs in the BSS support 2MHz, 4MHz, 8MHz, 16MHz, and / or other channel bandwidth operating modes. Carrier Sense and / or Network Allocation Vector (NAV) settings can depend on the status of the primary channel. If the primary channel is busy, for example, because an STA (which only supports the 1MHz operating mode) is transmitting to the AP, the entire available band can be considered busy even if most of the band remains idle and available.

[0081] In the United States, the available frequency band for 802.11ah is from 902MHz to 928MHz. In South Korea, the available frequency band is from 917.5MHz to 923.5MHz. In Japan, the available frequency band is from 916.5MHz to 927.5MHz. Depending on the country code, the total bandwidth available for 802.11ah is from 6MHz to 26MHz.

[0082] Figure 1DThis diagram illustrates a system diagram of RAN513 and CN515 according to an embodiment. As described above, RAN513 can use NR radio technology to communicate with WTRU502a, 502b, and 502c via air interface 516. RAN513 can also communicate with CN515.

[0083] RAN513 may include gNBs 580a, 580b, and 580c; however, it should be understood that RAN513 may include any number of gNBs while remaining consistent with the embodiments. Each of gNBs 580a, 580b, and 580c may include one or more transceivers for communicating with WTRUs 502a, 502b, and 502c via air interface 516. In one embodiment, gNBs 580a, 580b, and 580c may implement MIMO technology. For example, gNBs 580a and 580b may utilize beamforming to transmit signals to and / or receive signals from gNBs 580a, 580b, and 580c. Therefore, gNB 580a may, for example, use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 502a. In one embodiment, gNBs 580a, 580b, and 580c may implement carrier aggregation technology. For example, gNB 580a may transmit multiple component carriers (not shown) to WTRU 502a. A subset of these component carriers may be on unlicensed spectrum, while the remaining component carriers may be on licensed spectrum. In embodiments, gNB580a, 580b, and 580c may implement Cooperative Multipoint (CoMP) technology. For example, WTRU502a may receive coordinated transmissions from gNB580a and gNB580b (and / or gNB580c).

[0084] The WTRU502a, 502b, and 502c can communicate with the gNB580a, 580b, and 580c using transmissions associated with scalable parameter configuration (numerology). For example, the OFDM symbol spacing and / or OFDM subcarrier spacing can vary for different transmissions, different cells, and / or different portions of the radio transmission spectrum. The WTRU502a, 502b, and 502c can communicate with the gNB580a, 580b, and 580c using subframes or transmission time intervals (TTIs) with various or scalable lengths (e.g., containing different numbers of OFDM symbols and / or continuously varying absolute time lengths).

[0085] gNB580a, 580b, and 580c can be configured to communicate with WTRU502a, 502b, and 502c in standalone and / or non-standalone configurations. In standalone configuration, WTRU502a, 502b, and 502c can communicate with gNB580a, 580b, and 580c without needing to access other RANs (e.g., eNodeB560a, 560b, and 560c). In standalone configuration, WTRU502a, 502b, and 502c can utilize one or more of gNB580a, 580b, and 580c as mobility anchors. In standalone configuration, WTRU502a, 502b, and 502c can communicate with gNB580a, 580b, and 580c using signals in unlicensed frequency bands. In a non-standalone configuration, WTRU502a, 502b, and 502c can communicate / connect with gNB580a, 580b, and 580c, as well as with another RAN such as eNodeB560a, 560b, and 560c. For example, WTRU502a, 502b, and 502c can implement DC principles to communicate essentially simultaneously with one or more gNB580a, 580b, and 580c, and one or more eNodeB560a, 560b, and 560c. In a non-standalone configuration, eNodeB560a, 560b, and 560c can act as mobility anchors for WTRU502a, 502b, and 502c, and gNB580a, 580b, and 580c can provide additional coverage and / or throughput for serving WTRU502a, 502b, and 502c.

[0086] Each of gNB580a, 580b, and 580c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, network slicing support, dual connectivity, interoperability between NR and E-UTRA, routing user plane data to User Plane Functions (UPF) 584a and 584b, and routing control plane information to Access and Mobility Management Functions (AMF) 582a and 582b, etc. Figure 1D As shown, gNB580a, 580b, and 580c can communicate with each other via the Xn interface.

[0087] Figure 1D The CN515 shown may include at least one AMF582a, 582b, at least one UPF584a, 584b, at least one Session Management Function (SMF)583a, 583b, and possibly a Data Network (DN)585a, 585b. Although each of the foregoing elements is depicted as part of the CN515, it will be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0088] AMF582a and 582b can connect to one or more of gNB580a, 580b, and 580c in RAN513 via the N2 interface and can act as control nodes. For example, AMF582a and 582b can be responsible for authenticating users of WTRU502a, 502b, and 502c, supporting network slicing (e.g., handling different PDU sessions with different needs), selecting specific SMF583a and 583b, managing registration areas, terminating NAS signaling, mobility management, and so on. Network slices can be used by AMF582a and 582b to customize CN support for WTRU502a, 502b, and 502c based on the service types used by WTRU502a, 502b, and 502c. For example, different network slices can be established for different use cases, such as services relying on Ultra Reliable Low Latency (URLLC) access, services relying on Enhanced Massive Mobile Broadband (eMBB) access, and services for Machine Type Communication (MTC) access. The AMF562 can provide control plane functions for switching between RAN513 and other RANs (not shown) that employ other radio technologies, such as LTE, LTE-A Pro, and / or non-3GPP access technologies such as WiFi.

[0089] SMF583a and 583b can connect to AMF582a and 582b in CN515 via the N11 interface. SMF583a and 583b can also connect to UPF584a and 584b in CN515 via the N4 interface. SMF583a and 583b can select and control UPF584a and 584b, and configure the routing of services through UPF584a and 584b. SMF583a and 583b can perform other functions, such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notifications. PDU session types can be IP-based, non-IP-based, Ethernet-based, etc.

[0090] UPF584a and 584b can connect to one or more of gNB580a, 580b, and 580c in RAN513 via the N3 interface. This provides WTRU502a, 502b, and 502c with access to packet-switched networks such as the Internet510, facilitating communication between WTRU502a, 502b, and 502c and IP-enabled devices. UPF584 and 584b can perform other functions such as routing and forwarding packets, enforcing user plane policies, supporting multihomed PDU sessions, handling user plane QoS, buffering downlink packets, providing mobility anchoring, and so on.

[0091] CN515 can facilitate communication with other networks. For example, CN515 may include an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) or may communicate with an IP gateway that serves as an interface between CN515 and PSTN508. Furthermore, CN515 may provide WTRU502a, 502b, 502c with access to other networks 512, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, WTRU502a, 502b, 502c may be connected to local data networks (DNs) 585a, 585b via the N3 interface to UPF584a, 584b and the N6 interface between UPF584a, 584b and DN585a, 585b.

[0092] Given Figure 1A-1D and Figure 1A-1D The functions described herein with respect to one or more of the following can be performed by one or more emulation devices (not shown): WTRU502a-d, base station 514a-b, eNodeB560a-c, MME562, SGW564, PGW566, gNB580a-c, AMF582a-b, UPF584a-B, SMF583a-b, DN585a-b, and / or any (one or more) other devices described herein. An emulation device can be one or more devices configured to emulate one or more of the functions described herein. For example, an emulation device can be used to test other devices and / or simulate network and / or WTRU functions.

[0093] Simulation devices can be designed to perform one or more tests on other devices in laboratory and / or carrier network environments. For example, one or more simulation devices can perform one or more or all functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices within the communication network. One or more simulation devices can perform one or more or all functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. Simulation devices can be directly coupled to another device for testing purposes and / or can perform tests using over-the-air wireless communication.

[0094] One or more emulation devices may perform one or more functions, including all functions, rather than being implemented / deployed as part of a wired and / or wireless communication network. For example, emulation devices may be used in test scenarios within test laboratories and / or non-deployed (e.g., testing) wired and / or wireless communication networks to perform testing of one or more components. One or more emulation devices may be test rigs. Emulation devices may transmit and / or receive data using direct RF coupling and / or wireless communication via RF circuitry (e.g., which may include one or more antennas).

[0095] This application describes several aspects, including tools, features, examples, models, methods, etc. Many of these aspects are described as specific and, at least to illustrate individual characteristics, are generally described in a way that may sound restrictive. However, this is for the purpose of clarity and does not limit the application or scope of those aspects. In fact, all the different aspects can be combined and interchanged to provide other aspects. Furthermore, these aspects can also be combined and interchanged with those described in earlier submissions.

[0096] The aspects described and contemplated in this application can be implemented in many different forms. Figures 1-20 described herein provide some examples, but other examples are contemplated, and the discussion of Figures 1-20 does not limit the breadth of implementations. At least one aspect generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions thereon stored thereon for encoding or decoding video data according to any of the described methods, and / or computer-readable storage media having a bitstream generated according to any of the described methods stored thereon.

[0097] In this application, the terms "reconstruction" and "decoding" are used interchangeably, as are the terms "pixel" and "sample," and the terms "image," "picture," and "frame." Typically, but not necessarily, "decoding" is used on the decoder side. The term "reconstruction" is used on the encoder side and / or the decoder side.

[0098] This document describes various methods, and each method includes one or more steps or actions for implementing the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions may be modified or combined. Furthermore, terms such as "first" and "second" may be used in various examples to modify elements, components, steps, operations, etc., such as "first decoding" and "second decoding." Unless specifically required, the use of these terms does not imply an ordering of the modified operations. Therefore, in this example, the first decoding does not need to be performed before the second decoding and may occur, for example, before, during, or in a time period overlapping with the second decoding.

[0099] The various methods and other aspects described in this application can be used to modify the module, for example, such as Figure 2 and Figure 3 The decoding modules of the video encoder 100 and decoder 200 shown are illustrated. Furthermore, aspects of the invention are not limited to VVC or HEVC, and can be applied to, for example, other standards and recommendations (whether pre-existing or developed in the future), as well as any extensions of such standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically excluded, the aspects described in this application may be used individually or in combination.

[0100] Various numerical values ​​are used in this application, such as CU sizes of 4×8, 4×N, 8×4, and 8×N, minimum prediction block width of four samples, PU containing 32 or more samples, etc. Specific values ​​are for illustrative purposes, and the aspects described are not limited to these specific values.

[0101] Block-based video decoding systems, such as VVC, can include block-based hybrid video decoding frameworks. Figure 2 A block diagram of an example block-based hybrid video coding system 200 is shown. Variations of this encoder 200 can be envisioned, but for clarity, encoder 200 is described below without all anticipated variations. The input video signal 202 can be processed block-by-block (e.g., decoder unit (CU)). CUs can contain up to 128×128 pixels in size. Decoder tree units (CTUs) can be partitioned into CUs to accommodate varying local characteristics based on quadtree / binary / tritree structures. CUs can be used as basic units for prediction and transformation without further partitioning. In a multi-type tree structure, CTUs can be partitioned from quadtree structures. Quadtree leaf nodes (e.g., each quadtree leader node) can be further partitioned from binary and tritree structures.

[0102] Reference Figure 2For an input video block (e.g., a macroblock (MB) or CU), spatial prediction 260 or motion prediction 262 can be performed. Spatial prediction (e.g., intra-frame prediction) uses pixels from decoded neighboring blocks in the same video picture and / or slice to predict the current video block. Spatial prediction can reduce the spatial redundancy inherent in the video signal. Temporal prediction (e.g., referred to as inter-frame prediction or motion-compensated prediction) uses pixels from decoded video pictures to predict the current video block. Temporal prediction can reduce the temporal redundancy inherent in the video signal. The temporal prediction signal for the CU can be signaled via one or more motion vectors (MVs), which can indicate the amount and direction of motion between the current CU and its temporal reference. If multiple reference pictures are supported, a reference picture index can be signaled to the decoder. The reference index can be used to identify which reference picture in the reference picture storage 264 the temporal prediction signal can come from.

[0103] Following spatial and / or temporal prediction, mode determination 280 in the encoder can select the prediction mode, for example, based on a rate-distortion optimization mechanism. At 216, the prediction block can be subtracted from the current video block. The prediction residual can be decorrelated using transform module 204 and quantization module 206 to achieve the target bit rate. The quantized residual coefficients can be inversely quantized at 210 and inversely transformed at 212 to form the reconstructed residual. At 226, the reconstructed residual can be added back to the prediction block to form the reconstructed video block. At 266, before placing the reconstructed video block into reference image storage 264, loop filters such as deblocking filters and / or adaptive loop filters can be applied to the reconstructed video block. The reference images in reference image storage 264 can be used to decode future video blocks. An output video bitstream 220 can then be formed. Decoding mode (e.g., inter-frame or intra-frame), prediction mode information, motion information, and / or quantized residual coefficients can be sent to entropy decoding unit 208 to be compressed and packaged to form bitstream 220.

[0104] Figure 3A general block diagram of an example block-based video decoder 300 is shown. Video bitstream 302 can be received, unpacked, and / or entropy-decoded at entropy decoding unit 308. Decoding mode and / or prediction information can be sent to spatial prediction unit 360 (e.g., if intra-frame decoding) and / or to temporal prediction unit 362 (e.g., if inter-frame decoding) to form prediction blocks. Residual transform coefficients can be sent to inverse quantization unit 310 and inverse transform unit 312 to reconstruct residual blocks. At 326, prediction blocks and residual blocks can be added. The reconstructed blocks can undergo in-loop filtering 366 and can be stored in reference picture storage 364. The reconstructed video in reference picture storage 364 can be used to drive a display device and / or predict future video blocks. For example, decoder 300 can determine ISP enabled for the current block after receiving, unpacking, and / or entropy-decoding video bitstream 302. Decoding mode information (e.g., intra-frame decoding mode) can be sent to spatial prediction unit 360 to form prediction blocks. Decoder 300 can determine the widths of S sub-partitions of the current block based on the unpacked and / or entropy-decoded video bitstream 302. For each sub-partition in the current block, the decoder can determine the corresponding prediction block based on the sub-partition width and the minimum prediction block width. The minimum prediction block width can be a predetermined value for the prediction block width. Decoder 300 can determine reference samples in the spatial prediction unit 360. Decoder 300 can use the reference samples to predict the prediction block.

[0105] Figure 4 A block diagram illustrating an example system in which various aspects and examples are implemented is shown. System 1000 can be implemented as a device comprising the various components described below and configured to perform one or more aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of System 1000 can be implemented individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one example, the processing and encoder / decoder elements of System 1000 are distributed across multiple ICs and / or discrete components. In various examples, System 1000 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various examples, System 1000 is configured to implement one or more aspects described herein.

[0106] System 1000 includes at least one processor 1010 configured to execute instructions loaded therein for implementing various aspects, such as those described herein. Processor 1010 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). System 1000 includes a storage device 1040, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 1040 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices. In this example, a device including one or more processors may be configured to perform a determination to enable ISP for the current block. The one or more processors may be configured to perform the determination of the sub-partition width of the current block. The one or more processors may be configured to perform the determination of the corresponding prediction block for each sub-partition in the current block, based on the sub-partition width and the minimum prediction block width. The one or more processors may be configured to perform the determination of a reference sample. The one or more processors may be configured to use the reference sample to perform the prediction of the prediction block. The apparatus including one or more processors may be a decoder or an encoder.

[0107] Signals may be transmitted to enable the device to perform one or more steps of this document. For example, a device may include: an access unit configured to access data enabling the device to perform one or more steps of this document; and a transmitter configured to transmit the data. A method may include accessing data enabling the device to perform one or more steps of this document and transmitting the data. A computer-readable medium may include data enabling the device to perform one or more steps of this document.

[0108] System 1000 includes an encoder / decoder module 1030 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents one or more modules that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both encoding and decoding modules. Alternatively, the encoder / decoder module 1030 may be implemented as a separate element of system 1000 or may be incorporated into processor 1010 as a combination of hardware and software as known to those skilled in the art.

[0109] Program code to be loaded onto processor 1010 or encoder / decoder 1030 to execute the various aspects described in this document may be stored in storage device 1040 and subsequently loaded onto memory 1020 for execution by processor 1010. According to various examples, one or more of processor 1010, memory 1020, storage device 1040, and encoder / decoder module 1030 may store one or more of various items during the execution of the processes described in this document. These stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0110] In some examples, the memory within processor 1010 and / or encoder / decoder module 1030 is used to store instructions and provide working memory for processing during encoding or decoding. However, in other examples, external memory (e.g., the processing device may be processor 1010 or encoder / decoder module 1030) is used for one or more of these functions. External memory may be memory 1020 and / or storage device 1040, such as volatile memory and / or non-volatile flash memory. In several examples, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one example, fast external dynamic volatile memory such as RAM is used as working memory for video decoding and decoding operations, such as for MPEG-2 (MPEG stands for Moving Picture Experts Group; MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Decoding; also known as H.265 and MPEG-H Part 2), or VVC (General Video Decoding, a new standard developed by JVET, the Joint Video Experts Group).

[0111] As shown in box 1130, input to the components of system 1000 can be provided through various input devices. Such input devices include, but are not limited to, (i) a radio frequency (RF) section that receives, for example, RF signals transmitted over the air by a broadcasting company, (ii) a component (COMP) input terminal (or a set of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Figure 4 Other examples not shown include composite videos.

[0112] In various examples, as known in the art, the input device of block 1130 has associated corresponding input processing elements. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting a signal band to a band); (ii) down-converting the selected signal; (iii) further band-limiting to a narrower band to select, for example, a signal band, which in some examples may be referred to as a channel; (iv) demodulating the down-converted and band-limited signal; (v) performing error correction; and (vi) demultiplexing to select a desired data packet stream. The RF section of various examples includes one or more elements performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners performing various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one example of a set-top box, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band. Various examples rearrange the order of the aforementioned (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various examples, the RF section includes an antenna. In examples, the device may include the apparatus described herein and at least one of: (i) an antenna configured to receive signals including data representing an image; (ii) a band limiter configured to limit the received signals to a frequency band including the data representing the image; or (iii) a display configured to display an image.

[0113] Additionally, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 1000 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented as needed, for example, within a separate input processing IC or processor 1010. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed within a separate interface IC or within processor 1010. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, processor 1010 and encoder / decoder 1030, which operate in conjunction with memory and storage elements to process the data streams as needed for presentation on the output device.

[0114] Various components of system 1000 can be provided within an integrated housing. Within this integrated housing, the various components can be interconnected and data can be transmitted therebetween using a suitable connection arrangement 1140, such as an internal bus known in the art, including inter-IC (I2C) bus, wiring, and printed circuit board.

[0115] System 1000 includes a communication interface 1050, which enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data via the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 1060 may be implemented, for example, in a wired and / or wireless medium.

[0116] In various examples, wireless networks such as Wi-Fi networks (e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers)) are used to stream or otherwise provide data to system 1000. In these examples, the Wi-Fi signal is received via a communication channel 1060 and a communication interface 1050 suitable for Wi-Fi communication. The communication channel 1060 in these examples is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other examples use a set-top box that delivers data via an HDMI connection of input block 1130 to provide streaming data to system 1000. Still other examples use an RF connection of input block 1130 to provide streaming data to system 1000. As mentioned above, various examples provide data in a non-streaming manner. Additionally, various examples use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0117] System 1000 can provide output signals to various output devices, including display 1100, speaker 1110, and other peripheral devices 1120. Various examples of display 1100 include one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a flexible display, and / or a foldable display. Display 1100 can be used in televisions, tablets, laptops, cellular phones (mobile phones), or other devices. Display 1100 can also be integrated with other components (e.g., as in a smartphone) or standalone (e.g., an external monitor for a laptop computer). In various examples, other peripheral devices 1120 include one or more of a standalone digital video disc (or digital multifunction disc) (DVR, for both terms), a disc player, a stereo system, and / or a lighting system. Various examples utilize one or more peripheral devices 1120 that provide functionality based on the output of system 1000. For example, a disc player performs the function of playing the output of system 1000.

[0118] In various examples, signaling is used to transmit control signals between system 1000 and display 1100, speaker 1110, or other peripheral devices 1120, employing communication protocols such as AV links, consumer electronics controls (CEC), or other communication protocols that enable device-to-device control with or without user intervention. Output devices may be communicatively coupled to system 1000 via dedicated connections through corresponding interfaces 1070, 1080, and 1090. Alternatively, output devices may be connected to system 1000 via communication interface 1050 using communication channel 1060. In electronic devices such as televisions, display 1100 and speaker 1110 may be integrated into a single unit along with other components of system 1000. In various examples, display interface 1070 includes a display driver, such as a timing controller (TCon) chip.

[0119] For example, if the RF section of input 1130 is part of a separate set-top box, then display 1100 and speaker 1130 can alternatively be separated from one or more other components. In various examples where display 1100 and speaker 1130 are external components, the output signal can be provided via dedicated output connections, such as HDMI ports, USB ports, or COMP outputs.

[0120] These examples can be executed by processor 1010, computer software implemented by hardware or a combination of hardware and software. As a non-limiting example, these examples can be implemented by one or more integrated circuits. Memory 1020 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. Processor 1010 can be of any type suitable for the technical environment, as a non-limiting example, and can include one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0121] Various implementations involve decoding. As used herein, "decoding" can include, for example, all or part of a process performed on a received encoded sequence to produce a final output suitable for display. In various examples, such a process includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various examples, such a process may also or alternatively include a process performed by the decoder of the various implementations described herein, for example, determining the corresponding prediction block based on the sub-partition width and the minimum prediction block width for a sub-partition in the current block.

[0122] As further examples, in one example, "decoding" refers only to entropy decoding; in another example, "decoding" refers only to differential decoding; and in yet another example, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to specifically refer to a subset of operations or generally to a broader decoding process will be clear based on the specific context of the description and is believed to be fully understood by those skilled in the art.

[0123] Various implementations involve encoding. In a manner similar to the above discussion of “decoding,” “encoding,” as used herein, can include, for example, all or part of a process performed on an input video sequence to produce an encoded bitstream. In various examples, such a process includes one or more processes typically performed by an encoder, such as partitioning, differential coding, transform, quantization, and entropy coding. In various examples, such a process may also, or alternatively, include a process performed by an encoder of the various implementations described herein, for example, determining the corresponding prediction block based on the sub-partition width and the minimum prediction block width for a sub-partition in the current block.

[0124] As further examples, in one example, "encoding" refers only to entropy encoding; in another example, "encoding" refers only to differential encoding; and in yet another example, "encoding" refers to a combination of differential and entropy encoding. Whether the phrase "encoding process" is intended to specifically refer to a subset of operations or generally to a broader encoding process will be clear based on the specific context of the description and is believed to be fully understood by those skilled in the art.

[0125] Note that the grammatical elements used here are descriptive terms. Therefore, the use of other grammatical element names is not excluded.

[0126] When the accompanying drawings are presented as flowcharts, it should be understood that they also provide block diagrams of the corresponding devices. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that they also provide flowcharts of the corresponding methods / processes.

[0127] Various examples involve rate-distortion optimization. Specifically, during the encoding process, a balance or trade-off between rate and distortion is typically considered, often with constraints on computational complexity. Rate-distortion optimization is generally formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. Different approaches exist to address rate-distortion optimization problems. For example, these methods can be based on extensive testing of all encoding options (including all considered modes or decoding parameter values), providing a complete evaluation of their decoding costs and the correlated distortion of the decoded and decoded reconstructed signals. Faster methods can also be used to save encoding complexity, particularly by calculating approximate distortion based on predicting or predicting the residual signal rather than the reconstructed signal. A hybrid of these two approaches can also be used, for example, by using approximate distortion only for some possible encoding options and full distortion for others. Other methods evaluate only a subset of possible encoding options. More generally, many methods employ any of a variety of techniques to perform optimization, but optimization is not necessarily a complete evaluation of both decoding costs and correlated distortion.

[0128] The implementations and aspects described herein can be implemented, for example, in a method or process, apparatus, software program, data stream, or signal. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the features in question can be implemented in other forms (e.g., apparatus or program). For example, an apparatus can be implemented with appropriate hardware, software, and firmware. The method can be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as computers, cellular phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0129] References to “an example” or “an example” or “an implementation” or “an implementation”, and their other variations, imply that a particular feature, structure, characteristic, etc., described in connection with that example is included in at least one example. Therefore, the phrases “in an example” or “in the example” or “in an implementation” or “in the implementation” appearing in various places in this application, and their other variations, do not necessarily all refer to the same example.

[0130] Additionally, this application may involve "determining" various types of information. Determining information may include one or more of, for example, estimated information, calculated information, predicted information, or information retrieved from memory. Obtaining may include receiving, determining, identifying, and / or retrieving.

[0131] Furthermore, this application may relate to "accessing" various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or more of these.

[0132] Furthermore, this application may relate to "receiving" various types of information. Like "access," receiving is intended to be a broad term. Receiving information may include, for example, accessing information or retrieving information (e.g., from memory) or more. Moreover, "receiving" is generally referred to in one or more ways during operations such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0133] It should be understood that, for example, in the cases of “A / B,” “A and / or B,” and “at least one of A and B,” the use of any of the following “ / ,” “and / or,” and “at least one of…” is intended to cover the selection of only the first listed option (A), or only the selection of only the second listed option (B), or the selection of both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C,” such phrases are intended to cover the selection of only the first listed option (A), or only the selection of only the second listed option (B), or only the selection of only the third listed option (C), or only the selection of only the first and second listed options (A and B), or only the selection of only the first and third listed options (A and C), or only the selection of only the second and third listed options (B and C), or the selection of all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to multiple listed items.

[0134] Furthermore, as used herein, the word "signal" refers, among other things, to indicating something to the corresponding decoder. For example, in some examples, the encoder signals a specific index indicating the selection of samples to be used for predicting a sub-partition. In this way, the same parameter is used on both the encoder and decoder sides in the examples. Thus, for example, the encoder can transmit (explicit signaling) a specific parameter to the decoder so that the decoder can use the same specific parameter. Conversely, if the decoder already has the specific parameter as well as other parameters, signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select the specific parameter. Bit savings are achieved in various examples by avoiding the transmission of any actual function. It should be understood that signaling can be done in various ways. For example, in various examples, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the foregoing refers to the verb form of the word "signal," the word "signal" can also be used as a noun in this document.

[0135] As will be apparent to those skilled in the art, implementations can generate various formatted signals to carry, for example, information that can be stored or transmitted. This information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, the signal may be formatted to carry a bitstream of the described example. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding the data stream and modulating a carrier wave using the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

[0136] We have described several examples. The features of these examples may be provided individually or in any combination across a variety of claimed classes and types. Furthermore, the examples may include one or more of the following features, devices, or aspects, individually or in any combination, across a variety of claimed classes and types: Although the features and elements have been described above in specific combinations, those skilled in the art will understand that each feature or element can be used alone or in any combination with other features and elements. Furthermore, the methods described herein can be implemented in a computer program, software, or firmware incorporated into a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROMs and digital multifunction discs (DVDs). The processor associated with the software can be used to implement a radio frequency transceiver used in a WTRU, UE, terminal, base station, RNC, or any host computer.

[0137] Intra-frame decoding tools, such as the Intra-Frame Sub-Partition (ISP) tool, can be used. The ISP tool can be applied to the luma channel. The ISP tool can divide the CU into multiple sub-partitions (e.g., two or four sub-partitions). Sub-partitions can include multiple (e.g., at least 16) samples. Dividing can be performed in the horizontal and / or vertical directions. CU segmentation can be performed in the horizontal and / or vertical directions. Figure 5 Examples of partitions or CU-segments that may be allowed for a CU (e.g., a 4×8 pixel CU or an 8×4 pixel CU) are shown. Figure 5 As shown, the ISP tool can split a CU into two sub-partitions. The ISP partition type can be vertical or horizontal. Figure 6 An example of a partition that can be allowed for a CU (e.g., a CU larger than 4×8 pixels or 8×4 pixels) is shown. Figure 6 As shown, for relatively large CUs (e.g., CUs larger than 4×8 pixels or 8×4 pixels), the ISP tool can divide the CU into four sub-partitions. Intra-prediction modes can be used for these sub-partitions. For example, the same intra-prediction mode can be used for sub-partitions within the CU.

[0138] For example, when the intra-mode used for the ISP is a member of the Most Probable Mode (MPM) list, the MPM flag can be set (e.g., set to one). For the ISP, the intra-reference sample smoothing filter and / or location-dependent intra-prediction combination (PDPC) can be disabled. Reference line zero can be used for the ISP. Multi-reference line intra-prediction can be disabled.

[0139] For horizontal splits, sub-partitions can be processed from top to bottom. For vertical splits, sub-partitions can be processed from left to right. Sub-partitions can be predicted, and residuals can be added to the predicted sub-partitions, for example, to produce reconstructed sub-partitions. Samples from the reconstructed sub-partitions can be used to predict the next sub-partition. Figure 7 A graph showing example predictions in an intra-frame sub-segment (ISP) is displayed. Figure 7 In (a)-(c), the horizontally divided sub-partitions are processed. Figure 7 (a) shows a CU comprising four sub-partitions A, B, C and D. Figure 7 (b) shows that sub-partition A can be predicted and reconstructed, and one or more samples in the bottom row of the reconstructed sub-partition A can be used to predict sub-partition B. Figure 7 (c) shows that sub-partition B can be predicted and reconstructed, and one or more samples in the bottom row of the reconstructed sub-partition B can be used to predict sub-partition C. Figure 7 As shown in (b), sub-partition A (e.g., the topmost sub-partition) can be predicted using reference samples above and to the left of the CU. Sub-partition A can be reconstructed by adding the residuals obtained after one or more of entropy decoding, inverse quantization, and inverse transform. One or more samples in the bottom row of the reconstructed sub-partition A can be used to predict sub-partition B, which is then reconstructed. The prediction and reconstruction can be repeated for subsequent sub-partitions.

[0140] For reference-line zero intra-prediction, multi-reference-line intra-prediction, and ISP, separate MPM lists can be maintained (e.g., a separate MPM list of size six). The MPM lists can hold indications (e.g., signaling) of the intra-modes to be used. The MPM list constructions for reference-line zero intra-prediction, multi-reference-line intra-prediction, and ISP can differ, for example, as... Figure 8 , Figure 9 and Figures 10a-10c As shown. Figure 8 This is an example flowchart for constructing an MPM list for zero-frame intra-prediction of a reference line. Figure 9 This is an example flowchart for constructing an MPM list for intra-frame prediction using multiple reference lines. Figures 10a-10c An example flowchart for constructing an MPM list for an ISP is shown. MPM list construction (e.g., as...) Figure 8 , Figure 9 and Figures 10a-10cAs shown in the table, the intra-frame modes of the adjacent left CU (denoted as 'A') and / or the above CU (denoted as 'B') can be considered during list construction. Multiple conditions (e.g., four conditions) can be checked, for example, using modes 'A' and 'B'. If the conditions are met, an MPM list can be generated. For example, each met condition can result in a separate MPM list. Table 1 shows the four conditions that can be used in MPM list construction (e.g., ...). Figure 8 , Figure 9 and Figures 10a-10c Examples of the three MPM list constructions shown, and their corresponding logical conditions (according to intra-frame modes A and B). Serial Number condition Equivalent logical conditions 1 A and B are sufficiently similar, and both are in angular mode. (A = B) && (A > 1) 2 A and B are different, and both are angle modes. (A≠B) && (A and B) > 1 3 A and B are different, and either of them is an angle mode. (A≠B) && (A or B) > 1 4 Default value (none of the above conditions are met) Table 1 - In Figure 8 , Figure 9 and Figures 10a-10c Example conditions used in the list construction shown.

[0141] like Figure 8 , Figure 9 and / or Figures 10a-10c As shown, the MPM list can include indices (e.g., from zero to five) and their associated intra-frame modes. Indices closer to zero can be closer to the top of the MPM list. For example, as... Figure 8 As shown, the default conditions can generate an MPM list that includes intra-frame mode 'A' as its zeroth element. The minimum and maximum values ​​of 'A' and 'B' are represented as minAB and maxAB, respectively, and '%' represents the modulo operator. Figures 10a-10c As shown, for example in ISP, the type of segmentation (e.g., horizontal segmentation or vertical segmentation) can be considered in the MPM construction.

[0142] Figure 11 This is an example flowchart for constructing a unified MPM list. Figure 11 The unified MPM list construction for intra-frame decoding tools (e.g., reference line zero intra-frame prediction, multi-reference line intra-frame prediction, and ISP) is shown. The unified MPM list can be modified for multi-reference line intra-frame prediction and ISP. In the case of multi-reference line intra-frame prediction, for example, since DC and planar modes are not used for multi-reference line intra-frame prediction, the DC mode and planar mode can be skipped from the unified MPM list. In the case of ISP, for example, since the DC mode is not used for ISP, the DC mode can be skipped from the unified MPM list or replaced by the angle mode in the unified MPM list.

[0143] In ISP, sub-partitions can be predicted and reconstructed from top to bottom or left to right based on the ISP partition type (e.g., sequential prediction). For example, when using horizontal partitioning, sub-partitions can be predicted and reconstructed from top to bottom. When using vertical partitioning, sub-partitions can be predicted and reconstructed from left to right. For example, sub-partitions can be predicted (e.g., subsequent sub-partitions) when previous sub-partitions have been reconstructed. In some examples, sub-partition prediction can be independent of the reconstruction of adjacent or previous partitions. This can reduce latency.

[0144] One or more MPM list construction processes can correspond to reference line zero intra-prediction, multi-reference line intra-prediction, and ISP. In the example, the individual MPM list construction processes used for different intra-decoding tools (e.g., reference line zero intra-prediction, multi-reference line intra-prediction, and ISP) can be unified. The unified MPM list can be based on multiple conditions. The MPM list can change when the corresponding conditions are met. For example, each condition can produce a different MPM list. A unified MPM list can be constructed.

[0145] Systems, methods, and means are disclosed for implementing an ISP, for example, by reducing latency in the ISP and / or simplifying the MPM list construction process. In some examples, prediction samples from previously adjacent sub-partitions can be used to predict subsequent sub-partitions. In some examples, prediction samples or reconstructed samples from previously adjacent sub-partitions can be used to predict subsequent sub-partitions. One or more examples in this paper concerning horizontal partitioning can also be applied to vertical partitioning. In vertical partitioning, relevant rows can be replaced by columns, and relevant bottom rows can be replaced by right columns. One or more examples in this paper can reduce latency.

[0146] Previous sub-partition prediction samples can be used as reference samples for prediction. ISP can perform sub-partition prediction and reconstruction sequentially. In some examples, sub-partition prediction may not depend on reconstructed samples from its neighboring sub-partitions. The first sub-partition can be predicted using the upper reference row and left reference column of the adjacent CU. The first sub-partition can be the topmost sub-partition used for horizontal splitting. The first sub-partition can be the leftmost sub-partition used for vertical splitting. Prediction can be performed for the remaining sub-partitions.

[0147] It can perform CU-based predictions, such as predicting sub-partitions, transform units (TUs), or prediction units (PUs). For example, if N This indicates the number of sub-partitions in the predicted CU. Therefore, in CU-based prediction, the prediction can be performed as follows. The first... N -1 is the bottom row (e.g., for horizontal splitting) or right column (e.g., for vertical splitting) of each of the sub-partitions. Figure 12Including (a) and (b), examples of CU-based predictions are shown, where samples from the reference lines above and to the left of adjacent CUs are used to predict the bottom rows of sub-partitions A, B, and C. Figure 12 As shown, CU can be horizontally divided into four sub-partitions A, B, C, and D. The bottom row of each of the first three sub-partitions A, B, and C can be predicted, for example, as... Figure 12 (a) and Figure 12 As shown in (b). These rows (or columns in the case of vertical segmentation) can be predicted using the upper reference row and left reference column of the adjacent CU. These rows (or columns in the case of vertical segmentation) can be predicted using the same prediction mode as the intra-frame mode of the CU. The predicted rows (or columns) can be used to predict adjacent sub-partitions. For example, as Figure 12 As shown in (b), the predicted bottom row samples of sub-partition B can be used to predict sub-partition C. Each of the sub-partitions of CU (e.g., N -1 sub-partitions) can be predicted.

[0148] For example, once the prediction is complete, reconstruction can be performed based on a sub-partition or on the entire CU. If reconstruction is performed on a sub-partition basis, residuals can be generated by applying inverse quantization and inverse transform to the sub-partition. If the reconstruction process is performed on the CU (e.g., the entire CU), then residuals can be generated for the entire CU. CU-based prediction can, for example, reduce latency for the ISP.

[0149] It can perform sub-partition-based predictions. Prediction of sub-partitions can be performed continuously using, for example, prediction samples from previous sub-partitions. Figure 13 An example of prediction based on sub-partitions is shown. Figure 13 As shown, CU can be horizontally divided into four sub-partitions: A, B, C, and D. (As...) Figure 13 As shown, the first sub-partition can be predicted using the upper reference row and left reference column of the adjacent CU. In the example, the first sub-partition could be the topmost sub-partition A. Figure 13 As shown in (a), the predicted sample from the bottom row of A and having a left-hand reference column can be used to predict sub-partition B. Reference Figure 13 (b) The bottom row prediction samples of sub-partition B (e.g., together with the left reference column) can be used to predict sub-partition C. This process can be repeated until the last sub-partition is predicted. The following procedure can be performed on vertically segmented sub-partitions: Figure 13 The prediction based on sub-partitions is shown in the figure.

[0150] For example, once the prediction is complete, reconstruction can be performed based on a sub-partition or on the entire CU. If reconstruction is performed on a sub-partition basis, residuals can be generated by applying inverse quantization and inverse transform to the sub-partition. If the reconstruction process is performed on the CU (e.g., the entire CU), then residuals can be generated for the entire CU. CU-based prediction can, for example, reduce latency for the ISP.

[0151] Figures 14(a)-(c) are schematic diagrams illustrating examples of pipeline stages for: (a) ISP, (b) ISP using sub-partition-based prediction, and (c) ISP using prediction based on multiple sub-partitions or multi-sub-partition prediction. Figures 14(a) and 14(b) illustrate pipelines for ISPs, and when considering CUs with sub-partitions A and B, the pipeline for ISPs can utilize sub-partition-based prediction. Figures 14(a) and 14(b) illustrate the various stages of the pipeline: prediction; inverse quantization and inverse transform; and reconstruction. Figure 14(c) illustrates the pipeline stages for an ISP based on multi-sub-partition intra-frame prediction (e.g., relative to the original ISP). Prediction of sub-partition B can be independent of reconstruction of sub-partition A. For example, as shown in Figures 14(a)-(c), latency reduction can be achieved.

[0152] For example, hybrid prediction can be performed to predict sub-partitions, TUs, or PUs. Sub-partitions can be predicted using prediction samples from neighboring sub-partitions or reconstructed samples from neighboring sub-partitions. For prediction samples, CU-based predictions or sub-partition-based predictions can be used. Figure 15 An example of hybrid prediction is shown, where sub-partitions can be predicted using either reconstructed samples or predicted samples from neighboring sub-partitions. For example... Figure 15 As shown, for example, the bottom row of sub-partition A can be predicted using CU-based prediction as described herein. For example, the residuals of sub-partitions A, B, C, and D can be determined by utilizing inverse quantization and inverse transform operations. For example, after obtaining the residuals, sub-partitions A and B can be predicted. Sub-partition A can be predicted using the upper reference row and left reference column of the adjacent CU. Sub-partition B can be predicted using the bottom predicted row of sub-partition A and the left reference column of the CU. Sub-partitions A and B can be reconstructed. The bottom reconstructed row and left reference column of sub-partition B can be used to predict sub-partition C. The bottom predicted row and left reference column of sub-partition C can be used to predict sub-partition D, for example, using the sub-partition-based prediction described herein.

[0153] For example, as described in this paper, the final sub-partition can be predicted from reconstructed samples from adjacent sub-partitions. For instance, the final sub-partition may be the furthest from one of the reference lines. In horizontal segmentation, the bottom sub-partition may be the furthest from the upper reference line. This can improve the accuracy of the prediction of the final sub-partition.

[0154] Hybrid prediction can be used to predict CU. For example, one or more sub-partitions of a CU can be predicted based on reconstructed samples of another (one or more) sub-partitions, and one or more sub-partitions of a CU can be predicted based on predicted samples of other (one or more) sub-partitions. Figure 16 An example of mixed forecasting is shown. Figure 16 In this case, sub-partition D can be predicted by reconstructing samples from the bottom row of sub-partition C. For example... Figure 16 As shown in (a), sub-partition C can be predicted from the bottom row prediction samples of sub-partition B using sub-partition-based prediction. Figure 16 As shown in (b), sub-partition C can be predicted from the bottom row prediction samples of sub-partition B by utilizing CU-based prediction.

[0155] The prediction type (e.g., CU-based prediction, sub-partition-based prediction, or hybrid prediction) can be selected based on the CU size. For example, hybrid prediction can be used for larger CUs (e.g., CUs larger than 16×16 pixels). Sub-partition-based prediction can be used for smaller CUs (e.g., CUs up to 16×16 pixels).

[0156] The prediction type can be selected based on the intra-frame mode. For intra-frame angular modes, sub-partition-based prediction can be used, while for non-angular modes, CU-based prediction can be used.

[0157] One or more prediction types described herein can be used in conjunction with regular ISP prediction, where prediction and reconstruction of sub-partitions are performed sequentially (e.g., top-down or left-to-right). Partition-based prediction described herein can be applied to relatively small CUs, such as 8×4 pixels or 4×8 pixels, and regular ISP can be used for larger CUs (e.g., CUs larger than 8×4 pixels or 4×8 pixels). In some cases, regular ISP can be disabled for relatively small CUs (e.g., CUs of 8×4 pixels and 4×8 pixels), while ISP can be applied to larger CUs (e.g., CUs larger than 8×4 pixels and 4×8 pixels).

[0158] In the example, samples for predicting sub-partitions can be adaptively selected. For example, the selected samples could be reconstructed samples, samples predicted from CU-based predictions, or samples predicted from sub-partition-based predictions. An index can be signaled to the video decoding device in the bitstream. The index can be signaled for each CU. The index can indicate the selection of samples used by the video encoding device. Table 2 shows examples of selections available in the adaptive scheme, for example, when considering CUs with four sub-partitions. As shown in Table 2, index 2 uses CU-based predictions for sub-partition B, sub-partition-based predictions for sub-partition C, and predictions based on reconstructed samples (e.g., ISP) for sub-partition D. As shown in Table 2, the adaptive scheme can restrict sub-partition A to use ISP and sub-partition B to use CU-based predictions. As further shown in Table 2, the number of selections in the adaptive scheme can be limited to a subset of the listed selections. Signaling overhead for the index can be reduced. For example, the number of selections can be limited to four selections corresponding to indices 0, 1, 2, and 4, as shown in Table 2. The prediction accuracy of the final sub-partition D can be improved with these limited choices. index Subpartition B Subpartition C Subpartition D 0 CU recon recon 1 CU CU recon 2 CU sub-part recon 3 CU recon CU 4 CU recon sub-part 5 CU CU sub-part 6 CU sub-part CU 7 CU sub-part sub-part 8 CU CU CU Table 2 lists the different options for sub-partition prediction using the adaptive scheme. Sub-partition A can use ISP. The following symbols are used in Table 2: CU, indicating CU-based prediction; sub-part, indicating sub-partition-based prediction; and recon, indicating prediction from reconstructed samples.

[0159] Intra-frame prediction of sub-partitions can be based on reference samples adjacent to the current block, such as CU-adjacent reference samples. Figure 17 Examples of prediction based on multiple sub-partitions or intra-frame prediction based on multiple sub-partitions are shown. Intra-frame prediction of two or more sub-partitions can be performed in parallel. Figure 17 As shown, intra-prediction of two or more sub-partitions can be performed simultaneously (e.g., all at once) based on CU neighbor reference samples associated with each sub-partition. This can reduce the reconstruction latency of each ISP sub-partition. Reconstruction of multiple consecutive ISP sub-partitions can be performed in parallel. Figure 14(c) illustrates the pipeline stages of an ISP based on multi-sub-partition intra-prediction (e.g., relative to the original ISP).

[0160] The PU to be intra-predicted may include transform units (TUs). PUs and TUs may have the same size. In some examples, a PU may include two or more TUs. TUs may include sub-partitions. By merging multiple TUs into a larger PU (e.g., larger than a single TU), intra-prediction of multiple TUs can be performed simultaneously. TU decoding / reconstruction can be performed in parallel. This can reduce TU decoding and / or processing latency.

[0161] The device can perform one or more of the following: CU-based intra-frame prediction, sub-partition-based intra-frame prediction, and multiple sub-partition-based intra-frame prediction. Figure 18 Examples of ISP sub-partitions for horizontally and vertically partitioned CUs are shown. Figure 18 As shown, ISP can be applied to CUs of various sizes. Figure 18 Examples of various vertically partitioned CUs and horizontally partitioned CUs are provided, such as 4×8, 4×N, 8×4, and 8×N. The device can receive an indication of whether the sub-partition type (e.g., the type of ISP) is horizontally or vertically partitioned. For example, based on one or more examples herein, the device can perform prediction based on either vertical or horizontal partitioning. Based on an indication that the sub-partition type is horizontally partitioned, the device can perform sub-partitioning in the horizontal direction. Based on an indication that the sub-partition type is vertically partitioned, the device can perform sub-partitioning in the vertical direction. In the example, the device can predict prediction blocks based on the type of ISP.

[0162] A prediction block width (PU) for a decoded block can be defined as the number of samples measured from one side of the PU to the other. The minimum prediction block width can be used in conjunction with the prediction of sub-partitions. A PU may include two or more sub-partitions (e.g., TU sub-partitions) that are vertically or horizontally divided. Figure 19C and 19D As shown in the diagram. The prediction block width can be the sub-partition width or a multiple of the sub-partition width, for example, as... Figure 19C and 19D As shown in the diagram. The sub-partition width and minimum prediction block width can be used to determine the prediction blocks used for prediction. For example, when the sub-partitioning type is vertical splitting, a prediction block with multiple sub-partitions can be used for prediction when the sub-partition width is less than the minimum prediction block width. Figure 19C and 19D As shown, the minimum prediction block width can be 4 samples. When the sub-partition width is less than 4 samples, the prediction block width can be 4 samples. When the sub-partition width is equal to or greater than the minimum prediction block width, a prediction block with sub-partitions can be used for prediction. The prediction block width can be the sub-partition width.

[0163] Figure 19A An example is shown, in which Figure 18 The 1×N (N≥16) and 2×N (N≥8) sub-partitions shown can be removed and replaced by 4×N (e.g., N>8) sub-partitions. A 1×N sub-partition can represent a sub-partition with one sample width and N sample heights. A 2×N sub-partition can represent a sub-partition with two sample widths and N sample heights. A 4×N sub-partition can represent a sub-partition with four sample widths and N sample heights.

[0164] A 4×N sub-partition has a width of four samples and a height of N samples. The sub-partition width and / or height can be used to sub-partition the current block for intra-frame prediction. The sub-partition width of the current block can be determined at least in part based on whether the ISP type of the current block is vertical or horizontal, such as... Figure 18 As shown in the image.

[0165] The minimum prediction block width can be four samples. As described in this paper, the PU can include one or more horizontally or vertically partitioned sub-partitions. A PU can include one or more 4×N sub-partitions, for example, because 1×N (N≥16) and 2×N (N≥8) sub-partitions are replaced by 4×N sub-partitions. This can be combined... Figure 19A The example shown demonstrates how to perform an ISP using intra-frame prediction based on multiple sub-partitions. The minimum prediction block width can be used to determine the PU corresponding to the respective sub-partition.

[0166] Predictions for prediction blocks can be based on neighboring samples of the CU and / or reconstructed sub-partitions. For example... Figure 19A As shown, replacing 1×N (N≥16) and 2×N (N≥8) sub-partitions with 4×N sub-partitions results in no vertical segmentation ISP for 4×8 and 4×N (N≥16) CUs. Sub-partition prediction can be based on prediction units of at least 4 sample widths. Prediction of prediction blocks in such 4×8 or 4×N (N≥16) CUs can be based on CU neighboring samples rather than, for example, reconstructed sub-partitions. Replacing 1×N (N≥16) and 2×N (N≥8) sub-partitions with 4×N sub-partitions results in two 4×N vertical segmentation sub-partitions for 8×N (N≥8) CUs, highlighted by bold and underlined font. Prediction of prediction blocks in such 8×N (N≥8) CUs can be based on CU neighboring samples and reconstructed sub-partitions.

[0167] For certain CU sizes, the PU can include multiple TU sub-partitions. A 4×8 CU can represent a CU or decoder block with a width of four samples and a height of eight samples. A 4×N CU can represent a CU or decoder block with a width of four samples and a height of N samples. The PU can contain multiple TU sub-partitions, for example, for CUs of sizes 4×8, 4×N, 8×4, and 8×N.

[0168] In one or more examples in this paper, the CU and decode block are used interchangeably, the PU and predict block are used interchangeably, and the TU and transform block are used interchangeably.

[0169] In the example, the device can determine that ISP is enabled for the current block. The device can determine the sub-partition width of the current block. For each sub-partition in the current block, the device can determine the corresponding prediction block based on the sub-partition width and the minimum prediction block width. The device can use reference samples adjacent to the prediction block to predict the prediction block. For example, the reference samples may include CU-adjacent reference samples and / or samples from the reconstructed sub-partitions.

[0170] A reference sample may be adjacent to a CU, PU, ​​or TU. For example, a reference sample adjacent to a PU may include reference samples that are either near or spatially separated from the PU. A reference sample adjacent to a PU may be included in a reference sample within an adjacent sub-partition. A reference sample adjacent to a PU may be included in a reference sample within an adjacent CU, PU, ​​or TU. An adjacent sub-partition may be near or spatially separated from the PU.

[0171] You can remove N×2 (N≥8) subpartitions and replace them with N×4 subpartitions. Figure 19B An example is shown, in which, except for using Figure 19A The 4×N subpartitions shown replace the 1×N (N≥16) and 2×N (N≥8) subpartitions, Figure 18 The N×2 (N≥8) subpartitions shown can be removed and replaced with N×4 subpartitions. For example... Figure 19B As shown, except for the absence of a vertical partitioning ISP for 4×8 and 4×N (N≥16) CUs, for example, no horizontal partitioning ISP can be applied to 8×4 CUs. Two N×4 horizontal partitioning sub-partitions can be applied to N×8 (N≥8) CUs, which are highlighted according to the bold and underlined font. This can be combined... Figure 19B The example shown demonstrates how to perform an ISP using intra-frame prediction based on multiple sub-partitions.

[0172] A PU may include 32 or more samples. Figure 19C An example is shown, where, for example, except for using Figure 19A The 4×N sub-partitions shown replace the 1×N (N≥16) and 2×N (N≥8) sub-partitions, and the PU can contain 32 samples. For example, this variation from 16 samples per PU to 32 samples per PU can be achieved by forcing a minimum PU size for intra-frame prediction. In the example, the minimum PU size can include a 4×8 or 8×4 PU size. Figure 19C Use bold and underline to highlight. For example... Figure 19C As shown, the smallest PU size (e.g., 4×8 PU or 8×4 PU) can involve a TU sub-partition of the same PU size or two TU sub-partitions of half a PU size (e.g., 4×4). This can be combined... Figure 19CThe example shown demonstrates how to perform an intra-frame prediction using multiple sub-partitions. For instance, when the PU can include 32 or more samples, multi-sub-partition prediction as described herein can be performed.

[0173] Figure 19D An example is shown, in which Figure 19A , 19B The examples shown in 19C can be applied simultaneously. In the examples, 1×N, 2×N, and N×2 sub-partition removal can be applied simultaneously with a 4×8 or 8×4 minimum PU size constraint, for example, with the worst-case ISP throughput of 32 samples per PU cycle. This can be combined with... Figure 19D The example shown demonstrates how to perform an ISP using intra-frame prediction based on multiple sub-partitions.

[0174] It can provide systems, methods, and means for constructing MPM lists. Figure 20 An example of unified MPM list generation / construction is shown. In the unified MPM list construction example, multiple conditions (e.g., the four conditions listed in Table 1) can be checked to determine the combined MPM list. For example, conditions 1 and 3 listed in Table 1 can be combined into a single condition. Table 3 shows an example of the conditions for unified MPM list construction. As shown in Table 3, a single condition (e.g., condition (b)) can be used to check whether intra-frame mode A or intra-frame mode B is an angular mode. Serial Number condition Equivalent logical conditions Equivalent condition number in Table 1 （a） A and B are completely different perspective modes. (A≠B) && (A and B) > 1 2 （b） A or B is the angle mode. (A or B) > 1 N / A （c） Default value (none of the above conditions are met) - 4 Table 3 shows example conditions for constructing a unified MPM list.

[0175] In the example, such as Figure 11 As shown, the MPM list generated when the conditions listed in Table 3 (e.g., condition (b)) are met can be the same as the MPM list generated when the conditions listed in Table 1 (e.g., condition 3) are met. Figure 20 The example of constructing a unified MPM list is shown in the figure.

Claims

1. A device for video decoding, comprising: The processor is configured as follows: The block is determined to be divided into a first sub-partition and a second sub-partition; Use the reference sample of the block to predict the first sample of the second sub-partition; Use samples from the first sub-partition to predict the second sample from the second sub-partition; as well as The block is decoded based on the first and second samples predicted from the second sub-partition.

2. The apparatus of claim 1, wherein, The processor is further configured to: Using a row of samples adjacent to the block as reference samples, multiple samples of the second sub-partition are predicted, wherein the multiple samples include the first sample; and The first sub-partition is used to predict one or more remaining samples in the second sub-partition, wherein the one or more remaining samples include the second sample.

3. The apparatus of claim 2, wherein, The processor is further configured to: Determine a row of samples adjacent to the block, wherein the row of samples includes a reference sample of the block; and The first sub-partition is predicted using a row of samples adjacent to the block.

4. The device of claim 1, wherein the block is divided in a vertical direction, the first sub-partition is to the left of the second sub-partition, and the processor is further configured to: The rightmost column of samples in the second sub-partition is predicted using a column of reference samples on the left side of the block, wherein the rightmost column of samples in the second sub-partition includes the first sample, and wherein a column of reference samples on the left side of the block includes the reference sample of the block.

5. The device of claim 1, wherein the block is divided in a horizontal direction, the first sub-partition is above the second sub-partition, and the processor is further configured to: A row of reference samples above the block is used to predict a row of samples at the bottom of the second sub-partition, wherein the row of samples at the bottom of the second sub-partition includes the first sample, and wherein the row of reference samples above the block includes the reference sample of the block.

6. The device according to claim 5, wherein, The next row of samples at the bottom of the second sub-partition is then predicted using a column of neighboring samples on the left side of the block.

7. The device according to claim 1, wherein, The processor is further configured to: Determine the intra-prediction mode associated with the block, wherein the intra-prediction mode associated with the block is used to predict the first sample of the second sub-partition.

8. The device according to claim 1, wherein, The processor is further configured to: Predicting multiple sub-partitions of the block, wherein the multiple sub-partitions include a first sub-partition and a second sub-partition; and Obtain the residual associated with the block, wherein the block is decoded based on the plurality of sub-partitions and the residual.

9. The device according to claim 1, wherein, The processor is further configured to: Predict multiple sub-partitions of the block, wherein the multiple sub-partitions include a first sub-partition and a second sub-partition; Obtain the residual associated with the first sub-partition; as well as The first sub-partition is reconstructed based on the residual and the first sub-partition, and the block is decoded based on the reconstructed first sub-partition.

10. The device according to claim 1, wherein, The block is divided using intra-frame sub-partitions (ISPs).

11. An apparatus for video encoding, comprising: The processor is configured as follows: The block is determined to be divided into a first sub-partition and a second sub-partition; Use the reference sample of the block to predict the first sample of the second sub-partition; Use samples from the first sub-partition to predict the second sample from the second sub-partition; as well as The block is encoded based on the first and second samples predicted from the second sub-partition.

12. The device according to claim 11, wherein, The processor is further configured to: Using a row of samples adjacent to the block as reference samples, multiple samples of the second sub-partition are predicted, wherein the multiple samples include the first sample; and The first sub-partition is used to predict one or more remaining samples in the second sub-partition, wherein the one or more remaining samples include the second sample.

13. The device according to claim 12, wherein, The processor is further configured to: Determine a row of samples adjacent to the block, wherein the row of samples includes a reference sample of the block; and The first sub-partition is predicted using a row of samples adjacent to the block.

14. The device of claim 11, wherein the block is divided in a vertical direction, the first sub-partition is to the left of the second sub-partition, and the processor is further configured to: The rightmost column of samples in the second sub-partition is predicted using a column of reference samples on the left side of the block, wherein the rightmost column of samples in the second sub-partition includes the first sample, and wherein a column of reference samples on the left side of the block includes the reference sample of the block.

15. A method for video decoding, comprising: The block is determined to be divided into a first sub-partition and a second sub-partition; Use the reference sample of the block to predict the first sample of the second sub-partition; Use samples from the first sub-partition to predict the second sample from the second sub-partition; as well as The block is decoded based on the first and second samples predicted from the second sub-partition.

16. The method of claim 15, further comprising: A row of samples adjacent to the block is used as a reference sample to predict multiple samples in the second sub-partition, wherein the multiple samples include the first sample; as well as The first sub-partition is used to predict one or more remaining samples in the second sub-partition, wherein the one or more remaining samples include the second sample.

17. The method of claim 15, further comprising: Determine a row of samples adjacent to the block, wherein the row of samples includes a reference sample of the block; as well as The first sub-partition is predicted using a row of samples adjacent to the block.

18. A method for video encoding, comprising: The block is determined to be divided into a first sub-partition and a second sub-partition; Use the reference sample of the block to predict the first sample of the second sub-partition; Use samples from the first sub-partition to predict the second sample from the second sub-partition; as well as The block is encoded based on the first and second samples predicted from the second sub-partition.

19. The method of claim 18, further comprising: A row of samples adjacent to the block is used as a reference sample to predict multiple samples in the second sub-partition, wherein the multiple samples include the first sample; as well as The first sub-partition is used to predict one or more remaining samples in the second sub-partition, wherein the one or more remaining samples include the second sample.

20. The method of claim 18, further comprising: Determine a row of samples adjacent to the block, wherein the row of samples includes a reference sample of the block; as well as The first sub-partition is predicted using a row of samples adjacent to the block.

21. A base station configured to operate within a radio access network (RAN), the base station comprising: One or more transceivers; as well as A control circuit that can be operatively coupled to one or more transceivers. The base station is configured as follows: Transmitting and receiving wireless signals on one or more carrier frequencies associated with a cell; Provide wireless service coverage to a geographic area associated with the cell, the geographic area being fixed or subject to change over time; as well as Operate one or more carrier frequencies in licensed spectrum, unlicensed spectrum, or a combination thereof.

Images