Intra-fragmentation related intra-frame coding

Abstract

This invention provides systems, methods, and tools for decoding and / or encoding a coding unit (CU). An intra-frame prediction mode for the CU can be determined. A segmentation mode can be determined based on the intra-frame prediction mode to generate multiple sub-partitions in the CU. Prediction of a first sub-partition among the multiple sub-partitions in the CU can be based on a reference sample in a second sub-partition among the multiple sub-partitions in the CU. The CU can be decoded and / or encoded, for example, based on the determined segmentation mode.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Serial No. 62 / 901,497, filed on September 17, 2019, the entire disclosure of which is incorporated herein by reference. Background Technology

[0003] Video coding systems can be used to compress digital video signals, for example, to reduce the storage and / or transmission bandwidth associated with such signals. Video coding systems can include block-based, wavelet-based, and / or object-based systems. Hybrid block-based video coding systems can be deployed. Summary of the Invention

[0004] Systems, methods, and tools for decoding and / or encoding CUs using segmentation patterns determined based on intra-frame prediction patterns for coding units (CUs) are described.

[0005] One or more processors may be configured to determine an intra-prediction mode for the CU. One or more processors may be configured to determine a segmentation mode based on the intra-prediction mode to generate multiple sub-partitions in the CU, wherein prediction of a first sub-partition among the multiple sub-partitions in the CU may be based on a reference sample in a second sub-partition among the multiple sub-partitions in the CU. One or more processors may be configured to decode the CU based on the determined segmentation mode.

[0006] One or more processors may be configured to determine an intra-prediction mode for the CU. One or more processors may be configured to determine a segmentation mode based on the intra-prediction mode to generate multiple sub-partitions in the CU, wherein the prediction of a first sub-partition among the multiple sub-partitions in the CU is based on a reference sample in a second sub-partition among the multiple sub-partitions in the CU. One or more processors may be configured to encode the CU based on the determined segmentation mode.

[0007] An apparatus may include the means as described above and at least one of: (i) an antenna configured to receive a signal including data representing an image, (ii) a band limiter configured to limit the received signal to a frequency band including the data representing the image, or (iii) a display configured to display an image. Non-limiting examples of such apparatus may include a TV, a mobile phone, a tablet computer, or a set-top box (STB). An apparatus may include an access unit configured to access data (e.g., including residuals) generated based on a segmentation pattern of a CU determined according to the means as described in any of the above combinations, and a transmitter configured to transmit data (e.g., including residuals).

[0008] A method may include determining an intra-prediction mode for a control unit (CU). The method may include determining a segmentation mode based on the intra-prediction mode to generate multiple sub-partitions in the CU, wherein a prediction of a first sub-partition among the multiple sub-partitions in the CU is based on a reference sample in a second sub-partition among the multiple sub-partitions in the CU. The method may include decoding the CU based on the determined segmentation mode.

[0009] A method may include determining an intra-prediction mode for a control unit (CU). The method may include determining a segmentation mode based on the intra-prediction mode to generate multiple sub-partitions in the CU, wherein a prediction of a first sub-partition among the multiple sub-partitions in the CU is based on a reference sample in a second sub-partition among the multiple sub-partitions in the CU. The method may include encoding the CU based on the determined segmentation mode.

[0010] The device may employ the method described above, and such a device may include at least one of: (i) an antenna configured to receive a signal including data representing an image, (ii) a band limiter configured to limit the received signal to a frequency band including the data representing the image, or (iii) a display configured to display the image. Non-limiting examples of such devices include televisions, mobile phones, tablet computers, or set-top boxes (STBs).

[0011] The apparatus may employ these methods, and such apparatus may include an access unit configured to access data (e.g., including residuals) generated based on a segmentation pattern determined for the CU, and a transmitter configured to transmit data (e.g., including residuals).

[0012] A non-transitory computer-readable medium may include data content generated according to any one of the methods. The non-transitory computer-readable medium may enable one or more processors to execute any one of the methods.

[0013] The computer program product may include instructions for performing any of the methods when executed by one or more processors.

[0014] The signal may include a residual generated based on the CU segmentation pattern determined according to any of the methods described.

[0015] The method may include accessing data comprising residuals generated based on a partitioning pattern of a CU determined according to any one of the methods, and transmitting data comprising the residuals. Attached Figure Description

[0016] Figure 1A This is a system diagram illustrating an exemplary communication system.

[0017] Figure 1B It shows that it can be used Figure 1AA system diagram of an exemplary wireless transmit / receive unit (WTRU) used within the communication system shown.

[0018] Figure 1C It shows that it can be used Figure 1A The diagram shows an exemplary radio access network (RAN) and an exemplary core network (CN) used within the communication system.

[0019] Figure 1D It shows that it can be used Figure 1A The system diagram shows another exemplary RAN and another exemplary CN used within the communication system shown.

[0020] Figure 2 This is a schematic diagram illustrating an exemplary video encoder.

[0021] Figure 3 This is a schematic diagram illustrating an example of a video decoder.

[0022] Figure 4 An example of a system in which various aspects and examples can be implemented is shown.

[0023] Figure 5 Exemplary partitioning of CUs that allow for 4×8 and 8×4 is shown.

[0024] Figure 6 An exemplary partitioning that allows coding units (CUs) larger than 4×8 or 8×4 is shown.

[0025] Figure 7 An example of the prediction process in an intra-fractional sub-partition (ISP) is shown: (a) the CU includes four sub-partitions A, B, C and D; (b) sub-partition A can be predicted and reconstructed, and the bottom reconstructed row can be used to predict B; and (c) the bottom row of the reconstructed sample of B can be used to predict C.

[0026] Figure 8 Examples of intra-frame prediction modes are shown (e.g., including directional modes).

[0027] Figure 9 An exemplary reference sample for wide-angle intra-frame prediction is shown, which has (a) a relatively large width (compared to height) or (b) a relatively large height (compared to width).

[0028] Figure 10 An example of multiple reference lines (MRLs) used for intra-frame prediction is shown.

[0029] Figure 11 Examples of ISP segmentation modes limited by intra-prediction modes are shown: (a) horizontal ISP segmentation for intra-mode; and (b) vertical ISP segmentation for intra-mode.

[0030] Figure 12 Examples of a method 1200 for decoding or encoding a CU using a segmentation mode based on an intra-frame prediction mode, according to one or more examples described herein, are shown. Detailed Implementation

[0031] Figure 1A This is a schematic diagram illustrating an exemplary communication system 100 in which one or more of the disclosed examples may be implemented. Communication system 100 may be a multiple access system providing content such as voice, data, video, messaging, broadcasting, etc., to multiple wireless users. Communication system 100 enables multiple wireless users to access such content through the sharing of system resources (including wireless bandwidth). For example, communication system 100 may employ one or more channel access methods, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal FDMA (OFDMA), Single Carrier FDMA (SC-FDMA), Zero-Tail Unique Word DFT Extended OFDM (ZT UW DTS-s OFDM), Unique Word OFDM (UW-OFDM), Resource Block Filtered OFDM, Filter Bank Multicarrier (FBMC), etc.

[0032] like Figure 1AAs shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, public switched telephone network (PSTN) 108, Internet 110, and other networks 112. However, it should be understood that the disclosed examples contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, and 102d can be any type of device configured to operate and / or communicate in a wireless environment. As an example, WTRUs 102a, 102b, 102c, and 102d (any of which may be referred to as a “station” and / or “STA”) may be configured to transmit and / or receive wireless signals and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscription-based units, pagers, cellular phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated processing chain environments), consumer electronics devices, devices operating on commercial and / or industrial wireless networks, etc. Any of WTRUs 102a, 102b, 102c, and 102d may be interchangeably referred to as a UE.

[0033] The communication system 100 may also include base station 114a and / or base station 114b. Each of base stations 114a and 114b may be any type of device configured to wirelessly interface with at least one of WTRUs 102a, 102b, 102c, and 102d to facilitate access to one or more communication networks, such as CN106 / 115, Internet 110, and / or other networks 112. As an example, base stations 114a and 114b may be base transceiver stations (BTS), Node Bs, evolved Node Bs, home Node Bs, home evolved Node Bs, gNBs, NR Node Bs, site controllers, access points (APs), wireless routers, etc. Although base stations 114a and 114b are each depicted as a single element, it should be understood that base stations 114a and 114b may include any number of interconnected base stations and / or network elements.

[0034] Base station 114a may be part of RAN 104 / 113, which may also include other base stations and / or network elements (not shown), such as base station controllers (BSCs), radio network controllers (RNCs), relay nodes, etc. Base station 114a and / or base station 114b may be configured to transmit and / or receive radio signals on one or more carrier frequencies (which may be referred to as cells (not shown)). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage of radio services to a specific geographic area, which may be relatively fixed or changeable over time. A cell may be further divided into cell sectors. For example, the cell associated with base station 114a may be divided into three sectors. Thus, in one example, base station 114a may include three transceivers, i.e., one transceiver per sector of the cell. In one example, base station 114a may employ multiple-input multiple-output (MIMO) technology and may utilize multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in desired spatial directions.

[0035] Base stations 114a and 114b can communicate with one or more of WTRUs 102a, 102b, 102c, and 102d via air interface 116, which can be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). Any suitable radio access technology (RAT) can be used to establish air interface 116.

[0036] More specifically, as noted above, the communication system 100 may be a multiple access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, base stations 114a and WTRUs 102a, 102b, and 102c in RAN 104 / 113 may implement radio technologies such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may use Wideband CDMA (WCDMA) to establish air interfaces 115 / 116 / 117. WCDMA may include communication protocols such as High-Speed ​​Packet Access (HSPA) and / or evolved HSPA (HSPA+). HSPA may include High-Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High-Speed ​​UL Packet Access (HSUPA).

[0037] In one example, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as evolved UMTS terrestrial radio access (E-UTRA), which can use Long Term Evolution (LTE) and / or Advanced LTE (LTE-A) and / or Advanced LTE Pro (LTE-A Pro) to establish air interface 116.

[0038] In one example, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as NR radio access, which can use New Radio (NR) to establish air interface 116.

[0039] In one example, base station 114a and WTRUs 102a, 102b, and 102c can implement multiple radio access technologies. For example, base station 114a and WTRUs 102a, 102b, and 102c can, for instance, use a dual connectivity (DC) principle to implement both LTE and NR radio access together. Therefore, the air interface used by WTRUs 102a, 102b, and 102c can be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations (e.g., eNBs and gNBs).

[0040] In the example, base station 114a and WTRUs 102a, 102b, and 102c can implement radio technologies such as IEEE 802.11 (i.e., Wi-Fi), IEEE 802.16 (i.e., WiMAX), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Provisional Standard 2000 (IS-2000), Provisional Standard 95 (IS-95), Provisional Standard 856 (IS-856), Global System for Mobile Communications (GSM), GSM Enhanced Data Rate Evolution (EDGE), and GSM EDGE (GERAN).

[0041] Figure 1ABase station 114b can be, for example, a wireless router, a home node B, a home evolution node B, or an access point, and can utilize any suitable RAT to facilitate wireless connectivity in local areas such as commercial locations, homes, vehicles, campuses, industrial facilities, air corridors (e.g., for use by drones), roads, etc. In one example, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.11 to establish a wireless local area network (WLAN). In one example, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.15 to establish a wireless personal area network (WPAN). In one example, base station 114b and WTRUs 102c, 102d can utilize cellular-based RATs (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish picocells or femtocells. Figure 1A As shown, base station 114b may have a direct connection to Internet 110. Therefore, base station 114b may not need to access Internet 110 via CN 106 / 115.

[0042] RAN 104 / 113 can communicate with CN 106 / 115, which can be any type of network configured to provide voice, data, application, and / or Voice over Internet Protocol (VoIP) services to one or more of WTRU102a, 102b, 102c, and 102d. Data can have different Quality of Service (QoS) requirements, such as different throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. CN 106 / 115 can provide call control, billing services, location-based services, prepaid calling, internet connectivity, video distribution, etc., and / or perform advanced security functions such as user authentication. Although not explicitly stated... Figure 1A As shown, but it should be understood that RAN 104 / 113 and / or CN 106 / 115 can communicate directly or indirectly with other RANs that use the same RAT as RAN 104 / 113 or a different RAT. For example, in addition to being connected to RAN 104 / 113 which can utilize NR radio technology, CN 106 / 115 can also communicate with another RAN (not shown) that uses GSM, UMTS, CDMA2000, WiMAX, E-UTRA, or WiFi radio technology.

[0043] CN 106 / 115 may also act as a gateway for WTRU 102a, 102b, 102c, 102d to access PSTN 108, Internet 110, and / or other networks 112. PSTN 108 may include a circuit-switched telephone network providing Common Old-Style Telephone Service (POTS). Internet 110 may include a global system of interconnected computer networks and devices using common communication protocols such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and / or Internet Protocol (IP) from the TCP / IP Internet Protocol suite. Network 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, network 112 may include another CN connected to one or more RANs, which may use the same RAT as RAN 104 / 113 or a different RAT.

[0044] Some or all of the WTRUs 102a, 102b, 102c, and 102d in the communication system 100 may include multi-mode capabilities (e.g., WTRUs 102a, 102b, 102c, and 102d may include multiple transceivers for communicating with different wireless networks via different wireless links). For example, Figure 1A The WTRU102c shown can be configured to communicate with a base station 114a that can employ cellular-based radio technology and with a base station 114b that can employ IEEE 802 radio technology.

[0045] Figure 1B This is a system diagram illustrating an exemplary WTRU 102. (See diagram below.) Figure 1B As shown, WTRU 102 may include a processor 118, a transceiver 120, a transmitting / receiving element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power supply 134, a Global Positioning System (GPS) chipset 136, and / or other peripheral devices 138, etc. It should be understood that WTRU 102 may include any sub-combination of the foregoing elements.

[0046] Processor 118 can be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. Processor 118 can perform signal encoding, data processing, power control, input / output processing, and / or any other functions that enable WTRU 102 to operate in a wireless environment. Processor 118 can be coupled to transceiver 120, which can be coupled to transmitting / receiving element 122. Although Figure 1BThe processor 118 and transceiver 120 are depicted as separate components, but it should be understood that the processor 118 and transceiver 120 may be integrated together in an electronic package or chip.

[0047] Transmitting / receiving element 122 may be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) via air interface 116. For example, in one example, transmitting / receiving element 122 may be an antenna configured to transmit and / or receive RF signals. In one example, transmitting / receiving element 122 may be a transmitter / detector configured to transmit and / or receive, for example, IR, UV, or visible light signals. In one example, transmitting / receiving element 122 may be configured to transmit and / or receive both RF signals and optical signals. It should be understood that transmitting / receiving element 122 may be configured to transmit and / or receive any combination of wireless signals.

[0048] Although the transmitting / receiving element 122 is in Figure 1B While depicted as a single element, WTRU 102 may include any number of transmitting / receiving elements 122. More specifically, WTRU 102 may employ MIMO technology. Thus, in one example, WTRU 102 may include two or more transmitting / receiving elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals via air interface 116.

[0049] Transceiver 120 can be configured to modulate signals transmitted by transmitting / receiving element 122 and demodulate signals received by transmitting / receiving element 122. As noted above, WTRU 102 may have multi-mode capability. Therefore, transceiver 120 may include multiple transceivers to enable WTRU 102 to communicate via various RATs (such as NR and IEEE 802.11).

[0050] The processor 118 of WTRU 102 is coupled to a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) unit or an organic light-emitting diode (OLED) display unit) and can receive user input data therefrom. The processor 118 can also output user data to the speaker / microphone 124, keypad 126, and / or display / touchpad 128. Furthermore, the processor 118 can access information from any type of suitable memory (such as non-removable memory 130 and / or removable memory 132) and store data in any type of suitable memory. Non-removable memory 130 may include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. Removable memory 132 may include a user identity module (SIM) card, memory stick, secure digital storage (SD) card, etc. In this example, the processor 118 can access information from memory that is not physically located on WTRU 102 (e.g., on a server or home computer (not shown)) and store data in that memory.

[0051] The processor 118 may receive power from the power supply 134 and may be configured to distribute and / or control power to other components in the WTRU 102. The power supply 134 may be any suitable device for powering the WTRU 102. For example, the power supply 134 may include one or more dry cell battery packs (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0052] The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) about the current location of the WTRU 102. In addition to or instead of the information from the GPS chipset 136, the WTRU 102 may receive location information from base stations (e.g., base stations 114a, 114b) via air interface 116 and / or determine its location based on the timing of signals received from two or more nearby base stations. It should be understood that the WTRU 102 may acquire location information using any suitable location determination method.

[0053] The processor 118 may also be coupled to other peripheral devices 138, which may include one or more software and / or hardware modules providing additional features, functions, and / or wired or wireless connectivity. For example, peripheral device 138 may include an accelerometer, electronic compass, satellite transceiver, digital camera (for photos and / or video), Universal Serial Bus (USB) port, vibration device, television transceiver, hands-free headset, etc. Modules, FM radio units, digital music players, media players, video game player modules, internet browsers, virtual reality and / or augmented reality (VR / AR) devices, activity trackers, etc. Peripheral devices 138 may include one or more sensors, which may be one or more of the following: gyroscopes, accelerometers, Hall effect sensors, magnetometers, orientation sensors, proximity sensors, temperature sensors, time sensors; geolocation sensors; altimeters, light sensors, touch sensors, magnetometers, barometers, gesture sensors, biometric sensors, and / or humidity sensors.

[0054] WTRU 102 may include a full-duplex radio for which the transmission and reception of some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous. The full-duplex radio may include an interference management unit 139 for reducing and / or substantially eliminating self-interference through signal processing via hardware (e.g., a choke) or via a processor (e.g., a separate processor (not shown) or via processor 118). In one example, WTRU 102 may include a half-duplex radio for which the transmission and reception of some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) or downlink (e.g., for reception) may be concurrent and / or simultaneous.

[0055] Figure 1C This is a system diagram illustrating exemplary RAN 104 and CN 106. As described above, RAN 104 can communicate with WTRUs 102a, 102b, and 102c via air interface 116 using E-UTRA radio technology. RAN 104 can also communicate with CN 106.

[0056] RAN 104 may include evolved Node Bs 160a, 160b, and 160c, but it should be understood that RAN 104 may include any number of evolved Node Bs. Each evolved Node B 160a, 160b, and 160c may include one or more transceivers to communicate with WTRUs 102a, 102b, and 102c via air interface 116. In one example, evolved Node Bs 160a, 160b, and 160c may implement MIMO technology. Therefore, evolved Node B 160a may, for example, use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 102a.

[0057] Each of the evolved nodes B 160a, 160b, and 160c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, and user scheduling in the UL and / or DL, etc. Figure 1C As shown, evolution nodes B 160a, 160b, and 160c can communicate with each other via the X2 interface.

[0058] Figure 1C The CN 106 shown may include a Mobility Management Entity (MME) 162, a Serving Gateway (SGW) 164, and a Packet Data Network (PDN) Gateway (or PGW) 166. While each of the foregoing elements is depicted as part of the CN 106, it should be understood that any of these elements may be owned and / or operated by an entity other than a CN operator.

[0059] The MME 162 can connect to each of the evolved nodes B 162a, 162b, and 162c in RAN 104 via the S1 interface and can be used as a control node. For example, the MME 162 can be responsible for authenticating users of WTRUs 102a, 102b, and 102c, bearer activation / deactivation, selecting a specific serving gateway during the initial attachment of WTRUs 102a, 102b, and 102c, etc. The MME 162 can provide control plane functions for handover between RAN 104 and other RANs (not shown) employing other radio technologies such as GSM and / or WCDMA.

[0060] The SGW 164 can connect to each of the evolved Nodes B 160a, 160b, and 160c in RAN 104 via the S1 interface. The SGW 164 typically routes and forwards user data packets to and from WTRUs 102a, 102b, and 102c. The SGW 164 can perform other functions such as anchoring the user plane during inter-evolved Node B handovers, triggering paging when DL data is available for WTRUs 102a, 102b, and 102c, and managing and storing the context of WTRUs 102a, 102b, and 102c.

[0061] SGW 164 can be connected to PGW 166, which provides WTRU 102a, 102b, 102c with access to packet-switched networks (such as Internet 110) to facilitate communication between WTRU 102a, 102b, 102c and IP-enabled devices.

[0062] CN 106 can facilitate communication with other networks. For example, CN 106 can provide WTRUs 102a, 102b, and 102c with access to a circuit-switched network (such as PSTN 108) to facilitate communication between WTRUs 102a, 102b, and 102c and conventional landline communication equipment. For example, CN 106 may include an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between CN 106 and PSTN 108, or be able to communicate with such an IP gateway. Additionally, CN 106 can provide WTRUs 102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0063] Despite WTRU in Figures 1A to 1D While described as a wireless terminal, it is conceivable that in some examples, such a terminal could (e.g., temporarily or permanently) use a wired communication interface with a communication network.

[0064] In the example, other network 112 could be a WLAN.

[0065] A WLAN in Infrastructure Basic Services Set (BSS) mode may have an access point (AP) for the BSS and one or more sites (STAs) associated with the AP. The AP may have access or an interface to a distribution system (DS) or another type of wired / wireless network that carries traffic to and / or carries traffic out of the BSS. Traffic originating outside the BSS and destined for a STA can reach and be delivered to the STA via the AP. Traffic originating from a STA and destined for a destination outside the BSS can be sent to the AP for delivery to the appropriate destination. Traffic between STAs within the BSS can be sent via the AP, for example, where a source STA can send traffic to the AP, and the AP can deliver the traffic to the destination STA. Traffic between STAs within the BSS can be considered and / or referred to as point-to-point traffic. Point-to-point traffic can be sent between source and destination STAs (e.g., directly between them) using Direct Link Establishment (DLS). In the example, the DLS may use 802.11e DLS or 802.11z Tunneled DLS (TDLS). WLANs using the Standalone BSS (IBSS) mode may not have an access point (AP), and STAs within the IBSS or using the IBSS (e.g., all STAs) can communicate directly with each other. The IBSS communication mode may sometimes be referred to as the "ad-hoc" communication mode in this document.

[0066] When operating in 802.11ac infrastructure mode or a similar mode, the AP can transmit beacons on a fixed channel, such as the primary channel. The primary channel can be of fixed width (e.g., a 20 MHz bandwidth) or dynamically set via signaling. The primary channel can be the operating channel of the BSS and can be used by the STA to establish a connection with the AP. In an example, Carrier Sense Multiple Access / Collision Avoidance (CSMA / CA) can be implemented, for example, in an 802.11 system. For CSMA / CA, each STA (including the AP) can listen on the primary channel. If the primary channel is listened to / detected and / or determined to be busy by a particular STA, that STA can back off. A single STA (e.g., only one station) can transmit in a given BSS at any given time.

[0067] High-throughput (HT) STAs can communicate using a 40MHz wide channel, for example, by combining a primary 20MHz channel with adjacent or non-adjacent 20MHz channels to form a 40MHz wide channel.

[0068] The Very High Throughput (VHT) STA supports channels with widths of 20MHz, 40MHz, 80MHz, and / or 160MHz. 40MHz and / or 80MHz channels can be formed by combining consecutive 20MHz channels. A 160MHz channel can be formed by combining eight consecutive 20MHz channels, or by combining two non-consecutive 80MHz channels (this can be referred to as an 80+80 configuration). For the 80+80 configuration, after channel coding, data can be split into two streams by a segment parser. Each stream can be processed individually using Inverse Fast Fourier Transform (IFFT) and time-domain processing. These streams can be mapped to two 80MHz channels, and data can be transmitted via the transmitting STA. At the receiver of the receiving STA, the operations described above for the 80+80 configuration can be reversed, and the combined data can be sent to Media Access Control (MAC).

[0069] 802.11af and 802.11ah support operating modes below 1 GHz. Compared to those used in 802.11n and 802.11ac, 802.11af and 802.11ah reduce channel operating bandwidth and carrier. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV white space (TVWS) spectrum, while 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. As an example, 802.11ah may support instrument-type control / machine-type communication, such as MTC devices in macro coverage areas. MTC devices may have certain capabilities, such as limited capabilities, including support (e.g., only support) certain bandwidths and / or limited bandwidths. MTC devices may include batteries with battery life exceeding a threshold (e.g., to maintain a very long battery life).

[0070] WLAN systems supporting multiple channels, as well as channel bandwidths such as 802.11n, 802.11ac, 802.11af, and 802.11ah, include channels that can be designated as primary channels. A primary channel can have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel can be set and / or limited by STAs operating in the BSS (each supporting a minimum bandwidth operating mode). In the 802.11ah example, for STAs supporting (e.g., only supporting) a 1MHz mode (e.g., MTC type devices), the primary channel can be 1MHz wide, even if the AP and other STAs in the BSS support 2MHz, 4MHz, 8MHz, 16MHz, and / or other channel bandwidth operating modes. Carrier Sense and / or Network Allocation Vector (NAV) settings can depend on the status of the primary channel. If the primary channel is busy, for example, because an STA (supporting only the 1MHz operating mode) is transmitting to the AP, the entire available band can be considered busy even if most of the band remains idle and potentially available.

[0071] In the United States, the available frequency bands for 802.11ah are 902MHz to 928MHz. In South Korea, the available frequency bands are 917.5MHz to 923.5MHz. In Japan, the available frequency bands are 916.5MHz to 927.5MHz. The total available bandwidth for 802.11ah ranges from 6MHz to 26MHz, depending on the country code.

[0072] Figure 1D This is a system diagram illustrating exemplary RAN 113 and CN 115. As noted above, RAN 113 can communicate with WTRUs 102a, 102b, and 102c via air interface 116 using NR radio technology. RAN 113 can also communicate with CN 115.

[0073] RAN 113 may include gNBs 180a, 180b, and 180c, but it should be understood that RAN 113 may include any number of gNBs. Each of gNBs 180a, 180b, and 180c may include one or more transceivers for communication with WTRUs 102a, 102b, and 102c via air interface 116. In one example, gNBs 180a, 180b, and 180c may implement MIMO technology. For example, gNBs 180a and 180b may utilize beamforming to transmit signals to and / or receive signals from gNBs 180a, 180b, and 180c. Therefore, gNB 180a may, for example, use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 102a. In one example, gNBs 180a, 180b, and 180c may implement carrier aggregation technology. For example, gNB 180a can transmit multiple component carriers to WTRU 102a (not shown). A subset of these component carriers may be on unlicensed spectrum, while the remaining component carriers may be on licensed spectrum. In one example, gNBs 180a, 180b, and 180c can implement Cooperative Multipoint (CoMP) technology. For example, WTRU 102a can receive cooperative transmissions from gNB 180a and gNB 180b (and / or gNB 180c).

[0074] WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using transmissions associated with scalable parameter sets. For example, OFDM symbol spacing and / or OFDM subcarrier spacing can vary depending on different transmissions, different cells, and / or different portions of the radio transmission spectrum. WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using subframes or transmission time intervals (TTIs) of various or scalable lengths (e.g., containing different numbers of OFDM symbols and / or continuously varying absolute time lengths).

[0075] gNBs 180a, 180b, and 180c can be configured to communicate with WTRUs 102a, 102b, and 102c in standalone and / or non-standalone configurations. In standalone configuration, WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c without accessing other RANs (e.g., evolved Node Bs 160a, 160b, and 160c). In standalone configuration, WTRUs 102a, 102b, and 102c can use one or more of gNBs 180a, 180b, and 180c as mobility anchors. In standalone configuration, WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using signals in unlicensed frequency bands. In a non-standalone configuration, WTRUs 102a, 102b, and 102c can communicate or connect to gNBs 180a, 180b, and 180c, and also communicate or connect to other RANs (such as evolved Node Bs 160a, 160b, and 160c). For example, WTRUs 102a, 102b, and 102c can implement DC principles to communicate substantially simultaneously with one or more gNBs 180a, 180b, and 180c and one or more evolved Node Bs 160a, 160b, and 160c. In a non-standalone configuration, evolved Node Bs 160a, 160b, and 160c can be used as mobility anchors for WTRUs 102a, 102b, and 102c, and gNBs 180a, 180b, and 180c can provide additional coverage and / or throughput for serving WTRUs 102a, 102b, and 102c.

[0076] Each of gNBs 180a, 180b, and 180c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, network slicing support, dual connectivity, interoperability between NR and E-UTRA, routing of user plane data to User Plane Functions (UPF) 184a and 184b, routing of control plane information to Access and Mobility Management Functions (AMF) 182a and 182b, etc. Figure 1D As shown, gNB180a, 180b, and 180c can communicate with each other via the Xn interface.

[0077] Figure 1DThe CN 115 shown may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one Session Management Function (SMF) 183a, 183b, and possibly a Data Network (DN) 185a, 185b. Although each of the foregoing elements is depicted as part of the CN 115, it should be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0078] AMF 182a and 182b can connect to one or more of gNBs 180a, 180b, and 180c via the N2 interface in RAN 113 and can be used as control nodes. For example, AMF 182a and 182b can be responsible for authenticating users of WTRU 102a, 102b, and 102c, supporting network slicing (e.g., handling different PDU sessions with different requirements), selecting specific SMF 183a and 183b, managing registration areas, terminating NAS signaling, mobility management, etc. AMF 182a and 182b can use network slicing to customize CN support for WTRU 102a, 102b, and 102c based on the type of service used by WTRU 102a, 102b, and 102c. For example, different network slices can be established for different use cases, such as services that rely on Ultra-Reliable Low Latency (URLLC) access, services that rely on Enhanced Mobile Broadband (eMBB) access, and services for Machine Type Communication (MTC) access. The AMF162 can provide control plane functions for handover between RAN 113 and other RANs (not shown) that employ other radio technologies, such as LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies, such as WiFi.

[0079] SMFs 183a and 183b can connect to AMFs 182a and 182b in CN 115 via the N11 interface. SMFs 183a and 183b can also connect to UPFs 184a and 184b in CN 115 via the N4 interface. SMFs 183a and 183b can select and control UPFs 184a and 184b, and configure traffic routing through UPFs 184a and 184b. SMFs 183a and 183b can perform other functions, such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notifications. PDU session types can be IP-based, non-IP-based, Ethernet-based, etc.

[0080] UPF 184a and 184b can connect via the N3 interface to one or more of the gNBs 180a, 180b, and 180c in RAN 113. These gNBs can provide WTRU 102a, 102b, and 102c with access to packet-switched networks (such as Internet 110) to facilitate communication between WTRU 102a, 102b, 102c and IP-enabled devices. UPF 184 and 184b can perform other functions such as routing and forwarding packets, enforcing user plane policies, supporting multihomed PDU sessions, handling user plane QoS, buffering downlink packets, and providing mobility anchoring.

[0081] CN 115 may facilitate communication with other networks. For example, CN 115 may include an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between CN 115 and PSTN 108, or may communicate with such an IP gateway. Additionally, CN 115 may provide WTRU 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one example, WTRU 102a, 102b, 102c may be connected to DN185a, 185b via UPF 184a, 184b through the N3 interface to UPF 184a, 184b and the N6 interface between UPF 184a, 184b and local data networks (DNs) 185a, 185b.

[0082] Given Figures 1A to 1D as well as Figures 1A to 1D The corresponding descriptions herein refer to one or more of the functions described below, which may be performed by one or more emulation devices (not shown): WTRU102a-d, base station 114a-b, evolved Node B 160a-c, MME 162, SGW 164, PGW 166, gNB 180a-c, AMF182a-ab, UPF 184a-b, SMF 183a-b, DN 185a-b, and / or any other device described herein. An emulation device may be one or more devices configured to mimic one or more of the functions described herein. For example, an emulation device may be used to test other devices and / or simulate network and / or WTRU functions.

[0083] Simulation devices can be designed to perform one or more tests on other devices in laboratory and / or carrier network environments. For example, the one or more simulation devices may perform one or more or all functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices within the communication network. The one or more simulation devices may perform one or more or all functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. Simulation devices may be directly coupled to another device for testing purposes and / or may use over-the-air wireless communication to perform tests.

[0084] The one or more emulation devices may perform one or more (including all) functions without being implemented / deployed as part of a wired and / or wireless communication network. For example, the emulation devices may be used in test scenarios within a test laboratory and / or non-deployed (e.g., testing) wired and / or wireless communication networks to perform testing of one or more components. The one or more emulation devices may be test equipment. Direct RF coupling and / or wireless communication via RF circuitry (e.g., which may include one or more antennas) may be used by the emulation devices to transmit and / or receive data.

[0085] This application describes several aspects, including tools, features, examples, models, methods, etc. Many of these aspects are described in a particular manner, and are generally described in a way that may sound restrictive, at least to illustrate individual characteristics. However, this is for clarity and does not limit the application or scope of these aspects. In fact, all the different aspects can be combined and interchanged to provide further aspects. Furthermore, these aspects can also be combined and interchanged with aspects described in earlier filings.

[0086] The aspects described and envisioned in this application can be implemented in many different forms. Figures 5 to 12 Some examples are provided, but other examples are also envisioned. Figures 5 to 12 The discussion does not limit the breadth of specific implementations. At least one of these aspects generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a bitstream generated or encoded. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions stored thereon for encoding or decoding video data according to any of the methods, and / or computer-readable storage media having a bitstream generated according to any of the methods stored thereon.

[0087] In this application, the terms "reconstruction" and "decoding" are used interchangeably, as are the terms "pixel" and "sample," and the terms "image," "picture," and "frame." Generally, but not necessarily, the term "reconstruction" is used at the encoding end, while "decoding" is used at the decoding end.

[0088] This document describes various methods, and each method includes one or more steps or actions for implementing the method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or purpose of specific steps and / or actions may be modified or combined. Additionally, in various examples, terms such as "first," "second," etc., may be used to modify elements, components, steps, operations, etc., such as "first decoding" and "second decoding." Unless specifically required, the use of such terms does not imply a modification of the order of operations. Therefore, in this example, the first decoding does not need to be performed before the second decoding and may occur, for example, before, during, or in overlapping time periods of the second decoding.

[0089] The various methods and other aspects described in this application can be used to modify, for example... Figure 2 and Figure 3 The illustrated video encoder 200 and decoder 300 modules are, for example, decoding modules. Furthermore, the subject matter disclosed herein can be applied to, for example, any type, format, or version of video encoding (whether described in a standard or a recommendation), whether pre-existing or future-developed, and any extensions to such standards and recommendations. Unless otherwise indicated or technically excluded, the aspects described in this application may be used alone or in combination.

[0090] Various numerical values, such as angles and indices, are used in the examples describing this application. These and other specific values ​​are used for the purpose of describing the examples, and the aspects described are not limited to these specific values.

[0091] Figure 2 This is a schematic diagram illustrating an exemplary video encoder. Variations of the exemplary encoder 200 are contemplated, but encoder 200 is described below for clarity, without describing all anticipated variations.

[0092] Before being encoded, the video sequence may undergo pre-coding processing (201), such as applying color transformations to the input color image (e.g., a transformation from RGB 4:4:4 to YCbCr 4:2:0), or performing remapping of the input image components to obtain a signal distribution that is more resilient to compression (e.g., histogram equalization using one of the color components). Metadata may be associated with pre-processing and appended to the bitstream.

[0093] In encoder 200, the image is encoded by encoder elements as described below. The image to be encoded is partitioned (202) and processed in units such as coding units (CUs). For example, each unit is encoded using either an intra-frame mode or an inter-frame mode. When a unit is encoded in intra-frame mode, it performs intra-frame prediction (260). In inter-frame mode, motion estimation (275) and compensation (270) are performed. The encoder determines (205) which of the intra-frame mode or inter-frame mode is used to encode the unit and indicates the intra-frame / inter-frame decision by, for example, a prediction mode flag. For example, the prediction residual is calculated by subtracting (210) the prediction block from the original image block.

[0094] The predicted residual is then transformed (225) and quantized (230). The quantized transform coefficients, motion vectors, and other syntax elements are entropy encoded (245) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., encode the residual directly without applying the transform or quantization process.

[0095] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (255) to reconstruct the image block. A loop filter (265) is applied to the reconstructed image to perform, for example, unblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored in a reference image buffer (280).

[0096] Figure 3 This is a schematic diagram illustrating an example of a video decoder. In the exemplary decoder 300, the bitstream is decoded by decoder elements, as described below. The video decoder 300 generally performs operations similar to... Figure 2 The encoding process is the reverse of the decoding process. Encoder 200 may also perform video decoding as part of the encoding of video data. For example, encoder 200 may perform one or more video decoding steps as presented herein. The encoder may, for example, reconstruct the decoded image to maintain synchronization with the decoder relative to one or more of the following: a reference picture, entropy coding context, and other decoder-related state variables.

[0097] Specifically, the decoder's input includes a video bitstream, which can be generated by the video encoder 200. First, entropy decoding (330) is performed on the bitstream to obtain transform coefficients, motion vectors, and other encoded information. Picture partitioning information indicates how the picture should be partitioned. Therefore, the decoder can partition (335) the picture based on the decoded picture partitioning information. The transform coefficients are dequantized (340) and inverse transformed (350) to decode the prediction residuals. The decoded prediction residuals and prediction blocks are combined (355) to reconstruct the image blocks. Prediction blocks (370) can be obtained from intra-frame prediction (360) or motion-compensated prediction (i.e., inter-frame prediction) (375). A loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (380).

[0098] The decoded image may also undergo post-decoding processing (385), such as inverse color transformation (e.g., a transformation from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping of the remapping process performed in the pre-encoding process (201). Post-decoding processing may utilize metadata derived in the pre-encoding process and signaled in the bitstream. In one example, the decoded image (e.g., after applying a loop filter (365) and / or after post-decoding processing (385), if post-decoding processing is used) may be sent to a display device for presentation to the user.

[0099] Figure 4 This is a schematic diagram illustrating an example of a system in which the various aspects and examples described herein may be implemented. System 400 may be embodied as a device that includes the various components described below and is configured to perform one or more aspects of the aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of system 400 may be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one example, the processing and encoder / decoder elements of system 400 are distributed across multiple ICs and / or discrete components. In various examples, system 400 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input ports and / or output ports. In various examples, system 400 is configured to implement one or more aspects of the aspects described in this document.

[0100] System 400 includes at least one processor 410 configured to execute instructions loaded thereon for implementing various aspects, such as those described in this document. Processor 410 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 400 includes at least one memory 420 (e.g., a volatile memory device and / or a non-volatile memory device). System 400 includes a storage device 440 that may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 440 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0101] System 400 includes an encoder / decoder module 430 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 430 may include its own processor and memory. The encoder / decoder module 430 represents a module that can be included in a device to perform encoding and / or decoding functions. It is well known that a device may include one or both of an encoding module and a decoding module. Alternatively, the encoder / decoder module 430 may be implemented as a separate element of system 400 or may be incorporated within processor 410 as a combination of hardware and software known to those skilled in the art.

[0102] Program code to be loaded onto processor 410 or encoder / decoder 430 to execute the various aspects described in this document may be stored in storage device 440 and subsequently loaded onto memory 420 for execution by processor 410. According to various examples, one or more of processor 410, memory 420, storage device 440, and encoder / decoder module 430 may store one or more items from various categories during the execution of the processes described in this document. Such stored items may include, but are not limited to, input video, decoded or partially decoded video, bitstreams, matrices, variables, and intermediate or final results of processing equations, formulas, operations, and operational logic.

[0103] In some examples, the memory within processor 410 and / or encoder / decoder module 430 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other examples, external memory (e.g., the processing device may be processor 410 or encoder / decoder module 430) is used for one or more of these functions. External memory may be memory 420 and / or storage device 440, such as volatile memory and / or non-volatile flash memory. In several examples, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one example, fast external volatile memory (such as RAM) is used as working memory for video encoding and decoding operations. Examples of encoding and decoding operations include MPEG-2 (MPEG stands for Moving Picture Experts Group; MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Universal Video Coding, a new standard developed by the Joint Video Experts Group JVET).

[0104] As shown in block 445, inputs to the components of system 400 can be provided through various input devices. Such input devices include, but are not limited to: (i) a radio frequency (RF) section that receives, for example, RF signals transmitted over the air by a broadcaster; (ii) component (COMP) input terminals (or a set of COMP input terminals); (iii) universal serial bus (USB) input terminals; and / or (iv) high-definition multimedia interface (HDMI) input terminals. Figure 4 Other examples not shown include composite video.

[0105] In various examples, the input device of block 445 has associated corresponding input processing elements as known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting the signal band to a band); (ii) down-converting the selected signal; (iii) further band-limiting to a narrower band to select (e.g.,) a signal band that may be referred to as a channel in some examples; (iv) demodulating the down-converted and band-limited signal; (v) performing error correction; and (vi) demultiplexing to select the desired data packet stream. The RF section of various examples includes one or more elements for performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners that perform various functions among these functions, such as down-converting received signals to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or to baseband. In one set-top box example, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to the desired frequency band. Various examples rearrange the order of the above (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various examples, the RF section includes an antenna.

[0106] Furthermore, USB and / or HDMI terminals may include corresponding interface processors for connecting system 400 to other electronic devices across USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, within a separate input processing IC or within processor 410. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed, within a separate interface IC or within processor 410. Demodulated streams, error-corrected streams, and demultiplexed streams are provided to various processing elements, including, for example, processor 410 and encoder / decoder 430, which operate in conjunction with memory and storage elements to process the data streams as needed for presentation on the output device.

[0107] Various components of system 400 can be housed within an integrated housing. Within the integrated housing, the various components can be interconnected and data can be transferred between these components using a suitable connection arrangement 425 (e.g., internal buses known in the art, including inter-chip (I2C) buses, wiring, and printed circuit boards).

[0108] System 400 includes a communication interface 450 capable of communicating with other devices via a communication channel 460. The communication interface 450 may include, but is not limited to, a transceiver configured to transmit and receive data via the communication channel 460. The communication interface 450 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 460 may be implemented, for example, within a wired and / or wireless medium.

[0109] In various examples, wireless networks such as Wi-Fi networks, such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers), are used to stream or otherwise provide data to system 400. In these examples, the Wi-Fi signal is received via a communication channel 460 and a communication interface 450 suitable for Wi-Fi communication. The communication channel 460 in these examples is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other cross-platform communications. Other examples use a set-top box to provide streaming data to system 400, delivering data via an HDMI connection to input block 445. Still other examples use an RF connection to input block 445 to provide streaming data to system 400. As mentioned above, various examples provide data in a non-streaming manner. Additionally, various examples use wireless networks other than Wi-Fi, such as cellular networks or... network.

[0110] System 400 can provide output signals to various output devices, including a display 475, speakers 485, and other peripheral devices 495. Various examples of the display 475 include one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 475 can be used in televisions, tablets, laptops, mobile phones, or other devices. The display 475 can also be integrated with other components (e.g., in a smartphone) or standalone (e.g., an external monitor for a laptop). In various examples, other peripheral devices 495 include one or more of a standalone digital video disc (or digital universal disc) (for both terms being DVD), a disc player, a stereo system, and / or a lighting system. Various examples use one or more peripheral devices 495 that provide functionality based on the output of system 400. For example, a disc player performs the function of playing the output of system 400.

[0111] In various examples, signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols capable of device-to-device control with or without user intervention is used to transmit control signals between system 400 and display 475, speaker 485, or other peripheral devices 495. Output devices can be communicatively coupled to system 400 via dedicated connections through corresponding interfaces 470, 480, and 490. Alternatively, output devices can be connected to system 400 via communication interface 450 using communication channel 460. Display 475 and speaker 485 may be integrated into a single unit with other components of system 400 in electronic devices such as televisions. In various examples, display interface 470 includes display drivers, such as, for example, a timing controller (TCon) chip.

[0112] Alternatively, if the RF section of input 445 is part of a separate set-top box, the display 475 and speaker 485 may be separate from one or more other components. In various examples where the display 475 and speaker 485 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0113] These examples can be implemented by computer software implemented by processor 410, or by hardware, or by a combination of hardware and software. As a non-limiting example, these examples can be implemented by one or more integrated circuits. Memory 420 can be of any type suitable for the technical environment, and as a non-limiting example, it can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. Processor 410 can be of any type suitable for the technical environment, and as a non-limiting example, it can encompass one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0114] Various specific implementations participate in decoding. As used in this application, "decoding" may encompass all or part of a process performed, for example, on a received coded sequence to produce a final output suitable for display. In various examples, such a process includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various examples, such a process may also optionally include processes performed by a decoder of the various specific implementations described in this application, such as determining a segmentation mode based on an intra-frame prediction mode to generate multiple sub-partitions in a CU, for example, predicting a first sub-partition in a CU based on a reference sample in a second sub-partition in a CU, etc.

[0115] As another example, in one example, "decoding" refers only to entropy decoding; in another example, "decoding" refers only to differential decoding; and in yet another example, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" specifically refers to a subset of operations or broadly refers to a wider decoding process will be clear based on the specific context of the description and is believed to be well understood by those skilled in the art.

[0116] Various specific implementations participate in encoding. In a manner similar to the discussion above regarding “decoding,” the term “encoding,” as used herein, can encompass, for example, all or part of the process performed on an input video sequence to produce an encoded bitstream. In various examples, such processes include one or more processes typically performed by an encoder, such as partitioning, differential coding, transform, quantization, and entropy coding. In various examples, such processes may also optionally include processes performed by an encoder of the various specific implementations described herein, such as determining a segmentation mode based on an intra-frame prediction mode to generate multiple sub-partitions in the CU, for example, predicting a first sub-partition in the CU based on a reference sample in a second sub-partition among multiple sub-partitions in the CU, etc.

[0117] As another example, in one example, "decoding" refers only to entropy decoding; in another example, "decoding" refers only to differential decoding; and in yet another example, "decoding" refers to a combination of differential decoding and entropy decoding. Whether the phrase "encoding process" specifically refers to a subset of operations or broadly refers to a wider encoding process will be clear based on the specific context of the description and is believed to be well understood by those skilled in the art.

[0118] It should be noted that the syntax elements used in this article, such as the coding unit syntax intra_subpartitions_split_flag[x0][y0], predIntraMode, IntraSubPartitionsSplitType, and intra_subpartitions_mode_flag[x0][y0] in the luma intra-prediction mode, are descriptive terms. Therefore, they do not preclude the use of other syntax element names.

[0119] When the accompanying drawings are presented as flowcharts, it should be understood that block diagrams of the corresponding devices are also provided. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that flowcharts of the corresponding methods / processes are also provided.

[0120] Various example decodings are provided. Content-adaptive transforms can be applied. Specifically, the device can receive a video bitstream representing content. The video bitstream may include quantized transform coefficients of one or more blocks. A precision factor (e.g., offset) can be obtained (e.g., determined or signaled). The precision factor may have one or more precision values ​​to be used in encoder or decoder operation, or one or more offset values ​​to be used in quantization or dequantization. The precision factor may be associated with a block for performing at least one decoding function on that block. In one example, the precision factor may be based on the magnitude of the transform coefficients of the block. The precision factor can reduce the maximum magnitude of the transform coefficients to fit 16 bits.

[0121] The specific embodiments and aspects described herein may be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even if discussed only in the context of a single form of specific embodiment (e.g., discussed only as a method), specific embodiments of the discussed features may also be implemented in other forms (e.g., apparatus or program). Apparatus may be implemented, for example, in suitable hardware, software, and firmware. These methods may be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as, for example, computers, mobile phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0122] References to “an example” or “an example” or “an embodiment” or “an embodiment” and their other variations mean that a particular feature, structure, characteristic, etc., described in connection with that example is included in at least one example. Therefore, the phrases “in an example” or “in the example” or “in an embodiment” or “in the embodiment” and any other variations appearing in various places throughout this application do not necessarily refer to the same example.

[0123] Additionally, this application may involve "determining" various types of information. Determining information may include, for example, one or more of estimated information, calculated information, predicted information, or information retrieved from memory. Obtaining may include receiving, retrieving, constructing, generating, and / or determining.

[0124] Furthermore, this application may relate to "accessing" various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or more of these.

[0125] Furthermore, this application may relate to "receiving" various types of information. Like "access," "receiving" is intended to be a broad term. Receiving information may include, for example, accessing information or retrieving information (e.g., from memory) or more. Moreover, "receiving" typically involves one or more of the following during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0126] It should be understood that, for example, in the cases of “A / B,” “A and / or B,” and “at least one of A and B,” the use of any of the following “ / ,” “and / or,” and “at least one” is intended to cover selecting only the first listed option (A), or only the second listed option (B), or selecting both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C,” such phrases are intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or selecting all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to as many items as possible listed.

[0127] Moreover, as used herein, the term "signaling" refers to (among other things) instructing the corresponding decoder to do something. Encoder signals may include, for example, indications of intra-prediction modes, indications of enabled ISPs, etc. Thus, in one example, the same parameters are used at both the encoder and decoder. Therefore, for example, the encoder may transmit specific parameters (explicit signaling) to the decoder so that the decoder can use the same specific parameters. Conversely, if the decoder already has specific parameters as well as other parameters, signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select specific parameters. Bit savings are achieved in various examples by avoiding the transmission of any actual functions. It should be understood that signaling can be implemented in many ways. For example, in various examples, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the verb form of the term "signal" was mentioned above, the term "signal" can also be used as a noun in this article.

[0128] It will be apparent to those skilled in the art that the embodiments may produce various signals formatted to carry, for example, storable or transmissible information. The information may include, for example, instructions for performing a method or data generated by one of the embodiments. For example, the signal may be formatted to carry a bit stream of the example described. Such signals may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding the data stream and using a modulated carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal can be transmitted via a variety of different wired or wireless links. The signal may be stored on a processor-readable medium.

[0129] This document describes numerous examples. Features of the examples may be provided individually or in any combination across various claim classes and types. Furthermore, examples may include one or more of the features, devices, or aspects described individually or in any combination across various claim classes and types. For example, features described herein may be implemented in a bitstream or signal including information generated as described herein. This information may allow a decoder to decode the bitstream, the encoder, bitstream, and / or decoder being implemented according to any embodiment of the described embodiments. For example, features described herein may be implemented by creating and / or transmitting and / or receiving and / or decoding a bitstream or signal. For example, features described herein may implement a method, process, apparatus, medium storing instructions, medium storing data, or signal. For example, features described herein may be implemented by a TV, set-top box, mobile phone, tablet computer, or other electronic device performing decoding. The TV, set-top box, mobile phone, tablet computer, or other electronic device may display (e.g., using a monitor, screen, or other type of display) the resulting image (e.g., an image reconstructed from the residual of a video bitstream). The TV, set-top box, mobile phone, tablet computer, or other electronic device may receive a signal including an encoded image and perform decoding.

[0130] Various coding structures can be used. Flexible multi-type tree block partitioning (e.g., quadtree, binary tree, and / or ternary tree partitioning) can be performed. Intra-frame prediction can be performed. For example, one or more (e.g., 65) angular intra-frame prediction directions can be used, including wide-angle prediction, chroma component linear model (CCLM), and / or matrix-based intra-frame prediction (MP). Inter-frame prediction can be performed. One or more prediction directions can be predetermined. For example, affine motion models, sub-block temporal motion vector prediction (SbTMVP), adaptive motion vector precision, decoder-side motion vector correction (DMVR), triangular partitioning, combined intra-frame and inter-frame prediction (CIIP), merge mode with motion vector difference (MMVD), bidirectional optical flow (BDOF), pixel-refined optical flow (PROF), and / or dual prediction weighted average (BPWA) can be used. Transform, quantization, and coefficient coding can be performed. For example, multiple master transform selections (using DCT2, DST7, and DCT8), quadratic transform coding with low-frequency indivisible transform (LFNST), dependent quantization (using a maximum QP increasing from 51 to 63), and / or modified transform coefficient coding can be used. Loop filters (e.g., generalized adaptive loop filters (GALF)) can be used. Screen content coding (e.g., intra-block copy (IBC) and / or palette mode (PLT) for 4:4:4 content) can be performed. 360-degree video coding (e.g., horizontal surround motion compensation) can be performed.

[0131] The Intra-Frame Sub-Partition (ISP) mode can be used to encode (e.g., split) the CU into multiple sub-partitions.

[0132] An ISP can be a method of sequentially encoding a CU (e.g., an intra-mode CU) into multiple sub-partitions. For example, the CU can be divided into multiple sub-partitions based on a partitioning mode. A sub-partition can be a portion of a CU. Some or all of the sub-partitions can share the same intra-mode of the CU (e.g., the entire CU). Encoding overhead from signaling intra-mode variables used for the sub-partitions can be saved.

[0133] The ISP can be applied to the luma component. In one example, the ISP cannot be applied to the chroma component. In one example, the ISP can divide the intra-frame mode CU into multiple (e.g., two or four) sub-partitions. (Each) partition may include multiple (e.g., at least 16) samples. Partitioning or CU segmentation can be performed in some (e.g., horizontal or vertical) direction. In one or more examples, partitioning, dividing, or segmenting are used interchangeably. Figure 5 Exemplary partitioning for CUs that allow 4×8 and 8×4 is shown. For relatively small CUs (e.g., sizes 4×8 and 8×4), the ISP can partition the CU into two sub-partitions, such as... Figure 5 As shown. Figure 6An exemplary partitioning is shown that allows for CUs larger than 4×8 or 8×4. Relatively large CUs can be divided into four sub-partitions, for example... Figure 6 As shown. Intra-prediction modes can be used for some (e.g., all) sub-partitions. The MPM (Most Probable Mode) flag can be set to one. For example, the intra-prediction mode used for ISP can be a member of the MPM list. Intra-reference sample smoothing filters can be disabled for ISP. Reference line zeros can be used for ISP. The use of multi-reference line intra-prediction can be disabled.

[0134] For horizontal splitting (pattern), sub-partitions can be processed from top to bottom. For vertical splitting (pattern), sub-partitions can be processed from left to right. Each sub-partition can be predicted (e.g., each time). Residuals can be added to the sub-partitions to produce reconstructed sub-partitions. Samples from the reconstructed sub-partitions can be used to predict the next sub-partition.

[0135] Figure 7 An example of the prediction process in an ISP is shown: (a) a CU may include four sub-partitions A, B, C, and D; (b) sub-partition A (e.g., the second sub-partition) can be predicted and reconstructed, and the bottom reconstructed row of sub-partition A can be used to predict sub-partition B (e.g., the first sub-partition); and (c) the bottom row of the reconstructed sample of sub-partition B can be used to predict sub-partition C (e.g., the third sub-partition). Figure 7 As shown, at (b), the topmost sub-partition A can first be predicted using the upper and left reference lines (e.g., reference samples in the reference CU), and reconstructed by adding the residuals obtained after entropy decoding, inverse quantization, and inverse transform. The bottom row of the reconstructed sub-partition A can be used to predict sub-partition B, and sub-partition B can then be reconstructed. This process can be repeated for subsequent sub-partitions.

[0136] In one example, for the prediction unit (PU), 1×N or 2×N sub-partitions are not allowed. For the ISP PU, a minimum of 4×N sub-partitions is allowed, for example, to enhance hardware throughput performance. For the transform unit (TU), 1×N or 2×N sub-partitions are allowed, for example, to maintain the beneficial effect of the obtained coding efficiency.

[0137] Extensions to intra-prediction modes may include more-angle intra-prediction modes, an increase of 33 to 65 angles, and wide-angle prediction modes (e.g., for non-square blocks), such as... Figure 8 As shown. Figure 8 Examples of intra-prediction modes are shown (e.g., including directional modes). An intra-prediction mode with an index of 0 can be a planar mode, and an intra-prediction mode with an index of 1 can be a DC mode. Intra-prediction modes with indices of 2 to 66 can be used for angular intra-prediction modes of square blocks. Intra-prediction mode indices of -1 to -14 and 67 to 80 can be used for wide-angle intra-prediction modes of non-square blocks.

[0138] For example, the intra-frame wide-angle mode can be used for non-square coded blocks.

[0139] For certain intra-frame prediction directions, the direction can be defined as clockwise from 45 degrees to -135 degrees. For square coded blocks with equal width and height, a standard intra-frame angle pattern can be used for intra-frame prediction (e.g., indices from 2 to 66, such as...). Figure 8 (As shown). Wide-angle intra-prediction modes (e.g., indices -1 to -14 and 67 to 80) can be used, for example, for non-square coded blocks with unequal widths and heights. In one example, the original mode index can be used to signal the replaced mode, which is then remapped to the wide-angle mode index after parsing. The total number of intra-prediction modes can remain unchanged (e.g., 67), and the intra-mode coding can remain unchanged.

[0140] Wide-angle mode can be shown in Figure 9 The figure illustrates an exemplary reference sample for wide-angle intra-frame prediction, which has (a) a relatively large width (compared to height) or (b) a relatively large height (compared to width). Table 1 illustrates an exemplary wide-angle intra-frame prediction mode mapping process. As shown in Table 1, the original intra-frame mode index can be mapped to the mode index value of the wide-angle mode. As shown in Table 1, nW and nH can represent the width and height of the coding block, respectively. Table 1 provides an exemplary wide-angle intra-frame prediction mode mapping process.

[0141] Table 1

[0142]

[0143]

[0144] It can perform intra-frame prediction using multiple reference lines (MRL).

[0145] MRL intra-frame prediction can use more reference lines for intra-frame prediction. Figure 10 An example of multiple reference lines (MRLs) used for intra-frame prediction is shown. Figure 10 An example depicting four reference lines. (Example follows) Figure 10 As shown, samples for segments A and F do not need to be extracted from the reconstructed neighboring samples. Samples for segments A and F can be filled with the nearest samples from segments B and E, respectively. Some intra-frame image predictions can use the nearest reference line (e.g., reference line 0). In MRL, multiple additional lines can be used (e.g., two additional lines, including reference line 1 and reference line 3).

[0146] Intra-prediction factors can be generated using the index (mrl_idx) of the selected reference line. In the example, for reference line indices greater than 0, additional reference line modes (e.g., only) can be included in the MPM list, and / or the mpm index (e.g., only) can be signaled if no remaining modes are available. Reference line indices can be signaled before intra-prediction modes, and planar and DC modes can be excluded from intra-prediction modes if non-zero reference line indices are signaled. MRL can be disabled for the first line of a block within a CTU, for example, to prevent the use of extended reference samples outside the current CTU line.

[0147] Intra-frame reference sample smoothing filtering can be applied.

[0148] Smoothing filters can be applied to reference samples of intra-frame modes, for example, to improve prediction performance. In some examples, different reference sample smoothing filters can be applied to integer bevel angles (e.g., reference full-pixel reference samples) and fractional bevel angles (e.g., reference fractional-pixel reference samples). In one example, the [1 2 1] / 4 3-tap smoothing filter can be applied to intra-frame modes with indices of 0 (i.e., planar mode), -14, -12, -10, -6, 2, 34, 66, 72, 76, 78, 80, and a block size greater than 32. In some examples, it is not applied if MRL or ISP is used.

[0149] For fractional bevel angles, a 4-tap Gaussian filter can be used. Not all fractional bevel angles are suitable for reference sample smoothing filtering. In the examples, for angles that are nearly horizontal or vertical, a 4-tap interpolation filter (e.g., as described herein) can be used. A 4-tap interpolation filter may not produce a very large smoothing effect.

[0150] It can perform intra-frame angle mode interpolation filtering.

[0151] For example Figure 8One or more intra-frame modes of angle shown (e.g., intra-frame modes from mode indices -14 to 80) can be used to generate predicted samples using filtering of reference samples. For the luma component, filtering can be performed using a 4-tap component (e.g., using a 4-tap filter as specified in Table 3). 2-tap linear interpolation can be used for the chroma component. One or more (e.g., two) types of 4-tap filtering may be present. For example, a first (e.g., cubic) filter (e.g., for interpolation filtering) and / or a second (e.g., Gaussian) filter (e.g., for smoothing filtering) can be used. For example, a 4-tap Gaussian smoothing filter can be applied if the minimum angular mode distance from the horizontal or vertical direction is greater than a certain value (e.g., a predetermined value, such as a threshold). For example, a 4-tap cubic interpolation filter can be applied if the minimum angular mode distance from the horizontal or vertical direction is not greater than a threshold.

[0152] The selection decision can be represented by the variable "filterFlag". The variable "filterFlag" can be derived, for example, as shown in Table 2.

[0153] Table 2

[0154]

[0155] Table 3 provides examples of interpolation filter coefficients fC and fG.

[0156] Table 3

[0157]

[0158]

[0159] Table 4 provides examples of intraHorVerDistThres[nTbS] for various transform block sizes nTbS.

[0160] Table 4

[0161] nTbS=2 nTbS=3 nTbS = 4 nTbS = 5 nTbS = 6 nTbS = 7 intraHorVerDistThres[nTbS] 24 14 2 0 0 0

[0162] The parameter `refFilterFlag` indicates whether intra-frame reference sample smoothing filtering has been performed earlier. The parameter `refIdx` represents the index of the reference line, and if not equal to 0, indicates multiple reference lines (MRLs) for intra-frame mode coding. The parameter `IntraSubPartitionSplitType` indicates whether ISP is used and / or, if used, whether horizontal or vertical splitting is used. The variable `minDistVerHor` represents the minimum mode angular distance relative to the horizontal or vertical direction, compared to a block size-related value (e.g., a threshold), for example, to choose between a cubic (if `filterFlag=0`) and a Gaussian (if `filterFlag=1`) 4-tap filter used for interpolation. This decision condition may be referred to as the "MDIS" condition.

[0163] In one example, the 4-tap filter involved for the luminance component can be defined in Table 3. In Table 3, fC and fG can represent the cubic interpolation filter and the Gaussian smoothing filter, respectively.

[0164] For the luminance component, one or more angular intra-frame prediction samples can be obtained (e.g., derived) as described herein. Without loss of generality, it can be assumed that multiple reference lines can be omitted and a single line of intra-frame prediction reference samples (e.g., the top and left boundaries of reference samples from adjacent coded CUs or blocks) can be used.

[0165] For intra-frame mode indices greater than or equal to a threshold (e.g., 34), such as using a reference sample above the top boundary of the current coding block, an x-offset (e.g., a horizontal offset) can be calculated for a predicted sample at position (x, y) within the current coding block. The x-offset can be calculated based on the intra-frame prediction angle and / or the row (e.g., vertical) distance to the top boundary of the reference sample. The parameters iIdx and iFact can be defined as follows:

[0166] iIdx=(((y+1)*intraPredAngle)>>5) (1)

[0167] iFact=((y+1)*intraPredAngle)&31 (2)

[0168] Where iIdx can represent the full pixel portion of the x-offset, and iFact can represent the fractional pixel portion of the x-offset.

[0169] The variable intraPredAngle captures the effect from the corresponding intra-prediction direction. Fractional positions can be calculated to a specific precision (e.g., 1 / 32 pixel).

[0170] Depending on the intra-frame mode orientation, if the orientation is considered to be close to horizontal or vertical, the variable / parameter (e.g., filterFlag) can be set to one value (e.g., 0), otherwise to another value (e.g., 1). In one example, the variable / parameter could indicate whether a filter is used.

[0171] One or more interpolation filter coefficients fT[j] (e.g., where j = 0…3) can be determined (e.g., derived) as follows:

[0172] fT[j]=filterFlag? fG[iFact][j]:fC[iFact][j] (3)

[0173] The values ​​of the predicted samples predSamples[x][y] can be determined (e.g., derived) as follows:

[0174]

[0175] The variable ref[x] can represent the top boundary reference sample at position (x, -1) (e.g., where y = -1 indicates the top boundary). Clip(·) can be a function that clips a value into a series of pixel values ​​(e.g., valid pixel values).

[0176] For one or more (e.g., all) other intra-mode indices (e.g., less than 34), the y-offset (e.g., vertical offset) can be derived, for example, using a left-boundary reference sample on the left side of the current coding block, based on the intra-prediction angle and the column (e.g., horizontal) distance to the left boundary of the reference sample. The corresponding formula is as follows:

[0177] iIdx=(((x+1)*intraPredAngle)>>5) (5)

[0178] iFact=((x+1)*intraPredAngle)&31 (6)

[0179] One or more interpolation filter coefficients fT[j] can be derived as follows (e.g., where j = 0…3):

[0180] fT[j]=filterFlag? fG[iFact][j]:fC[iFact][j] (7)

[0181] The values ​​of the predicted samples predSamples[x][y] can be derived as follows:

[0182]

[0183] For one or more chromaticity components, a 2-tap linear filter can be applied to the fractional reference sample location, for example, as follows. If the parameter iFact is not equal to 0, the value of the predicted sample predSamples[x][y] can be obtained (e.g., derived) as follows:

[0184] predSamples[x][y]=((32–iFact)*ref[x+iIdx+1]+iFact*ref[x+iIdx+2]+16)>>5 (9)

[0185] If the parameter iFact equals 0, the value of the predicted sample predSamples[x][y] can be obtained (e.g., derived) as follows:

[0186] predSamples[x][y]=ref[x+iIdx+1] (10)

[0187] In the example, a cubic filter can (e.g., always) be used for interpolation of MRL and / or ISP. For regular intra-frame mode, the MDIS conditional check can be used to select between a cubic filter and a Gaussian filter.

[0188] The coding unit syntax can be used in conjunction with the lumen intra-prediction mode. Table 5 provides exemplary coding unit syntaxes for the lumen intra-prediction mode.

[0189] Table 5

[0190]

[0191] The index `intra_luma_ref_idx` signals the index of multiple reference lines (e.g., one of the reference lines) in the MRL. The indicators `tra_subpartitions_mode_flag` and `intra_subpartitions_split_flag` signal the ISP mode and split mode, respectively. The remainder of the flag variables can be used for MPM and intra-prediction mode derivation.

[0192] For entropy encoding indicating `intra_luma_not_planar_flag`, multiple (e.g., two) context encoding bins can be used. Which one is used depends on the indicator `intra_subpartitions_mode_flag` (e.g., whether ISP is used). If `intra_subpartitions_mode_flag` is equal to 0 (indicating ISP is not used), the context encoding bin with index 1 (e.g., the second bin) can be used. Otherwise, the first context encoding bin (e.g., the one with index 0) can be used.

[0193] For a given CU, the ISP segmentation mode may or may not depend on the intra-prediction mode. In the example, the ISP segmentation mode may depend on the CU's intra-prediction mode, such that in a sub-partition derived from the ISP segmentation mode, the intra-prediction of a later-encoded (e.g., first) sub-partition can reference a reconstructed sample from a previously-encoded (e.g., second) ISP sub-partition. An ISP segmentation mode that depends on the intra-prediction mode may have coding advantages or benefits over an ISP segmentation mode that does not depend on the intra-prediction mode.

[0194] In some examples, if the CU is encoded in ISP mode, for example due to the use of 1×N, 2×N, N×2 TU and other CU sizes with large width-to-height differences (e.g., 4×32, 4×64, etc.), intra-prediction reference sample filtering may be disabled, and in this example, ISP segmentation mode constraints based on intra-prediction mode may not be used.

[0195] For example, for 1×N, 2×N, and N×2 TUs, if the CU intra-frame prediction mode references the top boundary reference sample in the case of 1×N and 2×N TUs, or the left boundary reference sample in the case of N×2 TUs, then filtering with more reference samples can lead to larger fluctuations in prediction performance or degradation along a longer dimension of the TU.

[0196] The ISP segmentation mode can be determined based on the intra-frame prediction mode (e.g., subject to restrictions). The CU can be processed based on the determined segmentation mode (e.g., ...). Figure 12(As shown in 1230). In ISP segmentation mode, a sub-partition (e.g., the first sub-partition) can reference previously reconstructed sub-partition (e.g., the second) samples used for intra-prediction. In one example, for 1×N, 2×N, and N×2 TUs, intra-prediction can reference reference samples on one side of its longer dimension (e.g., the side with N samples, but not the shorter side with 1 or 2 samples). The risk of prediction performance having too much fluctuation or degradation within the 1×N, 2×N, and N×2 TUs can be significantly reduced. Limitations on intra-reference filtering are not required. Intra-reference filtering can be performed. Similar to additional filtering for non-ISP modes, better coding efficiency can be achieved for ISP modes.

[0197] Table 6 provides examples of intra-frame reference sample filtering in certain cases.

[0198] Table 6

[0199]

[0200] In some examples, reference sample smoothing filtering may not be applied to the ISP or MRL. Table 6 illustrates the differences in intra-frame reference sample filtering application across ISP, MRL, and regular intra-frame modes. A unified approach applicable to ISP, MRL, and regular intra-frame modes can be used. This unified approach supports consistency across ISP, MRL, and regular intra-frame modes.

[0201] One or more ISP-related methods may be used, including: ISP segmentation mode constraints based on intra-prediction mode and unified intra-reference sample filtering for MRL, ISP and regular intra-mode.

[0202] The ISP segmentation mode can be selected based on the intra-prediction mode. The intra-prediction mode can be encoded before the ISP segmentation mode indicating that the segmentation mode has a segmentation direction (e.g., horizontal or vertical).

[0203] In one example, given the CU's angle intra-prediction mode, if the CU is encoded into the ISP, the CU can be divided into multiple (e.g., 2 or 4) sub-partitions that can be encoded sequentially. After one sub-partition is encoded, the next sub-partition can use the reconstructed samples of the previous (e.g., the preceding) sub-partition as its reference samples for intra-prediction. The CU can be segmented horizontally or vertically. When using a certain ISP segmentation mode, based on the provided CU intra-prediction angle, the next sub-partition may or may not reference the reconstructed samples of the previous sub-partition.

[0204] In an example, the ISP splitting mode may depend on the intra prediction mode of the CU, such that for a certain intra prediction mode of the CU, a sub-partition intra prediction may be used to indicate the ISP splitting mode (e.g., only the ISP splitting mode) for the reconstructed samples of the previously encoded sub-partition.

[0205] The coding overhead for signaling the ISP splitting mode can be reduced in many cases, and thus the overall coding performance can be improved. For example, Figure 11 such a case is shown in which an example of the ISP splitting mode restricted by the intra prediction mode is illustrated.

[0206] The intra prediction mode for the CU (e.g., the directional mode) can be identified by an intra prediction mode index. Figure 8 One or more intra prediction modes in the intra prediction mode can be shown. The CU intra prediction mode index can be represented by an input variable named "predIntraMode". As Figure 12 shown in 1210 of, for example, the intra prediction mode for the CU can be determined using the intra prediction mode index. The ISP splitting mode can be indicated or encoded into a variable named intra_subpartitions_split_flag. The intra prediction mode can be classified into multiple (e.g., three) sets, i.e., {S1|predIntraMode>predIntraMode0}, {S2|predIntraMode<predIntraMode1 and predIntraMode is not equal to 0 and predIntraMode is not equal to 1}, {S3|remaining intra prediction modes, e.g., nodes that do not belong to S1 or S2}. The intra prediction mode can belong to one set (e.g., only one of S1, S2, and S3). predintramode 0 and predintramode 1 can be predetermined intra prediction modes. The parameters predintramode 0 and predintramode 1 can be dependent on the CU size.

[0207] Figure 11 An example of a horizontal ISP split indicating the top boundary reference sample of the intra mode is shown at, for example, (a) of. A certain indication (e.g., intra_subpartitions_split_flag) can be determined (e.g., inferred) as a value indicating the horizontal ISP split. As Figure 12 shown in the 1220 of, the split mode can be determined based on the intra prediction mode to generate multiple sub-partitions in the CU. The split mode can include a vertical split or a horizontal split. As Figure 11As shown in (a), if predIntraMode belongs to set S1 (e.g., angular intra-frame prediction can refer to the top boundary reference sample), then intra_subpartitions_split_flag can be inferred to be equal to 0 indicating the horizontal ISP split and not encoded in the bitstream. In one or more examples in this paper, the coded blocks and CUs are used interchangeably. Figure 11 As shown in (a) and (b), a CU may include a square CU with equal width and height, and multiple sub-partitions in a CU may be rectangular.

[0208] Figure 11 Example (b) shows an example of vertical ISP segmentation of the left boundary reference sample in intra-frame mode. A certain indicator (e.g., intra_subpartitions_split_flag) can be inferred as a value indicating the vertical ISP segmentation. For example... Figure 11 As shown in (b), if predIntraMode belongs to set S2 (e.g., angular intra-frame prediction can refer to the left boundary reference sample), then intra_subpartitions_split_flag can be inferred to be equal to 1 indicating vertical ISP split and not encoded in the bitstream.

[0209] A certain indicator (e.g., intra_subpartitions_split_flag) can be determined (e.g., inferred) as a value indicating vertical or horizontal ISP splitting. If predIntraMode does not belong to set S1 or set S2, then predIntraMode may belong to S3, and intra_subpartitions_split_flag may be encoded as 0 or 1 in the bitstream.

[0210] In one example, for a first intra-prediction mode (e.g., predIntraMode <= 18), the prediction samples of the (e.g., the current) coded block may reference a left boundary reference sample. For a second intra-prediction mode (e.g., predIntraMode from 18 to 33), the prediction of a block (e.g., the current block prediction) may reference both a left boundary reference sample and a top boundary reference sample. The portion of the referenced top boundary reference sample may increase from slightly more than 0% of the block (e.g., for predIntraMode = 19) to slightly less than 50% (e.g., for predIntraMode = 33). For a third intra-prediction mode (e.g., predIntraMode >= 50), the prediction samples of the coded block (e.g., the current coded block) may include, for example, a reference to a top boundary reference sample. For a fourth intra-prediction mode (e.g., predIntraMode from 49 to 34), the prediction of a block (e.g., the current block prediction) may reference both a left boundary reference sample and a top boundary reference sample. The portion of the left boundary reference sample mentioned can be increased from slightly more than 0% of the block (e.g., for predIntraMode=49) to about 50% (e.g., for predIntraMode=34). A CU can be divided into multiple PUs (e.g., a PU may include a single subpartition or multiple partitions). Figure 11 (a) and Figure 11 (b) shows an example of a CU that has been divided into multiple sub-partitions. CUs can be processed based on a partitioning pattern (e.g., using multiple sub-partitions generated based on a partitioning pattern).

[0211] An angular intra-frame mode may reference the left boundary reference sample more than the top boundary reference sample, or vice versa. In the example, predIntraMode0 can be set to 33, and predIntraMode1 can be set to 34. Set S1 (or S2) can represent an angular intra-frame mode that references the left boundary reference sample more than the top boundary reference sample. Set S2 can represent an angular intra-frame mode that references the top boundary reference sample more than the left boundary reference sample.

[0212] A reference sample at a location (e.g., the boundary of a coding unit) may be located on or adjacent to that location (e.g., the current CU). A reference sample at the left boundary may be located on or adjacent to the left boundary of a CU (e.g., the current CU). A reference sample at the top boundary may be located on or adjacent to the top boundary of a CU (e.g., the current CU). "Adjacent" can include different types of neighbors, such as neighboring blocks, neighboring sub-blocks, neighboring pixels, and / or samples adjacent to a boundary. For example, neighboring samples can be samples that are spatially or temporally neighbors. A boundary reference sample is a reference sample adjacent to a boundary, where the boundary can be any type of boundary. For example, a boundary reference sample may be adjacent to the boundary of a block, sub-block, CU, and / or PU.

[0213] An angular intra-frame mode can reference the left boundary reference sample instead of the top boundary reference sample, or vice versa. In the example, predIntraMode0 can be set to 19, and predIntraMode1 can be set to 49. S1 can represent an angular intra-frame mode that references the left boundary reference sample entirely, but not either the top or bottom boundary reference sample. S2 can represent an angular intra-frame mode that references the top boundary reference sample entirely, but not either the left or bottom boundary reference sample.

[0214] In the example, predIntraMode0 can be set to a value between 19 and 33, and predIntraMode1 can be set to a value between 49 and 34.

[0215] For example, one or more examples as described in this article can be used to process wide-angle intra-frame modes.

[0216] The ISP segmentation mode derivation and / or constraints described herein can be applied to square coded blocks with equal width and height, such as square coded blocks with only equal width and height. In the example, the ISP segmentation mode derivation and / or constraints described herein may not be applicable to non-square coded blocks that may have wide-angle intra-prediction modes.

[0217] In the examples, the methods described herein can be applied to both square and non-square coded blocks. The original mode indicator predIntraMode can be examined and mapped to (e.g., as described herein) values ​​for non-square coded blocks. The mode indicator predIntraMode can be converted to cover both regular-angle and wide-angle intra-frame modes. The determination of the ISP segmentation mode as described herein can be based on the converted mode indicator predIntraMode that covers both regular-angle and wide-angle intra-frame modes.

[0218] The syntax of the encoding units can support the application of ISP segmentation mode restrictions, as shown in Table 7 or Table 8, for example. The examples in this paper can be used to resolve the encoding dependency of the indicator `inta_luma_not_planar_flag` on the indicator `intra_subpartitons_mode_flag`.

[0219] In one example, the signal notification and / or reception of the indicator `intra_subpartitons_split_flag` can be sent after some (e.g., all) MPM variables. As shown in Table 7, the choice of the context encoding bin for `intra_luma_not_planar_flag` can be the same as in Table 5 (e.g., depending on `intra_subpartitons_mode_flag`). The position of the indicator `intra_subpartitons_mode_flag` can be the same as in Table 5. The signal notification and / or reception of the indicator `intra_subpartitons_split_flag` can be sent after all MPM variables. This improves encoding efficiency. Table 7 provides an exemplary encoding unit syntax where the indicator `intra_subpartitons_split_flag` is located after the MPM variables.

[0220] Table 7

[0221]

[0222]

[0223] In one example, the dependency of the intra_luma_not_planar_flag context encoding bin on the ISP mode can be eliminated. ISP-related flag variables (e.g., both intra_subpartitions_mode_flag and intra_subpartitons_split_flag) can be signaled and / or received after MPM variables. As shown in Table 8, an exemplary encoding unit syntax is illustrated, indicating that intra_subpartitions_mode_flag and intra_subpartitons_split_flag can be placed after all MPM variables. This improves clarity.

[0224] Table 8

[0225]

[0226] It can unify the filtering of intra-frame reference samples.

[0227] As a result of one or more examples in this paper, for CU sizes of 1×N, 2×N, N×2, or other sizes with large width-to-height differences (e.g., 4×32, 4×64, etc.), sub-partition intra-prediction can reference the lines along the longer side of the sub-partition using reference samples. Sub-partition intra-prediction can also avoid referencing the lines along the shorter (e.g., very short) side of the sub-partition using reference samples. Referencing the lines along the longer side of the sub-partition using reference samples can lead to better prediction performance than using reference samples along the very short side of the sub-partition. In the examples, one or more restrictions on applying intra-reference sample smoothing filtering to the ISP mode can be skipped. Better coding efficiency can be achieved.

[0228] In some examples, intra-frame reference sample filtering (e.g., including smoothing and interpolation filtering) used for ISP and MRL may differ from that of the regular intra-frame mode. A unified approach can be used for ISP, MRL, and the regular intra-frame mode.

[0229] In one example, intra-reference sample filtering in regular intra-frame mode can be the same as intra-reference sample filtering in ISP intra-frame mode. For ISP, MRL, and regular intra-frame modes, reference sample smoothing filtering may not be used for integer bevel angles, and a 4-tap cubic filter for interpolation filtering can be used for fractional bevel angles. Table 9 provides an example of unified intra-reference sample filtering for ISP and MRL.

[0230] Table 9

[0231] ISP MRL Normal intra-frame mode Smoothing filter for integer bevel angles N / A N / A N / A Interpolation filtering for fractional slope 4-tap third-order filter 4-tap third-order filter 4-tap third-order filter

[0232] In one example, intra-reference sample filtering for ISP and MRL intra-mode can be performed in the same way as intra-reference sample filtering for regular intra-mode. For ISP, MRL, and regular intra-mode, a 3-tap reference sample smoothing filter can be applied to integer bevel angles, and a 4-tap cubic filter or a Gaussian filter can be applied to fractional bevel angles for interpolation filtering. Table 10 provides an example of uniform intra-reference sample filtering for regular intra-mode.

[0233] Table 10

[0234]

[0235] The sub-segmentation mode (ISP) can be determined (e.g., restricted) based on the intra-prediction mode. The ISP segmentation mode can depend on the CU intra-prediction mode, such that in a sub-segment derived from the ISP segmentation mode, the intra-prediction of a later-encoded (e.g., first) sub-segment can be referenced to the reconstructed sample of a previously-encoded (e.g., second) ISP sub-segment.

[0236] In the example, the ISP segmentation mode determination (e.g., derivation and / or restriction) as described herein may not be applicable to non-square coded blocks that may have wide-angle intra-prediction modes. In the example, the methods described herein may be applicable to both square and non-square coded blocks. The ISP segmentation derivation as described herein may be based on a conversion mode indicator predIntraMode that covers both regular-angle and wide-angle intra-prediction modes.

[0237] A unified approach can be used for ISP, MRL, and regular intra-frame modes. In one example, intra-reference sample filtering for regular intra-frame mode can be consistent with intra-reference sample filtering for ISP intra-frame mode. For ISP, MRL, and regular intra-frame modes, reference sample smoothing filtering may not be applied for integer bevel angles, and a 4-tap cubic filter for interpolation filtering can be applied for fractional bevel angles. In one example, intra-reference sample filtering for ISP and MRL intra-frame modes can be consistent with intra-reference sample filtering for regular intra-frame mode. For ISP, MRL, and regular intra-frame modes, 3-tap reference sample smoothing filtering can be applied for integer bevel angles, and a 4-tap cubic filter or a Gaussian filter can be applied for interpolation filtering of fractional bevel angles.

[0238] Figure 12 An example of method 1200 for decoding or encoding a CU using a segmentation mode based on an intra-prediction mode, according to one or more examples described herein, is shown. The examples and other examples disclosed herein are operable according to exemplary method 1200, for example, a device including one or more processors that can perform the method. At 1210, the intra-prediction mode of the coding unit (CU) is determined, for example, using an input variable “predIntraMode”. At 1220, a segmentation mode based on the intra-prediction mode is determined to generate multiple sub-segments in the CU (e.g., as shown in the original text). Figure 11 (a) and Figure 11 (b) shows). The prediction of the first sub-partition among multiple sub-partitions in the CU can be based on a reference sample in the second sub-partition among multiple sub-partitions in the CU, as described herein (e.g., in Figure 7 (in the middle). At 1230, the CU can be decoded or encoded based on the determined segmentation pattern (e.g., as shown in the middle). Figure 11 (a) and Figure 11 (b) shown). When Figure 12 When this method is applied to the decoder, steps 1210, 1220, and 1230 can be performed by the decoder. Step 1230 may require decoding of the CU based on the determined segmentation pattern. Figure 12 When the method is applied to the encoder, 1210, 1220, and 1230 can be performed by the encoder. 1230 may require encoding the CU based on the determined segmentation pattern.

[0239] This document describes numerous embodiments. Features of the embodiments may be provided individually or in any combination across various claim classes and types. Furthermore, embodiments may include one or more of the features, devices, or aspects described individually or in any combination across various claim classes and types (such as, for example, any of the following).

[0240] like Figure 12 The methods described herein may, for example, be executed by a decoder based on one or more examples as described herein. The decoder (e.g., Figure 3 The decoder 300 can be configured, for example, to use an intra-prediction mode index to determine the intra-prediction mode for the CU, as described herein. The decoder can be configured to determine a segmentation mode based on the intra-prediction mode to generate multiple sub-partitions in the CU, for example... Figure 11 (a) and Figure 11 As shown in (b). The prediction of the first sub-partition among multiple sub-partitions in the CU can be based on a reference sample in the second sub-partition among multiple sub-partitions in the CU, as described herein. The decoder can be configured to decode the CU based on the determined segmentation pattern, for example, as described herein.

[0241] The decoder can be configured to divide the CU into multiple sub-partitions based on a determined segmentation pattern and reconstruct a second sub-partition including the reference samples based on multiple samples in the reference CU, for example... Figure 11 (a) and Figure 11 As shown in (b). The decoder can be configured to determine, based on the intra-frame prediction mode, whether to decode the CU based on a first plurality of reference samples adjacent to the left boundary of the CU, a second plurality of reference samples adjacent to the top boundary of the CU, or at least one of the first plurality of reference samples and at least one of the second plurality of reference samples, as described herein.

[0242] The segmentation mode can be a horizontal segmentation mode. The segmentation mode can also be a vertical segmentation mode. When the intra-frame prediction mode indicates that the CU is decoded based on a first plurality of reference samples adjacent to the left boundary of the CU, the segmentation mode can be a vertical segmentation mode. When the intra-frame prediction mode indicates that the CU is decoded based on a second plurality of reference samples adjacent to the top boundary of the CU, the segmentation mode can be determined based on whether the first plurality of reference samples are more than the second plurality of reference samples.

[0243] Intra-subpartitions (ISP) can be enabled. The ISP splitting mode can be indicated in intra_subpartitions_split_flag.

[0244] Intra-prediction modes can be identified by an intra-prediction mode index and can be associated with a predetermined prediction direction. The intra-prediction mode used for the CU can be a vertical prediction mode, such as... Figure 8 As shown. The intra-frame prediction mode used for CU can be a horizontal prediction mode, such as... Figure 8 As shown. The intra-frame prediction mode used for the CU can be an angle prediction mode, such as... Figure 8 As shown. The intra-frame prediction mode used for the CU can be a DC prediction mode, such as... Figure 8 As shown. The intra-frame prediction mode used for the CU can be a planar prediction mode, such as... Figure 8 As shown. A CU can be a square CU with equal width and height, and multiple sub-partitions can be rectangular (e.g., as shown). Figure 11 (as shown in 11(a) and 11(b)). The CU can be a non-square CU, and the intra-frame prediction mode can be a wide-angle intra-frame prediction mode.

[0245] Decoding tools and techniques, including one or more of entropy decoding, inverse quantization, inverse transform, and differential decoding, can be used to implement [the following] in the decoder. Figure 12 The methods described herein. These decoding tools and techniques can be used to achieve one or more of the following: for example, using an intra-prediction mode index to determine the intra-prediction mode for the CU, as described herein; determining a segmentation mode based on the intra-prediction mode to generate multiple sub-partitions in the CU, for example, Figure 11 As shown in (a) and 11(b), the prediction of the first sub-partition in a plurality of sub-partitions of the CU can be based on a reference sample in the second sub-partition in a plurality of sub-partitions of the CU, as described herein; the CU is decoded based on a determined segmentation pattern, as described herein; the CU is divided into a plurality of sub-partitions based on the determined segmentation pattern and the second sub-partition including the reference sample is reconstructed based on a plurality of samples in a reference CU, as described herein. Figure 11 (a) and Figure 11As shown in (b); the intra-frame prediction mode determines whether the CU is decoded based on a first plurality of reference samples adjacent to the left boundary of the CU, a second plurality of reference samples adjacent to the top boundary of the CU, or at least one of the first plurality of reference samples and at least one of the second plurality of reference samples, as described in this paper; under the condition that the intra-frame prediction mode indicates decoding of the CU based on the first plurality of reference samples adjacent to the left boundary of the CU, a vertical segmentation mode is selected; under the condition that the intra-frame prediction mode indicates decoding of the CU based on the second plurality of reference samples adjacent to the top boundary of the CU, a horizontal segmentation mode is selected for the CU; under the condition that the intra-frame prediction mode indicates decoding based on the first plurality of reference samples adjacent to the left boundary of the CU, a horizontal segmentation mode is selected for the CU; under the condition that the intra-frame prediction mode indicates decoding ... first plurality of reference samples adjacent to the left boundary of the CU, a horizontal segmentation mode is selected for the CU; under the condition that the intra-frame prediction mode indicates decoding based on the first plurality of reference samples adjacent to the left boundary of the CU, a horizontal segmentation mode is selected for the CU. Under the condition that the CU is decoded based on a first plurality of reference samples adjacent to the left boundary of the CU and a second plurality of reference samples adjacent to the top boundary of the CU, the following actions are taken: selecting a segmentation mode based on whether the first plurality of reference samples are more than the second plurality of reference samples; determining whether the ISP is enabled or disabled for the CU; receiving or setting the intra_subpartitions_split_flag for the segmentation mode of the ISP; identifying the intra prediction mode based on the intra prediction mode index and determining the predetermined prediction direction associated with the intra prediction mode; selecting a wide-angle intra prediction mode for non-square CUs; ​​and other decoder behaviors related to any of the above.

[0246] like Figure 12 The methods described herein may, for example, be performed by an encoder based on one or more examples as described herein. The encoder (e.g., Figure 2 The encoder 200 can be configured, for example, to use an intra-prediction mode index to determine the intra-prediction mode for the CU, as described herein. The encoder can be configured to determine a segmentation mode based on the intra-prediction mode to generate multiple sub-partitions in the CU, for example... Figure 11 (a) and Figure 11 As shown in (b). The prediction of the first sub-partition among multiple sub-partitions in the CU can be based on a reference sample in the second sub-partition among multiple sub-partitions in the CU, as described herein. The encoder can be configured to encode the CU based on the determined segmentation pattern, for example, as described herein.

[0247] The encoder can be configured to divide the CU into multiple sub-partitions based on a determined segmentation pattern and reconstruct a second sub-partition including the reference samples based on multiple samples in the reference CU, for example... Figure 11 (a) and Figure 11As shown in (b). The encoder can be configured to determine, based on an intra-frame prediction mode, whether to encode the CU based on a first plurality of reference samples adjacent to the left boundary of the CU, a second plurality of reference samples adjacent to the top boundary of the CU, or at least one of the first plurality of reference samples and at least one of the second plurality of reference samples, as described herein. The encoder can be configured to select a vertical segmentation mode for the CU when the intra-frame prediction mode indicates that the CU should be encoded based on the first plurality of reference samples adjacent to the left boundary of the CU. The encoder can be configured to select a horizontal segmentation mode for the CU when the intra-frame prediction mode indicates that the CU should be encoded based on the second plurality of reference samples adjacent to the top boundary of the CU. The encoder can be configured to select a segmentation mode based on whether the first plurality of reference samples are more than the second plurality of reference samples when the intra-frame prediction mode indicates that the CU should be encoded based on both the first plurality of reference samples adjacent to the left boundary of the CU and the second plurality of reference samples adjacent to the top boundary of the CU.

[0248] Encoding tools and techniques, including one or more of quantization, entropy coding, inverse quantization, inverse transform, and differential coding, can be used to implement, in the encoder, such as Figure 11 The methods described herein. These encoding tools and techniques can be used to: for example, use an intra-prediction mode index to determine the intra-prediction mode for the CU, as described herein; determine a segmentation mode based on the intra-prediction mode to generate multiple sub-partitions in the CU, for example, Figure 11 As shown in (a) and 11(b), the prediction of the first sub-partition in a plurality of sub-partitions of the CU can be based on a reference sample in the second sub-partition in a plurality of sub-partitions of the CU, as described herein; the CU is encoded based on a determined segmentation pattern, for example as described herein; the CU is divided into a plurality of sub-partitions based on the determined segmentation pattern and the second sub-partition including the reference sample is reconstructed based on a plurality of samples in a reference CU, for example as... Figure 11 (a) and Figure 11As shown in (b); the intra-frame prediction mode determines whether to encode the CU based on a first plurality of reference samples adjacent to the left boundary of the CU, a second plurality of reference samples adjacent to the top boundary of the CU, or at least one of the first plurality of reference samples and at least one of the second plurality of reference samples, as described herein; a vertical segmentation mode is selected when the intra-frame prediction mode indicates encoding of the CU based on the first plurality of reference samples adjacent to the left boundary of the CU; a horizontal segmentation mode is selected for the CU when the intra-frame prediction mode indicates encoding of the CU based on the second plurality of reference samples adjacent to the top boundary of the CU; and a horizontal segmentation mode is selected for the CU when the intra-frame prediction mode indicates encoding based on the first plurality of reference samples adjacent to the left boundary of the CU. Given that the CU is encoded based on a first plurality of reference samples adjacent to the left boundary of the CU and a second plurality of reference samples adjacent to the top boundary of the CU, the following actions are taken: selecting a segmentation mode based on whether the first plurality of reference samples are more than the second plurality of reference samples; determining whether the ISP is enabled or disabled for the CU; receiving or setting the intra_subpartitions_split_flag for the segmentation mode of the ISP; identifying the intra-prediction mode based on the intra-prediction mode index and determining the predetermined prediction direction associated with the intra-prediction mode; selecting a wide-angle intra-prediction mode for non-square CUs; ​​and other encoder behaviors related to any of the above.

[0249] Syntax elements can be inserted into the signaling, for example, to enable the decoder to identify and execute such as... Figure 12 Indications associated with the method or method used. For example, syntax elements may include ISP indications, indications of intra-prediction modes, indications of parameters used by the decoder to perform one or more examples herein, for example, to indicate to the decoder whether one or more of them are enabled or disabled.

[0250] For example, the syntax elements applied at the decoder can be selected and / or applied, such as Figure 12 The method described herein. For example, the decoder may receive an instruction to enable ISP for the CU. Based on this instruction, the decoder may perform actions such as... Figure 12 The method described above is used to determine the segmentation pattern of the CU.

[0251] The encoder can adjust the prediction residual based on one or more examples presented in this paper. For example, the residual can be obtained by subtracting the predicted video patch from the original image patch. For example, the encoder can predict the video patch based on a determined segmentation pattern. The encoder can obtain the original image patch and subtract the predicted video patch from the original image patch to generate the prediction residual.

[0252] A bitstream or signal may include one or more syntax elements or variations thereof. For example, a bitstream or signal may include syntax elements that indicate whether any of the following—ISP, intra-prediction mode, parameters used by the decoder to perform one or more examples herein—is enabled, active, or disabled / inactive.

[0253] Bitstreams or signals may include syntax that conveys information generated according to one or more examples in this document. For example, in the execution of... Figure 12 The example shown generates information or data. The generated information or data can be conveyed in the syntax included in the bitstream or signal.

[0254] Syntax elements that enable the decoder to adapt the residuals to the signal in a manner corresponding to how they are used by the encoder can be inserted. For example, one or more examples in this paper can be used to generate residuals.

[0255] A method, process, apparatus, medium for storing instructions, medium for storing data, or signal may be used to create and / or transmit and / or receive and / or decode a bitstream or signal comprising one or more syntax elements or variations thereof.

[0256] A method, process, apparatus, medium for storing instructions, medium for storing data, or signal may be used to create and / or transmit and / or receive and / or decode according to any of the examples described.

[0257] TVs, set-top boxes, mobile phones, tablets, or other electronic devices may determine the segmentation mode of the CU based on the intra-frame prediction mode associated with the CU, according to any of the examples described.

[0258] TVs, set-top boxes, mobile phones, tablets, or other electronic devices may determine the segmentation mode of the CU based on the intra-frame prediction mode associated with the CU, according to any of the examples described, and display (e.g., using a monitor, screen, or other type of display) the resulting image.

[0259] TVs, set-top boxes, mobile phones, tablets, or other electronic devices may select (e.g., using a tuner) a channel to receive signals including coded images, and determine the segmentation mode of the CU based on the intra-frame prediction mode associated with the CU, according to any of the examples described.

[0260] TVs, set-top boxes, mobile phones, tablets, or other electronic devices may receive (e.g., using an antenna) an air signal comprising an encoded image and determine the segmentation mode of the CU based on an intra-frame prediction mode associated with the CU, according to any of the examples described.

[0261] Although features and elements have been described above in specific combinations, those skilled in the art will understand that each feature or element may be used alone or in any combination with other features and elements. Furthermore, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted over wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (such as internal hard disks and removable disks), magneto-optical media, and optical media (such as CD-ROM disks and digital versatile optical discs (DVDs)). A processor associated with the software may be used to implement a radio frequency transceiver for a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. An apparatus for video decoding, comprising one or more processors, wherein the one or more processors are configured to: Determine the intra-prediction mode for the coded block; The segmentation mode is inferred based on the intra-frame prediction mode to generate multiple sub-partitions in the coding block, wherein the prediction of a first sub-partition among the multiple sub-partitions in the coding block is based on a reference sample in a second sub-partition among the multiple sub-partitions in the coding block, and wherein the segmentation mode is inferred as a vertical segmentation mode based on the conditions indicated by the intra-frame prediction mode for decoding the coding block based on multiple reference samples adjacent to the left boundary of the coding block. as well as The encoded block is decoded based on the inferred segmentation pattern.

2. An apparatus for video encoding, comprising one or more processors, wherein the one or more processors are configured to: Determine the intra-prediction mode for the coded block; A segmentation mode is determined based on the intra-frame prediction mode to generate multiple sub-partitions in the coding block, wherein the prediction of a first sub-partition among the multiple sub-partitions in the coding block is based on a reference sample in a second sub-partition among the multiple sub-partitions in the coding block, and wherein a vertical segmentation mode is determined based on the conditions indicated by the intra-frame prediction mode for decoding the coding block based on multiple reference samples adjacent to the left boundary of the coding block. as well as The coded block is encoded based on the determined segmentation pattern.

3. The apparatus of claim 1, wherein the one or more processors are further configured to: Determine the intra-prediction mode for the second coding block; and Based on the intra-frame prediction mode indication, the condition for decoding the second coding block based on a second plurality of reference samples adjacent to the top boundary of the second coding block is inferred to determine the horizontal segmentation mode for the second coding block.

4. The apparatus of claim 1, wherein the one or more processors are further configured to: Determine the intra-prediction mode for the second coding block; and Based on the condition that the second coding block is decoded based on a first plurality of reference samples adjacent to the top boundary of the second coding block and a second plurality of reference samples adjacent to the left boundary of the second coding block, the segmentation mode for the second coding block is inferred based on the determination result of whether the first plurality of reference samples are more than the second plurality of reference samples.

5. The apparatus of claim 1 or claim 2, wherein the one or more processors are configured to: The coding block is divided into the plurality of sub-partitions based on the segmentation pattern; and The second sub-partition, including the reference sample, is reconstructed based on multiple samples in the reference coding block.

6. The apparatus of claim 2, wherein the one or more processors are configured to: Generate residuals based on the coded blocks; and The residual is included in the video data, but the indication of the determined segmentation pattern is not included in the video data.

7. The apparatus according to any one of claims 1 to 4 and 6, wherein intra-framing sub-partitioning (ISP) is enabled for the coded block.

8. A method for video decoding, the method comprising: Determine the intra-prediction mode for the coded block; The segmentation mode is inferred based on the intra-frame prediction mode to generate multiple sub-partitions in the coding block, wherein the prediction of a first sub-partition among the multiple sub-partitions in the coding block is based on a reference sample in a second sub-partition among the multiple sub-partitions in the coding block, and wherein the vertical segmentation mode is inferred based on the conditions indicated by the intra-frame prediction mode for decoding the coding block based on multiple reference samples adjacent to the left boundary of the coding block. as well as The encoded block is decoded based on the inferred segmentation pattern.

9. A method for video encoding, the method comprising: Determine the intra-prediction mode for the coded block; A segmentation mode is determined based on the intra-frame prediction mode to generate multiple sub-partitions in the coding block, wherein the prediction of a first sub-partition among the multiple sub-partitions in the coding block is based on a reference sample in a second sub-partition among the multiple sub-partitions in the coding block, and wherein a vertical segmentation mode is determined based on the conditions indicated by the intra-frame prediction mode for decoding the coding block based on multiple reference samples adjacent to the left boundary of the coding block. as well as The coded block is encoded based on the determined segmentation pattern.

10. The method of claim 8, further comprising: Determine the intra-prediction mode for the second coding block; as well as Based on the intra-frame prediction mode indication, the condition for decoding the second coding block based on a second plurality of reference samples adjacent to the top boundary of the second coding block is inferred to determine the horizontal segmentation mode for the second coding block.

11. The method of claim 8, wherein the method further comprises: Determine the intra-prediction mode for the second coding block; The intra-frame prediction mode determines whether the second coding block is decoded based on a first plurality of reference samples adjacent to the left boundary of the second coding block, a second plurality of reference samples adjacent to the top boundary of the second coding block, or at least one of the first plurality of reference samples and at least one of the second plurality of reference samples. The segmentation mode for the second coding block is inferred based on the determination result of decoding the second coding block based on the first plurality of reference samples adjacent to the left boundary of the second coding block, the second plurality of reference samples adjacent to the top boundary of the second coding block, or based on at least one of the first plurality of reference samples and at least one of the second plurality of reference samples.

12. The method of claim 9, wherein the method further comprises bypassing the instruction to send the determined segmentation pattern.

13. The method according to claim 8 or claim 9, wherein the method further comprises: The coding block is divided into the multiple sub-partitions based on the segmentation pattern; as well as The second sub-partition, including the reference sample, is reconstructed based on multiple samples in the reference coding block.

14. A non-transitory computer-readable medium comprising computer-executable instructions for causing one or more processors to perform the method according to any one of claims 8 to 13.

Images