Intra-frame prediction using block segmentation

By determining the partition type based on the width and height of video blocks and using specific intra-frame prediction modes and techniques to perform partition prediction on video blocks, the problem of low efficiency in intra-frame prediction for video blocks with unequal widths and heights in existing technologies is solved, achieving more efficient video decoding.

CN123125860APending Publication Date: 2026-11-17INTERDIGITAL CE PATENT HOLDINGS SAS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202580027760.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-04-15
Filing Date
2025-03-24
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing video decoding systems suffer from inefficiencies in intra-frame prediction mode selection and partitioning when processing video blocks of varying widths and heights.

Method used

By determining the partition type based on the width and height of the video block, and using horizontal or vertical intra-frame prediction modes to predict the partitions of the video block, the selection of intra-frame prediction modes and partition processing are optimized by combining technologies such as TIMD, DIMD, MRL, MIP, MPM or TMRL.

Benefits of technology

It improves the prediction efficiency and encoding/decoding performance of video blocks, especially for video blocks with varying widths and heights, thereby enhancing the quality and efficiency of video decoding.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123125860A_ABST
    Figure CN123125860A_ABST
Patent Text Reader

Abstract

The example device can: obtain blocks with unequal widths and heights; determine an intra-prediction mode applicable to the block; determine the partition type of the block based on the width and height of the block; partition the block into a first partition and a second partition based on the partition type, wherein the first partition of the block will be predicted using a horizontal or vertical intra-prediction mode; predict the first partition of the block using a horizontal or vertical intra-prediction mode, and predict the second partition of the block using the same intra-prediction mode (e.g., which is not a horizontal or vertical intra-prediction mode); and decode the block based on the first and second partitions of the block.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Cross-reference to related applications This application claims the benefit of European Patent Application No. 24315164.4, filed on April 15, 2024, the entire contents of which are incorporated herein by reference. Background Technology

[0002] Video decoding systems can be used to compress digital video signals, for example, to reduce the storage and / or transmission bandwidth required for such signals. Video decoding systems can include, for example, block-based systems, wavelet-based systems, and / or object-based systems. Summary of the Invention

[0003] This paper describes systems, devices, and methods for intra-frame prediction using block splitting.

[0004] In the example, a device used for video decoding (e.g., a video decoder) can obtain a video block. The width and height of the video block may not be equal. The device can determine an intra-prediction mode applicable to the video block. The device can determine the partition type of the video block based on its width and height (e.g., the partition type may be determined as a horizontal partition type based on the video block's height being greater than its width). The device can partition the video block into a first partition and a second partition based on the partition type (e.g., the first partition can be predicted using either a horizontal or vertical intra-prediction mode). The device can predict the first partition using either a horizontal or vertical intra-prediction mode and predict the second partition using an intra-prediction mode applicable to the video block. The intra-prediction mode applicable to the video block may not be a horizontal or vertical intra-prediction mode. The device can decode the video block based on the first and second partitions.

[0005] In the example, the device may receive a partitioning indicator that indicates a video block (e.g., a video block with unequal widths and heights) will be partitioned based on the width and height of the video block. The device may also receive a prediction scheme indicator that indicates whether a vertical intra-frame prediction mode or a horizontal intra-frame prediction mode will be used to predict the first partition of the video block. The first partition of the video block can be predicted according to the prediction scheme indicator. For example, if the prediction scheme indicator indicates that a vertical intra-frame prediction mode will be used to predict the first partition of the video block, the device can use the vertical intra-frame prediction mode to predict the first partition of the video block. If the prediction scheme indicator indicates that a horizontal intra-frame prediction mode will be used to predict the first partition of the video block, the device can use the horizontal intra-frame prediction mode to predict the first partition of the video block.

[0006] In the example, the device can determine whether a vertical intra-prediction mode or a horizontal intra-prediction mode will be used to predict the first partition of the video block based on the partition type of the video block. The first partition of the video block can be predicted based on the determination of whether a vertical or horizontal intra-prediction mode will be used to predict the first partition of the video block. For example, if the partition type of the video block is vertical, the device can use the vertical intra-prediction mode to predict the first partition of the video block. If the partition type of the video block is horizontal, the device can use the horizontal intra-prediction mode to predict the first partition of the video block.

[0007] The device can determine whether to enable one of the following for a video block: Template-based Intra-Prediction (TIMD), Decoder-Side Intra-Mode Derivation (DIMD), Multiple Reference Lines (MRL), Matrix-based Intra-Prediction (MIP), Most Probable Mode (MPM), or Template-based Multiple Reference Lines (TMRL). The intra-prediction mode applicable to the video block can be determined based on whether one of TIMD, DIMD, MRL, MIP, MPM, or TMRL is enabled. For example, the device can determine whether TIMD is enabled for a video block and determine the intra-prediction mode for the video block based on TIMD. The device can use a vertical / horizontal intra-prediction mode to predict the first partition of the video block and use the intra-prediction mode determined using TIMD to predict the second partition of the video block.

[0008] In some examples, only the vertical intra-frame prediction mode may be allowed for the first partition of a video block, and the first partition of a video block may be predicted using the vertical intra-frame prediction mode.

[0009] In the example, the device can determine the prediction of a video block based on the predictions of the first and second partitions of the video block. For example, the prediction of the video block could be a simple average or a weighted average of the predictions of the first and second partitions. The device can decode the residual coefficients associated with the video block. The device can apply the decoded residual coefficients to the prediction of the video block. The video block can be symmetrically partitioned into a first and second partition. The video block can be asymmetrically partitioned into a first and second partition.

[0010] In the example, a device for video encoding (e.g., a video encoder) can obtain a video block. The width and height of the video block may not be equal. The device can determine an intra-prediction mode applicable to the video block. The device can determine the partition type of the video block based on the width and height of the video block (e.g., the partition type of the video block can be determined as a horizontal partition type based on the height of the video block being greater than its width). The device can partition the video block into a first partition and a second partition based on the partition type (e.g., the first partition of the video block can be predicted using a horizontal or vertical intra-prediction mode). The device can predict the first partition of the video block using a horizontal or vertical intra-prediction mode and predict the second partition of the video block using an intra-prediction mode applicable to the video block. The intra-prediction mode applicable to the video block may not be a horizontal or vertical intra-prediction mode. The device can encode the video block based on the first and second partitions of the video block.

[0011] In the example, the device can send a partitioning indicator that indicates a video block (e.g., a video block with unequal widths and heights) will be partitioned based on the width and height of the video block. The device can also send a prediction scheme indicator that indicates whether a vertical intra-frame prediction mode or a horizontal intra-frame prediction mode will be used to predict the first partition of the video block. The first partition of the video block can be predicted according to the prediction scheme indicator. For example, if the prediction scheme indicator indicates that a vertical intra-frame prediction mode will be used to predict the first partition of the video block, the device can use the vertical intra-frame prediction mode to predict the first partition of the video block. If the prediction scheme indicator indicates that a horizontal intra-frame prediction mode will be used to predict the first partition of the video block, the device can use the horizontal intra-frame prediction mode to predict the first partition of the video block.

[0012] In the example, the device can determine whether a vertical intra-prediction mode or a horizontal intra-prediction mode will be used to predict the first partition of the video block based on the partition type of the video block. The first partition of the video block can be predicted based on the determination of whether a vertical or horizontal intra-prediction mode will be used to predict the first partition of the video block. For example, if the partition type of the video block is vertical, the device can use the vertical intra-prediction mode to predict the first partition of the video block. If the partition type of the video block is horizontal, the device can use the horizontal intra-prediction mode to predict the first partition of the video block.

[0013] The device can determine whether to enable one of the following for a video block: TIMD, DIMD, MRL, MIP, MPM, or TMRL. The intra-prediction mode applicable to the video block can be determined based on the enabled option of TIMD, DIMD, MRL, MIP, MPM, or TMRL. For example, the device can determine whether to enable TIMD for a video block and determine the intra-prediction mode for the video block based on TIMD. The device can use a vertical / horizontal intra-prediction mode to predict the first partition of the video block and use the intra-prediction mode determined using TIMD to predict the second partition of the video block.

[0014] In some examples, only the vertical intra-frame prediction mode may be allowed for the first partition of a video block, and the first partition of a video block may be predicted using the vertical intra-frame prediction mode.

[0015] In the example, the device can determine the prediction of a video block based on the predictions of the first partition and the second partition. For example, the prediction of a video block can be a simple average or a weighted average of the predictions of the first and second partitions. A video block can be symmetrically partitioned into first and second partitions. A video block can be asymmetrically partitioned into first and second partitions.

[0016] In the example, predictions made for the first partition of a video block using either a horizontal or vertical intra-prediction mode, and predictions made for the second partition of a video block using an intra-prediction mode applicable to the video block, can be evaluated for predictions of an unpartitioned video block. The device can determine that the first prediction will be used to encode the video block. The device can send a partitioning indicator that indicates that the video block (e.g., a video block with unequal widths and heights) will be partitioned based on the width and height of the video block. Attached Figure Description

[0017] Figure 1A This is a system diagram illustrating an example communication system in which one or more of the disclosed embodiments may be implemented.

[0018] Figure 1B The illustration shows a method according to one embodiment. Figure 1A The diagram shows a system diagram of an example wireless transmit / receive unit (WTRU) used in a communication system.

[0019] Figure 1C The illustration shows a method according to one embodiment. Figure 1A The diagram illustrates a system diagram of an example radio access network (RAN) and an example core network (CN) used in the communication system.

[0020] Figure 1D The illustration shows a method according to one embodiment. Figure 1A The illustrated system diagram shows yet another example RAN and yet another example CN used in the communication system.

[0021] Figure 2 This is a diagram illustrating an example block-based video encoder.

[0022] Figure 3 This is a diagram showing an example video decoder.

[0023] Figure 4 This is a diagram illustrating examples of systems that can achieve various aspects and examples.

[0024] Figure 5 The illustration shows an example of a WxH block partition, such as H = 4 and W = 8.

[0025] Figure 6 The illustration shows an example of a WxH block partition, such as H = 16 and W = 32.

[0026] Figure 7 The illustration shows an example block partitioning in the Spatial Geometric Partitioning Pattern (SGPM).

[0027] Figure 8 An example template is illustrated (e.g., a template used in SGPM to derive the SGPM candidate list).

[0028] Figure 9 The illustration shows an example of block segmentation when the width of the block is greater than the height of the block.

[0029] Figure 10 The illustration shows an example of block segmentation when the block height is greater than the block width.

[0030] Figure 11 The illustration shows an example of asymmetric block segmentation when the width of the block is greater than the height of the block.

[0031] Figure 12 The illustration shows an example of asymmetric block segmentation when the height of a block is greater than the width of the block. Detailed Implementation

[0032] A detailed description of illustrative embodiments will now be described with reference to the accompanying drawings. While this description provides detailed examples of possible implementations, it should be noted that these details are intended to be exemplary and in no way limit the scope of this application.

[0033] Figure 1AThis diagram illustrates an example communication system 100 in which one or more of the disclosed embodiments may be implemented. The communication system 100 may be a multi-access system that provides content such as voice, data, video, messaging, and broadcasting to multiple wireless users. The communication system 100 enables multiple wireless users to access such content by sharing system resources, including wireless bandwidth. For example, the communication system 100 may employ one or more channel access methods, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal FDMA (OFDMA), Single Carrier FDMA (SC-FDMA), Zero-Tail Unique Word DFT-Spread Spectrum OFDM (ZT UW DTS-s OFDM), Unique Word OFDM (UW-OFDM), Resource Block Filtered OFDM, Filter Bank Multicarrier (FBMC), and so on.

[0034] like Figure 1A As shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, Public Switched Telephone Network (PSTN) 108, Internet 110, and other networks 112. However, it should be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, and 102d may be any type of device configured to operate and / or communicate in a wireless environment. For example, WTRUs 102a, 102b, 102c, and 102d (any of which may be referred to as a “station” and / or “STA”) may be configured to transmit and / or receive wireless signals and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscription-based units, pagers, cellular phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated processing chain environments), consumer electronics devices, devices operating on commercial and / or industrial wireless networks, etc. Any of WTRUs 102a, 102b, 102c, and 102d may be interchangeably referred to as a UE.

[0035] The communication system 100 may also include base station 114a and / or base station 114b. Each of base stations 114a and 114b may be any type of device configured to wirelessly interface with at least one of WTRUs 102a, 102b, 102c, and 102d to facilitate access to one or more communication networks, such as CN 106 / 115, the Internet 110, and / or other networks 112. For example, base stations 114a and 114b may be base transceiver stations (BTS), Node-B, eNode B, home node B, home eNode B, gNB, NR NodeB, site controller, access point (AP), wireless router, etc. Although base stations 114a and 114b are each depicted as a single element, it should be understood that base stations 114a and 114b may include any number of interconnected base stations and / or network elements.

[0036] Base station 114a may be part of RAN 104 / 113, which may also include other base stations and / or network elements (not shown), such as base station controllers (BSCs), radio network controllers (RNCs), relay nodes, etc. Base station 114a and / or base station 114b may be configured to transmit and / or receive radio signals on one or more carrier frequencies, which may be referred to as cells (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage of a specific geographic area for a radio service, which may be relatively fixed or may change over time. A cell may also be partitioned into cell sectors. For example, the cell associated with base station 114a may be partitioned into three sectors. Thus, in one embodiment, base station 114a may include three transceivers, i.e., one for each sector of the cell. In one embodiment, base station 114a may employ multiple-input multiple-output (MIMO) technology and may use multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in a desired spatial direction.

[0037] Base stations 114a and 114b can communicate with one or more of WTRUs 102a, 102b, 102c, and 102d via air interface 116. Air interface 116 can be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). Any suitable radio access technology (RAT) can be used to establish air interface 116.

[0038] More specifically, as described above, the communication system 100 can be a multi-access system and can employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, base stations 114a and WTRUs 102a, 102b, and 102c in RAN 104 / 113 can implement radio technologies such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which can establish air interfaces 115 / 116 / 117 using Wideband CDMA (WCDMA). WCDMA can include communication protocols such as High-Speed ​​Packet Access (HSPA) and / or evolved HSPA (HSPA+). HSPA can include High-Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High-Speed ​​UL Packet Access (HSUPA).

[0039] In one embodiment, base station 114a and WTRUs 102a, 102b, 102c may implement radio technologies such as evolved UMTS terrestrial radio access (E-UTRA), which may use Long Term Evolution (LTE) and / or Advanced LTE (LTE-A) and / or Advanced LTE Pro (LTE-A Pro) to establish air interface 116.

[0040] In one embodiment, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as NR radio access, which can establish an air interface 116 using a new radio (NR).

[0041] In one embodiment, base station 114a and WTRUs 102a, 102b, and 102c can implement multiple radio access technologies. For example, base station 114a and WTRUs 102a, 102b, and 102c can jointly implement LTE radio access and NR radio access, for example, using the dual connectivity (DC) principle. Therefore, the air interface used by WTRUs 102a, 102b, and 102c can be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations (e.g., eNBs and gNBs).

[0042] In other embodiments, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as IEEE 802.11 (i.e., Wi-Fi), IEEE 802.16 (i.e., WiMAX), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Provisional Standard 2000 (IS-2000), Provisional Standard 95 (IS-95), Provisional Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data Rate GSM Evolution (EDGE), GSMEDGE (GERAN), etc.

[0043] For example, Figure 1A Base station 114b can be a wireless router, home node B, home eNode B, or access point, and can utilize any suitable RAT to facilitate wireless connectivity in a local area, such as commercial locations, homes, vehicles, campuses, industrial facilities, air corridors (e.g., for drone use), roads, etc. In one embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.11 to establish a wireless local area network (WLAN). In one embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, base station 114b and WTRUs 102c, 102d can utilize cellular-based RATs (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish picocells or femtocells. Figure 1A As shown, base station 114b can be directly connected to the Internet 110. Therefore, base station 114b may not need to access the Internet 110 via CN 106 / 115.

[0044] RAN 104 / 113 can communicate with CN 106 / 115, which can be any type of network configured to provide voice, data, application, and / or Voice over Internet Protocol (VoIP) services to one or more of WTRUs 102a, 102b, 102c, and 102d. Data can have different Quality of Service (QoS) requirements, such as different throughput requirements, latency requirements, fault tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. CN 106 / 115 can provide call control, billing services, location-based services, prepaid calling, internet connectivity, video distribution, and / or perform advanced security functions such as user authentication. Although in Figure 1AAlthough not shown, it should be understood that RAN104 / 113 and / or CN 106 / 115 can communicate directly or indirectly with other RANs that use the same RAT as or a different RAT than RAN 104 / 113. For example, in addition to being connected to RAN 104 / 113, which may utilize NR radio technology, CN106 / 115 can also communicate with another RAN (not shown) that uses GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technology.

[0045] CN 106 / 115 can also serve as a gateway for WTRU 102a, 102b, 102c, 102d to access PSTN 108, the Internet 110, and / or other networks 112. PSTN 108 may include a circuit-switched telephone network providing Common Old-Style Telephone Service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices using common communication protocols such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and / or Internet Protocol (IP) from the TCP / IP Internet Protocol suite. Network 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, network 112 may include another CN connected to one or more RANs, which may use the same RAT as RAN 104 / 113 or a different RAT.

[0046] Some or all of the WTRUs 102a, 102b, 102c, and 102d in the communication system 100 may include multi-mode capabilities (e.g., WTRUs 102a, 102b, 102c, and 102d may include multiple transceivers for communicating with different wireless networks via different wireless links). For example... Figure 1A The WTRU 102c shown can be configured to communicate with base station 114a, which may employ cellular-based radio technology, and to communicate with base station 114b, which may employ IEEE 802 radio technology.

[0047] Figure 1B This is a system diagram illustrating example WTRU 102. (Example:) Figure 1B As shown, among other things, WTRU 102 may include, in particular, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power supply 134, a global positioning system (GPS) chipset 136, and / or other peripheral devices 138, etc. It should be understood that WTRU 102 may include any sub-combination of the foregoing elements while remaining consistent with the embodiments.

[0048] Processor 118 may be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. Processor 118 may perform signal encoding, data processing, power control, input / output processing, and / or any other functions that enable WTRU 102 to operate in a wireless environment. Processor 118 may be coupled to transceiver 120, which may be coupled to transmitting / receiving element 122. Although Figure 1B The processor 118 and transceiver 120 are depicted as separate components, but it should be understood that the processor 118 and transceiver 120 may be integrated together in an electronic package or chip.

[0049] Transmitting / receiving element 122 can be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) over air interface 116. For example, in one embodiment, transmitting / receiving element 122 can be an antenna configured to transmit and / or receive RF signals. In one embodiment, transmitting / receiving element 122 can be, for example, a transmitter / detector configured to transmit and / or receive IR, UV, or visible light signals. In yet another embodiment, transmitting / receiving element 122 can be configured to transmit and / or receive both RF and optical signals. It should be understood that transmitting / receiving element 122 can be configured to transmit and / or receive any combination of wireless signals.

[0050] Although the transmitting / receiving element 122 is in Figure 1B While depicted as a single element, WTRU 102 may include any number of transmit / receive elements 122. More specifically, WTRU 102 may employ MIMO technology. Thus, in one embodiment, WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals on air interface 116.

[0051] Transceiver 120 can be configured to modulate signals transmitted by transmitting / receiving element 122 and demodulate signals received by transmitting / receiving element 122. As described above, WTRU 102 can have multi-mode capability. Therefore, for example, transceiver 120 may include multiple transceivers to enable WTRU 102 to communicate via multiple RATs, such as NR and IEEE 802.11.

[0052] The processor 118 of WTRU 102 can be coupled to a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) unit or an organic light-emitting diode (OLED) display unit) and can receive user input data therefrom. The processor 118 can also output user data to the speaker / microphone 124, keypad 126, and / or display / touchpad 128. Furthermore, the processor 118 can access and store information from any type of suitable memory, such as non-removable memory 130 and / or removable memory 132. Non-removable memory 130 may include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. Removable memory 132 may include a user identification module (SIM) card, memory stick, secure digital storage (SD) card, etc. In other embodiments, the processor 118 can access and store information from memory that is not physically located on WTRU 102 (e.g., a server or home computer (not shown)).

[0053] The processor 118 can receive power from the power supply 134 and can be configured to distribute and / or control power to other components in the WTRU 102. The power supply 134 can be any suitable device that powers the WTRU 102. For example, the power supply 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0054] The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) about the current location of the WTRU 102. In addition to, or instead of, information from the GPS chipset 136, the WTRU 102 may receive location information on the air interface 116 from base stations (e.g., base stations 114a, 114b) and / or determine its location based on the timing of signals received from two or more nearby base stations. It should be understood that the WTRU 102 may acquire location information using any suitable location determination method while remaining consistent with the embodiments.

[0055] The processor 118 may be further coupled to other peripheral devices 138, which may include one or more software and / or hardware modules providing additional features, functions, and / or wired or wireless connectivity. For example, peripheral devices 138 may include accelerometers, electronic compasses, satellite transceivers, digital cameras (for photos and / or videos), Universal Serial Bus (USB) ports, vibration devices, television transceivers, hands-free headsets, Bluetooth® modules, FM radio units, digital music players, media players, video game player modules, internet browsers, virtual reality and / or augmented reality (VR / AR) devices, activity trackers, etc. Peripheral devices 138 may include one or more sensors, such as gyroscopes, accelerometers, Hall effect sensors, magnetometers, orientation sensors, proximity sensors, temperature sensors, time sensors; geolocation sensors; altimeters, light sensors, touch sensors, magnetometers, barometers, attitude sensors, biosensors, and / or humidity sensors.

[0056] WTRU 102 may include a full-duplex radio for which the transmission and reception of some or all signals (e.g., signals associated with specific subframes for both UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous. The full-duplex radio may include an interference management unit to reduce and / or substantially eliminate self-interference via hardware (e.g., chokes) or via signal processing by a processor (e.g., a separate processor (not shown) or via processor 118). In one embodiment, WTRU 102 may include a half-duplex radio for which the transmission and reception of some or all signals (e.g., signals associated with specific subframes for UL (e.g., for transmission) or downlink (e.g., for reception)) may be concurrent and / or simultaneous.

[0057] Figure 1C This diagram illustrates a system diagram of RAN 104 and CN 106 according to an embodiment. As described above, RAN 104 can communicate with WTRUs 102a, 102b, and 102c via air interface 116 using E-UTRA radio technology. RAN 104 can also communicate with CN 106.

[0058] RAN 104 may include eNode-Bs 160a, 160b, and 160c; however, it should be understood that RAN 104 may include any number of eNode-Bs while remaining consistent with the embodiments. eNode-Bs 160a, 160b, and 160c may each include one or more transceivers for communicating with WTRUs 102a, 102b, and 102c on air interface 116. In one embodiment, eNode-Bs 160a, 160b, and 160c may implement MIMO technology. Therefore, for example, eNode-B 160a may use multiple antennas to transmit and / or receive radio signals from WTRU 102a.

[0059] Each of the eNode-B 160a, 160b, and 160c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, etc. Figure 1C As shown, eNode-B 160a, 160b, and 160c can communicate with each other on the X2 interface.

[0060] Figure 1C The CN 106 shown may include a Mobility Management Entity (MME) 162, a Serving Gateway (SGW) 164, and a Packet Data Network (PDN) Gateway (or PGW) 166. While each of the foregoing elements is described as part of CN 106, it should be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0061] The MME 162 can connect to each of the eNode-B 160a, 160b, and 160c in RAN104 via the S1 interface and can act as a control node. For example, the MME 162 can be responsible for authenticating users of WTRUs 102a, 102b, and 102c, bearer activation / deactivation, selecting a specific serving gateway during the initial attachment of WTRUs 102a, 102b, and 102c, etc. The MME 162 can provide control plane functions for handover between RAN104 and other RANs (not shown) employing other radio technologies such as GSM and / or WCDMA.

[0062] The SGW 164 can connect to each of the eNode Bs 160a, 160b, and 160c in RAN104 via the S1 interface. The SGW 164 can typically route and forward user data packets to / from WTRUs 102a, 102b, and 102c. The SGW 164 can perform other functions, such as anchoring the user plane during inter-eNode B handover, triggering paging when DL data is available for WTRUs 102a, 102b, and 102c, and managing and storing the context of WTRUs 102a, 102b, and 102c.

[0063] SGW 164 can connect to PGW 166, which can provide WTRU 102a, 102b, 102c with access to packet-switched networks such as Internet 110, so as to facilitate communication between WTRU 102a, 102b, 102c and IP-enabled devices.

[0064] CN 106 can facilitate communication with other networks. For example, CN 106 can provide WTRU 102a, 102b, and 102c with access to a circuit-switched network such as PSTN 108, facilitating communication between WTRU 102a, 102b, and 102c and traditional landline communication equipment. For example, CN 106 may include, or be able to communicate with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that acts as an interface between CN 106 and PSTN 108. Furthermore, CN 106 can provide WTRU 102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0065] Despite WTRU in Figure 1A-1D While described as a wireless terminal, it is conceivable that, in some representative embodiments, such a terminal may use (e.g., temporarily or permanently) a wired communication interface with a communication network.

[0066] In a representative embodiment, another network 112 may be a WLAN.

[0067] A WLAN in Infrastructure Basic Services Set (BSS) mode can have an access point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP can access or interface with a distributed system (DS) or another type of wired / wireless network that transmits traffic to and / or out of the BSS. Traffic originating outside the BSS destined for a STA can reach and be delivered to the STA via the AP. Traffic originating from a STA destined for an external BSS can be sent to the AP for delivery to the appropriate destination. For example, traffic between STAs within the BSS can be transmitted via the AP, where the source STA can send traffic to the AP, and the AP can deliver traffic to the destination STA. Traffic between STAs within the BSS can be considered and / or referred to as peer-to-peer traffic. Peer-to-peer traffic can be transmitted between source and destination STAs (e.g., directly between them) using Direct Link Establishment (DLS). In some representative embodiments, the DLS can use 802.11e DLS or 802.11z Tunneled DLS (TDLS). A WLAN using the Standalone BSS (IBSS) mode may not have an access point (AP), and STAs within the IBSS or using the IBSS (e.g., all STAs) can communicate directly with each other. The IBSS communication mode is sometimes referred to here as a "self-organizing" communication mode.

[0068] When using 802.11ac infrastructure operating mode or a similar operating mode, the AP can transmit beacons on a fixed channel, such as the primary channel. The primary channel can be of a fixed width (e.g., a wide bandwidth of 20 MHz) or dynamically set via signaling. The primary channel can be the operating channel of the BSS and can be used by the STA to establish a connection with the AP. In some representative embodiments, such as in an 802.11 system, Carrier Sense Multiple Access (CSMA / CA) with collision avoidance can be implemented. For CSMA / CA, each STA, including the AP, can sense the primary channel. If a particular STA senses / detects and / or determines that the primary channel is busy, that particular STA can back off. A single STA (e.g., only one station) can transmit at any given time within a given BSS.

[0069] High-throughput (HT) STAs can communicate using a 40 MHz wide channel, for example, by combining a primary 20 MHz channel with adjacent or non-adjacent 20 MHz channels.

[0070] Very High Throughput (VHT) STAs can support channels with widths of 20 MHz, 40 MHz, 80 MHz, and / or 160 MHz. 40 MHz and / or 80 MHz channels can be formed by combining consecutive 20 MHz channels. A 160 MHz channel can be formed by combining eight consecutive 20 MHz channels, or by combining two non-consecutive 80 MHz channels, which can be referred to as an 80+80 configuration. For the 80+80 configuration, after channel coding, the data passes through a segment resolver, which splits the data into two streams. Each stream can be processed separately using Inverse Fast Fourier Transform (IFFT) and time-domain processing. These streams can be mapped onto two 80 MHz channels, and the data can be transmitted by the transmitting STA. At the receiver of the receiving STA, the operation of the 80+80 configuration can be reversed, and the combined data can be sent to the Media Access Control (MAC).

[0071] 802.11af and 802.11ah support operating modes below 1 GHz. The channel operating bandwidth and carrier in 802.11af and 802.11ah are reduced compared to those used in 802.11n and 802.11ac. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV whitespace (TVWS) spectrum, while 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11ah can support metering-type control / machine-type communications, such as MTC devices in macro coverage areas. MTC devices may have certain capabilities, such as limited capabilities, including support for (e.g., only) certain and / or limited bandwidths. MTC devices may include batteries with a battery life exceeding a threshold (e.g., to maintain a very long battery life).

[0072] WLAN systems that can support multiple channels and channel bandwidths, such as 802.11n, 802.11ac, 802.11af, and 802.11ah, include channels that can be designated as the primary channel. The bandwidth of the primary channel can be equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel can be set and / or limited by the STA among all STAs operating in the BSS that supports the minimum bandwidth operating mode. In the example of 802.11ah, for STAs that support (e.g., only support) the 1 MHz mode (e.g., MTC type devices), the primary channel can be 1 MHz wide, even if the AP and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and / or other channel bandwidth operating modes. Carrier Sense and / or Network Allocation Vector (NAV) settings may depend on the status of the primary channel. If the primary channel is busy, for example, because an STA (which only supports the 1 MHz operating mode) is transmitting to the AP, the entire available band can be considered busy, even if most of the band remains idle and can be available.

[0073] In the United States, the available frequency band for 802.11ah is from 902 MHz to 928 MHz. In South Korea, the available frequency band is from 917.5 MHz to 923.5 MHz. In Japan, the available frequency band is from 916.5 MHz to 927.5 MHz. The total available bandwidth for 802.11ah is 6 MHz to 26 MHz, depending on the country code.

[0074] Figure 1D This diagram illustrates a system diagram of RAN 113 and CN 115 according to one embodiment. As described above, RAN 113 can communicate with WTRUs 102a, 102b, and 102c via air interface 116 using NR radio technology. RAN 113 can also communicate with CN 115.

[0075] RAN 113 may include gNBs 180a, 180b, and 180c; however, it should be understood that RAN 113 may include any number of gNBs while remaining consistent with the embodiments. gNBs 180a, 180b, and 180c may each include one or more transceivers for communicating with WTRUs 102a, 102b, and 102c on air interface 116. In one embodiment, gNBs 180a, 180b, and 180c may implement MIMO technology. For example, gNBs 180a and 180b may utilize beamforming to transmit signals to and / or receive signals from gNBs 180a, 180b, and 180c. Therefore, for example, gNB 180a may use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 102a. In one embodiment, gNBs 180a, 180b, and 180c can implement carrier aggregation technology. For example, gNB 180a can transmit multiple component carriers (not shown) to WTRU 102a. A subset of these component carriers can be on unlicensed spectrum, while the remaining component carriers can be on licensed spectrum. In one embodiment, gNBs 180a, 180b, and 180c can implement Coordinated Multipoint (CoMP) technology. For example, WTRU 102a can receive coordinated transmissions from gNBs 180a and 180b (and / or gNB 180c).

[0076] WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using transmissions associated with scalable numberology. For example, the OFDM symbol spacing and / or OFDM subcarrier spacing can differ for different transmissions, different cells, and / or different portions of the radio transmission spectrum. WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using subframes or transmission time intervals (TTIs) of various or scalable lengths (e.g., including a variable number of OFDM symbols and / or a continuously variable absolute time).

[0077] gNBs 180a, 180b, and 180c can be configured to communicate with WTRUs 102a, 102b, and 102c in standalone and / or non-standalone configurations. In standalone configuration, WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c without accessing other RANs (e.g., eNode-Bs 160a, 160b, and 160c). In standalone configuration, WTRUs 102a, 102b, and 102c can utilize one or more of gNBs 180a, 180b, and 180c as mobility anchors. In standalone configuration, WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using signals in unlicensed frequency bands. In a non-standalone configuration, WTRUs 102a, 102b, and 102c can communicate / connect with gNBs 180a, 180b, and 180c, while also communicating / connecting with another RAN such as eNode-Bs 160a, 160b, and 160c. For example, WTRUs 102a, 102b, and 102c can implement DC principles to communicate substantially simultaneously with one or more gNBs 180a, 180b, and 180c, as well as one or more eNode-Bs 160a, 160b, and 160c. In a non-standalone configuration, eNode-Bs 160a, 160b, and 160c can act as mobility anchors for WTRUs 102a, 102b, and 102c, and gNBs 180a, 180b, and 180c can provide additional coverage and / or throughput for serving WTRUs 102a, 102b, and 102c.

[0078] Each of gNBs 180a, 180b, and 180c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, network slicing support, dual connectivity, interoperability between NR and E-UTRA, routing user plane data to User Plane Functions (UPF) 184a and 184b, and routing control plane information to Access and Mobility Management Functions (AMF) 182a and 182b, etc. Figure 1D As shown, gNB 180a, 180b, and 180c can communicate with each other on the Xn interface.

[0079] Figure 1DThe CN 115 shown may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one Session Management Function (SMF) 183a, 183b, and possibly a Data Network (DN) 185a, 185b. While each of the foregoing elements is depicted as part of the CN 115, it should be understood that any of these elements may be owned and / or operated by an entity other than a CN operator.

[0080] AMF 182a and 182b can connect to one or more of gNBs 180a, 180b, and 180c in RAN 113 via the N2 interface and can act as control nodes. For example, AMF 182a and 182b can be responsible for authenticating users of WTRU 102a, 102b, and 102c, supporting network slicing (e.g., handling different PDU sessions with different requirements), selecting specific SMF183a and 183b, managing registration areas, terminating NAS signaling, mobility management, and so on. AMF 182a and 182b can use network slicing to customize CN support for WTRU 102a, 102b, and 102c based on the service types used by WTRU 102a, 102b, and 102c. For example, different network slices can be created for different use cases, such as services relying on Ultra Reliable Low Latency Communication (URLLC) access, services relying on Enhanced Massive Mobile Broadband (eMBB) access, services for Machine Type Communication (MTC) access, and so on. AMF 182 can provide control plane functions for handover between RAN 113 and other RANs (not shown) employing other radio technologies such as LTE, LTE-A, LTE-A Pro and / or non-3GPP access technologies such as WiFi.

[0081] SMFs 183a and 183b can connect to AMFs 182a and 182b in CN 115 via the N11 interface. SMFs 183a and 183b can also connect to UPFs 184a and 184b in CN 115 via the N4 interface. SMFs 183a and 183b can select and control UPFs 184a and 184b, and configure the routing of services through UPFs 184a and 184b. SMFs 183a and 183b can perform other functions, such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, providing downlink data notifications, etc. PDU session types can be IP-based, non-IP-based, Ethernet-based, etc.

[0082] UPF 184a and 184b can be connected via the N3 interface to one or more of the gNBs 180a, 180b, and 180c in RAN 113. This N3 interface provides WTRU 102a, 102b, and 102c with access to packet-switched networks (such as Internet 110) to facilitate communication between WTRU 102a, 102b, 102c and IP-enabled devices. UPF 184 and 184b can perform other functions such as routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering downlink packets, providing mobility anchoring, and so on.

[0083] CN 115 can facilitate communication with other networks. For example, CN 115 may include, or be able to communicate with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that acts as an interface between CN 115 and PSTN 108. Furthermore, CN 115 can provide WTRUs 102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, WTRUs 102a, 102b, and 102c may be connected to local data networks (DNs) 185a and 185b via the N3 interface to UPFs 184a and 184b and the N6 interface between UPFs 184a and 184b and DNs 185a and 185b.

[0084] Given Figure 1A-1D as well as Figure 1A-1D The functions described herein, including one or more of the following, can be performed by one or more emulation devices (not shown): WTRU 102a-d, base station 114a-b, eNode-B160a-c, MME 162, SGW 164, PGW 166, gNB180a-c, AMF 182a-b, UPF 184a-b, SMF183a-b, DN185a-b, and / or any other devices(s) described herein. An emulation device can be one or more devices configured to emulate one or more of the functions described herein. For example, an emulation device can be used to test other devices and / or simulate network and / or WTRU functions.

[0085] Simulation devices can be designed to perform tests on one or more other devices in laboratory and / or carrier network environments. For example, one or more simulation devices can perform one or more or all functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices within the communication network. One or more simulation devices can perform one or more or all functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. Simulation devices can be directly coupled to another device for testing purposes and / or can perform tests using over-the-air wireless communication.

[0086] One or more simulation devices may perform one or more functions, including all functions, rather than being implemented / deployed as part of a wired and / or wireless communication network. For example, simulation devices may be used to test test scenarios in laboratory and / or non-deployment (e.g., testing) wired and / or wireless communication networks to implement the testing of one or more components. One or more simulation devices may be test devices. Simulation devices may transmit and / or receive data using direct RF coupling and / or wireless communication via RF circuitry (e.g., which may include one or more antennas).

[0087] This application describes various aspects, including tools, features, examples or embodiments, models, methods, etc. Many of these aspects are described in detail, and are generally described in a manner that may sound limiting, at least for the purpose of illustrating the various characteristics. However, this is for the purpose of clarity and does not limit the application or scope of these aspects. In fact, all the different aspects can be combined and interchanged to provide other aspects. Furthermore, these aspects can also be combined and interchanged with aspects described in previous applications.

[0088] The aspects described and envisioned in this application can be implemented in many different forms. Figure 5-12 Some embodiments may be provided, but other embodiments are conceivable. Figure 5-12 The discussion does not limit the breadth of implementation. At least one aspect generally relates to video encoding and decoding, and at least one other aspect generally relates to the transmission of generated or encoded bitstreams. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions stored thereon for encoding or decoding video data according to any of the described methods, and / or computer-readable storage media having bitstreams generated according to any of the described methods stored thereon.

[0089] In this application, the terms “reconstruction” and “decoding” are used interchangeably, the terms “pixel” and “sample” are used interchangeably, and the terms “image”, “picture” and “frame” are used interchangeably.

[0090] The terms HDR (High Dynamic Range) and SDR (Standard Dynamic Range) may be used in this disclosure. These terms generally convey specific values ​​of dynamic range to those skilled in the art. However, alternative embodiments are also contemplated, wherein a reference to HDR is understood to mean “higher dynamic range” and a reference to SDR is understood to mean “lower dynamic range”. Such alternative embodiments are not limited to any specific value of dynamic range that may often be associated with the terms “high dynamic range” and “standard dynamic range”.

[0091] This document describes various methods, and each method includes one or more steps or actions for implementing the method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined. Furthermore, terms such as "first" and "second" can be used in various embodiments to modify elements, components, steps, operations, etc., such as, for example, "first decoding" and "second decoding." Unless specifically required, the use of these terms does not imply an ordering of the modified operations. Therefore, in this example, the first decoding does not need to be performed before the second decoding and can occur, for example, before, during, or in a time period overlapping with the second decoding.

[0092] The various methods and other aspects described in this application can be used to modify the video encoder 200 and decoder 300 (respectively, as shown in the following examples). Figure 2 and Figure 3 Modules (as shown) include, for example, intra-frame prediction and entropy coding and / or decoding modules (260, 360, 245, 330). Furthermore, the subject matter disclosed herein proposes aspects not limited to VVC or HEVC and can be applied to, for example, any type, format, or version of video decoding, whether described in standards or recommendations, whether pre-existing or future-developed, and any extensions to such standards and recommendations (e.g., including VVC and HEVC). Unless otherwise stated or technically excluded, the aspects described in this application may be used individually or in combination.

[0093] Various numerical values, such as coefficients and block sizes, are used in the examples described in this application. These and other specific values ​​are for illustrative purposes only, and the aspects described are not limited to these specific values.

[0094] Figure 2 This is a diagram illustrating an example video encoder (e.g., an example block-based hybrid video encoder) 200. Variations of the example encoder 200 are envisioned, but for clarity, encoder 200 is described below without describing all anticipated variations.

[0095] Before being encoded, the video sequence may undergo pre-coding (201), such as applying a color transform to the input color image (e.g., a conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input image components to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization with one of the color components). Metadata may be associated with this pre-processing and appended to the bitstream.

[0096] In encoder 200, the image is encoded by encoder elements as described below. The image to be encoded is partitioned (202) and processed in units, for example, decoding units (CUs). Each unit is encoded using, for example, intra-frame or inter-frame modes. When a unit is encoded in intra-frame mode, it performs intra-frame prediction (260). In inter-frame mode, motion estimation (275) and compensation (270) are performed. The encoder determines (205) which of the intra-frame or inter-frame modes to use to decode the unit and indicates the intra-frame / inter-frame decision by, for example, a prediction mode flag. For example, the prediction residual is calculated by subtracting (210) the prediction block from the original image block.

[0097] The predicted residual is then transformed (225) and quantized (230). The quantized transform coefficients, along with the motion vector and other syntax elements, are entropy encoded (245) to output a bitstream. The encoder can skip the transform and apply the quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., the residual is directly decoded without applying either the transform or quantization process.

[0098] The encoder decodes the coded blocks to provide a reference for further prediction. The quantized transform coefficients are dequantized (240) and inversely transformed (250) to decode the prediction residuals. The decoded prediction residuals and prediction blocks are combined (255) to reconstruct the image blocks. An in-loop filter (265) is applied to the reconstructed image to perform, for example, deblocking / SAO (Sample Adaptive Shift) filtering, thereby reducing coding artifacts. The filtered image is stored in a reference image buffer (280).

[0099] Figure 3 This is a diagram illustrating an example of a video decoder 300. In the example decoder 300, the bitstream is decoded by decoder elements as described below. The video decoder 300 typically performs the same operations as... Figure 2 The encoding process described herein is the inverse of the decoding process. Encoder 200 typically also performs video decoding as part of the encoded video data.

[0100] Specifically, the input to the decoder includes a video bitstream, which can be generated by the video encoder 200. The bitstream is first entropy decoded (330) to obtain transform coefficients, motion vectors, and other decoding information. Image partitioning information indicates how the image is partitioned. Therefore, the decoder can partition the image according to the decoded image partitioning information (335). The transform coefficients are dequantized (340) and inverse transformed (350) to decode the prediction residuals. The decoded prediction residuals and prediction blocks are combined (355) to reconstruct image blocks. The prediction blocks can be obtained from intra-frame prediction (360) or motion-compensated prediction (i.e., inter-frame prediction) (375) (370). An in-loop filter (365) is applied to the reconstructed image. The filtered image is stored at a reference image buffer (380). For the same image, the contents of the reference image buffer 380 on the decoder 300 side can be the same as the contents of the reference image buffer 280 on the encoder 200 side.

[0101] The decoded image may undergo further post-decoding processing (385), such as inverse color transformation (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping, which is the inverse of the remapping process performed in the pre-encoding process (201). Post-decoding processing may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0102] Figure 4 This is a diagram illustrating an embodiment of a system in which the various aspects and examples described herein can be implemented. System 400 can be implemented as a device including the various components described below and configured to perform one or more aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 400 may be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one example, the processing and encoder / decoder elements of system 400 are distributed across multiple ICs and / or discrete components. In various embodiments, system 400 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 400 is configured to implement one or more aspects described in this document.

[0103] System 400 includes at least one processor 410 configured to execute instructions loaded thereon for implementing various aspects, such as those described in this document. Processor 410 may include embedded memory, input / output interfaces, and various other circuitry, as known in the art. System 400 includes at least one memory 420 (e.g., a volatile memory device and / or a non-volatile memory device). System 400 includes a storage device 440, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 440 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0104] System 400 includes an encoder / decoder module 430 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 430 may include its own processor and memory. The encoder / decoder module 430 represents multiple modules that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Furthermore, the encoder / decoder module 430 may be implemented as a separate element of system 400, or it may be incorporated within processor 410 as a combination of hardware and software, as is known to those skilled in the art.

[0105] Program code to be loaded onto processor 410 or encoder / decoder 430 to execute the various aspects described in this document may be stored in storage device 440 and subsequently loaded onto memory 420 for execution by processor 410. According to various embodiments, one or more of processor 410, memory 420, storage device 440, and encoder / decoder module 430 may store one or more various items during the execution of the processes described in this document. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0106] In some embodiments, the memory within the processor 410 and / or encoder / decoder module 430 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, external memory (e.g., the processing device may be the processor 410 or the encoder / decoder module 430) is used for one or more of these functions. External memory may be memory 420 and / or storage device 440, such as dynamically volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one embodiment, a fast external dynamic volatile memory, such as RAM, is used as working memory for video encoding and decoding operations, such as MPEG-2 (MPEG stands for Moving Picture Experts Group; MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Decoding; also known as H.265 and MPEG-H Part 2), or VVC (Various Video Decoding, a new standard being developed by JVET (Joint Video Experts Team)).

[0107] Inputs can be provided to the components of system 400 through various input devices as shown in block 445. Such input devices include, but are not limited to, (i) a radio frequency (RF) section that receives, for example, RF signals transmitted over the air by a broadcaster, (ii) component (COMP) input terminals (or a set of COMP input terminals), (iii) universal serial bus (USB) input terminals, and / or (iv) high-definition multimedia interface (HDMI) input terminals. Figure 4 Other examples not shown include composite video.

[0108] In various embodiments, the input device of block 445 has associated corresponding input processing elements, as known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also referred to as selecting a signal, or limiting the signal band to a band), (ii) down-converting the selected signal, (iii) further band-limiting to a narrower band to select, for example, a signal band, which may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section in various embodiments includes one or more elements performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners performing various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or to baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band. Various embodiments rearrange the order of the above (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0109] USB and / or HDMI terminals may include corresponding interface processors for connecting system 400 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented as needed, for example, within a separate input processing IC or within processor 410. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed within a separate interface IC or within processor 410. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, processor 410 and encoder / decoder 430, which operate in conjunction with memory and storage elements to process the data streams as needed for presentation on the output device.

[0110] Various components of system 400 can be housed within an integrated housing, where they can be interconnected and transmit data therebetween using a suitable connection arrangement 425, such as an internal bus known in the art, including inter-IC (I2C) bus, wiring, and printed circuit board.

[0111] System 400 includes a communication interface 450 that enables communication with other devices via a communication channel 460. The communication interface 450 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 460. The communication interface 450 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 460 may be implemented, for example, within a wired and / or wireless medium.

[0112] In various embodiments, data is streamed or otherwise provided to system 400 using a wireless network such as Wi-Fi (e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers)). In these examples, Wi-Fi signals are received on communication channel 460 and communication interface 450, which are adapted for Wi-Fi communication. Communication channel 460 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communication. Other embodiments use a set-top box that delivers data via an HDMI connection to input block 445 to provide streaming data to system 400. Still other embodiments use an RF connection to input block 445 to provide streaming data to system 400. As described above, various embodiments provide data in a non-streaming manner. Furthermore, various embodiments also use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0113] System 400 can provide output signals to various output devices, including a display 475, a speaker 485, and other peripheral devices 495. The display 475 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a flexible display, and / or a foldable display. The display 475 can be used in a television, tablet computer, laptop, cellular phone (mobile phone), or other device. The display 475 can also be integrated with other components (e.g., in a smartphone) or stand alone (e.g., an external monitor for a laptop). In various examples of embodiments, other peripheral devices 495 include one or more of a standalone digital video disc (or digital multifunction disc) (DVR, for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 495 that provide functionality based on the output of system 400. For example, a disc player performs the function of playing the output of system 400.

[0114] In various embodiments, signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols are used to communicate control signals between system 400 and display 475, speaker 485, or other peripheral devices 495, enabling device-to-device control with or without user intervention. Output devices may be communicatively coupled to system 400 via dedicated connections through corresponding interfaces 470, 480, and 490. Alternatively, output devices may be connected to system 400 via communication interface 450 using communication channel 460. In electronic devices such as, for example, televisions, display 475 and speaker 485 may be integrated with other components of system 400 in a single unit. In various embodiments, display interface 470 includes display drivers, such as, for example, a timing controller (TCon) chip.

[0115] For example, if the RF portion of input 445 is part of a separate set-top box, then display 475 and speaker 485 may alternatively be separated from one or more other components. In various embodiments where display 475 and speaker 485 are external components, the output signal may be provided via a dedicated output connection including, for example, an HDMI port, a USB port, or a COMP output.

[0116] The embodiments may be executed by computer software implemented by processor 410, or by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. Memory 420 may be of any type suitable for the technical environment and may be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. Processor 410 may be of any type suitable for the technical environment, and as a non-limiting example, processor 410 may comprise one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0117] Various implementations involve decoding. As used herein, “decoding” can encompass, for example, all or part of the process performed on a received encoded sequence to produce a final output suitable for display. In various embodiments, such a process includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such a process also includes, or alternatively includes, processes performed by a decoder of the various implementations described herein, such as: obtaining a video block where the width and height of the video block are not equal; determining an intra-prediction mode suitable for the video block; determining a partition type of the video block based on the width and height of the video block; partitioning the video block into a first partition and a second partition based on the partition type, wherein the first partition of the video block will be predicted using a horizontal or vertical intra-prediction mode; predicting the first partition of the video block using a horizontal or vertical intra-prediction mode, and predicting the second partition of the video block using an intra-prediction mode suitable for the video block, wherein the intra-prediction mode suitable for the video block may not be a horizontal or vertical intra-prediction mode; and decoding the video block based on the first and second partitions, etc. A block can be, for example, a decoding block.

[0118] As a further embodiment, in one example "decoding" refers only to entropy decoding, in another embodiment "decoding" refers only to differential decoding, and in yet another embodiment "decoding" refers to a combination of entropy decoding and differential decoding. It will be clear, and is considered well understood by those skilled in the art, whether the phrase "decoding process" is intended to specifically refer to a subset of operations or to refer to a broader decoding process, based on the context of the specific description.

[0119] Various implementations involve encoding. In a manner similar to the discussion of “decoding” above, “encoding,” as used herein, can encompass all or part of the process performed, for example, on an input video sequence to produce an encoded bitstream. In various embodiments, such a process includes one or more processes typically performed by an encoder, such as partitioning, differential coding, transform, quantization, and entropy coding. In various embodiments, such a process also includes, or alternatively includes, processes performed by encoders of various implementations described herein, such as: obtaining a video block, wherein the width and height of the video block are not equal; determining an intra-prediction mode suitable for the video block; determining a partition type of the video block based on the width and height of the video block; partitioning the video block into a first partition and a second partition based on the partition type, wherein the first partition of the video block will be predicted using a horizontal or vertical intra-prediction mode; predicting the first partition of the video block using a horizontal or vertical intra-prediction mode, and predicting the second partition of the video block using an intra-prediction mode suitable for the video block, wherein the intra-prediction mode suitable for the video block may not be a horizontal or vertical intra-prediction mode; and encoding the video block based on the first and second partitions of the video block, etc. The block may be, for example, a decoded block.

[0120] As a further example, in one embodiment "encoding" refers only to entropy encoding, in another embodiment "encoding" refers only to differential encoding, and in yet another embodiment "encoding" refers to a combination of differential and entropy encoding. It will be clear, and is considered well understood by those skilled in the art, whether the phrase "encoding process" is intended to specifically refer to a subset of operations or to refer to a broader encoding process, based on the context of the specific description.

[0121] Note the syntax elements used in this article, for example, NO_SPLIT , HOR_SPLIT , VER_SPLIT , ISPmode , MPMflag , splitFlag "etc." are descriptive terms. Therefore, they do not preclude the use of other syntactic element names.

[0122] When the accompanying drawings are presented as flowcharts, it should be understood that they also provide block diagrams of the corresponding devices. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that they also provide flowcharts of the corresponding methods / processes.

[0123] Various embodiments involve rate-distortion optimization. Specifically, during the encoding process, a balance or trade-off between rate and distortion is typically considered, usually with constraints on computational complexity. Rate-distortion optimization is generally formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. Different approaches exist for solving rate-distortion optimization problems. For example, these approaches can be based on extensive testing of all encoding options, including all considered modes or decoding parameter values, and a full evaluation of their decoding costs and associated distortion of the reconstructed signal after encoding and decoding. Faster methods can also be used to save encoding complexity, particularly by calculating approximate distortion based on prediction or prediction of the residual signal (rather than the reconstructed signal). A hybrid of these two approaches can also be used, such as by using approximate distortion for only some of the possible encoding options and full distortion for others. Other methods evaluate only a subset of the possible encoding options. More generally, many methods employ any of a variety of techniques to perform optimization, but optimization is not necessarily a full evaluation of decoding costs and associated distortion.

[0124] The implementations and aspects described herein can be implemented, for example, as methods or processes, apparatuses, software programs, data streams, or signals. Even if discussed only in the context of a single implementation (e.g., discussed only as a method), the implementation of the features in question can be implemented in other forms (e.g., apparatuses or programs). Apparatuses can be implemented, for example, in suitable hardware, software, and firmware. Methods can be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as, for example, computers, cellular phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0125] References to “an embodiment,” “an example,” “an implementation,” or “implementation,” and other variations thereof, mean that a particular feature, structure, characteristic, etc., described in connection with that embodiment is included in at least one embodiment. Therefore, the phrases “in an embodiment,” “in an embodiment,” “in an example,” “in an implementation,” or “in an implementation,” and any other variations appearing in different places throughout the application, do not necessarily refer to the same embodiment or example.

[0126] Furthermore, this application may refer to "determining" various information fragments. Determining information may include, for example, one or more of estimated information, calculated information, predicted information, or information retrieved from memory. Obtaining may include receiving, retrieving, constructing, generating, and / or determining.

[0127] Furthermore, this application may refer to "accessing" various information fragments. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or one or more of these.

[0128] Furthermore, this application may refer to "receiving" various pieces of information. Like "accessing," receiving is intended to be a broad term. Receiving information may include, for example, accessing information or retrieving information (e.g., from memory) or one or more of it. Moreover, "receiving" is generally referred to in one way or another during operations such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0129] It should be understood that any use of “ / ”, “and / or”, and “at least one” below (e.g., in the cases of “A / B”, “A and / or B”, and “at least one of A and B”) is intended to include selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As another example, in the cases of “A, B, and / or C” and “at least one of A, B, and C”, such wording is intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended with the number of items listed.

[0130] Furthermore, as used herein, the term "signal" refers to instructing the corresponding decoder to something, among other things. In this way, in one embodiment, the same parameters are used on both the encoder and decoder sides. Thus, for example, the encoder can send (explicit signaling) a specific parameter to the decoder so that the decoder can use the same specific parameter. Conversely, if the decoder already has that specific parameter as well as other parameters, signaling can be used without sending (implicit signaling) to simply allow the decoder to know and select that specific parameter. Bit savings are achieved in various embodiments by avoiding the transmission of any actual functions. It should be understood that signaling can be done in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the foregoing refers to the verb form of the word "signal," the word "signal" can (e.g., may also) be used as a noun herein.

[0131] As will be apparent to those skilled in the art, implementations can generate various signals formatted to carry information that can be stored or transmitted. The information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, the signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. The signal may be transmitted via a variety of different wired or wireless links, as known. The signal may be stored on a processor-readable medium.

[0132] This document describes numerous embodiments. Features of the embodiments may be provided individually or in any combination across various claim classes and types. Furthermore, embodiments may include one or more of the features, devices, or aspects described herein, individually or in any combination across various claim classes and types. For example, the features described herein may be implemented in a bitstream or signal including information generated as described herein. This information may allow for a decoder to decode the bitstream, an encoder according to any of the described embodiments, a bitstream, and / or a decoder. For example, the features described herein may be implemented by creating and / or transmitting and / or receiving and / or decoding a bitstream or signal. For example, the features described herein may be implemented as a method, process, apparatus, medium storing instructions, medium storing data, or signal. For example, the features described herein may be implemented by a TV, set-top box, cellular phone, tablet computer, or other electronic device performing decoding. The TV, set-top box, cellular phone, tablet computer, or other electronic device may display (e.g., using a monitor, screen, or other type of display) the obtained image (e.g., an image reconstructed from a residual of a video bitstream). The TV, set-top box, cellular phone, tablet computer, or other electronic device may receive a signal including an encoded image and perform decoding.

[0133] This paper describes systems, devices, and methods for intra-frame prediction using block segmentation.

[0134] In the example, a device for video decoding (e.g., a video decoder) can obtain a video block. The width and height of the video block may not be equal. The device can determine an intra-prediction mode applicable to the video block. The device can determine the partition type of the video block based on its width and height (e.g., the partition type may be determined as a horizontal partition type based on the video block's height being greater than its width). The device can partition the video block into a first partition and a second partition based on the partition type (e.g., the first partition can be predicted using either a horizontal or vertical intra-prediction mode). The device can predict the first partition using either a horizontal or vertical intra-prediction mode and predict the second partition using an intra-prediction mode applicable to the video block. The intra-prediction mode applicable to the video block may not be a horizontal or vertical intra-prediction mode. The device can decode the video block based on the first and second partitions.

[0135] In the example, the device may receive a partitioning indicator that indicates a video block (e.g., a video block with unequal widths and heights) will be partitioned based on the width and height of the video block. The device may also receive a prediction scheme indicator that indicates whether a vertical intra-frame prediction mode or a horizontal intra-frame prediction mode will be used to predict the first partition of the video block. The first partition of the video block can be predicted according to the prediction scheme indicator. For example, if the prediction scheme indicator indicates that a vertical intra-frame prediction mode will be used to predict the first partition of the video block, the device can use the vertical intra-frame prediction mode to predict the first partition of the video block. If the prediction scheme indicator indicates that a horizontal intra-frame prediction mode will be used to predict the first partition of the video block, the device can use the horizontal intra-frame prediction mode to predict the first partition of the video block.

[0136] In the example, the device can determine whether a vertical intra-prediction mode or a horizontal intra-prediction mode will be used to predict the first partition of the video block based on the partition type of the video block. The first partition of the video block can be predicted based on the determination of whether a vertical or horizontal intra-prediction mode will be used to predict the first partition of the video block. For example, if the partition type of the video block is vertical, the device can use the vertical intra-prediction mode to predict the first partition of the video block. If the partition type of the video block is horizontal, the device can use the horizontal intra-prediction mode to predict the first partition of the video block.

[0137] The device can determine whether to enable one of the following for a video block: Template-based Intra-Prediction (TIMD), Decoder-Side Intra-Mode Derivation (DIMD), Multiple Reference Lines (MRL), Matrix-based Intra-Prediction (MIP), Most Probable Mode (MPM), or Template-based Multiple Reference Lines (TMRL). The intra-prediction mode applicable to the video block can be determined based on whether one of TIMD, DIMD, MRL, MIP, MPM, or TMRL is enabled. For example, the device can determine whether TIMD is enabled for a video block and determine the intra-prediction mode for the video block based on TIMD. The device can use a vertical / horizontal intra-prediction mode to predict the first partition of the video block and use the intra-prediction mode determined using TIMD to predict the second partition of the video block.

[0138] In some examples, only the vertical intra-frame prediction mode may be allowed for the first partition of a video block, and the first partition of a video block may be predicted using the vertical intra-frame prediction mode.

[0139] In the example, the device can determine the prediction of a video block based on the predictions of the first and second partitions of the video block. For example, the prediction of the video block could be a simple average or a weighted average of the predictions of the first and second partitions. The device can decode the residual coefficients associated with the video block. The device can apply the decoded residual coefficients to the prediction of the video block. The video block can be symmetrically partitioned into a first and second partition. The video block can be asymmetrically partitioned into a first and second partition.

[0140] In the example, a device for video encoding (e.g., a video encoder) can obtain a video block. The width and height of the video block may not be equal. The device can determine an intra-prediction mode applicable to the video block. The device can determine the partition type of the video block based on the width and height of the video block (e.g., the partition type of the video block can be determined as a horizontal partition type based on the height of the video block being greater than its width). The device can partition the video block into a first partition and a second partition based on the partition type (e.g., the first partition of the video block can be predicted using a horizontal or vertical intra-prediction mode). The device can predict the first partition of the video block using a horizontal or vertical intra-prediction mode and predict the second partition of the video block using an intra-prediction mode applicable to the video block. The intra-prediction mode applicable to the video block may not be a horizontal or vertical intra-prediction mode. The device can encode the video block based on the first and second partitions of the video block.

[0141] In the example, the device can send a partitioning indicator that indicates a video block (e.g., a video block with unequal widths and heights) will be partitioned based on the width and height of the video block. The device can also send a prediction scheme indicator that indicates whether a vertical intra-frame prediction mode or a horizontal intra-frame prediction mode will be used to predict the first partition of the video block. The first partition of the video block can be predicted according to the prediction scheme indicator. For example, if the prediction scheme indicator indicates that a vertical intra-frame prediction mode will be used to predict the first partition of the video block, the device can use the vertical intra-frame prediction mode to predict the first partition of the video block. If the prediction scheme indicator indicates that a horizontal intra-frame prediction mode will be used to predict the first partition of the video block, the device can use the horizontal intra-frame prediction mode to predict the first partition of the video block.

[0142] In the example, the device can determine whether a vertical intra-prediction mode or a horizontal intra-prediction mode will be used to predict the first partition of the video block based on the partition type of the video block. The first partition of the video block can be predicted based on the determination of whether a vertical or horizontal intra-prediction mode will be used to predict the first partition of the video block. For example, if the partition type of the video block is vertical, the device can use the vertical intra-prediction mode to predict the first partition of the video block. If the partition type of the video block is horizontal, the device can use the horizontal intra-prediction mode to predict the first partition of the video block.

[0143] The device can determine whether to enable one of the following for a video block: TIMD, DIMD, MRL, MIP, MPM, or TMRL. The intra-prediction mode applicable to the video block can be determined based on the enabled option of TIMD, DIMD, MRL, MIP, MPM, or TMRL. For example, the device can determine whether to enable TIMD for a video block and determine the intra-prediction mode for the video block based on TIMD. The device can use a vertical / horizontal intra-prediction mode to predict the first partition of the video block and use the intra-prediction mode determined using TIMD to predict the second partition of the video block.

[0144] In some examples, only the vertical intra-frame prediction mode may be allowed for the first partition of a video block, and the first partition of a video block may be predicted using the vertical intra-frame prediction mode.

[0145] In the example, the device can determine the prediction of a video block based on the predictions of the first partition and the second partition. For example, the prediction of a video block can be a simple average or a weighted average of the predictions of the first and second partitions. A video block can be symmetrically partitioned into first and second partitions. A video block can be asymmetrically partitioned into first and second partitions.

[0146] In the example, predictions made for the first partition of a video block using either a horizontal or vertical intra-prediction mode, and predictions made for the second partition of a video block using an intra-prediction mode applicable to the video block, can be evaluated for predictions of an unpartitioned video block. The device can determine that the first prediction will be used to encode the video block. The device can send a partitioning indicator that indicates that the video block (e.g., a video block with unequal widths and heights) will be partitioned based on the width and height of the video block.

[0147] Intra-prediction tools (e.g., TIMD, DIMD, MRL, MIP, IntraTMP, MPM, or TMRL) can be used to derive an intra-prediction mode for the current block, which can be used to predict a second partition of the current block using intra-prediction that leverages block segmentation as described in one or more examples herein.

[0148] Example prediction tools / patterns as described in one or more examples herein may include decoder-side intra-frame pattern derivation (DIMD) and / or template-based intra-frame pattern derivation (TIMD). In these patterns (e.g., two patterns), a prediction pattern (e.g., an intra-frame prediction pattern) can be derived using reconstructed pixels (e.g., template pixels) surrounding the current block in the up and left directions. In DIMD, the template of the reconstructed pixels can be analyzed to infer the template's orientation, from which a directional pattern (e.g., five directional patterns) can be selected. The prediction signal can be generated by blending the patterns (e.g., five patterns) with planar patterns. In TIMD, the prediction pattern (e.g., an intra-frame prediction pattern) can be tested on the template of the reconstructed pixels. An optimal pattern (e.g., two optimal patterns) can be selected (e.g., the pattern that minimizes the sum of absolute transform differences (SATD) between the template of the reconstructed pixels and their predictions). The prediction signal can be generated by applying the optimal pattern or by blending those patterns (e.g., two patterns), depending on their prediction SATD.

[0149] This paper provides several features associated with decoder-side intra-frame mode derivation (DIMD).

[0150] When applying DIMD, one or more (e.g., five) intra-frame modes can be derived from reconstructed neighboring samples (e.g., by analyzing the directionality of the content around the current block). These (e.g., five) predictors can be combined with planar mode predictors (e.g., using weights derived from a histogram of gradients (HoG)). The HoG can be computed on an L-shaped template formed by the reconstructed samples (e.g., an L-shaped template with three sample widths / heights). Vertical and horizontal Sobel filters can be applied to the L-shaped template to obtain gradients (e.g., obtaining gradients at each location where vertical and horizontal Sobel filters are applied), and the HoG can be incremented by the magnitude of the gradients (e.g., the magnitude of each gradient increments the bins associated with that gradient direction). The direction associated with the HoG bin with the highest cumulative magnitude can be selected as the derived DIMD mode. The predictors obtained with the derived DIMD modes can be blended to form the final DIMD prediction. Uniform blending or spatial blending can be used (e.g., where DIMD predictors are combined with planar predictors, e.g., using weights that depend on the relative magnitudes of the modes in the HoG).

[0151] The deduced intra-frame pattern can be included in the master list of most probable intra-frame patterns (MPMs). The DIMD process can be performed before constructing the MPM list. The master deduced intra-frame pattern of a DIMD block can be stored with the block. The master deduced intra-frame pattern can be used for constructing the MPM list of neighboring blocks.

[0152] The region adjacent to the reconstructed sample (e.g., used to compute the gradient histogram) can depend on the availability of the reconstructed sample. The region of the current WxH brightness CB decoding reference sample can extend towards the upper right (e.g., up to W additional columns if available). The region of the current WxH brightness CB decoding reference sample can extend towards the lower left (e.g., up to H additional rows if available).

[0153] It can calculate the DIMD histogram.

[0154] A histogram can be computed by applying horizontal and vertical Sobel filters to pixels within a template (e.g., with a width of 3) surrounding the block. For example, for (e.g., each) pixel, the following filters can be applied: .

[0155] if Gy and Gx If neither of these two values ​​is zero, then the orientation of the current pixel is calculated as follows: orientation Otherwise, if Gy is zero, the direction can be calculated / treated as vertical. Otherwise, the orientation can be calculated / treated as horizontal.

[0156] A histogram can be calculated by summing the values ​​for each orientation (e.g., summing all values). A histogram may not show whether the template has no orientation. For example, DIMD may not output a DC or planar pattern.

[0157] Template-based multi-reference line intra-frame prediction (TMRL) modes can combine reference lines and prediction modes, and use template matching methods to construct a candidate combination list. The indices of the candidate combination list can be decoded to indicate which reference line and / or prediction mode can be used to decode the current block. The MRL for non-TIMD portions can be replaced by TMRL modes.

[0158] The TMRL mode can expand the reference row candidate list and / or the intra-prediction mode candidate list. For example, the expanded reference row candidate list can include rows {1, 3, 5, 7, 12}. Constraints regarding the top CTU row can remain unchanged. For example, the size of the intra-prediction mode candidate list can be set to 10. The construction of the intra-prediction mode candidate list can be similar to that of the MPM. Planar modes can be excluded from the intra-prediction mode candidate list. DC modes can be added after one or more (e.g., five) modes of neighboring PUs and / or one or more DIMD modes (e.g., if they are not included). One or more angle modes with delta angles from ±1 to ±4 can be added. The accuracy of angle prediction can be extended from 65 to 129. Non-adjacent locations can be added as candidates to construct the intra-prediction candidate list. If neighboring or non-adjacent blocks are decoded using SGPM or GPM modes, the intra-prediction modes of those blocks can be replaced by partitioned angles.

[0159] The TMRL candidate list can be constructed as follows. For a block, there can be multiple (e.g., 5x10=50) combinations of extended reference lines and allowed intra-prediction modes. Since the extended reference lines can start from reference line 1, the region covered by reference line 0 can be used for template matching. The SAD cost on the template region can be computed between predicted samples and reconstructed samples (e.g., generated from 50 combinations). Multiple (e.g., 20) combinations with the lowest SAD cost can be selected in ascending order to form the TMRL candidate list.

[0160] The index of the TMRL candidate list can be decoded (e.g., instead of directly decoding the reference line and intra-frame mode) to indicate which combination of reference line and prediction mode can be used to decode the current block.

[0161] Matrix-weighted intra-frame prediction (MIP) can use H reconstructed neighbor boundary samples from the left row of the block and W reconstructed neighbor boundary samples from the top row of the block as input. If reconstructed samples are unavailable, they can be generated (e.g., as in intra-frame prediction). The generation of the predicted signal can be based on (e.g., averaging, matrix-vector multiplication, and linear interpolation). One or more predefined (fixed) weighting matrices corresponding to various MIP modes can be used.

[0162] Multi-reference row (MRL) prediction can use an MRL candidate list, which may include, for example, two (2) additional reference rows, such as {1, 2}. The rows (e.g., each row) can be tested against predictions utilizing a prediction pattern from the MPM list (e.g., the exclusion plane (PLANAR) pattern). The selected reference row index can be encoded as a flag (e.g., multiRefIdx). The flag (e.g., multiRefIdx) can be (e.g., set to have a value) zero (0), one (1), or two (2). Predictions may or may not be merged. In some examples, predictions may be based on a single reference row. In some examples, for example, ISP may be disallowed if / when multiRefIdx equals one (1) or two (2).

[0163] This paper presents several features associated with intra-frame prediction utilizing block segmentation.

[0164] One or more specifications can support intra-frame prediction involving the partitioning of a block into sub-partitions. For example, intra-frame prediction utilizing block partitioning can be implemented based on intra-frame prediction with sub-partitions (ISP) or spatial geometric partitioning mode (SGPM). In ISP, a decoding unit (CU) can be partitioned into two or four equal-sized sub-partitions (e.g., horizontally or vertically). The sub-partitions can be encoded and decoded (e.g., sequentially) using the prediction mode of the CU. In SGPM, a CU can be partitioned into two parts (e.g., along a predefined direction). The parts can be encoded using two different prediction modes (e.g., they can be mixed to obtain the final prediction). SGPM can be more flexible than ISP (e.g., in terms of partitioning direction and allowed prediction modes). The features(s) described herein can be less complex than SGPM. The features(s) described herein can support different prediction modes (e.g., two different prediction modes) for parts obtained by vertical or horizontal partitioning (e.g., two parts).

[0165] The CU can be divided into parts (e.g., two parts) based on (e.g., depending on) its shape. If the CU is flat (e.g., a flat shape, such as its width being greater than its height), the CU can be divided vertically. If the CU is tall (e.g., its height being greater than its width), the CU can be divided horizontally. The CU can be divided at (pre)defined locations. A part (e.g., one of two parts) can be determined (e.g., constrained) to have (e.g., strictly) vertical or horizontal prediction patterns (e.g., depending on the segmentation type). The other part can be predicted using the prediction pattern of the CU. The prediction pattern can be derived using a template. The prediction pattern can be signaled (e.g., as with any other prediction pattern). The predictions of the parts (e.g., both parts) can be combined (e.g., as in SGPM) to obtain a final prediction.

[0166] The features described in this paper can be adopted / implemented in the standard to support the described block partitioning.

[0167] In intra-frame prediction, a target block (e.g., sometimes referred to as a decoding unit (CU)) can be intra-frame predicted as (e.g., a single) block, or segmented into multiple parts (e.g., these multiple parts can be predicted individually). ISP and SGPM can be classified separately from other tools (e.g., such as DIMD, TIMD, Intra-Template Matching Prediction (IntraTMP), MIP, TMRL, MRL, etc.). In ISP, a target CU can be divided into multiple (e.g., two or four) equal-sized sub-partitions. Sub-partitions can be decoded using the prediction pattern of the CU (e.g., sequentially). Sub-partitions can use decoded samples from neighboring sub-partitions (e.g., benefiting from their availability). Neighboring sub-partitions can be the immediate neighbors of the current sub-partition. In SGPM, a CU can be segmented into (e.g., two) geometric partitions based on (pre)defined segmentation directions. Predictions can be determined / predicted using one or more (e.g., two) prediction patterns. (e.g., two) predictions can be mixed to arrive at a final prediction. (e.g., two) prediction patterns and / or segmentation directions can be determined from a list (e.g., a list derived using template derivation).

[0168] SGPM can be (e.g., very) complex (e.g., and more flexible than ISP) due to the number of prediction patterns, segmentation orientation, and template usage. The features(s) described herein relate to a simpler technique where a rectangular block is segmented into parts (e.g., two parts), where one part (e.g., one of the parts) is predicted based on (e.g., depending on) the segmentation type using a vertical or horizontal pattern (e.g., strictly vertical or horizontal). In this case, the other part can be predicted using the CU's prediction pattern. The predictions can be available at the encoder. The technique described herein eliminates the need to examine the additional complexity in the rate-distortion (RD) performance of the block segmentation. If the predictions utilizing the block segmentation produce better RD performance for the current CU, the encoder can signal an indication of the result (e.g., with a 1-bit flag). The decoder can receive this indication and perform predictions based on it (e.g., perform the same predictions after decoding the flag value).

[0169] This paper provides (multiple) features associated with intra-frame prediction (ISP) with sub-partitions.

[0170] The ISP tool can divide the lumen intra-prediction block vertically or horizontally into sub-partitions (e.g., 2 or 4 sub-partitions), depending on the block size (e.g., as shown in Table 1). Sub-partitions can (e.g., must) have at least 16 samples. A 4x4 block may not be sub-partitioned. For example, a 4x8 block can be horizontally divided into (e.g., only) two 4x4 sub-partitions (e.g., each). As another example, an 8x4 block can be vertically divided into (e.g., only) two 4x4 sub-partitions (e.g., each). As another example, a 16x4 block can be vertically divided into (e.g., only) four 4x4 sub-partitions (e.g., each). In some examples (e.g., for hardware implementation convenience), the sub-partition width (e.g., minimum size) can be (e.g., constrained to) 4 during the intra-prediction step. In this scenario, a 4x8 block can be vertically partitioned (e.g., only) into two 2x8 sub-partitions (e.g., each) for transform decoding and / or entropy decoding of the transform coefficients (e.g., the 4x8 block may not be vertically partitioned during the intra-frame prediction step). In another example, a 4x16 block can be vertically partitioned (e.g., only) into four 1x16 sub-partitions (e.g., each) for transform decoding and / or entropy decoding of the transform coefficients (e.g., the 4x16 block may not be vertically partitioned during the intra-frame prediction step). Figure 5 and Figure 6 An example of block segmentation is illustrated. block size Number of sub-partitions 4×4 1 8×4 2 16x16, 16x32, 32x16 4

[0171] Table 1: Number of sub-partitions depending on block size Figure 5 The illustration shows an example partition of a WxH block, such as H = 4 and W = 8. Figure 6 The illustration shows an example partition of a WxH block, such as H = 16 and W = 32.

[0172] For each sub-partition (e.g., each of the sub-partitions), a prediction can be constructed using the parent CU's decoding prediction mode. The prediction signal can be included (e.g., added to) the decoded residual signal of the sub-partition. The decoded residual signal can be generated by entropy decoding, inverse quantization, and inverse transform of the coefficients sent by the encoder to reconstruct the pixels in the sub-partition. The reconstructed values ​​of each sub-partition (e.g., except for the first sub-partition) can be available to generate the prediction for the next sub-partition.

[0173] Sub-partitions can be processed sequentially (e.g., in normal order), regardless of the intra-prediction mode and the segmentation used. For example, the first sub-partition to be processed could be the sub-partition that includes the top-left sample of the CU. The remaining sub-partitions can be processed sequentially by continuing downwards (e.g., for horizontal segmentation) or to the right (e.g., for vertical segmentation). The segmentation type of the CU can be transmitted using an indication. For example, this indication can be a bit. Bit "0" can indicate no segmentation (e.g., NO_SPLIT). Bits "10" and "11" can indicate horizontal segmentation (HOR_SPLIT) and vertical segmentation (e.g., VER_SPLIT), respectively.

[0174] One or more decoding techniques can be used / supported. For example, decoding techniques may include intra prediction with multiple reference rows (MRL), matrix-based intra prediction (MIP), template-based intra prediction (TIMD), decoder-side mode derivation (DIMD), spatial GPM (SGPM), intra prediction with template matching (IntraTmp), etc. A target block may be eligible for ISP if (e.g., only if) a first reference row is used (e.g., and MIP, DIMD, SGPM, IntraTmp, etc. are disabled). For example, a CU can be tested against ISP if (e.g., only if) flags indicating the mentioned mode are disabled (e.g., all are disabled). If the block has an MRL index other than 0, or if MIP, DIMD, SGPM, or IntraTmp, etc., are enabled, the ISP decoding mode will be inferred as 0. In this case, the segmentation flag may not be decoded. In this case, the CU (e.g., the entire CU) can be decoded as (e.g., a single) unit (e.g., without sub-segments).

[0175] This paper provides several features associated with the Spatial Geometric Partitioning Pattern (SGPM).

[0176] SGPM can refer to an intra-prediction tool that simulates GPM used for inter-frame prediction. SGPM can involve partitioning the decoded block into (e.g., two) parts. Intra-prediction modes can be generated (e.g., two different intra-prediction modes) (e.g., one intra-prediction mode for each part). For example, predictions for the CU can be constructed (e.g., two predictions for the entire CU) (e.g., one prediction mode for each part). Predictions can be mixed according to the partitioning direction (e.g., so that each part obtains its corresponding prediction), for example, as... Figure 7 As shown.

[0177] Figure 7 The illustration shows an example block segmentation in SGPM. Blocks can be segmented by partitioning patterns or orientations (e.g., any of the 26 partitioning patterns or orientations). This segmentation can produce two parts (e.g., each part is associated with its own prediction pattern).

[0178] Partitioning and associated prediction information can be represented in the bitstream. For example, SGPM can employ a template-reordered candidate list (e.g., where each candidate in the list includes a combination of a partitioned mode and two intra-frame prediction modes). Candidate indices can be signaled in the bitstream. A template can be used to generate this candidate list. Figure 8 The diagram illustrates an example shape of the template. For possible combinations of (e.g., one) partitioned mode and one or more (e.g., two) intra-prediction modes (e.g., each), predictions can be generated for the template (e.g., where partition weights extend to the template). These combinations can be ranked (e.g., in ascending order) based on the associated SAD between the template's predictions and reconstructions. The length of the candidate list can be set to 16. Candidates in the candidate list can be considered the most probable SGPM combinations for the current block. The encoder and decoder can construct the same candidate list based on the template.

[0179] Figure 8 The illustration shows an example template (e.g., a template used in SGPM to derive the SGPM candidate list). The candidate list can include a segmentation direction and two prediction patterns for each segmentation direction.

[0180] To reduce the complexity of constructing the candidate list, the number of possible partitioning modes and the number of possible intra-prediction modes can be pruned. For example, 26 out of 64 partitioning modes can be used. MPMs selected from 67 intra-prediction modes can be used (e.g., only MPMs). The encoder can (e.g., during fast traversal) check decoding performance using the candidates in the list (e.g., all 16). The encoder can shorten the candidate list for a full RD performance test (e.g., only). If a candidate produces the best decoding performance (e.g., among all modes), the encoder can signal to the relevant parties. sgpmFlag And / or (e.g., subsequently) the candidate indices in the list of binary-encoded candidates. The decoder can decode... sgpmFlag (For example, when parsing a bitstream). If the flag is decoded to 1, the decoder can decode the candidate indices in the SGPM list. In this case, the decoder can perform template prediction (e.g., the entire template prediction process) to derive two SGPM prediction patterns for (e.g., each) segmentation direction to derive a list of the best 16 candidates. Based on (e.g., using) the decoded candidate indices, the decoder can decode the two prediction patterns and the segmentation direction. Using the two prediction patterns, the decoder can predict the current block. The decoder can mix the two predictions (e.g., in the same way as the encoder) to arrive at the final prediction.

[0181] This paper provides several features associated with predictions utilizing block segmentation.

[0182] Segmentation is more likely to occur at object boundaries. To keep additional signaling overhead low, rectangular blocks can be segmented (e.g., only rectangular blocks can be segmented). For example, blocks with unequal widths and heights can be segmented.

[0183] A block with a width greater than its height can be vertically divided (e.g., divided into two parts), such as... Figure 9 As shown.

[0184] In the first configuration of the example, the prediction mode associated with the first part of the block (e.g., a sub-partition) can be equal to the prediction mode of the CU. The prediction mode associated with the second part of the block can be equal to (e.g., strict) the vertical mode (e.g., intra-mode index 50).

[0185] In the second configuration of the example, the prediction mode associated with the first part of the block (e.g., a sub-partition) can be equal to a vertical mode or a horizontal mode (e.g., a strictly vertical mode or a strictly horizontal mode). The prediction mode associated with the second part of the block can be equal to the prediction mode of the CU. In both the first and second configurations, the segmentation can be symmetrical (e.g., the first and second parts have the same shape).

[0186] The first configuration (e.g., only the first configuration) can be used for intra-block segmentation to limit signaling overhead. If blocks are segmented, the encoder / decoder can (e.g., implicitly) assume that the two parts have different prediction modes. To limit decoder complexity, templates can be omitted from deriving the prediction modes (e.g., SGPM does use templates).

[0187] Figure 9 The illustration shows an example block segmentation where the width of the decoded block is greater than its height (e.g., when W > H). For example, the partition type of a video block can be determined as a vertical partition type based on the width being greater than its height. The partition type of a video block can be determined as a horizontal partition type based on its height being greater than its width.

[0188] Figure 9 The left side of the diagram illustrates the first example configuration (e.g., the prediction mode of the first part is equal to the prediction mode of the CU, and the prediction mode of the second part is equal to the strictly vertical mode, for example). Figure 9 Intra-frame mode 50 (in the frame). Figure 9 The right side of the diagram illustrates a second example configuration (e.g., the prediction mode of the first part is equal to the strict vertical mode or the strict horizontal mode, and the prediction mode of the second part is equal to the prediction mode of the CU).

[0189] Figure 10 The illustration shows an example block segmentation if the height of the decoded block is greater than the width of the decoded block (e.g., in the case where H > W). In this case, the block can be horizontally divided into two parts. Figure 10 The left side of the diagram illustrates the first example configuration (e.g., the prediction mode of the first part is equal to the prediction mode of CU, and the prediction mode of the second part is equal to the strict level mode). Figure 10 The right side of the diagram illustrates a second example configuration (e.g., the prediction mode of the first part is equal to the strict vertical mode or the strict horizontal mode, and the prediction mode of the second part is equal to the prediction mode of the CU).

[0190] Segmentation (e.g., horizontal segmentation) can be symmetrical (e.g., similar to the case of vertical segmentation). For example, the two parts can have the same shape. A first configuration (e.g., only the first configuration) can be used for intra-frame block segmentation (e.g., to limit signaling overhead).

[0191] Prediction can involve one or more of the following actions. The following example uses a first configuration as an example (e.g., regardless of whether the segmentation is vertical or horizontal). The decoder can (e.g., firstly) use the prediction pattern of the CU to predict the first part (e.g., followed by PDPC (if applicable)). The decoder can use either a vertical or horizontal pattern to predict the second part (e.g., depending on whether the segmentation is vertical or horizontal). The decoder can perform PDPC against the second part (e.g., using the prediction samples from the last column or row of the first part). The decoder can mix the two predictions (e.g., with weights suitable for symmetrical vertical or horizontal segmentation). If the prediction pattern of the CU is equal to the prediction pattern of the second part (e.g., vertical if the width is greater than the height, horizontal if the height is greater than the width), block segmentation may not be performed.

[0192] The following example uses a second configuration as an example. The decoder can (e.g., first) use a vertical or horizontal pattern to predict the first part (e.g., followed by PDPC). The decoder can use the prediction pattern of the CU to predict the second part. The decoder can perform PDPC against the second part (e.g., using the predicted sample of the last column or row in the first part). The decoder can mix the two predictions (e.g., with weights suitable for symmetrical vertical or horizontal splitting).

[0193] If the encoder allows (e.g., only allows) the first configuration, a single bit (e.g., only one bit is needed) can be used to signal whether the rectangular block is split or not (e.g., a partition indicator). In the example, whether to use splitting can be based on an evaluation. For example, this evaluation can be based on a rate-distortion comparison (e.g., comparing the rate-distortion associated with splitting / partitioning with the rate-distortion associated with the prediction of the entire CU / block without splitting / partitioning). Signaling may not be necessary for the rectangular blocks (e.g., because they are not split). If both configurations are allowed, two bits of signaling can be used. For example, the first bit (e.g., a partition indicator) can be used to signal whether the block is split. The second bit (e.g., a partition scheme indicator) can be used to signal the splitting configuration. If vertical and horizontal prediction modes (e.g., both) are allowed for the first partition in the second configuration, signaling bits (e.g., required, e.g., additional) can be used to indicate the mode used (e.g., if the first two bits indicate splitting with the second configuration). In the example, the horizontal or vertical prediction mode can be selected based on an evaluation (e.g., RD ​​performance). If (e.g., two) split configurations are allowed, where (e.g., two) vertical and horizontal modes are allowed in the second configuration, then the signaling scheme can be given as follows: '0' for no split, '10' for splitting with the first configuration, '110' for splitting with the second configuration where the prediction mode for the first partition is vertical, and '111' for splitting with the second configuration where the prediction mode for the first partition is horizontal. Table 2 summarizes this encoding scheme. Split configuration Divider coding Undivided 0 0 Segmented by configuration 1 1 10 The second configuration is used for segmentation, and the prediction pattern of the first part is vertical. 2 110 The prediction pattern of the first part is divided by the second configuration and is horizontal. 3 111

[0194] Table 2: Encoding of block segmentation information using segmentation markers To limit the number of patterns, the encoder can decide to apply block segmentation to a set of patterns (e.g., only to a selected set of patterns). For example, the encoder can apply block segmentation only to patterns in the Most Probable Pattern (MPM) list. In this case, the bits representing the block segmentation can be encoded if (e.g., only if) MPMflag (For example, it indicates whether the prediction pattern of the CU belongs to the MPM list) is enabled.

[0195] The encoder can decide to perform block segmentation if the prediction mode is DIMD, TIMD, or TMRL. In any of these cases, segmentation flag information can be encoded to indicate whether a block is segmented (e.g., as in configuration one or two). If a block is segmented, the decoder may not apply the usual mixing operations (e.g., as done in DIMD, TIMD, TMRL, etc.). For example, the decoder can use (e.g., only) DIMD / TIMD / TMRL modes and vertical / horizontal modes depending on the segmentation configuration for CU prediction.

[0196] In the example, asymmetric partitioning (e.g., instead of symmetric partitioning) can be used, such as... Figure 11 and Figure 12 As shown. Figure 11 The illustration shows an example of asymmetric block segmentation when the width of the block is greater than the height of the block. Figure 11 Asymmetric block partitioning when W>H. Left: Configuration 1; Right: Configuration 2. Figure 12 The illustration shows an example of asymmetric block segmentation when the height of a block is greater than the width of the block. Figure 12 Asymmetric block partitioning when H>W. Left: Configuration 1; Right: Configuration 2. Figure 12 In the first case (e.g., vertical segmentation), two vertical segments at W / 4 and (3W / 4) are allowed. In the second case (e.g., horizontal segmentation), two horizontal segments at H / 4 and (3H / 4) are allowed. One or more of the segments can be allowed by the encoder (e.g., for each segmentation type).

[0197] The features(s) in one or more examples described herein may differ from the ISP.

[0198] In ISP, the current block can be partitioned into two or four sub-partitions (e.g., using the same prediction pattern (e.g., the prediction pattern of the CU) for prediction). Among the features disclosed herein, the two parts of the CU can have different prediction patterns. For example, one part can have the prediction pattern of the CU, and the other part can have a vertical or horizontal pattern (e.g., depending on the partitioning configuration).

[0199] In ISP, sub-partitions can be processed sequentially (e.g., where a sub-partition is decoded after a previous sub-partition has been decoded). In ISP, the reference sample and prediction in a sub-partition (e.g., one) can depend on the decoded sample values ​​in the previous sub-partition. Among the features disclosed herein, the CU can be decoded after the predictions in both parts are added to the decoding residual of the CU (e.g., the entire CU) (e.g., after one step).

[0200] In ISP, the ISPmode flag can be encoded to indicate whether a block will (e.g., needs to) be segmented. If the ISPmode flag indicates that a block will be segmented, then the ISPmode flag will indicate the type of segmentation to be applied (e.g., vertical or horizontal). Among the features disclosed herein, segmentation information can be encoded to indicate which type of segmentation configuration the encoder uses for prediction. The segmentation configuration can indicate the prediction mode used in both parts.

[0201] The features(s) in one or more examples described herein may differ from SGPM.

[0202] In SGPM, the current block can be divided into two parts (e.g., where the division direction is selected from 26 predefined directions, excluding vertical and horizontal divisions at the middle). The division direction may be independent of the block shape. Among the features disclosed herein, the division may be (e.g., only) (e.g., limited to) vertical and horizontal. The division direction may depend on the block shape.

[0203] In SGPM, the predicted patterns of two parts with a segmentation direction can be unequal. The predicted patterns can be determined based on a template. To limit complexity, patterns can be selected (e.g., only) from / in the MPM set. Among the features(s) disclosed herein, the predicted patterns of the two parts can differ. One of the predicted patterns can be fixed (e.g., vertical or horizontal, depending on the block shape). The other predicted pattern can be determined by the encoder (e.g., using a normal search of all possible angular patterns excluding vertical / horizontal patterns from the selected set).

[0204] In SGPM, the encoder and decoder can derive a list of 16 entries, each entry having a segmentation direction and two associated prediction modes for the two parts. The encoder can encode indices of candidates in the list for prediction (e.g., enabling the decoder to decode the segmentation direction and the two associated prediction modes). In the features disclosed herein, the encoder / decoder may not derive (e.g., any) a list. For a given block shape, the segmentation configuration can be fixed. The encoding of the segmentation information can be done directly.

[0205] In SGPM, the prediction process may include predicting the CU (e.g., the entire CU) using two associated patterns. The two predictions can be blended based on the segmentation direction. A similar approach can be followed with the features(s) disclosed herein. Parts can be predicted individually (e.g., the first part, followed by the second part). PDPC can be used after (e.g., each) the predictions.

[0206] SGPM may not be combined with DIMD, TIMD, etc. The features disclosed herein can be (e.g., easily) combined, where the prediction pattern for one part is a DIMD / TIMD pattern and another part has a vertical / horizontal pattern (e.g., depending on the block shape).

[0207] The features disclosed herein may not be considered an adaptation of SGPM. Any coincidences (e.g., in the case of asymmetric segmentation, the occurrence of the segmentation direction in SGPM is the same as that in the disclosed features, and / or two associated prediction patterns are the same as those described) are merely coincidences (e.g., because the candidates in the SGPM list are selected based on their scores with the template predictions).

[0208] This document describes example implementation scenarios for the disclosed features(s). Codecs (e.g., based on standards) may be used and / or assumed.

[0209] In the first example, the encoder may allow block segmentation for rectangular blocks (e.g., only for rectangular blocks) for (e.g., only for) intra-frame modes in the MPM list. If the MPMflag is decoded to 1, the decoder can decode the segmentation flag. Based on the decoded value of the segmentation flag, the decoder can decide: segment the block and apply prediction in both parts, or not segment the CU and perform prediction on (e.g., the entire CU) using the decoded mode. The decoder can add the decoded residual to the prediction to reconstruct the CU. If the segmentation flag is encoded as non-zero, the ISPMode flag is assumed to be zero.

[0210] In the second example, the encoder can allow block segmentation of a rectangular block using (e.g., only) the DIMD mode. If the decoder decodes the DIMD flag to 1, the decoder can decode the segmentation flag information. If the decoded segmentation flag is non-zero, the decoder can segment the block. The decoder can perform prediction in two parts (e.g., using the DIMD mode in one part and a vertical / horizontal segment in another part, depending on the segmentation flag information). If the decoded segmentation flag is zero, the decoder can perform prediction on (e.g., the entire CU) (e.g., in the manner typically done in DIMD). The decoder can add the decoded residual to the prediction to reconstruct the CU.

[0211] In the third example, the encoder can allow block segmentation of a rectangular block using (e.g., only) a TIMD pattern. If the decoder decodes the TIMD flag to 1, the decoder can decode the segmentation flag information. If the decoded segmentation flag is non-zero, the decoder can segment the block. The decoder can perform prediction in two parts (e.g., using the TIMD pattern in one part and a vertical / horizontal segment in another, depending on the segmentation flag information). If the decoded segmentation flag is zero, the decoder can perform prediction on (e.g., the entire CU) (e.g., in the manner typically done in TIMD). The decoder can add the decoded residual to the prediction to reconstruct the CU.

[0212] In the fourth example, the encoder can allow block segmentation of a rectangular block using (e.g., only) a TMRL pattern. If the decoder decodes the TMRL flag to 1, the decoder can decode the segmentation flag information. If the decoded segmentation flag is non-zero, the decoder can segment the block. The decoder can perform prediction in two parts (e.g., using the TMRL pattern with the corresponding reference row in one part and the vertical / horizontal portion of the same reference row in another part, depending on the segmentation flag information). If the decoded segmentation flag is zero, the decoder can perform prediction on (e.g., the entire CU) (e.g., in the manner typically done in TMRL). The decoder can add the decoded residual to the prediction to reconstruct the CU.

[0213] In the fifth example, the encoder may allow block splitting with (e.g., any) a predicted pattern if that pattern is not equal to a vertical or horizontal pattern (e.g., depending on the block splitting configuration). To keep the additional encoder complexity to a minimum, the encoder can test the performance of the allowed splitting configuration by quickly traversing the qualified patterns of the shortened list. In this case, the encoder may encode a special flag (e.g., called splitFlag) to indicate whether the block prediction will (e.g., needs to) be split with both patterns (e.g., including the decoded pattern and a vertical or horizontal pattern, depending on the splitting direction).

[0214] Any combination(s) of the described examples can be used. The features(s) described in this article can improve compression efficiency.

[0215] The decoder can partition a video block into a first partition and a second partition, and predict the first partition using a horizontal or vertical intra-frame prediction mode (e.g., determined based on the partition type of the video block or based on an indicator), and predict the second partition using an intra-frame prediction mode determined to be suitable for the video block. Decoding tools and techniques include, for example, one or more of entropy decoding, inverse quantization, inverse transform, and differential decoding (e.g., ...). Figure 3(As shown) can be used to enable the implementation of one or more examples as described herein in a decoder. For example, these decoding tools and techniques can be used to enable one or more of the following: obtaining video blocks with unequal widths and heights; determining an intra-prediction mode applicable to the video blocks; determining the partition type of the video blocks based on the width and height of the video blocks; partitioning the video blocks into a first partition and a second partition based on the partition type, and determining that the first partition of the video blocks will be predicted using a horizontal or vertical intra-prediction mode; predicting the first partition of the video blocks using a horizontal or vertical intra-prediction mode, and predicting the second partition of the video blocks using an intra-prediction mode applicable to the video blocks (e.g., an intra-prediction mode applicable to the video blocks). The intra-prediction mode may not be horizontal or vertical (intra-prediction mode); the video block is decoded based on the first and second partitions of the video block; a partition indicator is received, which indicates that the video block (e.g., with unequal width and height) will be partitioned based on the width and height of the video block; a prediction scheme indicator is received, which indicates whether a vertical or horizontal intra-prediction mode will be used to predict the first partition of the video block; the first partition of the video block is predicted according to the prediction scheme indicator; the partition type of the video block determines whether a vertical or horizontal intra-prediction mode will be used to predict the first partition of the video block; according to... The process involves determining whether the vertical or horizontal intra-prediction mode will be used to predict the first partition of a video block; enabling one of TIMD, DIMD, MRL, MIP, MPM, or TMRL for the video block; determining the intra-prediction mode applicable to the video block based on the enabled TIMD, DIMD, MRL, MIP, MPM, or TMRL; receiving a partition indicator that indicates the video block (e.g., a video block with unequal width and height) will be partitioned based on the width and height of the video block; and determining whether only the vertical intra-prediction mode can be allowed for the first partition of the video block. / Or use a vertical intra-frame prediction mode to predict the first partition of a video block; symmetrically partition the video block into a first and second partition; asymmetrically partition the video block into a first and second partition; determine the prediction of the video block based on the prediction of the first and second partitions; decode the residual coefficients associated with the video block; apply the decoded residual coefficients to the prediction of the video block; determine the prediction of the video block as a simple average or weighted average of the predictions of the first and second partitions; determine the partition type of the video block as a horizontal partition type based on the fact that the height of the video block is greater than the width of the video block; etc.

[0216] The encoder can partition a video block into a first partition and a second partition, and predict the first partition using a horizontal or vertical intra-frame prediction mode (e.g., determined based on the partition type of the video block or the RD performance of the horizontal or vertical intra-frame prediction mode), and predict the second partition using an intra-frame prediction mode determined to be suitable for the video block. Encoding tools and techniques including one or more of quantization, entropy decoding, inverse quantization, inverse transform, and differential decoding (e.g., ... Figure 2(As shown) can be used to enable the implementation of one or more examples as described herein in an encoder. For example, these encoding tools and techniques can be used to enable one or more of the following: obtaining video blocks with unequal widths and heights; determining an intra-prediction mode applicable to the video blocks; determining the partition type of the video blocks based on the width and height of the video blocks; partitioning the video blocks into a first partition and a second partition based on the partition type, and determining that the first partition of the video blocks will be predicted using a horizontal or vertical intra-prediction mode; predicting the first partition of the video blocks using a horizontal or vertical intra-prediction mode, and predicting the second partition of the video blocks using an intra-prediction mode applicable to the video blocks (e.g., intra-prediction mode applicable to the video blocks). The mode may not be a horizontal or vertical intra-prediction mode); video blocks are encoded based on the first and second partitions of the video block; a partition indicator is sent, indicating that the video block (e.g., with unequal width and height) will be partitioned based on the width and height of the video block; a prediction scheme indicator is sent, indicating whether a vertical or horizontal intra-prediction mode will be used to predict the first partition of the video block; the first partition of the video block is predicted according to the prediction scheme indicator; the partition type of the video block determines whether a vertical or horizontal intra-prediction mode will be used to predict the first partition of the video block; the vertical intra-prediction mode... The process involves determining whether the horizontal intra-prediction mode will be used to predict the first partition of a video block; determining whether one of TIMD, DIMD, MRL, MIP, MPM, or TMRL is enabled for the video block; determining the intra-prediction mode applicable to the video block based on the enabled one of TIMD, DIMD, MRL, MIP, MPM, or TMRL; determining whether only the vertical intra-prediction mode can be allowed for the first partition of the video block, using the vertical intra-prediction mode to predict the first partition of the video block, and / or sending a partition indicator that indicates the video block (e.g., a video block with unequal width and height) will be predicted based on the video block's... The width and height of the video block are partitioned; the video block is symmetrically partitioned into a first partition and a second partition; the video block is asymmetrically partitioned into a first partition and a second partition; the prediction of the video block is determined based on the prediction of the first partition and the prediction of the second partition; the residual coefficients associated with the video block are sent; the prediction of the video block is determined as a simple average or weighted average of the predictions of the first partition and the second partition; the partitioning type of the video block is determined as a horizontal partitioning type based on the video block height being greater than the video block width; the partitioning type of the video block is determined as a vertical partitioning type based on the video block width being greater than the video block height.Evaluate the first prediction (e.g., prediction of a video block without partitions) for a second prediction of the video block (e.g., prediction of a first partition of the video block using a horizontal or vertical intra-frame prediction mode and prediction of a second partition of the video block using an intra-frame prediction mode applicable to the video block); determine that the first prediction will be used to encode the video block, and / or send a partition indicator indicating that the video block (e.g., having unequal widths and heights) will be partitioned based on the width and height of the video block; and so on.

[0217] Although the features and elements have been described above in specific combinations, those skilled in the art will understand that each feature or element can be used alone or in any combination with other features and elements. Furthermore, the methods described herein can be implemented as a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROMs and digital multifunction discs (DVDs). The processor associated with the software can be used to implement a radio frequency transceiver used in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. A device for video decoding, comprising: The processor is configured as follows: Obtain a video block where the width and height of the video block are not equal; Determine the intra-frame prediction mode applicable to the video block; The partition type of a video block is determined based on its width and height. Based on the partition type, the video block is divided into a first partition and a second partition of the video block, wherein the first partition of the video block will be predicted using either a horizontal or vertical intra-frame prediction mode. The first partition of the video block is predicted using a horizontal or vertical intra-prediction mode, and the second partition of the video block is predicted using an intra-prediction mode applicable to the video block, wherein the intra-prediction mode applicable to the video block is not a horizontal or vertical intra-prediction mode. as well as The video block is decoded based on the first and second partitions of the video block.

2. A device for video encoding, comprising: The processor is configured as follows: Obtain a video block where the width and height of the video block are not equal; Determine the intra-frame prediction mode applicable to the video block; The partition type of a video block is determined based on its width and height. Based on the partition type, the video block is divided into a first partition and a second partition of the video block, wherein the first partition of the video block will be predicted using either a horizontal or vertical intra-frame prediction mode. The first partition of the video block is predicted using a horizontal or vertical intra-prediction mode, and the second partition of the video block is predicted using an intra-prediction mode applicable to the video block, wherein the intra-prediction mode applicable to the video block is not a horizontal or vertical intra-prediction mode. as well as Video blocks are encoded based on the first and second partitions of the video block.

3. The device of claim 1, wherein the processor is further configured to: Receive a partition indicator, wherein the partition indicator indicates that video blocks with unequal widths and heights will be partitioned based on the width and height of the video blocks; and Receive a prediction scheme indicator, wherein the prediction scheme indicator indicates whether a vertical intra-frame prediction mode or a horizontal intra-frame prediction mode will be used to predict the first partition of a video block, wherein the first partition of the video block is predicted according to the prediction scheme indicator.

4. The device of claim 1 or claim 2, wherein the processor is further configured to determine, based on the partition type of the video block, whether a vertical intra-prediction mode or a horizontal intra-prediction mode will be used to predict the first partition of the video block, wherein the first partition of the video block is predicted according to the determination of whether a vertical intra-prediction mode or a horizontal intra-prediction mode will be used to predict the first partition of the video block.

5. The device according to claim 1 or claim 2, wherein the processor is further configured to: Determine whether one of the following is enabled for the video block: Template-based intra-prediction (TIMD), decoder-side intra-mode derivation (DIMD), multiple reference lines (MRL), matrix-based intra-prediction (MIP), most probable mode (MPM), or template-based multiple reference lines (TMRL), wherein the intra-prediction mode applicable to the video block is determined based on one of TIMD, DIMD, MRL, MIP, MPM, or TMRL being enabled.

6. The apparatus of claim 1, wherein the processor is further configured to receive a partition indicator, wherein the partition indicator indicates that video blocks having unequal widths and heights will be partitioned based on the width and height of the video blocks, wherein only vertical intra-frame prediction mode is allowed for the first partition of the video blocks, and the first partition of the video blocks uses the vertical intra-frame prediction mode for prediction.

7. The device of claim 1, wherein the video block is symmetrically partitioned into a first partition and a second partition of the video block, or asymmetrically partitioned into a first partition and a second partition of the video block, and wherein the processor is further configured to: The prediction of a video block is determined based on the prediction of the first partition and the prediction of the second partition of the video block; Decoding the residual coefficients associated with the video blocks; and The residual coefficients from the decoding are applied to the prediction of video blocks.

8. The device of claim 7, wherein the prediction of the video block is a simple average or weighted average of the predictions of the first partition of the video block and the predictions of the second partition of the video block.

9. The device of claim 1, wherein the partition type of the video block is determined as a horizontal partition type based on the height of the video block being greater than the width of the video block.

10. The apparatus of claim 2, wherein the prediction of a first partition of a video block using a horizontal or vertical intra-frame prediction mode and the prediction of a second partition of a video block using an intra-frame prediction mode suitable for the video block are first predictions, and wherein the processor is further configured to: Prediction evaluation of unpartitioned video blocks: First prediction; The first prediction is used to encode the video block; and Send a partition indicator, wherein the partition indicator indicates that video blocks with unequal widths and heights will be partitioned based on the width and height of the video blocks.

11. A method for video decoding, comprising: Obtain a video block where the width and height of the video block are not equal; Determine the intra-frame prediction mode applicable to the video block; The partition type of a video block is determined based on its width and height. Based on the partition type, the video block is divided into a first partition and a second partition of the video block, wherein the first partition of the video block will be predicted using either a horizontal or vertical intra-frame prediction mode. The first partition of the video block is predicted using a horizontal or vertical intra-prediction mode, and the second partition of the video block is predicted using an intra-prediction mode applicable to the video block, wherein the intra-prediction mode applicable to the video block is not a horizontal or vertical intra-prediction mode. as well as The video block is decoded based on the first and second partitions of the video block.

12. A method for video encoding, comprising: Obtain a video block where the width and height of the video block are not equal; Determine the intra-frame prediction mode applicable to the video block; The partition type of a video block is determined based on its width and height. Based on the partition type, the video block is divided into a first partition and a second partition of the video block, wherein the first partition of the video block will be predicted using either a horizontal or vertical intra-frame prediction mode. The first partition of the video block is predicted using a horizontal or vertical intra-prediction mode, and the second partition of the video block is predicted using an intra-prediction mode applicable to the video block, wherein the intra-prediction mode applicable to the video block is not a horizontal or vertical intra-prediction mode. as well as Video blocks are encoded based on the first and second partitions of the video block.

13. The method of claim 11, further comprising: Receive a partition indicator, wherein the partition indicator indicates that video blocks with unequal widths and heights will be partitioned based on the width and height of the video blocks; as well as Receive a prediction scheme indicator, wherein the prediction scheme indicator indicates whether a vertical intra-frame prediction mode or a horizontal intra-frame prediction mode will be used to predict the first partition of a video block, wherein the first partition of the video block is predicted according to the prediction scheme indicator.

14. The method of claim 11 or claim 12, wherein the method further comprises determining, based on the partition type of the video block, whether a vertical intra-prediction mode or a horizontal intra-prediction mode will be used to predict a first partition of the video block, wherein the first partition of the video block is predicted according to the determination of whether a vertical intra-prediction mode or a horizontal intra-prediction mode will be used to predict the first partition of the video block.

15. The method of claim 11 or claim 12, wherein the method further comprises determining that one of the following is enabled for a video block: template-based intra-prediction (TIMD), decoder-side intra-mode derivation (DIMD), multiple reference lines (MRL), matrix-based intra-prediction (MIP), most probable mode (MPM), template-based multiple reference lines (TMRL), wherein the intra-prediction mode applicable to the video block is determined based on one of TIMD, DIMD, MRL, MIP, MPM, or TMRL being enabled.