Spatial geometric partition mode

Abstract

Systems, methods, and instrumentalities associated with spatial geometric partition patterns (SGPMs) are disclosed. In an example, a device (e.g., a video encoder and / or a video decoder) may derive a first candidate intra prediction mode (IPM) based on a first template of a coded block. The device may derive a second candidate IPM based on a second template of the coded block. The device may process the coding block based on the first candidate IPM and the second candidate IPM.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of European Provisional Patent Application No. 22306007.0 filed on July 5, 2022, the contents of which are hereby incorporated herein by reference. Background Art

[0003] Video coding systems may be used to compress digital video signals, for example, to reduce the storage and / or transmission bandwidth required for such signals. Video coding systems may include, for example, block-based, wavelet-based, and / or object-based systems. Summary of the invention

[0004] Systems, methods, and means associated with a spatial geometric partitioning mode (SGPM) are disclosed. In an example, a video decoding device may derive a first candidate intra prediction mode (IPM) based on a first template of a coding block. The device may derive a second candidate IPM based on a second template of the coding block. The device may decode the coding block based on the first candidate IPM and the second candidate IPM. The second template may be different from the first template.

[0005] The apparatus may add the first candidate IPM and the second candidate IPM to an IPM candidate list associated with the coding block; and may decode the coding block based on the IPM candidate list.

[0006] The device may select a third candidate IPM based on the partition mode of the coding block. The device may add the first candidate IPM, the second candidate IPM, and the third candidate IPM to an IPM candidate list associated with the coding block. The coding block may be decoded based on the IPM candidate list.

[0007] The coding block may include a first partition and a second partition. The device may generate an IPM candidate list associated with the coding block based on the first candidate IPM and the second candidate IPM. The device may determine a first prediction mode for the first partition and a second prediction mode for the second partition based on a spatial geometric partition mode (SGPM) index and the IPM candidate list.

[0008] The device may determine that the SGPM is used for the coding block. Based on determining that the SGPM is used for the coding block, the device may obtain the first template and the second template of the coding block for IPM derivation. The second template may be different from the first template.

[0009] The device may determine that an SGPM is used for the coding block. Based on determining that the SGPM is used for the coding block, the device may obtain the first template and the second template of the coding block for IPM derivation. The device may sort candidates in the SGPM candidate list associated with the coding block based on a third template of the coding block. The third template of the coding block may include the first template and the second template.

[0010] The apparatus may determine the first template and the second template for the coding block based on a partition mode associated with the coding block.

[0011] The coding block may include a first partition and a second partition. The device may determine the first template of the coding block based on the first partition. The device may decode the first partition of the coding block based on at least the first candidate IPM derived based on the first template. The device may determine the second template of the coding block based on the second partition. The device may decode the second partition of the coding block based on at least the second candidate IPM derived based on the second template.

[0012] The device may obtain a left template of the decoding block as the first template. The device may obtain a top template of the decoding block as the second template. The device may obtain a left template with a width of 1 of the decoding block as the first template. The device may obtain a top template with a height of 1 of the decoding block as the second template.

[0013] When deriving the first candidate IPM based on the first template, the device may obtain a most probable mode (MPM) associated with the coding block. The device may determine a prediction of the first template corresponding to the MPM. The device may select the first candidate IPM from the MPM based on comparing the first template with the prediction corresponding to the most probable mode.

[0014] Systems, methods, and means associated with a spatial geometric partitioning mode (SGPM) are disclosed. In an example, a video encoding device may derive a first candidate intra prediction mode (IPM) based on a first template of a coding block. The device may derive a second candidate IPM based on a second template of the coding block. The device may encode the coding block based on the first candidate IPM and the second candidate IPM.

[0015] The apparatus may add the first candidate IPM and the second candidate IPM to an IPM candidate list associated with the coding block; and may encode the coding block based on the IPM candidate list.

[0016] The device may select a third candidate IPM based on the partition mode of the coding block. The device may add the first candidate IPM, the second candidate IPM, and the third candidate IPM to an IPM candidate list associated with the coding block. The coding block may be encoded based on the IPM candidate list.

[0017] The coding block may include a first partition and a second partition. The device may determine a first prediction mode for the first partition and a second prediction mode for the second partition. The device may generate an IPM candidate list associated with the coding block based on the first candidate IPM and the second candidate IPM. The device may determine a special geometric partition mode (SGPM) index based on the first prediction mode and the second prediction mode. The device may include the index in the IPM candidate list. The device may determine that the SGPM is used for the coding block. Based on determining that the SGPM is used for the coding block, the device may obtain the first template and the second template of the coding block for IPM derivation. The second template may be different from the first template.

[0018] The device may determine that an SGPM is used for the coding block. Based on determining that the SGPM is used for the coding block, the device may obtain the first template and the second template of the coding block for IPM derivation. The device may obtain a third template of the coding block including the first template and the second template. The device may sort candidates in an SGPM candidate list associated with the coding block based on the third template of the coding block. The device may determine an SGPM index based on the sorted candidates.

[0019] The apparatus may determine the first template and the second template for the coding block based on a partition mode associated with the coding block.

[0020] The coding block may include a first partition and a second partition. The device may determine the first template of the coding block based on the first partition. The device may encode the first partition of the coding block based on at least the first candidate IPM derived based on the first template. The device may determine the second template of the coding block based on the second partition. The device may encode the second partition of the coding block based on at least the second candidate IPM derived based on the second template.

[0021] When deriving the first candidate IPM based on the first template, the device may obtain a most probable mode (MPM) associated with the coding block. The device may determine a prediction of the first template corresponding to the MPM. The device may select the first candidate IPM from the MPM based on comparing the first template with the prediction corresponding to the most probable mode.

[0022] The device may select a third candidate IPM based on a partition mode of the coding block. The device may sort the combination of the first candidate IPM, the second candidate IPM, and / or the third candidate IPM in ascending order based on a sum of absolute errors (SAD) between a prediction of the coding block and a reconstruction of the coding block.

[0023] Systems, methods, and means associated with a spatial geometric partitioning mode (SGPM) are disclosed. In an example, a device may obtain characteristics associated with a current block. Based on one or more of the characteristics, an intra-prediction mode candidate list size may be determined for the SGPM associated with the current block. An intra-prediction mode candidate list for the current block may be obtained based on the determined intra-prediction mode candidate list size. In an example, an order of intra-prediction mode candidate types may be determined based on the characteristics. The intra-prediction mode candidate list may be obtained based on the determined order.

[0024] In an example, a device may obtain characteristics associated with a current block. Based on one or more of the characteristics, a partition mode candidate list size may be determined for an SGPM associated with the current block. A partition mode candidate list for the current block may be obtained based on the determined partition mode candidate list size. In an example, a selection of a partition mode candidate may be determined based on the characteristics.

[0025] In an example, a device may obtain characteristics associated with a current block. Based on one or more of the characteristics, a SGPM candidate list size associated with the current block may be determined. An SGPM candidate list associated with the current block may be obtained based on the determined SGPM candidate list size. The SGPM candidate list may be used to decode the current block. The characteristics may include the following: frame resolution, slice type, block size, and / or block shape.

[0026] The systems, methods, and means described herein may relate to decoders. In some examples, the systems, methods, and means described herein may relate to encoders. In some examples, the systems, methods, and means described herein may relate to signals (e.g., signals from encoders and / or received by decoders). A computer-readable medium may include instructions for causing one or more processors to perform the methods described herein. A computer program product may include instructions that, when executed by one or more processors, cause the one or more processors to perform the methods described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1A is a system diagram illustrating an example communication system in which one or more disclosed embodiments may be implemented.

[0028] Figure 1Bis an example of a method that can be used according to an embodiment of the present invention. Figure 1A A system diagram of an example wireless transmit / receive unit (WTRU) for use within an illustrated communication system.

[0029] Figure 1C is an example of a method that can be used according to an embodiment of the present invention. Figure 1A System diagram of an example Radio Access Network (RAN) and an example Core Network (CN) for use within an illustrated communication system.

[0030] Figure 1D is an example of a method that can be used according to an embodiment of the present invention. Figure 1A System diagram of a further example RAN and a further example CN for use within the illustrated communication system.

[0031] Figure 2 An example video encoder is illustrated.

[0032] Figure 3 An example video decoder is illustrated.

[0033] Figure 4 Examples of systems in which various aspects and examples may be implemented are illustrated.

[0034] Figure 5 An example of directional intra prediction with reference to neighboring samples is shown.

[0035] Figure 6 An example of intra-mode coding associated with 67 intra-prediction modes (IPMs) is shown.

[0036] Figure 7 An example of a current W×H block to be predicted using wide-angle intra prediction is shown.

[0037] Figure 8 An example of matrix-weighted intra prediction (MIP) is shown.

[0038] Fig. 9 An example of decoder-side intra prediction mode derivation (DIMD) is shown.

[0039] Fig.10 An example of an IPM that derives DIMD from a gradient in a template is shown.

[0040] Fig.11 An example of a template for deriving a fused IPM for template-based intra mode derivation (TIMD) is shown.

[0041] Fig.12 An example of 4 reference lines associated with a multiple reference line (MRL) is shown.

[0042] Fig.13An example of intra-frame sub-partitioning (ISP) is shown.

[0043] Fig.14 An example of a Most Probable Mode (MPM) list is shown.

[0044] Fig.15 An example of signaling, at the encoder side, the intra prediction mode selected for predicting the current CU is shown.

[0045] Fig.16 An example of 64 partitions for a geometric partitioning mode (GPM) is shown.

[0046] Fig.17 Example blending in GPM is shown.

[0047] Fig.18 An example of GPM intra mode is shown.

[0048] Fig.19 An example of non-rectangular partitions in inter prediction for a portion of a picture is shown.

[0049] Fig. 20 An example of a piecewise smooth image model is shown.

[0050] Fig.21 A shows an example of a Space Geometric Partitioning Pattern (SGPM).

[0051] Fig.21 B shows an example of a partition mode and two IPMs associated with a SGPM signaled in the video bitstream.

[0052] Fig.21 C shows the partition mode associated with the SGPM signaled in the video bitstream and the derived combination of two IPMs.

[0053] Fig. 22 An example of a prediction obtained for a template using the partition weights extended to the template is shown.

[0054] Fig.23 An example of a flow chart of the SGPM is shown.

[0055] Fig.24 An example of signaling, at the encoder side, the intra prediction mode selected for predicting the current CU is shown, including the addition of SGPM.

[0056] Fig.25 An example of the derivation of possible IPM candidates for SGPM is shown.

[0057] Fig.26 Examples of possible partitioning mode candidates selected for SGPM are shown.

[0058] Fig. 27 Examples of selected partitioning pattern candidates for the proposed reduction of SGPM are shown.

[0059] Fig.28 An example of examining and sorting possible combinations to derive a SGPM candidate list is shown.

[0060] Fig.29 An example of determining the size of the IPM candidate list based on characteristics is shown.

[0061] Fig.30 An example of determining the size of a partition candidate list based on characteristics is shown.

[0062] Fig.31 An example of determining the size of the IPM candidate list based on characteristics is shown.

[0063] Fig.32 An example of determining the size of a partition candidate list based on characteristics is shown. DETAILED DESCRIPTION

[0064] A more detailed understanding may be obtained from the following description given by way of example in conjunction with the accompanying drawings.

[0065] Figure 1A 1 is a diagram illustrating an example communication system 100 in which one or more disclosed embodiments may be implemented. The communication system 100 may be a multiple access system that provides content such as voice, data, video, messaging, broadcast, etc. to multiple wireless users. The communication system 100 may enable multiple wireless users to access such content through sharing of system resources (including wireless bandwidth). For example, the communication system 100 may employ one or more channel access methods such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single carrier FDMA (SC-FDMA), zero tail unique word DFT spread OFDM (ZT UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block filtered OFDM, and filter bank multi-carrier (FBMC), etc.

[0066] like Figure 1AAs shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, public switched telephone network (PSTN) 108, Internet 110 and other networks 112, but it should be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, 102d (any of which may be referred to as a “station” and / or “STA”) may be configured to send and / or receive wireless signals and may include user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a subscription-based unit, a pager, a cellular phone, a personal digital assistant (PDA), a smart phone, a laptop, a netbook, a personal computer, a wireless sensor, a hotspot or Mi-Fi device, an Internet of Things (IoT) device, a watch or other wearable device, a head-mounted display (HMD), a vehicle, a drone, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in an industrial and / or automated process chain environment), a consumer electronic device, a device operating on a commercial and / or industrial wireless network, etc. Any of the WTRUs 102a, 102b, 102c, and 102d may be interchangeably referred to as a UE.

[0067] The communication system 100 may also include a base station 114a and / or a base station 114b. Each of the base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks, such as the CN 106 / 115, the Internet 110, and / or other networks 112. By way of example, the base stations 114a, 114b may be a base transceiver station (BTS), a Node B, an eNode B, a Home Node B, a Home eNode B, a gNB, an NR Node B, a site controller, an access point (AP), a wireless router, and the like. Although the base stations 114a, 114b are each depicted as a single element, it should be appreciated that the base stations 114a, 114b may include any number of interconnected base stations and / or network elements.

[0068] Base station 114a may be part of RAN 104 / 113, which may also include other base stations and / or network elements (not shown), such as base station controllers (BSC), radio network controllers (RNC), relay nodes, etc. Base station 114a and / or base station 114b may be configured to send and / or receive wireless signals on one or more carrier frequencies (which may be referred to as cells (not shown)). These frequencies may be in a licensed spectrum, an unlicensed spectrum, or a combination of a licensed spectrum and an unlicensed spectrum. A cell may provide coverage of wireless services to a specific geographic area, which may be relatively fixed or may change over time. A cell may be further divided into cell sectors. For example, a cell associated with base station 114a may be divided into three sectors. Therefore, in one embodiment, base station 114a may include three transceivers, i.e., one transceiver for each sector of the cell. In an embodiment, base station 114a may employ multiple-input multiple-output (MIMO) technology and may utilize multiple transceivers for each sector of the cell. For example, beamforming may be used to send and / or receive signals in a desired spatial direction.

[0069] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0070] More specifically, as noted above, the communication system 100 may be a multiple access system and may employ one or more channel access schemes such as CDMA, TDMA, FDMA, OFDMA, and SC-FDMA. For example, the base station 114a in the RAN 104 / 113 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may use Wideband CDMA (WCDMA) to establish the air interface 115 / 116 / 117. WCDMA may include communication protocols such as High Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High Speed ​​UL Packet Access (HSUPA).

[0071] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA) that may establish the air interface 116 using Long Term Evolution (LTE) and / or Advanced LTE (LTE-A) and / or Advanced LTE Pro (LTE-A Pro).

[0072] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as NR radio access, which may establish the air interface 116 using New Radio (NR).

[0073] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may together implement LTE radio access and NR radio access, for example using the dual connectivity (DC) principle. Thus, the air interface utilized by the WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions transmitted to / from multiple types of base stations (e.g., eNBs and gNBs).

[0074] In other embodiments, the base station 114a and the WTRUs 102a, 102b, 102c may implement radio technologies such as IEEE 802.11 (i.e., Wireless Fidelity (WiFi)), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile Communications (GSM), GSM Enhanced Data Rates for Evolution (EDGE), and GSM EDGE (GERAN).

[0075] Figure 1AThe base station 114b in may be, for example, a wireless router, a Home NodeB, a Home eNodeB, or an access point, and may utilize any suitable RAT to facilitate wireless connectivity in local areas such as commercial locations, homes, vehicles, campuses, industrial facilities, sky corridors (e.g., for use by drones), and roads. In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement radio technologies such as IEEE 802.11 to establish a wireless local area network (WLAN). In an embodiment, the base station 114b and the WTRUs 102c, 102d may implement radio technologies such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, the base station 114b and the WTRUs 102c, 102d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish a picocell or a femtocell. As Figure 1A As shown, base station 114b may have a direct connection to Internet 110. Therefore, base station 114b may not need to access Internet 110 via CN 106 / 115.

[0076] The RAN 104 / 113 may be in communication with the CN 106 / 115, which may be any type of network configured to provide voice, data, applications and / or Voice over Internet Protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have different quality of service (QoS) requirements, such as different throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. The CN 106 / 115 may provide call control, billing services, mobile location-based services, prepaid calling, Internet connectivity, video distribution, etc., and / or perform high-level security functions, such as user authentication. Although not described in detail in the text and video, the CN 106 / 115 may be configured to provide voice, data, applications and / or Voice over Internet Protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have different quality of service (QoS) requirements, such as different throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. The CN 106 / 115 may provide call control, billing services, mobile location-based services, prepaid calling, Internet connectivity, video distribution, etc., and / or perform high-level security functions, such as user authentication. Figure 1A Although not shown in the figure, it will be appreciated that the RAN 104 / 113 and / or CN 106 / 115 may be in direct or indirect communication with other RANs that employ the same RAT as the RAN 104 / 113 or a different RAT. For example, in addition to being connected to the RAN 104 / 113, which may utilize NR radio technology, the CN 106 / 115 may also be in communication with another RAN (not shown) that employs GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technology.

[0077] The CN 106 / 115 may also act as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and / or other networks 112. The PSTN 108 may include a circuit-switched telephone network that provides plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the Transmission Control Protocol (TCP), the User Datagram Protocol (UDP), and / or the Internet Protocol (IP) in the TCP / IP Internet protocol suite. The networks 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, the networks 112 may include another CN connected to one or more RANs, which may employ the same RAT as the RAN 104 / 113 or a different RAT.

[0078] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communication system 100 may include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks via different wireless links). Figure 1A The illustrated WTRU 102c may be configured to communicate with the base station 114a, which may employ a cellular-based radio technology, and with the base station 114b, which may employ an IEEE 802 radio technology.

[0079] Figure 1B is a system diagram illustrating an example WTRU 102. Figure 1B As shown, the WTRU 102 may include, among other things, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and / or other peripherals 138. It will be appreciated that the WTRU 102 may include any sub-combination of the foregoing elements while remaining consistent with an embodiment.

[0080] The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC) and state machine, etc. The processor 118 may perform signal decoding, data processing, power control, input / output processing, and / or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit / receive element 122. Although Figure 1B The processor 118 and the transceiver 120 are depicted as separate components, but it is understood that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.

[0081] The send / receive element 122 may be configured to send a signal to a base station (e.g., base station 114a) or receive a signal from the base station via an air interface 116. For example, in one embodiment, the send / receive element 122 may be an antenna configured to send and / or receive an RF signal. In an embodiment, the send / receive element 122 may be a transmitter / detector configured to send and / or receive, for example, IR, UV or visible light signals. In another embodiment, the send / receive element 122 may be configured to send and / or receive both RF signals and optical signals. It should be understood that the send / receive element 122 may be configured to send and / or receive any combination of wireless signals.

[0082] Although the transmit / receive element 122 Figure 1B Although depicted as a single element in the figure, the WTRU 102 may include any number of transmit / receive elements 122. More specifically, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.

[0083] The transceiver 120 may be configured to modulate signals to be transmitted by the transmit / receive element 122 and to demodulate signals received by the transmit / receive element 122. As noted above, the WTRU 102 may have multi-mode capabilities. For example, the transceiver 120 may include multiple transceivers for enabling the WTRU 102 to communicate via multiple RATs, such as NR and IEEE 802.11.

[0084] The processor 118 of the WTRU 102 may be coupled to a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light emitting diode (OLED) display unit) and may receive user input data from these devices. The processor 118 may also output user data to the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128. In addition, the processor 118 may access information from and store data in any type of suitable memory, such as a non-removable memory 130 and / or a removable memory 132. The non-removable memory 130 may include a random access memory (RAM), a read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from, and store data in, memory that is not physically located on the WTRU 102, such as on a server or a home computer (not shown).

[0085] The processor 118 may receive power from the power source 134 and may be configured to distribute and / or control power to the other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0086] The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to or in lieu of the information from the GPS chipset 136, the WTRU 102 may receive location information from a base station (e.g., base stations 114a, 114b) over the air interface 116 and / or determine its location based on the timing of signals received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information by any suitable location-determination method while remaining consistent with an embodiment.

[0087] The processor 118 may be further coupled to other peripherals 138, which may include one or more software modules and / or hardware modules that provide additional features, functionality, and / or wired or wireless connectivity. For example, the peripherals 138 may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (for photos and / or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, module, a frequency modulation (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, a virtual reality and / or augmented reality (VR / AR) device, and an activity tracker, etc. The peripheral device 138 may include one or more sensors, which may be one or more of the following: a gyroscope, an accelerometer, a Hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor; a geolocation sensor; an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, and / or a humidity sensor.

[0088] The WTRU 102 may include a full-duplex radio for which transmission and reception of some or all of the signals (e.g., associated with specific subframes for UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous. The full-duplex radio may include an interference management unit for reducing and or substantially eliminating self-interference via signal processing performed via hardware (e.g., a choke) or via a processor (e.g., a separate processor (not shown) or via processor 118). In an embodiment, the WRTU 102 may include a half-duplex radio for which transmission and reception of some or all of the signals (e.g., associated with specific subframes for UL (e.g., for transmission) or downlink (e.g., for reception)) may be concurrent and / or simultaneous.

[0089] Figure 1C 1 is a system diagram illustrating the RAN 104 and the CN 106 in accordance with an embodiment. As noted above, the RAN 104 may employ an E-UTRA radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 104 may also be in communication with the CN 106.

[0090] The RAN 104 may include evolved Node-Bs 160a, 160b, 160c, though it will be appreciated that the RAN 104 may include any number of evolved Node-Bs while remaining consistent with an embodiment. The evolved Node-Bs 160a, 160b, 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the evolved Node-Bs 160a, 160b, 160c may implement MIMO technology. Thus, the evolved Node-B 160a, for example, may use multiple antennas to transmit wireless signals to and / or receive wireless signals from the WTRU 102a.

[0091] Each of the evolved Node Bs 160a, 160b, 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, and scheduling of users in the UL and / or DL, among other things. Figure 1C As shown, the eNode-Bs 160a, 160b, 160c may communicate with one another via an X2 interface.

[0092] Figure 1C The illustrated CN 106 may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (or PGW) 166. While each of the foregoing elements is depicted as being part of the CN 106, it should be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0093] The MME 162 may be connected to each of the evolved Node-Bs 162a, 162b, 162c in the RAN 104 via an S1 interface and may serve as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, bearer activation / deactivation, and selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, 102c. The MME 162 may provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM and / or WCDMA.

[0094] The SGW 164 may be connected to each of the evolved Node-Bs 160a, 160b, 160c in the RAN 104 via an S1 interface. The SGW 164 may generally route and forward user data packets to / from the WTRUs 102a, 102b, 102c. The SGW 164 may perform other functions, such as anchoring the user plane during an inter-evolved Node-B handover, triggering paging when DL data is available for the WTRUs 102a, 102b, 102c, and managing and storing the context of the WTRUs 102a, 102b, 102c.

[0095] The SGW 164 may be connected to the PGW 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0096] The CN 106 may facilitate communications with other networks. For example, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the CN 106 may include, or may be in communication with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that acts as an interface between the CN 106 and the PSTN 108. In addition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0097] Although the WTRU Figures 1A to 1D Although described as wireless terminals, it is contemplated that in certain representative embodiments, such terminals may (eg, temporarily or permanently) use a wired communications interface with a communications network.

[0098] In a representative embodiment, the other network 112 may be a WLAN.

[0099] A WLAN in infrastructure basic service set (BSS) mode may have an access point (AP) for a BSS and one or more stations (STAs) associated with the AP. The AP may have access or an interface to a distribution system (DS) or another type of wired / wireless network that carries traffic to and / or out of the BSS. Traffic from outside the BSS to a STA may be reached by the AP and may be delivered to the STA. Traffic from a STA to a destination outside the BSS may be transmitted to the AP to be delivered to the corresponding destination. Traffic between STAs within the BSS may be transmitted by the AP, for example, where the source STA may transmit traffic to the AP, and the AP may deliver traffic to the destination STA. Traffic between STAs within the BSS may be considered and / or referred to as point-to-point traffic. Direct link establishment (DLS) may be used to transmit point-to-point traffic between a source STA and a destination STA (e.g., directly between them). In certain representative embodiments, the DLS may use 802.11e DLS or 802.11z tunnel DLS (TDLS). A WLAN using an independent BSS (IBSS) mode may not have an AP, and STAs within or using the IBSS (eg, all STAs in the STA) may communicate directly with each other. The IBSS communication mode may sometimes be referred to herein as an "ad hoc" communication mode.

[0100] When using the 802.11ac infrastructure operating mode or a similar operating mode, the AP may send beacons on a fixed channel, such as a primary channel. The primary channel may be a fixed width (e.g., a 20 MHz wide bandwidth) or a width dynamically set via signaling. The primary channel may be an operating channel of the BSS and may be used by the STA to establish a connection with the AP. In certain representative embodiments, carrier sense multiple access / collision avoidance (CSMA / CA) may be implemented, for example, in an 802.11 system. For CSMA / CA, a STA (e.g., each STA) (including the AP) may sense the primary channel. If the primary channel is sensed / detected and / or determined to be busy by a particular STA, the particular STA may back off. One STA (e.g., only one station) may transmit at any given time in a given BSS.

[0101] A high throughput (HT) STA may communicate using a 40 MHz wide channel, for example, via a combination of a 20 MHz primary channel and adjacent or non-adjacent 20 MHz channels to form a 40 MHz wide channel.

[0102] Very high throughput (VHT) STA can support 20MHz, 40MHz, 80MHz and / or 160MHz wide channels. 40MHz channels and / or 80MHz channels can be formed by combining continuous 20MHz channels. 160MHz channels can be formed by combining 8 continuous 20MHz channels, or by combining two non-continuous 80MHz channels (this can be called 80+80 configuration). For 80+80 configuration, after channel coding, the data can pass through a segment parser that can divide the data into two streams. Each stream can be processed by inverse fast Fourier transform (IFFT) and time domain processing separately. These streams can be mapped to two 80MHz channels, and data can be sent by the transmitter STA. At the receiver of the receiver STA, the above-mentioned operation for the 80+80 configuration can be reversed, and the combined data can be transmitted to the medium access control (MAC).

[0103] 802.11af and 802.11ah support operating modes below 1GHz. Channel operating bandwidths and carriers are reduced in 802.11af and 802.11ah relative to those used in 802.11n and 802.11ac. 802.11af supports 5MHz, 10MHz, and 20MHz bandwidths in the TV White Space (TVWS) spectrum, and 802.11ah supports 1MHz, 2MHz, 4MHz, 8MHz, and 16MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11ah may support instrument type control / machine type communications, such as MTC devices in macro coverage areas. MTC devices may have certain capabilities, such as limited capabilities, including support for (e.g., only support for) certain bandwidths and / or limited bandwidths. MTC devices may include batteries with battery life above a threshold (e.g., to maintain very long battery life).

[0104] WLAN systems that can support multiple channels and channel bandwidths (such as 802.11n, 802.11ac, 802.11af, and 802.11ah) include channels that can be designated as primary channels. The primary channel may have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel may be set and / or limited by a STA (which supports the minimum bandwidth operating mode) from all STAs operating in the BSS. In the example of 802.11ah, for STAs (e.g., MTC-type devices) that support (e.g., only support) a 1MHz mode, the primary channel may be 1MHz wide, even if the AP and other STAs in the BSS support 2MHz, 4MHz, 8MHz, 16MHz, and / or other channel bandwidth operating modes. Carrier sensing and / or network allocation vector (NAV) settings may depend on the state of the primary channel. If the primary channel is busy, for example, because a STA (which only supports a 1MHz operating mode) is sending to the AP, the entire available band may be considered busy even if most of the band remains idle and may be available.

[0105] In the United States, the available frequency band for 802.11ah is 902MHz to 928MHz. In South Korea, the available frequency band is 917.5MHz to 923.5MHz. In Japan, the available frequency band is 916.5MHz to 927.5MHz. The total bandwidth available for 802.11ah is 6MHz to 26MHz, depending on the country code.

[0106] Figure 1D 1 is a system diagram illustrating the RAN 113 and the CN 115 in accordance with an embodiment. As noted above, the RAN 113 may employ NR radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 113 may also be in communication with the CN 115.

[0107] The RAN 113 may include gNBs 180a, 180b, 180c, though it will be appreciated that the RAN 113 may include any number of gNBs while remaining consistent with an embodiment. The gNBs 180a, 180b, 180c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the gNBs 180a, 180b, 180c may implement MIMO technology. For example, the gNBs 180a, 180b may utilize beamforming to send signals to and / or receive signals from the gNBs 180a, 180b, 180c. Thus, the gNB 180a may, for example, use multiple antennas to send wireless signals to and / or receive wireless signals from the WTRU 102a. In an embodiment, the gNBs 180a, 180b, 180c may implement carrier aggregation technology. For example, the gNB 180a may transmit multiple component carriers to the WTRU 102a (not shown). A subset of these component carriers may be on an unlicensed spectrum, while the remaining component carriers may be on a licensed spectrum. In an embodiment, the gNBs 180a, 180b, 180c may implement coordinated multi-point (CoMP) technology. For example, the WTRU 102a may receive coordinated transmissions from the gNB 180a and the gNB 180b (and / or the gNB 180c).

[0108] The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using transmissions associated with parameter sets that may be scalable. For example, OFDM symbol spacing and / or OFDM subcarrier spacing may vary for different transmissions, different cells, and / or different portions of the wireless transmission spectrum. The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using subframes or transmit time intervals (TTIs) of varying or scalable lengths (e.g., containing different numbers of OFDM symbols and / or varying absolute time lengths over time).

[0109] The gNBs 180a, 180b, 180c may be configured to communicate with the WTRUs 102a, 102b, 102c in a standalone configuration and / or a non-standalone configuration. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c without accessing other RANs (e.g., such as the eNodeBs 160a, 160b, 160c). In a standalone configuration, the WTRUs 102a, 102b, 102c may use one or more of the gNBs 180a, 180b, 180c as mobility anchors. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using signals in an unlicensed band. In a non-standalone configuration, the WTRUs 102a, 102b, 102c may communicate / connect with the gNB 180a, 180b, 180c while also communicating / connecting with another RAN, such as the eNodeB 160a, 160b, 160c. For example, the WTRUs 102a, 102b, 102c may implement the DC principle to communicate with one or more gNBs 180a, 180b, 180c and one or more eNodeBs 160a, 160b, 160c substantially simultaneously. In a non-standalone configuration, the eNodeB 160a, 160b, 160c may act as a mobility anchor for the WTRUs 102a, 102b, 102c, and the gNB 180a, 180b, 180c may provide additional coverage and / or throughput for serving the WTRUs 102a, 102b, 102c.

[0110] Each of the gNBs 180a, 180b, 180c may be associated with a specific cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in UL and / or DL, support of network slicing, dual connectivity, interworking between NR and E-UTRA, routing of user plane data towards a user plane function (UPF) 184a, 184b and routing of control plane information towards an access and mobility management function (AMF) 182a, 182b, etc. Figure 1D As shown, gNBs 180a, 180b, and 180c may communicate with each other via an Xn interface.

[0111] Figure 1DThe illustrated CN 115 may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one session management function (SMF) 183a, 183b, and possible data networks (DNs) 185a, 185b. Although each of the foregoing elements is depicted as part of the CN 115, it should be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0112] The AMF 182a, 182b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via the N2 interface and may act as a control node. For example, the AMF 182a, 182b may be responsible for authenticating users of the WTRU 102a, 102b, 102c, support of network slicing (e.g., handling of different PDU sessions with different requirements), selecting a specific SMF 183a, 183b, management of registration areas, termination of NAS signaling and mobility management, etc. The AMF 182a, 182b may use network slicing in order to customize CN support for the WTRU 102a, 102b, 102c based on the type of services utilized by the WTRU 102a, 102b, 102c. For example, different network slices may be established for different use cases, such as services relying on ultra-reliable low latency (URLLC) access, services relying on enhanced mobile broadband (eMBB) access, and / or services for machine type communication (MTC) access, etc. The AMF 162 may provide a control plane function for switching between the RAN 113 and other RANs (not shown) that employ other radio technologies, such as LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies such as WiFi.

[0113] The SMF 183a, 183b may be connected to the AMF 182a, 182b in the CN 115 via the N11 interface. The SMF 183a, 183b may also be connected to the UPF 184a, 184b in the CN 115 via the N4 interface. The SMF 183a, 183b may select and control the UPF 184a, 184b, and configure the service routing performed by the UPF 184a, 184b. The SMF 183a, 183b may perform other functions, such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy implementation and QoS, and providing downlink data notifications. The PDU session type may be IP-based, non-IP-based, Ethernet-based, etc.

[0114] The UPF 184a, 184b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via the N3 interface, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, for example, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The UPF 184, 184b may perform other functions, such as routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering downlink packets, and providing mobility anchoring, etc.

[0115] The CN 115 may facilitate communications with other networks. For example, the CN 115 may include or may communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between the CN 115 and the PSTN 108. In addition, the CN 115 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, the WTRUs 102a, 102b, 102c may be connected to local data networks (DNs) 185a, 185b via the UPF 184a, 184b via an N3 interface to the UPF 184a, 184b and an N6 interface between the UPF 184a, 184b and the DN 185a, 185b.

[0116] Given that Figures 1A to 1D as well as Figures 1A to 1D Corresponding to the description of the present invention, one or more or all of the functions described herein for one or more of the following may be performed by one or more simulation devices (not shown): WTRU102a-102d, base station 114a-114b, evolved Node B 160a-160c, MME 162, SGW 164, PGW 166, gNB 180a-180c, AMF 182a-182b, UPF 184a-184b, SMF 183a-183b, DN 185a-185b and / or any other device described herein. The simulation device may be one or more devices configured to mimic one or more or all of the functions described herein. For example, the simulation device may be used to test other devices and / or simulate network and / or WTRU functions.

[0117] The simulation device may be designed to implement one or more tests of other devices in a laboratory environment and / or in an operator network environment. For example, one or more simulation devices may perform one or more functions or all functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network in order to test other devices within the communication network. One or more simulation devices may perform one or more functions or all functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. The simulation device may be directly coupled to another device for testing purposes and / or may use over-the-air wireless communications to perform testing.

[0118] One or more simulation devices can perform one or more (including all) functions without being implemented / deployed as part of a wired and / or wireless communication network. For example, the simulation device can be used in a test scenario in a test laboratory and / or a non-deployed (e.g., testing) wired and / or wireless communication network to implement testing of one or more components. One or more simulation devices can be test equipment. Direct RF coupling and / or wireless communication via RF circuitry (e.g., which can include one or more antennas) can be used by the simulation device to send and / or receive data.

[0119] The present application describes various aspects, including tools, features, examples, models, methods, etc. Many aspects of these aspects are specifically described, and at least in order to illustrate individual characteristics, these many aspects are usually described in a manner that may sound limited. However, this is to describe clearly and does not limit the application or scope of these aspects. In fact, all different aspects can be combined and interchanged to provide further aspects. In addition, these aspects can also be combined and interchanged with the aspects described in the previous submission.

[0120] The aspects described and contemplated in this application can be implemented in many different forms. Figures 5 to 32 Some examples may be provided, but others are also contemplated. Figures 5 to 32 The discussion does not limit the breadth of specific implementations. At least one of these aspects generally relates to video encoding and decoding, and at least one other aspect generally relates to generating a bitstream, storing a bitstream, and / or sending a generated or encoded bitstream. These aspects and other aspects may be implemented as methods, apparatus, a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium having stored thereon a bitstream generated according to any of the described methods. As used herein, a bitstream may or may not be sent.

[0121] In this application, the terms "reconstruction" and "decoding" are used interchangeably, the terms "pixel" and "sample" are used interchangeably, and the terms "image", "picture" and "frame" are used interchangeably.

[0122] Various methods are described herein, and each of these methods includes one or more steps or actions for implementing the described methods. Unless the correct operation method requires a specific order of steps or actions, the order and / or purpose of specific steps and / or actions can be modified or combined. Additionally, terms such as "first", "second" and the like can be used to modify elements, parts, steps, operations, etc. in various examples, such as, for example, "first decoding" and "second decoding". Unless specifically required, the use of such terms does not imply the ordering of the modification operation. Therefore, in this example, the first decoding does not need to be performed before the second decoding, and can, for example, occur before, during, or in a time period overlapping with the second decoding.

[0123] The various methods and other aspects described in this application can be used to modify e.g. Figure 2 The video encoder 200 shown in FIG. Figure 3 Modules (e.g., decoding modules) of the decoder 300 shown. In addition, the subject matter disclosed herein can be applied to, for example, any type, format, or version of video decoding, whether described in a standard or described in a recommendation, whether pre-existing or developed in the future, as well as extensions of any such standards and recommendations. Unless otherwise specified or technically excluded, the aspects described in this application can be used alone or in combination.

[0124] Various numerical values ​​are used in the examples describing the present application, such as 0, 1, 2, 3, 4, 6, 7, 8, 11, 16, 18, 26, 33, 45, 50, 64, 65, 66, 67, 80, 129, 131, 135, 1456, etc. These specific values ​​and other specific values ​​are used for the purpose of describing the examples, and the described aspects are not limited to these specific values.

[0125] Figure 2 is a diagram illustrating an example video encoder. Variations of the example encoder 200 are contemplated, but the following describes encoder 200 for clarity without describing all contemplated variations.

[0126] Before being encoded, the video sequence may undergo pre-encoding processing (201), for example, applying a color transform to an input color picture (e.g., conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of input picture components to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and attached to the bitstream.

[0127] In encoder 200, a picture is encoded by encoder elements as described below. The picture to be encoded is partitioned (202) and processed in units such as coding units (CUs). Each unit is encoded, for example, using intra mode or inter mode. When a unit is encoded in intra mode, the unit performs intra prediction (260). Motion estimation (275) and compensation (270) are performed in inter mode. The encoder decides (205) which of intra mode or inter mode to use to encode the unit, and indicates the intra / inter decision by, for example, a prediction mode flag. The prediction residual is calculated, for example, by subtracting (210) the predicted block from the original image block.

[0128] The prediction residual is then transformed (225) and quantized (230). The quantized transform coefficients, along with motion vectors and other syntax elements, are entropy coded (245) to output a bitstream. The encoder may skip the transform and apply quantization directly to the untransformed residual signal. The encoder may bypass both the transform and quantization, i.e., directly code the residual without applying the transform or quantization process.

[0129] The encoder decodes the encoded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The image block is reconstructed by combining the decoded prediction residual and the predicted block (255). An in-loop filter (265) is applied to the reconstructed picture to perform, for example, deblocking / SAO (sample adaptive offset) filtering to reduce coding artifacts. The filtered image is stored at a reference picture buffer (280).

[0130] Figure 3 is a diagram illustrating an example of a video decoder. In the example decoder 300, a bitstream is decoded by decoder elements as described below. The video decoder 300 generally performs the same Figure 2 The decoding process is the reverse of the encoding process described in . The encoder 200 also typically performs video decoding as part of encoding the video data.

[0131] Specifically, the input of the decoder includes a video bitstream, which may be generated by the video encoder 200. First, the bitstream is entropy decoded (330) to obtain transform coefficients, motion vectors, and other decoded information. The picture partition information indicates how the picture is partitioned. Therefore, the decoder can divide the picture according to the decoded picture partition information (335). The transform coefficients are dequantized (340) and inverse transformed (350) to decode the prediction residual. The image block is reconstructed by combining the decoded prediction residual and the predicted block (355). The predicted block can be obtained (370) from intra-frame prediction (360) or motion compensated prediction (i.e., inter-frame prediction) (375). An in-loop filter (365) is applied to the reconstructed image. The filtered image is stored at a reference picture buffer (380).

[0132] The decoded picture may also undergo post-decoding processing (385), such as an inverse color transform (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or an inverse remapping that performs the inverse of the remapping performed in the pre-encoding process (201). The post-decoding processing may use metadata derived in the pre-encoding process and signaled in the bitstream. In an example, the decoded image (e.g., after applying the in-loop filter (365) and / or after the post-decoding processing (385), if post-decoding processing is used) may be transmitted to a display device for presentation to a user.

[0133] Figure 4 It is a diagram showing an example of a system in which various aspects and examples described herein can be implemented. System 400 may be embodied as a device including various components described below, and is configured to perform one or more aspects of the various aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances and servers. The elements of system 400 may be embodied in a single integrated circuit (IC), multiple ICs and / or discrete components, either individually or in combination. For example, in at least one example, the processing and encoder / decoder elements of system 400 are distributed over multiple ICs and / or discrete components. In various examples, system 400 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through a dedicated input port and / or output port. In various examples, system 400 is configured to implement one or more aspects of the various aspects described in this document.

[0134] The system 400 includes at least one processor 410 configured to execute instructions loaded therein for implementing, for example, various aspects described in this document. The processor 410 may include embedded memory, input and output interfaces, and various other circuit systems as known in the art. The system 400 includes at least one memory 420 (e.g., a volatile memory device and / or a non-volatile memory device). The system 400 includes a storage device 440, which may include a non-volatile memory and / or a volatile memory, including but not limited to an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a magnetic disk drive, and / or an optical disk drive. As a non-limiting example, the storage device 440 may include an internal storage device, an attached storage device (including a storage device that can be detached and cannot be detached), and / or a network-accessible storage device.

[0135] The system 400 includes an encoder / decoder module 430, which is configured to process data, for example, to provide encoded video or decoded video, and the encoder / decoder module 430 may include its own processor and memory. The encoder / decoder module 430 represents one or more modules that may be included in the device to perform encoding and / or decoding functions. As is well known, a device may include one or both of an encoding module and a decoding module. Additionally, the encoder / decoder module 430 may be implemented as a separate element of the system 400, or may be incorporated into the processor 410 as a combination of hardware and software known to those skilled in the art.

[0136] Program code to be loaded onto the processor 410 or the encoder / decoder 430 to perform various aspects described in this document may be stored in the storage device 440 and subsequently loaded onto the memory 420 for execution by the processor 410. According to various examples, one or more of the processor 410, the memory 420, the storage device 440, and the encoder / decoder module 430 may store one or more of the various items during the execution of the processes described in this document. Such stored items may include, but are not limited to, input video, decoded video or partially decoded video, bitstreams, matrices, variables, and intermediate or final results of processing equations, formulas, operations, and operation logic.

[0137] In some examples, memory inside the processor 410 and / or the encoder / decoder module 430 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other examples, memory external to the processing device (e.g., the processing device may be the processor 410 or the encoder / decoder module 430) is used for one or more of these functions. The external memory may be a memory 420 and / or a storage device 440, such as a dynamic volatile memory and / or a non-volatile flash memory. In several examples, the external non-volatile flash memory is used to store, for example, an operating system for a television. In at least one example, a fast external dynamic volatile memory (such as RAM) is used as working memory for video encoding and decoding operations.

[0138] Input to the elements of system 400 may be provided through various input devices as indicated in block 445. Such input devices include, but are not limited to: (i) a radio frequency (RF) section that receives an RF signal, such as sent over the air by a broadcaster; (ii) a component (COMP) input terminal (or a collection of COMP input terminals); (iii) a universal serial bus (USB) input terminal; and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Other examples ( Figure 4 ) includes composite video.

[0139] In various examples, the input device of block 445 has associated corresponding input processing elements as known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also referred to as selecting a signal, or band limiting a signal to a frequency band); (ii) down-converting the selected signal; (iii) again band-limiting to a narrower frequency band to select a signal band that may be referred to as a channel in some examples; (iv) demodulating the down-converted and band-limited signal; (v) performing error correction; and / or (vi) demultiplexing to select a desired data packet stream. Various examples of the RF section include one or more elements for performing these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or to baseband. In a set-top box example, the RF part and its associated input processing element receive the RF signal sent by wired (for example, cable) medium, and filter to the desired frequency band by filtering, down-conversion and again to perform frequency selection.Various examples rearrange the order of above-mentioned (and other) elements, remove some elements in these elements, and / or add other elements that perform similar or different functions.Adding element can include inserting element between existing elements, such as, inserting amplifier and analog-to-digital converter.In various examples, the RF part includes antenna.

[0140] The USB and / or HDMI terminals may include corresponding interface processors for connecting the system 400 to other electronic devices across the USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, in a separate input processing IC or in the processor 410, if necessary. Similarly, various aspects of USB or HDMI interface processing may be implemented, if necessary, in a separate interface IC or in the processor 410. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, the processor 410 and the encoder / decoder 430, which operate in conjunction with the memory and storage elements to process the data streams as needed for presentation on the output device.

[0141] The various elements of system 400 may be arranged within an integrated housing in which the various elements may be interconnected and data transmitted between these elements using a suitable connection arrangement 425 (e.g., an internal bus known in the art, including an inter-chip (I2C) bus, wiring, and printed circuit boards).

[0142] System 400 includes a communication interface 450 that enables communication with other devices via a communication channel 460. Communication interface 450 may include, but is not limited to, a transceiver configured to send and receive data through communication channel 460. Communication interface 450 may include, but is not limited to, a modem or a network card, and communication channel 460 may be implemented, for example, within a wired and / or wireless medium.

[0143] In various examples, data is streamed or otherwise provided to the system 400 using a wireless network, such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi-Fi signals of these examples are received via a communication channel 460 and a communication interface 450 suitable for Wi-Fi communications. The communication channel 460 of these examples is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other cross-top communications. Other examples provide streaming data to the system 400 using a set-top box that delivers data via an HDMI connection of an input box 445. Still other examples use an RF connection of an input box 445 to provide streaming data to the system 400. As described above, the various examples provide data in a non-streaming manner. Additionally, the various examples use wireless networks other than Wi-Fi, such as cellular networks or network.

[0144] System 400 can provide output signals to various output devices, including display 475, speaker 485 and other peripheral devices 495. Display 475 of various examples includes one or more of, for example, a touch screen display, an organic light emitting diode (OLED) display, a curved display and / or a foldable display. Display 475 can be used for a television, a tablet computer, a laptop, a cellular phone (mobile phone) or other devices. Display 475 can also be integrated with other components (e.g., as in a smart phone), or it can be separate (e.g., an external monitor for a laptop). In various examples, other peripheral devices 495 include one or more of a stand-alone digital video disc (or digital versatile disc) (DVD, for both terms), a disc player, a stereo system and / or a lighting system. Various examples use one or more peripheral devices 495, which provide functions based on the output of system 400. For example, a disc player performs the function of playing the output of system 400.

[0145] In various examples, control signals are communicated between the system 400 and the display 475, speaker 485, or other peripheral device 495 using signaling such as AV.Link, consumer electronics control (CEC), or other communication protocols that enable device-to-device control with or without user intervention. Output devices may be communicatively coupled to the system 400 via dedicated connections through respective interfaces 470, 480, and 490. Alternatively, the output devices may be connected to the system 400 via a communication interface 450 using a communication channel 460. The display 475 and speaker 485 may be integrated into a single unit with other components of the system 400 in an electronic device such as, for example, a television. In various examples, the display interface 470 includes a display driver, such as, for example, a timing controller (TCon) chip.

[0146] For example, if the RF portion of input 445 is part of a separate set-top box, the display 475 and speaker 485 may alternatively be separate from one or more of the other components. In various examples where the display 475 and speaker 485 are external components, the output signals may be provided via dedicated output connections including, for example, an HDMI port, a USB port, or a COMP output.

[0147] These examples may be performed by computer software implemented by processor 410 or by hardware or by a combination of hardware and software. As a non-limiting example, these examples may be implemented by one or more integrated circuits. As a non-limiting example, memory 420 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology (such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory). As a non-limiting example, processor 410 may be of any type suitable for the technical environment and may include one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0148] Various implementations involve decoding. As used in this application, "decoding" may encompass, for example, all or part of a process performed on a received encoded sequence to produce a final output suitable for display. In various examples, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various examples, such processes also include or alternatively include processes performed by a decoder of various implementations described in this application, such as deriving a first candidate intra-frame prediction mode (IPM) based on a first template of a decoding block; deriving a second candidate IPM based on a second template of the decoding block; and decoding the decoding block based on the first candidate IPM and the second candidate IPM.

[0149] As another example, in one example, "decoding" refers only to entropy decoding; in another example, "decoding" refers only to differential decoding; and in another example, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or broadly to a broader decoding process will be clear based on the context of the specific description and is considered to be well understood by those skilled in the art.

[0150] Various implementations involve encoding. In a manner similar to the discussion above about "decoding", "encoding" as used in this application may cover, for example, all or part of the processes performed on an input video sequence to produce an encoded bitstream. In various examples, such processes include one or more of the processes typically performed by an encoder, such as partitioning, differential encoding, transforms, quantization, and entropy encoding. In various examples, such processes also include or alternatively include processes performed by encoders of various implementations described in this application, such as deriving a first candidate intra-frame prediction mode (IPM) based on a first template of a decoding block; deriving a second candidate IPM based on a second template of the decoding block; and encoding the decoding block based on the first candidate IPM and the second candidate IPM.

[0151] As another example, in one example, "encoding" refers only to entropy encoding; in another example, "encoding" refers only to differential encoding; and in another example, "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding process" specifically refers to a subset of operations or broadly refers to a broader encoding process will be clear based on the context of the specific description and is believed to be well understood by those skilled in the art.

[0152] Note that the syntax elements used herein (e.g., coding syntax for the number of partition modes, the number of intra prediction mode candidates, the number of inter prediction mode candidates, the number of partition mode candidates, block size, partition, partition mode, slice type, etc.) are descriptive terms. Therefore, they do not exclude the use of other syntax element names.

[0153] When the figures are presented as flow charts, it should be understood that they also provide block diagrams of corresponding devices. Similarly, when the figures are presented as block diagrams, it should be understood that they also provide flow charts of corresponding methods / processes.

[0154] The specific implementations and aspects described herein may be implemented in, for example, methods or processes, devices, software programs, data streams, or signals. Even if discussed only in the context of a single form of specific implementation (e.g., discussed only as a method), the specific implementation of the features discussed may also be implemented in other forms (e.g., devices or programs). The device may be implemented in, for example, appropriate hardware, software, and firmware. The method may be implemented in, for example, a processor generally referring to a processing device, which includes, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes communication equipment, such as, for example, a computer, a mobile phone, a portable / personal digital assistant ("PDA"), and other equipment that facilitates information communication between end users.

[0155] References to "one example" or "an example" or "one implementation" or "an implementation" and other variations thereof mean that a particular feature, structure, characteristic, etc. described in connection with the example is included in at least one example. Thus, the phrases "in one example" or "in an example" or "in one implementation" or "in an implementation" and any other variations appearing in various places throughout this application are not necessarily all referring to the same example.

[0156] Additionally, the present application may involve "determining" various pieces of information. Determining information may include, for example, estimating information, calculating information, predicting information, or retrieving information from a memory. Obtaining may include receiving, retrieving, constructing, generating, and / or determining.

[0157] Furthermore, the present application may refer to "accessing" various pieces of information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0158] Additionally, the present application may refer to "receiving" various pieces of information. Like "accessing," receiving is intended to be a broad term. Receiving information may include, for example, one or more of accessing the information or retrieving the information (e.g., from a memory). Moreover, "receiving" generally involves in one way or another during operations such as, for example, storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0159] It should be understood that, for example, in the case of "A / B," "A and / or B," and "at least one of A and B," the use of any of the following " / ," "and / or," and "at least one of" is intended to encompass selecting only the first listed option (A), or selecting only the second listed option (B), or selecting both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such phrases are intended to encompass selecting only the first listed option (A), or selecting only the second listed option (B), or selecting only the third listed option (C), or selecting only the first listed option and the second listed option (A and B), or selecting only the first listed option and the third listed option (A and C), or selecting only the second listed option and the third listed option (B and C), or selecting all three options (A and B and C). This can be extended to as many items as listed, as would be apparent to one of ordinary skill in the art and related arts.

[0160] Moreover, as used herein, the word "signaling" refers in particular to indicating something to a corresponding decoder. The encoder signal may include, for example, intra-frame prediction mode candidates, the number of partition mode candidates, block size, slice type, etc. In this way, in the example, the same parameters are used at both the encoder side and the decoder side. Thus, for example, the encoder may send (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters and other parameters, signaling may be used without sending (implicit signaling) to simply allow the decoder to know and select specific parameters. Bit savings are achieved in various examples by avoiding sending any actual function. It should be understood that signaling can be implemented in various ways. For example, in various examples, one or more syntax elements, flags, etc. are used to signal information to the corresponding decoder. Although the verb form of the word "signaling" is mentioned above, the word may also be used as a noun "signal" in this article.

[0161] It will be apparent to one of ordinary skill in the art that a specific implementation may generate various signals formatted to carry, for example, storable or transmittable information. The information may include, for example, instructions for executing a method or data generated by one of the described specific implementations. For example, a signal may be formatted to carry a bit stream of the described examples. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier using the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is well known, the signal may be sent over a variety of different wired or wireless links. The signal may be stored on, or accessed or received from, a processor-readable medium.

[0162] Many examples are described herein. The features of the examples may be provided individually or in any combination across various claim categories and types. In addition, the examples may include one or more of the features, devices or aspects described herein individually or in any combination across various claim categories and types. For example, the features described herein may be implemented in a bitstream or signal including information generated as described herein. This information may allow a decoder to decode a bitstream, encoder, bitstream and / or decoder according to any of the embodiments described in the embodiments. For example, the features described herein may be implemented by creating and / or sending and / or receiving and / or decoding a bitstream or signal. For example, the features described herein may be implemented by a method, process, device, medium storing instructions, medium storing data or signal. For example, the features described herein may be implemented by a TV, a set-top box, a cell phone, a tablet computer or other electronic devices that perform decoding. A TV, a set-top box, a cell phone, a tablet computer or other electronic devices may display (e.g., using a monitor, a screen or other type of display) the resulting image (e.g., an image reconstructed from the residual of a video bitstream). A TV, a set-top box, a cell phone, a tablet computer or other electronic devices may receive a signal including an encoded image and perform decoding.

[0163] Intra prediction can be used to exploit correlations within local regions of a picture. For intra prediction, the texture of a picture region may be similar to the texture in a local neighborhood and may be predicted from the local neighborhood. Neighboring samples, such as samples from the line of samples above the current block and samples from the last column of a reconstructed block to the left of the current block, may be used for prediction.

[0164] Reference neighboring samples used to predict a current block may depend on a direction indicated by an intra prediction angle of a corresponding intra prediction mode (IPM). Figure 5 An example of directional intra prediction with reference neighboring samples is shown. For example, for horizontal prediction, reference neighboring samples from the left column can be used. For vertical prediction, reference neighboring samples from the previous row can be used. For bottom right diagonal prediction, reference neighboring samples from the top left can be applied. For bottom left diagonal prediction, reference neighboring samples from the top right can be applied.

[0165] Figure 6 An example of intra-mode decoding associated with 67 IPMs is shown. In order to capture arbitrary edge directions present in the video, the number of directional intra-modes can be expanded from 33 to 65, as shown in Figure 6 As depicted, the number of planar modes and DC modes may remain the same. Dense directional IPM may be applied to block sizes as well as luma and chroma intra prediction.

[0166] For a square CU, an angular IPM 2 - 66 can be used. The prediction mode can correspond to, for example, an angular intra prediction direction defined from 45 degrees to 135 degrees in the clockwise direction.

[0167] Wide - angle intra prediction can be used for non - square blocks. For non - square blocks, wide - angle IPM can be utilized to replace (e.g., adaptively replace) multiple angular IPMs. As Figure 6 shown by the dashed arrows in, the wide - angle modes beyond the lower - left direction mode can be indexed from 14 to 1, and the wide - angle modes beyond the upper - right direction can be indexed from 67 to 80. Flat blocks (W > H) and tall blocks (W < H) can use wide - angle modes to replace an equal number of regular - angle modes in opposite directions.

[0168] Figure 7 An example of a current W×H block to be predicted using wide - angle intra prediction is shown. As Figure 7 shown, a set of decoded reference samples can be used for the current W×H block to be predicted. The set of decoded reference samples can include an array of top - decoded reference samples of length 2W + 1 and an array of left - decoded reference samples of length 2H + 1. As Figure 7 shown, a relationship can be provided between the range of decoded reference samples around the current W×H block and the range of allowed intra - prediction angles.

[0169] Figure 8 An example of matrix - weighted intra prediction (MIP) is shown. To predict the samples of a rectangular block with width W and height H, MIP can take as input a row of H reconstructed adjacent boundary samples to the left of the block and a row of W reconstructed adjacent boundary samples above the block. In the example, if the reconstructed samples are not available, intra prediction (e.g., non - matrix - weighted intra prediction) can be used to generate the reconstructed samples. The generation of the prediction signal can be based on one or more of the following: averaging, matrix - vector multiplication, and / or linear interpolation (e.g., as Figure 8 shown).

[0170] For an intra - coded block, a flag mip_flag indicating whether the MIP mode is to be applied can be signaled.

[0171] Decoder - side intra - mode derivation (DIMD) can be used to derive the intra - mode for decoding a coding unit (CU). Fig. 9 An example of DIMD is shown. In the example, if DIMD is applied, an IPM (e.g., two IPMs) that may be the best IPM (e.g., two best IPMs) for predicting the current CU can be derived from an orientation gradient histogram (HOG) calculated from the neighboring pixels of the current block. As Fig. 9As shown, the IPM (eg, two predictors) and the planar mode predictor may be combined using weights derived from the HOG in the template.

[0172] Fig.10 An example of deriving an IPM of DIMD from a gradient in a template is shown. In the example, for the current CU, an IPM (e.g., two IPMs) may be derived from a gradient in a template, such as Fig.10 As shown. A HOG with 65 bins corresponding to 65 directional IPMs may be initialized to 0. For a decoded reference sample in the middle row or middle column of the template of the three rows of decoded reference samples above the current CU and the three columns of decoded reference samples to the left of the current CU, one or more of the following may be applied. A 3x3 horizontal Sobel filter and a 3x3 vertical Sobel filter, both centered on the decoded reference sample, may generate horizontal gradients G respectively. HOR and vertical gradient G VER . G HOR and G VER The sign of may indicate in which direction range (eg, four ranges) the perpendicular to the horizontal component G is found. HOR and the vertical component G VER The target direction of the gradient G. In the example, if |G VER |>|G HOR |, the anchor direction may correspond to the horizontal direction. In the example, if |G HOR |≥|G VER |, then the anchor direction may correspond to the vertical direction. The target direction may form an angle θ relative to the anchor direction. By discretizing a scaled version of tan(θ), the index i of the IPM whose direction is closest to the target direction may be found. The HOG bin of index i may be incremented by |G HOR |+|G VER |. The index of the largest HOG bin (eg, the two largest HOG bins) may be the index of the derived IPM (eg, the two derived IPMs). For intra-coded blocks, a flag dimd_flag indicating whether the DIMD mode is to be applied may be signaled.

[0173] Fusion may be used for template-based intra-mode derivation (TIMD).Intra-mode for coding a CU derived using fusion for TIMD may be described herein.

[0174] Fig.11 An example of a template for deriving a fused IPM for TIMD is shown. For an IPM in the most probable mode (MPM), the sum of absolute transform errors (SATD) between the predicted and reconstructed samples of the template can be expressed as Fig.11The IPM with the smallest SATD can be selected (e.g., the first two IPMs). For TIMD, the SATD can be calculated by Figure 6 Directions are inserted between the solid black arrows shown to expand the set of directional IPMs from 65 to 129. This may mean that the set of possible IPMs derived via TIMD collects 131 modes. Based on retaining the IPMs (e.g., two IPMs) from the first pass of testing involving the MPM list supplemented with the default mode, for a mode (e.g., one or more of the two modes), if the IPM is neither planar nor DC, TIMD may test its two closest extended directional IPMs in terms of predicting SATD. Based on SATD IPM2 <2*SATD IPM1 If the condition is true, these selected IPMs (e.g., the last two selected IPMs) may be fused with weights that may depend on the SATD of the IPMs (e.g., the two IPMs); otherwise, the first IPM may be used. For intra-coded blocks, a flag timd_flag indicating whether the TIMD mode is to be applied is signaled.

[0175] Multiple reference lines (MRL) intra prediction can use more reference lines for intra prediction. Fig.12 An example of 4 reference lines associated with MRL is shown. Samples of fragments A and F may not be available from reconstructed neighboring samples and may be filled with the closest samples from fragments B and E, respectively. Intra picture prediction may use the nearest reference line (e.g., reference line 0). In MRL, 2 lines may be used (e.g., additional lines such as reference line 1 and reference line 3). The index mrl_idx of the selected reference line may be signaled and used to generate intra prediction factors.

[0176] Intra sub-partitioning (ISP) may be used to vertically or horizontally split a luma intra prediction block into 2 or 4 sub-partitions according to the block size. Fig.13 An example of ISP is shown. The reconstructed sample values ​​of a sub-partition may be used to generate a prediction for the next sub-partition, and the sub-partitions may be processed repeatedly. The sub-partitions may satisfy the condition of having at least 16 samples and may share the same intra-mode.

[0177] For intra-coded blocks, a flag isp_flag indicating whether ISP is to be applied may be signaled. The syntax isp_mode may specify vertical or horizontal partitioning and may be signaled based on the condition that isp_flag is true.

[0178] Intra-prediction mode signaling may be provided. In an example, if the intra-prediction mode selected for predicting the current CU is not DIMD, MIP mode, or TIMD (e.g., the intra-prediction mode is one of the 67 IPMs as described herein), the index of the intra-prediction mode may be signaled using the MPM list of the CU. Block differential pulse code modulation (BDPCM), template-based intra prediction (TMP), intra-block copy (IBC), and palette coding may be activated for a video sequence (e.g., exclusively, such as screen content), and may be disabled when using the MPM list.

[0179] The MPM list may include a list of primary MPMs (e.g., 6 primary MPMs) and a list of secondary MPMs (e.g., 16 secondary MPMs), such as Fig.14 shown. Fig.14 An example of an MPM list is shown. The MPM list may be constructed by adding candidate IPM indexes in order from the candidate IPM index most likely to be selected as the IPM for predicting the current CU to the candidate IPM index least likely to be selected for predicting the current CU. Redundancy in the MPM list may be removed, for example, so that the MPM list may not include multiple identical IPM indices.

[0180] Fig.15 An example of signaling the intra prediction mode selected for predicting the current CU at the encoder side is shown. In the example, the intra prediction mode selected for predicting the current CU at the encoder side may be applied to the decoder side.

[0181] A geometric partition mode (GPM) can be used for better alignment of inter prediction boundaries with objects. A GPM may include a total of 64 partitions for inter prediction. In an example, if the geometric merge mode is used, the inter prediction boundary can be aligned with a geometrically positioned straight line (e.g., Fig.16 As shown) the CU is split into partitions (e.g., two partitions). Fig.16 An example of 64 partitions for a GPM is shown. The location of the partition line can be mathematically determined from the perspective of the specific partition. and distance offset ρ i Derived. Partitions in a CU can be inter-predicted using the motion parameters of the partition. Single prediction can be allowed for partitions. In an example, a partition can have a motion vector and a reference index. Based on the predicted partition, a blending process with adaptive weights can be used to adjust the sample values ​​along the partition edge.

[0182] The blending weights for positioning of a CU may be derived based on the distances between individual positions and the partition edge. The distance of a position (x, y) to a partition edge may be derived as follows:

[0183]

[0184] The weight of a portion of a geometric partition can be derived as follows:

[0185] wIdxL(x,y)=partIdx? 32+d(x,y):32-d(x,y)

[0186]

[0187] w1(x,y)=1-w0(x,y)

[0188] Here, partIdx depends on the angle index i. Fig.17 An example of weight w0 in an example blending process is shown.

[0189] Fig.18 An example of a GPM intra mode is shown. The GPM intra mode may be associated with adding the intra mode to the GPM to combine inter prediction and intra prediction. In a GPM with inter and intra prediction, a prediction sample (e.g., a final prediction sample) may be generated by weighting the inter prediction samples and the intra prediction samples for a region separated by the GPM. The inter prediction samples may be derived by the same scheme as the GPM, where the intra prediction samples may be derived by an IPM candidate list and an index signaled from the encoder. The IPM candidate list size may be predefined as 3. The available IPM candidates may be an angular mode parallel to the GPM block boundary (e.g., a parallel mode), an angular mode perpendicular to the GPM block boundary (e.g., a vertical mode), and a planar mode, such as Fig.18 shown.

[0190] like Fig.18 As shown, GPM with intra and intra prediction can be restricted to reduce the signaling overhead for IPM and avoid increasing the size of the intra prediction circuitry on the hardware decoder.

[0191] Fig.19 An example of non-rectangular partitioning for a portion of a picture is shown. Non-rectangular partitioning in prediction, such as diagonal partitioning (1010) and general geometric partitioning (1020), can be used to delineate the shape of an object (complex shape) from a background or other objects. Rectangular (e.g., including square) partitioning can be applied to intra frames, and objects with different characteristics can be included in intra-coded blocks. In an example, if a block has both a change region along a specific direction and a constant change region, or if a block has more than one change region along different directions, the change region may not be described by a single corresponding angular mode, a planar mode, or a DC mode.

[0192] Fig. 20 An example of a piecewise smooth image model is shown. In the example, if we consider Fig. 20 The piecewise smooth image model shown, where smooth regions (e.g., different smooth regions) with smooth properties (e.g., different smooth properties) are separated by an edge (1110), then using a single intra prediction model to predict the two regions may not be accurate. In the region near the edge, the region may be continuously partitioned into smaller square / rectangular blocks and decoded separately as smaller blocks. Smaller prediction blocks with similar data may incur overhead.

[0193] More than 2 IPMs for a CU may be supported (e.g., TIMD may support 2 IPMs; DIMD may support 3 IPMs). The blending process may be performed on one or more pixels with the same weight. To model such blocks, a geometric partitioning mode for intra prediction may be considered. GPM may be extended to intra prediction, which may be referred to as spatial GPM (SGPM). Fig.21 An example of a SGPM is shown in A. This may include a partition mode and an associated IPM (eg, two associated IPMs). Fig.21 B shows an example of a partition mode and two IPMs associated with a SGPM signaled in a video bitstream. Fig.21 B, overhead bits may be generated. In order to efficiently express partition and prediction information in the bitstream, a candidate list may be used, and the candidate index may be signaled in the bitstream. The candidates in the list may lead to a combination of partition mode and intra prediction mode (e.g., two prediction modes), such as Fig.21 As shown in C.

[0194] Fig.21 C shows the derived combination of partition mode and two IPMs associated with the SGPM signaled in the video bitstream. A template may be used to generate the candidate list. The shape of the template may be the same as the TIMD. For possible combinations of partition mode and intra prediction mode (e.g., two intra prediction modes), predictions may be generated for the template using the partition weights extended to the template, as Fig. 22 shown. Fig. 22 An example of a prediction generated for a template using partition weights extended to the template is shown. The combinations can be sorted in ascending order of the SATD between the prediction and reconstruction of the template. The length of the candidate list can be set equal to 16, and the candidates can be considered as the most likely SGPM combination for the current block. Fig.23An example of a flow chart of an SGPM is shown. The encoder and decoder may construct the same candidate list based on a template. At 2302, predictions may be generated for possible partition modes and intra prediction modes (IPMs) in the template, and SATDs may be calculated. At 2304, a combination of a partition mode and two IPMs may be checked, and the combined SATDs may be sorted in ascending order to construct a candidate list. At 2306, predictions may be generated for possible candidates in the current block, and the SATDs may be sorted to determine the final full rate distortion (RD) candidate. In order to reduce the complexity of constructing the candidate list, the number of possible partition modes and the number of possible intra prediction modes may be pruned. As described herein, 26 of the 64 partition modes may be used, and MPMs of the 67 intra prediction modes may be used.

[0195] SGPM can attempt to limit possible partitions and IPMs. A tradeoff can be made between complexity and performance (e.g., 0.26% gain for intra-frame configuration (AI) versus 172% encoding time and 114% decoding time increase). For example, if DIMD, TIMD, or MIP is used, SGPM can be disabled. For example, if SGPM is used (e.g., as Fig.24 ), MRL and ISP can be disabled. Fig.24 An example of signaling, at the encoder side, the intra prediction mode selected for predicting the current CU is shown, including the addition of SGPM.

[0196] One or more of the following may apply to SGPM: derivation of intra-frame prediction modes; selection of partition modes; combination of intra-frame prediction modes and partition modes; template for constructing the SGPM candidate list; length of the SGPM candidate list; and / or interaction between SGPM and other intra-frame modes.

[0197] The SGPM may include a partition mode and an associated IPM (eg, two associated IPMs) for an intra-coded block. The SGPM mode may interact with other intra modes such as DIMD, TIMD, and / or ISP.

[0198] Intra-frame prediction modes can be used to construct candidate lists. In order to reduce the complexity of constructing candidate lists, the number of possible intra-frame prediction modes can be pruned. In an example, an MPM of 67 intra-frame prediction modes can be used. An MPM associated with a coded block can be obtained. A prediction corresponding to the MPM for a first template can be determined. A first candidate intra-frame prediction mode (IPM) can be selected from the MPM. The selection can be based on a comparison of the first template with the prediction corresponding to the MPM. The prediction mode can be derived based on template matching, motion estimation, or pattern recognition. Inter-frame mode prediction can be used. The MPM can be used to select the most likely motion vector of a block in the current frame. The selection can be determined based on a comparison of the coded block with a corresponding prediction of the MPM.

[0199] Fig.25 An example of the derivation of possible IPM candidates for SGPM is shown. At 3001, an IPM list may be initialized. Available primary MPMs (e.g., 6 primary MPMs such as plane, IPMs for left / above / lower left / upper right / upper left blocks at 2504) and DIMD IPMs (e.g., 2 DIMD IPMs at 2506) may be included in a list of possible intra prediction modes. At 2508, for intra-coded slices (e.g., I slices), additional IPMs (e.g., DC, horizontal and / or vertical) may be inserted (e.g., inserted continuously) into the list. In an example, at 2510, if the size of the current possible IPM does not reach 4, this may indicate that up to 8 IPM candidates may be added to the list for intra-coded slices. Otherwise, at 2512, 3 additional IPMs may be inserted (e.g., inserted continuously) into the list, which may indicate that up to 11 IPM candidates may be added to the list for inter-coded slices (e.g., B / P slices). In an example, if an IPM candidate is inserted into the list, a redundancy check may be applied to find out whether the same IPM already exists in the list. In an example, for an I slice, the possible IPM candidates may be up to 8. 2 of the 8 IPM candidates may be selected for SGPM, which means that 2×8×Num PartitionMode predictions and calculate the SATD cost of the relevant template. Check 8×7×Num PartitionMode A comparison.

[0200] Possible IPM candidates for SGPM may be derived based on the dynamically changeable IPM list size. In an example, to reduce complexity, the maximum allowed IPM list size may be set to M (M<8). In an example, M may be 4. If the total number of available IPM candidates reaches M, the IPM candidate list construction process for SGPM may be terminated.

[0201] In an example, the allowed IPM list size (e.g., the maximum allowed IPM list size) may depend on the frame resolution, slice type, block size, and / or block shape. For frame resolution, for example, a larger IPM list size may be applied to content with a larger resolution (e.g., 4K content) to find the best IPM. For slice type, a smaller (e.g., or larger) IPM list size may be applied to an inter-coded slice. In an example, if the slice is a B slice, the corresponding IPM list size may be set to M=2. The block size (e.g., the width and / or height of the current block) may meet (e.g., be less than, greater than, equal to, etc.) a specific number. In such a case, a smaller (e.g., or larger) IPM list size may be applied to the block. In an example, if the block size is less than 8×8, the corresponding IPM list size may be set to M=2, and the number of pixels to be predicted (e.g., a small number of pixels) may not justify the search cost of the granularity (e.g., additional granularity).

[0202] The IPM candidate list can be constructed by including one or more of the following possible types of IPM candidates: planar; DIMD IPM (e.g., 2 DIMD IPMs); TIMD IPM (e.g., 2 TIMD IPMs); DC; or horizontal (e.g., IPM=18), vertical (e.g., IPM=50), diagonal (e.g., IPM=34) and / or anti-diagonal (e.g., IPM=66).

[0203] If the IPM list of length M is not complete based on the addition of the IPM candidates described herein, IPMs from the list of primary MPMs (e.g., 6 primary MPMs) and the list of secondary MPMs (e.g., 16 secondary MPMs) may be inserted at the end, for example, until an IPM candidate number (e.g., a maximum IPM candidate) is encountered.

[0204] In an example, the selected intra-modes may include intra-modes that are orthogonal to the derived TIMD and / or DIMD IPMs. These modes may include those found within 45 degrees before / after the DIMD mode and / or TIMD mode.

[0205] The order of IPM candidates to be added to the list described herein may depend on the slice type, block size, and / or block shape. The IPM candidates described herein may be replaced by other available IPM candidates (e.g., available IPMs from above / left / lower left / upper right / upper left neighboring blocks). The replacement may depend on the slice type, block size, and / or block shape. In an example, if the current slice is an inter-coded slice (e.g., a B slice), the order of adding the IPM candidates may be replaced so that DC may be inserted into the IPM list (e.g., first). In an example, the block shape (e.g., the ratio of the width (w) to the height (h) of the current block) may satisfy (e.g., be less than, greater than, equal to, etc.) a specific number, and the order of adding the IPM candidates or possible IPM candidates may be replaced. In an example, if the aspect ratio of the block is Then the available IPMs from above / upper right / upper left may be inserted into the IPM list (eg, first).

[0206] In an example, the DIMD and / or TIMD process may be called on the upper or left template (e.g., separately). An intra mode may be derived from the upper template (e.g., which may be oriented vertically) using TIMD and / or DIMD, and another intra mode may be derived from the left template (e.g., which may be oriented horizontally). The intra mode may be added to the candidate list. The intra mode may be considered as a possible mode for the candidate list.

[0207] Partition modes (e.g., possible partition modes) can be used to construct a candidate list. In order to reduce the complexity of constructing the candidate list, the number of possible partition modes can be pruned. In an example, 26 of the 64 partition modes can be used.

[0208] Fig.26 Examples of possible partitioning mode candidates selected for SGPM are shown. Fig.26 As shown, candidates can be marked in the gray dashed box. For 26 partition modes, 2×Num IPM ×26 predictions, and the relevant template SATD cost can be calculated. You can check Num IPM ×(Num IPM -1)×26 comparisons.

[0209] Possible partition mode candidates for SGPM may be selected.In an example, to further reduce complexity, the allowed partition mode list size (eg, maximum allowed partition mode list size) may be set to N (N<26) (eg, N may be 8). Fig. 27 An example of selected partition mode candidates for reduction of SGPM is shown.A block may be divided into two sub-partitions (eg, evenly) by the selection of a partition mode.

[0210] In an example, the allowed partition mode list size (e.g., the maximum allowed partition mode list size) may depend on the frame resolution, slice type, block size, and / or block shape. For slice type, a smaller partition mode list size may be applied to inter-coded slices. In an example, the slice type may be a B slice, and the corresponding list size of the partition mode may be set to N=4, where diagonal partitions (e.g., such as Fig. 27 Partitioning mode 10 shown), horizontal partitioning (e.g., Fig. 27 Partitioning mode 18 shown), vertical division (such as Fig. 27 36) and / or anti-diagonal partitioning (e.g., Fig. 27 55 shown in FIG. 5). The block size (e.g., the width and / or height of the current block) may satisfy (e.g., be less than, greater than, equal to, etc.) a certain number. A smaller (e.g., or larger) partition mode list size may be applied to the block. In an example, if the block size is less than 8×8, the corresponding list size of the partition mode may be set to N=1, where diagonal partitions (e.g., Fig. 27 Partition mode in 10).

[0211] In an example, a syntax element may be signaled to indicate the partition applied at the encoder side. The partition may be a correction to a segmentation line derived from template analysis. A value of a delta angle I may be signaled, where I may be 1, 2, etc., which shifts the partition from template analysis (e.g., the best partition) by + / - 1 degrees to obtain the applied partition.

[0212] The selection of a partition mode candidate may depend on the block shape. The block shape (e.g., the ratio of the width to the height of the current block) may satisfy (e.g., be less than, greater than, equal to, etc.) a certain number. Different sets of selected partition modes may be applied to the block. In an example, if the aspect ratio of the block is The corresponding set of partition mode candidates may include one or more partition modes, which may result in a sub-partition having an upper boundary with a larger surface area (such as Fig. 27 The partition modes 40, 42, 43, 45, 46, 47, 48, 49, 50, 51, 52, 53 and 54) are adjacent.

[0213] Combinations (e.g., possible combinations) may be used to construct a candidate list. For possible combinations of partition modes and intra prediction modes (e.g., two intra prediction modes), a prediction may be generated for the template using the partition weights extended to the template. The combinations may be sorted in ascending order of the SATD of the combination between the prediction and reconstruction of the template.

[0214] Fig.281 is an example flow chart for checking and sorting possible combinations to derive an SGPM candidate list. An SGPM may be determined for a coding block. A first template and a second template for a coding block derived from an IPM may be obtained based on the determination that the SGPM is for the coding block. In some examples, a first template and a second template for a coding block derived from an inter-frame prediction mode may be obtained. The second template may be different from the first template. A third template may be obtained that may include the first template and the second template. Candidates in an SGPM list associated with a coding block may be sorted based on the third template of the coding block. For example, the candidate combinations in the SGPM list may be sorted in ascending order of their corresponding SATDs between the prediction and reconstruction of the third template.

[0215] At 2800, an IPM can be utilized to generate a prediction. The template can be partitioned according to the partition mode, and the SATD costs of the two sub-partition templates can be calculated: SATD_PART1[m][p] and SATD_PART2[m][p]. At 2802, if the partition mode is in the SGPM candidate list, it can be determined at 2804 whether the first IPM m1 exists in the SGPM candidate list. At 2806, a second IPM m2 in the SGPM candidate list is identified, and it can be determined whether m2 is different from m1.

[0216] At 2808, the two IPMs {m1, m2} may be combined with the partition pattern "p", and the SATD cost of the entire template may be calculated. At 2810, the calculated SATD costs may be sorted in ascending order, and the SGPM candidate list may be updated. At 2812, possible combinations may be checked, and the maximum length of the SGPM candidate list may be reached.

[0217] like Fig.28 As shown, there may be Num IPM ×(Num IPM -1)×Num PartitionMode In an example, for an I slice, the possible IPM candidates may be up to 8, and the possible partition modes may be 26, which means 8×7×26=1456 comparisons. The search to identify 16 candidates may increase encoding and decoding time. The number of combinations of partitions and prediction modes may be reduced.

[0218] The combined restrictions on possible partitions and prediction mode candidates for constructing the SGPM candidate list may be described herein. In an example, to reduce complexity, after selecting a particular partition mode, one or more IPM candidates that are combined may be allowed based on the selected partition mode. In an example, the IPM candidate list size may be predefined as 3. A first candidate intra-frame prediction mode (IPM) may be derived based on a first template of the coding block. A second candidate IPM may be derived based on a second template of the coding block. The coding block may be decoded based on the first candidate IPM and the second candidate IPM. A third candidate IPM may be selected based on the partition mode of the coding block. The first candidate IPM, the second candidate IPM, and the third candidate IPM may be added to the IPM candidate list associated with the coding block. The coding block may be decoded based on the IPM candidate list. The inter-frame prediction mode may use a template to derive different inter-frame prediction candidate modes and decode the coding block based on the inter-frame prediction candidate mode.

[0219] The available IPM candidates may be an angular pattern parallel to the segmentation boundary (e.g., parallel pattern), an angular pattern perpendicular to the segmentation boundary (e.g., vertical pattern), and / or a planar pattern. In the example, if horizontal segmentation is selected (e.g., Fig. 27 The partition mode 18 shown in FIG. 1 ) may allow a horizontal mode (eg, a parallel mode), a vertical mode (eg, a perpendicular mode), and a planar mode to be combined.

[0220] The IPM candidate list size may be reduced to 2. In an example, a first candidate IPM and a second candidate IPM may be added to an IPM candidate list associated with a coding block. The coding block may be encoded and / or decoded based on the IPM candidate list. The available IPM candidates may be an angular mode (e.g., a parallel mode) and a planar mode parallel to a partition boundary.

[0221] The available IPM candidates may be an angle pattern parallel to the partition boundary and its closest angle pattern. In an example, the available IPM candidates may be an angle pattern perpendicular to the partition boundary and its closest angle pattern.

[0222] Depending on the selected partition mode candidate and the location of the sub-partitions, the available IPM candidates for one or more sub-partitions may be affected. In an example, a diagonal partition may be selected (e.g., Fig. 27 10). IPM candidates in the horizontal direction may be allowed for a sub-partition adjacent to the left boundary. IPM candidates in the vertical direction may be allowed for another sub-partition adjacent to the upper boundary.

[0223] For an IPM (eg, a first IPM), one or more IPM candidates may be allowed as a second IPM for combination.

[0224] A possible second IPM candidate may be an angular mode that is closer to an IPM (e.g., a first IPM), which means that the absolute difference of these IPM indices (e.g., two IPM indices) may be less than a number (e.g., a threshold T). The value of the threshold T may be predefined and fixed for a sequence, or may be signaled in a sequence parameter set (SPS), a view parameter set (VPS), a picture parameter set (PPS), and / or a picture header, or may be derived based on a block size. In an example, if a horizontal mode (e.g., IPM=18) is selected as an IPM (e.g., a first IPM), an angular mode having an IPM index in the range of (18-T, 18+T) may be allowed as a possible second IPM candidate.

[0225] A possible second IPM candidate may be an angular pattern perpendicular to the first IPM, which means that the absolute difference of these IPM indices (e.g., two IPM indices) may be greater than a number. In an example, if a horizontal pattern (e.g., IPM=18) is selected as the first IPM, a vertical pattern (e.g., such as IPM=50) may be allowed as a possible second IPM candidate.

[0226] A template may be used to generate a candidate list. The shape of the template may be the same as that of the TIMD, which may include 4 upper adjacent rows and 4 left adjacent columns of the current block, such as Fig. 22 As shown. There may be 2×Num generated for the template IPM ×Num PartitionMode predictions, and the associated SATD costs can be calculated for sorting SGPM candidates. The design of the template can affect the tradeoff between complexity and performance. In an example, the larger the size of the template, the higher the computational complexity the encoder / decoder can have (e.g., memory can be used to access adjacent samples). The larger the size of the template, the more accurate the estimation and selection of partitions and prediction modes may be.

[0227] A template for constructing a SGPM candidate list may be obtained. In an example, the template size may depend on the frame resolution, slice type, block size, and / or block shape. In an example, the block size (e.g., the width and / or height of the current block) may satisfy (e.g., be less than, greater than, equal to, etc.) a certain number. In an example, the template size may be set equal to 1. The height of the upper (e.g., top) template may be 1, and the width of the left template may be 1. The left template may be obtained as a first template, and the top template may be obtained as a second template. A smaller (e.g., or larger) template size may be applied to the block. In order to improve the accuracy of estimation and selection, if the block size is less than 8×8, the corresponding template may include one upper adjacent row and one left adjacent column of the current block. In an example, if the block size is greater than 16×16, the corresponding template may include the entire upper adjacent block and the entire left adjacent block of the current block. In order to reduce the complexity of calculation and memory access, a smaller block may use a larger template, and vice versa. For frame resolution, a larger template size may be applied to content with a larger resolution (e.g., 4K content) to find SGPM candidates. For slice types, a smaller (or larger) template size may be applied to inter-coded slices. In an example, if the slice is a B slice, the corresponding template size may include one above-neighboring row and one left-neighboring column of the current block set as M=2.

[0228] The template may depend on the location of the selected partition mode candidate and / or sub-partition. In an example, a first template and a second template of a coding block may be determined. The first coding block and the second coding block may be determined based on the partition mode associated with the coding block. The coding block may include a first partition and a second partition. The first template of the coding block may be determined based on the first partition. The first partition of the coding block may be encoded and / or decoded based on the first candidate IPM derived based on the first template. The second template of the coding block may be determined based on the second partition. The second partition of the coding block may be encoded and / or decoded based on the second candidate IPM derived based on the second template. The inter-frame prediction mode may be integrated, wherein the first template and the second template may be determined based on the corresponding partition of the coding block. The corresponding partition may be encoded and / or decoded based on the derived inter-frame prediction mode.

[0229] As described herein, if a diagonal split is selected (e.g., Fig. 27 For example, if the partition mode 10 shown in FIG. 10 is used, the IPM candidates in the horizontal direction may be allowed for the sub-partition adjacent to the left boundary, and the IPM candidates in the vertical direction may be allowed for another sub-partition adjacent to the upper boundary. In the example, the left template may be applied to derive the horizontal orientation mode, and the upper template may be applied to derive the vertical orientation mode.

[0230] In an example, the length of the possible candidate list may be fixed. A candidate list may be employed, and the candidate index may be signaled in the video data. The candidates in the list may be a combination of a partition mode and an intra prediction mode (e.g., two intra prediction modes). In an example, an IPM candidate list associated with a coding block may be generated based on a first candidate IPM and a second candidate IPM. A first prediction mode for a first partition and a second prediction mode for a second partition may be based on an SGPM index and an IPM candidate list.

[0231] The length of the candidate list may be set equal to 16, and the candidate may be considered the most likely SGPM combination for the current block. Full radio distortion optimization (RDO) may be performed on the 16 candidates. Coding time may increase. The SGPM index may use truncated binary coding, and the binary length of the SGPM index may depend on the length of the SGPM candidate list. A candidate list of length 16 may produce valid coding bits.

[0232] In an example, the length of possible SGPM candidates may be determined dynamically. To reduce complexity and bit cost, the length of the SGPM candidate list may be set to L (L<16), and the candidate list length may depend on the frame resolution, slice type, and / or block size. For slice type, a smaller (or larger) length SGPM candidate list may be applied to inter-coded slices. In an example, if the slice is a B slice, the corresponding length of the SGPM candidate list may be set to L=4. The block size (e.g., the width and / or height of the current block) may satisfy (e.g., less than, greater than, equal to, etc.) a number. In such cases, a smaller (or larger) length SGPM candidate list may be applied to the block. In an example, if the block size is less than 8×8, the corresponding length of the SGPM candidate list may be set to L=2. The small number of pixels to be predicted may not justify the search cost of the granularity.

[0233] In an example, based on applying full RDO to the candidates (e.g., the 16 candidates), a SATD check may be applied (e.g., first applied) to sort out the most likely candidates (e.g., one or two candidates) among the 16 candidates according to the minimum SATD cost. The retained candidates (e.g., the one or two retained candidates) may perform full RDO. The RDO candidate list length (e.g., the final RDO candidate list length) may depend on the frame resolution, slice type, and / or block size.

[0234] In an example, before applying full RDO to the candidates, a SATD check may be applied (e.g., applied first) to sort the candidates. The full RDO process may be performed for candidates whose SATD cost is less than (e.g., less than or equal to) a candidate (e.g., the best candidate, such as the smallest candidate). In an example, candidates whose SATD cost is 1.5 times lower than the minimum SATD cost may be processed in full RDO.

[0235] Interactions between SGPM, DIMD and / or TIMD may be provided. For example, if DIMD and / or TIMD are used (e.g., Fig.24 As shown), the SGPM may be disabled. The interaction between the SGPM, DIMD and / or TIMD may be described herein.

[0236] In an example, if DIMD is enabled, the SGPM may be tested. As described herein, the DIMD IPM (e.g., the two best DIMD IPMs) may be derived from the HOG. These predictors (e.g., the two predictors) and the plane mode predictor may be combined using weights derived from the HOG. The blended predictor Pred may be stored DIMD and the associated RDO cost DIMD . DIMDIPM (e.g., the two best DIMS IPMs) can be reused as possible intra prediction modes for SGPM (e.g., two possible intra prediction modes). 26 of the 64 partition modes (or the possible partition modes described herein) can be used in combination with DIMD IPM. The construction of the candidate list can be similar to the techniques described herein. Based on the Cost from the SGPM candidate list DIMD_SGPM and the related blending predictor Pred DIMD_SGPM Get the minimum RDO cost, executable and Cost DIMD A flag dimd_sgpm_flag indicating whether the SGPM mode is to be applied on top of the DIMD mode may be signaled. DIMD_SGPM <Cost DIMD If it is true, dimd_sgpm_flag may be set to true, an additional index dimd_sgpm_index may be signaled, and the blending predictor (eg, the final blending predictor) may be replaced with Pred DIMD_SGPM .

[0237] In an example, if TIMD is enabled, the SGPM may be tested. As described herein, the IPMs (e.g., the first two IPMs) that relate to the MPM list with the smallest SATD between the predicted sample and the reconstructed sample of the template may be retained in this pass (e.g., the first pass) of the test. For these modes (e.g., two modes), if the IPM is neither planar nor DC, the TIMD may test its closest extended directional IPM (e.g., the two closest extended directional IPMs) in terms of predicted SATD. In the conditional SATD IPM2 <2*SATD IPM1 If is true, the selected IPMs (eg, the last two selected IPMs) may be fused with weights that may depend on the SATD of the IPMs (eg, the two IPMs). The blended prediction factor Pred may be stored TIMD and the associated RDO cost TIMD TIMD IPMs (e.g., the two best TIMD IPMs) can be reused as intra prediction modes for SGPM. 26 of the 64 partition modes (or the possible partition modes described herein) can be used in combination with these TIMD IPMs (e.g., the two best TIMD IPMs). The construction of the candidate list can be similar to the techniques described herein. Based on the selection of Cost from the SGPM candidate list TIMD_SGPM and the related blending predictor Pred TIMD_SGPM Get the minimum RDO cost, executable and Cost TIMD A flag timd_sgpm_flag indicating whether the SGPM mode is to be applied on top of the TIMD mode may be signaled. TIMD_SGPM <Cost TIMD If it is true, timd_sgpm_flag may be set to true, an additional index timd_sgpm_index may be signaled, and the blending predictor (eg, the final blending predictor) may be replaced with Pred TIMD_SGPM In the example, if SATD IPM2 ≥2*SATD IPM1 If true, the first IPM may be used and the SGPM mode may not be applied.

[0238] Interaction between SGPM and ISP can be provided. For example, if SGPM is used (e.g., Fig.24), the ISP may be disabled. In an example, if the SGPM is enabled, the ISP may be tested. As described herein, a luma intra prediction block may be partitioned vertically or horizontally into 2 or 4 sub-partitions, depending on the block size. The reconstructed sample values ​​of the sub-partitions may be used to generate a prediction for the next sub-partition. Based on different IPMs (e.g., two different IPMs) applied to the sub-partitions (e.g., two sub-partitions) of the SGPM, respectively, for example, if a partition mode (e.g., a specific partition mode) of the SGPM is selected, the ISP may be allowed. If a partition mode of vertical partitioning (e.g., such as Fig. 27 0 / 1 / 36 / 37) and horizontal partitioning (e.g., Fig. 27 If the partition mode in 18 / 19 / 50 / 51) is selected, ISP is allowed.

[0239] The partition direction of the ISP may depend on the partition mode of the SGPM. In an example, if the SGPM is used to partition the current block vertically, different IPMs (e.g., two different IPMs) may be applied to the left subpartition and the right subpartition. In an example, if the ISP is used to partition the block vertically, the right subpartition may not benefit from utilizing the reconstructed samples available in the left subpartition due to their different IPMs. The partition direction of the ISP may be perpendicular to the partition interface of the SGPM. If one of the SGPM vertical partition modes is selected, the block may be horizontally divided into 2 or 4 subpartitions depending on the block size. The syntax isp_mode that signals the horizontal or vertical partition may be skipped.

[0240] Fig.29 is an example flow chart for determining the size of an IPM candidate list based on characteristics (eg, at the decoder side).

[0241] In an example, characteristics may be obtained. These characteristics may be associated with the current block. These characteristics may include one or more of the following: frame resolution, slice type, block size, and / or block shape. In an example, the frame resolution may be 4k. The slice type may be an intra-coded slice, such as an I slice. The slice type may be an inter-coded slice, such as a B slice. The block size may be associated with a width (w) and a height (h). In an example, the block size may be 8×8.

[0242] Based on one or more of a plurality of characteristics (e.g., frame resolution, slice type, block size, and / or block shape), an intra prediction mode candidate list size may be determined for an SGPM associated with a current block. In an example, if the slice type is an intra-coded slice and / or if the frame resolution is 4k, the list size may be set to 3. If the block size is 8×8 or less than 8×8, the list size may be set to 2. An intra prediction mode candidate list for the current block may be obtained based on the determined intra prediction mode candidate list size. In an example, obtaining the list may include: checking whether a potential IPM candidate is already in the list (e.g., a redundancy check); and whether adding a potential candidate will cause the list to exceed its determined candidate list size (e.g., using NUM_VALID_IPM as described herein).

[0243] In an example, based on one or more of a plurality of characteristics, a SGPM candidate list size associated with the current block may be determined. An SGPM candidate list associated with the current block may be obtained based on the determined SGPM candidate list size. The current block may be decoded based on the SGPM candidate list. The characteristics may include one or more of: frame resolution, slice type, block size, or block shape.

[0244] Fig.30 is an example flow chart for determining the size of a partition candidate list based on characteristics (eg, at the decoder side).

[0245] In an example, at 3002, one or more characteristics may be obtained. These characteristics may be associated with the current block. These characteristics may include one or more of the following: frame resolution, slice type, block size, and / or block shape. In an example, the frame resolution may be 4k. The slice type may be an intra-coded slice, such as an I slice. The slice type may be an inter-coded slice, such as a B slice. The block size may be associated with a width (w) and a height (h). In an example, the block size may be 8×8.

[0246] At 3004, based on one or more of a plurality of characteristics (e.g., frame resolution, slice type, block size, and / or block shape), a partition mode candidate list size may be determined for an SGPM associated with the current block. At 3006, a partition mode candidate list for the current block may be obtained based on the determined partition mode candidate list size. At 3008, the current block may be decoded based on the partition mode candidate list.

[0247] Fig.31 is an example flow chart for determining the size of an IPM candidate list based on characteristics (eg, at the encoder side).

[0248] In an example, at 3102, characteristics may be obtained. The characteristics may be associated with the current block. The characteristics may include one or more of: frame resolution, slice type, block size, and / or block shape. In an example, the frame resolution may be 4k. The slice type may be an intra-coded slice, such as an I slice. The slice type may be an inter-coded slice, such as a B slice. The block size may be associated with a width (w) and a height (h). In an example, the block size may be 8×8.

[0249] Based on one or more of a plurality of characteristics (e.g., frame resolution, slice type, block size, and / or block shape), at 3104, an intra-prediction mode candidate list size may be determined for an SGPM associated with the current block. In an example, if the slice type is an intra-coded slice and / or if the frame resolution is 4k, the list size may be set to 3. If the block size is 8×8 or less than 8×8, the list size may be set to 2. At 3106, an intra-prediction mode candidate list for the current block may be obtained based on the determined intra-prediction mode candidate list size. In an example, the generation may include: checking whether a potential IPM candidate is already in the list (e.g., a redundancy check); and whether adding a potential candidate will cause the list to exceed its determined candidate list size (e.g., using NUM_VALID_IPM as described herein).

[0250] In an example, based on one or more of a plurality of characteristics, a SGPM candidate list size associated with the current block may be determined. An SGPM candidate list associated with the current block may be obtained based on the determined SGPM candidate list size. At 3108, the current block may be encoded based on the SGPM candidate list. The characteristics may include one or more of: frame resolution, slice type, block size, or block shape.

[0251] Fig.32 is an example flow chart for determining the size of a partition candidate list based on characteristics (eg, at the encoder side).

[0252] In an example, at 3202, characteristics may be obtained. The characteristics may be associated with the current block. The characteristics may include one or more of: frame resolution, slice type, block size, and / or block shape. In an example, the frame resolution may be 4k. The slice type may be an intra-coded slice, such as an I slice. The slice type may be an inter-coded slice, such as a B slice. The block size may be associated with a width (w) and a height (h). In an example, the block size may be 8×8.

[0253] At 3204, based on one or more of a plurality of characteristics (e.g., frame resolution, slice type, block size, and / or block shape), a partition mode candidate list size may be determined for an SGPM associated with the current block. At 3206, a partition mode candidate list for the current block may be obtained based on the determined partition mode candidate list size. At 3208, the current block may be encoded based on the partition mode candidate list.

[0254] Fig.33 3306, the coding block may be processed (e.g., encoded and / or decoded) based on the first candidate IPM and the second candidate IPM.

[0255] Although the features and elements are described above in specific combinations, it will be understood by those of ordinary skill in the art that each feature or element may be used alone or in any combination with other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (sent via a wired or wireless connection) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (such as built-in hard disks and removable disks), magneto-optical media, and optical media (such as CD-ROM disks and digital versatile disks (DVDs)). A processor associated with the software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host.

Claims

1. A video decoding device, the device comprising a processor: The processor is configured to: deriving a first candidate intra prediction mode (IPM) based on a first template of the decoding block; deriving a second candidate IPM based on a second template of the decoding block; as well as The coding block is decoded based on the first candidate IPM and the second candidate IPM.

2. The apparatus of claim 1, wherein the processor is further configured to: The first candidate IPM and the second candidate IPM are added to an IPM candidate list associated with the coding block, wherein the coding block is decoded based on the IPM candidate list.

3. The apparatus according to any one of claims 1 and 2, wherein the processor is further configured to: selecting a third candidate IPM based on a partition mode of the coding block; and The first candidate IPM, the second candidate IPM, and the third candidate IPM are added to an IPM candidate list associated with the coding block, wherein the coding block is decoded based on the IPM candidate list.

4. The apparatus according to any one of claims 1 to 3, wherein the decoding block comprises a first partition and a second partition, and wherein the processor is further configured to: generating an IPM candidate list associated with the coding block based on the first candidate IPM and the second candidate IPM; and A first prediction mode for the first partition and a second prediction mode for the second partition are determined based on a spatial geometry partition mode (SGPM) index and the IPM candidate list.

5. The apparatus according to any one of claims 1 to 3, wherein the processor is further configured to: determining a SGPM for the decoding block; and Based on determining that the SGPM is for the coding block, the first template and the second template of the coding block are obtained for IPM derivation, wherein the second template is different from the first template.

6. The apparatus according to any one of claims 1 to 3, wherein the processor is further configured to: determining a SGPM for the decoding block; Based on determining that the SGPM is used for the decoding block, obtaining the first template and the second template of the decoding block for IPM derivation; Obtaining a third template of the decoding block including the first template and the second template; as well as A plurality of candidates in an SGPM candidate list associated with the coding block are ordered based on the third template of the coding block.

7. The apparatus according to any one of claims 1 to 3, wherein the processor is further configured to: The first template and the second template of the coding block are determined based on a partition mode associated with the coding block.

8. The apparatus according to any one of claims 1 to 3, wherein the decoding block comprises a first partition and a second partition, and wherein the processor is further configured to: determining the first template of the decoding block based on the first partition; decoding the first partition of the coding block based at least on the first candidate IPM derived based on the first template; determining the second template of the decoding block based on the second partition; as well as The second partition of the coding block is decoded based on at least the second candidate IPM derived based on the second template.

9. The apparatus according to any one of claims 1 to 3, wherein the processor is further configured to: obtaining a left template of the decoding block as the first template; and A top template of the decoding block is obtained as the second template.

10. The apparatus according to any one of claims 1 to 3, wherein the processor is further configured to: Obtaining a left template with a width of 1 of the decoding block as the first template; and A top template with a height of 1 of the decoding block is obtained as the second template.

11. The apparatus according to any one of claims 1 to 3, wherein the processor is configured to derive the first candidate IPM based on the first template further comprising the processor being configured to: Obtaining a plurality of most probable modes (MPMs) associated with the decoding block; determining a plurality of predictions of the first template corresponding to the plurality of MPMs; and The first candidate IPM is selected from the plurality of MPMs based on comparing the first template to the plurality of predictions corresponding to the plurality of most probable modes.

12. A video encoding device, the device comprising a processor: The processor is configured to: deriving a first candidate intra prediction mode (IPM) based on a first template of the decoding block; deriving a second candidate IPM based on a second template of the decoding block; as well as The coding block is encoded based on the first candidate IPM and the second candidate IPM.

13. The apparatus of claim 12, wherein the processor is further configured to: The first candidate IPM and the second candidate IPM are added to an IPM candidate list associated with the coding block, wherein the coding block is encoded based on the IPM candidate list.

14. The apparatus according to any one of claims 12 and 13, wherein the processor is further configured to: selecting a third candidate IPM based on a partition mode of the coding block; and The first candidate IPM, the second candidate IPM, and the third candidate IPM are added to an IPM candidate list associated with the coding block, wherein the coding block is encoded based on the IPM candidate list.

15. The apparatus according to any one of claims 12 to 14, wherein the decoding block comprises a first partition and a second partition, and wherein the processor is further configured to: determining a first prediction mode for the first partition and a second prediction mode for the second partition; generating an IPM candidate list associated with the decoding block based on the first candidate IPM and the second candidate IPM; determining a spatial geometry partitioning mode (SGPM) index based on the first prediction mode and the second prediction mode; as well as The index is included in the IPM candidate list.

16. The apparatus according to any one of claims 12 to 14, wherein the processor is further configured to: determining a SGPM for the decoding block; and Based on determining that the SGPM is for the coding block, the first template and the second template of the coding block are obtained for IPM derivation, wherein the second template is different from the first template.

17. The apparatus according to any one of claims 12 to 14, wherein the processor is further configured to: determining a SGPM for the decoding block; Based on determining that the SGPM is used for the decoding block, obtaining the first template and the second template of the decoding block for IPM derivation; Obtaining a third template of the decoding block including the first template and the second template; sorting a plurality of candidates in a SGPM candidate list associated with the coding block based on the third template of the coding block; and An SGPM index is determined based on the ranked plurality of candidates.

18. The apparatus according to any one of claims 12 to 14, wherein the processor is further configured to: The first template and the second template of the coding block are determined based on a partition mode associated with the coding block.

19. The apparatus according to any one of claims 12 to 14, wherein the decoding block comprises a first partition and a second partition, and wherein the processor is further configured to: determining the first template of the decoding block based on the first partition; encoding the first partition of the coding block based at least on the first candidate IPM derived based on the first template; determining the second template of the decoding block based on the second partition; as well as The second partition of the coding block is encoded based on at least the second candidate IPM derived based on the second template.

20. The apparatus of any one of claims 12 to 14, wherein the processor is configured to derive the first candidate IPM based on the first template comprises the processor being configured to: Obtaining a plurality of most probable modes (MPMs) associated with the decoding block; determining a plurality of predictions of the first template corresponding to the plurality of MPMs; as well as The first candidate IPM is selected from the plurality of MPMs based on comparing the first template to the plurality of predictions corresponding to the plurality of most probable modes.

21. The apparatus according to any one of claims 12 to 14, wherein the processor is further configured to: selecting a third candidate IPM based on a partition mode of the coding block; and Combinations of one or more of the first candidate IPM, the second candidate IPM, and the third candidate IPM are ranked in ascending order based on a sum of absolute differences (SAD) between a prediction of the coding block and a reconstruction of the coding block.

22. The apparatus of any one of claims 1 to 21, further comprising a memory operatively connected to the processor.

23. A method for video decoding, the method comprising: deriving a first candidate intra prediction mode (IPM) based on a first template of the decoding block; deriving a second candidate IPM based on a second template of the decoding block; as well as The coding block is decoded based on the first candidate IPM and the second candidate IPM.

24. The method according to claim 23, wherein the method further comprises: The first candidate IPM and the second candidate IPM are added to an IPM candidate list associated with the coding block, wherein the coding block is decoded based on the IPM candidate list.

25. The method according to any one of claims 23 and 24, wherein the method further comprises: selecting a third candidate IPM based on the partition mode of the decoding block; as well as The first candidate IPM, the second candidate IPM, and the third candidate IPM are added to an IPM candidate list associated with the coding block, wherein the coding block is decoded based on the IPM candidate list.

26. The method according to any one of claims 23 to 25, wherein the decoding block comprises a first partition and a second partition, and wherein the method further comprises: generating an IPM candidate list associated with the decoding block based on the first candidate IPM and the second candidate IPM; as well as A first prediction mode for the first partition and a second prediction mode for the second partition are determined based on a spatial geometry partition mode (SGPM) index and the IPM candidate list.

27. The method according to any one of claims 23 to 25, wherein the method further comprises: determining a SGPM for the decoding block; as well as Based on determining that the SGPM is for the coding block, the first template and the second template of the coding block are obtained for IPM derivation, wherein the second template is different from the first template.

28. The method according to any one of claims 23 to 25, wherein the method further comprises: determining a SGPM for the decoding block; Based on determining that the SGPM is used for the decoding block, obtaining the first template and the second template of the decoding block for IPM derivation; Obtaining a third template of the decoding block including the first template and the second template; as well as A plurality of candidates in an SGPM candidate list associated with the coding block are ordered based on the third template of the coding block.

29. The method according to any one of claims 23 to 25, wherein the method further comprises: The first template and the second template of the coding block are determined based on a partition mode associated with the coding block.

30. The method according to any one of claims 23 to 25, wherein the decoding block comprises a first partition and a second partition, and wherein the method further comprises: determining the first template of the decoding block based on the first partition; decoding the first partition of the coding block based at least on the first candidate IPM derived based on the first template; determining the second template of the decoding block based on the second partition; as well as The second partition of the coding block is decoded based on at least the second candidate IPM derived based on the second template.

31. The method according to any one of claims 23 to 25, wherein the method further comprises: Obtaining a left template of the decoding block as the first template; as well as A top template of the decoding block is obtained as the second template.

32. The method according to any one of claims 23 to 25, wherein the method further comprises: Obtaining a left template with a width of 1 of the decoding block as the first template; as well as A top template with a height of 1 of the decoding block is obtained as the second template.

33. The method according to any one of claims 23 to 25, wherein deriving the first candidate IPM based on the first template further comprises: Obtaining a plurality of most probable modes (MPMs) associated with the decoding block; determining a plurality of predictions of the first template corresponding to the plurality of MPMs; as well as The first candidate IPM is selected from the plurality of MPMs based on comparing the first template to the plurality of predictions corresponding to the plurality of most probable modes.

34. A method for video encoding, the method comprising: deriving a first candidate intra prediction mode (IPM) based on a first template of the decoding block; deriving a second candidate IPM based on a second template of the decoding block; as well as The coding block is encoded based on the first candidate IPM and the second candidate IPM.

35. The method of claim 34, wherein the method further comprises: The first candidate IPM and the second candidate IPM are added to an IPM candidate list associated with the coding block, wherein the coding block is encoded based on the IPM candidate list.

36. The method according to any one of claims 34 and 35, wherein the method further comprises: selecting a third candidate IPM based on the partition mode of the decoding block; as well as The first candidate IPM, the second candidate IPM, and the third candidate IPM are added to an IPM candidate list associated with the coding block, wherein the coding block is encoded based on the IPM candidate list.

37. The method according to any one of claims 34 to 36, wherein the decoding block comprises a first partition and a second partition, and wherein the method further comprises: determining a first prediction mode for the first partition and a second prediction mode for the second partition; generating an IPM candidate list associated with the decoding block based on the first candidate IPM and the second candidate IPM; determining a spatial geometry partitioning mode (SGPM) index based on the first prediction mode and the second prediction mode; as well as The index is included in the IPM candidate list.

38. The method according to any one of claims 34 to 36, wherein the method further comprises: determining a SGPM for the decoding block; as well as Based on determining that the SGPM is for the coding block, the first template and the second template of the coding block are obtained for IPM derivation, wherein the second template is different from the first template.

39. The method according to any one of claims 34 to 36, wherein the method further comprises: determining a SGPM for the decoding block; Based on determining that the SGPM is used for the decoding block, obtaining the first template and the second template of the decoding block for IPM derivation; Obtaining a third template of the decoding block including the first template and the second template; sorting a plurality of candidates in a SGPM candidate list associated with the coding block based on the third template of the coding block; and An SGPM index is determined based on the ranked plurality of candidates.

40. The method according to any one of claims 34 to 36, wherein the method further comprises: The first template and the second template of the coding block are determined based on a partition mode associated with the coding block.

41. The method according to any one of claims 34 to 36, wherein the decoding block comprises a first partition and a second partition, and wherein the method further comprises: determining the first template of the decoding block based on the first partition; encoding the first partition of the coding block based at least on the first candidate IPM derived based on the first template; determining the second template of the decoding block based on the second partition; as well as The second partition of the coding block is encoded based on at least the second candidate IPM derived based on the second template.

42. The method of any one of claims 34 to 36, wherein deriving the first candidate IPM based on the first template further comprises: Obtaining a plurality of most probable modes (MPMs) associated with the decoding block; determining a plurality of predictions of the first template corresponding to the plurality of MPMs; as well as The first candidate IPM is selected from the plurality of MPMs based on comparing the first template to the plurality of predictions corresponding to the plurality of most probable modes.

43. The method according to any one of claims 34 to 36, wherein the method further comprises: selecting a third candidate IPM based on the partition mode of the decoding block; as well as Combinations of one or more of the first candidate IPM, the second candidate IPM, and the third candidate IPM are ranked in ascending order based on a sum of absolute differences (SAD) between a prediction of the coding block and a reconstruction of the coding block.

44. A computer program product stored on a non-transitory computer readable medium and comprising program code instructions for implementing the steps of the method according to any one of claims 23 to 43 when executed by a processor.

45. A computer program comprising program code instructions for implementing the steps of the method according to any one of claims 23 to 43 when executed by a processor.

46. ​​Video data comprising information representing a decoded block encoded according to one of the methods of any one of claims 34 to 43.

Images