Classification-based post-filter rescaling

Abstract

A codec can perform classification-based post-filtering rescaling. A scaling parameter can be adapted to an associated classification. The scaling parameter supports modification of post-filtering correction. A classifier can reuse an existing classifier used by post-filtering or another classifier. An adaptive classifier can support: classification-based scaling correction; neural network (NN)-based post-filtering with classification-based scaling correction; and / or signaling of a scaling parameter. A codec can determine a classification associated with a reconstructed picture sample. A scaling parameter associated with the classification can be determined. A scaling correction can be determined based on the scaling parameter. The scaling correction can be applied to the reconstructed picture sample. A codec can perform neural network processing to determine a classification associated with a reconstructed picture sample. A scaling parameter associated with the classification can be determined. A scaling correction can be determined based on the scaling parameter. The scaling correction can be applied to the reconstructed picture sample.

Description

[0001] Cross-reference to related applications This application claims the benefit of European Patent Application No. 24305563.9, filed on 8 April 2024, the contents of which are incorporated herein by reference in their entirety. Background Technology

[0002] Video codec systems can be used to compress digital video signals, for example, to reduce the storage and / or transmission bandwidth required for such signals. Video codec systems can include, for example, block-based, wavelet-based, and / or object-based systems. Summary of the Invention

[0003] Systems, methods, and tools for a device configured for one or more of video encoding / decoding (e.g., encoding and / or decoding), distribution, or consumption are disclosed.

[0004] Systems, methods, and tools for classification-based post-filter rescaling are disclosed. Scaling parameters can be adapted to the associated classification. Scaling parameters can allow modification of the post-filter correction. The classifier can reuse an existing classifier that can be used by the post-filter or another classifier. The adaptive classifier can provide one or more of the following: classification-based scaling correction; neural network (NN)-based post-filtering with classification-based scaling correction; and / or signaling of the scaling parameters.

[0005] A device can be configured to provide classification-based scaling correction. The device (e.g., an encoder for video encoding or a decoder for video decoding) can determine corrections for reconstructed samples, such as filter corrections. The device can determine the classification associated with the reconstructed image sample. The device can determine the classification associated with the reconstructed image sample by determining the classification index associated with the reconstructed image sample. The determined classification may be a classification previously used to determine post-filtering.

[0006] This device can determine one or more scaling parameters associated with classification. These scaling parameters can indicate scaling values ​​and / or offset values. The device can determine scaling corrections based on these scaling parameters and corrections. The device can then apply these scaling corrections to reconstructed image samples.

[0007] In the example, the device can determine a classification associated with filtering. The device can, for example, determine one or more filtering parameters based on the classification associated with filtering. The device can determine to use the classification associated with filtering as the classification associated with the reconstructed image sample.

[0008] In the example, the device can determine that a category is associated with a minimum and a maximum value. The device can also determine one or more scaling parameters between the minimum and maximum values.

[0009] A device can be configured to provide neural network-based post-filtering with classification-based correction. The device (e.g., an encoder for video encoding or a decoder for video decoding) can perform neural network processing to determine the correction for reconstructed image samples. The device can determine the classification associated with the reconstructed image samples. The device can determine the classification index associated with the reconstructed image samples.

[0010] This device can determine one or more scaling parameters associated with classification. It can then determine scaling corrections based on these scaling parameters and the resulting corrections. Finally, it can apply these scaling corrections to reconstructed image samples.

[0011] In the example, the device can perform neural network-based filtering on reconstructed image samples to generate filtered reconstructed image samples. The device can then determine scaling corrections from the filtered reconstructed image samples.

[0012] In the example, the device can determine scaling correction from the filtered reconstructed image sample and one or more scaling parameters.

[0013] A device can be configured to provide signaling for scaling parameters. The device (e.g., an encoder for video encoding) can determine a correction for a reconstructed image sample. The device can determine a classification associated with the reconstructed image sample. The device can determine one or more scaling parameters associated with the classification. The device can determine a scaling correction based on one or more scaling parameters and the correction. The device can apply the scaling correction to the reconstructed image sample. The device can insert instructions associated with one or more scaling parameters into the bitstream of the reconstructed image sample.

[0014] In the example, the device can determine an index value associated with one or more scaling parameters, where the index value is associated with a scaling parameter value table. The device can then insert the index value associated with the scaling parameter value table into the bitstream. The scaling parameter value table can be one of multiple tables that include scaling parameter values.

[0015] In the example, reconstructed image samples can be associated with codec parameters such as, for example, time identifier index, image sequence count (POC), slice type, and / or quantizer parameters (QP). The scaling parameter value table can reflect scaling parameter values ​​associated with previous frames linked to a time identifier index or another codec parameter.

[0016] In the example, the device can determine that one or more scaling parameters should not be applied to the second reconstructed image sample. The device can insert an index value indicating that scaling parameters should not be applied to the second reconstructed image sample into the bitstream.

[0017] In the example, the device can determine a combination of index values ​​associated with one or more scaling parameters. The device can then insert this combination of index values ​​associated with one or more scaling parameters into the bitstream.

[0018] In this example, the device can be configured to determine a classification associated with filtering. The device can, for example, determine one or more filtering parameters based on the classification associated with filtering. The device can determine that the classification associated with the reconstructed image sample is equal to the classification associated with filtering.

[0019] In the example, the device can determine that a category is associated with a minimum and a maximum value. The device can also determine one or more scaling parameters between the minimum and maximum values.

[0020] A device (e.g., a decoder for video decoding) can be configured to receive a bitstream of video data. For a reconstructed image sample, the bitstream may include an indication associated with one or more scaling parameters. The device can determine one or more scaling parameters based on the indication. The device can determine a correction for the reconstructed sample. The device can determine a scaling correction based on one or more scaling parameters and the correction. The device can apply the scaling correction to the reconstructed image sample.

[0021] In the example, the indication associated with one or more scaling parameters may include an index value associated with a scaling parameter value table. The device can retrieve one or more scaling parameters from the scaling parameter value table.

[0022] In the example, the scaling parameter value table can be one of multiple tables that include scaling parameter values.

[0023] In the example, reconstructed image samples can be associated with codec parameters such as, for example, time identifier index, image sequence count (POC), slice type, or quantizer parameter (QP). In the example, the scaling parameter value table can reflect scaling parameter values ​​associated with previous frames linked to the codec parameters.

[0024] In the example, the bitstream may (e.g., further) include a second indication associated with the second reconstructed image sample. This second indication associated with the second reconstructed image sample may indicate that scaling parameters are not applied to the second reconstructed image. The device can then correct the second reconstructed image sample without applying scaling parameters.

[0025] In the example, the indication associated with one or more scaling parameters may include a combination of index values ​​associated with one or more scaling parameters. The device can retrieve one or more scaling parameters from at least one scaling parameter value table based on the combination of index values. Attached Figure Description

[0026] Figure 1AThis is a system diagram illustrating an example communication system that can implement one or more of the disclosed embodiments.

[0027] Figure 1B The illustration shows that, according to the embodiment, it is possible to... Figure 1A The diagram shows a system diagram of an example wireless transmit / receive unit (WTRU) used in a communication system.

[0028] Figure 1C The illustration shows that, according to the embodiment, it is possible to... Figure 1A The diagram illustrates a system diagram of an example radio access network (RAN) and an example core network (CN) used within a communication system.

[0029] Figure 1D The illustration shows that, according to the embodiment, it is possible to... Figure 1A The diagram shows another example RAN and another example CN used in the communication system.

[0030] Figure 2 The illustration shows a sample video encoder.

[0031] Figure 3 The illustration shows an example video decoder.

[0032] Figure 4 The illustration shows an example of a system that can implement various aspects and examples.

[0033] Figure 5 The diagram illustrates a flowchart of an example of a classification-based post-filtering process.

[0034] Figure 6 The diagram illustrates a flowchart of an example of a post-filtering process with rescaling correction.

[0035] Figure 7 An example of the SAO BO mode is illustrated, where the pixel range of 0 to 255 (8 bits) can be evenly divided into 32 frequency bands.

[0036] Figure 8 The diagram illustrates a flowchart of an example of a classification-based scaling correction process.

[0037] Figure 9 The diagram illustrates a flowchart of an example of post-filtering with classification-based scaling correction. Detailed Implementation

[0038] A more detailed understanding can be obtained from the following description given with reference to the accompanying drawings and examples.

[0039] Figure 1AThis diagram illustrates an example communication system 100 in which one or more of the disclosed embodiments may be implemented. The communication system 100 may be a multiple access system that provides content (such as voice, data, video, messaging, broadcasting, etc.) to multiple wireless users. The communication system 100 enables multiple wireless users to access such content by sharing system resources (including wireless bandwidth). For example, the communication system 100 may employ one or more channel access methods, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal FDMA (OFDMA), Single Carrier FDMA (SC-FDMA), Zero-Tail Unique Word DFT Extended OFDM (ZT UW DTS-s OFDM), Unique Word OFDM (UW-OFDM), Resource Block Filtered OFDM, Filter Bank Multicarrier (FBMC), etc.

[0040] like Figure 1A As shown, the communication system 100 may include wireless transceiver units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, public switched telephone network (PSTN) 108, Internet 110, and other networks 112. Although it will be appreciated, the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, 102d can be any type of device configured to operate and / or communicate in a wireless environment. For example, WTRUs 102a, 102b, 102c, and 102d (any of which may be referred to as a “station” and / or “STA”) may be configured to transmit and / or receive wireless signals and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscription-based units, pagers, cellular phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated processing chain scenarios), consumer electronics devices, devices operating on commercial and / or industrial wireless networks, etc. Any of WTRUs 102a, 102b, 102c, and 102d may be interchangeably referred to as a UE.

[0041] The communication system 100 may also include base station 114a and / or base station 114b. Each of base stations 114a and 114b may be any type of device configured to wirelessly interface with at least one of WTRUs 102a, 102b, 102c, and 102d to facilitate access to one or more communication networks, such as CN 106 / 115, the Internet 110, and / or other networks 112. For example, base stations 114a and 114b may be base transceiver stations (BTS), Node-B, eNode B, home node B, home eNode B, gNB, NR NodeB, site controller, access point (AP), wireless router, etc. Although base stations 114a and 114b are depicted as single elements, it will be understood that base stations 114a and 114b may include any number of interconnected base stations and / or network elements.

[0042] Base station 114a may be part of RAN 104 / 113, which may also include other base stations and / or network elements (not shown), such as base station controllers (BSCs), radio network controllers (RNCs), relay nodes, etc. Base station 114a and / or base station 114b may be configured to transmit and / or receive radio signals on one or more carrier frequencies, which may be referred to as cells (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage for a specific geographic area for radio services, which may be relatively fixed or may change over time. A cell may be further divided into cell sectors. For example, the cell associated with base station 114a may be divided into three sectors. Therefore, in one embodiment, base station 114a may include three transceivers, i.e., one transceiver for each sector of the cell. In one embodiment, base station 114a may employ multiple-input multiple-output (MIMO) technology, and multiple transceivers may be used for each sector of the cell. For example, beamforming can be used to transmit and / or receive signals in a desired spatial direction.

[0043] Base stations 114a and 114b can communicate with one or more of WTRUs 102a, 102b, 102c, and 102d via air interface 116, which can be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). Air interface 116 can be established using any suitable radio access technology (RAT).

[0044] More specifically, as noted above, the communication system 100 can be a multiple access system and can employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, base station 114a in RAN 104 / 113, and WTRUs 102a, 102b, and 102c, can implement radio technologies such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which can use Wideband CDMA (WCDMA) to establish air interfaces 115 / 116 / 117. WCDMA can include communication protocols such as High-Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA can include High-Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High-Speed ​​UL Packet Access (HSUPA).

[0045] In one embodiment, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which can use Long Term Evolution (LTE) and / or LTE-A Advanced (LTE-A) and / or LTE-A Pro Advanced (LTE-A Pro) to establish air interface 116.

[0046] In one embodiment, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as NR radio access, which can use a new radio (NR) to establish an air interface 116.

[0047] In one embodiment, base station 114a and WTRUs 102a, 102b, and 102c can implement multiple radio access technologies. For example, base station 114a and WTRUs 102a, 102b, and 102c can jointly implement LTE radio access and NR radio access, for example, using the dual connectivity (DC) principle. Therefore, the air interface utilized by WTRUs 102a, 102b, and 102c can be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations (e.g., eNBs and gNBs).

[0048] In other embodiments, base station 114a and WTRUs 102a, 102b, 102c can implement the following radio technologies, such as IEEE 802.11 (i.e., WiFi), IEEE 802.16 (i.e., WiMAX), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Provisional Standard 2000 (IS-2000), Provisional Standard 95 (IS-95), Provisional Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data Rate GSM Evolution (EDGE), GSMEDGE (GERAN), etc.

[0049] Figure 1A Base station 114b can be, for example, a wireless router, a home node B, a home eNode B, or an access point, and can utilize any suitable RAT to facilitate wireless connectivity in a local area, such as a commercial area, home, vehicle, campus, industrial facility, air corridor (e.g., for drone use), road, etc. In one embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.11 to establish a wireless local area network (WLAN). In one embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, base station 114b and WTRUs 102c, 102d can utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish a picocell or femtocell. Figure 1A As shown, base station 114b can have a direct connection to Internet 110. Therefore, base station 114b may not need to access Internet 110 via CN 106 / 115.

[0050] RAN 104 / 113 can communicate with CN 106 / 115, which can be any type of network configured to provide voice, data, application, and / or Voice over Internet Protocol (VoIP) services to one or more of WTRUs 102a, 102b, 102c, and 102d. Data may have different Quality of Service (QoS) requirements, such as different throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. CN 106 / 115 can provide call control, billing services, location-based services, prepaid calling, internet connectivity, video distribution, etc., and / or perform advanced security functions, such as user authentication. Although... Figure 1AAlthough not shown, it will be understood that RAN104 / 113 and / or CN 106 / 115 can communicate directly or indirectly with other RANs that use the same RAT as RAN 104 / 113 or a different RAT. For example, in addition to being connected to RAN 104 / 113, which can utilize NR radio technology, CN106 / 115 can also communicate with another RAN (not shown) that uses GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technology.

[0051] CN 106 / 115 can also act as a gateway for WTRU 102a, 102b, 102c, 102d to access PSTN 108, the Internet 110, and / or other networks 112. PSTN 108 may include a circuit-switched telephone network providing Common Old-Style Telephone Service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices using common communication protocols such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and / or Internet Protocol (IP) from the TCP / IP Internet Protocol suite. Network 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, network 112 may include another CN connected to one or more RANs, which may use the same RAT as RAN 104 / 113 or a different RAT.

[0052] Some or all of the WTRUs 102a, 102b, 102c, and 102d in communication system 100 may include multi-mode capabilities (e.g., WTRUs 102a, 102b, 102c, and 102d may include multiple transceivers for communicating with different wireless networks via different wireless links). For example, Figure 1A The WTRU 102c shown can be configured to communicate with base station 114a, which can use cellular-based radio technology, and base station 114b, which can use IEEE 802 radio technology.

[0053] Figure 1B This is a system diagram illustrating example WTRU 102. (Example:) Figure 1B As shown, WTRU 102 may include a processor 118, a transceiver 120, a transmitting / receiving element 122, a speaker / microphone 124, a keyboard 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power supply 134, a Global Positioning System (GPS) chipset 136, and / or other peripheral devices 138, etc. It will be appreciated that WTRU 102 may include any sub-combination of the above-described elements while remaining consistent with the embodiments.

[0054] Processor 118 may be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. Processor 118 may perform signal encoding, data processing, power control, input / output processing, and / or any other functions that enable WTRU 102 to operate in a wireless environment. Processor 118 may be coupled to transceiver 120, which may be coupled to transmitting / receiving element 122. Although Figure 1B The processor 118 and transceiver 120 are depicted as separate components, but it will be understood that the processor 118 and transceiver 120 can be integrated together in an electronic package or chip.

[0055] Transmitting / receiving element 122 can be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) via air interface 116. For example, in one embodiment, transmitting / receiving element 122 can be an antenna configured to transmit and / or receive RF signals. In one embodiment, transmitting / receiving element 122 can be a transmitter / detector configured to transmit and / or receive, for example, IR, UV, or visible light signals. In yet another embodiment, transmitting / receiving element 122 can be configured to transmit and / or receive both RF and optical signals. It will be appreciated that transmitting / receiving element 122 can be configured to transmit and / or receive any combination of wireless signals.

[0056] Although the transmitting / receiving element 122 is in Figure 1B While depicted as a single element, WTRU 102 may include any number of transmitting / receiving elements 122. More specifically, WTRU 102 may employ MIMO technology. Thus, in one embodiment, WTRU 102 may include two or more transmitting / receiving elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals via air interface 116.

[0057] Transceiver 120 can be configured to modulate signals to be transmitted by transmitting / receiving element 122 and demodulate signals received by transmitting / receiving element 122. As noted above, WTRU 102 can have multi-mode capability. Thus, for example, transceiver 120 may include multiple transceivers for enabling WTRU 102 to communicate, for example, via multiple RATs (such as NR and IEEE 802.11).

[0058] The processor 118 of WTRU 102 can be coupled to a speaker / microphone 124, a keyboard 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) unit or an organic light-emitting diode (OLED) display unit) and can receive user input data from them. The processor 118 can also output user data to the speaker / microphone 124, keyboard 126, and / or display / touchpad 128. Additionally, the processor 118 can access information from any type of suitable memory (such as non-removable memory 130 and / or removable memory 132) and store data in that memory. Non-removable memory 130 may include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. Removable memory 132 may include a subscriber identity module (SIM) card, memory stick, secure digital storage (SD) card, etc. In other embodiments, the processor 118 can access information from memory that is not physically located on WTRU 102 (such as on a server or home computer (not shown)) and store data in that memory.

[0059] The processor 118 can receive power from the power supply 134 and can be configured to distribute and / or control the power going to other components in the WTRU 102. The power supply 134 can be any suitable device for powering the WTRU 102. For example, the power supply 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0060] The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) about the current location of the WTRU 102. In addition to, or instead of, information from the GPS chipset 136, the WTRU 102 may receive location information from base stations (e.g., base stations 114a, 114b) via the air interface 116 and / or determine its location based on the timing of signals received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information using any suitable location determination method, while remaining consistent with the embodiments.

[0061] The processor 118 may be further coupled to other peripheral devices 138, which may include one or more software and / or hardware modules providing additional features, functions, and / or wired or wireless connectivity. For example, peripheral devices 138 may include accelerometers, electronic compasses, satellite transceivers, digital cameras (for photos and / or video), Universal Serial Bus (USB) ports, vibration devices, television transceivers, hands-free headsets, Bluetooth® modules, FM radio units, digital music players, media players, video game player modules, internet browsers, virtual reality and / or augmented reality (VR / AR) devices, activity trackers, etc. Peripheral devices 138 may include one or more sensors, which may be one or more of the following: gyroscopes, accelerometers, Hall effect sensors, magnetometers, orientation sensors, proximity sensors, temperature sensors, time sensors, geolocation sensors, altimeters, light sensors, touch sensors, magnetometers, barometers, gesture sensors, biometric sensors, and / or humidity sensors.

[0062] WTRU 102 may include a full-duplex radio, for which the transmission and reception of some or all signals (e.g., associated with specific subframes for both UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous. The full-duplex radio may include an interference management unit to reduce and / or substantially eliminate self-interference via signal processing performed via hardware (e.g., a choke) or via a processor (e.g., a separate processor (not shown) or via processor 118). In one embodiment, WTRU 102 may include a half-duplex radio, for which the transmission and reception of some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) or downlink (e.g., for reception)) may be concurrent and / or simultaneous.

[0063] Figure 1C The diagram illustrates a system diagram of RAN 104 and CN 106 according to one embodiment. As indicated above, RAN 104 employs E-UTRA radio technology to communicate with WTRUs 102a, 102b, and 102c via air interface 116. RAN 104 can also communicate with CN 106.

[0064] RAN 104 may include eNode-Bs 160a, 160b, and 160c, although it will be understood that RAN 104 may include any number of eNode-Bs while remaining consistent with the embodiments. eNode-Bs 160a, 160b, and 160c may each include one or more transceivers for communicating with WTRUs 102a, 102b, and 102c via air interface 116. In one embodiment, eNode-Bs 160a, 160b, and 160c may implement MIMO technology. Therefore, eNode-B 160a may, for example, use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 102a.

[0065] Each of the eNode-B 160a, 160b, and 160c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, and user scheduling in the UL and / or DL, etc. Figure 1C As shown, eNode-B 160a, 160b, and 160c can communicate with each other via the X2 interface.

[0066] Figure 1C The CN 106 shown may include a Mobility Management Entity (MME) 162, a Serving Gateway (SGW) 164, and a Packet Data Network (PDN) Gateway (or PGW) 166. While each of the foregoing elements is depicted as part of CN 106, it will be understood that any of these elements may be owned and / or operated by an entity other than a CN operator.

[0067] The MME 162 can connect to each of the eNode-Bs 160a, 160b, and 160c in RAN 104 via the S1 interface and can act as a control node. For example, the MME 162 can be responsible for authenticating users of WTRUs 102a, 102b, and 102c, bearer activation / deactivation, selecting a specific serving gateway during the initial attachment of WTRUs 102a, 102b, and 102c, etc. The MME 162 can provide control plane functions for handover between RAN 104 and other RANs (not shown) employing other radio technologies (such as GSM and / or WCDMA).

[0068] The SGW 164 can connect to each of the eNode Bs 160a, 160b, and 160c in RAN 104 via the S1 interface. The SGW 164 can typically route and forward user data packets to / from WTRUs 102a, 102b, and 102c. The SGW 164 can perform other functions, such as anchoring the user plane during inter-eNode B handover, triggering paging when DL data is available to WTRUs 102a, 102b, and 102c, and managing and storing the context of WTRUs 102a, 102b, and 102c.

[0069] SGW 164 can be connected to PGW 166, which can provide WTRU 102a, 102b, 102c with access to a packet-switched network (such as Internet 110) to facilitate communication between WTRU 102a, 102b, 102c and IP-enabled devices.

[0070] CN 106 facilitates communication with other networks. For example, CN 106 can provide WTRUs 102a, 102b, and 102c with access to a circuit-switched network (such as PSTN 108) to facilitate communication between WTRUs 102a, 102b, and 102c and conventional terrestrial line communication equipment. For example, CN 106 may include or communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that acts as an interface between CN 106 and PSTN 108. Additionally, CN 106 can provide WTRUs 102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0071] Despite WTRU in Figures 1A to 1D While described as a wireless terminal, it is envisioned that, in some representative embodiments, such a terminal may use (e.g., temporarily or permanently) a wired communication interface with a communication network.

[0072] In a representative embodiment, the other network 112 may be a WLAN.

[0073] In Infrastructure Basic Services Set (BSS) mode, a WLAN may have an access point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP may have access or an interface to a distribution system (DS) or another type of wired / wireless network that carries traffic into and / or out of the BSS. Traffic originating outside the BSS destined for a STA can be reached via the AP and delivered to the STA. Traffic from a STA to a destination outside the BSS can be sent to the AP for delivery to the appropriate destination. Traffic between STAs within the BSS can be sent via the AP, for example, where a source STA can send traffic to the AP, and the AP can deliver the traffic to the destination STA. Traffic between STAs within the BSS can be considered and / or referred to as peer-to-peer traffic. Peer-to-peer traffic can be sent between source and destination STAs (e.g., directly between them) using a direct link setup (DLS). In some representative embodiments, the DLS may use 802.11e DLS or 802.11z Tunneled DLS (TDLS). A WLAN using the Standalone BSS (IBSS) mode can exist without an access point (AP), and STAs within the IBSS or using the IBSS (e.g., all STAs) can communicate directly with each other. The IBSS communication mode may sometimes be referred to as the "ad-hoc" communication mode in this document.

[0074] When operating in 802.11ac infrastructure mode or a similar mode, the AP can transmit beacons on a fixed channel, such as a primary channel. The primary channel can be of fixed width (e.g., a 20 MHz bandwidth) or dynamically set via signaling. The primary channel can be the operating channel of the BSS and can be used by the STA to establish a connection with the AP. In some representative embodiments, Carrier-Sensed Multiple Access with Collision Avoidance (CSMA / CA) can be implemented, for example, in an 802.11 system. For CSMA / CA, STAs including the AP (e.g., each STA) can sense the primary channel. If the primary channel is sensed / detected and / or determined to be busy by a particular STA, that STA can back off. A single STA (e.g., only one station) can transmit in a given BSS at any given time.

[0075] High-throughput (HT) STAs can communicate using a 40MHz wide channel, for example, by combining a primary 20MHz channel with adjacent or non-adjacent 20MHz channels to form a 40MHz wide channel.

[0076] Very High Throughput (VHT) STAs can support channels with widths of 20MHz, 40MHz, 80MHz, and / or 160MHz. 40MHz and / or 80MHz channels can be formed by combining consecutive 20MHz channels. A 160MHz channel can be formed by combining eight consecutive 20MHz channels or by combining two non-consecutive 80MHz channels (which can be referred to as an 80+80 configuration). For the 80+80 configuration, after channel coding, data is passed through a segment resolver that divides the data into two streams. Inverse Fast Fourier Transform (IFFT) processing and time-domain processing can be performed separately on each stream. The streams can be mapped onto two 80MHz channels, and the data can be transmitted by the STA performing the transmission. At the receiver of the receiving STA, the above operations for the 80+80 configuration can be reversed, and the combined data can be sent to the Media Access Control (MAC).

[0077] Operating modes below 1 GHz are supported by 802.11af and 802.11ah. The channel operating bandwidth and carrier used in 802.11af and 802.11ah are reduced compared to those used in 802.11n and 802.11ac. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV white space (TVWS) spectrum, while 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to representative embodiments, 802.11ah can support instrument-type control / machine-type communications, such as MTC devices in macro coverage areas. MTC devices may have certain capabilities, such as limited capabilities, including support for (e.g., only support) certain and / or limited bandwidths. MTC devices may include batteries with a lifespan exceeding a threshold (e.g., to maintain a very long battery life).

[0078] WLAN systems that can support multiple channels and channel bandwidths (such as 802.11n, 802.11ac, 802.11af, and 802.11ah) include a channel that can be designated as the primary channel. The primary channel can have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel can be set and / or limited by the STA that supports the minimum bandwidth operating mode among all STAs operating in the BSS. In the 802.11ah example, for STAs that support (e.g., only support) the 1MHz mode (e.g., MTC type devices), the primary channel can be 1MHz wide, even if the AP and other STAs in the BSS support 2MHz, 4MHz, 8MHz, 16MHz, and / or other channel bandwidth operating modes. Carrier sensing and / or Network Allocation Vector (NAV) settings can depend on the status of the primary channel. If the primary channel is busy, for example because an STA (which only supports the 1MHz operating mode) is transmitting to the AP, the entire available band may be considered busy, even if most of the band is still idle and available.

[0079] In the United States, the available frequency band for 802.11ah is from 902MHz to 928MHz. In South Korea, the available frequency band is from 917.5MHz to 923.5MHz. In Japan, the available frequency band is from 916.5MHz to 927.5MHz. Depending on the country code, the total available bandwidth for 802.11ah ranges from 6MHz to 26MHz.

[0080] Figure 1D The diagram illustrates a system diagram of RAN 113 and CN 115 according to one embodiment. As indicated above, RAN 113 may employ NR radio technology to communicate with WTRUs 102a, 102b, and 102c via air interface 116. RAN 113 may also communicate with CN 115.

[0081] RAN 113 may include gNBs 180a, 180b, and 180c, although it will be understood that RAN 113 may include any number of gNBs while remaining consistent with the embodiments. gNBs 180a, 180b, and 180c may each include one or more transceivers for communicating with WTRUs 102a, 102b, and 102c via air interface 116. In one embodiment, gNBs 180a, 180b, and 180c may implement MIMO technology. For example, gNBs 180a and 180b may utilize beamforming to transmit signals to and / or receive signals from gNBs 180a, 180b, and 180c. Therefore, gNB 180a may, for example, use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 102a. In one embodiment, gNBs 180a, 180b, and 180c can implement carrier aggregation technology. For example, gNB 180a can transmit multiple component carriers to WTRU 102a (not shown). A subset of these component carriers can be on unlicensed spectrum, while the remaining component carriers can be on licensed spectrum. In one embodiment, gNBs 180a, 180b, and 180c can implement Cooperative Multipoint (CoMP) technology. For example, WTRU 102a can receive cooperative transmissions from gNBs 180a and 180b (and / or gNB 180c).

[0082] WTRUs 102a, 102b, and 102c can use transmissions associated with scalable digitization. For example, OFDM symbol spacing and / or OFDM subcarrier spacing can vary depending on different transmissions, different cells, and / or different portions of the radio transmission spectrum. WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using subframes of various or scalable lengths or transmission time intervals (TTIs) (e.g., containing different numbers of OFDM symbols and / or absolute times of varying durations).

[0083] gNBs 180a, 180b, and 180c can be configured to communicate with WTRUs 102a, 102b, and 102c in standalone and / or non-standalone configurations. In standalone configuration, WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c without access to other RANs (e.g., eNode-Bs 160a, 160b, and 160c). In standalone configuration, WTRUs 102a, 102b, and 102c can utilize one or more of gNBs 180a, 180b, and 180c as mobility anchors. In standalone configuration, WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using signals in unlicensed frequency bands. In a non-standalone configuration, WTRUs 102a, 102b, and 102c can communicate with / connect to gNBs 180a, 180b, and 180c, and simultaneously communicate with / connect to another RAN (such as eNode-Bs 160a, 160b, and 160c). For example, WTRUs 102a, 102b, and 102c can implement DC principles to communicate substantially simultaneously with one or more gNBs 180a, 180b, and 180c and one or more eNode-Bs 160a, 160b, and 160c. In a non-standalone configuration, eNode-B 160a, 160b, and 160c can act as mobility anchors for WTRU 102a, 102b, and 102c, and gNB 180a, 180b, and 180c can provide additional coverage and / or throughput to serve WTRU 102a, 102b, and 102c.

[0084] Each of gNBs 180a, 180b, and 180c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, network slicing support, dual connectivity, networking between NR and E-UTRA, routing of user plane data to User Plane Functions (UPF) 184a and 184b, routing of control plane information to Access and Mobility Management Functions (AMF) 182a and 182b, etc. Figure 1D As shown, gNB 180a, 180b, and 180c can communicate with each other via the Xn interface.

[0085] Figure 1DThe CN 115 shown may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one Session Management Function (SMF) 183a, 183b, and possibly a Data Network (DN) 185a, 185b. While each of the foregoing elements is depicted as part of the CN 115, it will be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0086] AMF 182a and 182b can connect to one or more of the gNBs 180a, 180b, and 180c in RAN 113 via the N2 interface and can act as control nodes. For example, AMF 182a and 182b can be responsible for authenticating users of WTRU 102a, 102b, and 102c, supporting network slicing (e.g., handling different PDU sessions with different requirements), selecting specific SMF183a and 183b, managing registration areas, terminating NAS signaling, mobility management, etc. Network slices can be used by AMF 182a and 182b to customize CN support for WTRU 102a, 102b, and 102c based on the service types utilized by WTRU 102a, 102b, and 102c. For example, different network slices can be built for different use cases, such as services that rely on Ultra Reliable Low Latency (URLLC) access, services that rely on Enhanced Massive Mobile Broadband (eMBB) access, and services for Machine Type Communication (MTC) access. AMF 162 can provide control plane functions for handover between RAN 113 and other RANs (not shown) employing other radio technologies, such as LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies, such as WiFi.

[0087] SMFs 183a and 183b can connect to AMFs 182a and 182b in CN 115 via the N11 interface. SMFs 183a and 183b can also connect to UPFs 184a and 184b in CN 115 via the N4 interface. SMFs 183a and 183b can select and control UPFs 184a and 184b, and configure service routes through UPFs 184a and 184b. SMFs 183a and 183b can perform other functions, such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notifications. PDU session types can be IP-based, non-IP-based, Ethernet-based, etc.

[0088] UPF 184a and 184b can connect to one or more of gNBs 180a, 180b, and 180c in RAN 113 via the N3 interface. These gNBs can provide WTRU 102a, 102b, and 102c with access to packet-switched networks (such as the Internet 110) to facilitate communication between WTRU 102a, 102b, 102c and IP-enabled devices. UPF 184a and 184b can perform other functions such as routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering downlink packets, and providing mobility anchoring.

[0089] CN 115 can facilitate communication with other networks. For example, CN 115 may include or communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that acts as an interface between CN 115 and PSTN 108. Additionally, CN 115 can provide WTRUs 102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, WTRUs 102a, 102b, and 102c can connect to local data networks (DNs) 185a and 185b via the N3 interface to UPFs 184a and 184b and the N6 interface between UPFs 184a and 184b and DNs 185a and 185b.

[0090] Given Figures 1A to 1D as well as Figures 1A to 1D The corresponding descriptions may be performed by one or more emulation devices (not shown) to perform one or more of the functions described herein with respect to one or more of the following: WTRU 102a to 102d, base stations 114a to 114b, eNode-B 160a to 160c, MME 162, SGW 164, PGW 166, gNB 180a to 180c, AMF 182a to 182b, UPF 184a to 184b, SMF 183a to 183b, DN 185a to 185b, and / or any other devices described herein. An emulation device may be one or more devices configured to emulate one or more of the functions described herein. For example, an emulation device may be used to test other devices and / or simulate network and / or WTRU functions.

[0091] Simulation devices can be designed to perform tests on one or more other devices in a laboratory environment and / or a carrier network environment. For example, one or more simulation devices can perform one or more or all of their functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices within the communication network. One or more simulation devices can perform one or more or all of their functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. Simulation devices can be directly coupled to another device for testing purposes and / or can use over-the-air wireless communication to perform tests.

[0092] One or more emulation devices may perform one or more functions without being implemented / deployed as part of a wired and / or wireless communication network. For example, emulation devices may be used to test scenarios in a laboratory and / or non-deployed (e.g., testing) wired and / or wireless communication networks to enable testing of one or more components. One or more emulation devices may be test equipment. Direct RF coupling and / or wireless communication via RF circuitry (e.g., which may include one or more antennas) may be used by the emulation devices to transmit and / or receive data.

[0093] This application describes various aspects, including tools, features, examples, models, methods, etc. Many of these aspects are specifically described and are generally described in a way that may sound restrictive, at least to illustrate individual characteristics. However, this is for clarity and does not limit the application or scope of these aspects. In fact, all the different aspects can be combined and interchanged to provide other aspects. Moreover, these aspects can also be combined and interchanged with aspects described in earlier documents.

[0094] The aspects described and envisioned in this application can be implemented in many different forms. Figure 5-9 Some examples can be provided, but other examples are also envisioned. Figure 5-9 The discussion does not limit the breadth of implementation methods. At least one aspect typically involves video encoding and decoding, and at least one other aspect typically involves transmitting a generated or encoded bitstream. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions stored thereon for encoding or decoding video data according to any of the described methods, and / or computer-readable storage media having bitstreams generated according to any of the described methods stored thereon.

[0095] In this application, the terms “reconstruction” and “decoding” are used interchangeably, the terms “pixel” and “sample” are used interchangeably, and the terms “image”, “picture” and “frame” are used interchangeably.

[0096] This document describes various methods, and each method includes one or more steps or actions to implement the described method. Unless a specific order of steps or actions is required for the method to operate correctly, the order and / or use of specific steps and / or actions can be modified or combined. Additionally, in various examples, terms such as "first," "second," etc., may be used to modify elements, components, steps, operations, etc., such as, for example, "first decoding" and "second decoding." Unless specifically required, the use of such terms does not imply an ordering of the modified operations. Therefore, in this example, the first decoding does not need to be performed before the second decoding and can occur, for example, before, during, or within a time period overlapping with the second decoding.

[0097] The various methods and other aspects described in this application can be used to modify... Figure 2 and Figure 3 The illustrated video encoder 200 and decoder 300 modules are, for example, decoding modules. Furthermore, the subject matter disclosed herein can be applied to, for example, any type, format, or version of video encoding (whether described in standards or recommendations, whether pre-existing or future-developed) and any extensions to such standards and recommendations. Unless otherwise indicated or technically excluded, the various aspects described in this application can be used individually or in combination.

[0098] Various numerical values ​​are used in the examples described in this application, such as correction values, scaling values, bandwidth values, filter values, coefficient values, minimum values, maximum values, flag values, offset values, chroma formats (e.g., 4:2:0, 4:2:2, 4:4:4), block sizes or codec unit sizes (e.g., height, width), partition sizes, number of search regions, number of sub-regions or sub-regions, search step size, number of reconstructed blocks, number of entries in the list, precision, number of directions, number of patterns, number of candidates, candidate order, number of samples, number of references, bit values, sub-block sizes, number of parameters, multiplier values, constant values, range values, minimum values, maximum values, threshold values, index values, number of predicted patterns, number of reference lines, index values, flag values, weight values, etc. These and other specific values ​​are used to describe the examples, and the various aspects described are not limited to these specific values.

[0099] Figure 2 This is a diagram illustrating an example video encoder. Variations of the example encoder 200 are envisioned, but for clarity, encoder 200 is described below without depicting all anticipated variations.

[0100] Before encoding, the video sequence can undergo pre-coding (201), such as applying color transformations to the input color image (e.g., from RGB 4:4:4 to YCbCr 4:2:0), or performing remapping on the input image components to obtain a more resilient signal distribution to compression (e.g., using histogram equalization of one of the color components). Metadata can be associated with pre-processing and appended to the bitstream.

[0101] In encoder 200, as described below, the image is encoded by encoder elements. The image to be encoded is partitioned (202) and processed in units, for example, coding units (CUs). Each unit is encoded using, for example, an intra-frame or inter-frame mode. When a unit is encoded in intra-frame mode, it performs intra-frame prediction (260). In inter-frame mode, motion estimation (275) and compensation (270) are performed. The encoder determines (205) which of the intra-frame or inter-frame modes to use to encode the unit and indicates the intra-frame / inter-frame decision by, for example, a prediction mode flag. For example, the prediction residual is calculated by subtracting (210) the prediction block from the original image block.

[0102] The predicted residual is then transformed (225) and quantized (230). The quantized transform coefficients, along with the motion vector and other syntactic elements (such as image partitioning information), are entropy encoded (245) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass the transform and quantization, i.e., the residual is directly encoded without applying the transform or quantization process.

[0103] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (240) and inversely transformed (250) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (255) to reconstruct the image block. An in-loop filter (265) is applied to the reconstructed image to perform, for example, deblocking / SAO (Sample Adaptive Shift) / ALF (Adaptive Loop Filtering) filtering to reduce coded artifacts. The filtered image is stored in the reference image buffer (280).

[0104] Figure 3 This is a diagram illustrating an example video decoder. In example decoder 300, the bitstream is decoded by decoder elements, as described below. Figure 2 As described, the video decoder 300 typically performs a decoding phase that is the reciprocal of the encoding phase. The encoder 200 typically also performs video decoding as part of the encoding of the video data.

[0105] Specifically, the input to the decoder includes a video bitstream, which can be generated by the video encoder 200. The bitstream is first entropy decoded (330) to obtain transform coefficients, prediction modes, motion vectors, and other encoded information. Image partitioning information indicates how the image is partitioned. Therefore, the decoder can partition the image according to the decoded image partitioning information (335). The transform coefficients are dequantized (340) and inverse transformed (350) to decode the prediction residuals. The decoded prediction residuals and prediction blocks are combined (355) to reconstruct the image blocks. The prediction blocks (370) can be obtained from intra-frame prediction (360) or motion-compensated prediction (i.e., inter-frame prediction) (375). An in-loop filter (365) is applied to the reconstructed image. The filtered image is stored at a reference image buffer (380). In some examples (e.g., for a given image), the contents of the reference image buffer 380 on the decoder 300 side can be the same as the contents of the reference image buffer 280 on the encoder 200 side (e.g., for the same image).

[0106] The decoded image can also undergo post-decoding processing (385), such as inverse color transformation (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping, which is the inverse operation of the remapping process performed in pre-encoding processing (201). Post-decoding processing can use metadata derived in pre-encoding processing and signaled in the bitstream. In the example, the decoded image (e.g., after applying an in-loop filter (365), and / or after post-decoding processing (385), if post-decoding processing is used) can be sent to a display device for presentation to the user.

[0107] Figure 4 This is a diagram illustrating system examples that can implement the various aspects and examples described herein. System 400 can be implemented as a device including the various components described below and configured to perform one or more aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 400 can be implemented individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one example, the processing and encoder / decoder elements of system 400 are distributed across multiple ICs and / or discrete components. In various examples, system 400 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various examples, system 400 is configured to implement one or more aspects described in this document.

[0108] System 400 includes at least one processor 410 configured to execute instructions loaded therein to implement various aspects, such as those described in this document. Processor 410 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 400 includes at least one memory 420 (e.g., a volatile memory device and / or a non-volatile memory device). System 400 includes a storage device 440, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 440 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0109] System 400 includes an encoder / decoder module 430, which is configured to process data, for example, to provide encoded or decoded video, and the encoder / decoder module 430 may include its own processor and memory. The encoder / decoder module 430 represents multiple modules that can be included in a device to perform encoding and / or decoding functions. It is well known that a device may include one or both encoding and decoding modules. Additionally, the encoder / decoder module 430 may be implemented as a separate element of system 400, or it may be incorporated into processor 410 as a combination of hardware and software known to those skilled in the art.

[0110] Program code to be loaded onto processor 410 or encoder / decoder 430 to execute the various aspects described in this document may be stored in storage device 440 and subsequently loaded onto memory 420 for execution by processor 410. Depending on various examples, one or more of processor 410, memory 420, storage device 440, and encoder / decoder module 430 may store one or more various items during the execution of the processes described in this document. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from equations, formulas, operations, and operational logic processing.

[0111] In some examples, the memory within processor 410 and / or encoder / decoder module 430 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other examples, external memory (e.g., the processing device may be processor 410 or encoder / decoder module 430) is used for one or more of these functions. External memory may be memory 420 and / or storage device 440, such as volatile memory and / or non-volatile flash memory. In several examples, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one example, fast external volatile memory, such as RAM, is used as working memory for video encoding and decoding operations.

[0112] As indicated in block 445, inputs can be provided to the components of system 400 through various input devices. Such input devices include, but are not limited to, (i) a radio frequency (RF) section that receives, for example, RF signals transmitted over the air by a broadcaster, (ii) component (COMP) input terminals (or sets of COMP input terminals), (iii) universal serial bus (USB) input terminals, and / or (iv) high-definition multimedia interface (HDMI) input terminals. Figure 4 Other examples not shown include composite video.

[0113] In various examples, the input device of block 445 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for (i) selecting a desired frequency (also known as selecting a signal, or limiting a signal band to a certain band), (ii) down-converting the selected signal, (iii) further band-limiting it to a narrower band to select, for example, a signal band that may be referred to as a channel in some examples, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and / or (vi) demultiplexing to select a desired stream of data packets. The RF section of various examples includes one or more elements performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, down-converters, demodulators, error correctors, and demultiplexers. The RF section may include tuners performing various of these functions, including, for example, down-converting received signals to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one set-top box example, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to the desired frequency band. Various examples rearrange the order of the aforementioned (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as, for example, inserting amplifiers and analog-to-digital converters. In various examples, the RF section includes an antenna.

[0114] USB and / or HDMI terminals may include corresponding interface processors for connecting system 400 to other electronic devices over USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, within a separate input processing IC or within processor 410. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed within a separate interface IC or within processor 410. Demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, processor 410 and an encoder / decoder 430 operating in combination with memory and storage elements, to process the data streams as needed for presentation on an output device.

[0115] Various components of system 400 can be provided within an integrated housing. Within the integrated housing, various components can be interconnected using a suitable connection arrangement 425 and transmit data therebetween, such as internal buses known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards.

[0116] System 400 includes a communication interface 450 capable of communicating with other devices via a communication channel 460. The communication interface 450 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 460. The communication interface 450 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 460 may be implemented, for example, within a wired and / or wireless medium.

[0117] In various examples, wireless networks such as Wi-Fi networks, such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers), are used to stream or otherwise provide data to system 400. In these examples, the Wi-Fi signal is received via a communication channel 460 and a communication interface 450 adapted for Wi-Fi communication. The communication channel 460 in these examples is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other examples use a set-top box to provide streaming data to system 400, delivering data via an HDMI connection to input block 445. Other examples use an RF connection to input block 445 to provide streaming data to system 400. As indicated above, various examples provide data in a non-streaming manner. Additionally, various examples use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth® networks.

[0118] System 400 can provide output signals to various output devices, including a display 475, a speaker 485, and other peripheral devices 495. Various examples of the display 475 include one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 475 can be used in a television, tablet computer, laptop computer, mobile phone, or other device. The display 475 can also be integrated with other components (e.g., in a smartphone) or standalone (e.g., as an external monitor for a laptop computer). In various examples, other peripheral devices 495 include one or more of a standalone digital video disc (or digital universal disc) (DVD, for both terms), a disc player, a stereo system, and / or a lighting system. Various examples use one or more peripheral devices 495 that provide functionality based on the output of system 400. For example, a disc player performs the function of playing the output of system 400.

[0119] In various examples, signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable inter-device control with or without user intervention is used to transmit control signals between system 400 and display 475, speaker 485, or other peripheral devices 495. Output devices can be communicatively coupled to system 400 via dedicated connections through corresponding interfaces 470, 480, and 490. Alternatively, output devices can be connected to system 400 via communication interface 450 using communication channel 460. Display 475 and speaker 485 can be integrated into a single unit with other components of system 400 in electronic devices such as, for example, televisions. In various examples, display interface 470 includes display drivers, such as, for example, a timing controller (TCon) chip.

[0120] For example, if the RF section of input 445 is part of a separate set-top box, then display 475 and speaker 485 can alternatively be separated from one or more other components. In various examples where display 475 and speaker 485 are external components, the output signal can be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0121] These examples can be implemented by computer software implemented by processor 410, or by hardware, or by a combination of hardware and software. As a non-limiting example, these examples can be implemented by one or more integrated circuits. Memory 420 can be of any type suitable for the technical environment, and as a non-limiting example, it can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. Processor 410 can be of any type suitable for the technical environment, and as a non-limiting example, it can encompass one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0122] Various implementations include decoding. As used in this application, "decoding" can encompass all or part of the process performed on, for example, a received encoded sequence to produce a final output suitable for display. In various examples, such a process includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various examples, such a process also includes, or alternatively includes, processes performed by a decoder of the various implementations described in this application, such as determining a classification associated with a reconstructed image sample; determining one or more scaling parameters associated with the classification; determining a scaling correction based on one or more scaling parameters; applying the scaling correction to the reconstructed image sample; performing neural network processing to determine a classification associated with the reconstructed image sample; determining one or more scaling parameters associated with the classification; determining a scaling correction based on one or more scaling parameters; applying the scaling correction to the reconstructed image sample; receiving a bitstream of video data, which includes indications associated with one or more scaling parameters for the reconstructed image sample; determining one or more scaling parameters based on the indications; determining a scaling correction based on one or more scaling parameters; applying the scaling correction to the reconstructed image sample, etc.

[0123] As other examples, in one example, "decoding" refers only to entropy decoding; in another, "decoding" refers only to differential decoding; and in yet another, "decoding" refers to a combination of entropy decoding and differential decoding. It will be clear, and those skilled in the art, whether the phrase "decoding process" is intended to specifically refer to a subset of operations or generally to the broader decoding process, based on the specific context of the description.

[0124] Various implementations include encoding. Similar to the above discussion of "decoding," the term "encoding" as used in this application can encompass, for example, all or part of a process performed on an input video sequence to produce an encoded bitstream. In various examples, such a process includes one or more processes typically performed by an encoder, such as partitioning, differential coding, transform, quantization, and entropy coding. In various examples, such a process also includes, or alternatively includes, processes performed by an encoder of the various implementations described in this application, such as determining a classification associated with a reconstructed image sample; determining one or more scaling parameters associated with the classification; determining a scaling correction based on one or more scaling parameters; applying the scaling correction to the reconstructed image sample; performing neural network processing to determine a classification associated with the reconstructed image sample; determining one or more scaling parameters associated with the classification; determining a scaling correction based on one or more scaling parameters; applying the scaling correction to the reconstructed image sample; determining a classification associated with the reconstructed image sample; determining one or more scaling parameters associated with the classification; determining a scaling correction based on one or more scaling parameters; applying the scaling correction to the reconstructed image sample; inserting an indication associated with one or more scaling parameters into the bitstream of the reconstructed image sample, etc.

[0125] As other examples, in one example, "encoding" refers only to entropy encoding; in another, "encoding" refers only to differential encoding; and in yet another, "encoding" refers to a combination of differential and entropy encoding. It will be clear, and those skilled in the art, whether the phrase "encoding process" is intended to specifically refer to a subset of operations or generally to a broader encoding process, based on the specific context of the description.

[0126] Note that the syntactic elements used in this article (such as bitdepth, numAlt, apsId, idxCorr, nbGroup, use_alf_scale_flag, pcSlice, getTileGroupNumAps, alf_scale_group_shift, alf_scale_group_idx_corr, etc.) are descriptive terms. Therefore, they do not preclude the use of other syntactic element names.

[0127] When the accompanying drawings are presented as flowcharts, it should be understood that they also provide block diagrams of the corresponding devices. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that they also provide flowcharts of the corresponding methods / processes.

[0128] The implementations and aspects described herein can be implemented, for example, in methods or processes, apparatuses, software programs, data streams, or signals. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the features under discussion can also be implemented in other forms (e.g., apparatuses or programs). For example, an apparatus can be implemented with appropriate hardware, software, and firmware. These methods can be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as, for example, computers, cellular phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0129] References to “an example” or “an example” or “an implementation” or “an implementation” and their variations mean that a particular feature, structure, characteristic, etc., described in connection with the example is included in at least one example. Therefore, the phrases “in an example” or “in the example” or “in an implementation” or “in the implementation” and their other variations appearing throughout this application do not necessarily all refer to the same example.

[0130] Additionally, this application may relate to "determining" various information segments. Determining information may include one or more of the following: for example, estimated information, calculated information, predicted information, or information retrieved from memory. Obtaining may include receiving, retrieving, constructing, generating, and / or determining.

[0131] Furthermore, this application may relate to "accessing" various information segments. Accessing information may include one or more of the following: for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0132] Additionally, this application may relate to "receiving" various segments of information. Like "accessing," receiving is intended to be a broad term. Receiving information may include one or more of the following: for example, accessing information or retrieving information (e.g., from memory). Further, "receiving" is generally referred to in one or more ways during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0133] It should be understood that the use of any of the following “ / ”, “and / or”, and “…at least one of…”—for example, in the cases of “A / B”, “A and / or B”, and “at least one of A and B”—is intended to cover selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As further examples, in the cases of “A, B, and / or C” and “at least one of A, B, and C”, this wording is intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A, B, and C). Those skilled in the art and related fields will appreciate that this can be extended to as many items as possible listed.

[0134] Moreover, as used in this paper, the term "signal" refers to instructing the corresponding decoder to do something. Encoder signals can include, for example, prediction modes, scaling parameters, residual signals, metadata, encoding / decoding or motion representation modes and related signals (e.g., ITMP-related flags, LIC-related flags), partitioning modes, reference images, motion vector predictors (MVP), motion vector differences (MVD), indices, candidate lists, offsets, classifiers, filters, scaling parameters, etc. In this way, in the examples, the same parameters are used on both the encoder and decoder sides. Therefore, for example, the encoder can send (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. Conversely, if the decoder already has specific parameters as well as other parameters, then signaling can be used without sending them (implicit signaling) to simply allow the decoder to know and select specific parameters. Bit savings are achieved in various examples by avoiding the transmission of any actual functionality. It is important to understand that signaling can be implemented in various ways. For example, in various examples, one or more syntactic elements, flags, etc., are used to signal information to the corresponding decoder. Although the verb form of the word "signal" was mentioned earlier, the word "signal" can also be used as a noun in this article.

[0135] As will be apparent to those skilled in the art, implementations can generate various signals formatted to carry, for example, information that can be stored or transmitted. This information may include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal may be formatted to carry a bit stream of the described example. For example, such a signal may be formatted as an electromagnetic wave (e.g., using the radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. It is well known that signals can be transmitted via various wired or wireless links. Signals may be stored on, or accessed or received from, a processor-readable medium.

[0136] This document describes numerous examples. Features of the examples may be provided individually or in any combination, spanning various claim classes and types. Furthermore, examples may include one or more features, devices, or aspects described herein, individually or in any combination, spanning various claim classes and types. For example, features described herein may be implemented in a bitstream or signal including information generated as described herein. This information may allow a decoder to decode the bitstream according to any described embodiment, allow an encoder to encode the bitstream according to any described embodiment, and / or allow a decoder to operate according to any described embodiment. For example, features described herein may be implemented by creating and / or sending and / or receiving and / or decoding a bitstream or signal. For example, features described herein may be implemented by a method, process, apparatus, medium storing instructions, medium storing data, or signal. For example, features described herein may be implemented by a TV, set-top box, mobile phone, tablet computer, or other electronic device performing decoding. The TV, set-top box, mobile phone, tablet computer, or other electronic device may display (e.g., using a monitor, screen, or other type of display) a resulting image (e.g., an image reconstructed from the residual of the video bitstream). The TV, set-top box, mobile phone, tablet computer, or other electronic device may receive a signal including the encoded image and perform decoding.

[0137] In-loop post-filtering can be implemented using a video codec. The reconstructed image can be enhanced using post-filtering. For example, the post-filtering stage can be referred to as in-loop if / when the filtered image can be stored in a decoded picture buffer (DPB) for further use in inter-frame prediction.

[0138] In some examples, post-filtering can use additional data, also known as filter parameters, such as linear filter coefficients, sample offsets, etc. This data can be implicit (e.g., predefined values ​​that may be known on the decoder side), also known as fixed filters, or it can be explicit (e.g., values ​​encoded and decoded in the bitstream), also known as non-fixed or adaptive filters or sample adaptive offsets.

[0139] Figure 5 The diagram illustrates a flowchart showing an example of a classification-based post-filtering process. In some examples (e.g., ... Figure 5 As shown), image samples (e.g., codec tree units (CTUs)) (515) can be classified (520) into N classes Cl. i=0,..N-1 N filter parameters P i=0,..N-1 The set can be selected (e.g., implicit case) or encoded / decoded in the bitstream (e.g., explicit case). Classified as Cl k Each (e.g., each) sample can be represented by filter parameter P. k Perform filtering.

[0140] The post-filtering process can calculate (e.g., for each reconstructed sample) the correction (540) to be added (550) to the reconstructed sample. In the case of classification-based filtering, the correction can be based on a parameter (530) associated with the class (520) of the sample.

[0141] Post-filtering based on a neural network (NN) can compute corrections to be added to the reconstructed samples by inferring (applying) the NN. Classification may not be necessary, as it might be implicitly performed by the NN layers and stored in the latent space. Filter parameters can be coefficients of the NN, which can be (e.g., implicitly) known on the decoder side. For example, filter parameters (e.g., NN coefficients) can be chosen depending on encoding / decoding parameters (e.g., quantization parameters (QP)). The NN can use reconstructed samples and encoding / decoding information (e.g., residuals, unblocking filter parameters, etc.) at the input. The NN may be trained (e.g., or partially trained) on a set of data during an initial (e.g., offline) training phase. In some examples (e.g., ...), ... Figure 6 As shown), parameters that allow the export of scaling or offset parameters can be sent in the bitstream (545) to adapt to the correction (540). For example, an example of adaptive correction can be determined according to equations (1) and (2): corr' = scale x corr (1) corr' = corr + off (2).

[0142] Figure 6 The diagram illustrates a flowchart of an example of a post-filtering process with rescaling correction.

[0143] Sample adaptive offset (SAO) and / or cross-component SAO (CC-SAO) filters can be implemented. SAO band offset (BO) and edge offset (EO) modes can be classified (NC). Sample values ​​belonging to one or more classes can be modified, for example, by adding an offset off(n), which can be (e.g., explicitly) encoded / decoded in the bitstream (P). i=0…3 =off(i)).

[0144] For example, the BO band can use a classifier where the range of sample values ​​for component "c" (e.g., 0..255, 8 bits) can be split (e.g., uniformly split) into N(c) = 32 bands. Sample values ​​belonging to (NC-1) = 4 consecutive bands can be modified, for example, by adding an offset off(n), which can be (e.g., explicitly) encoded / decoded in the bitstream (P). i=0…3 =off(i)). Figure 7 An example of four (4) consecutive frequency bands is shown. Encoding and decoding can be performed for (NC-1) = 4 banded offset values ​​and the starting frequency band, for example, one encoding and decoding for each of the (NC-1) = 4 frequency bands (e.g., the offset of the remaining frequency bands is equal to zero).

[0145] Figure 7 An example of the SAO BO mode is illustrated, where the pixel range of 0 to 255 (8 bits) can be evenly divided into 32 frequency bands.

[0146] Similar to SAO, CCSAO can classify reconstructed samples into different BO or EO categories, which can be derived from the current component (e.g., Cb) and other components (e.g., Y, Cr). An offset can be derived (e.g., correctly) for each category. The offset value can be added to the reconstructed samples in each category.

[0147] For example, in CC-SAO BO mode, given a luminance / chrominance sample, three candidate samples can be selected to classify the given sample into different categories: a juxtaposed Y sample, a juxtaposed U sample, and a juxtaposed V sample. The sample values ​​of the three selected samples can (e.g., then) be classified into three different frequency bands. Y band U band V The joint index "i" can be used to indicate the class of a given sample. An offset can be signaled (e.g., one) and added to the reconstructed samples belonging to that class, for example, it can be formulated according to the set of equation (3): Where {Y col Ucol V col} can be three selected juxtaposed samples used to classify the current sample; {N Y , N U , N V} can be applied to {Y} respectively col U col V col The number of equal-divided frequency bands across the entire range; BD can be the internal codec bit depth; C rec and C' rec This could be reconstructed samples before and after applying CCSAO; and σ CCSAO [i] can be the value of the CCSAO offset that can be applied to the i-th BO category.

[0148] Similar to SAO, different classifiers can be applied to different local regions to further enhance the overall image quality. The parameters of each classifier (e.g., parameter N) Y N U N V (and offset) can be signaled, for example, at the frame level. For example, at the codec tree block (CTB) level, signaling and switching of the classifier to be used can be done, for example, explicitly. For example, one or more constraints may apply, for example, to each classifier. For example, {N} can be used. Y , N U , N V The maximum value of} is set to {16, 4, 4}. For example, the offset may be constrained to the range [-15, 15]. For example, the maximum number of classifiers per frame can be constrained to four (4).

[0149] Adaptive Loop Filter (ALF) and / or Cross Component ALF (CC-ALF) filters can be applied. The Adaptive Loop Filter (ALF) can classify reconstructed samples (e.g., luminance and / or chrominance) into distinct classes. (e.g., a) Class Index (Cl... k The filter parameter set P can be associated with each reconstructed sample. k It can be associated with (e.g., each) class index. For example, P k It can be a set of linear filter coefficients. An example of an ALF classifier can be represented, for example, according to equation (4): Where P k ={c i} i=-n…+n These can be filter coefficients, where xi is a sample neighbor of x, x = x0, and F k (x) is the filtered value of x.

[0150] One or more classifiers can be used. The type of classifier can be signaled in the bitstream. For example (e.g., in ALF), non-fixed filters can use the Laplacian operator or a band classifier. In some examples, other data based on reconstructed samples can be used for classification, such as residual samples.

[0151] A band classifier can classify brightness reconstructed samples into frequency bands with intervals between 0 and (1 << bit depth) (e.g., equal intervals), where the bit depth can be an integer bit depth representation of the reconstructed sample data. Similarly, a residual classifier can derive the class using the absolute value of the residual. A Laplacian classifier can use local gradients to derive the dominant direction and activity to derive the class.

[0152] Non-fixed ALF coefficients can be encoded and decoded. The non-fixed coefficients of the ALF can be derived for (e.g., one) frame and sent in the bitstream to (e.g., one) Adaptive Parameter Set (APS) nal unit for each class. An APS nal unit can include a set of "numAlt" filters (e.g., numAlt=4). For example, multiple classes of the filter set can be grouped so that coefficients from different classes can be the same, which can improve encoding and decoding efficiency. The APS can be indexed using "apsId". The CTU (Code-Decode Tree Unit) of subsequent frames can reuse ALF coefficients previously decoded from the APS.

[0153] Adaptive clipping can be performed. Minimum and maximum (min / max) sample values ​​can be encoded and decoded in the bitstream (e.g., slice header), for example, to define constrained dynamic range values. For example, clipping operations can be applied to filtering, sampling, interpolation, weighted prediction, weighted combination, reconstruction, and / or other stages, for example, to ensure that the generated predicted and reconstructed samples remain within a defined dynamic range. At the encoder, the minimum / maximum value can be obtained, for example, by scanning sample values ​​in the original image. The (e.g., original) image / slice can be pre-filtered before encoding, for example, using a motion-compensated temporal filter (MCTF). For example, the minimum / maximum value can be obtained by scanning sample values ​​in the pre-filtered image / slice.

[0154] For post-filter correction scaling, consider deriving a set of scaling parameters customized for classification.

[0155] By using appropriate classification to derive and applying different scaling parameters, the benefits of post-scaling filter correction in video encoding and decoding can be enhanced.

[0156] Considering ALF, the non-fixed coefficients may have been derived (e.g., optimized) for (e.g., one) frame and sent to the APS nal unit in the bitstream. For subsequent frames, the non-fixed coefficients may be suboptimal.

[0157] Systems, methods, and tools for classification-based post-filter rescaling are disclosed. Scaling parameters can be adapted to the associated classification. Scaling parameters can allow modification of post-filter corrections. The classifier can reuse an existing classifier that can be used by post-filtering or another classifier. The adaptive classifier can allow, for example, one or more of the following: classification-based scaling correction; NN-based post-filtering with classification-based scaling correction; and / or signaling of scaling parameters.

[0158] Classification-based scaling correction can be implemented. Scaling correction can be customized for, for example, a classification (CS). Figure 8 As shown in the example, the classifier (525) can allow the derivation (e.g., one) of class indices for each reconstructed sample. A set of scaling parameters can be defined. The correction can be scaled using a scaling parameter (535) associated with the class index. Examples of scaling parameters can be “scale” or “off”, as depicted in equations (1) and (2) respectively. Other examples of correction models can be represented, for example, according to equations (5), (6), and (7), where “corr” is the post-filter correction, {s i , o j} represents the scaling parameter, and "corr" is the scaling correction: corr' = s0x corr + s1x corr 2 (5) corr' = s0x corr + o0 (6) corr' = s0x corr + s1x corr 2 + o0 (7).

[0159] Figure 8 The diagram illustrates a flowchart of an example of a classification-based scaling correction process.

[0160] In some examples, the classifier CS can be the same as the classifier CF that can be used to derive filter parameters (e.g., filter coefficients).

[0161] The values ​​of (min;max) can be derived from the adaptive clipping information encoded and decoded in the bitstream (560). For example, if CS is a band classifier, the minimum and maximum bands can be defined relative to the clipping range.

[0162] Post-filtering based on neural networks (NNs) can be achieved using classification-based scaling correction.

[0163] In some examples (such as those described in this paper), NN-based post-filtering can use (e.g., an additional) classifier process. For example (e.g., ... Figure 9as shown in the example in ), the NN-based (or non-NN-based) module (570) may operate as a classifier to derive the class index of each sample to be corrected. For example, a scaling parameter (e.g., an optimal scaling parameter) may be derived for each frame to minimize the distortion of the original frame.

[0164] Figure 9 illustrates a flowchart showing an example of NN-based post-filtering (575) with classification-based scaling correction.

[0165] The scaling parameters may be signaled. One or more (e.g., a set or a subset) of the scaling parameters may be sent in the bitstream and / or may be known (e.g., implicitly) at the decoder. The scaling parameters may not be explicitly sent. An index (e.g., idxCorr) of a predetermined table (LUT) referencing scaling parameter values may be used. An index value (e.g., idxCorr=0) may correspond to no modification of the correction. One or more predetermined tables may be used. The selected table may depend on, for example, one or more coding parameters, such as the temporal identifier index (TId) of the current picture / slice, the QP base of the slice, the picture order count (POC), the slice type, etc. The scaling parameter (e.g., the index) may be coded in a slice header, a picture header, or the like, or in a video coding layer (VCL), for example using a context-adaptive binary arithmetic coding (CABAC) entropy encoder. For example, the table index may be coded with the first CTU data. The values of the table may be updated by scaling parameters (e.g., indices) previously coded in the bitstream. For example, the table may be updated with scaling parameters (e.g., indices) that may have been pre-coded in the bitstream of a previous frame, for example with the same TId.

[0166] The number of transmitted scaling parameter indices may be inferior to the number of CS classes. In some examples, unsent index values may be inferred, for example corresponding to "unmodified correction". Some classes may be grouped together. A (e.g., single) transmitted index may be associated with several classes. For example, the syntax element SG may be signaled (e.g., first), allowing the number of groups (nbGroup) to be derived. The nbGroup scaling parameter indices may be coded (e.g., subsequently). For example, nbGroup=1<<SG or nbGroup=SG.

[0167] In some examples (e.g., ALF post-filtering), for example if CS is the same as CF, the grouping of classes may be the same as the group of transmitted filter coefficients used for APS-ALF parameters.

[0168] The value of (min; max) may for example be derived from adaptive clipping information encoded and decoded in the bitstream. For example, if the classifier is a frequency band classifier, the scaling parameter(s) (e.g., indices) for classes corresponding to samples outside the clipping range may not be sent.

[0169] A syntax example for encoding and decoding the scaling parameter index in the slice header is depicted in Table 1.

[0170] Table 1 - Syntax example for encoding and decoding scaling parameter indices .

[0171] With reference to the syntax example in Table 1, the parameter use_alf_scale_flag equal to zero (0) may indicate, for example, that no scaling parameters are encoded and decoded and / or that the scaling parameter index may be inferred to be zero (0). The parameter use_alf_scale_flag equal to one (1) may indicate, for example, that one or more (e.g., some) scaling parameters are encoded and decoded. The parameter alf_scale_group_shift[i][j] may allow deriving the number of encoded and decoded scaling parameter indices for the j-th alternative filter of the i-th APS referenced in the current slice, which is equal to (1<<alf_scale_group_shift[i][j]). The parameter alf_scale_group_idx_corr[i][j][g] may be the scaling parameter index associated with the g-th group of classes.

[0172] Although features and elements are described above in particular combinations, those of ordinary skill in the art will appreciate that each feature or element may be used alone, or in any combination with other features and elements. In addition, the methods described herein may be implemented in a computer program, software or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted over wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, Read-Only Memory (ROM), Random Access Memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (such as internal hard disks and removable disks), magneto-optical media, and optical media (such as CD-ROM disks and Digital Versatile Disks (DVDs)). A radio frequency transceiver may be implemented using a processor associated with software for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

[0173] Examples of the present disclosure 1. An encoder apparatus, comprising: a processor, configured to: Determine the correction (corr) for the reconstructed image samples; Determine the classification (CS) associated with the reconstructed image samples; Determine one or more scaling parameters associated with the classification (CS); The scaling correction (corr') is determined based on one or more scaling parameters and corrections; and Scaling correction is applied to the reconstructed image sample.

[0174] 2. Based on the encoder device of Example 1, The processor configured to determine the classification associated with the reconstructed image sample is also configured to determine the classification index associated with the reconstructed image sample.

[0175] 3. Based on the encoder device of Example 1, The processor configured to determine the classification (CS) associated with the reconstructed image sample is also configured to determine the classification (CF) previously used to determine the post-filter correction.

[0176] 4. Based on the encoder device of Example 1, One or more scaling parameters indicate the scaling value and the offset value.

[0177] 5. Based on the encoder device of Example 1, The processor is also configured as follows: Determine the classification associated with filtering; and Based on the classification (CF) associated with the filter, determine one or more filter parameters; The processor configured to determine the classification (CS) associated with the reconstructed image sample is also configured to determine the classification (CF) associated with the filter as the classification associated with the reconstructed image sample.

[0178] 6. Based on the encoder device of Example 1, The processor configured to determine the classification (CS) associated with the reconstructed image sample is also configured to determine the classification associated with the minimum and maximum values; The processor configured to determine one or more scaling parameters associated with the classification is also configured to determine one or more scaling parameters between a minimum value and a maximum value.

[0179] 7. A method for video encoding, comprising: Determine the correction (corr) for the reconstructed image samples; Determine the classification (CS) associated with the reconstructed image samples; Determine one or more scaling parameters associated with the classification; The scaling correction (corr') is determined based on one or more scaling parameters and the correction (corr); and Scaling correction is applied to the reconstructed image sample.

[0180] 8. A decoder device, comprising: The processor is configured as follows: Determine the correction (corr) for the reconstructed image samples; Determine the classification (CS) associated with the reconstructed image samples; Determine one or more scaling parameters associated with the classification; The scaling correction (corr') is determined based on one or more scaling parameters and the correction (corr); and Scaling correction is applied to the reconstructed image sample.

[0181] 9. Based on the decoder device in Example 8, The processor configured to determine the classification associated with the reconstructed image sample is also configured to determine the classification index associated with the reconstructed image sample.

[0182] 10. Based on the decoder device in Example 8, The processor configured to determine the classification (CS) associated with the reconstructed image sample is also configured to determine the classification (CF) previously used to determine the post-filter correction.

[0183] 11. Based on the decoder device in Example 8, One or more scaling parameters indicate the scaling value and the offset value.

[0184] 12. Based on the decoder device in Example 8, The processor is also configured as follows: Determine the classification (CF) associated with the filter; and Based on the classification associated with filtering, one or more filtering parameters are determined; The processor configured to determine the classification (CS) associated with the reconstructed image sample is also configured to determine the classification (CF) associated with the filter as the classification associated with the reconstructed image sample.

[0185] 13. Based on the decoder device of Example 8, The processor configured to determine the classification (CS) associated with the reconstructed image sample is also configured to determine the classification associated with the minimum and maximum values; The processor configured to determine one or more scaling parameters associated with the classification is also configured to determine one or more scaling parameters between a minimum value and a maximum value.

[0186] 14. A method for video decoding, comprising: Determine the correction (corr) for the reconstructed image samples; Determine the classification associated with the reconstructed image samples; Determine one or more scaling parameters associated with the classification; Scaling correction is determined based on one or more scaling parameters; and Scaling correction is applied to the reconstructed image sample.

[0187] 15. An encoder device, comprising: The processor is configured as follows: Perform neural network processing to determine the correction (corr) for the reconstructed image samples; Determine the classification (CS) associated with the reconstructed image samples; Determine one or more scaling parameters associated with the classification; The scaling correction (corr') is determined based on one or more scaling parameters and corrections; and Scaling correction is applied to the reconstructed image sample.

[0188] 16. The encoder device according to Example 15, The processor configured to perform neural network processing to determine the classification associated with the reconstructed image sample is also configured to determine the classification index associated with the reconstructed image sample.

[0189] 17. The encoder device according to Example 15, The processor is also configured to perform neural network-based filtering on the reconstructed image samples to generate filtered reconstructed image samples; and The processor configured to determine scaling correction based on one or more scaling parameters is also configured to determine scaling correction from filtered reconstructed image samples.

[0190] 18. The encoder device according to Example 17, The processor configured to determine scaling correction from the filtered reconstructed image samples is also configured to determine scaling correction from the filtered reconstructed image samples and one or more scaling parameters.

[0191] 19. A method for video coding, comprising: Perform neural network processing to determine the correction (corr) for the reconstructed image samples; Determine the classification (CS) associated with the reconstructed image samples; Determine one or more scaling parameters associated with the classification; Scaling correction is determined based on one or more scaling parameters and corrections; and Scaling correction is applied to the reconstructed image sample.

[0192] 20. A decoder device, comprising: The processor is configured as follows: Perform neural network processing to determine the correction (corr) for the reconstructed image samples; Determine the classification (CS) associated with the reconstructed image samples; Determine one or more scaling parameters associated with the classification; Scaling correction is determined based on one or more scaling parameters and corrections; and Scaling correction is applied to the reconstructed image sample.

[0193] 21. According to the decoder device of Example 20, The processor configured to perform neural network processing to determine the classification associated with the reconstructed image sample is also configured to determine the classification index associated with the reconstructed image sample.

[0194] 22. Based on the decoder device of Example 20, The processor is also configured to perform neural network-based filtering on the reconstructed image samples to generate filtered reconstructed image samples; and The processor configured to determine scaling correction based on one or more scaling parameters is also configured to determine scaling correction from filtered reconstructed image samples.

[0195] 23. Based on the decoder device of Example 22, The processor configured to determine scaling correction from the filtered reconstructed image samples is also configured to determine scaling correction from the filtered reconstructed image samples and one or more scaling parameters.

[0196] 24. A method for video decoding, comprising: Perform neural network processing to determine the correction (corr) for the reconstructed image samples; Determine the classification (CS) associated with the reconstructed image samples; Determine one or more scaling parameters associated with the classification; Scaling correction is determined based on one or more scaling parameters and corrections; and Scaling correction is applied to the reconstructed image sample.

[0197] 25. An encoder device, comprising: The processor is configured as follows: Determine the correction (corr) for the reconstructed image samples; Determine the classification (CS) associated with the reconstructed image samples; Determine one or more scaling parameters associated with the classification; Scaling correction is determined based on one or more scaling parameters and corrections; Apply scaling correction to the reconstructed image samples; and Instructions associated with one or more scaling parameters are inserted into the bitstream of the reconstructed image samples.

[0198] 26. The encoder device according to Example 25, The processor is also configured to determine an index value associated with one or more scaling parameters, wherein the index value is associated with a table of scaling parameter values.

[0199] 27. The encoder device according to Example 26, The processor configured to insert an indication associated with one or more scaling parameters into the bitstream of the reconstructed image samples is also configured to insert an index value associated with a table of scaling parameter values ​​into the bitstream.

[0200] 28. The encoder device according to Example 27, The scaling parameter value table is one of several tables that include scaling parameter values.

[0201] 29. The encoder device according to Example 28, The reconstructed image samples are associated with a time identifier index; and The scaling parameter value table reflects the scaling parameter values ​​associated with the previous frame linked to the time identifier index.

[0202] 30. The encoder device according to Example 25, The processor is also configured to determine that one or more scaling parameters are not applied to the second reconstructed image sample; and The processor configured to insert an indication associated with one or more scaling parameters into the bitstream is also configured to insert an index value into the bitstream that indicates that scaling parameters are not applied to the second reconstructed image sample.

[0203] 31. The encoder device according to Example 25, The processor is also configured to determine a combination of index values ​​associated with one or more scaling parameters; and The processor configured to insert an indication associated with one or more scaling parameters into the bitstream is also configured to insert a combination of index values ​​associated with one or more scaling parameters into the bitstream.

[0204] 32. The encoder device according to Example 25, The processor is also configured as follows: Determine the classification associated with filtering; and Based on the classification associated with filtering, one or more filtering parameters are determined; and The processor configured to determine the classification associated with the reconstructed image sample is also configured to determine that the classification associated with the reconstructed image sample is equal to the classification associated with the filter.

[0205] 33. The encoder device according to Example 25, The processor configured to determine the classification associated with the reconstructed image samples is also configured to associate the classification with minimum and maximum values; and The processor configured to determine one or more scaling parameters associated with the classification is also configured to determine one or more scaling parameters between a minimum value and a maximum value.

[0206] 34. An encoding method, comprising: Determine the correction (corr) for the reconstructed image samples; Determine the classification associated with the reconstructed image samples; Determine one or more scaling parameters associated with the classification; Scaling correction is determined based on one or more scaling parameters and corrections; Apply scaling correction to the reconstructed image samples; and Instructions associated with one or more scaling parameters are inserted into the bitstream of the reconstructed image samples.

[0207] 35. A decoder device, comprising: The processor is configured as follows: Receive a bitstream of video data, which includes indications associated with one or more scaling parameters for reconstructed image samples; One or more scaling parameters are determined based on the indication; Determine the correction (corr) for the reconstructed image samples; Scaling correction is determined based on one or more scaling parameters and corrections; and Scaling correction is applied to the reconstructed image sample.

[0208] 36. Based on the decoder device of Example 35, The indications associated with one or more scaling parameters include index values ​​associated with a scaling parameter value table; and The processor configured to determine one or more scaling parameters based on an indication is also configured to retrieve one or more scaling parameters from a scaling parameter value table.

[0209] 37. Based on the decoder device of Example 36, The scaling parameter value table is one of several tables that include scaling parameter values.

[0210] 38. According to the decoder device of Example 36, The reconstructed image samples are associated with a time identifier index; and The scaling parameter value table reflects the scaling parameter values ​​associated with the previous frame linked to the time identifier index.

[0211] 39. Based on the decoder device of Example 35, The bitstream also includes a second indication associated with the second reconstructed image sample; The second indicator associated with the second reconstructed image sample indicates that scaling parameters should not be applied to the second reconstructed image; and The processor is also configured to correct the second reconstructed image sample without applying scaling parameters.

[0212] 40. Based on the decoder device in Example 35, The indication associated with one or more scaling parameters includes a combination of index values ​​associated with one or more scaling parameters; The processor configured to determine one or more scaling parameters based on an indication is also configured to retrieve one or more scaling parameters from at least one scaling parameter value table based on a combination of index values.

[0213] 41. A decoding method, comprising: Receive a bitstream of video data, which includes indications associated with one or more scaling parameters for reconstructed image samples; One or more scaling parameters are determined based on the indication; Determine the correction (corr) for the reconstructed image samples; Scaling correction is determined based on one or more scaling parameters and corrections; and Scaling correction is applied to the reconstructed image sample.

Claims

1. A device for video decoding, comprising: The processor is configured as follows: Determine the correction for the reconstructed image samples; Determine the classification associated with the reconstructed image sample; Determine one or more scaling parameters associated with the classification; The scaling correction is determined based on the one or more scaling parameters and the correction. as well as The scaling correction is applied to the reconstructed image sample.

2. A device for video encoding, comprising: The processor is configured as follows: Determine the correction for the reconstructed image samples; Determine the classification associated with the reconstructed image sample; Determine one or more scaling parameters associated with the classification; The scaling correction is determined based on the one or more scaling parameters and the correction. as well as The scaling correction is applied to the reconstructed image sample.

3. The device according to claim 1 or 2, The processor, which is configured to determine the classification associated with the reconstructed image sample, is also configured to determine the classification index associated with the reconstructed image sample.

4. The device according to claim 1 or 2, The processor, which is configured to determine the classification associated with the reconstructed image sample, is also configured to determine the classification previously used to determine the post-filter correction.

5. The device according to claim 1 or 2, The one or more scaling parameters therein indicate scaling and offset values.

6. The device according to claim 1 or 2, The processor is further configured to: Determine the classification associated with filtering; and Based on the classification associated with filtering, one or more filtering parameters are determined; The processor, which is configured to determine the classification associated with the reconstructed image sample, is also configured to determine that the classification associated with the filter is used as the classification associated with the reconstructed image sample.

7. The device according to claim 1 or 2, The processor, which is configured to determine the classification associated with the reconstructed image sample, is also configured to determine that the classification is associated with a minimum and a maximum value; The processor, which is configured to determine one or more scaling parameters associated with the classification, is also configured to determine one or more scaling parameters between the minimum value and the maximum value.

8. A method for video encoding, comprising: Determine the correction for the reconstructed image samples; Determine the classification associated with the reconstructed image sample; Determine one or more scaling parameters associated with the classification; The scaling correction is determined based on the one or more scaling parameters and the correction. as well as The scaling correction is applied to the reconstructed image sample.

9. A method for video decoding, comprising: Determine the correction for the reconstructed image samples; Determine the classification associated with the reconstructed image sample; Determine one or more scaling parameters associated with the classification; The scaling correction is determined based on the one or more scaling parameters; as well as The scaling correction is applied to the reconstructed image sample.

10. The method according to claim 8 or 9, Determining the classification associated with the reconstructed image sample also includes determining the classification index associated with the reconstructed image sample.

11. The method according to claim 8 or 9, Determining the classification associated with the reconstructed image sample also includes determining the classification previously used to determine the post-filtering correction.

12. The method according to claim 8 or 9, The one or more scaling parameters therein indicate scaling and offset values.

13. The method according to claim 8 or 9, further comprising: Determine the classification associated with filtering; as well as Based on the classification associated with filtering, one or more filtering parameters are determined; The determination of the classification associated with the reconstructed image sample also includes determining the classification associated with filtering as the classification associated with the reconstructed image sample.

14. The method according to claim 8 or 9, Determining the classification associated with the reconstructed image sample also includes determining that the classification is associated with a minimum and a maximum value; Determining one or more scaling parameters associated with the classification also includes determining one or more scaling parameters between the minimum value and the maximum value.

15. A computer program product stored on a computer-readable medium and comprising program code instructions to implement the steps of the method according to any one of claims 8 to 14 when executed by a processor.

Images