Ratio or code selection for LMCS

Abstract

Systems, methods, and means may be configured for coding tree unit (CTU) grid shifting. An exemplary video decoding device may receive a picture encoded based on a ratio value. The device may receive a ratio value corresponding to a picture. The device may decode the picture based on the ratio value. The ratio value may correspond to a picture corresponding to the maximum gain. Decoding a picture based on a ratio value may include decoding a picture based on the maximum gain.

Description

[Technical Field]

[0001] (Cross-reference of related applications) This application claims the benefit of European Provisional Patent Application No. EP23306573.9, filed on 21 September 2023, the entire disclosure thereof is incorporated herein by reference. [Background technology]

[0002] Video coding systems can be used to compress digital video signals, for example, to reduce the storage and / or transmission bandwidth of such signals. Video coding systems may include, for example, block-based, wavelet-based, and / or object-based systems. [Overview of the project]

[0003] Systems, methods, and means may be configured for ratio or code selection for Luma Mapping with Chroma Scaling (LMCS). An exemplary video encoding device may determine candidate ratio values ​​associated with a video picture. The device may determine the coding gain associated with each candidate ratio value. The device may select a ratio value from the candidate ratio values ​​based on their respective associated coding gains. The device may adjust the number of codewords of the mapping function associated with the video picture based on the selected ratio value. The device may encode the video picture based on the mapping function.

[0004] The device may partition the range of luma sample values ​​in a video picture into luma sample value partitions, each of which may correspond to a subrange of luma values. The device may determine the number of codewords for a luma sample value partition based on a selected ratio value. The device may map the luma sample value partition to updated luma sample values ​​based on the determined number of codewords for the luma sample value partition. The device may generate a mapping function based on the mapped luma sample value partition. The device may include indications of the selected ratio value in the video data, and the video picture may be further encoded based on these indications.

[0005] The device may determine whether to signal codewords individually or whether codewords are associated with a predefined code, and the indication of selected ratio values ​​may be included in the video data based on the determination that the codewords are associated with a predefined code.

[0006] The device may determine that a codeword is associated with a predefined code. The device may include an index associated with the codeword in the video data, and the video picture may be further encoded based on the codeword.

[0007] The device may determine at least one feature from a mapped videopicture sample associated with a videopicture using at least one ratio value from candidate ratio values, and the coding gain may be determined based on the determined at least one feature. The at least one feature may include at least one of the following: the minimum value of the mapped videopicture sample, the maximum value of the mapped videopicture sample, a segmental histogram of the mapped videopicture sample, segmental variance of the mapped videopicture sample, or the temporal variance between the mapped videopicture sample and a sample of a second videopicture different from the videopicture.

[0008] The device may apply a gradient boosting decision tree algorithm to candidate ratio values, and the coding gain may be determined based on applying the gradient boosting decision tree algorithm to the candidate ratio values.

[0009] An exemplary video decoding device may be configured to receive a video picture. The device may receive an indication of a ratio value corresponding to the video picture. Based on the ratio value, the device may adjust the number of codewords of the mapping function associated with the video picture. The device may decode the video picture based on the mapping function.

[0010] The device may partition the range of luma sample values ​​in a video picture into luma sample value partitions, each of which may correspond to a subrange of luma values. The device may determine the number of codewords for a luma sample value partition based on a selected ratio value. Based on the determined number of codewords for the luma sample value partition, the device may map the luma sample value partition to updated luma sample values. Based on the mapped luma sample value partition, the device may generate a mapping function.

[0011] The device can apply a mapping function to interpreted luma samples. The device can decode video pictures based on applying a mapping function to interpreted luma samples. The device can determine that a codeword belongs to a predefined codeword, and an indication of a ratio value can be received based on the determination that the codeword belongs to a predefined codeword.

[0012] An exemplary video decoding device may receive a picture (e.g., a video frame) encoded based on a ratio value. The device may receive a ratio value corresponding to the picture. The device may decode the picture based on the ratio value. The ratio value may correspond to a picture corresponding to the maximum gain. Decoding a picture based on a ratio value may include decoding a picture based on the maximum gain.

[0013] An exemplary video decoding device may receive a picture and an index. Based on the index, the device may determine the best codeword (CW) for a segment of the video (code) from a given set of candidate codes. Based on the code, the device may decode the picture.

[0014] An exemplary video encoding device may include receiving a picture. The device may estimate multiple encoding gains based on multiple ratio values. The device may determine the maximum gain from the encoding gains. The device may encode the picture based on the maximum gain. Estimating the encoding gain may include testing ratio values ​​from ratio values ​​on the picture. The device may determine that a ratio value corresponds to the maximum gain. Encoding the picture based on the maximum gain may include encoding the picture using the ratio value corresponding to the maximum gain.

[0015] An exemplary video encoding device may receive a picture. The device may evaluate multiple candidate codes. The device may use a machine learning model to compute features to predict the encoding efficiency (e.g., gain) of the candidate codes. The device may compare the encoding efficiency of a candidate code to the maximum gain. If the encoding efficiency of a candidate code is more efficient than the maximum gain, the device may select the code with the highest gain from the candidate codes. The device may select an index corresponding to the selected code. The device may encode the picture based on the index. [Brief explanation of the drawing]

[0016] [Figure 1A] FIG. 1 is a system diagram showing an exemplary communication system in which one or more disclosed embodiments may be implemented. [Figure 1B] FIG. 2 is a system diagram showing an exemplary wireless transmit / receive unit (WTRU) that may be used in the communication system shown in FIG. 1A, in accordance with an embodiment. [Figure 1C] FIG. 3 is a system diagram showing an exemplary radio access network (RAN) and an exemplary core network (CN) that may be used in the communication system shown in FIG. 1A, in accordance with an embodiment. [Figure 1D] FIG. 4 is a system diagram showing a further exemplary RAN and a further exemplary CN that may be used in the communication system shown in FIG. 1A, in accordance with an embodiment. [Figure 2] An exemplary video encoder is illustrated. [Figure 3] An exemplary video decoder is illustrated. [Figure 4] Examples of systems in which various aspects and examples may be implemented are illustrated. [Figure 5] An exemplary flowchart of interference for ratio selection is illustrated. [Figure 6] An exemplary flowchart of inference for a set of CODEs (e.g., the best set of codewords (CWs) per partition for a segment of video) is illustrated. [Figure 7] An exemplary flowchart of inference using features calculated based on mapped video is illustrated. [Description of Embodiments]

[0017] A more detailed understanding may be obtained from the following description, given by way of example in conjunction with the accompanying drawings.

[0018] FIG. 1A is a diagram illustrating an exemplary communication system 100 in which one or more disclosed embodiments may be implemented. The communication system 100 may be a multiple-access system that provides content such as voice, data, video, messaging, and broadcast to a plurality of wireless users. The communication system 100 may enable a plurality of wireless users to access such content through sharing of system resources including radio bandwidth. For example, the communication system 100 may employ one or more channel access methods such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), zero-tail unique-word DFT-Spread OFDM (ZT UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block filtered OFDM, and filter bank multicarrier (FBMC).

[0019] As shown in Figure 1A, the communication system 100 may include radio transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, public switched telephone network (PSTN) 108, the Internet 110, and other networks 112, but it will be understood that the disclosed embodiments are intended to be any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, and 102d may be any type of device configured to operate and / or communicate in a radio environment. For example, WTRU102a, 102b, 102c, 102d (any of which may be referred to as “Station” and / or “STA”) may be configured to transmit and / or receive radio signals and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscriber-based units, pagers, mobile phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearables, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in the context of industrial and / or automated processing chains), consumer electronics devices, and devices operating on commercial and / or industrial wireless networks. WTRU102a, 102b, 102c, and 102d can all be synonymously referred to as UE.

[0020] The communication system 100 may also include base stations 114a and / or base stations 114b. Each of the base stations 114a and 114b may be any type of device configured to radio interface with at least one of the WTRUs 102a, 102b, 102c, and 102d to facilitate access to one or more communication networks, such as CN 106 / 115, the Internet 110, and / or other networks 112. For example, base stations 114a and 114b may be a base transceiver station (BTS), node B, enode B, home node B, home enode B, gNB, NR node B, site controller, access point (AP), wireless router, etc. Although base stations 114a and 114b are shown as single elements, it will be understood that base stations 114a and 114b may include any number of interconnected base stations and / or network elements.

[0021] Base station 114a may be part of RAN 104 / 113, which may also include other base station and / or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), and relay nodes. Base station 114a and / or base station 114b may be configured to transmit and / or receive radio signals on one or more carrier frequencies, which may be referred to as cells (not shown). These frequencies may be licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. Cells may provide coverage of radio services to a particular geographic area, which may be relatively fixed or change over time. Cells may be further divided into cell sectors. For example, a cell associated with base station 114a may be divided into three sectors. Thus, in one embodiment, base station 114a may include three transceivers, i.e., one transceiver per sector of the cell. In one embodiment, the base station 114a may employ multiple-input multiple output (MIMO) technology and utilize multiple transceivers per sector of the cell. For example, beamforming may be used to transmit and / or receive signals in a desired spatial direction.

[0022] Base stations 114a and 114b may communicate with one or more WTRUs 102a, 102b, 102c, and 102d via an air interface 116, which may be any suitable radio communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0023] More specifically, as described above, the communication system 100 may be a multiple access system and may employ one or more channel access schemes such as CDMA, TDMA, FDMA, OFDMA, and SC-FDMA. For example, base stations 114a and WTRUs 102a, 102b, and 102c in RAN 104 / 113 may implement radio technologies such as Universal Mobile Telecommunications System (UMTS) and Universal Terrestrial Radio Access (UTRA), which may use wideband CDMA (WCDMA) to establish air interfaces 115 / 116 / 117. WCDMA may include communication protocols such as High-Speed ​​Packet Access (HSPA) and / or Advanced HSPA (HSPA+). HSPA may include High-Speed ​​Downlink Packet Access (HSDPA) and / or High-Speed ​​UL Packet Access (HSUPA).

[0024] In one embodiment, base stations 114a and WTRUs 102a, 102b, 102c may implement radio technologies such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish an air interface 116 using Long Term Evolution (LTE), LTE-Advanced (LTE-A), and / or LTE-Advanced Pro (LTE-A Pro).

[0025] In one embodiment, base stations 114a and WTRUs 102a, 102b, and 102c may implement radio technologies such as NR radio access, which may establish an air interface 116 using New Radio (NR).

[0026] In one embodiment, base station 114a and WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, base station 114a and WTRUs 102a, 102b, 102c may implement both LTE radio access and NR radio access using, for example, the dual connectivity (DC) principle. Thus, the air interface utilized by WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions to and from multiple types of base stations (e.g., eNBs and gNBs).

[0027] In other embodiments, base stations 114a and WTRUs 102a, 102b, and 102c may implement radio technologies such as IEEE 802.11 (i.e., Wireless Fidelity, WiFi), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access, WiMAX), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Provisional Standard 2000 (IS-2000), Provisional Standard 95 (IS-95), Provisional Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), and GSM EDGE (GERAN).

[0028] The base station 114b in Figure 1A may be, for example, a wireless router, home node B, home e-node B, or access point, and may utilize any suitable RAT to facilitate wireless connectivity in local areas such as offices, homes, vehicles, campuses, industrial facilities, aerial corridors (for use by drones), roads, etc. In one embodiment, the base station 114b and WTRU 102c, 102d may implement wireless technologies such as IEEE 802.11 to establish a wireless local area network (WLAN). In another embodiment, the base station 114b and WTRU 102c, 102d may implement wireless technologies such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, base stations 114b and WTRUs 102c, 102d may establish picocells or femtocells using cellular-based RATs (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.). As shown in Figure 1A, base station 114b may be directly connected to the internet 110. Therefore, base station 114b may not need to access the internet 110 via CN 106 / 115.

[0029] RAN104 / 113 may communicate with CN106 / 115, which may be any type of network configured to provide voice, data, applications, and / or Voice over Internet Protocol (VoIP) services to one or more of WTRU102a, 102b, 102c, and 102d. The data may have various quality of service (QoS) requirements, such as different throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, and mobility requirements. CN106 / 115 may provide call control, billing services, mobile location-based services, prepaid calls, internet connectivity, video distribution, etc., and / or perform high-level security functions such as user authentication. Although not shown in Figure 1A, it will be understood that RAN104 / 113 and / or CN106 / 115 can communicate directly or indirectly with other RANs employing the same or different RATs as RAN104 / 113. For example, in addition to connecting to RAN104 / 113 which may utilize NR radio technology, CN106 / 115 may also communicate with another RAN (not shown) employing GSM, UMTS, CDMA2000, WiMAX, E-UTRA, or WiFi radio technology.

[0030] CN106 / 115 may also function as a gateway for WTRU102a, 102b, 102c, 102d to access PSTN108, the Internet 110, and / or other networks 112. PSTN108 may include a circuit-switched telephone network providing plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices using common communication protocols such as the transmission control protocol (TCP), the user datagram protocol (UDP), and / or the Internet protocol (IP) of the TCP / IP Internet Protocol suite. Network 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, network 112 may include another CN connected to one or more RANs, one or more RANs may employ the same RAT as RAN104 / 113 or a different RAT.

[0031] Some or all of the WTRUs 102a, 102b, 102c, and 102d in the communication system 100 may include multimode functionality (for example, WTRUs 102a, 102b, 102c, and 102d may include multiple transceivers for communicating with different radio networks via different radio links). For example, WTRU 102c shown in Figure 1A may be configured to communicate with base station 114a, which may employ cellular-based radio technology, and base station 114b, which may employ IEEE 802 radio technology.

[0032] Figure 1B is a system diagram showing an exemplary WTRU 102. As shown in Figure 1B, the WTRU 102 may include, among other things, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power supply 134, a global positioning system (GPS) chipset 136, and / or other peripherals 138. It will be understood that the WTRU 102 may include any partial combination of the aforementioned elements while maintaining consistency with a particular embodiment.

[0033] The processor 118 may be a general-purpose processor, a dedicated processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor 118 may perform signal coding, data processing, power control, input / output processing, and / or any other functions that enable the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to a transceiver 120 which can be coupled to a transmit / receive element 122. Although Figure 1B shows the processor 118 and transceiver 120 as separate components, it will be understood that the processor 118 and transceiver 120 may be integrated together in an electronic package or chip.

[0034] The transmit / receive element 122 may be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) via the air interface 116. For example, in one embodiment, the transmit / receive element 122 may be an antenna configured to transmit and / or receive RF signals. In one embodiment, the transmit / receive element 122 may be an emitter / detector configured to transmit and / or receive, for example, IR signals, UV signals, or visible light signals. In yet another embodiment, the transmit / receive element 122 may be configured to transmit and / or receive both RF signals and optical signals. It will be understood that the transmit / receive element 122 may be configured to transmit and / or receive any combination of radio signals.

[0035] Although the transmit / receive element 122 is shown as a single element in Figure 1B, the WTRU 102 may include any number of transmit / receive elements 122. More specifically, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for sending and receiving radio signals via the air interface 116.

[0036] The transceiver 120 may be configured to modulate the signal to be transmitted by the transmit / receive element 122 and to demodulate the signal to be received by the transmit / receive element 122. As described above, the WTRU 102 may have multimode capabilities. Therefore, the transceiver 120 may include multiple transceivers to enable the WTRU 102 to communicate via multiple RATs, such as NR and IEEE 802.11.

[0037] The processor 118 of the WTRU102 may be coupled to a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light-emitting diode (OLED) display unit) and may receive user input from these. The processor 118 may also output user data to the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128. In addition, the processor 118 may access information from any type of suitable memory, such as non-removable memory 130 and / or removable memory 132, and may store data in memory. The non-removable memory 130 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from memory not physically located on the WTRU 102, such as on a server or home computer (not shown), and store data in memory.

[0038] The processor 118 may receive power from the power supply 134 and be configured to distribute and / or control power to other components within the WTRU 102. The power supply 134 may be any suitable device for supplying power to the WTRU 102. For example, the power supply 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0039] The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) about the current location of the WTRU 102. In addition to, or instead of, the WTRU 102 may determine its location by receiving location information from base stations (e.g., base stations 114a, 114b) via the air interface 116 and / or based on the timing of signals received from two or more nearby base stations. It will be understood that the WTRU 102 may acquire location information by any preferred location determination method while maintaining consistency with a particular embodiment.

[0040] The processor 118 may be further coupled to other peripherals 138, which may include one or more software and / or hardware modules that provide additional features, functions, and / or wired or wireless connectivity. For example, peripherals 138 may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (for photos and / or videos), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an internet browser, a virtual reality and / or augmented reality (VR / AR) device, an activity tracker, and the like. The peripheral device 138 may include one or more sensors, one or more of which may be a gyroscope, accelerometer, Hall effect sensor, magnetometer, compass sensor, proximity sensor, temperature sensor, time sensor, geolocation sensor, altimeter, light sensor, touch sensor, magnetometer, barometer, gesture sensor, biometric sensor, and / or humidity sensor.

[0041] WTRU102 may include a full-duplex radio in which the transmission and reception of some or all of a signal (e.g., a signal associated with specific subframes of both UL (e.g., for transmission) and downlink (e.g., for reception)) may occur in parallel and / or simultaneously. The full-duplex radio may include an interference management unit for reducing and / or substantially eliminating self-interference via either hardware (e.g., chokes) or signal processing via a processor (e.g., via a separate processor (not shown) or processor 118). In one embodiment, WRTU102 may include a half-duplex radio for the transmission and reception of some or all of a signal (e.g., associated with specific subframes of either UL (e.g., for transmission) or downlink (e.g., for reception)).

[0042] Figure 1C is a system diagram showing RAN104 and CN106 according to one embodiment. As described above, RAN104 can communicate with WTRU102a, 102b, and 102c via the air interface 116 using E-UTRA wireless technology. RAN104 can also communicate with CN106.

[0043] RAN104 may include e-nodes B160a, 160b, and 160c, but it will be understood that RAN104 may include any number of e-nodes B while maintaining consistency with a particular embodiment. Each of e-nodes B160a, 160b, and 160c may include one or more transceivers for communicating with WTRU102a, 102b, and 102c via the air interface 116. In one embodiment, e-nodes B160a, 160b, and 160c may implement MIMO technology. Thus, e-node B160a may, for example, use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU102a.

[0044] Each of the e-nodes B160a, 160b, and 160c may be associated with a specific cell (not shown) and may be configured to handle wireless resource management decisions, handover decisions, user scheduling in UL and / or DL, etc. As shown in Figure 1C, the e-nodes B160a, 160b, and 160c may communicate with each other via the X2 interface.

[0045] The CN106 shown in Figure 1C may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (or packet gateway, PGW) 166. Although each of the aforementioned elements is shown as part of CN106, it should be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0046] The MME162 can be connected to each of the e-nodes B162a, 162b, and 162c within RAN104 via the S1 interface and can function as a control node. For example, the MME162 may be responsible for user authentication of WTRU102a, 102b, and 102c, activation / deactivation of bearers, and selection of a specific serving gateway during the initial connection of WTRU102a, 102b, and 102c. The MME162 may provide control plane functionality for switching between RAN104 and other RANs (not shown) employing other radio technologies such as GSM and / or WCDMA.

[0047] The SGW164 can connect to each of the e-nodes B160a, 160b, and 160c within RAN104 via the S1 interface. The SGW164 can generally route and forward user data packets to and from WTRU102a, 102b, and 102c. The SGW164 can perform other functions, such as anchoring the user plane during e-node B handovers, triggering paging when DL data is available to WTRU102a, 102b, and 102c, and managing and remembering the context of WTRU102a, 102b, and 102c.

[0048] SGW164 can connect to PGW166, which allows WTRU102a, 102b, and 102c to access packet-switched networks such as the Internet 110, thereby facilitating communication between WTRU102a, 102b, and 102c and IP-enabled devices.

[0049] CN106 can facilitate communication with other networks. For example, CN106 can provide WTRU102a, 102b, and 102c with access to a circuit-switched network such as PSTN108 to facilitate communication between WTRU102a, 102b, and 102c and conventional terrestrial line communication devices. For example, CN106 may include, or communicate with, an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that functions as an interface between CN106 and PSTN108. In addition, CN106 may provide WTRU102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0050] While the WTRU is shown as a wireless terminal in Figures 1A to 1D, in certain representative embodiments, such a terminal is intended to be able to use a wired communication interface with a communication network (e.g., temporarily or permanently).

[0051] In a typical embodiment, the other network 112 may be a WLAN.

[0052] In Infrastructure Basic Service Set (BSS) mode, a WLAN may have access points (APs) for the BSS and one or more stations (STAs) associated with the APs. APs may have access to or interfaces with a Distribution System (DS) or another type of wired / wireless network that carries traffic into and / or out of the BSS. Traffic originating from outside the BSS to an STA may arrive via an AP and be delivered to the STA. Traffic originating from an STA to a destination outside the BSS may be sent to an AP and delivered to its respective destination. Traffic between STAs within the BSS may be transmitted, for example, via an AP, where a source STA sends traffic to an AP, and the AP delivers the traffic to the destination STA. Traffic between STAs within the BSS may be considered and / or referred to as peer-to-peer traffic. Peer-to-peer traffic may be transmitted between a source STA and a destination STA (for example, directly between them) using a direct link setup (DLS). In certain representative embodiments, the DLS may use 802.11e DLS or 802.11z tunneled DLS (TDLS). A WLAN using Independent BSS (IBSS) mode may not have APs, and STAs within or using IBSS (e.g., all STAs) may communicate directly with one another. The IBSS communication mode may be referred to herein as the “ad hoc” communication mode.

[0053] When using the 802.11ac infrastructure operating mode or a similar operating mode, an AP may transmit beacons on a fixed channel, such as the primary channel. The primary channel may be of a fixed width (e.g., a 20 MHz bandwidth) or a dynamically set width via signaling. The primary channel may be the operating channel of the BSS and may be used by the STA to establish a connection with the AP. In a particular representative embodiment, a Carrier Sense Multiple Access with Collision Avoidance (CSMA / CA) scheme may be implemented, for example, in an 802.11 system. In the case of CSMA / CA, the STA, including the AP (e.g., all STAs), may sense the primary channel. If the primary channel is sensed / detected and / or determined to be busy by a particular STA, that STA may backoff. One STA (e.g., only one station) may transmit on a given BSS at any given time.

[0054] A high-throughput (HT) STA can use a 40MHz wide channel for communication, for example, by forming a 40MHz wide channel through a combination of a primary 20MHz channel and adjacent or non-adjacent 20MHz channels.

[0055] Very High Throughput (VHT) STAs can support channels with widths of 20 MHz, 40 MHz, 80 MHz, and / or 160 MHz. 40 MHz and / or 80 MHz channels can be formed by combining consecutive 20 MHz channels. 160 MHz channels can be formed by combining eight consecutive 20 MHz channels or two discontinuous 80 MHz channels, which may be referred to as an 80+80 configuration. In the 80+80 configuration, data can be passed through a segment parser that, after channel coding, can split the data into two streams. Inverse Fast Fourier Transform (IFFT) processing and time-domain processing can be performed independently on each stream. The streams may be mapped to two 80 MHz channels, and the data can be transmitted by the transmitting STA. At the receiver of the receiving STA, the above operation for the 80+80 configuration may be reversed, and the combined data can be transmitted to Medium Access Control (MAC).

[0056] Sub-1 GHz operating modes are supported in 802.11af and 802.11ah. Channel operating bandwidth and carrier are reduced in 802.11af and 802.11ah compared to those used in 802.11n and 802.11ac. 802.11af supports bandwidths of 5 MHz, 10 MHz, and 20 MHz in the TV White Space (TVWS) spectrum, while 802.11ah supports bandwidths of 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz using the non-TVWS spectrum. According to a typical embodiment, 802.11ah may support meter-type control / machine-type communications, such as MTC devices in a macro coverage area. MTC devices may have limited functionality, including support for specific bandwidths and / or limited bandwidths (e.g., support only). MTC devices may include batteries with battery life exceeding a threshold (e.g., to maintain very long battery life).

[0057] A WLAN system that can support multiple channels and channel bandwidths such as 802.11n, 802.11ac, 802.11af, and 802.11ah includes a channel that can be designated as the primary channel. The primary channel may have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel may be set and / or limited by the STA that supports the minimum bandwidth operating mode among all STAs operating in the BSS. In the 802.11ah example, even if the AP and other STAs in the BSS support operating modes of 2MHz, 4MHz, 8MHz, 16MHz, and / or other channel bandwidths, the primary channel of an STA that supports 1MHz mode (e.g., only 1MHz mode) (e.g., an MTC type device) may be 1MHz wide. Carrier detection and / or Network Allocation Vector (NAV) settings may depend on the status of the primary channel. For example, if the primary channel is busy because an STA (which only supports 1MHz operating mode) is transmitting to the AP, the entire available frequency band may be considered busy, even though a large portion of the frequency band remains idle and could potentially be available.

[0058] In the United States, the available frequency band for 802.11ah is 902MHz to 928MHz. In South Korea, the available frequency band is 917.5MHz to 923.5MHz. In Japan, the available frequency band is 916.5MHz to 927.5MHz. The total available bandwidth for 802.11ah is 6MHz to 26MHz, depending on the country code.

[0059] Figure 1D is a system diagram showing RAN113 and CN115 according to one embodiment. As described above, RAN113 can communicate with WTRU102a, 102b, and 102c via air interface 116 using NR radio technology. RAN113 can also communicate with CN115.

[0060] RAN113 may include gNB180a, 180b, and 180c, but it will be understood that RAN113 may include any number of gNBs while maintaining consistency with a particular embodiment. Each of gNB180a, 180b, and 180c may include one or more transceivers for communicating with WTRU102a, 102b, and 102c via the air interface 116. In one embodiment, gNB180a, 180b, and 180c may implement MIMO technology. For example, gNB180a and 108b may use beamforming to transmit signals to and / or receive signals from gNB180a, 180b, and 180c. Thus, gNB180a may, for example, use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU102a. In some embodiments, gNB180a, 180b, and 180c may implement carrier aggregation technology. For example, gNB180a may transmit multiple component carriers to WTRU102a (not shown). A subset of these component carriers may be on the unlicensed spectrum, while the remaining component carriers may be on the licensed spectrum. In some embodiments, gNB180a, 180b, and 180c may implement coordinated multi-point (CoMP) technology. For example, WTRU102a may receive coordinated transmissions from gNB180a and gNB180b (and / or gNB180c).

[0061] WTRU102a, 102b, and 102c may communicate with gNB180a, 180b, and 180c using transmissions associated with scalable numerology. For example, OFDM symbol intervals and / or OFDM subcarrier intervals may vary for different transmissions, different cells, and / or different parts of the radio transmission spectrum. WTRU102a, 102b, and 102c may communicate with gNB180a, 180b, and 180c using subframes or transmit time intervals (TTIs) of varying or scalable lengths (e.g., containing varying numbers of OFDM symbols and / or spanning varying lengths of absolute time).

[0062] gNB180a, 180b, and 180c can be configured to communicate with WTRU102a, 102b, and 102c in standalone and / or non-standalone configurations. In a standalone configuration, WTRU102a, 102b, and 102c can communicate with gNB180a, 180b, and 180c without accessing other RANs (e.g., e-nodes B160a, 160b, and 160c). In a standalone configuration, WTRU102a, 102b, and 102c can utilize one or more of gNB180a, 180b, and 180c as mobility anchor points. In a standalone configuration, WTRU102a, 102b, and 102c can communicate with gNB180a, 180b, and 180c using signals in the unlicensed band. In a non-standalone configuration, WTRU102a, 102b, and 102c may communicate with and connect to gNB180a, 180b, and 180c, while also communicating with and connecting to other RANs such as enodes B160a, 160b, and 160c. For example, WTRU102a, 102b, and 102c may implement a DC principle for substantially simultaneous communication with one or more gNB180a, 180b, and 180c and one or more enodes B160a, 160b, and 160c. In a non-standalone configuration, enodes B160a, 160b, and 160c may function as mobility anchors for WTRU102a, 102b, and 102c, and gNB180a, 180b, and 180c may provide additional coverage and / or throughput to service WTRU102a, 102b, and 102c.

[0063] Each of the gNB180a, 180b, and 180c may be associated with a specific cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, support for network slicing, dual connectivity, interworking between NR and E-UTRA, routing of user plane data to User Plane Functions (UPFs) 184a and 184b, access to control plane information, and routing to Mobility Management Functions (AMFs) 182a and 182b. As shown in Figure 1D, the gNB180a, 180b, and 180c may communicate with each other via the Xn interface.

[0064] The CN115 shown in Figure 1D may include at least one AMF182a, 182b, at least one UPF184a, 184b, at least one Session Management Function (SMF)183a, 183b, and optionally a Data Network (DN)185a, 185b. Although each of the aforementioned elements is shown as part of the CN115, it will be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0065] AMF182a and 182b may be connected to one or more of gNB180a, 180b, and 180c in RAN113 via the N2 interface and may function as control nodes. For example, AMF182a and 182b may be responsible for user authentication of WTRU102a, 102b, and 102c, support for network slicing (e.g., handling different PDU sessions with different requirements), selection of specific SMF183a and 183b, management of registration areas, termination of NAS signaling, mobility management, etc. Network slicing may be used by AMF182a and 182b to customize CN support for WTRU102a, 102b, and 102c based on the type of services utilized by WTRU102a, 102b, and 102c. For example, different network slices can be established for various use cases, such as services relying on ultra-reliable low latency (URLLC) access, services relying on enhanced massive mobile broadband (eMBB) access, and services using machine-type communication (MTC) access. The AMF162 may provide control plane functionality for switching between RAN113 and other RANs (not shown) employing other radio technologies such as LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies like WiFi.

[0066] SMF183a and 183b may be connected to AMF182a and 182b in CN115 via the N11 interface. SMF183a and 183b may also be connected to UPF184a and 184b in CN115 via the N4 interface. SMF183a and 183b may select and control UPF184a and 184b and configure traffic routing through UPF184a and 184b. SMF183a and 183b may perform other functions such as managing and assigning UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notifications. The PDU session type may be IP-based, non-IP-based, Ethernet-based, etc.

[0067] UPF184a, 184b may be connected via the N3 interface to one or more of gNB180a, 180b, 180c in RAN113, thereby providing WTRU102a, 102b, 102c with access to a packet-switched network such as the Internet 110, facilitating communication between WTRU102a, 102b, 102c and IP-enabled devices. UPF184, 184b may perform other functions such as packet routing and forwarding, enforcement of user plane policies, support for multi-homed PDU sessions, processing of user plane QoS, buffering of downlink packets, and providing mobility anchors.

[0068] CN115 can facilitate communication with other networks. For example, CN115 may include, or communicate with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that functions as an interface between CN115 and PSTN108. In addition, CN115 may provide WTRU102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, WTRU102a, 102b, 102c may be connected to local data networks (DN) 185a, 185b via UPF184a, 184b through an N3 interface to UPF184a, 184b and an N6 interface between UPF184a, 184b and DN185a, 185b.

[0069] With regard to Figures 1A-1D and the corresponding descriptions in Figures 1A-1D, with respect to one or more of the WTRU102a-d, base stations 114a-b, e-nodes B160a-c, MME162, SGW164, PGW166, gNB180a-c, AMF182a-b, UPF184a-b, SMF183a-b, DN185a-b, and / or any other devices described herein, one or more of the functions described herein may be performed by one or more emulation devices (not shown). An emulation device may be one or more devices configured to emulate one or more of the functions described herein. For example, an emulation device may be used to test other devices and / or simulate network and / or WTRU functions.

[0070] Emulation devices may be designed to implement one or more tests of other devices in a laboratory and / or operator network environment. For example, one or more emulation devices may perform one or more or all of their functions while fully or partially implemented and / or deployed as part of a wired and / or wireless network to test other devices in a communications network. One or more emulation devices may perform one or more or all of their functions while temporarily implemented / deployed as part of a wired and / or wireless network. Emulation devices may be directly coupled to other devices for the purpose of performing tests and / or tests using over-the-air wireless communication.

[0071] One or more emulation devices may perform one or more functions, including all of the above, while not implemented / deployed as part of a wired and / or wireless communication network. For example, an emulation device may be used in a test laboratory test scenario, and / or in an undeployed (e.g., test) wired and / or wireless communication network, to implement testing of one or more components. One or more emulation devices may be test equipment. Direct RF coupling and / or wireless communication via RF circuitry (e.g., which may include one or more antennas) may be used by the emulation device to transmit and / or receive data.

[0072] This application describes various embodiments, including tools, features, examples, models, and approaches. Many of these embodiments are described in detail, often in a manner that sounds restrictive, at least to illustrate individual features. However, this is for clarity and not to limit the use or scope of these embodiments. In fact, all of the different embodiments may be combined and interchangeable to provide further embodiments. Furthermore, these embodiments may be combined and interchangeable with embodiments described in previous applications.

[0073] The embodiments described and contemplated herein may be implemented in many different forms. Figures 5 to 11 described herein may provide some examples, but other examples are also contemplated. The considerations in Figures 5 to 11 are not intended to limit the scope of implementation forms. At least one of these embodiments generally relates to video encoding and decoding, and at least one other embodiment generally relates to transmitting a generated or encoded bitstream. These and other embodiments may be implemented as a computer-readable storage medium storing instructions for encoding or decoding video data according to any of the methods, apparatus, or described methods, and / or a computer-readable storage medium storing a bitstream generated according to any of the described methods.

[0074] In this application, the terms “reconstructed” and “decoded” may be used interchangeably, the terms “pixel” and “sample” may be used interchangeably, and the terms “image,” “picture,” and “frame” may be used interchangeably.

[0075] Various methods are described herein, each of which includes one or more steps or actions to achieve the described method. Unless a particular order of steps or actions is required for the normal operation of the method, the order and / or use of any particular steps and / or actions may be modified or combined. In addition, terms such as “first,” “second,” etc., may be used in various embodiments to modify elements, components, steps, actions, etc., such as “first decryption” and “second decryption.” The use of such terms does not imply a modified order of actions unless specifically required. Thus, in this example, the first decryption does not need to be performed before the second decryption, but may occur, for example, before, during, or overlapping with the second decryption.

[0076] Various methods and other embodiments described herein may be used to modify modules, such as the decoding module of a video encoder 200 and decoder 300, as shown in Figures 2 and 3. Furthermore, the subject matter disclosed herein may be applied to any type, format, or version of video coding, whether, for example, it is described in a standard or recommendation, whether it is existing or to be developed in the future, and whether it is an extension of any such standard or recommendation. Unless otherwise specifically indicated or technically excluded, the embodiments described herein may be used individually or in combination.

[0077] Various numerical values, such as 1, 2, 4, 7, 8, 16, 32, and 64, are used in the embodiments described in this application. These and other specific values ​​are for illustrative purposes only, and the embodiments described are not limited to these specific values.

[0078] Figure 2 shows an exemplary video encoder 200. Variations of the exemplary encoder 200 are intended, but the encoder 200 is described below without describing all expected variations for clarity.

[0079] Before encoding, the video sequence may undergo pre-encoding processing (201), for example, by applying a color conversion (e.g., from RGB 4:4:4 to YCbCr 4:2:0) to the input color picture, or by remapping the input picture components (e.g., using histogram equalization of one of the color components) to obtain a signal distribution that is more resilient to compression. Metadata (e.g., including film grain parameters determined by pre-processing as described herein) may be associated with the pre-processing and appended to the bitstream.

[0080] In encoder 200, the picture is encoded by encoder elements as described below. The picture to be encoded is partitioned (202) and processed in units of coding units (Cu), for example. Each unit is encoded using either intra-mode or inter-mode, for example. If the unit is encoded in intra-mode, intra-prediction is performed (260). In inter-mode, motion estimation (275) and motion compensation (270) are performed. The encoder decides whether to use intra-mode or inter-mode for encoding the unit (205), and indicates the intra / inter decision, for example, by a prediction mode flag. The prediction residual is calculated, for example, by subtracting the predicted blocks from the original image blocks (210).

[0081] Next, the predicted residual is transformed (225) and quantized (230). The quantized transformation coefficients, as well as the motion vector and other syntactic elements, are entropy coded to output a bitstream (245). The encoder can skip the transformation and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transformation and quantization, i.e., the residual is coded directly without applying any transformation or quantization process.

[0082] The encoder decodes the encoded blocks to provide a reference for further prediction. The quantized transformation coefficients are inversely quantized (240), inversely transformed (250), and the predicted residuals are decoded. The decoded predicted residuals and the predicted blocks are combined (255) to reconstruct the image blocks. An in-loop filter (265) is applied to the reconstructed picture to reduce encoding artifacts, for example, by performing deblocking / SAO (Sample Adaptive Offset) filtering. The filtered image is stored in a reference picture buffer (280).

[0083] Figure 3 shows an example of a video decoder. In the exemplary decoder 300, the bitstream is decoded by the decoder elements as described below. The video decoder 300 generally performs a decoding path that is the reverse of the encoding path described in Figure 2. The encoder 200 also generally performs video decoding as part of encoding the video data.

[0084] In particular, the decoder input includes a video bitstream that may be generated by the video encoder 200. First, the bitstream is entropy-decoded to obtain transformation coefficients, motion vectors, and other coded information (330). Picture partitioning information indicates how the picture is partitioned. Thus, the decoder may partition the picture according to the decoded picture partitioning information (335). The transformation coefficients are inversely quantized (340) and inversely transformed (350) to decode the prediction residuals. The image blocks are reconstructed by combining the decoded prediction residuals and the predicted blocks (355). The predicted blocks may be obtained from intra-prediction (360) or motion-compensated prediction (i.e., inter-prediction) (375) (370). An in-loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (380).

[0085] The decoded picture may undergo further post-decoded processing (385), such as reverse color transformation (e.g., conversion from YcbCr4:2:0 to RGB4:4:4) or reverse remapping, which performs the reverse of the remapping process performed in pre-encoded processing (201). Post-decoded processing can use metadata derived in pre-encoded processing and signaled within the bitstream. For example, the decoded image (e.g., after applying the in-loop filter (365) and / or after post-decoded processing (385) if post-decoded processing is used) may be sent to a display device for rendering to the user.

[0086] Figure 4 shows an example of a system in which various embodiments and examples described herein may be implemented. System 400 may be embodied as a device comprising various components described below and configured to perform one or more of the embodiments described herein. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of System 400 may be embodied individually or in combination as a single integrated circuit (IC), multiple ICs, and / or individual components. For example, in at least one example, the processing elements and encoder / decoder elements of System 400 are distributed across multiple ICs and / or individual components. In various examples, System 400 is communicably coupled to one or more other systems or other electronic devices, for example, via a communication bus or via dedicated input and / or output ports. In various examples, System 400 is configured to implement one or more of the embodiments described herein.

[0087] System 400 includes, for example, at least one processor 410 configured to execute loaded instructions in order to implement various embodiments described herein. The processor 410 may include embedded memory, input / output interfaces, and various other circuits known in the art. System 400 includes at least one memory 420 (e.g., a volatile memory device and / or a non-volatile memory device). System 400 includes a storage device 440 which may include non-volatile memory and / or volatile memory, including, but not limited to, electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, magnetic disk drives, and / or optical disk drives. The storage device 440 may, in non-limiting examples, include an internal storage device, an attached storage device (including removable and non-removable storage devices), and / or a network-accessible storage device.

[0088] System 400 includes, for example, an encoder / decoder module 430 configured to process data to provide encoded or decoded video, the encoder / decoder module 430 of which may include its own processor and memory. The encoder / decoder module 430 represents a module that may be included in a device for performing encoding and / or decoding functions. As is known, the device may include one or both of the encoding and decoding modules. Furthermore, the encoder / decoder module 430 may be implemented as a separate element of System 400, or it may be incorporated into the processor 410 as a combination of hardware and software, as is known to those skilled in the art.

[0089] Program code loaded into the processor 410 or encoder / decoder 430 to perform various embodiments described herein may be stored in the storage device 440 and subsequently loaded into memory 420 for execution by the processor 410. According to various examples, one or more of the processor 410, memory 420, storage device 440, and encoder / decoder module 430 may store one or more of various items during the execution of the processes described herein. Such stored items may include, but are not limited to, input video, decoded video, or a portion of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0090] In some examples, internal memory of the processor 410 and / or encoder / decoder module 430 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other examples, external memory of the processing device (for example, the processing device may be either the processor 410 or the encoder / decoder module 430) is used for one or more of these functions. The external memory may be memory 420 and / or storage device 440, for example, dynamic volatile memory and / or non-volatile flash memory. In some examples, for example, external non-volatile flash memory is used to store the television's operating system. In at least one example, high-speed external dynamic volatile memory, such as RAM, is used as working memory for video encoding and decoding operations.

[0091] Inputs to the elements of System 400 can be provided via various input devices, as shown in Block 445. Such input devices include, but are not limited to, (i) a radio frequency (RF) section that receives RF signals transmitted over the air by a broadcasting station, for example, (ii) a component (COMP) input terminal (or a set of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Another example not shown in Figure 4 is composite video.

[0092] In various examples, the input device of block 445 has associated input processing elements, each of which is known in the art. For example, the RF portion may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or band-limiting a signal to a certain frequency band), (ii) down-converting the selected signal, (iii) again band-limiting to a narrower frequency band so as to select a signal frequency band, which (for example) in a particular embodiment may be referred to as a channel, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF portion of various examples includes one or more elements for performing these functions, e.g., frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF portion may include wavelength tuners that perform various of these functions, including, for example, down-converting the received signal to a lower frequency (e.g., an intermediate frequency or a frequency close to the baseband) or to the baseband. In one example of a set-top box, the RF section and its associated input processing elements perform frequency selection by receiving, filtering, down-converting, and filtering again to a desired frequency band of RF signals transmitted via a wired (e.g., cable) medium. In various examples, the order of the above (and other) elements may be rearranged, some of these elements may be removed, and / or other elements performing similar or different functions may be added. Adding elements may include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various examples, the RF section may include an antenna.

[0093] The USB and / or HDMI terminals may include their respective interface processors for connecting the system 400 to other electronic devices via USB and / or HDMI connections. It should be understood that various forms of input processing, such as Reed-Solomon error correction, may be implemented as needed, for example, in a separate input processing IC or within the processor 410. Similarly, forms of USB or HDMI interface processing may be implemented as needed, within a separate interface IC or within the processor 410. For example, demodulated, error-corrected, and demultiplexed streams may be provided to various processing elements, including a processor 410 and an encoder / decoder 430, which may, as needed, process the data stream for display on an output device.

[0094] Various elements of system 400 may be provided within an integrated housing, and these elements may be interconnected within the integrated housing and transmit data between them using an internal bus known in the art, such as an Inter-IC (Inter-IC, I2C) bus, wiring, and printed circuit board.

[0095] System 400 includes a communication interface 450 that enables communication with other devices via a communication channel 460. The communication interface 450 may, but is not limited to, include transceivers configured to transmit and receive data via the communication channel 460. The communication interface 450 may, but is not limited to, include a modem or a network card, and the communication channel 460 may be implemented, for example, in a wired and / or wireless medium.

[0096] In various examples, data is streamed to system 400 or otherwise provided using a wireless network such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE stands for Institute of Electrical and Electronics Engineers). In these examples, the Wi-Fi signal is received via a communication channel 460 and a communication interface 450 adapted for Wi-Fi communication. In these examples, communication channel 460 is typically connected to an access point or router that provides access to an external network, including the Internet, enabling streaming applications and other over-the-top communications. In other examples, the streamed data is provided to system 400 using a set-top box that distributes data via an HDMI connection in input block 445. Still other examples provide the streamed data to system 400 using an RF connection in input block 445. As described above, various examples provide data in a non-streaming manner. Additionally, various examples use wireless networks other than Wi-Fi, e.g., cellular networks or Bluetooth® networks.

[0097] System 400 can provide output signals to various output devices, including a display 475, a speaker 485, and other peripheral devices 495. Various examples of the display 475 include, for example, one or more of a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 475 may be for a television, tablet, laptop, mobile phone, or other device. The display 475 may also be integrated with other components (for example, in a smartphone) or separate (for example, an external monitor for a laptop). Other peripheral devices 495, in various examples, include one or more of a standalone digital video disc (or digital multipurpose disc) (both terms refer to DVDs), a disc player, a stereo system, and / or a lighting system. Various examples use one or more peripheral devices 495 that provide functions based on the output of System 400. For example, a disc player performs the function of playing back the output of System 400.

[0098] In various examples, control signals are communicated between the system 400 and the display 475, speaker 485, or other peripheral devices 495 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable inter-device control with or without user intervention. Output devices may be communicably coupled to the system 400 via dedicated connections through their respective interfaces 470, 480, and 490. Alternatively, output devices may be connected to the system 400 using a communication channel 460 via a communication interface 450. The display 475 and speaker 485 may be integrated into a single unit with other components of the system 400 in an electronic device such as a television. In various examples, the display interface 470 may include a display driver, such as a timing controller (TCon) chip.

[0099] The display 475 and speaker 485 may, alternatively, be separate from one or more of the other components, for example, if the RF portion of input 445 is part of a separate set-top box. In various examples where the display 475 and speaker 485 are external components, the output signal may be provided via a dedicated output connection, for example, including an HDMI port, a USB port, or a COMP output.

[0100] The example may be implemented by the processor 410, by hardware, or by computer software implemented by a combination of hardware and software. In a non-limiting example, the example may be implemented by one or more integrated circuits. The memory 420 may be of any type appropriate for the technical environment and, in a non-limiting example, may be implemented using any suitable data storage technology such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 410 may be of any type appropriate for the technical environment and, in a non-limiting example, may include one or more of microprocessors, general-purpose computers, dedicated computers, and processors based on multi-core architectures.

[0101] Various implementations involve decoding. As used in this application, “decoding” can encompass all or part of the processes performed on a received encoded sequence to produce, for example, a final output suitable for display. In various examples, such processes typically include one or more of the processes normally performed by a decoder, e.g., entropy decoding, inverse quantization, inverse transform, and differential decoding. In various examples, such processes may also, or alternatively, include processes performed by a decoder in one of the various implementations described in this application, e.g., receiving a video picture, receiving an indication of a ratio value corresponding to the video picture, adjusting the number of codewords of a mapping function associated with the video picture based on the ratio value, and decoding the video picture based on the mapping function.

[0102] For further examples, in one instance, “decoding” refers only to entropy decoding; in another instance, “decoding” refers only to differential decoding; and in yet another instance, “decoding” refers to a combination of entropy decoding and differential decoding. Whether the phrase “decoding process” is intended to refer specifically to a subset of operations or to a broader decoding process in general becomes clear from the context of the particular description and is expected to be well understood by those skilled in the art.

[0103] Various implementations involve encoding. As with the above considerations regarding "decoding," "encoding" as used in this application may encompass all or part of the processes performed on an input video sequence to generate an encoded bitstream. In various examples, such processes may include one or more of the processes typically performed by an encoder, for example, receiving a video picture, receiving an indication of a ratio value corresponding to the video picture, adjusting the number of codewords of a mapping function associated with the video picture based on the ratio value, and decoding the video picture based on the mapping function.

[0104] For further examples, in one instance, “encoding” refers only to entropy coding; in another instance, “encoding” refers only to differential coding; and in yet another instance, “encoding” refers to a combination of differential coding and entropy coding. Whether the phrase “encoding process” is intended to refer specifically to a subset of operations or to a broader encoding process in general becomes clear from the context of the particular description and is expected to be well understood by those skilled in the art.

[0105] It should be noted that the coding syntax used herein for syntactic elements, such as intensity intervals, grain parameters, block offsets, and scaling factors, is descriptive terminology. Therefore, it does not preclude the use of other syntactic element names.

[0106] When a diagram is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding device. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0107] The implementation forms and embodiments described herein may be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even if considered only in the context of a single form of implementation (e.g., considered only as a method), the implementation forms of the considered features may also be implemented in other forms (e.g., apparatus or programs). Apparatus may be implemented, for example, with appropriate hardware, software, and firmware. The method may be implemented in a processor, for example, a computer, microprocessor, integrated circuit, or programmable logic device, generally referring to a processing device. Processors also include communication devices, such as computers, mobile phones, portable / personal digital assistants ("personal digital assistants, PDAs"), and other devices that facilitate the communication of information between end users.

[0108] References to “one example” or “example,” or “one implementation” or “a certain implementation,” and other variations thereof, mean that the specific features, structures, characteristics, etc. described in relation to the example are included in at least one example. Therefore, the appearance of phrases such as “one example” or “in one example” or “in one implementation” or “in a certain implementation,” and any other variations, found in various places throughout this application, do not necessarily all refer to the same example.

[0109] Additionally, this application may refer to “determining” various types of information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from memory. Acquiring may include receiving, retrieving, structuring, generating, and / or determining.

[0110] Furthermore, this application may also refer to “accessing” various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0111] Additionally, this application may refer to “receiving” various types of information. Receiving is intended to be a broad term, similar to “accessing.” Receiving information may include, for example, accessing information or retrieving information (for example, from memory). Furthermore, “receiving” typically accompanies, in some way, operations such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0112] For example, in the cases of "A / B", "A and / or B", and "at least one of A and B", please understand that the use of any of the following " / ", "and / or", and "at least one of" is intended to cover the selection of only the first option (A), only the second option (B), or both options (A and B). As further examples, in the cases of "A, B, and / or C" and "at least one of A, B, and C", such phrasing is intended to cover the selection of only the first option (A), only the second option (B), only the third option (C), only the first and second options (A and B), only the first and third options (A and C), only the second and third options (B and C), or all three options (A, B, and C). This may be extended to the number of items listed, as is obvious to anyone with ordinary knowledge of this and related technologies.

[0113] Furthermore, as used herein, the word “signaling” refers, among other things, to pointing something to the corresponding decoder. Encoder signals may include, for example, the number of intensity intervals, the number of model values, grain parameters, grain identification, scaling factors, etc. In this way, in one example, the same parameters are used on both the encoder and decoder sides. Therefore, for example, the encoder can send certain parameters to the decoder (explicit signaling), and as a result the decoder can use the same certain parameters. Conversely, if the decoder already has certain parameters as well as other parameters, signaling may be used without transmission (implicit signaling) to allow the decoder to easily recognize and select the certain parameters. Bit savings are achieved in various examples by avoiding the transmission of any actual functions. It will be understood that signaling can be achieved in various ways. For example, one or more syntactic elements, flags, etc., may be used to signal information to the corresponding decoder in various examples. The preceding discussion concerns the verb form of the word "signaling," but the word "signaling" can also be used as a noun in this specification.

[0114] As will be apparent to those skilled in the art, implementations can generate a wide variety of signals, for example, that are formatted to carry information that can be stored or transmitted. This information may include, for example, instructions for performing a method, or data generated by one of the implementations described. For example, a signal may be formatted to carry the bitstream of the example described. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. The signal may be transmitted over a wide variety of different wired or wireless links, as is known. The signal may be stored in a processor-readable medium and may be accessed or received from a processor-readable medium.

[0115] Many examples are provided herein. The features of the examples may be provided individually or in any combination across various categories and types of claims. Furthermore, the examples may include, individually or in any combination across various categories and types of claims, one or more of the features, devices, or embodiments described herein. For example, the features described herein may be implemented in a bitstream or signal containing information generated as described herein. The information enables a decoder to decode the bitstream, and the encoder, bitstream, and / or decoder follow any of the embodiments described. For example, the features described herein may be implemented by creating, transmitting, receiving, and / or decoding a bitstream or signal. For example, the features described herein may be implemented by a method, process, apparatus, medium for storing instructions, medium for storing data, or signal. For example, the features described herein may be implemented by a TV, set-top box, mobile phone, tablet, or other electronic device that performs decoding. A TV, set-top box, mobile phone, tablet, or other electronic device may display the resulting image (e.g., an image from residual reconstruction of a video bitstream) (e.g., using a monitor, screen, or other type of display). A TV, set-top box, mobile phone, tablet, or other electronic device may receive and decode a signal containing the encoded image. The coding loop may be updated on the encoder and decoder side using a lumamapping function based on the reconstructed samples (e.g., already) generated in the current slice or picture. Compression efficiency may be affected. The bitrate may be reduced while maintaining quality, and / or quality may be improved while maintaining the bitrate. Features associated with block partitioning are described herein.

[0116] The features described herein may be associated with lumens mapping with chroma scaling (LMCS) for SDR content. In some sequences (e.g., for captured content), the data range of the lumens sample may or may not correspond to 1024 levels for a 10-bit signal. The data range of the lumens sample may correspond to the range of 64 to 940 (e.g., the standard range). The LMCS function may modify the lumens of the content to take advantage of its (e.g., 10-bit) capabilities. The mapping function may be applied to the lumens based on a pre-calculated LUT calculated on the encoder side. The LUT value may be defined by the encoder based on the characteristics of the content. A 10-bit representation (e.g., 1024 levels) may be divided into 16 segments (e.g., intervals) of 64 levels each. The derivation of the LMCS mapping function in the encoder may include the following steps in random access mode.

[0117] Video parameters can be calculated. For a video or video chunk, parameters such as the frame luma sample histogram and local variance can be extracted segment by segment on the I-frame. The histogram result can be given by 16 values ​​(corresponding to 16 segments). In the case of a 10-bit signal, a segment may correspond to 1 / 16 of the range of values ​​from 0 to 1023. Hist[0] may correspond to the occurrence of samples with luma values ​​from 0 to 63 on the picture (for example), Hist[1] may correspond to the occurrence of samples with luma values ​​from 64 to 127, and so on. Hist[i] for i=0 to 15 can be normalized such that Hist[0]+.+Hist

[15] =1.0.

[0118] The variance of a sample P with the value of Luma P(x,y) is 5 * The variance can be calculated by subtracting the square of the mean value on the window from the sum of the squared values ​​of the pixels on the window divided by the window size. The variance can be summed with the variances of the same section, and the section can be determined by the luma value of the current sample.

[0119] It can be determined whether to apply LMCS. If both Hist[0] and Hist

[15] are below a threshold (e.g., 0.001), the LMCS function may be activated (e.g., codewords (CW) from 0-63 and 960-1023 may or may not be used to code the sequence).

[0120] The global number of codewords can be calculated. 896 codes can be used to code values ​​between 64 and 959. The global number of codewords (NbCW) can be a multiple of 14. The NbCW can be determined based on a threshold calculated from the histogram and variance values.

[0121] CW can be refined per segment. The number of CWs per segment may be the same (for example, NbCW may be a multiple of 14). The refinement process may use segment histograms and variance values ​​to modify the number of CWs per segment (for example, listed under m_binCW[i] for segment at index i) around the initial value being calculated. The refinement process may include:

[0122]

number

[0123] A correction may be applied to ensure that the total number of CWs in each category does not exceed 1023.

[0124] The example may include QP <= 22. To determine whether NbCW is corrected, a threshold may be used to apply a process to QP <= 22. If NbCW is corrected, the value may be set to 924.

[0125] LUTs can be generated. In the example, a LUT can be generated corresponding to the number of CWs per segment. A LUT corresponding to the number of CWs per segment may be a forward mapping LUT and can be used when there is a transformation from an unmapped region to a mapped region. In parallel, a reverse mapping LUT can be generated per segment based on the same number of CWs. A reverse mapping LUT can be used when there is a transformation from a mapped region to an unmapped region. Forward and reverse mapping can be applied to video of frames (e.g., frames other than I-frames).

[0126] LMCS parameters can be signaled. LMCS parameters can be signaled by an encoder in APS.

[0127] [Table 1]

[0128] The decoder decodes this information, reconstructs forward and reverse mapping functions, and may use these functions during the decoding process (for example, forward mapping may be applied to interpredicted luma samples, and reverse mapping may be applied after the reconstruction process and before intra-loop filtering).

[0129] The features described herein may be associated with luminous mapping with chroma scaling (LMCS) for HDR content. In some SDR sequences (e.g., captured content SDR sequences), the data range of the luminous signal may or may not correspond to 1024 levels (e.g., a 10-bit signal), and may correspond to a range of 64 to 940. For HDR signals, when the HDR signal is in PQ format, the maximum value of 1023 may correspond to a peak of 10000 cd / m2, so the code range may be reduced, and this may or may not be used. Grading may use peaks of up to 1000, 4000, or 5000 cd / m2, which may result in codewords around 720, 850, and 875, respectively. The LMCS function may modify the luminous signal of the content to take advantage of the 10-bit capability. The mapping function may be applied to the luminous signal due to a pre-calculated function or LUT calculated on the encoder side. The function or LUT values ​​may be defined by the encoder based on the characteristics of the content. A 10-bit representation (1024 levels) can be divided into a given number of N segments (e.g., 16 intervals) of 1024 / N levels (e.g., 64 if 16 intervals are used). The luma mapping function may be defined as a piecewise linear function defined by N segments. As with SDR, the forward mapping function may be modeled by a piecewise linear model of 16 segments, for example.

[0130] For each input value x (e.g., [0, 1023] for a 10-bit signal), a forward mapping LUT can be constructed from a piecewise linear function. In parallel, an inverse mapping LUT can be generated based on the same piecewise linear function. The inverse mapping LUT can be used when a transformation from a mapped region to an unmapped region is required. Both forward and inverse mapping can be applied to video pictures.

[0131] A mapping function may correspond to, or be defined as, a set of CW numbers for each segment, where a certain (e.g., one) number of CWs may be assigned to each segment of the function.

[0132] The encoder may derive the LMCS function by extracting video parameters (e.g., histogram, variance) from the I-frame and calculating the number of codewords per segment based on the histogram and local variance. Since the number of CWs per segment is content-dependent, the value may be calculated by the encoder, and the number of CWs per segment may be transmitted in the bitstream for each I-frame for each segment. There may be (e.g., additional) costs for transmitting the data. The amount of data transmitted may be limited by selecting a limited number of predefined sets (e.g., 4, 8, or 16) of the number of CWs per segment transmitted. The encoder may determine the best set of CWs per segment for the current chunk (e.g., segment) of the video. As described herein, the best set of CWs per segment for the current chunk may be referred to as the code.

[0133] The code may be selected by performing a brute-force rate-distortion optimization (RDO), which may include encoding the picture for all possible codes and selecting the code that leads to the best rate-distortion value. For a given code, a full encoding of the picture may be performed. For a full encoding, a partitioning tree and coding mode may be selected from among the possible partitioning trees and available coding modes for the picture's CTU that leads to the best rate-distortion criterion. The rate-distortion criterion may include a metric that mixes the bitrate R (code) obtained when encoding the picture (or video) and the distortion D (code) corresponding to the distance between the decoded picture samples and the original picture samples, as shown in equation (1).

[0134]

number

[0135] The parameter (λ) is a parameter (e.g., a Lagrangian parameter) that may depend on the quantization parameter (QP) used to encode the picture. D may correspond to the mean squared error or the mean absolute error.

[0136] As described herein, the coding cost of signaling codes can be reduced. The best set of "CW per segment for the current chunk" (e.g., a code) can be selected, and the selection information can be indicated in the video data (e.g., transmitted in the bitstream). The process of estimating the code can be performed based on an analysis of the input picture to be encoded. The signaling mechanism can lead to a reduction in the signaling cost of codes.

[0137] Codes can be identified based on input features derived from input picture (or picture) samples used in a gradient boosting decision tree to select the best code from a set of possible codes.

[0138] Features derived from input picture samples can be obtained based on the minimum and maximum values ​​of the video, a histogram of the picture (or multiple picture) samples, the variance of the picture samples, temporal information, or parameters extracted from the picture (or multiple picture) samples.

[0139] A set of possible codes can be obtained from the initial code by applying a fixed ratio to the initial code's value. A set of possible codes can be obtained from the initial code by changing at least one (e.g., one) value of the initial code. Features can be computed from a mapped input picture sample using at least one (e.g., one) code from the set of possible codes. An exemplary process may involve signaling code-related information into the bitstream to reduce code signaling costs.

[0140] The features described herein may be associated with gradient boosting decision trees. The best code may be selected using a regression algorithm called the LightGBM algorithm. LightGBM may be a gradient boosting framework that uses a tree-based learning algorithm.

[0141] Once the LightGBM is trained, the inference process may aim to predict the gain when using a given code with input features computed on a picture. A set of input features used in the training and inference processes can be defined.

[0142] The examples described herein may be applicable to types of machine learning algorithms, such as random forests and neural networks with deep learning algorithms.

[0143] To train a LightGBM model, a dataset can be obtained by encoding a set of test sequences in different QPs (e.g., QP22, 27, 32, 37) using different codes. The set of codes from which the best code will be selected can be defined, for example, by applying a different ratio to the number of codewords in the initial code that can correspond to a discriminative mapping function or a predefined mapping function. The process of calculating the codes with ratio R may include: defining MIN and MAX values ​​for the current frame; defining the total number N of divisions between the division to which the MIN value belongs and the division to which the MAX value belongs; and applying the ratio R to the number of codewords for the divisions (e.g., ratio R is condition N). * 64 * R<2 n (This can satisfy the following conditions), where n may be the bit depth of the video sample.

[0144] In the example, the initial code, for a 10-bit signal, can correspond to 64 CWs for each of 16 segments. N=14 when the minimum video sample count is 64 and the maximum is 960. Applying a ratio R of 1.1 gives the number of CWs per segment as ROUND(64). * 1.1) means that it can be equal to 70. In the example, 14 * Since 70 = 980 < 1024, a ratio R = 1.1 may be acceptable.

[0145] In the example process, the initial code may be represented as a LUT, and the elements of the LUT may be scaled in equation (2) as follows: scaledLUT[x]=Min(2^n-1,round(R * LUT[x])), where x = 0 ~ 2^n(2), Here, round(.) can return the nearest integer value.

[0146] For a given segment i, nbCWs(i) is (scaledLUT[(i+1) *K]-scaledLUT[i * K]), where K=2^n / maxNbPieces, and maxNbPieces is the maximum number of segments (for example, for a 10-bit signal, maxNbPieces=16 and K=64).

[0147] For sequences and codes, coding gains and input feature values may be derived. The coding gain may be a BD-rate coding gain, a PSNR coding gain, or a probability of coding gain. The BD-rate coding gain may provide an average bitrate variation indication between an anchor (e.g., a sequence encoded with an initial code) and a test (e.g., a sequence encoded with a code) across four QPs. A negative value may indicate a bitrate reduction that may be associated with a coding gain. The PSNR coding gain may provide an average PSNR variation indication between an anchor (e.g., a sequence encoded with an initial code) and a test (e.g., a sequence encoded with a code) across four QPs. The probability of coding gain may give the average probability of coding gain between an anchor (e.g., a sequence encoded with an initial code) and a test (e.g., a sequence encoded with a code) across four QPs. The gain may be a BD-rate coding gain, a PSNR coding gain, or a probability of coding gain. The cost function of the training process may be the L1 or L2 distance between the predicted gain and the actual gain.

[0148] Exemplary features may be used for training and inference. The input features of the LightGBM model may include the following: the MIN and MAX values of picture samples, and / or the histogram of picture samples for each segment. For a certain segment, the number of pixels belonging to the segment may be calculated as follows.

[0149] [Equation]

[0150] The mean variance of pixel samples for each category can be calculated. For a given category, the mean variance of pixels belonging to that category can be calculated as follows:

[0151]

number

[0152] The temporal variance can be calculated between the current picture Icur and the motion-compensated distant picture Imc. In the example, the temporal variance between the current picture Icur and the distant picture with a temporal distance d can be calculated in equation (3) as follows:

[0153]

number

[0154] The parameters (e.g., the sum of squared errors and / or the number of non-zero errors) when applying the forward mapping function (fwdMap) and the inverse mapping function (invMap) to a picture sample can be shown in equations (4) and (5).

[0155]

number

[0156] For rounding purposes, the piecewise linear inverse mapping function may or may not be the exact inverse of the piecewise linear forward mapping function.

[0157] The characteristics of a picture or video (e.g., any other characteristics) can be used as features for a decision algorithm.

[0158] Figure 5 shows an exemplary flowchart of inference for ratio selection. Figure 5 provides an exemplary flowchart of the inference process in an example of code defined by a fixed ratio. The input to the process may be a picture, for example, the first picture of the clip to be coded or the chunk to be coded when a video is coded chunk by chunk (for example, when random access coding is performed using an intra picture coded in a bitstream). In 501, the parameter maxGain may be set to 0. The parameter may store the value of the maximum gain among the possible ratio values ​​to be tested. In 502, a loop may be performed on the possible values ​​of the ratio. In 503, the features to be considered may be calculated. In 504, inference may be performed using a machine learning model with the features from 503 as input. Gain estimation may be performed when using the tested ratio values. The estimated gain in 504 may be compared to maxGain in 505. In 506, if the estimated gain is higher than maxGain, the maxGain value may be updated with the gain, and the current ratio may be stored as best_ratio. If the estimated gain is not higher than maxGain, the next step (e.g., 507) may be performed. In 507, it may be checked whether the end of the loop over possible ratio values ​​has been reached. If the end of the loop has not been reached, the next ratio value (e.g., in 502) may be tested. If the end of the loop has been reached, the output of the inference process may be an estimate of the best ratio corresponding to the value best_ratio.

[0159] The number of CWs per segment can be defined individually for each segment. In the example, the set of codes from which the best code will be selected can be defined by modifying at least a certain number (e.g., one) of codewords for the segments of the initial code. The initial code may have an equal number of CWs per segment. The process of calculating the codes belonging to a set of codes may include: defining MIN and MAX values ​​for the current frame; defining the total number N of segments between the segment to which the MIN value belongs and the segment to which the MAX value belongs; and / or modifying at least a certain number (e.g., one) of codewords for the segments of the initial code (e.g., the set of codes may satisfy the condition Σ piece nbCW(category)<2 n (where n is the bit depth of the video sample). In the example, for a 10-bit signal, the initial code could correspond to 64 CWs for each of 16 divisions. If the minimum video is 64 and the maximum is 960, then N=14, and 896 levels out of 1023 may be used. It may be possible to consider codes where the number of CWs in at least one division is greater than 64. A set of codes can be defined in this way, and the best one for the sequence can be determined.

[0160] For sequences and codes, coding gain and input feature values ​​can be derived.

[0161] The cost function of the training process can be defined as the L1 or L2 distance between the predicted gain and the actual gain. Figure 6 shows an exemplary flowchart of inference for a set of codes. Figure 6 shows an exemplary inference process for an example of a code selected from the set of codes. The input to the process can be a picture, for example, the first picture of the clip to be coded or the chunk to be coded when a video is coded chunk by chunk (for example, when random access coding is performed using normal intra pictures coded in a bitstream). In 601, the parameter maxGain may be set to 0. maxGain may store the value of the maximum gain among the possible codes to be tested. In 602, a loop may be executed for the possible codes. In 603, features to be considered may be computed. In 604, inference may be performed using the features from 603 as input, with a machine learning model. In 603, an estimate of the gain may be performed when using the tested code. The estimated gain from 604 may be compared to maxGain in 605. In 606, if the estimated gain is higher than maxGain, the maxGain value may be updated with the gain, and the current code may be stored as best_code. If the estimated gain is not higher than maxGain, the next step may be performed (e.g., in 607). In 607, it may be checked whether the end of the loop across possible ratio values ​​has been reached. If the end of the loop has not been reached, the next code may be tested (e.g., in 602). If the end of the loop has been reached, the output of the inference process may be the estimated best code corresponding to the value best_code.

[0162] Features can be derived from mapped input picture samples using at least one (e.g., one) code from a set of possible codes. In the example, features can be extracted from a video to which a forward mapping function has (e.g., already) been applied. In the example, if the features extracted from the content are based on a histogram, the number of samples belonging to a given segment can be calculated as follows:

[0163]

number

[0164] Figure 7 shows an exemplary flowchart of inference using features calculated based on mapped video. Figure 7 shows an example flowchart of the inference process when the code is defined by a fixed ratio. The input to the process may be a picture, for example, the first picture of the clip to be coded or the chunk to be coded when video is coded chunk by chunk (for example, when random access coding is performed using normal intra pictures coded in a bitstream). In 701, the parameter maxGain may be set to 0. maxGain may store the value of the largest gain among the possible ratio values ​​to be tested. In 702, a loop may be performed for the possible values ​​of the ratio. In 703, the features to be considered may be calculated on the mapped video corresponding to the ratios (e.g., all of them). In 704, inference may be performed using a machine learning model with the features from 703 as input. Gain estimation may be performed when using the ratio values ​​tested. The estimated gain from 704 may be compared to maxGain in 705. In step 706, if the estimated gain is higher than maxGain, the maxGain value may be updated with the gain, and the best ratio may be stored as best_ratio. If the estimated gain is not higher than maxGain, the next step may be performed (e.g., in step 707). In step 707, it may be checked whether the end of the loop over possible ratio values ​​has been reached. If the end of the loop has not been reached, the next ratio value may be tested (e.g., in step 702). If the end of the loop has been reached, the output of the inference process may be an estimate of the best ratio corresponding to the value best_ratio.

[0165] Information related to the optimal code may be indicated in the video data (e.g., signaled in the bitstream). When LMCS is applied, the lmcs_data set may be signaled (e.g., on the encoder side) or parsed (e.g., on the decoder side), and indications (e.g., flags) may be added to the LMCS data. For example, the indication lmcs_cw_code_selection_flag may indicate whether LMCS codewords are transmitted individually or whether the codewords belong to a predefined code. When lmcs_cw_code_selection_flag is false, the codeword may be transmitted. When lmcs_cw_code_selection_flag is true, lmcs_cw_ratio_code_flag may be tested. When the indication lmcs_cw_ratio_code_flag is true, the ratio lmcs_cw_ratio may be transmitted. When lmcs_cw_ratio_code_flag is false, an index of a predefined code lmcs_cw_nb_codes (for example, from a predefined set of codes) may be sent.

[0166] [Table 2]

[0167] In the example, the process may include the following:

[0168]

number

[0169] In the example, one of lmcs_cw_ratio and lmcs_cw_code_idx may be signaled. In the example, lmcs_cw_ratio may correspond to an index used to fetch ratio values ​​from a predefined table of ratios.

[0170] While features and elements are described above in specific combinations, those skilled in the art will understand that each feature or element may be used alone or in any combination with other features and elements. In addition, the methods described herein may be implemented in computer programs, software, or firmware embedded in computer-readable media for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital multi-purpose disks (DVDs). A processor associated with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. A video encoding device, A processor is provided, and the processor is Determine the multiple candidate ratio values ​​associated with the video picture. Determine the coding gains associated with each of the multiple candidate ratio values ​​mentioned above. From the aforementioned multiple candidate ratio values, a ratio value is selected based on the associated coding gain of each of them. Based on the selected ratio value, adjust the number of codewords in the mapping function associated with the video picture. A video encoding device configured to encode the video picture based on the mapping function.

2. The aforementioned processor, The range of luma sample values ​​of the video picture is partitioned into multiple luma sample value segments corresponding to multiple subranges of each luma value, Based on the selected ratio value, the number of codewords for the plurality of luma sample value categories is determined. Based on the determined number of codewords for the plurality of luma sample value categories, the plurality of luma sample value categories are mapped to the updated luma sample values. The device according to claim 1, further configured to generate the mapping function based on the mapped plurality of luma sample value segments.

3. The aforementioned processor, The device according to claim 1 or 2, further configured to include an indication of the selected ratio value in the video data, wherein the video picture is further encoded based on the indication.

4. The aforementioned processor, The device according to claim 3, further configured to determine whether to signal a plurality of codewords individually or whether the plurality of codewords are associated with a predefined code, wherein the indication of the selected ratio value is included in the video data based on the determination that the plurality of codewords are associated with a predefined code.

5. The aforementioned processor, It is determined that the codeword is associated with a predefined code. The device according to claim 1 or 2, further configured to include an index associated with the codeword in the video data, wherein the video picture is further encoded based on the codeword.

6. The aforementioned processor, The device according to any one of claims 1 to 5, further configured to determine at least one feature from a plurality of mapped video picture samples associated with the video picture using at least one ratio value from the plurality of candidate ratio values, wherein the coding gain is determined based on the determined at least one feature.

7. The device according to claim 6, wherein the at least one feature comprises at least one of the following: the minimum value of the plurality of mapped video picture samples, the maximum value of the plurality of mapped video picture samples, a histogram for each segment of the plurality of mapped video picture samples, the variance for each segment of the plurality of mapped video picture samples, or the temporal variance between the plurality of mapped video picture samples and a sample of a second video picture different from the video picture.

8. The aforementioned processor, The device according to any one of claims 1 to 7, further configured to apply a gradient boosting decision tree algorithm to the plurality of candidate ratio values, wherein the coding gain is determined based on the application of the gradient boosting decision tree algorithm to the plurality of candidate ratio values.

9. A video decoding device, A processor is provided, and the processor is Receive video picture, The system receives an indication of the ratio value corresponding to the aforementioned video picture. Based on the ratio value, adjust the number of codewords in the mapping function associated with the video picture. A video decoding device configured to decode the video picture based on the mapping function.

10. The aforementioned processor, The range of luma sample values ​​of the video picture is partitioned into multiple luma sample value segments corresponding to multiple subranges of each luma value, Based on the ratio value, the number of codewords for the plurality of luma sample value categories is determined. Based on the determined number of codewords for the plurality of luma sample value categories, the plurality of luma sample value categories are mapped to the updated luma sample values. The device according to claim 9, further configured to generate the mapping function based on the mapped luma sample value segments.

11. The aforementioned processor, The mapping function is applied to multiple interpreted luma samples, The device according to claim 9 or 10, further configured to decode the video picture based on applying the mapping function to the plurality of interpredicted luma samples.

12. The aforementioned processor, The device according to any one of claims 8 to 11, further configured to determine that a plurality of codewords belong to a predefined codeword, wherein the indication of the ratio value is received based on the determination that the plurality of codewords belong to a predefined codeword.

13. The device according to any one of claims 1 to 12, further comprising a memory operably connected to the processor.

14. A method for video encoding, Determining multiple candidate ratio values ​​associated with a video picture, Determining each of the multiple coding gains associated with the aforementioned multiple candidate ratio values, From the aforementioned multiple candidate ratio values, a ratio value is selected based on the associated coding gain of each of them. Based on the selected ratio value, adjust the number of codewords in the mapping function associated with the video picture, A method comprising encoding the video picture based on the mapping function.

15. The method described above is The range of luma sample values ​​of the video picture is partitioned into multiple luma sample value segments corresponding to multiple subranges of each luma value, Based on the selected ratio value, the number of codewords for the plurality of luma sample value categories is determined, Based on the determined number of codewords for the plurality of luma sample value categories, the plurality of luma sample value categories are mapped to updated luma sample values. The method according to claim 14, further comprising generating the mapping function based on the mapped plurality of luma sample value segments.

16. The method described above is The device according to claim 14 or 15, further comprising including an indication of the selected ratio value in the video data, wherein the video picture is further encoded based on the indication.

17. The method described above is The method according to claim 16, further comprising determining whether to signal a plurality of codewords individually or whether the plurality of codewords are associated with a predefined code, wherein the indication of the selected ratio value is included in the video data based on the determination that the plurality of codewords are associated with a predefined code.

18. The method described above is Determining that a codeword is associated with a predefined code, The method according to claim 14 or 15, further comprising including an index associated with the codeword in the video data, wherein the video picture is further encoded based on the codeword.

19. The method described above is The method according to any one of claims 14 to 18, further comprising determining at least one feature from a plurality of mapped video picture samples associated with the video picture using at least one ratio value from the plurality of candidate ratio values, wherein the coding gain is determined based on the determined at least one feature.

20. The method according to claim 19, wherein the at least one feature includes at least one of the minimum value of the plurality of mapped video picture samples, the maximum value of the plurality of mapped video picture samples, a histogram for each segment of the plurality of mapped video picture samples, the variance for each segment of the plurality of mapped video picture samples, or the temporal variance between the plurality of mapped video picture samples and a sample of a second video picture different from the video picture.

21. The method described above is The method according to any one of claims 14 to 20, further comprising applying a gradient boosting decision tree algorithm to the plurality of candidate ratio values, wherein the coding gain is determined based on applying the gradient boosting decision tree algorithm to the plurality of candidate ratio values.

22. A method for video decoding, wherein the method is Receiving video pictures and, Receiving an indication of the ratio value corresponding to the aforementioned video picture, Based on the ratio value, adjust the number of codewords in the mapping function associated with the video picture, A method comprising decoding the video picture based on the mapping function.

23. The method described above is The range of luma sample values ​​of the video picture is partitioned into multiple luma sample value segments corresponding to multiple subranges of each luma value, Based on the ratio value, the number of codewords for the plurality of luma sample value categories is determined, Based on the determined number of codewords for the plurality of luma sample value categories, the plurality of luma sample value categories are mapped to updated luma sample values. The method according to claim 22, further comprising generating the mapping function based on the mapped plurality of luma sample value segments.

24. The method described above is Applying the aforementioned mapping function to multiple interpreted luma samples, The method according to claim 22 or 23, further comprising decoding the video picture based on applying the mapping function to the plurality of interpredicted luma samples.

25. The method described above is The method according to any one of claims 22 to 24, further comprising determining that a plurality of codewords belong to a predefined codeword, wherein the indication of the ratio value is received based on the determination that the plurality of codewords belong to a predefined codeword.

26. A computer-readable medium comprising a medium containing instructions for causing one or more processors to perform the method according to any one of claims 14 to 25.

27. A computer program product stored in a non-temporary computer-readable medium, which, when executed by a processor, includes program code instructions for performing steps of the method described in at least one of claims 14 to 25.

28. Video data comprising information representing a video picture encoded according to one of the methods described in any one of claims 14 to 21.

Images