Alternative prediction method for lifting wavelet transform in subdividing mesh surfaces

Abstract

Some embodiments of a method can include obtaining an input signal corresponding to a grid; splitting the input signal into a plurality of odd-indexed samples and a plurality of even-indexed samples, wherein the plurality of odd-indexed samples of the input signal correspond to vertices at a lower resolution level in the grid, and wherein the plurality of even-indexed samples of the input signal correspond to vertices at a higher resolution level in the grid; determining at least one sample of a prediction signal using at least three of the plurality of odd-indexed samples; and subtracting the prediction signal from the plurality of even-indexed samples to generate a wavelet coefficient signal.

Description

[0001] Cross-references to related applications This application claims the benefit of European Patent Application No. EP24305075, filed on January 11, 2024, entitled “ALTERNATIVE PREDICTION METHODS FORTHE LIFTING WAVELET TRANSFORM IN SUBDIVISION MESH SURFACES”, which is incorporated herein by reference in its entirety. Background Technology

[0002] ISO / IEC 23090-5:2021, “Information technology — Coded representation of immersive media — Part 5: Visual volumetric video-based coding (V3C) and video-based point cloud compression (V-PCC),” describes visual volumetric video-based coding (V3C). Video-based dynamic mesh coding (V-DMC) is a framework being developed to compress dynamic meshes by extending V3C. The V-DMC framework can involve different properties specified for a sequence of dynamic meshes. In V-DMC, the underlying static mesh codec can decode the mesh properties for each face or each vertex. These properties can provide additional information about the static mesh, such as color, texture coordinates, normals, reflection information, transparency information, and / or user-defined properties.

[0003] For V-DMC, a framework was developed by extending V3C. V3C is described in ISO / IEC 23090-5:2021 “Information technology — Coded representation of immersive media — Part 5: Visualvolumetric video-based coding (V3C) and video-based point cloud compression (V-PCC)”. Summary of the Invention

[0004] A first example method according to some embodiments may include: obtaining an input signal corresponding to a grid; segmenting the input signal into a plurality of odd-indexed samples and a plurality of even-indexed samples, wherein the plurality of odd-indexed samples of the input signal correspond to vertices at a lower resolution level in the grid, and wherein the plurality of even-indexed samples of the input signal correspond to vertices at a higher resolution level in the grid; using at least three of the plurality of odd-indexed samples to determine at least one sample of a prediction signal; and subtracting the prediction signal from the plurality of even-indexed samples to generate a wavelet coefficient signal.

[0005] Some embodiments of the first example method may further include subtracting wavelet coefficient signals from a plurality of odd-indexed samples.

[0006] For some embodiments of the first example method, determining at least one sample of the predicted signal may include determining at least one weighting value based on four samples from a plurality of odd-indexed samples, wherein the four samples are the nearest neighbors of at least one sample of the predicted signal.

[0007] In some embodiments of the first example method, the nearest neighbor is the vertex of a triangle that is adjacent to at least one sample of the predicted signal.

[0008] For some embodiments of the first example method, determining at least one weighting value based on four samples from a plurality of odd-indexed samples includes determining unequal weights for the four samples.

[0009] In some embodiments of the first example method, determining at least one weighting value is based on the relative distances of the four samples to at least one sample of the predicted signal.

[0010] For some embodiments of the first example method, determining at least one sample of the predicted signal includes determining at least one weighting value based on three samples from a plurality of odd-indexed samples, wherein the three samples are the nearest neighbors of at least one sample of the predicted signal.

[0011] In some embodiments of the first example method, the nearest neighbor is the vertex of a triangle that is adjacent to at least one sample of the predicted signal.

[0012] For some embodiments of the first example method, determining at least one weighted value based on three samples from a plurality of odd-indexed samples includes determining unequal weights for the three samples.

[0013] In some embodiments of the first example method, determining at least one weighting value is based on the relative distance between the three samples and at least one sample of the predicted signal.

[0014] In some embodiments of the first example method, at least one sample of the predicted signal may include the boundary points of the grid.

[0015] A first example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0016] A second example method according to some embodiments may include: obtaining a coarse input signal of a wavelet transform corresponding to a grid, wherein the coarse input signal of the wavelet transform includes a first plurality of samples; obtaining a wavelet coefficient input signal corresponding to the grid; using at least three of the first plurality of samples of the grid signal to determine at least one sample of a prediction signal; adding the prediction signal to the wavelet coefficient input signal to generate a second plurality of samples; and merging the first plurality of samples with the second plurality of samples to generate an output signal corresponding to the grid.

[0017] In some embodiments of the second example method, the first plurality of samples correspond to even-indexed samples of the output signal, and the second plurality of samples correspond to odd-indexed samples of the output signal.

[0018] Some embodiments of the second example method may further include subtracting the wavelet coefficient input signal from the coarse input signal of the wavelet transform.

[0019] In some embodiments of the second example method, determining at least one sample of the predicted signal may include determining at least one weighting value based on four samples from a first plurality of samples, wherein the four samples are the nearest neighbors of at least one sample of the predicted signal.

[0020] In some embodiments of the second example method, the nearest neighbor is the vertex of a triangle that is adjacent to at least one sample of the predicted signal.

[0021] For some embodiments of the second example method, determining at least one weighting value based on four samples of the first plurality of samples includes determining unequal weights for the four samples.

[0022] In some embodiments of the second example method, determining at least one weighting value is based on the relative distance between the four samples and at least one sample of the predicted signal.

[0023] For some embodiments of the second example method, determining at least one sample of the predicted signal includes determining at least one weighting value based on three samples from a first plurality of samples, wherein the three samples are the nearest neighbors of at least one sample of the predicted signal.

[0024] In some embodiments of the second example method, the nearest neighbor is the vertex of a triangle that is adjacent to at least one sample of the predicted signal.

[0025] For some embodiments of the second example method, determining at least one weighted value based on three samples of the first plurality of samples includes determining unequal weights for the three samples.

[0026] In some embodiments of the second example method, determining at least one weighting value is based on the relative distance between the three samples and at least one sample of the predicted signal.

[0027] In some embodiments of the second example method, at least one sample of the predicted signal includes the boundary points of the grid.

[0028] A second example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0029] A third example method according to some embodiments may include: obtaining an input signal corresponding to a grid; dividing the input signal into a first plurality of samples and a second plurality of samples, using at least three of the first plurality of samples to determine at least one sample of a prediction signal; and subtracting the prediction signal from the second plurality of samples to generate a wavelet coefficient signal.

[0030] In some embodiments of the second example method, a first plurality of samples of the input signal correspond to vertices at a lower resolution level in the grid, and a second plurality of samples of the input signal correspond to vertices at a higher resolution level in the grid.

[0031] In some embodiments of the second example method, a first plurality of samples of the input signal correspond to odd-indexed samples of the input signal, and a second plurality of samples of the input signal correspond to even-indexed samples of the input signal.

[0032] In some embodiments of the second example method, a first plurality of samples of the input signal correspond to vertices at a higher resolution level in the grid, and a second plurality of samples of the input signal correspond to vertices at a lower resolution level in the grid.

[0033] In some embodiments of the second example method, a first plurality of samples of the input signal correspond to even-indexed samples of the input signal, and a second plurality of samples of the input signal correspond to odd-indexed samples of the input signal.

[0034] A third example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0035] According to some embodiments, an example method for performing wavelet transform on an input signal corresponding to a grid may include: obtaining the input signal; segmenting the input signal into lower-resolution samples and higher-resolution samples; generating a prediction signal, wherein the prediction signal includes prediction samples corresponding to the higher-resolution samples, and wherein each prediction sample is generated based on at least three of the lower-resolution samples; and subtracting the prediction signal from the higher-resolution samples to generate a wavelet coefficient signal.

[0036] A fourth example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0037] According to some embodiments, an example method for performing wavelet transform on an input signal corresponding to a grid may include: segmenting the input signal into lower-resolution samples and higher-resolution samples; generating a prediction signal based on at least three lower-resolution samples for each sample of the prediction signal; and subtracting the prediction signal from the higher-resolution samples to generate a wavelet coefficient signal.

[0038] A fifth example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0039] A sixth example apparatus according to some embodiments may include at least one processor configured to perform any of the methods listed above.

[0040] A seventh example apparatus according to some embodiments may include a computer-readable medium storing instructions for causing one or more processors to perform any of the methods listed above.

[0041] An eighth example apparatus according to some embodiments may include at least one processor and at least one non-transitory computer-readable medium storing instructions for causing at least one processor to perform any of the methods listed above.

[0042] Example signals according to some embodiments may include signals that transmit wavelet coefficient signals generated according to the methods listed above. Attached Figure Description

[0043] Figure 1A This is a system diagram illustrating an example communication system according to some embodiments.

[0044] Figure 1B The illustration shows a method according to some embodiments. Figure 1AThe diagram shows a system diagram of an example wireless transmit / receive unit (WTRU) used in a communication system.

[0045] Figure 1C This is a system diagram illustrating a set of example interfaces for a system according to some embodiments.

[0046] Figure 2A This is a functional block diagram of a block-based video encoder according to some embodiments, such as an encoder for general video codec (VVC).

[0047] Figure 2B This is a functional block diagram of a block-based video decoder according to some embodiments, such as a decoder for VVC.

[0048] Figure 3A This is a schematic side view of an example waveguide display that can be used in extended reality (XR) applications according to some embodiments.

[0049] Figure 3B This is a schematic side view illustrating an example alternative display type that can be used for extended reality applications according to some embodiments.

[0050] Figure 3C This is a schematic side view illustrating an example alternative display type that can be used for extended reality applications according to some embodiments.

[0051] Figure 4A This is a schematic diagram illustrating an example basic mesh according to some embodiments.

[0052] Figure 4B This is a schematic diagram illustrating an example mesh after one subdivision iteration according to some embodiments.

[0053] Figure 4C This is a schematic diagram illustrating an example mesh after two subdivision iterations according to some embodiments.

[0054] Figure 4D This is a schematic diagram illustrating an example mesh after three subdivision iterations according to some embodiments.

[0055] Figure 4E This is a schematic diagram illustrating an example mesh after four subdivision iterations according to some embodiments.

[0056] Figure 4F This is a schematic diagram illustrating an example mesh after 5 subdivision iterations according to some embodiments.

[0057] Figure 5 This is a schematic diagram illustrating example wavelet coefficient calculation in linear polyhedral subdivision according to some embodiments.

[0058] Figure 6A This is a schematic diagram illustrating example rule connectivity of a triangular mesh according to some embodiments.

[0059] Figure 6B This is a schematic diagram illustrating an example of irregular connectivity of a triangular mesh according to some embodiments.

[0060] Figure 6C This is a schematic diagram illustrating an example of semi-regular connectivity of a triangular mesh according to some embodiments.

[0061] Figure 7A This is a diagram illustrating an example of forward lifting wavelet transform according to some embodiments.

[0062] Figure 7B This is a diagram illustrating an example of forward lifting wavelet transform according to some embodiments.

[0063] Figure 7C This is a diagram illustrating an example of inverse lifting wavelet transform according to some embodiments.

[0064] Figure 7D This is a diagram illustrating an example of inverse lifting wavelet transform according to some embodiments.

[0065] Figure 8A This is a schematic diagram illustrating an example manifold mesh according to some embodiments.

[0066] Figure 8B This is a schematic diagram illustrating an example non-manifold mesh according to some embodiments.

[0067] Figure 8C This is a schematic diagram illustrating an example non-manifold mesh according to some embodiments.

[0068] Figure 9A This is a schematic diagram illustrating an example closed mesh without boundaries according to some embodiments.

[0069] Figure 9B This is a schematic diagram illustrating an example open mesh with boundaries according to some embodiments.

[0070] Figure 10A This is a schematic diagram illustrating an example mesh containing new vertices not on the mesh boundaries, according to some embodiments.

[0071] Figure 10B This is a schematic diagram illustrating an example mesh containing new vertices on the mesh boundary according to some embodiments.

[0072] Figure 11 This is a flowchart illustrating the process of an example V-DMC intra-frame encoder according to some embodiments.

[0073] Figure 12 This is a flowchart illustrating an example V-DMC intra-frame decoder according to some embodiments.

[0074] Figure 13 This is a schematic diagram illustrating adjacent triangles of different sizes for new mesh vertices according to some embodiments.

[0075] Figure 14 This is a schematic diagram illustrating adjacent triangles of different sizes for new mesh vertices according to some embodiments.

[0076] Figure 15 This is a schematic diagram illustrating an example adjacent triangle of a new vertex on a mesh boundary according to some embodiments.

[0077] Figure 16 This is a flowchart illustrating an example process for performing a wavelet transform according to some embodiments.

[0078] Figure 17 This is a flowchart illustrating an example process for performing an inverse wavelet transform according to some embodiments.

[0079] The entities, connections, arrangements, etc., depicted in the figures and described in conjunction with the figures are presented in an illustrative rather than restrictive manner. Similarly, any and all statements or other indications regarding what a particular figure “depicts,” what a particular element or entity in a particular figure “is” or “has”, and any and all similar statements—which, in isolation and out of context, might be read as absolute and therefore restrictive—may only be correctly read as if constructively preceded by a clause such as “In at least one embodiment, …”. For the sake of brevity and clarity, this implicit introductory clause will not be repeated in the detailed description. Detailed Implementation

[0080] Figure 1A This is a schematic diagram illustrating an example communication system 100 in which one or more of the disclosed embodiments may be implemented. The communication system 100 may be a multiple access system that provides content such as voice, data, video, message delivery, and broadcasting to multiple wireless users. The communication system 100 may enable multiple wireless users to access such content by sharing system resources, including wireless bandwidth. For example, the communication system 100 may employ one or more channel access methods, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal FDMA (OFDMA), Single Carrier FDMA (SC-FDMA), Zero-Tail Unique Word DFT Spread Spectrum OFDM (ZT UW DTS-s OFDM), Unique Word OFDM (UW-OFDM), Resource Block Filtered OFDM, Filter Bank Multicarrier (FBMC), and so on.

[0081] like Figure 1A As shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106, public switched telephone network (PSTN) 108, Internet 110, and other networks 112. However, it will be appreciated that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, 102d can be any type of device configured to operate and / or communicate in a wireless environment. By way of example, any of WTRU 102a, 102b, 102c, and 102d may be referred to as a “station” and / or “STA”, may be configured to transmit and / or receive wireless signals, and may include user equipment (UE), mobile station, fixed or mobile subscriber unit, subscription-based unit, pager, cellular phone, personal digital assistant (PDA), smartphone, laptop, netbook, personal computer, wireless sensor, hotspot or Mi-Fi device, Internet of Things (IoT) device, watch or other wearable device, head-mounted display (HMD), vehicle, drone, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated processing chain scenarios), consumer electronics devices, devices operating on commercial and / or industrial wireless networks, and so on. Any of WTRU 102a, 102b, 102c, and 102d may be interchangeably referred to as a UE.

[0082] The communication system 100 may also include base station 114a and / or base station 114b. Each of base stations 114a and 114b may be any type of device configured to wirelessly interface with at least one of WTRUs 102a, 102b, 102c, and 102d to facilitate access to one or more communication networks such as CN 106, the Internet 110, and / or other networks 112. By way of example, base stations 114a and 114b may be a basic transceiver station (BTS), Node-B, eNode B, home Node B, home eNode B, gNB, NR NodeB, site controller, access point (AP), wireless router, etc. Although base stations 114a and 114b are each depicted as a single element, it will be appreciated that base stations 114a and 114b may include any number of interconnected base stations and / or network elements.

[0083] Base station 114a may be part of RAN 104 / 113, which may also include other base stations and / or network elements (not shown), such as base station controllers (BSCs), radio network controllers (RNCs), relay nodes, etc. Base station 114a and / or base station 114b may be configured to transmit and / or receive radio signals on one or more carrier frequencies, which may be referred to as cells (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage of a specific geographic area, which may be relatively fixed or may change over time. A cell may be further divided into cell sectors. For example, the cell associated with base station 114a may be divided into three sectors. Therefore, in one embodiment, base station 114a may include three transceivers, i.e., one transceiver for each sector of the cell. In embodiments, base station 114a may employ multiple-input multiple-output (MIMO) technology and may utilize multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in a desired spatial direction.

[0084] Base stations 114a and 114b can communicate with one or more of WTRUs 102a, 102b, 102c, and 102d via air interface 116. The air interface can be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). Air interface 116 can be established using any suitable radio access technology (RAT).

[0085] More specifically, as noted above, the communication system 100 can be a multiple access system and can employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, base station 114a in RAN 104 / 113 and WTRUs 102a, 102b, and 102c can implement radio technologies such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which can establish an air interface 116 using Wideband CDMA (WCDMA). WCDMA may include communication protocols such as High-Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High-Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High-Speed ​​UL Packet Access (HSUPA).

[0086] In the embodiment, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which can establish air interface 116 using Long Term Evolution (LTE) and / or Advanced LTE (LTE-A) and / or Advanced LTE Pro (LTE-A Pro).

[0087] In the embodiment, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as NR radio access, which can establish air interface 116 using new radio (NR).

[0088] In the embodiments, base station 114a and WTRUs 102a, 102b, and 102c can implement various radio access technologies. For example, base station 114a and WTRUs 102a, 102b, and 102c can, for example, use the dual connectivity (DC) principle to implement both LTE and NR radio access. Therefore, the air interface utilized by WTRUs 102a, 102b, and 102c can be characterized by various types of radio access technologies and / or transmissions sent to / from various types of base stations (e.g., eNBs and gNBs).

[0089] In other embodiments, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as IEEE 802.11 (i.e., Wi-Fi), IEEE 802.16 (i.e., Global Microwave Access Interoperability (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Provisional Standard 2000 (IS-2000), Provisional Standard 95 (IS-95), Provisional Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced GSM Evolution Data Rate (EDGE), GSM EDGE (GERAN), and so on.

[0090] Figure 1ABase station 114b can be, for example, a wireless router, a home Node B, a home eNode B, or an access point, and can utilize any suitable RAT to facilitate wireless connectivity in a local area, such as a business premises, residence, vehicle, campus, industrial facility, air corridor (e.g., for drone use), road, etc. In one embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.11 to establish a wireless local area network (WLAN). In another embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, base station 114b and WTRUs 102c, 102d can utilize cellular-based RATs (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish picocells or femtocells. Figure 1A As shown, base station 114b can have a direct connection to Internet 110. Therefore, it is not required that base station 114b access Internet 110 via CN 106.

[0091] RAN 104 / 113 can communicate with CN 106, which can be any type of network configured to provide voice, data, application, and / or Voice over Internet Protocol (VoIP) services to one or more of WTRUs 102a, 102b, 102c, and 102d. Data can have varying Quality of Service (QoS) requirements, such as different throughput requirements, latency requirements, fault tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. CN 106 can provide call control, billing services, location-based services, prepaid calling, internet connectivity, video distribution, etc., and / or perform advanced security functions such as user authentication. Although not explicitly stated... Figure 1A As shown, but will be understood, RAN 104 / 113 and / or CN 106 can communicate directly or indirectly with other RANs that use the same RAT as RAN 104 / 113 or a different RAT. For example, in addition to connecting to RAN 104 / 113, which may be using NR radio technology, CN 106 can also communicate with another RAN (not shown) using GSM, UMTS, CDMA2000, WiMAX, E-UTRA, or WiFi radio technology.

[0092] CN 106 can also serve as a gateway for WTRUs 102a, 102b, 102c, and 102d to access PSTN 108, the Internet 110, and / or other networks 112. PSTN 108 may include a circuit-switched telephone network providing Common Old-Style Telephone Service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices using common communication protocols such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and / or Internet Protocol (IP) from the TCP / IP Internet Protocol Suite. Network 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, network 112 may include another CN connected to one or more RANs, which may use the same RAT as RAN104 / 113 or a different RAT.

[0093] Some or all of the WTRUs 102a, 102b, 102c, and 102d in communication system 100 may include multi-mode capabilities (e.g., WTRUs 102a, 102b, 102c, and 102d may include multiple transceivers for communicating with different wireless networks via different wireless links). For example, Figure 1A The WTRU 102c shown can be configured to communicate with base station 114a, which may employ cellular-based radio technology, and with base station 114b, which may employ IEEE 802 radio technology.

[0094] Figure 1B The following diagram illustrates the system of example WTRU 102. Figure 1B As shown, among other things, WTRU 102 may include a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power supply 134, a Global Positioning System (GPS) chipset 136, and / or other peripheral devices 138. It will be appreciated that WTRU 102 may include any sub-combination of the foregoing elements while remaining consistent with the embodiments.

[0095] Processor 118 can be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. Processor 118 can perform signal encoding / decoding, data processing, power control, input / output processing, and / or any other functions that enable WTRU 102 to operate in a wireless environment. Processor 118 can be coupled to transceiver 120, and transceiver 120 can be coupled to transmitting / receiving element 122. Although... Figure 1B The processor 118 and transceiver 120 are depicted as separate components, but it will be understood that the processor 118 and transceiver 120 can be integrated together into an electronic package or chip.

[0096] Transmitting / receiving element 122 can be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) via air interface 116. For example, in one embodiment, transmitting / receiving element 122 can be an antenna configured to transmit and / or receive RF signals. In another embodiment, transmitting / receiving element 122 can be a transmitter / detector configured to transmit and / or receive, for example, IR, UV, or visible light signals. In yet another embodiment, transmitting / receiving element 122 can be configured to transmit and / or receive both RF signals and optical signals. It will be appreciated that transmitting / receiving element 122 can be configured to transmit and / or receive any combination of wireless signals.

[0097] Although the transmitting / receiving element 122 is in Figure 1B While depicted as a single element, WTRU 102 may include any number of transmit / receive elements 122. More specifically, WTRU 102 may employ MIMO technology. Thus, in one embodiment, WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals via air interface 116.

[0098] Transceiver 120 can be configured to modulate signals to be transmitted by transmitting / receiving element 122 and demodulate signals received by transmitting / receiving element 122. As noted above, WTRU 102 can have multi-mode capability. Thus, for example, transceiver 120 can include multiple transceivers for enabling WTRU 102 to communicate via various RATs such as NR and IEEE 802.11.

[0099] The processor 118 of WTRU 102 can be coupled to a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) unit or an organic light-emitting diode (OLED) display unit) and can receive user input data from these devices. The processor 118 can also output user data to the speaker / microphone 124, keypad 126, and / or display / touchpad 128. Furthermore, the processor 118 can access information from any type of suitable memory and store the data in memory such as non-removable memory 130 and / or removable memory 132. Non-removable memory 130 may include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. Removable memory 132 may include a subscriber identity module (SIM) card, memory stick, secure digital storage (SD) card, etc. In other embodiments, the processor 118 can access information from memory that is not physically located on WTRU 102 (such as on a server or home computer (not shown)) and store the data in that memory.

[0100] The processor 118 can receive power from the power supply 134 and can be configured to distribute and / or control power to other components in the WTRU 102. The power supply 134 can be any suitable device for powering the WTRU 102. For example, the power supply 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cell units, fuel cell units, etc.

[0101] The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) about the current location of the WTRU 102. In addition to, or alternatively to, the information from the GPS chipset 136, the WTRU 102 may receive location information from base stations (e.g., base stations 114a, 114b) via the air interface 116, and / or determine its location based on the timing of signals received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information using any suitable location determination method, while remaining consistent with the embodiments.

[0102] The processor 118 may be further coupled to other peripheral devices 138, which may include one or more software and / or hardware modules providing additional features, functions, and / or wired or wireless connectivity. For example, peripheral devices 138 may include accelerometers, electronic compasses, satellite transceivers, digital cameras (for photos and / or video), Universal Serial Bus (USB) ports, vibration devices, television transceivers, hands-free headsets, Bluetooth® modules, FM radio units, digital music players, media players, video game player modules, internet browsers, virtual reality and / or augmented reality (VR / AR) devices, activity trackers, etc. Peripheral devices 138 may include one or more sensors, which may be one or more of the following: gyroscopes, accelerometers, Hall effect sensors, magnetometers, orientation sensors, proximity sensors, temperature sensors, time sensors; geolocation sensors; altimeters, light sensors, touch sensors, magnetometers, barometers, gesture sensors, biometric sensors, and / or humidity sensors.

[0103] WTRU 102 may include a full-duplex radio, for which the transmission and reception of some or all signals (e.g., associated with a specific subframe of both UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous. The full-duplex radio may include an interference management unit to reduce and / or substantially eliminate self-interference via hardware (e.g., a choke) or via signal processing (e.g., a separate processor (not shown) or via processor 118). In embodiments, WTRU 102 may include a half-duplex radio, for which the transmission and reception of some or all signals (e.g., associated with a specific subframe of either UL (e.g., for transmission) or downlink (e.g., for reception) may be concurrent and / or simultaneous.

[0104] Despite WTRU in Figures 1A-1B While described as a wireless terminal, it is conceivable that in some representative embodiments, such a terminal may use (e.g., temporarily or permanently) a wired communication interface with a communication network.

[0105] In a representative embodiment, the other network 112 may be a WLAN.

[0106] Given Figures 1A-1B as well as Figures 1A-1B As described herein, one or more of the functions described herein may be performed by one or more emulation devices (not shown). An emulation device may be one or more devices configured to emulate one or more of the functions described herein. For example, an emulation device may be used to test other devices and / or simulate network and / or WTRU functions.

[0107] Simulation devices can be designed to perform tests on one or more other devices in laboratory and / or carrier network environments. For example, one or more simulation devices can perform one or more or all functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices within the communication network. One or more simulation devices can perform one or more or all functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. For testing purposes, simulation devices can be directly coupled to another device and / or can use over-the-air wireless communication to perform tests.

[0108] One or more emulation devices can perform one or more functions (including all functions) without being implemented / deployed as part of a wired and / or wireless communication network. For example, emulation devices can be used in test scenarios within test laboratories and / or non-deployed (e.g., test) wired and / or wireless communication networks to perform testing of one or more components. One or more emulation devices can be test rigs. Emulation devices can transmit and / or receive data using direct RF coupling and / or wireless communication via RF circuitry (e.g., which may include one or more antennas).

[0109] Figure 1C This is a system diagram illustrating an example set of interfaces of a system according to some embodiments. In some embodiments, an extended reality display device and its control electronics can be implemented. System 150 can be embodied as a device including the various components described below and configured to perform one or more aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 150 can be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 150 are distributed across multiple ICs and / or discrete components. In various embodiments, system 150 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 150 is configured to implement one or more aspects described in this document.

[0110] System 150 includes at least one processor 152 configured to execute instructions loaded therein for implementing various aspects, such as those described in this document. Processor 152 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 150 includes at least one memory 154 (e.g., a volatile memory device and / or a non-volatile memory device). System 150 may include a storage device 158, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 158 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0111] System 150 includes an encoder / decoder module 156 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 156 may include its own processor and memory. The encoder / decoder module 156 represents one or more modules that can be included in a device to perform encoding and / or decoding functions. It is well known that a device may include one or both encoding and decoding modules. Furthermore, the encoder / decoder module 156 may be implemented as a separate element of system 150, or it may be incorporated within processor 152 as a combination of hardware and software known to those skilled in the art.

[0112] Program code to be loaded onto processor 152 or encoder / decoder 156 to execute the various aspects described in this document may be stored in storage device 158 and subsequently loaded onto memory 154 for execution by processor 152. According to various embodiments, one or more of processor 152, memory 154, storage device 158, and encoder / decoder module 156 may store one or more various items during the execution of the processes described in this document. Such stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0113] In some embodiments, the memory within processor 152 and / or encoder / decoder module 156 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, external memory (e.g., processor 152 or encoder / decoder module 152) is used for one or more of these functions. External memory may be memory 154 and / or storage device 158, such as volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one embodiment, a fast external dynamic volatile memory, such as RAM, is used as working memory for video encoding and decoding operations, such as for MPEG-2 (MPEG stands for Moving Picture Experts Group; MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Codec; also known as H.265 and MPEG-H Part 2), or VVC (Various Video Codec, a new standard developed by the Joint Video Experts Group JVET).

[0114] As indicated in box 172, inputs can be provided to the components of system 150 via various input devices. Such input devices include, but are not limited to, (i) a radio frequency (RF) section that receives, for example, RF signals transmitted over the air by a broadcaster, (ii) component (COMP) input terminals (or a set of COMP input terminals), (iii) universal serial bus (USB) input terminals, and / or (iv) high-definition multimedia interface (HDMI) input terminals. Figure 1C Other examples not shown include composite video.

[0115] In various embodiments, the input device of block 172 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also referred to as selecting a signal, or band-limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) band-limiting it again to a narrower frequency band to select, for example, a signal band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section in various embodiments includes one or more elements for performing these functions, such as a frequency selector, signal selector, band limiter, channel selector, filter, downconverter, demodulator, error corrector, and demultiplexer. The RF section may include tuners that perform various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band. Various embodiments rearrange the order of the above (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0116] Additionally, the USB and / or HDMI terminals may include their respective interface processors for connecting system 150 to other electronic devices across the USB and / or HDMI connections. It will be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented as needed, for example, in a separate input processing IC or within processor 152. Similarly, as needed, various aspects of USB or HDMI interface processing may be implemented within a separate interface IC or within processor 152. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, processor 152 and encoder / decoder 156, which operate in conjunction with memory and storage elements to process the data streams as needed for presentation on the output device.

[0117] Various components of system 150 can be provided within an integrated housing, in which the various components can be interconnected and transmit data therebetween using a suitable connection arrangement 174, such as an internal bus known in the art, including inter-IC (I2C) bus, wiring, and printed circuit board.

[0118] System 150 includes a communication interface 160, which enables communication with other devices via a communication channel 162. The communication interface 160 may include, but is not limited to, a transceiver configured to transmit and receive data via the communication channel 162. The communication interface 160 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 162 may be implemented, for example, in a wired and / or wireless medium.

[0119] In various embodiments, data is streamed or otherwise provided to system 150 using a Wi-Fi network such as IEEE 802.11 (IEEE stands for Institute of Electrical and Electronics Engineers). The Wi-Fi signals in these embodiments are received via a communication channel 162 and a communication interface 160 adapted for Wi-Fi communication. The communication channel 162 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other embodiments use a set-top box to provide streaming data to system 150, transmitting data via an HDMI connection to input box 172. Still other embodiments use an RF connection to input box 172 to provide streaming data to system 150. As indicated above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0120] System 150 can provide output signals to various output devices, including display 176, speaker 178, and other peripheral devices 180. Display 176 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a flexible display, and / or a foldable display. Display 176 can be used in televisions, tablets, laptops, cellular phones (mobile phones), or other devices. Display 176 can also be integrated with other components (e.g., in a smartphone) or standalone (e.g., an external monitor for a laptop). In various embodiments, other peripheral devices 180 include one or more of a standalone digital video disc (or digital universal disc) (DVR, for both terms), disc player, stereo system, and / or lighting system. Various embodiments use one or more peripheral devices 180 that provide functionality based on the output of system 150. For example, a disc player performs the function of playing the output of system 150.

[0121] In various embodiments, signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols capable of enabling device-to-device control with or without user intervention transmits control signals between system 150 and display 176, speaker 178, or other peripheral devices 180. Output devices can be communicatively coupled to system 150 via dedicated connections through their respective interfaces 164, 166, and 168. Alternatively, output devices can be connected to system 150 via communication interface 160 using communication channel 162. Display 176 and speaker 178 can be integrated into a single unit with other components of system 150 in an electronic device, such as a television set. In various embodiments, display interface 164 includes a display driver, such as, for example, a timing controller (TCon) chip.

[0122] For example, if the RF portion of input 172 is part of a separate set-top box, then display 176 and speaker 178 can alternatively be separated from one or more other components. In various embodiments where display 176 and speaker 178 are external components, the output signal can be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0123] System 150 may include one or more sensor devices 168. Examples of sensor devices that may be used include one or more GPS sensors, gyroscope sensors, accelerometers, light sensors, cameras, depth cameras, microphones, and / or magnetometers. Such sensors can be used to determine information such as user position and orientation. Where system 150 is used as a control module (such as control modules 124, 132) for an extended reality display, the user's position and orientation can be used to determine how image data is rendered so that the user perceives the correct portion of a virtual object or scene from the correct viewpoint. In the case of a head-mounted display device, the device's own position and orientation can be used to determine the user's position and orientation for the purpose of rendering virtual content. In the case of other display devices such as telephones, tablets, computer monitors, or televisions, other inputs can be used to determine the user's position and orientation for the purpose of rendering content. For example, the user can use a touchscreen, keypad or keyboard, trackball, joystick, or other inputs to select and / or adjust the desired viewpoint and / or direction of observation. Where the display device has sensors such as accelerometers and / or gyroscopes, the viewpoint and orientation can be selected and / or adjusted based on the movement of the display device for the purpose of rendering content.

[0124] These embodiments can be implemented by computer software implemented by processor 152, or by hardware, or by a combination of hardware and software. As a non-limiting example, embodiments can be implemented by one or more integrated circuits. Memory 154 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. Processor 152 can be of any type suitable for the technical environment and, as a non-limiting example, can encompass one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0125] Block-based video encoding and decoding Like HEVC, VVC is built on a block-based hybrid video codec framework. Figure 2A A block diagram of a block-based hybrid video codec system 200 is given. Variations of the encoder 200 are envisioned, but for clarity, the encoder 200 is described below without all anticipated variations.

[0126] Before being encoded, the video sequence may undergo pre-coding processing 204, such as applying color transformations to the input color image (e.g., a conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing remapping of the input image components to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization with one of the color components). Metadata may be associated with the pre-processing and attached to the bitstream.

[0127] The input video signal 202, including the image to be encoded, is partitioned 206 and processed block by block, for example, in units of CUs. Different CUs may have different sizes. In VTM-1.0, the maximum size of a CU is 128x128 pixels. However, unlike HEVC, which partitions blocks solely based on quadtrees, in VTM-1.0, codec tree units (CTUs) are split into CUs to accommodate the varying local characteristics based on quadtrees / binaries / tritrees. Additionally, the concept of multiple partition unit types in HEVC is removed, meaning the separation of CUs, prediction units (PUs), and transform units (TUs) no longer exists in VVC-1.0; instead, each CU is always used as the basic unit for both prediction and transformation without further partitioning. In a multi-type tree structure, CTUs are first partitioned using a quadtree structure. Then, each quadtree leaf node can be further partitioned using binary and ternary tree structures. Different partitioning types can be used, such as quadtree partitioning, vertical binary partitioning, horizontal binary partitioning, vertical ternary partitioning, and horizontal ternary partitioning.

[0128] exist Figure 2AIn the encoder, spatial prediction 208 and / or temporal prediction 210 can be performed. Spatial prediction (or "intra-frame prediction") uses pixels from samples (called reference samples) of adjacent blocks already encoded and decoded in the same video picture / slice to predict the current video block. Spatial prediction reduces the spatial redundancy inherent in the video signal. Temporal prediction (also known as "inter-frame prediction" or "motion-compensated prediction") uses reconstructed pixels from already encoded and decoded video pictures to predict the current video block. Temporal prediction reduces the temporal redundancy inherent in the video signal. For a given CU, the temporal prediction signal can be signaled by one or more motion vectors (MVs) indicating the amount and direction of motion between the current CU and its temporal reference. Furthermore, if multiple reference pictures are supported, a reference picture index can be additionally sent, which identifies which reference picture in the reference picture storage 212 the temporal prediction signal originates from.

[0129] The mode decision block 214 in the encoder selects the optimal prediction mode, for example, based on a rate-distortion optimization method. This selection can be made after performing spatial and / or temporal predictions. Intra-frame / inter-frame decisions can be indicated by, for example, a prediction mode flag. The prediction block 216 is subtracted from the current video block to generate the prediction residual. The prediction residual is decorrelated 218 and quantized 220 using a transform. (For some blocks, the encoder can bypass the transform and quantization, in which case the residual can be directly encoded and decoded without applying the transform or quantization process.) The quantized residual coefficients are inversely quantized 222 and inversely transformed 224 to form the reconstructed residual, which is then added back to the prediction block 226 to form the reconstructed signal of the CU. Before the reconstructed CU is placed in the reference picture storage 212 and used for encoding and decoding future video blocks, the reconstructed CU can be further filtered 228 by in-loop filtering, such as deblocking / SAO (sample adaptive offset) filtering, to reduce coding artifacts. In order to form the output video bitstream 230, the encoding / decoding mode (inter-frame or intra-frame), prediction mode information, motion information and quantization residual coefficients are all sent to the entropy encoding / decoding unit (108) for further compression and packaging to form the bitstream.

[0130] Figure 2B A block diagram of a block-based video decoder 250 is given. In decoder 250, the bitstream is decoded by decoder elements, as described below. Figure 2A The video decoder 250 typically performs decoding delivery, which is the opposite of encoding delivery. The encoder 200 typically also performs video decoding as part of the encoded video data.

[0131] Specifically, the decoder's input includes a video bitstream 252, which can be generated by the video encoder 200. The video bitstream 252 is first unpacked and entropy-decoded in the entropy decoding unit 254 to obtain transform coefficients, motion vectors, and other encoding / decoding information. Picture partitioning information indicates how the pictures are partitioned. Therefore, the decoder can partition 256 pictures based on the decoded picture partitioning information. Encoding / decoding modes and prediction information are sent to the spatial prediction unit 258 (if intra-frame encoding / decoding) or the temporal prediction unit 260 (if inter-frame encoding / decoding) to form prediction blocks. Residual transform coefficients are sent to the inverse quantization unit 262 and the inverse transform unit 264 to reconstruct residual blocks. Then, at 266, the prediction blocks and residual blocks are added together to generate reconstructed blocks. The reconstructed blocks can be further filtered in the loop 268 before being stored in the reference picture storage 270 for predicting future video blocks.

[0132] The decoded image 272 can further undergo post-decoding processing 274, such as inverse color transformation (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping, which performs the inverse of the remapping process performed in pre-encoding processing 204. The post-decoding processing can utilize metadata derived in the pre-encoding process and signaled in the bitstream. The decoded and processed video can be sent to display device 276. Display device 276 can be a device separate from decoder 250, or decoder 250 and display device 276 can be components of the same device.

[0133] The various methods and other aspects described in this disclosure can be used to modify modules of the video encoder 200 or decoder 250. Furthermore, the systems and methods disclosed herein are not limited to VVC or HEVC and can be applied to, for example, other standards and recommendations, whether pre-existing or future-developed, and any extensions of such standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically excluded, the aspects described in this disclosure can be used alone or in combination.

[0134] Figure 3A This is a schematic side view illustrating an example waveguide display that can be used in extended reality (XR) applications according to some embodiments. The image is projected by an image generator 302. The image generator 302 can project the image using one or more of a variety of technologies. For example, the image generator 302 can be a laser beam scanning (LBS) projector, a liquid crystal display (LCD), a light-emitting diode (LED) display (including organic LED (OLED) or micro LED (µLED) displays), a digital light processor (DLP), a liquid crystal on silicon (LCoS) display, or other types of image generators or light engines.

[0135] The light representing the image 312 generated by the image generator 302 is coupled into the waveguide 304 via a diffraction coupler 306. The coupler 306 diffracts the light representing the image 312 into one or more diffraction orders. For example, a ray 308 representing a portion of the bottom of the image is diffracted by the coupler 306, and one of the diffraction orders 310 (e.g., the second order) is at an angle capable of propagating through the waveguide 304 via total internal reflection. The image generator 302 displays the image as guided by the control module 324, which operates to render image data, video data, point cloud data, or other displayable data.

[0136] At least a portion of the light 310, already coupled into waveguide 304 by diffraction input coupler 306, is coupled out of the waveguide by diffraction output coupler 314. At least some of the light coupled out of waveguide 304 replicates the incident angle of the light coupled into the waveguide. For example, in the illustration, out-coupled rays 316a, 316b, and 316c replicate the angle of the input coupled ray 308. Because the light emitted from the output coupler replicates the direction of the light entering the input coupler, the waveguide essentially replicates the original image 312. The user's eye 318 can focus on the replicated image.

[0137] exist Figure 3A In the example, the outgoing coupler 314 only couples a portion of the light on each reflection, allowing a single input beam (such as beam 308) to generate multiple parallel output beams (such as beams 316a, 316b, and 316c). Thus, even if the user's eye is not perfectly aligned with the center of the outgoing coupler, at least some light from each part of the image may reach the user's eye. For example, if eye 318 moves downwards, beam 316c may enter the eye even if beams 316a and 316b do not, so the user can still perceive the bottom of image 312 despite the positional shift. Therefore, the outgoing coupler 314 partially functions as an outgoing pupil expander in the vertical direction. The waveguide may also include one or more additional outgoing pupil expanders (…). Figure 3A (not shown in the image) to expand the exit pupil in the horizontal direction.

[0138] In some embodiments, waveguide 304 is at least partially transparent to light originating outside the waveguide display. For example, at least some light 320 from a real-world object (such as object 322) passes through waveguide 304, allowing the user to see the real-world object when using the waveguide display. Since the light 320 from the real-world object also passes through diffraction grating 314, there will be multiple diffraction orders, and therefore multiple images. To minimize the visibility of multiple images, it is desirable that the zero-order diffraction (without the deviation of 314) has high diffraction efficiency for light 320 and zero order, while higher diffraction orders are lower in energy. Therefore, in addition to extending and outcoupling the virtual image, the outcoupler 314 is preferably configured to allow the zero-order of the real-world image to pass through. In such an embodiment, the image displayed by the waveguide display can appear to be superimposed on the real world.

[0139] Figure 3B This is a schematic side view illustrating example alternative display types that can be used in extended reality applications according to some embodiments. In the XR head-mounted display device 330, a control module 332 controls a display 334, which may be an LCD, to display an image. The head-mounted display includes a partially reflective surface 336 that reflects (and in some embodiments, both reflects and focuses) the image displayed on the LCD to make the image visible to the user. The partially reflective surface 336 also allows at least some external light to pass through, allowing the user to see their surroundings.

[0140] Figure 3C This is a schematic side view illustrating example alternative display types that can be used in extended reality applications according to some embodiments. In the XR head-mounted display device 340, a control module 342 controls a display 344, which may be an LCD, to display an image. The image is focused by one or more lenses of a display optics 346 to make the image visible to the user. Figure 3C In this example, the external light does not reach the user's eyes directly. However, in some such embodiments, an external camera 348 can be used to capture images of the external environment and display such images on a display 344 along with any virtual content that may also be displayed.

[0141] The embodiments described herein are not limited to any particular type or structure of XR display device.

[0142] V-DMC uses a subdivision (or tessellation) process, starting with a coarse BASE MESH and then progressively adding vertices to obtain more geometric detail. The subdivision process is applied multiple iterations, which can be considered as adding levels of detail layer by layer. Each subdivision adds a new child vertex at the midpoint of the edge connecting its parent vertex and then encodes and decodes the displacement information to “move” that vertex to its true position. A lifting wavelet transform is used to transform the displacement values, and a key step in this transform is the “PREDICT” step, which predicts the displacement value for each component based on the displacement used to encode the parent vertex (x, y, and z displacement components are processed separately). Existing PREDICT steps are simply averages of the displacement values ​​of the parent vertex. This application discusses modifications to the PREDICT step of the lifting wavelet transform to utilize contributions from additional neighboring vertices. This application provides how to handle details from the center edge and mesh boundary edges, and also provides a fallback case for “non-manifold” mesh regions. Some embodiments weight the contributions from the parent vertex and additional neighboring vertices (e.g., based on topological or actual distances between vertices).

[0143] The current MPEG V-DMC ( Video-based dynamic mesh encoding and decoding The test model (TM) is based on the reference Mammou, K. et al. [V-CG] Apple's Dynamic Mesh Coding CfP Response ", MPEGInput Document m59281 (v4), ISO / IEC JTC 1 / SC 29 / WG 7 (2022)(" Mammou In V-DMC, the encoding and decoding of mesh geometry (the x, y, z positions of mesh vertices) is based on a variation of the subdivision wavelet scheme (see Lounsbery, M., " Multiresolution Analysis for Surfaces of Arbitrary Topological Type ", University of Washington (1994) (Doctoral Dissertation, Department of Computer Science and Engineering)" Lounsbery 1 (and Lounsbery, M. et al., ") Multiresolution Analysis for Surfaces of Arbitrary Topological Type ", 16:1 ACM Transactions on Graphics 34-73 (1997) (" Lounsbery 2 V-DMC uses lifting wavelet transform for encoding and decoding (Sweldens, W., " The Lifting Scheme: A Construction of Second Generation Wavelets ", 29: 2 SIAM J. Mathematical Analysis 511-546 (1998) (" Sweldens The subdivision wavelet scheme is based on surface subdivision.

[0144] Surface subdivision begins with a 3D mesh called a base mesh, which can be used for mesh simplification (or extraction) processes, as described by Garland, M. and Heckbert, P.S., “ Surface Simplification using Quadric Error , SIGGRAPH'97 (1997) (“ Garland The process described in the document () is obtained from a higher-resolution mesh. The base mesh contains a relatively small number of vertices and faces. A subdivision process is used to progressively refine the mesh, iteratively adding new vertices and faces to the mesh by subdividing existing faces into smaller sub-faces. The new vertices are then displaced to new locations according to some predefined rules to progressively refine the mesh shape, such as... Figures 4A to 4F The example is shown in the image.

[0145] Figure 4A This is a schematic diagram illustrating an example basic mesh according to some embodiments. Figure 4B This is a schematic diagram illustrating an example mesh after one subdivision iteration according to some embodiments. Figure 4C This is a schematic diagram illustrating an example mesh after two subdivision iterations according to some embodiments. Figure 4D This is a schematic diagram illustrating an example mesh after three subdivision iterations according to some embodiments. Figure 4E This is a schematic diagram illustrating an example mesh after four subdivision iterations according to some embodiments. Figure 4F This is a schematic diagram illustrating an example mesh after 5 subdivision iterations according to some embodiments.

[0146] There are many different surface subdivision schemes, see Benton for example. In V-DMC, the default choice is the midpoint subdivision scheme, which inserts a new vertex at the midpoint of each existing edge in each iteration.

[0147] Figures 4A-4F The diagram illustrates the subdivision process. Figure 4A The base mesh 402 to be refined is shown. Figures 4B to 4F Mesh models 404, 406, 408, 410, and 412 are shown after different numbers of subdivision iterations. Figure 4F Model 412 in the image has been rendered using interpolated shading to showcase its smoothness, while Figures 4A-4E Mesh 402, 404, 406, 408, and 410 are shown in their faceted form to illustrate the addition of new vertices and subfaces in each subdivision iteration.

[0148] As in Lounsberry 1 and Lounsberry 2The simplest form of the subdivision wavelet described in [the text] has linear time complexity for analysis and synthesis, and can be obtained by essentially equivalent to the lifting wavelet transform (…). Sweldens This is achieved through piecewise linear subdivision. In this form, the lowest resolution of the grid shape is approximately represented by the base grid, and the wavelet coefficients between any two consecutive resolution (subdivision) levels represent the difference between the predicted x, y, z positions of the "child" vertices at the higher resolution level and the predicted positions of these child positions from their "parent" vertices at the lower resolution level.

[0149] Figure 5 This is a schematic diagram illustrating example wavelet coefficient calculation in linear polyhedral subdivision according to some embodiments. For example, in Figure 5 In the middle, with the vertex of displacement (504) (of which) It was originally added to the lower resolution level. Midpoint of (508) The wavelet coefficients associated with the higher resolution level vertex at (506) (502) can be calculated as shown in Equation 1: .

[0150] In Equation 1, the wavelet coefficients are obtained yes The final location and its prediction The difference lies in the fact that the base mesh and thus the mesh geometry can be gradually refined by continuously adding more wavelet coefficients to the inserted vertices at a higher resolution (subdivision) level.

[0151] Figure 5 The illustration shows the wavelet coefficient calculation in a linear polyhedral subdivision of a 500-mesh grid. Two vertices in the example base mesh are labeled p and q. A new example vertex is labeled m (506) and inserted at the midpoint of an edge in the base mesh. The final vertex is labeled... (504), and the vertex represents the initial position at the midpoint. The vertex (predicted) inserted at (506) The final position of (504).

[0152] The subdivision wavelet framework relies on the fact that the connectivity of the underlying mesh can be refined in a predictable way (using a standard set of subdivision rules known to both the encoder and decoder) to obtain a final higher-resolution mesh. However, as is understood, most mesh models do not have predictable regular or semi-regular connectivity.

[0153] Figure 6AThis is a schematic diagram illustrating example rule connectivity of a triangular mesh according to some embodiments. Figure 6B This is a schematic diagram illustrating an example of irregular connectivity of a triangular mesh according to some embodiments. Figure 6C This is a schematic diagram illustrating an example of semi-regular connectivity of a triangular mesh according to some embodiments.

[0154] In V-DMC, the original input grid is first downsampled (e.g., by using...). Garland The method described in the text obtains a base mesh, and then applies a point subdivision scheme to iteratively upsample the base mesh again to achieve approximately the same resolution (number of vertices and faces) as the original mesh. Although in current V-DMC TM, the number of vertices and faces in the upsampled mesh is typically much greater than in the original mesh. Then, at the highest subdivision (resolution) level, the upsampled mesh vertices (…) x, y, z The difference between the vertex positions of the original mesh and the vertex positions of the subdivided mesh (where the point correspondence between the original and subdivided meshes is found through nearest-neighbor search) is represented as: Displacement vector A set of (or "displacement") vectors. These displacement vectors indicate how to deform the subdivided mesh surface to more closely approximate the original mesh surface. This is achieved by applying a linear lifting wavelet transform (...). Sweldens The displacement vector is transformed using a wavelet vector, and the resulting wavelet coefficients are encoded for transmission to the decoder.

[0155] Figures 6A-6C Examples of different connectivity types for triangular meshes are shown: Figure 6A Example 600 of regular connectivity is shown, where all vertices have the same number of incident edges. Figure 6B Example 630 of irregular connectivity is shown, where vertices can have different numbers of incident edges. Figure 6C Example 660 of semi-regular connectivity is shown, where most vertices have regular connectivity, while some vertices have irregular connectivity.

[0156] Figure 7A This is a diagram illustrating an example of a forward lifting wavelet transform according to some embodiments. Figure 7B This is a diagram illustrating an example of a forward lifting wavelet transform according to some embodiments. Figure 7C This is a diagram illustrating an example of inverse lifting wavelet transform according to some embodiments. Figure 7D This is a diagram illustrating an example of inverse lifting wavelet transform according to some embodiments.

[0157] The (positive) lifting wavelet transform 700 currently used in V-DMC is in Figure 7AThe process is described in the following steps: odd / even segmentation 702, prediction (P) 704, wavelet coefficient calculation 706, and update (U) block 708.

[0158] Figure 7B An example transform 720 using higher resolution level samples as odd index samples is shown, with the following steps: odd / even segmentation 722, prediction (P) 724, wavelet coefficient calculation 726, and update (U) block 728.

[0159] Figure 7C An example inverse transform 740 using higher resolution level samples as even index samples is shown, with the following steps: update (U) block 742, prediction (P) 744, calculation of even samples 746, and odd / even merging 748.

[0160] Figure 7D An example inverse transform 760 using higher resolution level samples as odd index samples is shown, with the following steps: updating (U) block 762, predicting (P) 764, calculating even samples 766, and odd / even merging 768.

[0161] The input to the odd / even segmentation block 702 is a signal (in this case, a displacement vector). x, y and z The signal is segmented into even-indexed and odd-indexed samples (the components are processed separately), such that each group contains half of the original signal. This segmentation step is sometimes referred to as a "lazy wavelet transform." In the context of a grid, odd-indexed and even-indexed samples correspond to vertices at different resolution levels. Odd-indexed samples typically correspond to vertices at lower resolution levels (similar to the result of a low-pass filter in a conventional wavelet transform), and even-indexed samples correspond to vertices at higher resolution levels. In some embodiments, odd-indexed samples may correspond to higher-resolution level samples, and even-indexed samples may correspond to lower-resolution level samples. Such a scenario occurs in... Figure 7B and Figure 7D As shown, the opposite scenario is... Figure 7A and Figure 7C As shown in the image.

[0162] Prediction (P) step 704 calculates the prediction (approximate) for even samples based on odd samples (or vice versa). The prediction is calculated as shown in Equation 2: in Indicates adding to the edge The new vertex in the middle, and and These are the vertices and The value of the signal at that location (in this case, the displacement value). Vertex The predicted displacement at the vertex is and The average value of the signal (displacement) on the signal.

[0163] To calculate the 706 wavelet coefficients, the predicted results are subtracted from the even-numbered samples, and the resulting residual values ​​are the wavelet coefficients. This calculation can be expressed by the formula shown in Equation 3: in Represents vertices The actual value of the signal (displacement) on the signal, and Indicates the newly added vertex The associated wavelet coefficients (which also have x, y, z The value of the component.

[0164] Update (U) step 708 adds wavelet coefficients to the odd-numbered signal samples (which represent a coarser “approximation” of the signal). This addition recovers some of the energy lost during signal segmentation and prepares the signal for the next prediction step (if any). In V-DMC, the current calculation is updated as shown in Equation 4: in Indicator Vertex The wavelet coefficients of the neighboring vertices. V-DMC™ can also choose to completely skip the update step.

[0165] Figure 7A It is shown that it is used in V-DMC and Mammou The forward lifting wavelet transform 700 is shown in the figure.

[0166] Figure 7C An inverse wavelet transform 740 is shown, used at the decoder to reconstruct the input signal (displacement) from a coarse approximation and wavelet coefficients, performed in the opposite direction to the forward wavelet transform. An inverse wavelet transform is also performed at the encoder to reconstruct the mesh geometry, allowing texture transfer to be applied prior to texture compression. See also Figure 11 .

[0167] This application describes a modification to the “prediction” step (Equation 2) of the lifting wavelet transform. Currently, the prediction is based on the linear interpolation (average) between the signals on the two parent vertices on the edge where the new child vertex is inserted.

[0168] In some embodiments, the signal value at a child vertex is predicted by using the signal value of its parent vertex on the same side and the signal values ​​at other vertices of the triangle adjacent to that side.

[0169] Figure 8A This is a schematic diagram illustrating an example manifold mesh according to some embodiments. Figure 8B This is a schematic diagram illustrating an example non-manifold mesh according to some embodiments. Figure 8C This is a schematic diagram illustrating an example non-manifold mesh according to some embodiments.

[0170] For the manifold triangular mesh 800 with boundaries, see [reference needed]. Figure 8A For a boundary-bounded manifold triangular mesh, each edge on the boundary has only one adjacent triangle, while edges away from the boundary each have two adjacent triangles. This application focuses on boundary-bounded and boundaryless manifold triangular meshes. Figure 8B and Figure 8C This refers to non-manifold meshes 830 and 860. For manifold meshes, each edge not on the boundary is shared by only two faces, not more, and a ring of connected faces must surround each vertex, not a broken ring, as shown below. Figure 8B As shown. Boundary edges are shared by only one face.

[0171] Figure 9A This is a schematic diagram illustrating an example closed mesh without boundaries according to some embodiments. Figure 9B This is a schematic diagram illustrating an example open mesh with boundaries according to some embodiments. Figure 9A An example of a closed grid 900 without boundaries is shown. Figure 9A In the triangle, each side has two adjacent triangles. Figure 9B An example of an open grid 950 with boundary 952 is shown.

[0172] Figure 10A This is a schematic diagram illustrating an example mesh containing new vertices not on the mesh boundaries, according to some embodiments. Figure 10B This is a schematic diagram illustrating an example mesh containing new vertices on the mesh boundary according to some embodiments.

[0173] In some embodiments, neighboring vertices are considered to predict the signal at the newly inserted vertex, examples of which are shown in Figure 10A and Figure 10B As shown in the diagram, considering not only the immediate neighbors of the new vertex allows for more accurate predictions of the signal value at the new vertex. This increased accuracy arises because the signal value can be influenced by the values ​​at all vertices of the adjacent triangles, not just by the value of its direct parent node that appears on the same edge as the new vertex.

[0174] Figure 10AAn example of a manifold triangular mesh 1000 is shown, in which edge 1002 of a newly inserted vertex 1004 is not on the mesh boundary. Point c (1004) is the newly inserted vertex. Points a (1006) and b (1008) are direct parent nodes. Points d (1010) and e (1012) are additional nearest neighbors, besides points a (1006) and b (1008), that can be used to predict the position of point c (1004), which can be represented as the signal value of the newly inserted vertex.

[0175] Figure 10B An example of a manifold triangular mesh 1050 is shown, where the new vertex is on the mesh boundary. Points a (1052) and b (1054) are direct parent nodes. Point d (1058) is an additional nearest neighbor, in addition to points a (1052) and b (1054), that can be used to predict the location of point c (1056).

[0176] In some scenarios, certain portions of the mesh may be manifold, while others may be non-manifold. In some implementations, manifold conditions can be checked, which could involve checking the number of adjacent triangles on each edge. This check can be performed before predicting the signal (position) of a newly inserted vertex on that edge.

[0177] If the grid in the corresponding region is manifold, then additional nearest neighbors can be used in the prediction. If the grid in the corresponding region is non-manifold, then midpoint prediction can be used, as shown in Equation 2.

[0178] Figure 11 This is a process diagram illustrating an example V-DMC intra encoder according to some embodiments. For general details regarding the V-DMC intra encoder 1100, see [link to documentation]. Mammou In some embodiments, the idea of ​​using additional nearest neighbors to predict the signal (position) of newly inserted vertices can be applied within wavelet transform block 1102 and inverse wavelet transform block 1104. Figure 11 Two blocks, 1102 and 1104, are shown. Such changes to these blocks will have an effect on the operation of the V-DMC intra-frame encoder 1100.

[0179] Figure 12 This is a flowchart illustrating an example V-DMC intra-frame decoder according to some embodiments. For general details regarding the V-DMC intra-frame decoder 1200, see [link to documentation]. Mammou In some embodiments, the idea of ​​using additional nearest neighbors to predict the signal (position) of a newly inserted vertex can be applied within the inverse wavelet transform block 1202, as shown below. Figure 12 As shown. Such a change to block 1202 will have an effect on the operation of the V-DMC intra-frame decoder 1200.

[0180] The following sections describe some variations of the prediction methods used for lifting wavelet transform, which take into account... Figure 10A and Figure 10B The neighboring vertices are shown in the diagram. Regardless of how the signal value at the newly inserted vertex is predicted, the mesh connectivity remains the same. A vertex remains on the edge where the inserted vertex is located (e.g., as shown in the diagram). Figure 10A and Figure 10B As illustrated, this ensures that the connectivity of the mesh in the neighborhood remains unchanged. In other words, the new vertex position (x, y, z) is still at the midpoint of the corresponding edge, regardless of how the signal at that vertex is predicted (the displacement value in the case of V-DMC).

[0181] Average of neighbors In some embodiments, the predicted signal at each new vertex can be calculated as the average of the signal values ​​of its nearest neighbors, such as... Figure 10A As shown in Equation 5: in This is the signal value corresponding to the vertex (e.g., the displacement value in V-DMC). For Figure 10B The corresponding prediction is shown in Equation 6: .

[0182] Weighted average of nearest neighbors In some embodiments, weights can be applied to signals from different neighboring vertices before calculating predictions. In some embodiments, the weights can be applied in a manner reproducible at the decoder, without relying on any additional data from the encoder. This avoids transmitting extra data to the decoder.

[0183] Weights of triangles of approximately the same size In some implementations, the weights can be calculated as the reciprocal of the number of "edges" between the vertex to be predicted and each of its nearest neighbors. Figure 10A and Figure 10B In the scenario shown, adjacent triangles are approximately equal in size and approximately equilateral, and the "distance" between vertices a and c, and between c and b, is approximately half the length of each side. This relationship arises because point c is initially inserted at the midpoint of the side connecting a and b. Therefore, for this scenario, the weights of the signals on a and b would be: In each case, the "distance" between c and d, and between c and e, will be an edge, so the weights of the signals on d and e will be 1. Figure 10A The scenario shown is predicted in Equations 7 and 8: .

[0184] for Figure 10B The situation shown is predicted as shown in Equation 9: .

[0185] Weights of triangles of different sizes Figure 13 This is a schematic diagram illustrating adjacent triangles of different sizes representing new mesh vertices according to some embodiments. Figure 13 As illustrated, for example, adjacent triangles 1302 and 1304 with an edge having a newly inserted vertex 1306 can have significantly different sizes. In this case, the default midpoint subdivision can be used. Alternatively, the scenario shown in Equation 10 below can be used.

[0186] In the subdivision surface level, before calculating the prediction of vertices at the next resolution level, the mesh triangles and associated vertex positions (plus the signals on those vertices) at a resolution level (or subdivision level, or Level of Detail (LoD)) are reconstructed. Thus, as... Figure 13 The diagram illustrates how the actual distances between a new vertex (initially inserted at the midpoint of an edge) and each of its nearest neighbors can be determined. In this case, the prediction for vertex c can be calculated as shown in Equation 10: .

[0187] distance It can be calculated as the corresponding vertex The 3D Euclidean distance between locations. In some embodiments, to save computation time and resources, the square root operation of the 3D Euclidean distance calculation can be skipped, and the squared Euclidean distance can be used instead.

[0188] Figure 14 This is a schematic diagram illustrating adjacent triangles of different sizes for new mesh vertices according to some embodiments. Figure 14 Example 1400 shows the distances 1402, 1404, 1406, and 1408 between vertex position coordinates 1410, 1412, 1414, and 1416, which can be used to calculate the distances and weights in Equation 10.

[0189] Figure 15 This is a schematic diagram illustrating an example adjacent triangle of a new vertex on a mesh boundary according to some embodiments. Similar to Equation 10, when an edge with an inserted vertex is on the mesh boundary, a weighted method can be used, such as... Figure 15 As shown in Equation 11: .

[0190] Figure 15Example 1500 shows the distances 1502, 1504, and 1506 between vertex position coordinates 1508, 1510, and 1512, which can be used to calculate the weights in Equation 11.

[0191] In some embodiments, the above method may not work well when the angle between adjacent triangles of an edge is too large (which can be measured and compared with a threshold). In such cases, some embodiments may default to using midpoint prediction, such as the midpoint prediction shown in Equation 2.

[0192] As understood, this application provides a novel approach to compute / determine the prediction of signals for newly inserted vertices on a subdivision surface using lifting wavelet transform to compute subdivision wavelets. While this application can be applied to the MPEG V-DMC standard, the ideas presented herein can be applied to other systems using lifting wavelet transform with manifold triangular meshes.

[0193] As understood, this application describes potentially better predictions with smaller prediction residuals or wavelet coefficients for encoding and transmission to the decoder by taking into account more nearest neighbors of the vertices whose signals are to be predicted.

[0194] In some embodiments, slightly more computation may occur than when calculating the midpoint prediction, due to the possible performance of additional checks. For example, before making a prediction, the adjacent triangles for each edge may be checked, including the side lengths and / or the angles of the adjacent triangles. If these adjacent triangles are efficiently stored in memory, they can be accessed and processed efficiently.

[0195] Figure 16 This is a flowchart illustrating an example process for performing a wavelet transform according to some embodiments. In some embodiments, example process 1600 may include obtaining an input signal 1602 corresponding to a grid. In some embodiments, example process 1600 may further include segmenting the input signal 1604 into a plurality of odd-indexed samples and a plurality of even-indexed samples, wherein the plurality of odd-indexed samples of the input signal correspond to vertices at a lower resolution level in the grid, and wherein the plurality of even-indexed samples of the input signal correspond to vertices at a higher resolution level in the grid. In some embodiments, example process 1600 may further include using at least three samples from the plurality of odd-indexed samples to determine at least one sample of a predicted signal 1606. In some embodiments, example process 1600 may further include subtracting the predicted signal 1608 from the plurality of even-indexed samples to generate a wavelet coefficient signal.

[0196] Figure 17This is a flowchart illustrating an example process for performing an inverse wavelet transform according to some embodiments. In some embodiments, example process 1700 may include obtaining 1702 a coarse input signal of the wavelet transform corresponding to a grid, wherein the coarse input signal of the wavelet transform includes a first plurality of samples. In some embodiments, example process 1700 may further include obtaining 1704 a wavelet coefficient input signal corresponding to the grid. In some embodiments, example process 1700 may further include using at least three of the first plurality of samples of the grid signal to determine 1706 at least one sample of a predicted signal. In some embodiments, example process 1700 may further include adding the predicted signal 1708 to the wavelet coefficient input signal to generate a second plurality of samples. In some embodiments, example process 1700 may further include merging the first plurality of samples with the second plurality of samples 1710 to generate an output signal corresponding to the grid.

[0197] While the methods and systems according to some embodiments are generally discussed in the context of extended reality (XR), some embodiments can be applied to any XR context, such as virtual reality (VR) / mixed reality (MR) / augmented reality (AR) contexts. Furthermore, although the term "head-mounted display (HMD)" is used herein according to some embodiments, for some embodiments, some embodiments can be applied to wearable devices capable of, for example, XR, VR, AR, and / or MR (which may or may not be attached to the head).

[0198] A first example method according to some embodiments may include: obtaining an input signal corresponding to a grid; segmenting the input signal into a plurality of odd-indexed samples and a plurality of even-indexed samples, wherein the plurality of odd-indexed samples of the input signal correspond to vertices at a lower resolution level in the grid, and wherein the plurality of even-indexed samples of the input signal correspond to vertices at a higher resolution level in the grid; using at least three of the plurality of odd-indexed samples to determine at least one sample of a prediction signal; and subtracting the prediction signal from the plurality of even-indexed samples to generate a wavelet coefficient signal.

[0199] Some embodiments of the first example method may further include subtracting wavelet coefficient signals from a plurality of odd-indexed samples.

[0200] For some embodiments of the first example method, determining at least one sample of the predicted signal may include determining at least one weighting value based on four samples from a plurality of odd-indexed samples, wherein the four samples are the nearest neighbors of at least one sample of the predicted signal.

[0201] In some embodiments of the first example method, the nearest neighbor is the vertex of a triangle that is adjacent to at least one sample of the predicted signal.

[0202] For some embodiments of the first example method, determining at least one weighting value based on four samples from a plurality of odd-indexed samples includes determining unequal weights for the four samples.

[0203] In some embodiments of the first example method, determining at least one weighting value is based on the relative distances of the four samples to at least one sample of the predicted signal.

[0204] For some embodiments of the first example method, determining at least one sample of the predicted signal includes determining at least one weighting value based on three samples from a plurality of odd-indexed samples, wherein the three samples are the nearest neighbors of at least one sample of the predicted signal.

[0205] In some embodiments of the first example method, the nearest neighbor is the vertex of a triangle that is adjacent to at least one sample of the predicted signal.

[0206] For some embodiments of the first example method, determining at least one weighted value based on three samples from a plurality of odd-indexed samples includes determining unequal weights for the three samples.

[0207] In some embodiments of the first example method, determining at least one weighting value is based on the relative distance between the three samples and at least one sample of the predicted signal.

[0208] In some embodiments of the first example method, at least one sample of the predicted signal may include the boundary points of the grid.

[0209] A first example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0210] A second example method according to some embodiments may include: obtaining a coarse input signal of a wavelet transform corresponding to a grid, wherein the coarse input signal of the wavelet transform includes a first plurality of samples; obtaining a wavelet coefficient input signal corresponding to the grid; using at least three of the first plurality of samples of the grid signal to determine at least one sample of a prediction signal; adding the prediction signal to the wavelet coefficient input signal to generate a second plurality of samples; and merging the first plurality of samples with the second plurality of samples to generate an output signal corresponding to the grid.

[0211] In some embodiments of the second example method, the first plurality of samples correspond to even-indexed samples of the output signal, and the second plurality of samples correspond to odd-indexed samples of the output signal.

[0212] Some embodiments of the second example method may further include subtracting the wavelet coefficient input signal from the coarse input signal of the wavelet transform.

[0213] In some embodiments of the second example method, determining at least one sample of the predicted signal may include determining at least one weighting value based on four samples from a first plurality of samples, wherein the four samples are the nearest neighbors of at least one sample of the predicted signal.

[0214] In some embodiments of the second example method, the nearest neighbor is the vertex of a triangle that is adjacent to at least one sample of the predicted signal.

[0215] For some embodiments of the second example method, determining at least one weighting value based on four samples of the first plurality of samples includes determining unequal weights for the four samples.

[0216] In some embodiments of the second example method, determining at least one weighting value is based on the relative distance between the four samples and at least one sample of the predicted signal.

[0217] For some embodiments of the second example method, determining at least one sample of the predicted signal includes determining at least one weighting value based on three samples from a first plurality of samples, wherein the three samples are the nearest neighbors of at least one sample of the predicted signal.

[0218] In some embodiments of the second example method, the nearest neighbor is the vertex of a triangle that is adjacent to at least one sample of the predicted signal.

[0219] For some embodiments of the second example method, determining at least one weighted value based on three samples of the first plurality of samples includes determining unequal weights for the three samples.

[0220] In some embodiments of the second example method, determining at least one weighting value is based on the relative distance between the three samples and at least one sample of the predicted signal.

[0221] In some embodiments of the second example method, at least one sample of the predicted signal includes the boundary points of the grid.

[0222] A second example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0223] A third example method according to some embodiments may include: obtaining an input signal corresponding to a grid; dividing the input signal into a first plurality of samples and a second plurality of samples, using at least three of the first plurality of samples to determine at least one sample of a prediction signal; and subtracting the prediction signal from the second plurality of samples to generate a wavelet coefficient signal.

[0224] In some embodiments of the second example method, a first plurality of samples of the input signal correspond to vertices at a lower resolution level in the grid, and a second plurality of samples of the input signal correspond to vertices at a higher resolution level in the grid.

[0225] In some embodiments of the second example method, a first plurality of samples of the input signal correspond to odd-indexed samples of the input signal, and a second plurality of samples of the input signal correspond to even-indexed samples of the input signal.

[0226] In some embodiments of the second example method, a first plurality of samples of the input signal correspond to vertices at a higher resolution level in the grid, and a second plurality of samples of the input signal correspond to vertices at a lower resolution level in the grid.

[0227] In some embodiments of the second example method, a first plurality of samples of the input signal correspond to even-indexed samples of the input signal, and a second plurality of samples of the input signal correspond to odd-indexed samples of the input signal.

[0228] A third example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0229] According to some embodiments, an example method for performing wavelet transform on an input signal corresponding to a grid may include: obtaining the input signal; segmenting the input signal into lower-resolution samples and higher-resolution samples; generating a prediction signal, wherein the prediction signal includes prediction samples corresponding to the higher-resolution samples, and wherein each prediction sample is generated based on at least three of the lower-resolution samples; and subtracting the prediction signal from the higher-resolution samples to generate a wavelet coefficient signal.

[0230] A fourth example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0231] According to some embodiments, an example method for performing wavelet transform on an input signal corresponding to a grid may include: segmenting the input signal into lower-resolution samples and higher-resolution samples; generating a prediction signal based on at least three lower-resolution samples for each sample of the prediction signal; and subtracting the prediction signal from the higher-resolution samples to generate a wavelet coefficient signal.

[0232] A fifth example apparatus according to some embodiments may include: a processor; and a non-transitory computer-readable medium storing instructions that, when executed by the processor, are operable to cause the apparatus to perform any of the methods listed above.

[0233] A sixth example apparatus according to some embodiments may include at least one processor configured to perform any of the methods listed above.

[0234] A seventh example apparatus according to some embodiments may include a computer-readable medium storing instructions for causing one or more processors to perform any of the methods listed above.

[0235] An eighth example apparatus according to some embodiments may include at least one processor and at least one non-transitory computer-readable medium storing instructions for causing at least one processor to perform any of the methods listed above.

[0236] Example signals according to some embodiments may include signals that transmit wavelet coefficient signals generated according to the methods listed above.

[0237] This disclosure describes a wide variety of aspects, including tools, features, embodiments, models, methods, etc. Many of these aspects are described in detail and are generally described in a manner that may sound restrictive, at least for the purpose of showing individual characteristics. However, this is for the purpose of clarity and does not limit the disclosure or scope of these aspects. In fact, all the different aspects can be combined and interchanged to provide more aspects. Furthermore, these aspects can also be combined and interchanged with aspects described in earlier applications.

[0238] The aspects described and contemplated in this disclosure can be implemented in many different forms. While specific embodiments are illustrated, other embodiments are contemplated, and the discussion of particular embodiments does not limit the breadth of implementation. At least one aspect generally relates to video encoding and decoding, and at least one other aspect generally relates to the transmission of a generated or encoded bitstream. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions stored thereon for encoding or decoding video data according to any of the described methods, and / or computer-readable storage media having a bitstream generated according to any of the described methods stored thereon.

[0239] In this disclosure, the terms "reconstruction" and "decoding" are used interchangeably, as are the terms "pixel" and "sample," and the terms "image," "picture," and "frame." Generally, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0240] The terms HDR (High Dynamic Range) and SDR (Standard Dynamic Range) generally convey specific values ​​of dynamic range to those skilled in the art. However, additional embodiments are also intended, wherein a reference to HDR is understood to mean "higher dynamic range," and a reference to SDR is understood to mean "lower dynamic range." Such additional embodiments are not bound by any specific values ​​of dynamic range that may often be associated with the terms "high dynamic range" and "standard dynamic range."

[0241] This document describes various methods, and each method includes one or more steps or actions for implementing the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined. Additionally, terms such as "first," "second," etc., can be used in various embodiments to modify elements, components, steps, operations, etc., such as, for example, "first decoding" and "second decoding." Unless specifically required, the use of these terms does not imply a sequence of modified operations. Therefore, in this example, the first decoding does not need to be performed before the second decoding and can occur, for example, before, during, or within a time period overlapping with the second decoding.

[0242] For example, various numerical values ​​may be used in this disclosure. Specific values ​​are for illustrative purposes, and the aspects described are not limited to these specific values.

[0243] The embodiments described herein can be implemented by computer software implemented by a processor, or by other hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments can be implemented by one or more integrated circuits. As a non-limiting example, the processor can be of any type suitable for the technical environment and can encompass one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0244] Various implementations involve decoding. As used in this disclosure, “decoding” can encompass, for example, all or part of the processing performed on a received encoded sequence to produce a final output suitable for display. In various embodiments, these processes include one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also or alternatively include processes performed by a decoder of various embodiments described in this disclosure, such as extracting an image from a tiled (packed) image, determining an upsampling filter to use, then upsampling the image, and flipping the image back to its intended orientation.

[0245] As a further example, in one embodiment, "decoding" refers only to entropy decoding; in another embodiment, "decoding" refers only to differential decoding; and in yet another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. It will be clear, based on the specific context of the description, whether the phrase "decoding process" is intended to specifically refer to a subset of operations or to refer to a broader decoding process.

[0246] Various implementations involve encoding. In a manner similar to the discussion of “decoding” above, the term “encoding” as used in this disclosure can encompass all or part of a process performed, for example, on an input video sequence to produce an encoded bitstream. In various embodiments, these processes include one or more processes typically performed by an encoder, such as partitioning, differential coding, transform, quantization, and entropy coding. In various embodiments, such processes also, or alternatively, include processes performed by an encoder of the various embodiments described in this disclosure.

[0247] As a further example, in one embodiment, "encoding" refers only to entropy encoding; in another embodiment, "encoding" refers only to differential encoding; and in yet another embodiment, "encoding" refers to a combination of differential and entropy encoding. It will be clear, based on the specific context of the description, whether the phrase "encoding process" is intended to refer specifically to a subset of operations or to a broader encoding process.

[0248] When a diagram is represented as a flowchart, it should be understood that it also provides a block diagram of the apparatus. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flowchart of the corresponding method / process.

[0249] Various embodiments involve rate-distortion optimization. Specifically, during the encoding process, a balance or trade-off between rate and distortion is typically considered, often with constraints on computational complexity. Rate-distortion optimization is generally formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. Different approaches exist to address the rate-distortion optimization problem. For example, these approaches can be based on extensive testing of all encoding options, including all considered modes or codec parameter values, and a complete evaluation of their encoding / decoding costs and the associated distortion of the encoded and decoded reconstructed signals. Faster methods can also be used to save encoding complexity, particularly by calculating approximate distortion based on predicting or predicting the residual signal rather than the reconstructed signal. A hybrid of these approaches can also be used, such as by using approximate distortion only for some possible encoding options and full distortion for others. Other methods evaluate only a subset of possible encoding options. More generally, many methods employ any of a wide variety of techniques to perform optimization, but optimization is not necessarily a complete evaluation of encoding / decoding costs and associated distortion.

[0250] The implementations and aspects described herein can be implemented, for example, as methods or processes, apparatuses, software programs, data streams, or signals. Even if discussed only in the context of a single implementation form (e.g., discussed only as a method), implementations of the discussed features can be implemented in other forms (e.g., apparatuses or programs). Apparatuses can be implemented, for example, in suitable hardware, software, and firmware. Methods can be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as, for example, computers, cellular phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0251] References to “an embodiment” or “an embodiment” or “a implementation” or “implementation”, and other variations thereof, mean that a particular feature, structure, characteristic, etc., described in connection with that embodiment is included in at least one embodiment. Therefore, the appearance of the phrases “in an embodiment” or “in an embodiment” or “in an implementation” or “in an implementation,” and any other variations appearing throughout this disclosure, do not necessarily all refer to the same embodiment.

[0252] Additionally, this disclosure may relate to "determining" various pieces of information. Determining information may include, for example, one or more of estimated information, calculated information, predicted information, or information retrieved from memory.

[0253] Furthermore, this disclosure may relate to “accessing” various information pieces. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or one or more of these.

[0254] Additionally, this disclosure may relate to “receiving” various pieces of information. Like “access,” receiving is intended to be a broad term. Receiving information may include, for example, accessing information or retrieving information (e.g., from memory) one or more. Furthermore, “receiving” is generally referred to in one way or another during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0255] It is important to understand that, for example, in the cases of “A / B,” “A and / or B,” and “at least one of A and B,” any of the following uses of “ / ,” “and / or,” and “at least one of…” are intended to cover selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As another example, in the cases of “A, B, and / or C” and “at least one of A, B, and C,” such wording is intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A, B, and C). This can be extended to many listed items.

[0256] Furthermore, as used herein, the word “signal” refers, among other things, to instructing the corresponding decoder to do something. For example, in some embodiments, the encoder signals a specific one of several parameters for selecting region-based filter parameters used for artifact removal filtering. In this way, in embodiments, the same parameter is used on both the encoder and decoder sides. Thus, for example, the encoder may transmit (explicit signaling) a specific parameter to the decoder so that the decoder can use the same specific parameter. Conversely, if the decoder already has the specific parameter as well as other parameters, then signaling can then be used without transmission (implicit signaling) to simply allow the decoder to know and select the specific parameter. Bit savings are achieved in various embodiments by avoiding the transmission of any actual functionality. It should be understood that signaling can be implemented in a variety of ways. For example, in various embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the verb form of the word “signal” has been referred to above, the word “signal” may also be used as a noun herein.

[0257] Implementations can generate a wide variety of signals that are formatted to carry information, such as information that can be stored or transmitted. This information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, a signal may be formatted to carry a bitstream of a described embodiment. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of a spectrum) or baseband signals. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. It is well known that signals can be transmitted via a wide variety of wired or wireless links. The signal may be stored on a processor-readable medium.

[0258] Several embodiments are described. Features of these embodiments may be provided individually or in any combination across various claim classes and types. Furthermore, embodiments may include one or more of the following features, devices, or aspects (individually or in any combination) across various claim classes and types: ● Adapt the residual at the encoder according to any of the embodiments discussed.

[0259] ● Includes one or more bitstreams or signals describing syntax elements or their variations.

[0260] ● Includes bit streams or signals that express the syntax of information generated according to any of the described embodiments.

[0261] ● Insert elements into the signaling syntax elements that enable the decoder to adapt the residuals in a manner corresponding to that used by the encoder.

[0262] ● Creating and / or transmitting and / or receiving and / or decoding bitstreams or signals comprising one or more of the described syntax elements or variations thereof.

[0263] ● Create and / or transmit and / or receive and / or decode according to any of the embodiments described.

[0264] ● Methods, processes, apparatus, media for storing instructions, media for storing data, or signals according to any of the described embodiments.

[0265] ● TVs, set-top boxes, mobile phones, tablets, or other electronic devices that perform filter parameter adaptation according to any of the described embodiments.

[0266] ● TVs, set-top boxes, mobile phones, tablets, or other electronic devices that perform filter parameter adaptation according to any of the described embodiments and display (e.g., using a monitor, screen, or other type of display) the resulting image.

[0267] ● TVs, set-top boxes, mobile phones, tablets, or other electronic devices that select (e.g., using a tuner) a channel to receive signals including encoded images and perform filter parameter adaptation according to any of the embodiments described.

[0268] ● TVs, set-top boxes, mobile phones, tablets or other electronic devices that receive signals including encoded images over the air (e.g., using an antenna) and perform filter parameter adaptation according to any of the embodiments described.

[0269] Note that the various hardware elements of one or more described embodiments are referred to as “modules”, which implement (i.e., perform, execute, etc.) the various functions described herein in conjunction with the respective modules. As used herein, a module includes hardware (e.g., one or more processors, one or more microprocessors, one or more microcontrollers, one or more microchips, one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), one or more memory devices) that a person skilled in the art would consider suitable for a given implementation. Each described module may also include instructions that are executable to implement one or more functions described as being implemented by the respective modules, and it should be noted that these instructions may take the form of hardware (i.e., hardwired) instructions, firmware instructions, software instructions, etc., or include hardware instructions, firmware instructions, software instructions, etc., and may be stored in any suitable non-transitory computer-readable medium, such as commonly referred to as RAM, ROM, etc.

[0270] Although the features and elements have been described above in specific combinations, one of those skilled in the art will appreciate that each feature or element can be used alone or in combination with other features and elements. Furthermore, the methods described herein can be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor storage devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROMs and digital multifunction discs (DVDs). The processor associated with the software can be used to implement a radio frequency transceiver used in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. A method comprising: Obtain the input signal corresponding to the grid; The input signal is divided into multiple odd-indexed samples and multiple even-indexed samples. The plurality of odd-indexed samples of the input signal correspond to vertices at lower resolution levels in the grid, and The plurality of even-indexed samples of the input signal correspond to vertices at higher resolution levels in the grid; At least three of the plurality of odd-indexed samples are used to determine at least one sample of the predicted signal; as well as The predicted signal is subtracted from the plurality of even-indexed samples to generate the wavelet coefficient signal.

2. The method of claim 1, further comprising subtracting wavelet coefficient signals from the plurality of odd-indexed samples.

3. The method according to any one of claims 1-2, The determination of the at least one sample of the predicted signal includes determining at least one weighting value based on four samples from the plurality of odd-indexed samples, and The four samples are the nearest neighbors of at least one sample of the predicted signal.

4. The method of claim 3, wherein the nearest neighbor is the vertex of a triangle adjacent to the at least one sample of the predicted signal.

5. The method according to any one of claims 3-4, wherein determining the at least one weighted value based on four of the plurality of odd-indexed samples comprises determining unequal weights of the four samples.

6. The method according to any one of claims 3-5, wherein determining the at least one weighting value is based on the relative distance between the four samples and the at least one sample of the predicted signal.

7. The method according to any one of claims 1-6, The determination of the at least one sample of the predicted signal includes determining at least one weighting value based on three samples from the plurality of odd-indexed samples, and The three samples are the nearest neighbors of at least one sample of the predicted signal.

8. The method of claim 7, wherein the nearest neighbor is the vertex of a triangle adjacent to the at least one sample of the predicted signal.

9. The method according to any one of claims 7-8, wherein determining the at least one weighted value based on three samples from the plurality of odd-indexed samples comprises determining unequal weights of the three samples.

10. The method according to any one of claims 7-9, wherein determining the at least one weighting value is based on the relative distance between the three samples and the at least one sample of the predicted signal.

11. The method according to any one of claims 1-10, wherein the at least one sample of the predicted signal comprises boundary points of the grid.

12. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by a processor, are operable to cause the apparatus to perform the method of any one of claims 1 to 11.

13. A method comprising: Obtain a coarse input signal corresponding to the wavelet transform of the grid. The coarse input signal of the wavelet transform includes a first set of multiple samples; Obtain the wavelet coefficient input signal corresponding to the grid; At least three of the first plurality of samples of the grid signal are used to determine at least one sample of the predicted signal; The predicted signal is added to the wavelet coefficient input signal to generate a second set of samples; as well as The first set of samples is combined with the second set of samples to generate an output signal corresponding to the grid.

14. The method according to claim 13, The first plurality of samples correspond to even-indexed samples of the output signal, and The second plurality of samples correspond to the odd-indexed samples of the output signal.

15. The method according to any one of claims 13-14, further comprising subtracting the wavelet coefficient input signal from the coarse input signal of the wavelet transform.

16. The method according to any one of claims 13-15, The determination of the at least one sample of the predicted signal includes determining at least one weighting value based on four samples from a first plurality of samples, and The four samples are the nearest neighbors of at least one sample of the predicted signal.

17. The method of claim 16, wherein the nearest neighbor is the vertex of a triangle adjacent to the at least one sample of the predicted signal.

18. The method of any one of claims 16-17, wherein determining the at least one weighted value based on four of the first plurality of samples comprises determining unequal weights of the four samples.

19. The method according to any one of claims 16-18, wherein determining the at least one weighting value is based on the relative distance between the four samples and the at least one sample of the predicted signal.

20. The method according to any one of claims 13-19, The determination of the at least one sample of the predicted signal includes determining at least one weighting value based on three samples from a first plurality of samples, and The three samples are the nearest neighbors of at least one sample of the predicted signal.

21. The method of claim 20, wherein the nearest neighbor is the vertex of a triangle adjacent to the at least one sample of the predicted signal.

22. The method according to any one of claims 20-21, wherein determining the at least one weighted value based on three samples from a first plurality of samples includes determining unequal weights for the three samples.

23. The method according to any one of claims 20-22, wherein determining the at least one weighting value is based on the relative distance between the three samples and the at least one sample of the predicted signal.

24. The method according to any one of claims 13-23, wherein the at least one sample of the predicted signal comprises boundary points of the grid.

25. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by a processor, are operable to cause the apparatus to perform the method of any one of claims 13 to 24.

26. A method comprising: Obtain the input signal corresponding to the grid; The input signal is divided into a first set of samples and a second set of samples. At least three of the first plurality of samples are used to determine at least one sample of the predicted signal; as well as The predicted signal is subtracted from the second set of samples to generate the wavelet coefficient signal.

27. The method according to claim 26, The first plurality of samples of the input signal correspond to vertices at lower resolution levels in the grid, and The second plurality of samples of the input signal correspond to vertices at a higher resolution level in the grid.

28. The method according to any one of claims 26-27, The first plurality of samples of the input signal correspond to the odd-indexed samples of the input signal, and The second plurality of samples of the input signal correspond to the even-indexed samples of the input signal.

29. The method according to claim 26, The first plurality of samples of the input signal correspond to vertices at a higher resolution level in the grid, and The second plurality of samples of the input signal correspond to vertices at a lower resolution level in the grid.

30. The method according to any one of claims 26 and 29, The first plurality of samples of the input signal correspond to the even-indexed samples of the input signal, and The second plurality of samples of the input signal correspond to the odd-indexed samples of the input signal.

31. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by a processor, are operable to cause the apparatus to perform the method of any one of claims 26 to 30.

32. A method for performing wavelet transform on an input signal corresponding to a grid, comprising: Obtain the input signal; The input signal is segmented into lower-resolution samples and higher-resolution samples; Generate prediction signals, The prediction signal includes prediction samples corresponding to higher resolution samples, and Each predicted sample is generated based on at least three lower-resolution samples; and The predicted signal is subtracted from the higher resolution sample to generate the wavelet coefficient signal.

33. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by a processor, are operable to cause the apparatus to perform the method of claim 32.

34. A method for performing wavelet transform on an input signal corresponding to a grid, comprising: The input signal is segmented into lower-resolution samples and higher-resolution samples; For each sample of the predicted signal, a predicted signal is generated based on at least three lower-resolution samples; as well as The predicted signal is subtracted from the higher resolution sample to generate the wavelet coefficient signal.

35. An apparatus comprising: processor; as well as A non-transitory computer-readable medium storing instructions that, when executed by a processor, are operable to cause the apparatus to perform the method of claim 34.

36. An apparatus comprising at least one processor configured to perform the method of any one of claims 1-11, 13-24, 26-30, 32 and 34.

37. An apparatus comprising a computer-readable medium storing instructions for causing one or more processors to perform the method of any one of claims 1-11, 13-24, 26-30, 32 and 34.

38. An apparatus comprising at least one processor and at least one non-transitory computer-readable medium storing instructions for causing the at least one processor to perform the method of any one of claims 1-11, 13-24, 26-30, 32 and 34.

39. A signal that transmits a wavelet coefficient signal generated by the method according to any one of claims 1-11, 26-30, 32 and 34.

Images