Point-based attribute transfer for textured meshes

Abstract

In some embodiments, the method includes: obtaining a source point cloud from a source mesh model; obtaining attributes for each of a plurality of source points in the source point cloud based on a source texture map associated with the source mesh model; obtaining a destination point cloud from a destination mesh model; obtaining attributes for each of a plurality of destination points in the destination point cloud based on attributes of one or more source points in the source point cloud; and setting attributes of pixels in the destination texture map based on attributes of points in the destination point cloud.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to European Patent Application No. 22306588.9, filed October 20, 2022, which is incorporated herein by reference in its entirety. [Background technology]

[0002] background

[0002] This disclosure relates to attribute transfer between mesh models. In some proposed techniques for coding dynamic mesh models, such as the MPEG V-Mesh Test Model, the input mesh to be coded is preprocessed into a form that can be compressed with greater compression efficiency, for example, by a decimation process that removes some of the original vertices, and a subdivision process that adds vertices at more evenly distributed positions that can be reconstructed at a lower bit rate than the original mesh. One such process is described in K. Mammou, J. Kim, A. Tourapis, and D. Podborski, "m59281-[V-CG] Apple's Dynamic Mesh Coding CfP Response," Apple Inc., 2022.

[0003]

[0003] Often, an input mesh model to be coded is associated with a corresponding texture map that conveys attributes (e.g., color) of locations on the surface defined by the input mesh. The vertices of the mesh model are associated with information indicating their corresponding locations (called UV coordinates) in the texture map. (The locations of other points on the mesh besides the vertices may be obtained by interpolation.) After the input mesh undergoes preprocessing and / or other processes (e.g., encoding and reconstruction), the original texture map no longer matches the newly created, preprocessed mesh model. To address this, a new texture map is created, with each vertex in the new mesh having a corresponding UV location in the new texture map. It is desirable to add attributes to the new texture map that accurately reflect the attributes of the original texture map. This process of assigning attributes (e.g., pixel values) to the new texture map is called attribute transfer. One example of an attribute transfer technique is the "nearest neighbor" technique, as described in Mammou et al.

[0004] . However, this technique can introduce errors, especially when the mesh geometry is strongly distorted, and does not easily allow filtering to be performed in geometric space. It is therefore desirable to explore alternative attribute transfer techniques that may avoid some or all of the problems that arise using known techniques. Summary of the Invention

[0004] overview

[0004] A method according to some embodiments includes obtaining a source point cloud from a source mesh model, obtaining attributes for each of a plurality of source points in the source point cloud based on a source texture map associated with the source mesh model, obtaining a destination point cloud from a destination mesh model, obtaining attributes for each of a plurality of destination points in the destination point cloud based on attributes of one or more source points of the source point cloud, and setting attributes of pixels in the destination texture map based on the destination point cloud.

[0005] In some embodiments, obtaining the source point cloud includes sampling a plurality of source points on a surface of a source mesh model, hi some embodiments, the source point cloud is obtained from the source mesh model by at least one of grid sampling, surface sampling, or map sampling.

[0006]

[0006] In some embodiments, obtaining attributes for a source point in the source point cloud includes determining UV coordinates in a source texture map corresponding to the 3D position of the source point in the source point cloud, and using the attributes at the UV coordinates in the source texture map as attributes for the source point in the source point cloud.

[0007] In some embodiments, obtaining the destination point cloud includes sampling a plurality of destination points on a surface of the destination mesh model, hi some embodiments, the destination point cloud is obtained from the destination mesh model by at least one of grid sampling, surface sampling, or map sampling.

[0008]

[0008] In some embodiments, obtaining the destination point cloud includes, for each of a plurality of pixels in the destination texture map, determining UV coordinates of each pixel in the destination texture map, and creating a destination point in the destination point cloud, the created point having a 3D position corresponding to the UV coordinates of each pixel.

[0009] In some embodiments, obtaining attributes for the destination point in the destination point cloud includes selecting at least one source point based on a location of the destination point and setting attributes of the destination point based on attributes of the at least one source point. In some such embodiments, a weighted sum of the attributes of the selected source points is used as the attributes of the destination point.

[0010] In some embodiments, setting attributes of a pixel in the destination texture map includes selecting at least one of the destination points based on a location of the pixel, and setting attributes of the pixel based on attributes of the selected destination point. In some such embodiments, a weighted sum of the attributes of the selected destination points is used as the attribute of the pixel.

[0011] Some embodiments are implemented in a mesh encoder, in which the source mesh model is an input mesh, the source texture map is an input texture map, the source mesh model is encoded into the bitstream as a static mesh and a set of displacements, a destination mesh model is reconstructed from the static mesh and the set of displacements, and the destination texture map is also encoded into the bitstream.

[0012] In some embodiments, the attribute includes at least one color component.

[0013] An apparatus according to some embodiments comprises one or more processors configured to perform any of the methods disclosed herein.

[0014]

[0014] A computer-readable medium (which may be a non-transitory storage medium) according to some embodiments includes instructions for causing one or more processors to perform any of the methods described herein.

[0015]

[0015] A computer program product according to some embodiments includes instructions that, when executed by one or more processors, cause the one or more processors to perform any of the methods described herein.

[0016]

[0016] Some embodiments include a computer-readable medium that stores a mesh that has been encoded using any of the methods described herein. [Brief explanation of the drawings]

[0017] BRIEF DESCRIPTION OF THE DRAWINGS [Figure 1A] FIG. 1 is a system diagram illustrating an example communication system in which one or more disclosed embodiments may be implemented. [Figure 1B]

[0018] 1B is a system diagram illustrating an exemplary wireless transmit / receive unit (WTRU) that may be used within the communication system shown in FIG. 1A, according to one embodiment. [Figure 1C]

[0019] FIG. 1 is a functional block diagram of a system used in some embodiments described herein. [Figure 2A]

[0020] FIG. 1 is a functional block diagram of a block-based video encoder such as the encoder used in VVC. [Figure 2B]

[0021] FIG. 1 is a functional block diagram of a block-based video decoder such as the decoder used in VVC. [Figure 3]

[0022] FIG. 1 is a functional block diagram of a dynamic mesh encoder based on the MPEG V-Mesh Test Model. [Figure 4]

[0023] 1 illustrates a schematic example of attribute transfer from a textured mesh A to a remeshed and reparameterized mesh B. [Figure 5]

[0024] 1 provides an overview of an attribute transfer method according to some embodiments. [Figure 6A]

[0025] 1 illustrates a schematic example of a mesh sampling method used in some embodiments. [Figure 6B]

[0025] Figure 1 illustrates a schematic example of a mesh sampling method used in some embodiments. [Figure 6C]

[0025] Figure 1 illustrates a schematic example of a mesh sampling method used in some embodiments. [Figure 7]

[0026] A summary of bit distortion (BD) rates obtained from experiments comparing attribute transfer techniques and parameters is presented. [Figure 8]

[0027] FIG. 1 is a schematic flow diagram illustrating a method according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0018] Exemplary Network for Implementation of the Embodiments

[0028] 1A illustrates an exemplary communication system 100 in which one or more disclosed embodiments may be implemented. The communication system 100 may be a multiple-access system that provides content, such as voice, data, video, messaging, and broadcasts, to multiple wireless users. The communication system 100 may enable multiple wireless users to access such content through the sharing of system resources, including wireless bandwidth. For example, the communication system 100 may employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), zero-tailed unique word DFT spread OFDM (ZT UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block filtered OFDM, filter bank multicarrier, etc.

[0019]

[0029] 1A, communications system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104, CN 106, public switched telephone network (PSTN) 108, Internet 110, and other networks 112, although it should be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, 102d (any of which may be referred to as a “station” and / or “STA”) may be configured to transmit and / or receive wireless signals and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscription base units, pagers, mobile phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearables, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated processing chain contexts), consumer electronics devices, devices operating in commercial and / or industrial wireless networks, etc. Any of the WTRUs 102a, 102b, 102c, and 102d may be referred to interchangeably as UEs.

[0020]

[0030] The communications system 100 may also include a base station 114a and / or a base station 114b. Each of the base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communications networks, such as the CN 106, the Internet 110, and / or other networks 112. By way of example, the base stations 114a, 114b may be a base transceiver station (BTS), a Node B, an eNode B, a Home Node B, a Home eNode B, a gNB, an NR Node B, a site controller, an access point (AP), a wireless router, etc. Although the base stations 114a, 114b are each depicted as a single element, it should be understood that the base stations 114a, 114b may include any number of interconnected base stations and / or network elements.

[0021]

[0031] The base station 114a may be part of the RAN 104, which may also include other base stations and / or network elements (not shown) (e.g., a base station controller (BSC), a radio network controller (RNC), relay nodes, etc.). The base station 114a and / or base station 114b may be configured to transmit and / or receive wireless signals on one or more carrier frequencies, which may be referred to as a cell (not shown). These frequencies may be in a licensed band, an unlicensed band, or a combination of licensed and unlicensed bands. A cell may provide wireless service coverage for a particular geographic area, which may be relatively fixed or may change over time. A cell may be further divided into cell sectors. For example, the cell associated with the base station 114a may be divided into three sectors. Thus, in one embodiment, the base station 114a may include three transceivers, i.e., one transceiver for each sector of the cell. In one embodiment, the base station 114a may employ multiple-input multiple-output (MIMO) technology and utilize multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in desired spatial directions.

[0022]

[0032] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0023]

[0033] More specifically, as noted above, the communications system 100 may be a multiple-access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, the base station 114a and the WTRUs 102a, 102b, 102c in the RAN 104 may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 116 using Wideband CDMA (WCDMA). WCDMA may include communications protocols such as High Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High Speed ​​UL Packet Access (HSUPA).

[0024]

[0034] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 116 using Long Term Evolution (LTE), and / or LTE-Advanced (LTE-A), and / or LTE-Advanced Pro (LTE-A Pro).

[0025]

[0035] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as New Radio (NR) radio access, which may establish the air interface 116 using NR.

[0026]

[0036] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may jointly implement LTE radio access and NR radio access, e.g., using a dual connectivity (DC) principle. Thus, the radio interface utilized by the WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions sent to and from multiple types of base stations (e.g., eNBs and gNBs).

[0027]

[0037] In other embodiments, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as IEEE 802.11 (i.e., Wireless Fidelity, WiFi), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access, WiMAX), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), or the like.

[0028]

[0038] 1A may be, for example, a wireless router, a Home Node B, a Home eNode B, or an access point, and may utilize any suitable RAT to facilitate wireless connectivity in a local area, such as a business, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a road, etc. In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.11 to establish a wireless local area network (WLAN). In an embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, the base station 114b and the WTRUs 102c, 102d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish a picocell or femtocell. 1A, the base station 114b may have a direct connection to the Internet 110. Thus, the base station 114b may not need to access the Internet 110 through the CN 106.

[0029]

[0039] The RAN 104 may communicate with the CN 106, which may be any type of network configured to provide voice, data, application, and / or voice over internet protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have various quality of service (QoS) requirements, such as different throughput, latency, error tolerance, reliability, data throughput, mobility, etc. The CN 106 may provide call control, billing services, mobile location-based services, prepaid calling, Internet connectivity, video distribution, etc., and / or perform high-level security functions such as user authentication. Although not shown in FIG. 1A , it should be understood that the RAN 104 and / or CN 106 may communicate directly or indirectly with other RANs that employ the same RAT as the RAN 104 or a different RAT. For example, in addition to being connected to the RAN 104, which may be utilizing NR radio technology, the CN 106 may also communicate with another RAN (not shown) that employs GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technology.

[0030]

[0040] The CN 106 may also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and / or other networks 112. The PSTN 108 may include a circuit-switched telephone network providing plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and / or Internet Protocol (IP) in the TCP / IP Internet protocol suite. The networks 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, the network 112 may include another CN connected to one or more RANs, which may employ the same RAT as the RAN 104 or a different RAT.

[0031]

[0041] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communications system 100 may include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks over different wireless links). For example, the WTRU 102c shown in FIG. 1A may be configured to communicate with a base station 114a, which may employ a cellular-based wireless technology, and a base station 114b, which may employ an IEEE 802.11 wireless technology.

[0032]

[0042] 1B is a system diagram illustrating an example WTRU 102. As shown in FIG. 1B, the WTRU 102 may include, among other things, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and / or other peripherals 138. It should be understood that the WTRU 102 may include any sub-combination of the above elements while remaining consistent with an embodiment.

[0033]

[0043] The processor 118 may be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor 118 may perform signal coding, data processing, power control, input / output processing, and / or any other function that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit / receive element 122. While FIG. 1B depicts the processor 118 and the transceiver 120 as separate components, it should be understood that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.

[0034]

[0044] The transmit / receive element 122 may be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) via the air interface 116. For example, in one embodiment, the transmit / receive element 122 may be an antenna configured to transmit and / or receive RF signals. In one embodiment, the transmit / receive element 122 may be an emitter / detector configured to transmit and / or receive IR, UV, or visible light signals, for example. In yet another embodiment, the transmit / receive element 122 may be configured to transmit and / or receive both RF and light signals. It should be understood that the transmit / receive element 122 may be configured to transmit and / or receive any combination of wireless signals.

[0035]

[0045] 1B, the transmit / receive element 122 is shown as a single element, the WTRU 102 may include any number of transmit / receive elements 122. More specifically, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.

[0036]

[0046] The transceiver 120 may be configured to modulate signals transmitted by the transmit / receive element 122 and demodulate signals received by the transmit / receive element 122. As mentioned above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers to enable the WTRU 102 to communicate via multiple RATs, such as NR and IEEE 802.11.

[0037]

[0047] The processor 118 of the WTRU 102 may be coupled to and may receive user input data from a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light emitting diode (OLED) display unit). The processor 118 may also output user data to the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128. Additionally, the processor 118 may access information from and store data in any type of suitable memory, such as non-removable memory 130 and / or removable memory 132. The non-removable memory 130 may include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, etc. In other embodiments, the processor 118 may access information from and store data in memory that is not physically located on the WTRU 102, such as on a server or home computer (not shown).

[0038]

[0048] The processor 118 may receive power from the power source 134 and may be configured to provide power to and / or control other components within the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0039]

[0049] The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to, or instead of, information from the GPS chipset 136, the WTRU 102 may receive location information from a base station (e.g., base stations 114a, 114b) via the air interface 116 and / or may determine its location based on the timing of signals received from two or more nearby base stations. It will be appreciated that the WTRU 102 may obtain location information by way of any suitable location-determination method while remaining consistent with an embodiment.

[0040]

[0050] The processor 118 may further be coupled to other peripherals 138, which may include one or more software and / or hardware modules that provide additional features, functionality, and / or wired or wireless connectivity. For example, the peripherals 138 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photos and / or videos), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, a Bluetooth module, a frequency modulation (FM) radio unit, a digital music player, a media player, a video game player module, an internet browser, a virtual reality and / or augmented reality (VR / AR) device, an activity tracker, etc. The peripherals 138 may include one or more sensors, which may include one or more of a gyroscope, an accelerometer, a Hall effect sensor, a magnetometer, a direction sensor, a proximity sensor, a temperature sensor, a time sensor, a geolocation sensor, an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, and / or a humidity sensor.

[0041]

[0051] The WTRU 102 may include a full-duplex radio over which transmission and reception of some or all of the signals associated with a particular subframe (e.g., for both UL (e.g., for transmission) and downlink (e.g., for reception)) may be performed in parallel and / or simultaneously. The full-duplex radio may include an interference management unit to reduce and or substantially eliminate self-interference through either hardware (e.g., a choke) or processor (e.g., via a separate processor (not shown) or processor 118) signal processing. In one embodiment, the WTRU 102 may include a half-duplex radio over which transmission and reception of some or all of the signals (e.g., associated with a particular subframe for either UL (e.g., for transmission) or downlink (e.g., for reception)) may be performed.

[0042]

[0052] Although the WTRUs are described in FIGS. 1A-1B as wireless terminals, it is contemplated that in certain representative embodiments such terminals may use a wired communication interface (e.g., temporary or permanent) with a communication network.

[0043]

[0053] In a representative embodiment, the other network 112 may be a WLAN.

[0044]

[0054] 1A-1B and the corresponding description, one or more or all of the functions described herein may be performed by one or more emulation devices (not shown). An emulation device may be one or more devices configured to emulate one or more or all of the functions described herein. For example, the emulation device may be used to test other devices and / or to simulate network and / or WTRU functions.

[0045]

[0055] The emulation devices may be designed to perform one or more tests of other devices in a laboratory environment and / or an operator network environment. For example, one or more emulation devices may perform one or more or all functions while fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices in the communication network. One or more emulation devices may perform one or more or all functions while temporarily implemented / deployed as part of a wired and / or wireless communication network. The emulation devices may be directly coupled to another device for testing purposes and / or may perform testing using over-the-air wireless communications.

[0046]

[0056] One or more emulation devices may perform one or more functions, including all functions, without being implemented / deployed as part of a wired and / or wireless communication network. For example, the emulation devices may be utilized in a test lab and / or in a test scenario in an undeployed (e.g., test) wired and / or wireless communication network to perform testing of one or more components. One or more emulation devices may be test equipment. Direct RF coupling and / or wireless communication via RF circuitry (which may include, e.g., one or more antennas) may be used by the emulation devices to transmit and / or receive data.

[0047] Exemplary System.

[0057] The embodiments described herein are not limited to being implemented on a WTRU. Such embodiments may also be implemented using other systems, such as the system of FIG. 1C. FIG. 1C is a block diagram of an example system in which various aspects and embodiments are implemented. System 1000 may be embodied as a device comprising various components described below and configured to perform one or more of the aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of system 1000, singly or in combination, may be embodied in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing elements and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or discrete components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices, for example, via a communication bus or via dedicated input and / or output ports. In various embodiments, the system 1000 is configured to implement one or more of the aspects described herein.

[0048]

[0058] The system 1000 includes at least one processor 1010 configured to execute loaded instructions, for example, to implement various aspects described herein. The processor 1010 may include embedded memory, input / output interfaces, and various other circuits known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 1000 includes a storage device 1040, which may include non-volatile memory and / or volatile memory, including, but not limited to, electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash, magnetic disk drives, and / or optical drives. The storage device 1040 may include, by way of non-limiting example, an internal storage device, an attached storage device (including removable and non-removable storage devices), and / or a network-accessible storage device.

[0049]

[0059] System 1000 includes, for example, an encoder / decoder module 1030 configured to process data to provide encoded or decoded video, which may include its own processor and memory. Encoder / decoder module 1030 represents a module or modules that may be included in a device for performing encoding and / or decoding functions. As is known, a device may include one or both of an encoding module and a decoding module. Additionally, encoder / decoder module 1030 may be implemented as a separate element of system 1000 or may be incorporated within processor 1010 as a combination of hardware and software, as known to those skilled in the art.

[0050]

[0060] Program code loaded onto the processor 1010 or the encoder / decoder 1030 to perform various aspects described herein may be stored in the storage device 1040 and then loaded onto the memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of various items during the execution of the processes described herein. Such stored items include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and arithmetic logic.

[0051]

[0061] In some embodiments, memory internal to the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide working memory for processing necessary during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be either the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory may be the memory 1020 and / or the storage device 1040, e.g., dynamic volatile memory and / or non-volatile flash memory. In some embodiments, the external non-volatile flash memory is used to store, for example, the television's operating system. In at least one aspect, a high-speed external dynamic volatile memory such as RAM is used as working memory for video coding and decoding operations, such as for MPEG-2 (MPEG refers to the Moving Picture Experts Group; MPEG-2 is also known as ISO / IEC 13818, 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a new standard being developed by the Joint Video Experts Team (JVET)).

[0052]

[0062] Input to the elements of system 1000 may be provided through various input devices, as shown in block 1130. Such input devices may include, but are not limited to, (i) a radio frequency (RF) section that receives, for example, RF signals broadcast by a broadcast station, (ii) a component (COMP) input terminal (or set of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high definition multimedia interface (HDMI) input terminal. Other examples, not shown in FIG. 1C, include composite video.

[0053]

[0063] In various embodiments, the input devices of block 1130 have associated respective input processing elements, as is known in the art. For example, the RF section may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or band-limiting a signal to a band of frequencies), (ii) downconverting the selected signal, (iii) band-limiting again to a narrower frequency band to select a signal frequency band, which in particular examples may be referred to as a channel (for example), (iv) demodulating the downconverted, band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements for performing these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a downconverter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner that performs various of these functions, including, for example, downconverting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency near baseband) or to baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted over a wired (e.g., cable) medium and perform frequency selection by filtering, downconverting, and filtering again to a desired frequency band. Various embodiments rearrange the order of the above (and other) elements, remove some of these elements, and / or add other elements to perform similar or different functions. Adding elements may include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0054]

[0064] Additionally, the USB and / or HDMI terminals may include respective interface processors for connecting system 1000 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of the input processing, e.g., Reed-Solomon error correction, may be implemented, for example, in a separate input processing IC or within processor 1010, as desired. Similarly, aspects of the USB or HDMI interface processing may be implemented, as desired, in a separate interface IC or within processor 1010. The demodulated, error corrected, and demultiplexed streams are provided to various processing elements, including, for example, an encoder / decoder 1030 operating in combination with processor 1010 and memory and storage elements, to process the data streams for presentation to output devices, as desired.

[0055]

[0065] The various elements of system 1000 may be provided within a unified housing in which the various elements may be interconnected and data may be transmitted between them using suitable connection arrangements 1140, such as internal buses known in the art, including the Inter-IC (I2C) bus, wiring, and printed circuit boards.

[0056]

[0066] The system 1000 includes a communication interface 1050 that enables communication with other devices over a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data over the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented in a wired and / or wireless medium, for example.

[0057]

[0067] In various embodiments, data is streamed or otherwise provided to system 1000 using a wireless network such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). The Wi-Fi signal in these embodiments is received via communication channel 1060 and communication interface 1050 adapted for Wi-Fi communication. Communication channel 1060 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to enable streaming applications and other over-the-top communications. Other embodiments provide streamed data to system 1000 using a set-top box that delivers data via an HDMI connection in input block 1130. Still other embodiments provide streamed data to system 1000 using an RF connection in input block 1130. As noted above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as a cellular network or a Bluetooth network.

[0058]

[0068] System 1000 may provide output signals to various output devices, including a display 1100, speakers 1110, and other peripheral devices 1120. Display 1100 in various embodiments includes, for example, one or more of a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. Display 1100 may be for a television, a tablet, a laptop, a mobile phone, or other device. Display 1100 may also be integrated with other components (e.g., as in a smartphone) or separate (e.g., an external monitor for a laptop). Other peripheral devices 1120, in various example embodiments, include one or more of a standalone digital video disc (or digital versatile disc) (both terms DVD), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functionality based on the output of system 1000. For example, a disc player performs the function of playing the output of system 1000.

[0059]

[0069] In various embodiments, control signals are communicated between system 1000 and display 1100, speakers 1110, or other peripheral devices 1120 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that allow control between devices with or without user intervention. Output devices may be communicatively coupled to system 1000 via dedicated connections via respective interfaces 1070, 1080, and 1090. Alternatively, output devices may be connected to system 1000 using communication channel 1060 via communication interface 1050. Display 1100 and speakers 1110 may be integrated into a single unit with other components of system 1000 within an electronic device such as a television. In various embodiments, display interface 1070 includes a display driver, such as a timing controller (T Con) chip.

[0060]

[0070] Alternatively, the display 1100 and speakers 1110 may be separate from one or more of the other components, for example, if the RF portion of the input 1130 is part of a separate set-top box. In various embodiments in which the display 1100 and speakers 1110 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0061]

[0071] The embodiments may be performed by computer software implemented by the processor 1010 or other hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 1020 may be of any type suitable for the technology environment and may be implemented using any suitable data storage technology, such as, by way of non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 1010 may be of any type suitable for the technology environment and may include, by way of non-limiting examples, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0062] Detailed Description Dynamic mesh coding.

[0072] 3 is a schematic block diagram of a mesh encoding process that may be employed in some embodiments. A source mesh model 302 is provided as an input mesh M(i) to the mesh encoding process. The source mesh model 302 is associated with a source texture map 304, which is evidenced as the input texture map A(i) to the encoding process. The input mesh is decimated at 306 to generate a base mesh m(i) with a reduced number of vertices, and a UV atlas is generated for the base mesh at 308. The base mesh is quantized at 310 and encoded at 312, and the compressed base mesh data is multiplexed into a dynamic mesh bitstream at 314. The compressed base mesh data is reconstructed in the encoder to generate a reconstructed base mesh m′(i) using a static mesh decoder 316. The reconstructed base mesh is subdivided at 318 by adding new vertices. At 320, a subdivision surface fitting process is performed by comparing the subdivided base mesh with the input mesh M(i) to determine a set of displacements d(i) that deform the vertices of the subdivided base mesh to more closely correspond to the surface defined by the input mesh M(i). At 322, these displacements may be updated to updated displacements d′(i) based on the difference between the original base mesh m(i) and the reconstructed base mesh m′(i). These updated displacements are encoded using a wavelet transform 324, which produces wavelet coefficients e′(i), which are quantized at 326 and packed into an image format at 328. At 330, a time-varying series of images representing the wavelet coefficients may be encoded using conventional video encoding techniques, and at 314, the encoded video may be multiplexed with data representing the compressed base mesh. At the encoder, the displacements are reconstructed from the encoded video through image unpacking 328, inverse quantization 330, and inverse wavelet transform 332 to produce a set of reconstructed displacements d''(i).At 334, a reconstructed base mesh M''(i) is obtained by dequantizing the reconstructed quantized base mesh m'(i), and at 336, the reconstructed base mesh M''(i) is subdivided. At 338, a reconstructed deformed mesh DM(i) is generated by applying the reconstructed set of displacements d''(i) to the reconstructed base mesh m'(i). The reconstructed deformed mesh DM(i) is used as a destination mesh model 340 for attribute transfer purposes.

[0063]

[0073] Using the reconstructed deformed mesh DM(i) (destination mesh model 340), the input mesh M(i) (source mesh model 302), and the input texture map A(i) (source texture map 304), an attribute transfer process is performed to provide attribute values ​​for the destination texture map A'(i) associated with the reconstructed deformed mesh DM(i). Pixels in the texture map A'(i) that are not associated with any triangle of the reconstructed deformed mesh DM(i) may be filled using a padding process 342. A color space conversion 344 may be performed, and the time-varying sequence of texture maps A'(i) may be encoded using conventional video encoding techniques 346, and at 314 the encoded video may be multiplexed with data representing the displaced and compressed base mesh into a bitstream 350. Patch information 348 may also be multiplexed into the bitstream.

[0064] Block-based video coding.

[0074] As noted above, the systems and methods disclosed herein may be used to code a textured mesh, which may be a dynamic textured mesh. In some embodiments, information representing the displacement of the dynamic mesh and / or information representing attributes of the mesh (e.g., texture information) may be coded using known video coding techniques. An overview of block-based video coding techniques that may be used in some embodiments is provided below.

[0065]

[0075] Among other things, the video coding standards HEVC and VVC are based on a block-based hybrid video coding framework. Figure 2A is a block diagram of a block-based hybrid video coding system 200. While variations of this encoder 200 are contemplated, encoder 200 is described below for clarity and does not describe all anticipated variations.

[0066]

[0076] Before being encoded, the video sequence may undergo pre-encoding processing (204), such as applying a color transformation to the input color picture (e.g., from RGB 4:4:4 to YCbCr 4:2:0) or performing a remapping of the input picture components to obtain a signal distribution that is more resistant to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and added to the bitstream.

[0067]

[0077] An input video signal 202 containing a picture to be coded is divided (206) into blocks, for example, by CUs, and processed. Different CUs may have different sizes. In VTM-1.0, a CU can be up to 128 x 128 pixels. However, unlike HEVC, which divides blocks based only on a quadtree, VTM-1.0 divides coding tree units (CTUs) into CUs based on a quadtree / binary / ternary tree to adapt to various local characteristics. In addition, the concept of multiple division unit types in HEVC has been removed. The separation of CUs, prediction units (PUs), and transform units (TUs) no longer exists in VVC-1.0. Instead, each CU is always used as the basic unit for both prediction and transformation without further division. In the multi-type tree structure, CTUs are first divided by a quadtree structure. Then, the leaf nodes of each quadtree can be further divided by a binary tree structure and a ternary tree structure. Different splitting types may be used, such as quadtree splitting, vertical binary tree splitting, horizontal binary tree splitting, vertical ternary tree splitting, and horizontal ternary tree splitting.

[0068]

[0078] The encoder of FIG. 2A may perform spatial prediction (208) and / or temporal prediction (210). Spatial prediction (or "intra-prediction") predicts a current video block using pixels from samples (called reference samples) of previously coded neighboring blocks within the same video picture / slice. Spatial prediction reduces spatial redundancy inherent in video signals. Temporal prediction (also called "inter-prediction" or "motion-compensated prediction") predicts a current video block using pixels reconstructed from previously coded video pictures. Temporal prediction reduces temporal redundancy inherent in video signals. The temporal prediction signal for a given CU may be signaled by one or more motion vectors (MVs), which indicate the amount and direction of motion between the current CU and its temporal reference. If multiple reference pictures are supported, a reference picture index may also be transmitted, which is used to identify which reference picture in the reference picture store (212) the temporal prediction signal comes from.

[0069]

[0079] A mode decision block (214) within the encoder selects the best prediction mode based, for example, on a rate-distortion optimization method. This selection may be made after spatial and / or temporal prediction has been performed. The intra / inter decision may be indicated, for example, by a prediction mode flag. The prediction block is subtracted from the current video block (216) to generate a prediction residual. The prediction residual is decorrelated using a transform (218) and quantization (220). (For some blocks, the encoder may bypass both the transform and quantization, in which case the residual may be coded directly without applying a transform or quantization process.) The quantized residual coefficients are inverse quantized (222) and inverse transformed (224) to form a reconstructed residual, which is then added back to the prediction block (226) to form the reconstructed signal for the CU. Further in-loop filtering, such as deblocking / SAO (Sample Adaptive Offset) filtering, may be applied to the reconstructed CUs (228) to reduce coding artifacts before they are placed in the reference picture store (212) and used to code future video blocks. To form the output video bitstream 230, the coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to the entropy coding unit (108) for further compression and packing to form the bitstream.

[0070]

[0080] Figure 2B shows a block diagram of a block-based video decoder 250. In the decoder 250, the bitstream is decoded by decoder elements as described below. The video decoder 250 generally performs a decoding pass that is the inverse of the encoding pass as shown in Figure 2A. The encoder 200 also generally performs video decoding as part of encoding the video data.

[0071]

[0081] In particular, the decoder input includes a video bitstream 252, which may be generated by the video encoder 200. The video bitstream 252 is first unpacked and entropy decoded in an entropy decoding unit 254 to obtain transform coefficients, motion vectors, and other coding information. Picture partition information indicates how the picture is partitioned. Thus, the decoder may partition the picture according to the decoded picture partition information (256). The coding mode and prediction information are sent to either a spatial prediction unit 258 (if intra-coded) or a temporal prediction unit 260 (if inter-coded) to form a prediction block. The residual transform coefficients are sent to an inverse quantization unit 262 and an inverse transform unit 264 to reconstruct a residual block. The prediction block and the residual block are then added together at 266 to generate a reconstructed block. The reconstructed block may further undergo in-loop filtering 268 before being stored in a reference picture store 270 for use in predicting future video blocks.

[0072]

[0082] The decoded picture 272 may further undergo post-decoding processing (274), such as an inverse color transform (e.g., YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping, which performs the inverse of the remapping process performed in pre-decoding processing (204). Post-decoding processing may use metadata obtained in pre-encoding processing and signaled in the bitstream. The decoded and processed video may be sent to a display device 276. The display device 276 may be a separate device from the decoder 250, or the decoder 250 and display device 276 may be components of the same device.

[0073]

[0083] Various methods and other aspects described in this disclosure may be used to modify modules of video encoder 200 or decoder 250. Furthermore, the systems and methods disclosed herein are not limited to VVC or HEVC, but may, for example, be applied to other standards and recommendations, whether existing or developed in the future, and extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically precluded, aspects described in this disclosure may be used individually or in combination.

[0074] Overview of exemplary embodiments.

[0084] The MPEG V-Mesh Test Model for exploring dynamic mesh coding is based on the techniques described in K. Mammou, J. Kim, A. Tourapis and D. Podborski, "m59281-[V-CG] Apple's Dynamic Mesh Coding CfP Response," Apple Inc, 2022. An overview of the intra-frame encoding process for the test model is provided in the flowchart of FIG. 3, which shows intra-frame encoding of frame i. In the illustrated technique, one operation performed by the encoder is called attribute transfer. This operation involves "reprojecting" the original color attributes stored in the texture map of the input mesh onto a new texture map associated with the mesh that has been distorted by the compression of the geometry.

[0075]

[0085] From a general perspective, textured mesh attribute transfer involves transferring color attributes from a mesh model A to another mesh model B. This attribute transfer can be used in the context of mesh compression, but is not limited to use in mesh compression and can also be used in any other context. Model A and model B are parameterized with their own UV atlas, and model A has an associated texture map MA containing color attributes that are mapped to its 3D surface using UV coordinates. Model B has a different parameterization, and its associated texture MB is initially created empty (each pixel is set to black) and then filled by the attribute transfer algorithm.

[0076]

[0086] The technique of Mammou et al. uses a nearest neighbor method for attribute transfer. For each pixel in texture map MB covered by the parameterized triangle of mesh B, a 3D point PB is created. Then, the nearest neighbor PA on the surface of model A is selected, and its color is used to fill the pixel in texture map MP based on point PB. For each triangle in model A, the nearest neighbor distance is calculated using the 3D distance between the point and the plane, while keeping point PA as the shortest distance. This technique performs an optimization to speed up the distance calculation process by simply attaching the triangle of model A to the vertex VA of model A that is closest to point PB. A kd-tree is used to accelerate the search for the nearest vertex VA. Finally, when extracting colors from the source texture, bilinear filtering of pixels is optionally used, or only the nearest pixel is used. Although this method is quite effective, it has some drawbacks. For example, due to the blurring caused by strong geometric distortion, some errors may occur when calculating the nearest geometry, leading to incorrect colors being obtained. Also, this method only allows filtering in source image space (no filtering in geometric space since only one sample / point can be found on the source model), which can also introduce distortions.

[0077]

[0087] FIG. 4 schematically illustrates an example of attribute transfer from a textured mesh A to a remeshed and reparameterized mesh B. For each pixel MB(i,j) in the destination attribute map, a corresponding point PB(u,v) in the destination texture UV space is determined 402. At 404, the point PB(u,v) is mapped to a corresponding point PB(x,y,z) in the destination 3D space. At 406, a point PA(x',y',z') is determined, where PA(x',y',z') is the point on the source mesh that is closest to point PB(x,y,z). At 408, the point PA(x',y',z') is mapped to a corresponding point PA(u',v') in the source texture UV space. At 410, the point PA(u',v') is then mapped to a coordinate MA(i',j') in the source attribute map. From this process, it may be determined that point MA(i',j') in the source attribute map corresponds to point MB(i,j) in the destination map. Based on this correspondence, the attribute value at point MA(i',j') in the source attribute map may be transferred to the pixel at point MB(i,j) in the destination attribute map at 412. The process may be repeated for each pixel MB(i,j) in the destination attribute map (414, 416).

[0078]

[0088] Some attribute transfer techniques for 3D point clouds exist and operate in 3D space, such as the technique described in U.S. Patent Application Publication No. 2019 / 0311502. In these techniques, source color samples have associated 3D coordinates (point locations). The locations can then be used to perform more advanced filtering.

[0079]

[0089] In this disclosure, an attribute transfer method using point-based spatial methods is proposed. The disclosed method may be used as an alternative to the attribute transfer method described in Mammou et al., or may be used in other contexts. The attribute transfer method described herein may improve the quality of color transfer for textured mesh models in one or more of the following ways: Improved colormetric PSNR after applying texture MB on distorted mesh B. -Improved compression of such MB texture maps. · Improved BD rate (e.g. Bjontegaard rate-distortion metric).

[0080]

[0090] In some embodiments, the attribute transfer methods disclosed herein are used to implement an "attribute transfer" block in a mesh coding method, such as the method shown in FIG.

[0081]

[0091] In an exemplary method, the source and destination meshes are sampled into a 3D point cloud (PC) that retains some additional information, namely, the color of the source point cloud (PC) and the UV coordinates of the target point cloud (PC). These models are then injected into a point cloud color transfer algorithm, which transfers color from the source PC to the destination PC. Finally, the color of each point in the destination point cloud is back-projected onto the target texture map using the associated UV coordinates. An overview of an attribute transfer method according to the present disclosure is shown in FIG. 5. In the exemplary method, a source mesh model 502 is obtained (e.g., as described above with respect to source mesh model 302) and a source texture map 504 is obtained (e.g., as described above with respect to source texture map 304). At 506, a sampling process is performed using the source mesh model and the associated source texture map to obtain a source point cloud 508. Sampling involves obtaining the color (or other relevant attribute) for each point in the source point cloud. A destination mesh model 510 is also obtained (e.g., as described above with respect to destination mesh model 340). At 512, a sampling process is performed using the destination mesh model to obtain a parameterized destination point cloud 514. The points of the destination point cloud 514 do not have assigned colors, but do have associated UV coordinates. At 516, attributes are transferred from the colored source point cloud 508 to the destination point cloud 514, resulting in a parameterized colored destination point cloud 518, where each point has both UV and color information. Based on the UV and color information of the points in the destination point cloud 518, a reprojection process 520 is used to obtain a texture map 522. In some embodiments, the reprojection process includes or is combined with a filtering process. While this example uses color as an example of an attribute to be transferred, it should be understood that attributes other than color may be transferred using this method and other methods described herein.

[0082]

[0092] The disclosed methods are not limited to the use of any one particular type of sampling, however, some types of sampling have been found to provide better transfer results or reduced complexity, as discussed below.

[0083]

[0093] In different embodiments, different types of point cloud attribute transfer methods may be used. Some examples are provided below. Finally, pixel color reprojection may be performed with or without additional filtering, depending on the type of target sampling used, as described in more detail below.

[0084] Exemplary sampling methods.

[0094] 6A-6C show schematic examples of mesh sampling methods used in some embodiments. The process of sampling a textured mesh results in a colored point cloud. A variety of different methods can be used, some of which are shown in FIGS. 6A-6C. Some methods, such as grid sampling (FIG. 6A) or surface sampling (FIG. 6B), are performed in 3D space to obtain the positions of points and their UV coordinates.

[0085]

[0095] An example of grid sampling is shown schematically in FIG. 6A. A cube 602 schematically represents one unit cell of a three-dimensional lattice in space. A triangle 604 schematically represents a face of a mesh bounded by vertices V1, V2, and V3. Grid sampling involves determining intersections between the mesh surface and the lines of a 3D grid. In this example, the intersections are points 606a, 606b, 606c, and 606d. These intersections (and any other intersections between the mesh and the grid) are used as points of a point cloud. Each of these points has corresponding UV coordinates, as shown in a corresponding UV map 608. The UV coordinates of each of the sampled points may be determined from the known UV coordinates of vertices V1, V2, and V3 using barycentric coordinates or other techniques. If the mesh being sampled is a source mesh grid associated with a texture map, the UV coordinates of each sampled point are used to obtain color (or other attributes) from the texture map associated with the mesh. If the mesh being sampled is a destination mesh model, the UV coordinates of each sampled point are stored to provide a parameterized destination point cloud (such as 514 above).

[0086]

[0096] Another sampling method used in some embodiments, sometimes called area sampling, is shown schematically in FIG. 6B . (For clarity, unit cell 602 is shown as if it were an opaque box to improve perspective cues.) In area sampling, sample points are distributed evenly across each face (such as face 604). While FIG. 6B shows samples as being taken as corners of a triangular subgrid at the face, other arrangements or numbers of points at the face may alternatively be used. In some embodiments, the number of samples taken at a face depends at least in part on the size of the face, with more samples taken at larger faces. The UV coordinates of each sample point in UV map 610 and the associated color (or other attribute) of the source map may be determined using the techniques described above with respect to FIG. 6A .

[0087]

[0097] In some embodiments, another sampling method, which may be called map sampling, is used, which involves constraining the sampling to correspond to occupied pixels in the texture map. Such a method is illustrated in FIG. 6C . For each triangle 604 in the model, per-vertex UV coordinates are used to determine the projection of the triangle onto the texture map. Then, for each pixel covered by the triangle, a sample is generated associated with the pixel's center and its color. For example, each sample 612a, 612b, 612c, etc. may be selected to correspond to the center of a pixel in UV map 614. Using UV mapping and barycentric coordinates, the 3D location of the sample in the 3D triangle can be determined. The associated color (or other attribute) of the source map is given by the corresponding pixel value in UV map 614. This type of mapping thus generates exactly one point per pixel in the texture that effectively covers the mesh (other pixels are not sampled). The generated sampling may not be as regular as the grid sampling of FIG. 6C ; some areas of the mesh surface may be densely sampled, while other areas may contain fewer samples.

[0088]

[0098] Between grid sampling and map sampling, grid sampling was found to be more regular in 3D model space, while map sampling was found to be more regular in 2D texture space. However, it should be noted that any other sampling method can be used with the attribute transfer methods described herein, and different sampling methods can be used together in the same grid. Two specific examples are discussed below, which may be appropriate depending on the transfer method used. One example uses high-density grid sampling for the source model and the target model. Another example uses high-density grid sampling for the source model and map sampling for the target model.

[0089]

[0099] In one embodiment, where a dense grid is used for source sampling and a dense grid is also used for destination sampling, the grid is regular in 3D space, but not necessarily in texture space. Using this embodiment, holes (unfilled portions of the destination map) may be introduced due to missing points that cover some pixels. Thus, map coverage may require the use of very dense sampling. Any remaining holes may then be filled by a padding step (FIG. 3). However, quality may be reduced due to interpolation from neighbors rather than actual transfer.

[0090]

[0100] In one embodiment, where a dense grid is used for source sampling and map sampling is used for the target, each sample of the target map is covered without any holes because the map is regular in 2D texture space. However, this technique may not be as effective for some transfer methods (e.g., V-PCC method) due to the non-regular 3D sampling of the target model. The map is regularly sampled in texture space but not in 3D space. Using dense grid sampling for the source helps to collect a large amount of data from the input signal for subsequent filtering and transfer.

[0091] Point cloud attribute transfer method.

[0101] The attribute transfer method performs a color transfer from the source point cloud to the target point cloud. Some complex attribute transfer methods can be used, such as those used in V-PCC. Alternatively, other simpler but still efficient 3D filtering methods can be used.

[0092]

[0102] In some embodiments, the point cloud attribute transfer algorithm described in U.S. Patent Application Publication No. 2019 / 0311502 may be used. When calculating the color of a target point, this method not only searches for the closest point in the source point cloud, but also performs additional searching and processing based on the closest point in the target point cloud.

[0093]

[0103] In other embodiments, the attribute transfer method may be based on average, linear, Gaussian, or other filtering. In such embodiments, for each point in the target point cloud, a search is performed for one or more of its closest neighbors in the source point cloud. The determination of which points are closest may be made in 2D UV space or 3D model space. Each point in the target point cloud may then be assigned a color (or other attribute) based on the selected one or more closest neighbors. This assignment may be performed using a variety of different techniques, including one or more of the following: Assigning the color of the nearest point in the source point cloud. Using the average of the colors of the k closest points in the source point cloud (where "nearest" can be determined in UV space or 3D space, or some combination of the two), where k is a predetermined number greater than 1. Using the average of all of the points in the source point cloud within a certain distance (where distance can be determined in UV space or 3D space, or some combination of both). Performing filtering by weighting the colors of the k-nearest or all points (in 2D UV space or 3D model space, or even color space) within a given distance range to a target point using linear or Gaussian or other blending functions based on 2D or 3D distance.

[0094]

[0104] In some embodiments, distance or other metrics in color space may also be used in combination with this filtering to enhance the weighting.

[0095]

[0105] Some embodiments operate in 3D space to avoid problems encountered with current test models. The point-based approach described herein is particularly useful in that it allows filtering in 3D space.

[0096]

[0106] An example of linear (inverse distance weighting) and Gaussian filtering, which will be used later for our experimental results, is given below: In this example, linear filtering is used if the parameter params.textureTransferSigma is zero, otherwise Gaussian filtering is used.

number

[0097]

[0107] The code above shows the weight calculation portion of the code for both the weights and the Gaussian sigma calculation.

[0098]

[0108] In summary, the linear filtering in this example uses the following weights for points whose distance d=kdtreeResult.indices(i) (as determined using a kd-tree search) to the 3D location of the target point: (In this example, offset=4.0 is used to prevent extremely high or infinite weights):

number

[0099]

[0109] And the calculation of the Gaussian weights:

number

number

[0100]

[0110] Each component of the point color is weighted and added to the final color of the target pixel (originally black).

number

[0101]

[0111] Although particular methods for transferring attributes from a source point cloud to a destination point cloud are described herein, it should be understood that the principles described herein are not limited to the use of any particular point cloud attribute transfer technique.

[0102] Pixel color reprojection.

[0112] In some embodiments, once all points of the destination point cloud have their colors extracted from the input point cloud by the selected point cloud color transfer algorithm, a pixel color reprojection process is performed to write the obtained colors of the target point cloud to the target texture map.

[0103]

[0113] Point cloud attribute transfer may encounter situations where two or more 3D points of the target point cloud project onto a single pixel (whose coordinates in image space and UV space are known). In this case, one or more of the following procedures can be followed: Use the color of the closest projected point to the pixel center (in UV or 3D space). Use the average of the colors of the k nearest neighbors of the projected point or all of them. (The experimental results presented below use the average of all points projected to the pixel.) Performing filtering by weighting the color of the k-nearest neighbors of a projected point or all points by their distance to the pixel center in 2D UV space or 3D model space, using a linear or Gaussian or any other blending function based on 2D or 3D distance.

[0104]

[0114] Distance in color space or other metrics may also be used in these filters to enhance the weighting.

[0105]

[0115] If filtering is performed in full 3D space, determining the 3D position associated with each pixel center may be done through rendering (e.g., by rasterization or ray tracing) the triangles of the target mesh in image space and storing the interpolated 3D positions using barycentric and UV coordinates in a floating-point rendering buffer.

[0106] Illustrative results.

[0116] Some embodiments include sampling for color filtering in 3D space, as opposed to color filtering in 2D texture space (e.g., bilinear filtering) for a texture mesh. The use of filtering in 3D space may provide one or more of the following advantages: Improved handling of model and texture transformations for filtering. Preventing artifacts on texture patch boundaries (also known as UV seams) that can occur when bilinear or other large filtering kernels are performed on the edges of texture patches.

[0107]

[0117] The experimental results shown in Fig. 7 were obtained using the MPEG V-Mesh Test Model modified by our method and the corresponding general test conditions. Fig. 7 presents an overview of the bit distortion (BD) rates calculated over N sequences of 30 frames for five target rates and AI and RA conditions. The Grid and Ibsm columns represent the BD rates of given metric features, and the AvgBdRate column shows the average sum of percentages for these features. The rightmost column represents the execution time (Envti / DecTi) and memory consumption (EncME / DecMe). The meaning of each row is as follows:

[0108] [Table 1]

[0109]

[0118] In these tests, pixel color reprojection always uses the average filtering of all points projected onto the target pixel.

[0110]

[0119] Exemplary embodiments enable filtering based on 3D spatial sampling, rather than using less accurate filtering performed in the 2D texture space of the source model. This results in more accurate results. Any spatial filtering can be used (linear, Gaussian, or other more complex methods based on point cloud geometry structure, such as that used in U.S. Patent Application Publication No. 2019 / 0311502). Experimental results show that the embodiments described herein outperform current test models in all cases. Finally, using linear or Gaussian filtering allows for reduced complexity with only a slight loss of quality, but a better overall gain (AvgBdRate), compared to using the method of U.S. Patent Application Publication No. 2019 / 0311502. This allows for faster processing with minimal loss of quality.

[0111]

[0120] Some embodiments may introduce some complexity due to the use of sampling and kd-tree searching in dense point clouds, however, embodiments that use map sampling on the target model reduce this complexity by providing the exact number of points that cover a pixel in the destination texture map.

[0112]

[0121] The exemplary embodiments may be implemented with coding standards such as MPEG V-MESH (V-DMC), or other coding standards, and may also be adapted for attribute transfer for use in other contexts.

[0113]

[0122] In this disclosure, the term "source point" refers to a point in a source point cloud, and the term "destination point" refers to a point in a destination point cloud. In this context, the modifiers "source" and "destination" are used only to identify the constituent point clouds of the respective points, and should not be understood to impose any further limitations on the individual points.

[0114] Further description of exemplary methods and systems.

[0123] As shown in Figure 8, in some embodiments, a method includes obtaining a source point cloud from a source mesh model (902). For each of a plurality of source points in the source point cloud, attributes are obtained based on a source texture map associated with the source mesh model (904). A destination point cloud is obtained from the destination mesh model (906). For each of a plurality of destination points in the destination point cloud, attributes are obtained based on attributes of one or more source points in the source point cloud (908). Attributes of pixels in the destination texture map are set based on the destination point cloud (910).

[0115]

[0124] In some embodiments, the above-described processes 902-910 may be implemented as a stand-alone attribute transfer process 912. In other embodiments, the attribute transfer process 912 is used as part of a process for encoding static or dynamic input mesh data 914. In such embodiments, input mesh geometry data may be obtained (916) for use as a source mesh model, and an input texture map may be obtained (918) for use as a source texture map. The encoder may encode the source mesh model (920), in some embodiments including preprocessing the source mesh model. The encoded mesh model may be provided in a bitstream for storage or delivery to a decoder. The encoded mesh model may also be reconstructed (922) by the encoder, and the reconstructed mesh model may be used as a destination mesh model. The resulting destination texture map produced in process 912 may also be encoded (924) and multiplexed into the bitstream.

[0116]

[0125] A method according to some embodiments includes obtaining a source point cloud from a source mesh model, obtaining attributes for each of a plurality of source points in the source point cloud based on a source texture map associated with the source mesh model, obtaining a destination point cloud from a destination mesh model, obtaining attributes for each of a plurality of destination points in the destination point cloud based on attributes of one or more source points of the source point cloud, and setting attributes of pixels in the destination texture map based on the destination point cloud.

[0117]

[0126] In some embodiments, the source point cloud is obtained from the source mesh model using grid sampling, surface sampling, or map sampling.

[0118]

[0127] In some embodiments, the source point cloud is obtained by, for each of a plurality of pixels in the source texture map, determining the UV coordinates of each pixel in the source texture map (e.g., the center of each pixel), and creating source points in the source point cloud, the created source points having 3D positions corresponding to the UV coordinates of each pixel. In some such embodiments, source points in the source point cloud are created only for occupied pixels of the source texture map.

[0119]

[0128] In some embodiments, the attributes of the created source points in the source point cloud are determined by the attributes of the respective pixels in the source texture map.

[0120]

[0129] In some embodiments, obtaining attributes for a source point in the source point cloud includes determining UV coordinates in a source texture map that correspond to a 3D position of the source point in the source point cloud, and using attributes at the UV coordinates in the source texture map as attributes for the source point in the source point cloud.

[0121]

[0130] In some embodiments, the destination point cloud is obtained from the destination mesh model using grid sampling, surface sampling, or map sampling.

[0122]

[0131] In some embodiments, obtaining the destination point cloud includes, for each of a plurality of pixels in the destination texture map, determining UV coordinates of each pixel in the destination texture map (e.g., coordinates of the center of the pixel), and creating a destination point in the destination point cloud, the created point having a 3D position corresponding to the UV coordinates of each pixel.

[0123]

[0132] In some embodiments, obtaining attributes for a destination point in the destination point cloud includes selecting at least one source point based on a location of the destination point, and setting attributes of the destination point based on attributes of the at least one source point.

[0124]

[0133] In some embodiments, selecting at least one source point based on the location of the destination point includes selecting a predetermined number of source points that are closest to the destination point (e.g., in 3D space or UV space).

[0125]

[0134] In some embodiments, the predetermined number of source points is one, and attributes of the source points are used as attributes of the destination points. In other embodiments, the predetermined number of source points is greater than one.

[0126]

[0135] In some embodiments, selecting at least one source point based on the location of the destination point includes selecting all source points within a predetermined distance (in 3D space or UV space) of the destination point.

[0127]

[0136] In some embodiments, the attributes of the destination point are a weighted sum of the attributes of selected source points. In some such embodiments, the weight of each source point in the weighted sum is determined based on the distance between each source point and the destination point (in 3D or UV space). The weight of each source point in the weighted sum can be, for example, a Gaussian function of distance or an inverse linear function of distance.

[0128]

[0137] In some embodiments, the attributes of the destination point are the average of the attributes of the selected source points.

[0129]

[0138] In some embodiments, setting attributes of a pixel in the destination texture map includes selecting at least one destination point based on a location of the pixel, and setting the attributes based on attributes of the at least one destination point.

[0130]

[0139] In some embodiments, selecting at least one destination point based on the position of the pixel (e.g., the position of the center of the pixel) includes selecting a predetermined number of destination points that are closest to the position of the pixel (in 3D space or UV space).

[0131]

[0140] In some embodiments, the predetermined number of destination points is one and the attributes of the destination points are used as the attributes of the pixels. In other embodiments, the predetermined number of destination points is greater than one.

[0132]

[0141] In some embodiments, selecting at least one destination point based on the pixel's location includes selecting all destination points within a predetermined distance (in 3D space or UV space) of the pixel's location (e.g., center).

[0133]

[0142] In some embodiments, selecting at least one destination point based on the location of the pixel includes selecting all destination points having UV coordinates within a boundary of the pixel.

[0134]

[0143] In some embodiments, the pixel's attributes are a weighted sum of the attributes of the selected destination points.

[0135]

[0144] In some embodiments, the weight of each destination point in the weighted sum is determined based on the distance (in 3D or UV space) between each destination point and the pixel's location (e.g., the pixel's center). The weight of each destination point in the weighted sum may be determined, for example, by a Gaussian function of the distance or an inverse linear function of the distance.

[0136]

[0145] In some embodiments, the pixel's attribute is the average of the attributes of the selected destination points.

[0137]

[0146] In some embodiments, the attribute includes at least one color component.

[0138]

[0147] In some embodiments, the destination mesh model is obtained from the source mesh model by encoding and reconstructing the source mesh model.

[0139]

[0148] Some embodiments further include encoding the destination texture map, for example using video encoding.

[0140]

[0149] Some embodiments include a dynamic mesh encoding method performed using the attribute transfer methods described herein. In some such embodiments, the source mesh model is an input mesh model M(i), the source texture map is an input texture map A(i), and the destination mesh model is a reconstructed deformed mesh DM(i).

[0141]

[0150] An apparatus according to some embodiments comprises one or more processors configured to perform any of the methods disclosed herein.

[0142]

[0151] A computer-readable medium (which may be a non-transitory storage medium) according to some embodiments includes instructions for causing one or more processors to perform any of the methods described herein.

[0143]

[0152] A computer program product according to some embodiments includes instructions that, when executed by one or more processors, cause the one or more processors to perform any of the methods described herein.

[0144]

[0153] This disclosure describes various aspects, including tools, features, embodiments, models, methods, and the like. Many of these aspects are described with specificity, often sounding limiting, to at least illustrate their individual characteristics. However, this is for clarity of description only and does not limit the disclosure or scope of the aspects. In fact, all of the different aspects can be combined and interchanged to provide further aspects. Furthermore, these aspects can also be combined and interchanged with aspects described in prior applications.

[0145]

[0154] The aspects described and contemplated in this disclosure can be implemented in many different forms. While some embodiments are specifically illustrated, other embodiments are contemplated, and discussion of a particular embodiment does not limit the scope of the implementations. At least one of the aspects generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects can be embodied as a method, an apparatus, a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium having stored thereon a bitstream generated according to any of the described methods.

[0146]

[0155] In this disclosure, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image," "picture," and "frame" may be used interchangeably. Typically, but not necessarily, the term "reconstructed" is used on the encoder side, and the term "decoded" is used on the decoder side.

[0147]

[0156] The terms HDR (high dynamic range) and SDR (standard dynamic range) often convey specific values ​​of dynamic range to those skilled in the art. However, further embodiments are contemplated in which reference to HDR is understood to mean "higher dynamic range" and reference to SDR is understood to mean "lower dynamic range." Such further embodiments are not constrained by any specific values ​​of dynamic range that may often be associated with the terms "high dynamic range" and "standard dynamic range."

[0148]

[0157] Various methods are described herein, each of which includes one or more steps or actions for achieving the described method. Unless a specific order of steps or actions is required for proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined. Additionally, terms such as “first,” “second,” etc. may be used in various embodiments to modify elements, components, steps, operations, etc. (e.g., “first decode” and “second decode,” etc.). The use of such terms does not imply any ordering of the modified operations unless specifically required. Thus, in this example, the first decode need not be performed before the second decode, but may occur before, during, or during an overlapping period with the second decode.

[0149]

[0158] For example, various numerical values ​​may be used in this disclosure, and the specific values ​​are for illustrative purposes only and the described aspects are not limited to these specific values.

[0150]

[0159] The embodiments described herein may be performed by computer software implemented by a processor or other hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The processor may be of any type suitable for the technology environment, and may include, as non-limiting examples, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0151]

[0160] Various embodiments include decoding. As used in this disclosure, "decoding" may encompass all or part of the processes performed on a received encoded sequence, e.g., to generate a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder (e.g., entropy decoding, inverse quantization, inverse transform, and differential decoding). In various embodiments, such processes additionally or alternatively include processes performed by the decoders of various embodiments described in this disclosure, e.g., extracting a picture from a tiled (packed) picture, determining an upsampling filter to use and then upsampling the picture, and flipping the picture to its intended orientation.

[0152]

[0161] As a further example, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or to the broader decoding process in general should be clear based on the context of the specific description.

[0153]

[0162] Various embodiments include encoding. Similar to what was discussed above with respect to "decoding," as used in this disclosure, "encoding" may encompass all or part of the processes performed on an input video sequence to, for example, generate an encoded bitstream. In various embodiments, such processes include one or more of the processes typically performed by an encoder (e.g., partitioning, differential encoding, transforming, quantizing, and entropy coding). In various embodiments, such processes additionally or alternatively include processes performed by the encoders of various embodiments described in this disclosure.

[0154]

[0163] As a further example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or to the broader encoding process in general should be clear based on the context of the specific description.

[0155]

[0164] Where a drawing is presented as a flow diagram, it should be understood that the drawing also provides a block diagram of the corresponding apparatus. Similarly, where a drawing is presented as a block diagram, it should be understood that the drawing also provides a flow diagram of the corresponding method / process.

[0156]

[0165] Various embodiments refer to rate-distortion optimization. In particular, during the encoding process, a balance or trade-off between rate and distortion is typically considered, often subject to computational complexity constraints. Rate-distortion optimization is typically formulated to minimize a rate-distortion function, which is a weighted sum of rate and distortion. There are various approaches to solving the rate-distortion optimization problem. For example, an approach may be based on extensive testing of all encoding options, including all considered modes or coding parameter values, along with a thorough evaluation of the coding cost and associated distortion of the reconstructed signal after coding and decoding. To limit encoding complexity, faster approaches may also be used, particularly those using approximate distortion calculations based on prediction or prediction residual signals rather than the reconstructed signal. These two approaches may also be used in combination, such as by using approximate distortion for only some of the possible encoding options and full distortion for others. Other approaches evaluate only a subset of the possible encoding options. More generally, many approaches employ any of a variety of techniques for performing the optimization, but the optimization does not necessarily involve a thorough evaluation of both the coding cost and the associated distortion.

[0157]

[0166] The embodiments and aspects described herein may be embodied in, for example, a method or process, an apparatus, a software program, a data stream, or a signal. Even if discussed with respect to only a single form of embodiment (e.g., discussed only as a method), the discussed feature implementation may also be embodied in other forms (e.g., an apparatus or a program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. A method may be implemented in, for example, a processor, which generally refers to a processing device including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end users.

[0158]

[0167] References to "one embodiment" or "one embodiment" or "one implementation" or "one embodiment," as well as other variations, mean that a particular feature, structure, characteristic, etc. described in connection with that embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in one embodiment" or "in one embodiment" or "in one embodiment" in various places throughout this disclosure, as well as any other variations, are not necessarily all referring to the same embodiment.

[0159]

[0168] Additionally, this disclosure may refer to "determining" various information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from memory.

[0160]

[0169] Additionally, this disclosure may refer to "accessing" various information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from a memory), storing information, transferring information, replicating information, calculating information, determining information, predicting information, or estimating information.

[0161]

[0170] Additionally, this disclosure may refer to "receiving" various information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" typically involves in some way an operation such as, for example, storing information, processing information, transmitting information, moving information, replicating information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0162]

[0171] It should be understood that the use of any of the following terms " / ," "and / or," and "at least one of" is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B), for example, "A / B," "A and / or B," and "at least one of A and B." As a further example, for "A, B, and / or C" and "at least one of A, B, and C," such phrases are intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), or the selection of only the first and third listed alternatives (A and C), or the selection of only the second and third listed alternatives (B and C), or the selection of all three alternatives (A, B, and C). This can be expanded as many times as there are listed items.

[0163]

[0172] Also, as used herein, the term "signal" refers, among other things, to indicating something to a corresponding decoder. For example, in a particular embodiment, an encoder signals a specific one of multiple parameters for region-based filter parameter selection for de-artifact filtering. In this way, in one embodiment, the same parameters are used at both the encoder and decoder sides. Thus, for example, the encoder may transmit a specific parameter to the decoder so that the decoder may use the same specific parameter (explicit signaling). Conversely, if the decoder already has a specific parameter and other parameters, signaling may be used without transmission (implicit signaling) to simply enable the decoder to know and select the specific parameter. By avoiding the transmission of any actual function, bit savings are achieved in various embodiments. It should be understood that signaling may be implemented in various ways. For example, in various embodiments, one or more syntax elements, flags, etc. are used to signal information to a corresponding decoder. Although the above refers to the verb form of the word "signal," the word "signal" may also be used herein as a noun.

[0164]

[0173] Implementations can generate various signals formatted to carry information, such as information that can be stored or transmitted. Information can include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal can be formatted to carry a bitstream of the described embodiments. Such a signal can be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting can include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal can be, for example, analog or digital information. The signal can be transmitted over a variety of different wired or wireless links, as is well known. The signal can be stored on a processor-readable medium.

[0165]

[0174] The inventors have described several embodiments, the features of which may be provided alone or in any combination across various claim categories and types.

[0166]

[0175] Although features and elements are described above in particular combinations, each feature or element can also be used alone or in any combination with the other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware embodied in a computer-readable medium for execution by a computer or processor. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random-access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs). A processor, together with software, may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. obtaining a source point cloud from a source mesh model; For each of a plurality of source points in the source point cloud, obtaining attributes based on a source texture map associated with the source mesh model; obtaining a destination point cloud from the destination mesh model; obtaining attributes for each of a plurality of destination points in the destination point cloud based on the attributes of one or more source points in the source point cloud; setting attributes of pixels in a destination texture map based on the destination point cloud; A method comprising:

2. at least, obtaining a source point cloud from a source mesh model; For each of a plurality of source points in the source point cloud, obtaining attributes based on a source texture map associated with the source mesh model; obtaining a destination point cloud from the destination mesh model; obtaining attributes for each of a plurality of destination points in the destination point cloud based on the attributes of one or more source points in the source point cloud; setting attributes of pixels in a destination texture map based on the destination point cloud; 1. An apparatus comprising: one or more processors configured to execute

3. One or more processors may include at least obtaining a source point cloud from a source mesh model; For each of a plurality of source points in the source point cloud, obtaining attributes based on a source texture map associated with the source mesh model; obtaining a destination point cloud from the destination mesh model; obtaining attributes for each of a plurality of destination points in the destination point cloud based on the attributes of one or more source points in the source point cloud; setting attributes of pixels in a destination texture map based on the destination point cloud; 1. A computer-readable medium containing instructions for executing:

4. 4. The method of claim 1, the apparatus of claim 2, or the computer-readable medium of claim 3, wherein obtaining the source point cloud comprises sampling a plurality of source points on a surface of the source mesh model.

5. The method of claim 1 or claim 4 when dependent on claim 1, the apparatus of claim 2 or claim 4 when dependent on claim 2, or the computer-readable medium of claim 3 or claim 4 when dependent on claim 3, wherein the source point cloud is obtained from the source mesh model by at least one of grid sampling, surface sampling, or map sampling.

6. Obtaining attributes for source points in the source point cloud includes: determining UV coordinates in the source texture map corresponding to 3D positions of the source points in the source point cloud; using attributes in the UV coordinates in the source texture map as the attributes for the source points in the source point cloud; 10. A method according to claim 1 or claim 4 or 5 when dependent on claim 1, an apparatus according to claim 2 or claim 4 or 5 when dependent on claim 2, or a computer-readable medium according to claim 3 or claim 4 or 5 when dependent on claim 3, comprising:

7. The method of claim 1 or any one of claims 4 to 6 when dependent on claim 1, the apparatus of claim 2 or any one of claims 4 to 6 when dependent on claim 2, or the computer-readable medium of claim 3 or any one of claims 4 to 6 when dependent on claim 3, wherein obtaining the destination point cloud comprises sampling a plurality of destination points on a surface of the destination mesh model.

8. 8. The method of claim 1 or any one of claims 4 to 7 when dependent on claim 1, the apparatus of claim 2 or any one of claims 4 to 7 when dependent on claim 2, or the computer-readable medium of claim 3 or any one of claims 4 to 7 when dependent on claim 3, wherein the destination point cloud is obtained from the destination mesh model by at least one of grid sampling, surface sampling, or map sampling.

9. Obtaining the destination point cloud includes, for each of a plurality of pixels in the destination texture map: determining UV coordinates for each pixel in the destination texture map; creating destination points within the destination point cloud, the created points having 3D positions corresponding to the UV coordinates of the respective pixels; A method according to claim 1 or any one of claims 4 to 8 when dependent on claim 1, an apparatus according to claim 2 or any one of claims 4 to 8 when dependent on claim 2, or a computer-readable medium according to claim 3 or any one of claims 4 to 8 when dependent on claim 3, comprising:

10. Obtaining attributes for destination points in the destination point cloud includes: selecting at least one source point based on the location of the destination point; setting attributes of the destination point based on the attributes of the at least one source point; A method according to claim 1 or any one of claims 4 to 9 when dependent on claim 1, an apparatus according to claim 2 or any one of claims 4 to 9 when dependent on claim 2, or a computer-readable medium according to claim 3 or any one of claims 4 to 9 when dependent on claim 3, comprising:

11. Obtaining attributes for destination points in the destination point cloud includes: selecting at least one source point based on the location of the destination point; A method according to claim 1 or any one of claims 4 to 10 when dependent on claim 1, an apparatus according to claim 2 or any one of claims 4 to 10 when dependent on claim 2, or a computer-readable medium according to claim 3 or any one of claims 4 to 10 when dependent on claim 3, wherein a weighted sum of attributes of the selected source points is used as the attribute of the destination point.

12. setting attributes of pixels in the destination point cloud; selecting at least one of the destination points based on a location of the pixel; setting the attributes of the pixel based on the attributes of the selected destination point; A method according to claim 1 or any one of claims 4 to 11 when dependent on claim 1, an apparatus according to claim 2 or any one of claims 4 to 11 when dependent on claim 2, or a computer-readable medium according to claim 3 or any one of claims 4 to 11 when dependent on claim 3, comprising:

13. setting attributes of pixels in the destination texture map; selecting at least one of the destination points based on a location of the pixel; A method according to claim 1 or any one of claims 4 to 12 when dependent on claim 1, an apparatus according to claim 2 or any one of claims 4 to 12 when dependent on claim 2, or a computer-readable medium according to claim 3 or any one of claims 4 to 12 when dependent on claim 3, wherein a weighted sum of attributes of the selected destination points is used as the attribute of the pixel.

14. Implemented in mesh coder, the source mesh model is an input mesh; the source texture map is an input texture map; the source mesh model is encoded into a bitstream as a static mesh and a set of displacements; the destination mesh model is reconstructed from the static mesh and the set of displacements; The method of claim 1 or any one of claims 4 to 13 when dependent on claim 1, the apparatus of claim 2 or any one of claims 4 to 13 when dependent on claim 2, or the computer-readable medium of claim 3 or any one of claims 4 to 13 when dependent on claim 3, wherein the destination texture map is encoded into the bitstream.

15. The method of claim 1 or any one of claims 4 to 14 when dependent on claim 1, the apparatus of claim 2 or any one of claims 4 to 14 when dependent on claim 2, or the computer-readable medium of claim 3 or any one of claims 4 to 14 when dependent on claim 3, wherein the attribute comprises at least one color component.

Images