Value-based updates for vertices in base mesh frames for dynamic mesh coding

Abstract

An apparatus involving improvement on update processing for linear lifting inverse transform is provided. The apparatus receives a compressed bitstream, decodes a base mesh sub-bitstream to reconstruct a base mesh, decodes a displacement sub-bitstream to reconstruct displacement wavelet coefficients, performs update processing using the displacement wavelet coefficients to generate updated displacement wavelet coefficients, performs prediction processing using the updated displacement wavelet coefficients to generate a displacement, and reconstructs a mesh frame based on the displacement. During the update processing, the apparatus determines a first valence indicating a number of connecting edges at a first vertex and a second valence indicating a number of connecting edges at a second vertex based on whether the first vertex and the second vertex forming an edge to which a current vertex belongs belong to the base mesh, and updates displacement wavelet coefficients of the first vertex and the second vertex based on the first valence and the second valence.

Description

Technical Field

[0001] This disclosure relates to dynamic mesh coding, and more specifically, to lifting wavelet transforms for, but not limited to, displacement. Background Technology

[0002] Currently, Working Group 07 of Subcommittee 29 of the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) (ISO / IEC SC29 / WG07) is working on developing a standard for video-based dynamic mesh compression. At the 14th meeting of ISO / IEC SC29 WG07 in January 2024, the seventh test model, V-DMCTMM 7.0, representing the current state of the standard, was established. A draft specification for video-based dynamic mesh compression is also available.

[0003] Based on the seventh test model V-DMC TMM 7.0 and the corresponding working draft WD 6.0 (WD 6.0 for V-DMC, ISO / IEC SC29 WG07 N00822, January 2024), the V-DMC encoder generates multiple sub-bitstreams (such as atlases, base meshes, displacements, and optional attributes). The V-DMC decoder decodes the displacement bitstream and performs an inverse linear lift transform, which includes update processing and prediction processing. In the update processing, the V-DMC decoder can use update weights derived from syntax elements included in the V-DMC bitstream. The V-DMC decoder can use valence-based update weights in the update processing, which results in a BD rate gain.

[0004] However, this method imposes additional computational and memory requirements on the V-DMC decoder. Therefore, a simplified method for valence-based updates is needed that preserves almost all of the BD rate gain while significantly reducing computational and memory requirements.

[0005] The descriptions set forth in the Background section should not be assumed to be prior art simply because they are set forth in the Background section. The Background section may describe aspects or embodiments of this disclosure. Summary of the Invention

[0006] Technical solution This disclosure relates to improvements to dynamic mesh coding, and more specifically, to improvements to the update processing of the linear lifting inverse transform in the test model V-DMC TMM 7.0 and the corresponding working draft WD 6.0.

[0007] According to embodiments of this disclosure, a simplified method for valence-based updates is introduced. This simplified method for valence-based updates preserves almost all of the BD rate gain while significantly reducing computational and memory requirements by using fixed values ​​for new vertices generated by the subdivision process.

[0008] One aspect of this disclosure provides an apparatus. The apparatus includes a communication interface and a processor. The communication interface is configured to receive a compressed bitstream comprising a base mesh sub-bitstream and a displacement sub-bitstream. The processor is operatively coupled to the communication interface. The processor is configured to reconstruct the base mesh. The processor is further configured to perform one or more subdivisions on the base mesh to generate a subdivided mesh comprising multiple vertices. The processor is further configured to decode the displacement sub-bitstream to reconstruct displacement wavelet coefficients for the multiple vertices. The processor is further configured to perform an update process using the displacement wavelet coefficients to generate updated displacement wavelet coefficients for the multiple vertices. The processor is further configured to perform a prediction process using the updated displacement wavelet coefficients to generate reconstructed displacements. The processor is further configured to reconstruct a mesh frame based on the reconstructed displacements and the subdivided mesh. The processor is further configured to, during the update process, determine a first valence indicating the number of connecting edges at the first vertex based on whether a first vertex forming an edge to which the current vertex belongs belongs to the base mesh, and update the displacement wavelet coefficients of the first vertex based on the first valence.

[0009] According to embodiments of this disclosure, if a vertex does not belong to the base mesh, the first valence is determined to be equal to a predetermined value.

[0010] According to embodiments of this disclosure, if a first vertex belongs to a base mesh, a first price is determined from an array of predetermined prices based on the index of the first vertex.

[0011] According to an embodiment of this disclosure, the predetermined value is 6.

[0012] According to embodiments of this disclosure, the processor is further configured to: during the update process, determine a second valence indicating the number of connecting edges at the second vertex based on whether the second vertex forming the edge to which the current vertex belongs belongs to the base mesh. The processor is also configured to: update the displacement wavelet coefficients of the second vertex based on the second valence.

[0013] According to embodiments of this disclosure, update weights are determined based on first grammatical elements, and the displacement wavelet coefficients of the first vertex are updated based on the first valence and the update weights.

[0014] According to embodiments of this disclosure, the compressed bitstream further includes a graph sub-bitstream, which includes a first syntax element for determining update weights. The processor is further configured to: determine update weights based on the first syntax element in the bitstream; and update the shift wavelet coefficients of the first vertex based on the first valence and the update weights.

[0015] According to embodiments of this disclosure, the compressed bitstream further includes a graph sub-bitstream, which includes a second syntax element for determining prediction weights. The processor is further configured to: determine the prediction weights based on the second syntax element in the bitstream; and determine the displacement of the current vertex based on the prediction weights and updated displacement wavelet coefficients of the first vertex.

[0016] One aspect of this disclosure provides a method. The method includes: receiving a compressed bitstream comprising a base grid sub-bitstream and a displacement sub-bitstream. The method further includes: reconstructing the base grid. The method further includes: performing one or more subdivisions on the base grid to generate a subdivided grid comprising multiple vertices. The method further includes: decoding the displacement sub-bitstream to reconstruct displacement wavelet coefficients for the multiple vertices. The method further includes: performing an update process using the displacement wavelet coefficients to generate updated displacement wavelet coefficients for the multiple vertices. The method further includes: performing a prediction process using the updated displacement wavelet coefficients to generate reconstructed displacements. The method further includes: reconstructing a grid frame based on the reconstructed displacements and the subdivided grid. The method further includes: determining a first valence indicating the number of connecting edges at the first vertex based on whether a first vertex forming an edge to which the current vertex belongs belongs to the base grid, and updating the displacement wavelet coefficients of the first vertex based on the first valence.

[0017] According to embodiments of this disclosure, if a vertex does not belong to the base mesh, the first valence is determined to be equal to a predetermined value.

[0018] According to embodiments of this disclosure, if a first vertex belongs to a base mesh, a first price is determined from an array of predetermined prices based on the index of the first vertex.

[0019] According to an embodiment of this disclosure, the predetermined value is 6.

[0020] According to embodiments of this disclosure, the processor is further configured to: during the update process, determine a second valence indicating the number of connecting edges at the second vertex based on whether the second vertex forming the edge to which the current vertex belongs belongs to the base mesh. The processor is also configured to: update the displacement wavelet coefficients of the second vertex based on the second valence.

[0021] According to embodiments of this disclosure, the compressed bitstream includes a graph sub-bitstream, which includes a first syntax element for determining update weights. The method further includes: determining update weights based on the first syntax element in the bitstream; and updating the shift wavelet coefficients of a first vertex based on the first valence and the update weights.

[0022] According to embodiments of this disclosure, the compressed bitstream further includes a graph sub-bitstream, which includes second syntax elements for determining prediction weights. The method further includes: determining prediction weights based on the second syntax elements in the bitstream; and determining the displacement of the current vertex based on the prediction weights and updated displacement wavelet coefficients of the first vertex.

[0023] One aspect of this disclosure provides an apparatus. The apparatus includes a communication interface and a processor. The processor is operatively coupled to the communication interface. The processor is configured to: reconstruct a base mesh; perform one or more subdivisions on the base mesh to generate a subdivided mesh including multiple vertices; determine displacements; perform prediction processing using the displacements to generate displacement wavelet coefficients; perform update processing using the displacement wavelet coefficients to generate updated displacement wavelet coefficients; encode the updated displacement wavelet coefficients to generate a compressed displacement bitstream; and transmit the compressed bitstream including the compressed displacement bitstream. The processor is further configured to: during the update processing, determine a first valence indicating the number of connecting edges at the first vertex based on whether a first vertex forming an edge to which the current vertex belongs belongs to the base mesh, and update the displacement wavelet coefficients of the first vertex based on the first valence.

[0024] According to embodiments of this disclosure, if a vertex does not belong to the base mesh, the first valence is determined to be equal to a predetermined value.

[0025] According to embodiments of this disclosure, if a first vertex belongs to a base mesh, a first price is determined from an array of predetermined prices based on the index of the first vertex.

[0026] According to an embodiment of this disclosure, the predetermined value is 6.

[0027] According to embodiments of this disclosure, the processor is further configured to: during the update process, determine a second valence indicating the number of connecting edges at the second vertex based on whether the second vertex forming the edge to which the current vertex belongs belongs to the base mesh. The processor is also configured to: update the displacement wavelet coefficients of the second vertex based on the second valence.

[0028] Price-based updates lead to higher compression efficiency, but at the cost of computational complexity. The simplified method in this disclosure retains the benefits of increased compression efficiency while avoiding the increased cost of computational complexity. Attached Figure Description

[0029] Figure 1 An example communication system 100 according to an embodiment of the present disclosure is shown.

[0030] Figure 2 and Figure 3 An example electronic device according to an embodiment of the present disclosure is shown.

[0031] Figure 4A block diagram of an encoder for encoding intra-frames according to an embodiment is shown.

[0032] Figure 5 A block diagram of a decoder according to an embodiment is shown.

[0033] Figure 6 and Figure 7 Basic block diagrams of the V-DMC encoder and V-DMC decoder according to embodiments are shown respectively.

[0034] Figure 8 An example of the reconstructed base mesh obtained from the encoder and decoder according to an embodiment is shown.

[0035] Figure 9 An example of a base grid that has already undergone one subdivision reconstruction is shown.

[0036] Figure 10 This is a flowchart illustrating the operation of a basic mesh encoder according to an embodiment.

[0037] Figure 11 This is a flowchart illustrating the operation of a wavelet transform performing prediction processing.

[0038] Figure 12 This is a flowchart illustrating the operation of the wavelet transform performing update processing.

[0039] Figure 13 This is a flowchart illustrating the operation of the basic mesh decoder according to an embodiment.

[0040] Figure 14 This is a flowchart illustrating the operation of the inverse wavelet transform performing update processing.

[0041] Figure 15 This is a flowchart illustrating the operation of the inverse wavelet transform performing prediction processing.

[0042] In one or more embodiments, not all of the components depicted in each figure may be required, and one or more embodiments may include additional components not shown in the figures. The arrangement and type of components may be changed without departing from the scope of this disclosure. Within the scope of this disclosure, additional components, different components, or fewer components may be used. Detailed Implementation

[0043] The detailed description set forth below with reference to the accompanying drawings is intended to describe various embodiments and not to represent the only possible implementation of the subject matter. Rather, the detailed description includes specific details for the purpose of providing a thorough understanding of the subject matter of the invention. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the scope of this disclosure. Therefore, the drawings and description are to be considered illustrative rather than restrictive in nature. Similar reference numerals denote similar elements.

[0044] With the ever-present availability of high-performance handheld devices such as smartphones, 360° video and 3D volumetric video are emerging as new ways to experience immersive content. While 360° video delivers an immersive, “real-life,” “being there” experience for consumers by capturing a 360° panoramic view of the world, 3D volumetric video offers a full 6DoF experience, allowing users to be immersed in and move within the content. Users can interactively change their viewpoint and dynamically view any part of the captured scene or object they desire. Display and navigation sensors track the user’s head movements in real time to determine the area of ​​360° video or volumetric content the user wants to view or interact with. Multimedia data with three-dimensional (3D) properties, such as point clouds or 3D polygon meshes, can be used in immersive environments.

[0045] A point cloud is a set of 3D points with attributes representing the surface or volume of an object, such as color, normals, reflectivity, and point size. Point clouds are commonly used in various applications, including games, 3D maps, visualization, medical applications, augmented reality, virtual reality, autonomous driving, multi-view replay, and 6DoF immersive media. If uncompressed, point clouds typically require significant bandwidth for transmission. Due to high bit rate requirements, point clouds are usually compressed before transmission. Compressing 3D objects, such as point clouds, typically requires dedicated hardware. To avoid using dedicated hardware to compress 3D point clouds, they can be transformed into traditional two-dimensional (2D) frames, which can then be compressed and reconstructed later for viewing by the user.

[0046] Polygonal 3D meshes (especially triangular meshes) are another common format for representing 3D objects. A mesh typically consists of a set of vertices, edges, and faces used to represent the surfaces of a 3D object. A triangular mesh is a simple polygonal mesh where the faces are simple triangles covering the surfaces of the 3D object. Typically, one or more attributes can exist associated with a mesh. In one scene, one or more attributes can be associated with each vertex in the mesh. For example, texture attributes (RGB) can be associated with each vertex. In another scene, each vertex can be associated with a pair of coordinates (u, v). The (u, v) coordinates point to a position in the texture map associated with the mesh. For example, the (u, v) coordinates can refer to the row index and column index in the texture map, respectively. A mesh can be thought of as a point cloud with additional connectivity information.

[0047] Point clouds or meshes can be dynamic, meaning they can change over time. In these cases, a point cloud or mesh at a specific moment can be referred to as a point cloud frame or a mesh frame, respectively.

[0048] Because point clouds and meshes contain large amounts of data, they need to be compressed for efficient storage and transmission. This is especially true for dynamic point clouds and meshes that may contain 60 or more frames per second.

[0049] The accompanying drawings and various embodiments used to describe the principles of this disclosure in this patent document are for illustrative purposes only and should not be construed as limiting the scope of this disclosure in any way. Those skilled in the art will understand that the principles of this disclosure can be implemented in any suitably arranged system or apparatus.

[0050] Figure 1 An example communication system 100 according to an embodiment of the present disclosure is shown. Figure 1 The embodiment of the communication system 100 shown is for illustrative purposes only. Other embodiments of the communication system 100 may be used without departing from the scope of this disclosure.

[0051] Communication system 100 includes a network 102 that facilitates communication between various components within the communication system 100. For example, network 102 can communicate between network addresses to transmit IP packets, Frame Relay frames, Asynchronous Transfer Mode (ATM) cells, or other information. Network 102 includes all or part of one or more local area networks (LANs), metropolitan area networks (MANs), wide area networks (WANs), global networks (such as the Internet), or any other communication system located in one or more locations.

[0052] In this example, network 102 facilitates communication between server 104 and various client devices 106-116. Client devices 106-116 may be, for example, smartphones, tablets, laptops, personal computers, TVs, interactive displays, wearable devices, HMDs, etc. Server 104 may represent one or more servers. Each server 104 includes any suitable computing or processing means capable of providing computing services to one or more client devices, such as client devices 106-116. Each server 104 may, for example, include one or more processing means, one or more memories storing instructions and data, and one or more network interfaces facilitating communication via network 102. As described in more detail below, server 104 may transmit compressed bitstreams representing point clouds or meshes to one or more display devices, such as client devices 106-116. According to embodiments of this disclosure, each server 104 may include an encoder.

[0053] Each client device 106-116 represents any suitable computing or processing device that interacts with at least one server (such as server 104) or other computing device via network 102. Client devices 106-116 include desktop computer 106, mobile phone or mobile device 108 (such as smartphone), PDA 110, laptop computer 112, tablet computer 114, and HMD 116. However, any other or additional client devices may be used in communication system 100. A smartphone represents a class of mobile devices 108 that are handheld devices with a mobile operating system and integrated mobile broadband cellular network connectivity for voice, short message service (SMS), and Internet data communication. HMD 116 can display a 360° scene including one or more dynamic or static 3D point clouds. According to embodiments of this disclosure, any of client devices 106-116 may include an encoder, a decoder, or both. For example, mobile device 108 may record 3D volumetric video and then encode the video so that it can be sent to one of client devices 106-116. In another example, laptop computer 112 can be used to generate a 3D point cloud or mesh, then encode the 3D point cloud or mesh and send it to one of client devices 106-116.

[0054] In this example, some client devices 108-116 communicate indirectly with network 102. For example, mobile device 108 and PDA 110 communicate via one or more base stations 118 (such as cellular base stations or eNodeBs (eNBs)). Additionally, laptop computer 112, tablet computer 114, and HMD 116 communicate via one or more wireless access points 120 (such as IEEE 802.11 wireless access points). It should be noted that these are for illustrative purposes only, and each client device 106-116 may communicate directly with network 102 or indirectly with network 102 via any suitable intermediary or network. According to embodiments of this disclosure, server 104 or any client device 106-116 may be used to compress point clouds or meshes, generate bitstreams representing point clouds or meshes, and send the bitstreams to another client device (such as any client device 106-116).

[0055] According to embodiments of this disclosure, any one of the client devices 106-114 can securely and efficiently transmit information to another device (such as, for example, server 104). Furthermore, any one of the client devices 106-116 can trigger information transmission between itself and server 104. Any one of the client devices 106-114 can be used as a VR display when attached to a head-mounted device via a bracket, and functions similarly to HMD 116. For example, when mobile device 108 is attached to the bracket system and worn on a user's eyes, mobile device 108 can perform functions similar to HMD 116. Mobile device 108 (or any other client device 106-116) can trigger information transmission between itself and server 104.

[0056] According to embodiments of this disclosure, any one of the client devices 106-116 or server 104 may perform the following operations: create a 3D point cloud or mesh, compress a 3D point cloud or mesh, send a 3D point cloud or mesh, receive a 3D point cloud or mesh, decode a 3D point cloud or mesh, render a 3D point cloud or mesh, or a combination thereof. For example, server 104 may then compress the 3D point cloud or mesh to generate a bitstream, and then send the bitstream to one or more of the client devices 106-116. As another example, one of the client devices 106-116 may compress the 3D point cloud or mesh to generate a bitstream, and then send the bitstream to another of the client devices 106-116 or server 104.

[0057] although Figure 1 An example of a communication system 100 is shown, but it is possible to modify it. Figure 1 Various changes can be made. For example, communication system 100 can include any number of each component in any suitable arrangement. Typically, computing and communication systems have a wide variety of configurations, and Figure 1 This disclosure is not intended to limit the scope to any particular configuration. Although Figure 1 An operating environment is shown in which the various features disclosed in this patent document can be used, but these features can be used in any other suitable system.

[0058] Figure 2 and Figure 3 An example electronic device according to an embodiment of the present disclosure is shown. In particular, Figure 2 Example server 200 is shown, and server 200 can represent Figure 1 Server 104 in the context of server 200 can represent one or more encoders, decoders, local servers, remote servers, cluster computers, components acting as a single seamless resource pool, cloud-based servers, etc. It can be... Figure 1 One or more of the client devices 106-116 or another server access server 200.

[0059] Server 200 may represent one or more local servers, one or more compression servers, or one or more encoding servers (such as encoders). According to embodiments of this disclosure, the encoder may perform decoding. Figure 2 As shown, server 200 includes bus system 205, which supports communication between at least one processing device (such as processor 210), at least one storage device 215, at least one communication interface 220 and at least one input / output (I / O) unit 225.

[0060] Processor 210 executes instructions that can be stored in memory 230. Processor 210 may include any suitable number and type of processors or other devices arranged in any suitable manner. Example types of processor 210 include microprocessors, microcontrollers, digital signal processors, field-programmable gate arrays, application-specific integrated circuits (ASICs), and discrete circuits.

[0061] According to embodiments of the present disclosure, processor 210 may encode a 3D point cloud or mesh stored in storage device 215. According to embodiments of the present disclosure, encoding the 3D point cloud also includes decoding the 3D point cloud or mesh to ensure that when the point cloud or mesh is reconstructed, the reconstructed 3D point cloud or mesh matches the 3D point cloud or mesh before encoding.

[0062] Memory 230 and persistent storage device 235 are examples of storage device 215, representing any structure capable of storing and facilitating the retrieval of information such as temporary or persistent data, program code, or other suitable information. Memory 230 may represent random access memory or any other suitable volatile or non-volatile storage device. For example, instructions stored in memory 230 may include instructions for decomposing a point cloud into patches, instructions for packing patches onto 2D frames, instructions for compressing 2D frames, and instructions for encoding 2D frames in a specific order to generate a bitstream. Instructions stored in memory 230 may also include instructions for use in omnidirectional 360° scenes (such as through VR headsets, such as...) Figure 1 Instructions for rendering point clouds on the HMD 116 (viewed). Persistent storage device 235 may contain one or more components or devices (such as read-only memory, hard disk drive, flash memory, or optical disk) that support long-term storage of data.

[0063] Communication interface 220 supports communication with other systems or devices. For example, communication interface 220 may include features that facilitate communication via... Figure 1 The network interface card or wireless transceiver communicates with network 102. Communication interface 220 can support communication via any suitable physical or wireless communication link. For example, communication interface 220 can send a bitstream containing a 3D point cloud to another device (such as one of client devices 106-116).

[0064] I / O unit 225 allows for data input and output. For example, I / O unit 225 can provide connectivity for user input via a keyboard, mouse, keypad, touchscreen, or other suitable input device. I / O unit 225 can also send output to a display, printer, or other suitable output device. However, it should be noted that I / O unit 225 can be omitted, such as when I / O interaction with server 200 occurs via a network connection.

[0065] It should be noted that, although Figure 2 Described as representing Figure 1 The server 104 may be used, but the same or similar architecture may be used in one or more client devices among various client devices 106-116. For example, desktop computer 106 or laptop computer 112 may have the same architecture as... Figure 2 The structures shown are the same or similar.

[0066] Figure 3 An example electronic device 300 is shown, and the electronic device 300 can represent Figure 1One or more of the client devices 106-116. Electronic device 300 may be a mobile communication device (such as, for example, a mobile station, user station, wireless terminal, desktop computer (similar to...)). Figure 1 Desktop computers 106), portable electronic devices (similar to) Figure 1 Mobile devices 108, PDA 110, laptop computer 112, tablet computer 114, or HMD 116, etc.). According to embodiments of this disclosure, Figure 1 One or more of the client devices 106-116 may include the same or similar configuration as electronic device 300. According to embodiments of this disclosure, electronic device 300 is an encoder, decoder, or both. For example, electronic device 300 may be used with data transmission, image or video compression, image or video decompression, encoding, decoding, and media rendering applications.

[0067] like Figure 3 As shown, electronic device 300 includes an antenna 305, a radio frequency (RF) transceiver 310, a transmit (TX) processing circuitry 315, a microphone 320, and a receive (RX) processing circuitry 325. The RF transceiver 310 may include, for example, an RF transceiver, a Bluetooth transceiver, a Wi-Fi transceiver, a Zigbee transceiver, an infrared transceiver, and various other wireless communication signals. Electronic device 300 also includes a speaker 330, a processor 340, an input / output (I / O) interface (IF) 345, an input 350, a display 355, a memory 360, and a sensor 365. The memory 360 includes an operating system (OS) 361 and one or more applications 362.

[0068] RF transceiver 310 receives incoming RF signals from antenna 305 from an access point (such as a base station, Wi-Fi router, or Bluetooth device) or other devices on network 102 (such as Wi-Fi, Bluetooth, cellular, 5G, LTE, LTE-A, WiMAX, or any other type of wireless network). RF transceiver 310 down-converts the incoming RF signals to generate an intermediate frequency (IF) or baseband signal. The IF or baseband signal is sent to RX processing circuitry 325, which generates a processed baseband signal by filtering, decoding, and / or digitizing the baseband or IF signal. RX processing circuitry 325 sends the processed baseband signal to speaker 330 (e.g., for voice data) or to processor 340 for further processing (e.g., for web browsing data).

[0069] The TX processing circuit 315 receives analog or digital voice data from the microphone 320, or other outgoing baseband data from the processor 340. Outgoing baseband data may include web data, email, or interactive video game data. The TX processing circuit 315 encodes, multiplexes, and / or digitizes the outgoing baseband data to generate a processed baseband or intermediate frequency (IF) signal. The RF transceiver 310 receives the processed baseband or IF signal from the TX processing circuit 315 and up-converts the baseband or IF signal into an RF signal transmitted via the antenna 305.

[0070] Processor 340 may include one or more processors or other processing devices. Processor 340 may execute instructions stored in memory 360 (such as OS 361) to control the overall operation of electronic device 300. For example, processor 340 may control the reception of forward channel signals and the transmission of reverse channel signals via RF transceiver 310, RX processing circuitry 325, and TX processing circuitry 315 according to well-known principles. Processor 340 may include any suitable number and type of processors or other devices arranged in any suitable manner. For example, according to embodiments of this disclosure, processor 340 includes at least one microprocessor or microcontroller. Example types of processor 340 include microprocessors, microcontrollers, digital signal processors, field-programmable gate arrays, application-specific integrated circuits, and discrete circuits.

[0071] Processor 340 is also capable of executing other processes and programs residing in memory 360, such as operations for receiving and storing data. Processor 340 may move data into or out of memory 360 as needed for the execution of the process. According to embodiments of this disclosure, processor 340 is configured to run one or more applications 362 based on OS 361 or in response to signals received from an external source or operator. For example, applications 362 may include encoders, decoders, VR or AR applications, camera applications (for still images and video), video call applications, email clients, social media clients, SMS messaging clients, virtual assistants, etc. According to embodiments of this disclosure, processor 340 is configured to receive and transmit media content.

[0072] The processor 340 is also integrated with an I / O interface 345, which provides the electronic device 300 with the ability to connect to other devices, such as client devices 106-114. The I / O interface 345 is the communication path between these accessories and the processor 340.

[0073] Processor 340 is also integrated with input 350 and display 355. An operator of electronic device 300 can use input 350 to input data or other information into electronic device 300. Input 350 may be a keyboard, touchscreen, mouse, trackball, voice input, or other device capable of acting as a user interface to allow user interaction with electronic device 300. For example, input 350 may include voice recognition processing, allowing user input of voice commands. In another example, input 350 may include a touch panel, (digital) pen sensor, key, or ultrasonic input device. Touch panel may recognize, for example, touch input using at least one method, such as capacitive, pressure-sensitive, infrared, or ultrasonic. Input 350 may be associated with sensor 365 and / or camera by providing additional input to processor 340. According to embodiments of this disclosure, sensor 365 includes one or more inertial measurement units (IMUs) (such as accelerometers, gyroscopes, and magnetometers), motion sensors, optical sensors, cameras, pressure sensors, heart rate sensors, altimeters, etc. Input 350 may also include control circuitry. In a capacitive scheme, input 350 can detect touch or proximity.

[0074] Display 355 may be a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic LED (OLED), an active-matrix OLED (AMOLED), or other display capable of rendering text and / or graphics (such as from websites, videos, games, images, etc.). The size of display 355 may be configured to fit within an HMD. Display 355 may be a single display or multiple displays capable of creating stereoscopic displays. According to embodiments of this disclosure, display 355 is a head-up display (HUD). Display 355 may display 3D objects (such as 3D point clouds or meshes).

[0075] Memory 360 is integrated with processor 340. A portion of memory 360 may include RAM, and another portion of memory 360 may include flash memory or other ROM. Memory 360 may include persistent storage devices (not shown), which represent any structure capable of storing and facilitating the retrieval of information such as data, program code, and / or other suitable information. Memory 360 may contain one or more components or devices (such as read-only memory, hard disk drive, flash memory, or optical disc) supporting long-term storage of data. Memory 360 may also contain media content. Media content may include various types of media (such as images, videos, 3D content, VR content, AR content, 3D point clouds, meshes, etc.).

[0076] The electronic device 300 also includes one or more sensors 365, which can measure physical quantities or detect the activation state of the electronic device 300 and convert the measured or detected information into electrical signals. For example, sensors 365 may include one or more buttons for touch input, a camera, a gesture sensor, an IMU sensor (such as a gyroscope or gyroscope sensor and accelerometer), an eye-tracking sensor, a barometric pressure sensor, a magnetic sensor or magnetometer, a grip sensor, a proximity sensor, a color sensor, a biophysical sensor, a temperature / humidity sensor, an illumination sensor, an ultraviolet (UV) sensor, an electromyography (EMG) sensor, an electroencephalography (EEG) sensor, an electrocardiogram (ECG) sensor, an IR sensor, an ultrasound sensor, an iris sensor, a fingerprint sensor, and color sensors (such as red-green-blue (RGB) sensors). Sensors 365 may also include control circuitry for controlling any of the included sensors.

[0077] As discussed in more detail below, one or more of these sensors 365 can be used to control the user interface (UI), detect UI input, determine the orientation and facing direction of the user's 3D content display recognition, etc. Any of these sensors 365 may be located within the electronic device 300, within an auxiliary device operatively connected to the electronic device 300, within a head-mounted device configured to house the electronic device 300, or within a single device in which the electronic device 300 includes the head-mounted device.

[0078] Electronic device 300 can create media content (such as generating virtual objects or capturing (or recording) content via a camera). Electronic device 300 can encode the media content to generate a bitstream, such that the bitstream can be sent directly to another electronic device, or indirectly (e.g., via...) Figure 1 The network 102) sends the bit stream to another electronic device. The electronic device 300 can receive the bit stream directly from the other electronic device, or indirectly from the other electronic device (e.g., via...). Figure 1 Network 102) receives bit streams.

[0079] although Figure 2 and Figure 3 An example of an electronic device is shown, but it is possible to... Figure 2 and Figure 3 Make various changes. For example, make them combinable, further subdivided, or omitted. Figure 2 and Figure 3 The various components within it, and additional components can be added as needed. As a specific example, processor 340 can be divided into multiple processors (such as one or more central processing units (CPUs) and one or more graphics processing units (GPUs)). Furthermore, as with computing and communication, electronic devices and servers can have a wide variety of configurations, and Figure 2and Figure 3 This disclosure is not limited to any particular electronic device or server.

[0080] Figure 4 A block diagram of an encoder for encoding intra-frames according to an embodiment is shown.

[0081] like Figure 4 As shown, the encoder 400 that encodes intra-frames according to an embodiment may include a quantizer 401, a static mesh encoder 403, a static mesh decoder 405, a shift updater 407, a wavelet transform 409, a quantizer 411, an image packer 413, a video encoder 415, an image unpacker 417, an inverse quantizer 419, an inverse wavelet transform 421, an inverse quantizer 423, a deformable mesh reconstructor 425, an attribute transfer module 427, a fill module 429, a color space converter 431, a video encoder 433, a multiplexer 435, and a controller 437.

[0082] The quantizer 401 can quantize the base mesh m(i) to generate a quantized base mesh. According to embodiments of this disclosure, the base mesh may have fewer vertices than the original mesh.

[0083] A static mesh encoder 403 can encode and compress a quantized base mesh to generate a compressed base mesh bitstream. According to embodiments of this disclosure, the base mesh can be compressed in a lossy or lossless manner. According to embodiments of this disclosure, an existing mesh codec (such as Draco) can be used to compress the base mesh.

[0084] The static grid decoder 405 can decode the compressed underlying grid bitstream to generate the reconstructed quantized underlying grid m'(i).

[0085] The displacement updater 407 can update the displacement d(i) based on the subdivided base mesh m(i) and the reconstructed quantized base mesh m'(i) to generate an updated displacement d'(i). The reconstructed base mesh can be subdivided, and the field between the original mesh and the subdivided reconstructed base mesh can then be determined. In inter-frame coding of the mesh frame, the base mesh can be encoded by sending vertex motion, rather than directly compressing the base mesh. In either case, a displacement field can be created. The displacement field, along with the modified property map, can be encoded using a video codec and is also included as part of the V-DMC bitstream.

[0086] Wavelet transform 409 can perform wavelet transform using the updated displacement d'(i) to generate displacement wavelet coefficients e(i). The wavelet transform may include a series of prediction and update lifting processes.

[0087] Quantizer 411 can quantize the displacement wavelet coefficients e(i) to generate quantized displacement wavelet coefficients e'(i). The quantized displacement wavelet coefficients can be represented by an array dispQuantCoeffArray.

[0088] Image packer 413 can pack the quantized displacement wavelet coefficients e'(i) into a 2D image including the packed quantized displacement wavelet coefficients dispQuantCoeffFrame. The 2D video frame may be referred to herein as a displacement frame or a displacement video frame.

[0089] The video encoder 415 can encode the packed quantized displacement wavelet coefficients dispQuantCoeffFrame to generate a compressed displacement bitstream.

[0090] Image unpacker 417 can unpack the packed quantized shift wavelet coefficients dispQuantCoeffFrame to generate an array of quantized shift wavelet coefficients dispQuantCoeffArray.

[0091] The dequantizer 419 can dequantize the array of quantized shift wavelet coefficients, dispQuantCoeffArray, to generate shift wavelet coefficients.

[0092] The inverse wavelet transform 421 can perform an inverse wavelet transform using the displacement wavelet coefficients to generate the reconstructed displacement d(i).

[0093] Dequantizer 423 can dequantize the reconstructed quantized base grid m'(i) to generate the reconstructed base grid m"(i).

[0094] The deformable mesh reconstructor 425 can generate a reconstructed deformable mesh DM(i) based on the reconstructed displacement D(i) and the reconstructed base mesh m(i).

[0095] The attribute transfer module 427 can update the attribute map A(i) based on the static / dynamic mesh m(i) and the reconstructed deformable mesh DM(i) to generate an updated attribute map A'(i). The attribute map can be a texture map, but other attributes can also be sent.

[0096] The filling module 429 can perform filling to fill blank areas in the updated property graph A'(i) in order to remove high-frequency components.

[0097] Color space converter 431 can perform color space conversion on the filled updated attribute graph A'(i).

[0098] The video encoder 433 can encode the output of the color space converter 431 to generate a compressed attribute bitstream.

[0099] Multiplexer 435 can multiplex the compressed base grid bitstream, the compressed displacement bitstream, and the compressed attribute bitstream to generate a compressed bitstream b(i).

[0100] The controller 437 can control the module of the encoder 400.

[0101] Figure 5 A block diagram of a decoder according to an embodiment is shown.

[0102] like Figure 5 As shown, the decoder 500 may include a demultiplexer 501, a switch 503, a static mesh decoder 505, a mesh buffer 507, a motion decoder 509, a basic mesh reconstructor 511, a switch 513, an inverse quantizer 515, a video decoder 521, an image unpacker 523, an inverse quantizer 525, a wavelet inverse transform 527, a deformable mesh reconstructor 529, a video decoder 531, and a color space converter 533.

[0103] Demultiplexer 501 can receive compressed bit stream b(i) from encoder 400 to extract compressed base grid bit stream, compressed displacement bit stream and compressed attribute bit stream from compressed bit stream b(i).

[0104] Switch 503 determines whether the compressed base mesh bitstream has inter-coded mesh frames or intra-coded mesh frames. If the compressed base mesh bitstream has inter-coded mesh frames, switch 503 can transmit the inter-coded mesh frames to motion decoder 509. If the compressed base mesh bitstream has intra-coded mesh frames, switch 503 can transmit the intra-coded mesh frames to static mesh decoder 505.

[0105] The static grid decoder 505 can decode intra-coded grid frames to generate reconstructed quantized base grid frames.

[0106] The grid buffer 507 can store the reconstructed quantization base grid frame and the inter-coded grid frame for future use in decoding subsequent inter-coded grid frames. The reconstructed quantization base grid frame can be used as a reference grid frame.

[0107] The motion decoder 509 can obtain the motion vector for the current inter-coded grid frame based on the data stored in the grid buffer 507 and the syntax elements for the current inter-coded grid frame in the bitstream. According to embodiments of this disclosure, the syntax elements for the current inter-coded grid frame in the bitstream can be the motion vector difference.

[0108] The base mesh reconstructor 511 can generate a reconstructed quantized base mesh frame based on the motion vector for the current inter-coded mesh frame by using syntax elements of the bitstream for the current inter-coded mesh frame.

[0109] If the compressed base grid bitstream has intra-coded grid frames, switch 513 can send the reconstructed quantized base grid frame from static grid decoder 505 to dequantizer 515. If the compressed base grid bitstream has inter-coded grid frames, switch 513 can send the reconstructed quantized base grid frame from static grid decoder 511 to dequantizer 515.

[0110] The dequantizer 515 can perform dequantization using the reconstructed quantized base grid frame to generate the reconstructed base grid frame m(i).

[0111] The video decoder 521 can decode the shifted bitstream to generate packed quantized shifted wavelet coefficients dispQuantCoeffFrame.

[0112] Image unpacker 523 can unpack the packed quantized shift wavelet coefficients dispQuantCoeffFrame to generate an array of quantized shift wavelet coefficients dispQuantCoeffArray.

[0113] The dequantizer 525 can perform dequantization using an array of quantized shift wavelet coefficients, dispQuantCoeffArray, to generate shift wavelet coefficients.

[0114] The inverse wavelet transform 527 can perform an inverse wavelet transform using displacement wavelet coefficients to generate displacements.

[0115] The deformable mesh reconstructor 529 can reconstruct deformable meshes based on displacement and the reconstructed base mesh frame m (i).

[0116] The video decoder 531 can decode the attribute bitstream to generate an attribute map before color space conversion.

[0117] The color space converter 533 can perform color space conversion on the attribute map from the video decoder 531 to reconstruct the attribute map.

[0118] The following documents are hereby incorporated herein by reference in their entirety, as if fully set forth herein: i) V-DMC TMM 7.0, ISO / IEC SC29 WG07 N00811, January 2024; ii) “V-DMC WD 6.0”, ISO / IEC SC29 WG07 N00822, January 2024; iii) “Text, video-based mesh coding of ISO / IEC DIS 23090-29”, ISO / IEC SC29 WG07 N01027, December 2024; and iv) “[V-DMC][EE4.7 – Test 7.1] Price-based adaptive boosting update”, ISO / IEC JTC 1 / SC 29 / WG 7 m66331 (m66331), January 2024.

[0119] Figure 6 A basic block diagram for a V-DMC encoder according to an embodiment is shown.

[0120] Reference Figure 6 According to an embodiment, the V-DMC encoder 600 includes a preprocessing unit 610, an atlas encoder 620, a basic mesh encoder 630, a displacement encoder 640, and a multiplexer 650. According to embodiments of this disclosure, the V-DMC encoder 600 may include a video encoder 660.

[0121] Preprocessing unit 610 can process dynamic mesh sequences to generate atlas information, a base mesh m(i), and a displacement d(i). According to embodiments of this disclosure, preprocessing unit 610 can also process dynamic mesh sequences to additionally generate attribute A(i). Atlas encoder 620 can encode the atlas information to generate an atlas sub-bitstream. Base mesh encoder 630 can encode the base mesh m(i) to generate a base mesh sub-bitstream. Displacement encoder 640 can encode the displacement d(i) to generate a displacement sub-bitstream. According to embodiments of this disclosure, video encoder 660 can encode attribute A(i) to generate an attribute sub-bitstream. Multiplexer 650 can combine the atlas sub-bitstream, the base mesh sub-bitstream, and the displacement sub-bitstream to generate a single V3C bitstream b(i). According to embodiments of this disclosure, multiplexer 650 can combine the atlas sub-bitstream, the base mesh sub-bitstream, the displacement sub-bitstream, and the attribute sub-bitstream to generate a single V3C bitstream b(i). Therefore, for each mesh frame, the V-DMC encoder creates a base mesh, which typically has fewer vertices than the original mesh. The base mesh is then compressed in a lossy or lossless manner to create a base mesh substream. The V-DMC encoder also generates a reconstructed base mesh.

[0122] Figure 7A basic block diagram for a V-DMC decoder according to an embodiment is shown.

[0123] Reference Figure 7 According to an embodiment, the V-DMC decoder 700 includes a demultiplexer 710, an atlas decoder 720, a base mesh decoder 730, a displacement decoder 740, a base mesh processing unit 750, a displacement processing unit 760, a mesh processing unit 770, and a reconstruction processing unit 780. According to embodiments of this disclosure, the V-DMC decoder 700 may include a video decoder 790.

[0124] Demultiplexer 710 can separate the V3C bitstream b(i) to generate an atlas sub-bitstream, a base grid sub-bitstream, and a displacement sub-bitstream. According to embodiments of this disclosure, demultiplexer 710 can also separate the V3C bitstream b(i) to additionally generate an attribute sub-bitstream. Atlas decoder 720 can decode the atlas sub-bitstream to generate atlas information. Base grid decoder 730 can decode the base grid sub-bitstream to generate a base grid m(i). Displacement decoder 740 can decode the displacement sub-bitstream to generate a displacement d(i). According to embodiments of this disclosure, video decoder 790 can decode the attribute sub-bitstream to generate attribute A(i). The base mesh processor 750 processes atlas information and the base mesh m(i) to generate a reconstructed base mesh m(i). The displacement processor 760 processes atlas information and displacement d(i) to generate a reconstructed displacement. The mesh processor 770 processes atlas information, the reconstructed base mesh m(i), and the reconstructed displacement matrix D(i) to generate a reconstructed deformable mesh DM(i). In some embodiments, the reconstruction processing unit 780 processes the reconstructed deformable mesh DM(i) and attribute A(i) to generate a reconstructed dynamic mesh sequence.

[0125] Figure 8 An example of a reconstructed base mesh obtained from the encoder and decoder according to an embodiment is shown. This mesh contains 7 vertices (A, B, ..., G). Edges AB, BC, CD, and DA can be considered as mesh boundaries. The remaining edges can be considered as non-boundary or interior edges. Typically, boundary edges belong to only a single triangle, while interior edges belong to multiple triangles.

[0126] The reconstructed base mesh can be subdivided once or multiple times. In the case of V-DMC, subdivision methods (such as midpoint subdivision, loop subdivision, normal subdivision, and Pythagorean mean-based subdivision) are used, where additional vertices are introduced for each edge in the current mesh. For midpoint subdivision, additional vertices are introduced at the midpoint of the edge. For loop subdivision, additional vertices are introduced as determined using a weighted average of the positions forming the edge. For normal subdivision, new edges are introduced to introduce additional vertices based on the generation of normal vectors. For Pythagorean mean subdivision, additional vertices are introduced as determined based on the average, harmonic mean, or geometric mean of the positions forming the edge. For each triangle from the current mesh, new edges are introduced to connect the three newly introduced vertices to each other. Thus, each triangle is subdivided into 4 triangles.

[0127] Figure 9 An example of a base mesh that has already undergone one subdivision reconstruction is shown. (Refer to...) Figure 9 This subdivision introduces 14 new vertices (H, I, ..., U). This process can be continued for further subdivisions. The total number of subdivisions is represented by `subdivisionIterationCount`. The number of vertices introduced by the k-th subdivision is represented by `levelOfDetailVertexCounts[k]`, where k = 0, 1, ..., (subdivisionIterationCount + 1), and `levelOfDetailVertexCounts[0]` indicates the number of vertices in the reconstructed base mesh. Each subdivision can also be referred to as a level of detail.

[0128] The total number of vertices N in the subdivided mesh can be expressed as shown in Equation 1 below: [Equation 1]

[0129] For each vertex in the subdivided mesh, the V-DMC encoder determines the displacement vector relative to the original mesh surface. Each displacement vector has three components, represented by x, y, and z. These can be relative to a canonical coordinate system or a local coordinate system, where x, y, and z represent the local normal, tangent, and bitangent directions, respectively. The 3D displacement vector can be represented as shown in Equation 2 below: [Equation 2]

[0130] On the encoder side, as implemented in the test model V-DMC TMM 7.0, the 3D displacement vectors are subjected to a forward linear lift transformation. The lift transformation process begins at the highest subdivision level and continues up to the first subdivision level. At each subdivision level, for each vertex, prediction and update processing are performed. A vertex is represented as v, and the vertices forming the edges to which vertex v belongs are represented by v1 and v2. The prediction process can then be represented as shown in Equation 3: [Equation 3]

[0131] After prediction processing, the V-DMC encoder can update the displacements corresponding to vertices v1 and v2, as shown in Equations 4 and 5: [Equation 4]

[0132] [Equation 5]

[0133] According to embodiments of this disclosure, the prediction weight `predWeight` and the update weight `updateWeight` can be functions of the subdivision level and index of the subdivision to which vertex v belongs. According to embodiments of this disclosure, the V-DMC encoder and V-DMC decoder can determine the prediction weight `predWeight` and the update weight `updateWeight` based on the subdivision level and index of the subdivision to which vertex v belongs.

[0134] According to embodiments of this disclosure, after a forward linear lift transform, the resulting transformed displacement field can be quantized. The quantized transformed displacement field can be packed into a video frame and then compressed using a video codec.

[0135] According to embodiments of this disclosure, arithmetic coding can be used to directly entropy encode the quantized transformed displacement field.

[0136] As specified in V-DMC WD 6.0 and implemented in the test model V-DMC TMM 7.0, the V-DMC decoder reconstructs the quantized transform-shift field by decoding the packed compressed video or by arithmetic decoding. The reconstructed transform-shift field is then subjected to an inverse lifting wavelet transform. The lifting process begins at the first subdivision level and continues up to the highest subdivision level. At each subdivision level, for each vertex belonging to that level, update and prediction processes are performed. The update process can be represented by Equations 6 and 7: [Equation 6]

[0137] [Equation 7]

[0138] After the update process, the prediction process is performed. The prediction process can be represented by Equation 8: [Equation 8]

[0139] The compressed V-DMC bitstream includes syntax elements related to updates and predictions, which allow the V-DMC decoder to derive updateWeight and predWeight. The encoder and decoder typically use the update weight.

[0140] As proposed in "[V-DMC][EE4.7 – Test 7.1] Price-Based Adaptive Boosting Update", ISO / IEC JTC 1 / SC 29 / WG 7 m66331 (m66331), the update weights can depend on the price of the corresponding vertex. On the encoder side, the update process can be modified as shown in Equations 9 and 10: [Equation 9]

[0141] [Equation 10]

[0142] Furthermore, on the decoder side, the update process can be modified, as shown in Equations 11 and 12: [Equation 11]

[0143] [Equation 12]

[0144] Here, K can be a constant that depends on the subdivision level and the syntax elements related to the update weights sent by signaling in the V-DMC bitstream.

[0145] The valence of vertex v can be represented by the number of other vertices it is directly connected to. Alternatively, the valence of vertex v can be considered as the number of edges that begin or end at vertex v.

[0146] Under typical test conditions, this method has been shown to produce a BD-rate gain in the range of 0.5% to 0.7% compared to TMM 6.0 anchors. However, to implement this method, it is necessary to determine the valences of v1 and v2 during runtime. These can be computationally complex operations requiring significant memory access. Alternatively, the valences for vertices in the subdivision hierarchy from 0 to (SubdivisionIterationCount-1) can be pre-calculated and stored in memory. However, depending on the number of vertices in the base mesh and the subdivisionIterationCount, the amount of memory required may be prohibitive.

[0147] To overcome this drawback, a simplified form can be used, which reduces memory and / or computational requirements while retaining almost all the gains in cost during update processing.

[0148] Figure 9 An example of a base grid that has already undergone one subdivision reconstruction is shown.

[0149] Reference Figure 9 Each new vertex generated by midpoint subdivision has a valence of 6. The only exception is when the new vertex is subdivided along edges belonging to the boundaries of the base mesh. In this case, the valence for the new vertex is 4. According to embodiments of this disclosure, a modified valence function, valenceMod, can be used. The function valenceMod can be represented by Equation 13: [Equation 13]

[0150] In embodiments using valenceMod, it is only necessary to determine and store valences for vertices belonging to the base mesh, thus saving significant computation and memory.

[0151] According to embodiments of this disclosure, the update process in the inverse lifting wavelet transform can be modified, as shown in Equations 14 and 15: [Equation 14]

[0152] [Equation 15]

[0153] According to embodiments of this disclosure, in addition to modifying the update process in the inverse lifting wavelet transform, the update process in the forward lifting wavelet transform is also modified, as shown in Equations 16 and 17: [Equation 16]

[0154] [Equation 17]

[0155] When a new vertex partitions the edges belonging to the boundary of the base mesh, its valence is 4. According to an embodiment of this disclosure, valenceMod can be determined as shown in Equation 18: [Equation 18]

[0156] According to embodiments of this disclosure, Equation 18 can be used in the update processing of the inverse lifting wavelet transform.

[0157] According to embodiments of this disclosure, Equation 18 can be used to modify the update process in both the forward lifting wavelet transform and the inverse lifting wavelet transform.

[0158] According to embodiments of this disclosure, a fixed value M can be used in the calculation of valenceMod. The value M can be fixed for all levels of detail (LODs), or it can be fixed for each level of detail (LOD). The function of valenceMod can be expressed as shown in Equation 19: [Equation 19]

[0159] According to embodiments of this disclosure, Equation 19 can be used in the update process of the inverse lifting wavelet transform.

[0160] According to embodiments of this disclosure, Equation 19 can be used in the update process of both the forward lifting wavelet transform and the inverse lifting wavelet transform.

[0161] Reference Figure 8 The base grid for reconstruction and Figure 9 The base mesh for subdivision reconstruction is reconstructed, and when subdivision is performed, the value of vertices existing in the reconstructed base mesh does not change.

[0162] According to embodiments of this disclosure, the value of vertices corresponding to the reconstructed base mesh can be determined based on the reconstructed base mesh before performing any subdivision.

[0163] According to embodiments of this disclosure, the valence of the vertices corresponding to the reconstructed base mesh can be obtained as the output of the base mesh decoder without needing to be re-determined.

[0164] According to embodiments of this disclosure, a syntax element may be included in the bitstream and may be signaled in the V-DMC extension asps_vdmc_extension of the atlas sequence parameter set. The syntax element may take the value 0, 1, or 2. If the value of the syntax element is 0, the update process is as described in WD 6.0 of V-DMC. If the value of the syntax element is 1, the update process is based on a valence as described in m66331. If the value of the syntax element is 2, a simplified valence-based update process as described in one of the above embodiments is used.

[0165] According to embodiments of this disclosure, two flags may be included in the bitstream and may be signaled in the V-DMC extension asps_vdmc_extension of the atlas sequence parameter set. A first flag indicates whether a price-based update is used. A value of 0 for the first flag indicates that an update process as described in WD 6.0 of V-DMC is used. A value of 1 for the first flag indicates that a price-based update process may be used. If the value of the first flag is 1, a second flag is signaled to indicate whether a simplified price-based update as described in one of the above embodiments is used. A value of 0 for the second flag indicates that an update process as described in m66331 is used. A value of 1 for the second flag indicates that a simplified price-based update as described in the above embodiments is used.

[0166] According to embodiments of this disclosure, a flag may be included in the bitstream and may be signaled in the V-DMC extension asps_vdmc_extension of the atlas sequence parameter set. This flag indicates whether a simplified valence-based update as described in none of the above embodiments is applicable. A value of 0 for the flag indicates that an update process as described in WD 6.0 of V-DMC is applicable. A value of 1 for the flag indicates that a simplified valence-based update process as described in one of the above embodiments is applicable.

[0167] According to embodiments of this disclosure, clause 11 in the text specification of V-DMC WD 6.0 is modified as shown in Tables 1-6 below.

[0168] [Table 1]

[0169]

[0170] In Table 1, the input for midpoint subdivision processing is changed to include the inputs and outputs listed above.

[0171] [Table 2]

[0172]

[0173] In Table 2, the array of 2D edges is determined by first setting all values ​​in the array `neighborCounts` to 0. Next, the function `ComputeNeighbors` is used to determine the values ​​of the array `neighborCounts`. For each position to be subdivided, a first position is determined as the current position. For each neighbor of this first position, a second position is determined as a neighbor of the first position, and if the second position is a position where new edges can be created (i.e., not a position for new subdivision), the edge array at the current index (`positionCounter`) is determined to be the edge created by the first and second positions.

[0174] [Table 3]

[0175]

[0176] In Table 3, the inputs for the inverse transform process are changed to include the inputs and outputs listed above.

[0177] [Table 4]

[0178] In Table 4, updateWeights and predWeights are determined based on values ​​derived from the syntax present in the compressed bitstream.

[0179] [Table 5]

[0180]

[0181] Table 5 shows the inverse wavelet transform, where, by implementing... Figure 14 The update process shown in the figure and Figure 15 The prediction process shown in the figure is used to generate the displacement array.

[0182] Figure 10 This is a flowchart illustrating the operation of the V-DMC encoder 400 according to an embodiment.

[0183] Reference Figure 10 The V-DMC encoder 400 encodes the original mesh frames.

[0184] At 1001, the V-DMC encoder 400 determines the array of displacements, dispArray. In an embodiment, the V-DMC encoder 400 can reconstruct the base mesh. In an embodiment, the V-DMC encoder 400 can perform one or more subdivisions on the base mesh to generate a subdivided mesh including multiple vertices.

[0185] In 1003, the wavelet transform 409 of the V-DMC encoder 400 performs prediction processing using the displacement array dispArray to generate the displacement wavelet coefficient array dispCoeffArray.

[0186] In 1005, the V-DMC encoder 400 determines an array of valences, each valence indicating the number of connected edges at a corresponding vertex in the base mesh.

[0187] At 1007, the wavelet transform 409 of the V-DMC encoder 400 performs an update process using the array of displacement wavelet coefficients dispCoeffArray to generate an updated array of displacement wavelet coefficients dispCoeffArray.

[0188] At 1009, the quantizer 411 of the V-DMC encoder 400 quantizes the updated array of displacement wavelet coefficients dispCoeffArray to generate an array of quantized displacement wavelet coefficients dispQuantCoeffArray.

[0189] At 1011, the image packer 413 of the V-DMC encoder 400 packs the array of quantized displacement wavelet coefficients dispQuantCoeffArray into a 2D image dispQuantCoeffFrame that includes the packed quantized displacement wavelet coefficients.

[0190] At 1013, the video encoder 415 of the V-DMC encoder 400 encodes the packed quantized displacement wavelet coefficients dispQuantCoeffFrame to generate a compressed displacement bitstream.

[0191] In step 1015, the V-DMC encoder 400 transmits a compressed bitstream comprising a compressed displacement bitstream and a atlas bitstream. According to embodiments of this disclosure, the atlas bitstream may include a first syntax element for determining prediction weights and a second syntax element for determining update weights. In embodiments, the compressed bitstream may also include a base grid sub-bitstream.

[0192] Figure 11 This is a flowchart illustrating the prediction processing operation of the wavelet transform 409 of the V-DMC encoder 400 according to an embodiment.

[0193] Reference Figure 11 The wavelet transform 409 determines the displacement wavelet coefficients.

[0194] At 1101, wavelet transform 409 determines the prediction weights.

[0195] At 1103, wavelet transform 409 determines the first vertex v1 and the second vertex v2 that form the edge to which the v-th vertex belongs.

[0196] In 1105, wavelet transform 409 determines the predicted displacement of the vth vertex based on the prediction weights, the displacement of the first vertex v1 in the array dispArray, and the displacement of the second vertex v2 in the array dispArray.

[0197] In 1107, wavelet transform 409 determines the displacement wavelet coefficients of the v-th vertex based on the predicted displacement of the v-th vertex and the displacement of the v-th vertex in the array dispArray, so as to store the displacement wavelet coefficients of the v-th vertex in the array dispCoeffArray of displacement wavelet coefficients.

[0198] Figure 12 This is a flowchart illustrating the update processing operation of the wavelet transform 409 of the V-DMC encoder 400 according to an embodiment.

[0199] Reference Figure 12 The wavelet transform 409 determines the updated displacement wavelet coefficients.

[0200] At 1201, wavelet transform 409 determines the updated weights.

[0201] At 1203, wavelet transform 409 determines the first vertex v1 and the second vertex v2 that form the edge to which the v-th vertex belongs.

[0202] At 1205, wavelet transform 409 determines a first valence indicating the number of connected edges at the first vertex v1. According to an embodiment of this disclosure, if the first vertex v1 belongs to the base mesh, the inverse wavelet transform 527 can determine the first valence from an array of predetermined valences based on the index of the first vertex v1. The array of predetermined valences requires memory and much prior computation. According to an embodiment of this disclosure, if the first vertex v1 does not belong to the base mesh, the inverse wavelet transform 527 can determine that the first valence is equal to a predetermined value. According to an embodiment of this disclosure, the predetermined value can be equal to 6. According to an embodiment of this disclosure, the predetermined value can be equal to 4, wherein the first vertex v1 belongs to edges belonging to the boundaries of the reconstructed base mesh. According to an embodiment of this disclosure, the predetermined value can be equal to a value fixed for all levels of detail. According to an embodiment of this disclosure, the predetermined value can be equal to a value fixed for each level of detail. Since no memory and prior computation are required when the first vertex v1 does not belong to the base mesh, computational and memory requirements can be significantly reduced.

[0203] At 1207, wavelet transform 409 determines a second valence indicating the number of connected edges at the second vertex v2. According to an embodiment of this disclosure, if the second vertex v2 belongs to the base mesh, the inverse wavelet transform 527 can determine the second valence from an array of predetermined valences based on the index of the second vertex v2. The array of predetermined valences requires memory and much prior computation. According to an embodiment of this disclosure, if the second vertex v2 does not belong to the base mesh, the inverse wavelet transform 527 can determine that the second valence is equal to a predetermined value. According to an embodiment of this disclosure, the predetermined value can be equal to 6. According to an embodiment of this disclosure, the predetermined value can be equal to 4, wherein the second vertex v2 belongs to edges belonging to the boundaries of the reconstructed base mesh. According to an embodiment of this disclosure, the predetermined value can be equal to a value fixed for all levels of detail. According to an embodiment of this disclosure, the predetermined value can be equal to a value fixed for each level of detail. Since no memory and prior computation are required when the second vertex v2 does not belong to the base mesh, computational and memory requirements can be significantly reduced.

[0204] At 1209, wavelet transform 409 updates the displacement wavelet coefficients of the first vertex v1 in the array dispCoeffArray based on the first valence, updated weights and displacement wavelet coefficients of the vth vertex in the array dispCoeffArray, to generate updated displacement wavelet coefficients of the first vertex v1, and stores the updated displacement wavelet coefficients of the first vertex v1 in the array dispCoeffArray.

[0205] In 1211, wavelet transform 409 updates the displacement wavelet coefficients of the second vertex v2 in the array dispCoeffArray based on the second valence, updated weights and displacement wavelet coefficients of the vth vertex in the array dispCoeffArray, to generate updated displacement wavelet coefficients of the second vertex v2, so as to store the updated displacement wavelet coefficients of the first vertex v2 in the array dispCoeffArray.

[0206] Figure 13 This is a flowchart illustrating the operation of the V-DMC decoder 500 according to an embodiment.

[0207] Reference Figure 13 The V-DMC decoder 500 decodes the V3C bitstream to generate a deformable mesh.

[0208] At 1301, the V-DMC decoder 500 receives a compressed bitstream including a compressed displacement bitstream and a atlas bitstream. According to embodiments of this disclosure, the atlas bitstream may include a first syntax element for determining update weights and a second syntax element for determining prediction weights. In embodiments, the compressed bitstream may also include a base grid sub-bitstream. The V-DMC decoder 500 may decode the base grid sub-bitstream to reconstruct the base grid. The V-DMC decoder 500 may perform one or more subdivisions on the base grid to generate a subdivided grid including multiple vertices.

[0209] At 1303, the video decoder 521 of the V-DMC decoder 500 decodes the shift bitstream to generate an array of packed quantized shift wavelet coefficients, dispQuantCoeffFrame.

[0210] At 1305, the image unpacker 523 of the V-DMC decoder 500 unpacks the array of quantized shift wavelet coefficients dispQuantCoeffFrame to generate an array of quantized shift wavelet coefficients dispQuantCoeffArray.

[0211] At 1307, the dequantizer 525 of the V-DMC decoder 500 performs dequantization using an array of quantized shift wavelet coefficients, dispQuantCoeffArray, to generate an array of shift wavelet coefficients, dispCoeffArray.

[0212] In 1309, the V-DMC decoder 500 determines an array of valences, each valence indicating the number of connected edges at a corresponding vertex in the base mesh.

[0213] In 1311, the wavelet inverse transform 527 of the V-DMC decoder 500 performs an update process using an array of displacement wavelet coefficients, dispCoeffArray, to generate an updated array of displacement wavelet coefficients, dispCoeffArray.

[0214] At 1313, the wavelet inverse transform 527 of the V-DMC decoder 500 performs prediction processing using the updated array of displacement wavelet coefficients dispCoeffArray to generate the array of displacements dispArray.

[0215] In 1315, the deformable mesh reconstructor 529 of the V-DMC decoder 500 reconstructs the mesh frame recMeshFrame based on a displacement array dispArray. In an embodiment, the deformable mesh reconstructor 529 of the V-DMC decoder 500 may reconstruct the mesh frame based on the reconstructed displacement and subdivision mesh.

[0216] Figure 14 This is a flowchart illustrating the update processing operation of the wavelet inverse transform 527 of the V-DMC decoder 500 according to an embodiment.

[0217] Reference Figure 14 The inverse wavelet transform 527 determines the updated displacement wavelet coefficients.

[0218] At 1401, the wavelet inverse transform 527 determines the updated weights based on the first syntax element.

[0219] At 1403, the wavelet inverse transform 527 determines the first vertex v1 and the second vertex v2 that form the edge to which the v-th vertex belongs.

[0220] At 1405, the wavelet inverse transform 527 determines a first valence indicating the number of connected edges at the first vertex v1. According to an embodiment of this disclosure, if the first vertex v1 belongs to the base mesh, the wavelet inverse transform 527 can determine the first valence from an array of predetermined valences based on the index of the first vertex v1. The array of predetermined valences requires memory and much prior computation. According to an embodiment of this disclosure, if the first vertex v1 does not belong to the base mesh, the wavelet inverse transform 527 can determine that the first valence is equal to a predetermined value. According to an embodiment of this disclosure, the predetermined value can be equal to 6. According to an embodiment of this disclosure, if the first vertex v1 belongs to an edge belonging to the boundary of the reconstructed base mesh, the predetermined value can be equal to 4. According to an embodiment of this disclosure, the predetermined value can be equal to a value fixed for all levels of detail. According to an embodiment of this disclosure, the predetermined value can be equal to a value fixed for each level of detail. Since no memory and prior computation are required when the first vertex v1 does not belong to the base mesh, computational and memory requirements can be significantly reduced.

[0221] At 1407, the wavelet inverse transform 527 determines a second valence indicating the number of connected edges at the second vertex v2. According to an embodiment of this disclosure, if the second vertex v2 belongs to the base mesh, the wavelet inverse transform 527 can determine the second valence from an array of predetermined valences based on the index of the second vertex v2. The array of predetermined valences requires memory and much prior computation. According to an embodiment of this disclosure, if the second vertex v2 does not belong to the base mesh, the wavelet inverse transform 527 can determine that the first valence is equal to a predetermined value. According to an embodiment of this disclosure, the predetermined value can be equal to 6. According to an embodiment of this disclosure, the predetermined value can be equal to 4, wherein the second vertex v2 belongs to edges belonging to the boundaries of the reconstructed base mesh. According to an embodiment of this disclosure, the predetermined value can be equal to a value fixed for all levels of detail. According to an embodiment of this disclosure, the predetermined value can be equal to a value fixed for each level of detail. Since no memory and prior computation are required when the second vertex v2 does not belong to the base mesh, computational and memory requirements can be significantly reduced.

[0222] At 1409, the inverse wavelet transform 527 updates the displacement wavelet coefficients of the first vertex v1 based on the first valence, updated weights, and the displacement wavelet coefficients of the vth vertex in the array dispCoeffArray to generate updated displacement wavelet coefficients of the first vertex v1, and stores the updated displacement wavelet coefficients of the first vertex v1 in the array dispCoeffArray.

[0223] In 1411, the wavelet inverse transform 527 updates the displacement wavelet coefficients of the second vertex v2 based on the second valence, updated weights and displacement wavelet coefficients of the vth vertex in the array dispCoeffArray to generate updated displacement wavelet coefficients of the second vertex v2, and stores the updated displacement wavelet coefficients of the second vertex v2 in the array dispCoeffArray.

[0224] Figure 15 This is a flowchart illustrating the prediction processing operation of the wavelet inverse transform 527 of the V-DMC decoder 500 according to an embodiment.

[0225] Reference Figure 15 The inverse wavelet transform 527 determines the displacement.

[0226] In 1501, the wavelet inverse transform 527 determines the prediction weights based on the second syntax element.

[0227] In 1503, the inverse wavelet transform 527 determines the first vertex v1 and the second vertex v2 that form the edge to which the v-th vertex belongs.

[0228] In 1505, the inverse wavelet transform 527 determines the predicted displacement wavelet coefficient of the v-th vertex based on the predicted weights, the updated displacement wavelet coefficients of the first vertex v1 in the array of updated displacement wavelet coefficients dispCoeffArray, and the updated displacement wavelet coefficients of the second vertex v2 in the array of updated displacement wavelet coefficients dispCoeffArray.

[0229] In 1507, the inverse wavelet transform 527 determines the displacement of the v-th vertex based on the predicted displacement wavelet coefficients of the v-th vertex and the displacement wavelet coefficients of the v-th vertex in the array of updated displacement wavelet coefficients dispCoeffArray, so as to store the displacement of the v-th vertex in the array of displacements dispArray.

[0230] Various descriptive frames, units, modules, components, methods, operations, instructions, items, and algorithms can be implemented or executed using processing circuitry.

[0231] Unless otherwise specified, references to elements in the singular form do not imply one and only one, but rather one or more. For example, "module" can refer to one or more modules. Without further constraints, elements beginning with "a," "the," or "the" do not exclude the existence of other identical elements.

[0232] Titles and subtitles (where applicable) are used for convenience only and do not limit the technical scope of this subject matter. The term "exemplary" is used to indicate that it is used as an example or illustration. Where terms such as "comprising," "having," "carrying," "including," etc., are used, such terms are intended to be inclusive in a manner similar to the interpretation of the term "comprising" as a transitional word in the claims. Relational terms such as "first" and "second" can be used to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between these entities or actions.

[0233] Phrases such as aspect, that aspect, on the other hand, some aspects, one or more aspects, implementation, that implementation, another implementation, some implementations, one or more implementations, embodiment, that embodiment, another embodiment, some embodiments, one or more embodiments, configuration, that configuration, another configuration, some configurations, one or more configurations, subject matter, disclosure, this disclosure, other variations thereof, etc., are for convenience and do not imply that the disclosure associated with such phrases is essential to the subject matter, or that such disclosure applies to all configurations of the subject matter. The disclosure associated with such phrases may apply to all configurations or one or more configurations. The disclosure associated with such phrases may provide one or more examples. Phrases such as aspect or some aspects may refer to one or more aspects, and this similarly applies to other foregoing phrases.

[0234] The phrase "at least one" preceding a series of items (any of which are separated by the terms "and" or "or") modifies the entire list, not each member of the list. The phrase "at least one of..." does not require the selection of at least one item; rather, it allows for the inclusion of at least one item from any given list, and / or at least one item from any combination of items, and / or at least one item from each of the items. For example, each of the phrases "at least one of A, B, and C" or "at least one of A, B, or C" refers to: only A, only B, or only C; any combination of A, B, and C; and / or at least one of each of A, B, and C.

[0235] It should be understood that the specific order or hierarchy of the disclosed processes, operations, or procedures is an illustration of exemplary methods. Unless otherwise expressly stated, it should be understood that the specific order or hierarchy of processes, operations, or procedures may be performed in a different order. Some processes, operations, or procedures may be performed simultaneously or as part of one or more other processes, operations, or procedures. The appended method claims (where applicable) present the elements of various processes, operations, or procedures in a sample order and are not intended to limit one to the specific order or hierarchy presented. These may be performed serially, linearly, in parallel, or in different orders. It should be understood that the described instructions, operations, and systems may generally be integrated together in a single software / hardware product or packaged into multiple software / hardware products.

[0236] This disclosure is provided to enable any person skilled in the art to practice the various aspects described herein. In some cases, well-known structures and components are shown in block diagram form to avoid obscuring the concept of the subject matter. This disclosure provides various examples of the subject matter, and the subject matter is not limited to these examples. Various modifications to these aspects will be apparent to those skilled in the art, and the principles described herein can be applied to other aspects.

[0237] All structural and functional equivalents of elements of the various aspects described throughout this disclosure that are known or will be known by one of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, whether or not the contents of this disclosure are expressly recited in the claims, they are not intended to be offered to the public. No claim element should be interpreted in accordance with the provisions of paragraph 6 of 35 U.S.SC § 112 unless the element is expressly recited using the phrase “apparatus for…” or, in the case of a method claim, using the phrase “process for…”.

[0238] The title, background art, description of the drawings, abstract, and figures are incorporated herein by reference and are provided as illustrative examples rather than limiting descriptions. These are provided so that they are not intended to limit the scope or meaning of the claims. Furthermore, in the detailed description, illustrative examples may be provided, and various features may be grouped together in various implementations for the purpose of simplifying the disclosure. The disclosed methods should not be construed as reflecting an intention to require more features than expressly recited in each claim. Rather, as reflected in the following claims, the inventive subject matter exists in a subset of all features of a single disclosed configuration or operation. The following claims are incorporated herein by reference, wherein each claim itself is a separately claimed subject matter.

[0239] The embodiments are provided merely as examples for understanding the invention. The embodiments are not intended and should not be construed as limiting the scope of the invention in any way. Although specific embodiments and examples have been provided, those skilled in the art will understand from the disclosure herein that changes can be made to the illustrated embodiments and examples without departing from the scope of the invention.

[0240] The claims are not intended to be limited to the aspects described herein, but rather to encompass the full scope consistent with the wording of the claims and to cover all legal equivalents. Nevertheless, no claim is intended to include subject matter that does not comply with applicable patent law, nor should it be interpreted in this manner.

Claims

1. An apparatus for decoding video, the apparatus comprising: The communication interface is configured to receive (1301) a compressed bit stream comprising a base grid sub-bit stream and a displacement sub-bit stream; as well as A processor, operably coupled to the communication interface; the processor is configured to: The base grid sub-bitstream is decoded to reconstruct the base grid. The base mesh is subdivided once or multiple times to generate a subdivided mesh that includes multiple vertices. The shifted sub-bit stream is decoded (1303) to reconstruct the shifted wavelet coefficients for the plurality of vertices. The displacement wavelet coefficients are used to perform an update process (1311) to generate updated displacement wavelet coefficients for the plurality of vertices. During the update process, the processor is configured to: Based on whether the first vertex forming the edge to which the current vertex belongs belongs to the base mesh, a first quotient indicating the number of connecting edges at the first vertex is determined, and The displacement wavelet coefficients of the first vertex are updated based on the first valence. The updated displacement wavelet coefficients are used to perform a (1313) prediction process to generate the reconstructed displacement, and Based on the reconstructed displacement and the subdivided mesh, reconstruct (1315) mesh frames.

2. The device as claimed in claim 1, wherein, If the vertex does not belong to the base mesh, the first price is determined to be equal to a predetermined value.

3. The device as claimed in claims 1 to 2, wherein, If the first vertex belongs to the base grid, the first price is determined from the array of predetermined prices based on the index of the first vertex.

4. The device as claimed in claims 1 to 2, wherein, The predetermined value is 6.

5. The device as claimed in claims 1 to 4, wherein, During the update process, the processor is also configured to: Based on whether the second vertex forming the edge to which the current vertex belongs belongs to the base mesh, a second valence indicating the number of connecting edges at the second vertex is determined, and The displacement wavelet coefficients of the second vertex are updated based on the second valence.

6. The device as claimed in claims 1 to 5, wherein, The update weight is determined based on the first syntax element, and Based on the first price and the updated weight, the displacement wavelet coefficients of the first vertex are updated.

7. The device as claimed in claims 1 to 6, wherein: The compressed bitstream further includes a graph atlas sub-bitstream, which includes a first syntax element for determining update weights. The update weight is determined based on the first syntax element in the bitstream, and Based on the first price and the updated weight, the displacement wavelet coefficients of the first vertex are updated.

8. The device as claimed in claims 1 to 7, wherein: The compressed bitstream also includes a graph atlas sub-bitstream, which includes second syntax elements for determining prediction weights. The prediction weights are determined based on the second syntax element in the bitstream, and The displacement of the current vertex is determined based on the predicted weights and the updated displacement wavelet coefficients of the first vertex.

9. A method for decoding video, the method comprising: Receive (1301) a compressed bitstream including a basic grid sub-bitstream and a displacement sub-bitstream. The base grid sub-bitstream is decoded to reconstruct the base grid. The base mesh is subdivided once or multiple times to generate a subdivided mesh that includes multiple vertices. The shifted sub-bit stream is decoded (1303) to reconstruct the shifted wavelet coefficients for the plurality of vertices. The displacement wavelet coefficients are used to perform an update process (1311) to generate updated displacement wavelet coefficients for the plurality of vertices. The operation of performing the update process described in (1311) includes: Based on whether the first vertex forming the edge to which the current vertex belongs belongs to the base mesh, a first quotient indicating the number of connecting edges at the first vertex is determined, and The displacement wavelet coefficients of the first vertex are updated based on the first valence. The updated displacement wavelet coefficients are used to perform a (1313) prediction process to generate the reconstructed displacement, and Based on the reconstructed displacement and the subdivided mesh, reconstruct (1315) mesh frames.

10. The method of claim 9, wherein, If the vertex does not belong to the base mesh, the first price is determined to be equal to a predetermined value.

11. The method as claimed in claims 9 to 10, wherein, If the first vertex belongs to the base grid, the first price is determined from the array of predetermined prices based on the index of the first vertex.

12. The method as claimed in claims 9 to 11, wherein, The predetermined value is 6.

13. The method as claimed in claims 9 to 12, in, The operation of performing the update process described in (1311) further includes: Based on whether the second vertex forming the edge to which the current vertex belongs belongs to the base mesh, a second valence indicating the number of connecting edges at the second vertex is determined, and The displacement wavelet coefficients of the second vertex are updated based on the second valence.

14. The method as claimed in claims 9 to 13, wherein: The compressed bitstream includes a graph atlas sub-bitstream, which includes a first syntax element for determining update weights. The update weight is determined based on the first syntax element in the bitstream, and Based on the first price and the updated weight, the displacement wavelet coefficients of the first vertex are updated.

15. An apparatus for encoding video, the apparatus comprising: Communication interface; as well as A processor, operably coupled to the communication interface; The processor is configured to: Rebuild the base grid. The base mesh is subdivided once or multiple times to generate a subdivided mesh that includes multiple vertices. Determine the displacement (1001). The displacement is used to perform a (1003) prediction process to generate displacement wavelet coefficients. The displacement wavelet coefficients are used to perform an update process (1007) to generate updated displacement wavelet coefficients. During the update process, the processor is configured to: Based on whether the first vertex forming the edge to which the current vertex belongs belongs to the base mesh, a first quotient indicating the number of connecting edges at the first vertex is determined, and The displacement wavelet coefficients of the first vertex are updated based on the first valence. The updated displacement wavelet coefficients are encoded (1013) to generate a compressed displacement bitstream, and The transmission (1015) includes the compressed bit stream of the compressed shift bit stream.

Images