Signaling of zero coefficients for displacement coding

Abstract

An apparatus for mesh decoding is provided. The apparatus includes processing circuitry. The processing circuit is used for receiving a code stream, and the code stream comprises displacement information of a grid in a current grid frame and basic grid information. The displacement information indicates at least two displacements associated with the base grid and the grid. The base grid includes a subset of at least two vertices of the grid. The processing circuit is used for determining the last non-zero quantized wavelet coefficient in the at least two non-zero quantized wavelet coefficients in the coefficient array associated with the at least two shifts based on the index information included in the code stream. The processing circuitry is to reconstruct the grid based on the at least two displacements indicated by the at least two non-zero quantized wavelet coefficients.

Description

Incorporation by Reference

[0001] This application claims priority to U.S. Patent Application No. 18 / 644,022, filed on April 23, 2024, entitled “Signaling Zero Coefficients For Displacement Coding,” which claims priority to U.S. Provisional Application No. 63 / 461,551, filed on April 24, 2023, entitled “Signaling Zero Coefficients for Displacement Coding in Mesh Compression,” and U.S. Provisional Application No. 63 / 462,934, filed on April 28, 2023, entitled “Signaling Zero Coefficients for Displacement Coding.” The entire disclosures of these prior applications are incorporated herein by reference in their entirety. Technical Field

[0002] The description of this application generally relates to aspects of trellis coding. Background Art

[0003] The background description provided herein is for the purpose of generally presenting the context of the present application. The work of the presently named inventors, including the work described in the background section and aspects of this specification that may not be prior art at the time of filing, is neither explicitly nor implicitly admitted to be prior art to the present application.

[0004] Image / video compression can help to transmit image / video data across different devices, storage devices and networks with minimal quality degradation. In some instances, video codec technology can compress video based on spatial and temporal redundancy. In an example, a video codec can use a technique called intra-frame prediction, which can compress images based on spatial redundancy. For example, intra-frame prediction can use reference data from the current image being reconstructed for sample prediction. In another instance, a video codec can use a technique called inter-frame prediction, which can compress images based on temporal redundancy. For example, inter-frame prediction can use previously reconstructed images and motion compensation to predict samples in the current image. Motion compensation can be indicated by a motion vector (MV).

[0005] Advances in three-dimensional (3D) filming, modeling, and rendering have facilitated the use of 3D content across a variety of platforms and devices. For example, a baby's first steps are filmed in one continent, while grandparents in another can see (and in some cases interact with) and enjoy a fully immersive experience with the child. To achieve this realism, models have become more complex, and the creation and use of these models involves large amounts of data. 3D meshes are widely used to render this immersive content. Summary of the invention

[0006] Aspects of the present application include code streams, methods, and apparatus for trellis encoding / decoding. In some examples, an apparatus for trellis encoding / decoding includes a processing circuit.

[0007] According to one aspect of the present application, a device for grid decoding is provided. The device includes a processing circuit. The processing circuit is used to receive a code stream, which includes displacement information and basic grid information of a grid in a current grid frame. The displacement information indicates at least two displacements associated with the basic grid and the grid. The basic grid includes a subset of at least two vertices of the grid. The processing circuit is configured to determine the last non-zero quantized wavelet coefficient of at least two non-zero quantized wavelet coefficients in a coefficient array associated with at least two displacements based on index information included in the code stream. The processing circuit is used to reconstruct the grid based on at least two displacements indicated by the at least two non-zero quantized wavelet coefficients.

[0008] In an example, the index information identifies the index of the last non-zero quantized wavelet coefficient in the coefficient array. The processing circuit is configured to determine the last non-zero quantized wavelet coefficient in the coefficient array associated with the index identified by the index information.

[0009] In an example, the processing circuit is used to determine the index of the second to last non-zero quantized wavelet coefficient based on the index information. The processing circuit is used to determine the index difference between the second to last non-zero quantized wavelet coefficient and the last non-zero quantized wavelet coefficient in the coefficient array based on the index information. The processing circuit is used to determine the index of the last non-zero quantized wavelet coefficient in the coefficient array as the sum of the index of the second to last non-zero quantized wavelet coefficient and the index difference.

[0010] In an example, the processing circuit is used to determine the index of the second to last non-zero quantized wavelet coefficient based on the index information. The processing circuit is used to determine the index parameter based on the index information, the index parameter being equal to the index difference minus one. The index difference is the position difference between the second to last non-zero quantized wavelet coefficient and the last non-zero quantized wavelet coefficient in the coefficient array. The processing circuit is used to determine the index of the last non-zero quantized wavelet coefficient in the coefficient array based on the index of the second to last non-zero quantized wavelet coefficient and the index parameter.

[0011] In an example, the index information identifies the second to last non-zero quantized wavelet coefficient in the coefficient array. The processing circuit is configured to determine the last non-zero quantized wavelet coefficient in the coefficient array as zero.

[0012] In an example, the processing circuit is used to determine the index of the first non-zero quantized wavelet coefficient of the last at least two non-zero quantized wavelet coefficients in the coefficient array based on the index information. The last at least two non-zero quantized wavelet coefficients include the first non-zero quantized wavelet coefficient and at least one subsequent non-zero quantized wavelet coefficient. The processing circuit is used to determine the index parameter of each of the at least one subsequent non-zero quantized wavelet coefficient based on the index information. The index parameter of the corresponding subsequent non-zero quantized wavelet coefficient is equal to the index difference minus one. The index difference is the position difference between the corresponding subsequent non-zero quantized wavelet coefficient and the previous non-zero quantized wavelet coefficient of the subsequent non-zero quantized wavelet coefficient in the coefficient array. The processing circuit is used to determine the index of each of the at least one subsequent non-zero quantized wavelet coefficient based on the index of the first non-zero quantized wavelet coefficient and the index parameter of the corresponding subsequent non-zero quantized wavelet coefficient.

[0013] In an example, the processing circuit is used to determine the index of the first non-zero quantized wavelet coefficient of the last at least two non-zero quantized wavelet coefficients in the coefficient array based on the index information. The processing circuit is used to determine the index parameter of each wavelet coefficient in the subset of the last at least two non-zero quantized wavelet coefficients based on the index information. The subset of the last at least two non-zero quantized wavelet coefficients includes at least two subsequent non-zero quantized wavelet coefficients of the first non-zero quantized wavelet coefficient. The index parameter of each wavelet coefficient in the at least two subsequent non-zero quantized wavelet coefficients is equal to the index difference minus one. The index difference is the position difference between the corresponding subsequent non-zero quantized wavelet coefficient and the previous non-zero quantized wavelet coefficient of the subsequent non-zero quantized wavelet coefficient in the coefficient array. The processing circuit is used to determine the index of each wavelet coefficient in the subset of the last at least two non-zero quantized wavelet coefficients based on the index of the first non-zero quantized wavelet coefficient and the index parameter of the corresponding subsequent non-zero quantized wavelet coefficient.

[0014] According to another aspect of the present application, a method for mesh encoding is provided. In the method, at least two displacements of a mesh in a current mesh frame are determined. The at least two displacements are associated with a base mesh and a mesh. The base mesh includes a subset of at least two vertices of the mesh. A wavelet transform is performed on the at least two displacements to generate at least two wavelet coefficients. The at least two wavelet coefficients are quantized to generate a coefficient array including at least two non-zero quantized wavelet coefficients. The last non-zero quantized wavelet coefficient is determined in the coefficient array. Index information is encoded based on the last non-zero quantized wavelet coefficient of the coefficient array in the code stream.

[0015] In an example, to encode the index information, the index of the last non-zero quantized wavelet coefficient in the coefficient array is encoded.

[0016] In an example, to determine the last non-zero quantized wavelet coefficient, the last non-zero quantized wavelet coefficient and the second to last non-zero quantized wavelet coefficient are determined in the coefficient array. To encode the index information, the index of the second to last non-zero quantized wavelet coefficient in the coefficient array is encoded. The index difference between the second to last non-zero quantized wavelet coefficient and the last non-zero quantized wavelet coefficient in the coefficient array is encoded.

[0017] In the example, to determine the last non-zero quantized wavelet coefficient, the last non-zero quantized wavelet coefficient and the second to last non-zero quantized wavelet coefficient in the coefficient array are determined. To encode the index information, the index of the second to last non-zero quantized wavelet coefficient in the coefficient array is encoded. An index parameter equal to the index difference minus one is encoded. The index difference is the position difference between the second to last non-zero quantized wavelet coefficient and the last non-zero quantized wavelet coefficient in the coefficient array.

[0018] In an example, to determine the last non-zero quantized wavelet coefficient, the last non-zero quantized wavelet coefficient is determined to be zero. The second to last non-zero quantized wavelet coefficient in the coefficient array is determined. To encode index information, the index of the second to last non-zero quantized wavelet coefficient in the coefficient array is encoded.

[0019] In the example, in order to determine the last non-zero quantized wavelet coefficient, the last at least two non-zero quantized wavelet coefficients in the coefficient array are determined. The last at least two non-zero quantized wavelet coefficients include the first non-zero quantized wavelet coefficient and at least one subsequent non-zero quantized wavelet coefficient. In order to encode the index information, the index of the first non-zero quantized wavelet coefficient in the last at least two non-zero quantized wavelet coefficients in the coefficient array is encoded. The index parameter of each wavelet coefficient in at least one subsequent non-zero quantized wavelet coefficient is encoded. The index parameter of the corresponding subsequent non-zero quantized wavelet coefficient is equal to the index difference minus one. The index difference is the position difference between the corresponding subsequent non-zero quantized wavelet coefficient in the coefficient array and the previous non-zero quantized wavelet coefficient of the corresponding subsequent non-zero quantized wavelet coefficient.

[0020] In an example, to determine the last non-zero quantized wavelet coefficient, the last at least two non-zero quantized wavelet coefficients in the coefficient array are determined. The last at least two non-zero quantized wavelet coefficients include the first non-zero quantized wavelet coefficient and a subset of the last at least two non-zero quantized wavelet coefficients. The subset of the last at least two non-zero quantized wavelet coefficients includes at least two subsequent non-zero quantized wavelet coefficients of the first non-zero quantized wavelet coefficient. To encode index information, the index of the first non-zero quantized wavelet coefficient of the at least two last non-zero quantized wavelet coefficients in the coefficient array is encoded. The index parameter of each of the at least two subsequent non-zero quantized wavelet coefficients in the subset of the last at least two non-zero quantized wavelet coefficients is encoded. The index parameter of each of the at least two subsequent non-zero quantized wavelet coefficients is equal to the index difference minus one. The index difference is the position difference between a corresponding subsequent non-zero quantized wavelet coefficient and a previous non-zero quantized wavelet coefficient of the corresponding subsequent non-zero quantized wavelet coefficient in the coefficient array.

[0021] According to another aspect of the present application, a method for mesh data processing is provided. In the method, a code stream of mesh data is processed according to a format rule. In an example, the code stream includes displacement information and base mesh information of a mesh in a current mesh frame. The displacement information indicates at least two displacements associated with a base mesh and a mesh. The base mesh includes a subset of at least two vertices of the mesh. The format rule specifies that the last non-zero quantized wavelet coefficient of at least two non-zero quantized wavelet coefficients in a coefficient array associated with at least two displacements is determined based on index information included in the code stream. The format rule specifies that a mesh is processed based on at least two displacements indicated by at least two non-zero quantized wavelet coefficients.

[0022] Aspects of the present application also provide an apparatus for trellis coding. The apparatus for trellis coding comprises a processing circuit configured to implement any of the described methods for trellis coding.

[0023] Aspects of the present application also provide a method for trellis decoding, which includes implementing any one of the methods by a trellis decoding apparatus.

[0024] Aspects of the present application also provide a non-transitory computer-readable medium storing instructions that, when executed by a computer, cause the computer to perform any of the described methods for trellis decoding / encoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Other features, properties and various advantages of the subject matter of the present application will become more apparent from the following detailed description and accompanying drawings, in which:

[0026] Figure 1 is a schematic diagram of a block diagram of an example communication system (100).

[0027] Figure 2 is a schematic diagram of a block diagram of an example decoder.

[0028] Figure 3 is a schematic diagram of a block diagram of an encoder example.

[0029] Figure 4 It is a schematic diagram of an example of an encoding method (400) for grid processing according to one aspect of the present application.

[0030] Figure 5 is a schematic diagram of an example of a pre-processing step (500) according to one aspect of the present application.

[0031] Figure 6 It is a schematic diagram of a decoding method (600) for grid processing according to one aspect of the present application.

[0032] Figure 7 is a schematic diagram of an example of a subdivision scheme according to one aspect of the present application.

[0033] Figure 8 An overview flow chart of a trellis decoding method according to some aspects of the present application is shown.

[0034] Fig. 9 An overview flow chart of a trellis encoding method for some aspects of the present application is shown.

[0035] Fig.10 is a schematic diagram of a computer system on the one hand. DETAILED DESCRIPTION

[0036] Figure 1 A block diagram of a video processing system (100) in some examples is shown. The video processing system (100) is an example of the application of the subject matter of the present application, a video encoder and a video decoder in a streaming environment. The subject matter disclosed in the present application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, streaming services, storing compressed video on digital media including CDs, DVDs, memory sticks, etc., and the like.

[0037] The video processing system may include an acquisition subsystem (113), which may include a video source (101). The video source (101) may include one or more images captured by a camera and / or generated by a computer. For example, a digital camera is used to create, for example, an uncompressed video image stream (102). In an embodiment, the video picture stream (102) includes samples captured by the digital camera. Compared to the encoded video data (104) (or the encoded video bitstream), the video picture stream (102) is depicted as a thick line to emphasize the high data volume of the video picture stream, and the video picture stream (102) can be processed by an electronic device (120), which includes a video encoder (103) coupled to the video source (101). The video encoder (103) may include hardware, software, or a combination of hardware and software to implement or implement various aspects of the disclosed subject matter as described in more detail below. Compared to the video picture stream (102), the encoded video data (104) (or the encoded video bitstream (104)) is depicted as a thin line to emphasize the lower amount of data of the encoded video data (104) (or the encoded video bitstream (104)), which can be stored on the streaming server (105) for future use. One or more streaming client subsystems, such as Figure 1 A client subsystem (106) and a client subsystem (108) in a streaming server (105) may access a streaming server (105) to retrieve a copy (107) and a copy (109) of the encoded video data (104). The client subsystem (106) may include, for example, a video decoder (110) in an electronic device (130). The video decoder (110) decodes the incoming copy (107) of the encoded video data and generates an output video picture stream (111) that can be presented on a display (112) (e.g., a display screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (104), the video data (107), and the video data (109) (e.g., a video bitstream) may be encoded according to certain video encoding / compression standards. Examples of such standards include ITU-T H.265. In an embodiment, the video coding standard under development is informally referred to as Versatile Video Coding (VVC), and the present application may be used in the context of the VVC standard.

[0038] It should be noted that the electronic device (120) and the electronic device (130) may include other components (not shown). For example, the electronic device (120) may include a video decoder (not shown), and the electronic device (130) may also include a video encoder (not shown).

[0039] Figure 2is a block diagram of an example of a video decoder (210). The video decoder (210) may be provided in an electronic device (230). The electronic device (230) may include a receiver (231) (eg, a receiving circuit). The video decoder (210) may be used to replace Figure 1 A video decoder (110) of an embodiment.

[0040] The receiver (231) may receive one or more encoded video sequences (e.g., in a bitstream) to be decoded by the video decoder (210). In one aspect, the encoded video sequences are received one at a time, wherein the decoding of each encoded video sequence is independent of the decoding of the other encoded video sequences. The encoded video sequence may be received from a channel (201), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (231) may receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that may be forwarded to their respective consuming entities (not shown). The receiver (231) may separate the encoded video sequence from the other data. To prevent network jitter, a buffer memory (215) may be coupled between the receiver (231) and the entropy decoder / parser (220) (hereinafter referred to as "parser (220)"). In some applications, the buffer memory (215) is part of the video decoder (210). In other cases, the buffer memory (215) may be set outside the video decoder (210) (not shown). In other cases, a buffer memory (not shown) is set outside the video decoder (210) to, for example, prevent network jitter, and another buffer memory (215) may be configured inside the video decoder (210) to, for example, handle broadcast timing. When the receiver (231) receives data from a storage / forward device with sufficient bandwidth and controllability or from an isochronous network, it may not be necessary to configure the buffer memory (215), or the buffer memory may be made smaller. Of course, in order to use on a service packet network such as the Internet, a buffer memory (215) may also be required. The buffer memory may be relatively large and may have an adaptive size, and may be at least partially implemented in an operating system or a similar element (not shown) outside the video decoder (210).

[0041] The video decoder (210) may include a parser (220) to reconstruct symbols (221) from the encoded video sequence. The types of symbols include information for managing the operation of the video decoder (210) and potential information for controlling a display device such as a display device (212) (e.g., a display screen) that is not part of the electronic device (230) but can be coupled to the electronic device (230), such as Figure 2As shown in . The control information for the display device may be a parameter set fragment (not indicated) of Supplemental Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (220) may parse / entropy decode the received coded video sequence. The encoding of the coded video sequence may be performed according to a video coding technique or standard, and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, and the like. The parser (220) may extract a subgroup parameter set of at least one subgroup of the subgroups of pixels in the video decoder from the coded video sequence based on at least one parameter corresponding to the group. The subgroup may include a Group of Pictures (GOP), a picture, a tile, a slice, a macroblock, a Coding Unit (CU), a block, a Transform Unit (TU), a Prediction Unit (PU), and the like. The parser (420) may also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, and so on.

[0042] The parser (220) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (215), thereby creating symbols (221).

[0043] Depending on the type of coded video picture or part of coded video picture (e.g., inter-picture and intra-picture, inter-block and intra-block) and other factors, the reconstruction of symbol (221) may involve multiple different units. Which units are involved and how they are involved may be controlled by subgroup control information parsed by parser (220) from the coded video sequence. For the sake of brevity, such subgroup control information flow between parser (220) and the multiple units below is not described.

[0044] In addition to the functional blocks already mentioned, the video decoder (210) can be conceptually subdivided into several functional units as described below. In a practical embodiment operating under commercial constraints, many of these units interact closely with each other and can be integrated with each other. However, for the purpose of describing the disclosed subject matter, the conceptual subdivision into the following functional units is appropriate.

[0045] The first unit is a sealer / inverse transform unit (251). The sealer / inverse transform unit (251) receives quantized transform coefficients as symbols (221) from the parser (220) and control information, including which transform method to use, block size, quantization factor, quantization scaling matrix, etc. The sealer / inverse transform unit (251) can output a block including sample values, which can be input into an aggregator (255).

[0046] In some cases, the output samples of the scaler / inverse transform unit (251) may belong to an intra-coded block; that is, a block that does not use predictive information from a previously reconstructed picture, but may use predictive information from a previously reconstructed portion of the current picture. Such predictive information may be provided by an intra-picture prediction unit (252). In some cases, the intra-picture prediction unit (252) generates surrounding blocks of the same size and shape as the block being reconstructed using reconstructed information extracted from a current picture buffer (258). For example, the current picture buffer (258) buffers a partially reconstructed current picture and / or a fully reconstructed current picture. In some cases, the aggregator (255) adds the prediction information generated by the intra-prediction unit (252) to the output sample information provided by the scaler / inverse transform unit (251) on a per-sample basis.

[0047] In other cases, the output samples of the scaler / inverse transform unit (251) may belong to an inter-frame coded and potentially motion compensated block. In this case, the motion compensated prediction unit (253) may access the reference picture memory (257) to extract samples for prediction. After the extracted samples are motion compensated according to the symbols (221), these samples may be added to the output of the scaler / inverse transform unit (251) (in this case referred to as residual samples or residual signals) by the aggregator (255) to generate output sample information. The acquisition of the predicted samples by the motion compensated prediction unit (253) from the address in the reference picture memory (257) may be controlled by a motion vector, and the motion vector is provided to the motion compensated prediction unit (253) in the form of the symbols (221), for example, including X, Y and reference picture components. Motion compensation may also include interpolation of sample values ​​extracted from the reference picture memory (257) when using sub-sample accurate motion vectors, motion vector prediction mechanisms, etc.

[0048] The output samples of the aggregator (255) may be used by various loop filtering techniques in a loop filter unit (256). The video compression techniques may include in-loop filter techniques that are controlled by parameters included in the encoded video sequence (also referred to as the encoded video bitstream) and that are available to the loop filter unit (256) as symbols (221) from the parser (220). However, in other embodiments, the video compression techniques may also be responsive to meta-information obtained during decoding of a previous (in decoding order) portion of an encoded picture or encoded video sequence, and to previously reconstructed and loop filtered sample values.

[0049] The output of the loop filter unit (256) may be a sample stream that may be output to a display device (212) and stored in a reference picture memory (257) for subsequent inter-picture prediction.

[0050] Once fully reconstructed, certain coded pictures may be used as reference pictures for future prediction. For example, once the coded picture corresponding to the current picture is fully reconstructed, and the coded picture is identified as a reference picture (e.g., by the parser (220)), the current picture buffer (258) may become part of the reference picture memory (257), and a new current picture buffer may be reallocated before starting to reconstruct a subsequent coded picture.

[0051] The video decoder (210) may perform decoding operations according to a predetermined video compression technique, such as in the ITU-T H.265 standard. The encoded video sequence may conform to the syntax specified by the video compression technique or standard used in the sense that the encoded video sequence follows the syntax of the video compression technique or standard and the profile recorded in the video compression technique or standard. Specifically, the profile may select certain tools from all the tools available in the video compression technique or standard as the only tools available for use under the profile. For compliance, the complexity of the encoded video sequence is also required to be within the range defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum picture size, the maximum frame rate, the maximum reconstruction sampling rate (measured in, for example, mega samples per second), the maximum reference picture size, etc. In some cases, the limits set by the hierarchy may be further limited by the Hypothetical Reference Decoder (HRD) specification and metadata of the HRD buffer management signaled in the encoded video sequence.

[0052] In one aspect, the receiver (231) may receive additional (redundant) data along with the encoded video. The additional data may be part of the encoded video sequence. The additional data may be used by the video decoder (210) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0053] Figure 3 is a block diagram of an example of a video encoder (303). The video encoder (303) is disposed in an electronic device (320). The electronic device (320) includes a transmitter (340) (eg, a transmission circuit). The video encoder (303) may be used to replace Figure 1 A video encoder (103) in an embodiment.

[0054] The video encoder (303) can be used to obtain the video source (301) (not Figure 3 In another embodiment, the video source (301) is a part of the electronic device (320) to receive video samples, and the video source can collect video images to be encoded by the video encoder (303). In another embodiment, the video source (301) is a part of the electronic device (320).

[0055] The video source (301) may provide a source video sequence in the form of a digital video sample stream to be encoded by the video encoder (303), wherein the digital video sample stream may have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service system, the video source (301) may be a storage device storing previously prepared videos. In a video conferencing system, the video source (301) may be a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual pictures that are given motion when viewed sequentially. The pictures themselves may be constructed as a spatial pixel array, wherein each pixel may include one or more samples depending on the sampling structure, color space, etc. used. The following description focuses on the samples.

[0056] According to one aspect, the video encoder (303) can encode and compress pictures of a source video sequence into an encoded video sequence (343) in real time or under any other time constraints required. Implementing the appropriate encoding speed is a function of the controller (350). In some aspects, the controller (350) controls other functional units as described below and is functionally coupled to these units. For the sake of brevity, the coupling is not shown in the figure. The parameters set by the controller (350) may include rate control related parameters (picture skipping, quantizer, lambda value of rate distortion optimization technology, etc.), picture size, group of pictures (group of pictures, GOP) layout, maximum motion vector search range, etc. The controller (350) can be used to have other suitable functions that are related to the video encoder (303) optimized for a certain system design.

[0057] In some aspects, the video encoder (303) operates in an encoding loop. As a simple description, in an embodiment, the encoding loop may include a source encoder (330) (e.g., responsible for creating symbols, such as a symbol stream, based on an input picture to be encoded and a reference picture) and a (local) decoder (333) embedded in the video encoder (303). The decoder (333) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data. The reconstructed sample stream (sample data) is input to a reference picture memory (334). Since the decoding of the symbol stream produces bit-accurate results that are independent of the decoder location (local or remote), the contents of the reference picture memory (334) are also bit-accurate between the local encoder and the remote encoder. In other words, the reference picture samples "seen" by the prediction part of the encoder are exactly the same as the sample values ​​that the decoder will "see" when using the prediction during decoding. This basic principle of reference picture synchronization (and the drift that occurs when synchronization cannot be maintained, such as due to channel errors) is also used in some related technologies.

[0058] The operation of the "local" decoder (333) may be combined with, for example, Figure 2 The "remote" decoder described in detail for the video decoder (210) is identical. However, additional brief reference is made to Figure 2 , when symbols are available and the entropy encoder (345) and parser (220) are capable of losslessly encoding / decoding the symbols into an encoded video sequence, the entropy decoding portion of the video decoder (210), including the buffer memory (215) and the parser (220), may not be fully implemented in the local decoder (333).

[0059] According to one aspect, any decoder technology other than parsing / entropy decoding present in the decoder is present in the corresponding encoder in the same or substantially the same functional form. Accordingly, the present application focuses on decoder operation. The description of encoder technology can be simplified because encoder technology is mutually inverse to the decoder technology described comprehensively. A more detailed description is needed in some areas and is provided below.

[0060] During operation, in some embodiments, the source encoder (330) may perform motion compensated predictive coding. The motion compensated predictive coding predictively encodes an input picture with reference to one or more previously encoded pictures from a video sequence designated as "reference pictures." In this manner, the encoding engine (332) encodes the differences between pixel blocks of the input picture and pixel blocks of a reference picture that may be selected as a prediction reference for the input picture.

[0061] The local video decoder (333) may decode the encoded video data of the picture that may be designated as the reference picture based on the symbol created by the source encoder (330). The operation of the encoding engine (332) may be a lossy process. When the encoded video data is available at the video decoder ( Figure 3 When the video sequence is decoded at a remote location (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (333) replicates the decoding process that may be performed by the video decoder on the reference picture and may cause the reconstructed reference picture to be stored in the reference picture memory (334). In this way, the video encoder (303) may locally store a copy of the reconstructed reference picture that has common content (absent transmission errors) with the reconstructed reference picture to be obtained by the remote video decoder.

[0062] The predictor (335) may perform a prediction search for the encoding engine (332). That is, for a new picture to be encoded, the predictor (335) may search the reference picture memory (334) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference picture motion vectors, block shapes, etc., that may serve as appropriate prediction references for the new picture. The predictor (335) may operate pixel-by-pixel based on sample blocks to find a suitable prediction reference. In some cases, based on the search results obtained by the predictor (335), it may be determined that the input picture may have prediction references taken from a plurality of reference pictures stored in the reference picture memory (334).

[0063] The controller (350) may manage encoding operations of the source encoder (330), including, for example, setting parameters and subgroup parameters for encoding video data.

[0064] The outputs of all the above functional units may be entropy encoded in an entropy encoder (345). The entropy encoder (345) performs lossless compression on the symbols generated by the various functional units according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc., thereby converting the symbols into a coded video sequence.

[0065] The transmitter (340) may buffer the encoded video sequence created by the entropy encoder (345) in preparation for transmission over a communication channel (360), which may be a hardware / software link to a storage device where the encoded video data will be stored. The transmitter (340) may combine the encoded video data from the video encoder (303) with other data to be transmitted, such as encoded audio data and / or ancillary data streams (source not shown).

[0066] The controller (350) may manage the operation of the video encoder (303). During encoding, the controller (350) may assign a certain coded picture type to each coded picture, but this may affect the coding techniques that can be applied to the corresponding picture. For example, a picture may generally be assigned to any of the following picture types:

[0067] An intra picture (I picture) may be a picture that can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh ("IDR") pictures.

[0068] A predictive picture (P picture) may be a picture that can be encoded and decoded using intra prediction or inter prediction, which uses at most one motion vector and a reference index to predict sample values ​​for each block.

[0069] Bidirectional predictive pictures (B pictures), which can be pictures that can be encoded and decoded using intra prediction or inter prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for reconstruction of a single block.

[0070] The source picture may typically be spatially subdivided into blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples), and coded block by block. These blocks may be predictively coded with reference to other (already coded) blocks, which are determined according to the coding allocation applied to the block's corresponding picture. For example, blocks of an I picture may be non-predictively coded, or the blocks may be predictively coded (spatial prediction or intra prediction) with reference to already coded blocks of the same picture. Blocks of pixels of a P picture may be predictively coded by spatial prediction with reference to one previously coded reference picture or by temporal prediction. Blocks of a B picture may be predictively coded by spatial prediction with reference to one or two previously coded reference pictures or by temporal prediction.

[0071] The video encoder (303) may perform encoding operations according to a predetermined video encoding technique or standard, such as ITU-T H.265 Recommendation. In operation, the video encoder (303) may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in an input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard used.

[0072] In an embodiment, the transmitter (340) may transmit additional data when transmitting the encoded video. The source encoder (330) may include such data as part of the encoded video sequence. The additional data may include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0073] The captured video may be taken as a plurality of source pictures (video pictures) in a temporal sequence. Intra-picture prediction (often simplified to intra-prediction) exploits spatial correlations in a given picture, while inter-picture prediction exploits (temporal or other) correlations between pictures. In an embodiment, a particular picture being encoded / decoded is divided into blocks, and the particular picture being encoded / decoded is referred to as the current picture. When a block in the current picture is similar to a reference block in a reference picture that was previously encoded in the video and is still buffered, the block in the current picture may be encoded by a vector called a motion vector. The motion vector points to a reference block in a reference picture, and in the case where multiple reference pictures are used, the motion vector may have a third dimension that identifies the reference picture.

[0074] In some aspects, bidirectional prediction techniques may be used in inter-picture prediction. According to the bidirectional prediction technique, two reference pictures are used, for example, a first reference picture and a second reference picture that are both before the current picture in the video in decoding order (but may be in the past and future in display order, respectively). A block in the current picture may be encoded by a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. Specifically, the block may be predicted by a combination of the first reference block and the second reference block.

[0075] In addition, merge mode technology can be used in inter-picture prediction to improve coding efficiency.

[0076] According to some aspects of the present application, predictions such as inter-picture prediction and intra-picture prediction are performed in units of blocks (e.g., polygonal or triangular blocks). For example, according to the HEVC standard, pictures in a video picture sequence are divided into coding tree units (CTUs) for compression, and the CTUs in the pictures have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. In general, a CTU includes three coding tree blocks (CTBs), which are a luminance CTB and two chrominance CTBs. Furthermore, each CTU can be split into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be split into a 64×64 pixel CU, or 4 32×32 pixel CUs, or 16 16×16 pixel CUs. In an embodiment, each CU is analyzed to determine a prediction type for the CU, such as an inter-prediction type or an intra-prediction type. In addition, depending on temporal and / or spatial predictability, the CU is split into one or more prediction units (PUs). Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In one aspect, the prediction operation in encoding (encoding / decoding) is performed in units of prediction blocks. Taking the luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, and the like.

[0077] It should be noted that the video encoder (103) and the video encoder (303) and the video decoder (110) and the video decoder (210) may be implemented using any suitable technology. In one aspect, the video encoder (103) and the video encoder (303) and the video decoder (110) and the video decoder (210) may be implemented using one or more integrated circuits. In another aspect, the video encoder (103) and the video encoder (303) and the video decoder (110) and the video decoder (210) may be implemented using one or more processors that execute software instructions.

[0078] Aspects of the present application include techniques for signaling zero coefficients for displacement encoding in trellis compression.

[0079] A mesh may include at least two polygons that describe the surface of a volumetric object. Each polygon of a mesh may be defined by the vertices of the polygon in a three-dimensional (3D) space and information about how the vertices are connected (which may be referred to as connectivity information). In some aspects, vertex attributes, such as color, normal, etc., may be associated with vertices (or mesh vertices). Mapping information may also be used to associate attributes (or vertex attributes) with the surface of a mesh, wherein the mapping information parameterizes the mesh using a two-dimensional (2D) attribute map. This mapping may be described by a set of parameter coordinates called UV coordinates or texture coordinates associated with mesh vertices. A 2D attribute map may be used to store high-resolution attribute information, such as textures, normals, displacements, etc. High-resolution attribute information may be used for various purposes, such as texture mapping and shading.

[0080] Dynamic mesh sequences may require a large amount of data because the dynamic mesh may include a large amount of information that changes over time. Therefore, efficient compression techniques can be used to store and transmit such content. Mesh compression standards, such as Information and Communication (IC) mesh compression, MESHGRID, and frame-based animated mesh compression (FAMC), were previously developed by the Moving Picture Experts Group (MPEG) to address dynamic meshes with continuous connectivity, time-varying geometry, and vertex attributes. However, these standards may not take into account time-varying attribute graphs and connectivity information. Digital Content Creation (DCC) tools can generate such dynamic meshes. However, for volume acquisition technology, generating continuous connectivity dynamic meshes may be challenging, especially when real-time is required. Existing standards may not support this type of content (e.g., continuous connectivity dynamic meshes). The present application includes various aspects for a new mesh compression standard that can directly handle dynamic meshes with time-varying connectivity information and possibly with time-varying attribute graphs. Mesh compression can target lossy and lossless compression for various applications such as real-time communications, storage, free viewpoint video, augmented reality (AR) and virtual reality (VR). Features such as random access and scalable / progressive coding can also be considered.

[0081] Figure 4 An example of an encoding method (400) based on mesh processing of a related video codec (such as MPEG V-Mesh TMv1.0) according to one aspect of the present application is shown. Figure 4As shown, the encoding method (400) may include a preprocessing step (400A) and an encoding step (400B). The preprocessing step (400A) may be configured to generate a base grid m(i) of the current frame and a displacement field d(i) of the current frame including a displacement vector according to an input grid M(i) of the current frame. The encoding step (400B) may be configured to encode the base grid m(i), the displacement field d(i) and the texture information of the base grid m(i). The displacement field d(i) of the current frame may include a displacement vector. The index i may refer to the current frame. In one aspect, a mode decision method may be performed in the encoding method (400) to determine whether the current frame applies inter-frame coding (also known as inter-frame prediction or inter-frame mode), intra-frame coding (also known as intra-frame prediction or intra-frame mode), etc. For example, the mode decision method may compare the cost of the intra-frame mode and the cost of the inter-frame mode, and determine the encoding mode of the base grid m(i) of the current frame based on which cost is smaller. In some examples, skip mode is used to encode or decode (e.g., encode or decode) the base grid m(i). In an example, skip mode is a special mode of inter mode. For example, the base grid m(i) can be encoded by intra coding, or inter coding, or skip mode.

[0082] Still reference Figure 4 , the preprocessing step (400A) may include a mesh simplification process (402), a parameterization process (404) (such as a graph parameterization process) and a subdivision surface fitting process (406). The mesh simplification process (402) is used to downsample the vertices of the input mesh M(i) to generate a simplified mesh dm(i), which may include at least two simplified (or downsampled) vertices. The number of at least two simplified vertices is less than the number of vertices of the input mesh M(i). The parameterization process (404) (such as a graph parameterization process) is used to map the simplified mesh dm(i) to a planar domain, such as mapping to a UV graph (or UV map) to generate a re-parameterized mesh pm(i). In an example, the graph parameterization can be performed based on a video processing tool (such as a UVAtlas tool). The subdivision surface fitting process (406) is used to take the reparameterized mesh pm(i) and the input mesh M(i) as input and generate a base mesh m(i) and a displacement field d(i) including a displacement vector or a set of displacements. In an example of the subdivision surface fitting process (406), pm(i) is subdivided to obtain a subdivided mesh using a subdivision scheme such as iterative interpolation. Iterative interpolation includes inserting a new point in the middle of each edge of the reparameterized mesh pm(i) in each iteration. Any suitable subdivision scheme can be applied to subdivide pm(i). The displacement field d(i) is calculated by determining the closest point on the surface of the input mesh M(i) to each vertex of the subdivided mesh.

[0083] Advantages of subdividing meshes may include that the subdivided mesh has a subdivision structure that allows efficient compression while faithfully approximating the input mesh. Improved compression efficiency may be obtained due to the following properties. The simplified mesh dm(i) may have a small number of vertices and may be encoded and transmitted using a smaller number of bits than the input mesh M(i) or the subdivided mesh. Figure 4 , a base mesh m(i) can be generated using a simplified mesh dm(i). In the example, the base mesh m(i) is a simplified mesh dm(i). Since the subdivided mesh can be generated based on the subdivision method, the decoder can automatically generate the subdivided mesh when decoding the base mesh or the simplified mesh (for example, without using any information other than the subdivision scheme and the subdivision iteration count). On the decoder side, the displacement field d(i) can be generated by decoding the displacement vectors associated with the vertices of the subdivided mesh. In addition to allowing spatial / quality scalability, the subdivision structure also allows efficient transforms, such as wavelet decomposition, which can provide high compression performance.

[0084] For simplicity, the preprocessing step (500) applied to a two-dimensional (2D) curve may be used to illustrate the preprocessing step (400A) that may be applied to an input mesh such as a 3D mesh. The preprocessing step (400A) and the preprocessing step (500) are similar, except that the 3D mesh may be replaced by a 2D curve.

[0085] Figure 5 An example of a preprocessing step (500) according to one aspect of the present application is shown. Figure 5As shown, an input 2D curve (502) (represented by a 2D polyline) can be downsampled to generate a base curve, such as a polyline, referred to as a "simplified" curve (504). A subdivision scheme can then be applied to the simplified polyline (504) to generate a "subdivided" curve (506). In an example, the subdivision scheme can be an iterative interpolation scheme. The iterative interpolation scheme can include inserting a new point in the middle of each edge of the polyline (or simplified curve) (504) at each iteration. For example, a point (510) can be inserted in an edge (508) of the simplified curve (504). In the example, the edge (508) is between a point (512) and a point (514). In addition, a point (522) can be added between the point (512) and the point (510), and a point (516) can be added between the point (510) and the point (514). The subdivided polyline (506) is then deformed to generate a shifted curve (518). The shifted curve (518) may be a better approximation of the input curve (502) than the subdivided curve (506). For example, a displacement vector (e.g., (520)) is calculated for each vertex (e.g., vertex (510)) of the subdivided curve (506) so that the shape of the shifted curve (518) is as close as possible to the shape of the input curve (502). An advantage of the subdivided curve (506) is that the subdivided curve (506) has a subdivision structure that allows for more efficient compression while faithfully approximating the input curve (502).

[0086] The simplified curve (504) may have a smaller number of points and may be encoded and transmitted using a limited number of bits. Since the subdivision curves may be generated based on the subdivision scheme, the decoder may automatically generate the subdivision curves when decoding the base curve or the simplified curve (e.g., without using any information other than the subdivision method and the subdivision iteration count). The shifted curves are generated by decoding the displacement vectors associated with the subdivision curve vertices. In addition to allowing spatial / quality scalability, the subdivision structure also allows efficient transforms, such as wavelet decomposition, which may provide high compression performance.

[0087] Still reference Figure 5 In an example, the input mesh M(i) may include an input 2D curve (502). The base mesh m(i) may include a simplified curve (504) formed by downsampling the vertices of the input 2D curve (502). The displacement field dm(i) may include at least two displacement vectors, such as Figure 5 The displacement vector (520) shown in .

[0088] The encoding step (400B) may include base mesh encoding (408), displacement encoding (410), texture encoding (412), etc. The base mesh encoding (408) is used to encode the geometric information of the base mesh m(i) associated with the current frame. In intra-frame coding, the base mesh m(i) may be first quantized (e.g., using uniform quantization) and then encoded (e.g., by using a coding mode determined by a mode decision method). The coding mode may be an inter-frame mode, an intra-frame mode, a skip mode, etc. The encoder used to perform intra-frame encoding on the base mesh m(i) may be referred to as a static mesh encoder. In inter-frame coding, a reference base mesh associated with a reference frame indicated by index j (e.g., a reconstructed and quantized reference base mesh m'(j)) may be used to predict the base mesh m(i) associated with the current frame indicated by index i. The displacement encoding (410) is used to encode the displacement field d(i) generated in the preprocessing step (400A). The displacement field d(i) may include a set of displacement vectors (or displacements) associated with subdivided mesh vertices. Texture encoding (412) is used to encode the attribute information of the base mesh m(i). The attribute information may include texture, normal, color, etc. The attribute information may be encoded based on a suitable codec such as High Efficiency Video Coding (HEVC) or Universal Video Coding (VVC).

[0089] In one aspect, reference Figure 4 , the mesh encoding method (such as the encoding process (400)) starts with preprocessing (e.g., preprocessing step (400A)). The preprocessing can convert the input mesh (e.g., input dynamic mesh) M(i) into a base mesh m(i) and a displacement field d(i) including a set of displacements (or a set of displacement vectors). The encoding step (400B) can compress the output of the preprocessing (e.g., m(i), d(i), etc.) and generate a compressed code stream b(i). The compressed code stream b(i) can include a compressed base mesh code stream, a compressed displacement field code stream, a compressed attribute code stream, etc.

[0090] Figure 6An example of a decoding method (600) for grid processing according to an aspect of the present application is shown. The decoding method (600) may include a decoding step (605) and a post-processing step (610). A compressed code stream b(i) may be fed into the decoding step (605). In the example, for example for lossless transmission, the compressed code stream b(i) is the output b(i) of the encoding process (400). The decoding step (605) may extract various sub-code streams, such as a compressed base grid sub-stream, a compressed displacement field sub-stream, a compressed attribute sub-stream, etc. The decoding step (605) may decompress these sub-code streams to generate the following components: patch metadata indicated by metadata(i), a decoded base grid m"(i), a decoded displacement field (including displacement) d"(i), a decoded attribute map A"(i), etc.

[0091] In one aspect, the base grid substream can be fed into a grid decoder to generate a reconstructed quantized base grid m'(i). The decoded base grid (or reconstructed base grid) m"(i) can be obtained by applying inverse quantization to m'(i). The displacement field substream, including the encoded packed and quantized wavelet coefficients, can be decoded by a video and / or image decoder. Image unpacking and inverse quantization can be applied to the packed and quantized wavelet coefficients to reconstruct the wavelet coefficients to obtain unpacked and unquantized transform coefficients (e.g., wavelet coefficients). An inverse wavelet transform can be applied to the unpacked and unquantized wavelet coefficients to generate a decoded displacement field (or reconstructed displacement) d"(i).

[0092] The decoded components (e.g., including metadata(i), m”(i), d”(i), A”(i), etc.) may be fed into a post-processing step (610). A mesh (also referred to as a decoded / reconstructed mesh) M”(i) may be generated by the post-processing step (610) based on m”(i) and d”(i). In an example, the mesh M”(i) (also referred to as a reconstructed deformed mesh DM(i)) may be obtained by subdividing m”(i) using a subdivision scheme and applying reconstructed displacements d”(i) to the vertices of the subdivided mesh. In an example, DM(i) may include a shifted curve (518). In an example, when the encoding method (400), the decoding method (600), and the transmission are lossless, the mesh M”(i) may be the same as the input mesh M(i). When one of the encoding process (400), the decoding process (600), and the transmission is lossy, M”(i) is different from M(i). In various examples, the difference (if any) between M”(i) and M(i) may be relatively small. In the example, the attribute graph A"(i) is also generated by the post-processing step (610).

[0093] In one aspect, the base grid may be intra-coded, inter-coded, or encoded with a skip mode, etc. In an example, the skip mode may be a special mode of the inter-mode, in which the base grid m(i) of the current frame indicated by index i is the same as the base grid m(j) of the reference frame indicated by index (also referred to as frame index) j. When the inter-mode is applied to encode the base grid in the current frame, the encoder may generate a predicted base grid for the current frame based on the reconstructed base grid of the reference frame. In an example, as in MPEG V-DMC WD 2.0, the reference frame is a frame immediately preceding the current frame in display order. The frame index i of the current frame indicates the display order. When the frame index of the current frame is i, the frame index of the reference frame is (i-1). In an example, the current frame and the reference frame are in the same group of frames (GoF).

[0094] In one aspect, for example in MPEG V-DMC WD 2.0, the mesh coding method starts with preprocessing. The preprocessing can convert the input dynamic mesh (denoted as M(i)) into a base mesh m(i) and a set of displacements d(i). The encoder can compress the base mesh m(i) and the displacements d(i) to generate a compressed code stream b(i).

[0095] Preprocessing can include mesh simplification, followed by atlas parameterization, and then subdivision surface fitting, as shown in Figure 4 As shown. Mesh simplification can use simplification techniques to simplify the input mesh M(i) and produce a simplified mesh dm(i). The simplified mesh dm(i) can then be reparameterized. The resulting mesh can be denoted as pm(i). Subdivision surface fitting can take the reparameterized mesh pm(i) and the input mesh M(i) as input and produce a base mesh m(i) with the set of displacements d(i).

[0096] An example of subdivision surface fitting is given in Figure 7 As shown in Figure 7 As shown, the re-parameterized mesh pm(i) (702) can be subdivided by applying a subdivision method (such as a midpoint subdivision scheme). The midpoint subdivision scheme can subdivide each triangle into 4 sub-triangles at each subdivision iteration, such as Figure 7 For example, in the initial iteration S 0 At the first iteration S, the reparameterized mesh pm(i) (702) may include two triangles (701) and (703). 1 At least two vertices, such as vertices (704) and (706), can be generated according to the midpoint subdivision scheme. Accordingly, triangle (701) is subdivided into four smaller triangles, and triangle (703) is also subdivided into four smaller triangles. In the second iteration S 2At , at least two vertices, such as vertices (708) and (710), can be generated according to the midpoint subdivision scheme. 1 Each triangle formed at the second iteration S 2 The time is subdivided into 4 triangles.

[0097] The displacement field d(i) may be computed by determining the closest point on the surface of the original (or input) mesh M(i) to each vertex of the subdivided mesh.

[0098] The encoder may optionally encode a set of displacement vectors associated with the subdivided mesh vertices, referred to as the displacement field d(i). In an example, the reconstructed quantized base mesh m'(i) may be used to update the displacement field d(i) to generate an updated displacement field d'(i). A wavelet transform may then be applied to d'(i) and a set of wavelet coefficients may be generated. The wavelet coefficients may then be quantized and packed into a 2D image / video. The quantized and packed wavelet coefficients may be compressed using arithmetic coding, a conventional image / video encoder, or any other encoder.

[0099] After quantization, there may be many zeros in the quantized wavelet coefficients.In the present application, a method and system for signaling displacement-coded zero coefficients in grid compression are provided.

[0100] In one aspect, an array of coefficients (or coefficient array) will be encoded. The coefficient array may include at least two coefficients. For example, these coefficients include wavelet coefficients. The wavelet coefficients can be generated by performing a transform (e.g., a wavelet transform) on at least two displacements. In one aspect, at the end of the coefficient array, one or more zero coefficients may exist. Table 1 shows an example of a coefficient array including at least two zero coefficients. An index of each coefficient in the coefficient array is provided, such as a first index of the first coefficient, a second index of the second coefficient, and a last index of the last coefficient. In addition, the last at least two non-zero coefficients are indicated in the coefficient array, such as the last non-zero coefficient (e.g., 1), the second to last non-zero coefficient (e.g., 11), and the first non-zero coefficient (e.g., 3). Table 1. Example of coefficient array and last non-zero coefficient signaling

[0101] The present application includes aspects for determining the last non-zero quantized wavelet coefficient (e.g., the last non-zero quantized wavelet coefficient in a coefficient array). The last non-zero quantized wavelet coefficient can be determined based on index information. Examples of index information include an index of the last non-zero quantized wavelet coefficient or other index information that can be used to determine the last non-zero quantized wavelet coefficient.

[0102] In one aspect, a last non-zero coefficient, such as a last non-zero quantized wavelet coefficient, is identified in the coefficient array, and a position index of the last non-zero quantized wavelet coefficient in the coefficient array is signaled or otherwise indicated.

[0103] For example, as shown in Table 1, the last non-zero coefficient (e.g., 1) is identified by the encoder and the corresponding index (e.g., 6) is signaled. In an example, the last non-zero coefficient can be derived from the index information. The last non-zero coefficient can be derived by adding a predefined value to the index value signaled in the index information (e.g., index value+1).

[0104] In one aspect, a non-zero coefficient of at least two non-zero coefficients and a last non-zero coefficient of at least two non-zero coefficients are identified. Index information indicates an index of the non-zero coefficient and an index difference between the index of the non-zero coefficient and the index of the last non-zero coefficient. In an example, the last two non-zero coefficients are determined, such as the last two quantized wavelet coefficients. The second to last non-zero quantized wavelet coefficient (or the first of the last two quantized wavelet coefficients) is signaled or otherwise indicated. In addition, the index difference between the last two non-zero quantized wavelet coefficients is signaled or otherwise indicated.

[0105] For example, as shown in Table 1, the index (e.g., 4) of the penultimate non-zero quantized wavelet coefficient (e.g., 11) is signaled or otherwise indicated. In addition, the index difference between the last non-zero quantized wavelet coefficient (e.g., 1) and the penultimate non-zero quantized wavelet coefficient (e.g., 11) is signaled or otherwise indicated, e.g., 2.

[0106] In one aspect, the last two non-zero coefficients, e.g., the last two non-zero quantized wavelet coefficients, are identified. The penultimate non-zero quantized wavelet coefficient (or the first of the last two quantized wavelet coefficients) is signaled or otherwise indicated. An index difference (between the last two non-zero quantized wavelet coefficients) may be indicated. The index difference may be indicated by the index difference minus a predefined value. For example, the index difference minus 1 is further signaled.

[0107] For example, as shown in Table 1, the index (e.g., 4) of the second to last non-zero quantized wavelet coefficient (e.g., 11) is signaled. In addition, the index difference minus a predefined value, such as 1, is signaled. The index difference indicates the difference between the index (e.g., 6) of the last non-zero quantized wavelet coefficient (e.g., 1) in the coefficient array and the index (e.g., 4) of the second to last non-zero quantized wavelet coefficient (e.g., 11).

[0108] In one aspect, the last two non-zero coefficients, e.g., the last two non-zero quantized wavelet coefficients, are identified in the coefficient array. The second to last non-zero quantized wavelet coefficient (or the first of the last two quantized wavelet coefficients) is signaled or otherwise indicated. The last non-zero quantized wavelet coefficient (or the second of the last two quantized wavelet coefficients) may be lossily coded to zero.

[0109] For example, as shown in Table 1, the index (eg, 4) of the second to last non-zero quantized wavelet coefficient (eg, 11) is signaled. In addition, the last non-zero quantized wavelet coefficient (eg, 1) is encoded as zero according to lossy coding.

[0110] In one aspect, the last at least two non-zero quantized wavelet coefficients are identified, including the last N non-zero quantized wavelet coefficients, where N>=2. The first non-zero quantized wavelet coefficient among the last N non-zero quantized wavelet coefficients is indicated by a signal (counting from the first coefficient in the coefficient array). In an example, the index information includes or otherwise indicates the index of the first non-zero quantized wavelet coefficient. For the last non-zero quantized wavelet coefficient following the first non-zero quantized wavelet coefficient, the difference between the index of the first non-zero quantized wavelet coefficient and the index of the last non-zero quantized wavelet coefficient following the first non-zero quantized wavelet coefficient may be included in the index information or otherwise indicated in the index information. For example, the index difference of each of the last N non-zero quantized wavelet coefficients minus 1 is further indicated by a signal or otherwise indicated. In some aspects, when the index of the last N non-zero quantized wavelet coefficients is represented as i1 <i2<…<i N , i1 is signaled or otherwise indicated when j=1, 2, ..., and N-1. j+1 –i j –1).

[0111] In an example, as shown in Table 1, the last three non-zero quantized wavelet coefficients with indices 0, 4, and 6, respectively, are identified. The index (e.g., 1) of the first non-zero quantized wavelet coefficient (e.g., 3) is signaled. In addition, the index difference of the second-to-last non-zero quantized wavelet coefficient minus 1 is signaled. The index difference is defined as the difference between the index of the first non-zero quantized wavelet coefficient (e.g., 0) and the index of the second-to-last non-zero quantized wavelet coefficient (e.g., 4). Therefore, the index difference of the second-to-last non-zero quantized wavelet coefficient is defined as 4-0=4. Therefore, the index difference of the second-to-last non-zero quantized wavelet coefficient minus 1 is signaled as 3. Similarly, the index difference of the last non-zero quantized wavelet coefficient minus 1 is signaled. The index difference is defined as the difference between the index of the second-to-last non-zero quantized wavelet coefficient (e.g., 4) and the index of the last non-zero quantized wavelet coefficient (e.g., 6). Therefore, the index difference of the last non-zero quantized wavelet coefficient is defined as 6-4 = 2. Therefore, the index difference of the last non-zero quantized wavelet coefficient minus 1 is signaled as 1.

[0112] In one aspect, the last at least two non-zero quantized wavelet coefficients are identified, including the last N non-zero quantized wavelet coefficients, where N>=2. In an example, the index information includes, or otherwise indicates, the index of the first non-zero quantized wavelet coefficient. For the last non-zero quantized wavelet coefficient following the first non-zero quantized wavelet coefficient, the difference between the index of the first non-zero quantized wavelet coefficient and the index of the last non-zero quantized wavelet coefficient following the first non-zero quantized wavelet coefficient may be included in the index information, or otherwise indicated in the index information. For example, the first non-zero quantized wavelet coefficient of the N non-zero quantized wavelet coefficients (counting from the first coefficient in the coefficient array) is indicated by a signal or otherwise. In addition, the first M indices of the last N non-zero quantized wavelet coefficients are indicated by a signal or otherwise subtracted from a predefined value (e.g., 1), where 0<=M<=N-1. In some aspects, when the index of the last N non-zero quantized wavelet coefficients is represented as i1 <i2<…<i N , where N>=2, i1 is indicated by a signal or otherwise indicated, and (i j+1 –i j –1), where j = 1, 2, …, M.

[0113] In an example, the last at least two non-zero quantized wavelet coefficients in the coefficient array are determined, for example, the last N non-zero quantized wavelet coefficients. The last at least two non-zero quantized wavelet coefficients include the first non-zero quantized wavelet coefficient and a subset of the last at least two non-zero quantized wavelet coefficients. The subset of the last at least two non-zero quantized wavelet coefficients includes at least two non-zero quantized wavelet coefficients subsequent to the first non-zero quantized wavelet coefficient. The index of the first non-zero quantized wavelet coefficient of the last at least two non-zero quantized wavelet coefficients in the coefficient array is encoded (or represented by a signal). The index parameter of each of the at least two subsequent non-zero quantized wavelet coefficients in the subset of the last at least two non-zero quantized wavelet coefficients is encoded. The index parameter of each of the at least two subsequent non-zero quantized wavelet coefficients is equal to the index difference minus one. The index difference is the position difference between a corresponding subsequent non-zero quantized wavelet coefficient and a previous non-zero quantized wavelet coefficient of the corresponding subsequent non-zero quantized wavelet coefficient in the coefficient array.

[0114] In an example, as shown in Table 1, the last 3 non-zero quantized wavelet coefficients with indices 0, 4, and 6 are identified. The index (e.g., 1) of the first non-zero quantized wavelet coefficient (e.g., 3) is signaled. Additionally, two index differences minus 1 are signaled for the last 3 non-zero quantized wavelet coefficients. The first of the two index differences minus 1 is equal to the index difference between the index of the first non-zero quantized wavelet coefficient (e.g., 0) and the index of the second to last non-zero quantized wavelet coefficient (e.g., 4) minus 1. Accordingly, the first of the first 2 index differences minus 1 is signaled as 3. Similarly, the second of the two index differences minus 1 is equal to the difference between the index of the second to last non-zero quantized wavelet coefficient (e.g., 4) and the index of the last non-zero quantized wavelet coefficient (e.g., 6) minus 1. Therefore, the second of the two index differences minus 1 is signaled as 1.

[0115] Figure 8An overview flow chart of a method (800) of one aspect of the present application is shown. The method (800) can be used in an apparatus. The apparatus may include a grid decoder (e.g., a dynamic grid decoder) and a video decoder. The video decoder is used, for example, to decode a displacement component encoded using a video codec. In various aspects, the method (800) is performed by a processing circuit, such as a processing circuit that performs the functions of a video decoder (110), a processing circuit that performs the functions of a video decoder (210), a grid decoder, etc. In some aspects, the method (800) is implemented by software instructions, so when the processing circuit executes the software instructions, the processing circuit executes the method (800). The method starts at (S801) and proceeds to (S810).

[0116] At (S810), a code stream is received, the code stream including displacement information of a mesh in a current mesh frame and base mesh information. The displacement information indicates at least two displacements associated with the base mesh and the mesh. The base mesh includes a subset of at least two vertices of the mesh.

[0117] At (S820), a last non-zero quantized wavelet coefficient among at least two non-zero quantized wavelet coefficients in a coefficient array associated with at least two displacements is determined based on index information included in the code stream.

[0118] At (S830), a mesh is reconstructed based on at least two displacements indicated by at least two non-zero quantized wavelet coefficients.

[0119] In an example, the index information identifies the index of the last non-zero quantized wavelet coefficient in the coefficient array. The last non-zero quantized wavelet coefficient is determined in the coefficient array based on the identified index.

[0120] In an example, the index of the second to last non-zero quantized wavelet coefficient is determined based on the index information. The index difference between the second to last non-zero quantized wavelet coefficient and the last non-zero quantized wavelet coefficient in the coefficient array is determined based on the index information. The index of the last non-zero quantized wavelet coefficient in the coefficient array is determined as the sum of the index of the second to last non-zero quantized wavelet coefficient and the index difference.

[0121] In an example, the index of the second to last non-zero quantized wavelet coefficient is determined based on the index information. An index parameter is determined based on the index information, which is equal to the index difference minus one. The index difference is the position difference between the second to last non-zero quantized wavelet coefficient and the last non-zero quantized wavelet coefficient in the coefficient array. The index of the last non-zero quantized wavelet coefficient in the coefficient array is determined based on the index of the second to last non-zero quantized wavelet coefficient and the index parameter.

[0122] In an example, the index information identifies the second to last non-zero quantized wavelet coefficient in the coefficient array. The last non-zero quantized wavelet coefficient in the coefficient array is determined to be zero.

[0123] In an example, the index of the first non-zero quantized wavelet coefficient of the last at least two non-zero quantized wavelet coefficients in the coefficient array is determined based on the index information. The last at least two non-zero quantized wavelet coefficients include the first non-zero quantized wavelet coefficient and at least one subsequent non-zero quantized wavelet coefficient. The index parameter of each of the at least one subsequent non-zero quantized wavelet coefficients is determined based on the index information. The index parameter of the corresponding subsequent non-zero quantized wavelet coefficient is equal to the index difference minus one. The index difference is the position difference between the corresponding subsequent non-zero quantized wavelet coefficient in the coefficient array and the previous non-zero quantized wavelet coefficient of the corresponding subsequent non-zero quantized wavelet coefficient. The index of each of the at least one subsequent non-zero quantized wavelet coefficients is determined based on the index of the first non-zero quantized wavelet coefficient and the index parameter of the corresponding subsequent non-zero quantized wavelet coefficient.

[0124] In an example, the index of the first non-zero quantized wavelet coefficient of the last at least two non-zero quantized wavelet coefficients in the coefficient array is determined based on the index information. The index parameter of each wavelet coefficient in the subset of the last at least two non-zero quantized wavelet coefficients is determined based on the index information. The subset of the last at least two non-zero quantized wavelet coefficients includes at least two subsequent non-zero quantized wavelet coefficients of the first non-zero quantized wavelet coefficient. The index parameter of each wavelet coefficient in the at least two subsequent non-zero quantized wavelet coefficients is equal to the index difference minus one. The index difference is the position difference between the corresponding subsequent non-zero quantized wavelet coefficient in the coefficient array and the previous non-zero quantized wavelet coefficient of the corresponding subsequent non-zero quantized wavelet coefficient. The index of each wavelet coefficient in the subset of the last at least two non-zero quantized wavelet coefficients is determined based on the index of the first non-zero quantized wavelet coefficient and the index parameter of the corresponding subsequent non-zero quantized wavelet coefficient.

[0125] Then, the method proceeds to (S899) and terminates.

[0126] The method (800) may be adjusted appropriately. One or at least two steps in the method (800) may be modified and / or omitted. An additional one or at least two steps may be added. Any suitable order of implementation may be used.

[0127] Fig. 9An overview flow chart of a method (900) of one aspect of the present application is shown. The method (900) can be used in an apparatus. The apparatus may include a grid decoder (e.g., a dynamic grid decoder) and a video encoder. The video decoder is used, for example, to decode a displacement component encoded using a video codec. In various aspects, the method (900) is performed by a processing circuit, such as a processing circuit that performs the functions of a video encoder (103), a processing circuit that performs the functions of a video encoder (303), a grid decoder, etc. In some aspects, the method (900) is implemented by software instructions, so when the processing circuit executes the software instructions, the processing circuit executes the method (900). The method starts at (S901) and proceeds to (S910).

[0128] At (S910), at least two displacements of a mesh in a current mesh frame are determined. The at least two displacements are associated with a base mesh and a mesh. The base mesh includes a subset of at least two vertices of the mesh.

[0129] At (S920), a wavelet transform is performed on the at least two displacements to generate at least two wavelet coefficients.

[0130] At (S930), at least two wavelet coefficients are quantized to generate a coefficient array including at least two non-zero quantized wavelet coefficients.

[0131] At (S940), the last non-zero quantized wavelet coefficient in the coefficient array is determined.

[0132] At (S950), index information is encoded based on the last non-zero quantized wavelet coefficient of the coefficient array in the code stream. In an example, the index information is encoded based on the index of the last non-zero quantized wavelet coefficient.

[0133] In an example, to encode the index information, the index of the last non-zero quantized wavelet coefficient in the coefficient array is encoded.

[0134] In an example, determining the last non-zero quantized wavelet coefficient includes determining the last non-zero quantized wavelet coefficient and the second to last non-zero quantized wavelet coefficient in the coefficient array. To encode the index information, the index of the second to last non-zero quantized wavelet coefficient in the coefficient array is encoded. The index difference between the second to last non-zero quantized wavelet coefficient and the last non-zero quantized wavelet coefficient in the coefficient array is encoded.

[0135] In an example, determining the last non-zero quantized wavelet coefficient includes determining the last non-zero quantized wavelet coefficient and the second to last non-zero quantized wavelet coefficient in the coefficient array. To encode the index information, the index of the second to last non-zero quantized wavelet coefficient in the coefficient array is encoded. An index parameter equal to the index difference minus one is encoded. The index difference is the position difference between the second to last non-zero quantized wavelet coefficient and the last non-zero quantized wavelet coefficient in the coefficient array.

[0136] In an example, to determine the last non-zero quantized wavelet coefficient, the last non-zero quantized wavelet coefficient is determined to be zero. The second to last non-zero quantized wavelet coefficient in the coefficient array is determined. To encode index information, the index of the second to last non-zero quantized wavelet coefficient in the coefficient array is encoded.

[0137] In the example, in order to determine the last non-zero quantized wavelet coefficient, the last at least two non-zero quantized wavelet coefficients in the coefficient array are determined. The last at least two non-zero quantized wavelet coefficients include the first non-zero quantized wavelet coefficient and at least one subsequent non-zero quantized wavelet coefficient. In order to encode the index information, the index of the first non-zero quantized wavelet coefficient of the last at least two non-zero quantized wavelet coefficients in the coefficient array is encoded. The index parameter of each wavelet coefficient in at least one subsequent non-zero quantized wavelet coefficient is encoded. The index parameter of the corresponding subsequent non-zero quantized wavelet coefficient is equal to the index difference minus one. The index difference is the position difference between the corresponding subsequent non-zero quantized wavelet coefficient in the coefficient array and the previous non-zero quantized wavelet coefficient of the corresponding subsequent non-zero quantized wavelet coefficient.

[0138] In an example, to determine the last non-zero quantized wavelet coefficient, the last at least two non-zero quantized wavelet coefficients in the coefficient array are determined. The last at least two non-zero quantized wavelet coefficients include the first non-zero quantized wavelet coefficient and a subset of the last at least two non-zero quantized wavelet coefficients. The subset of the last at least two non-zero quantized wavelet coefficients includes at least two subsequent non-zero quantized wavelet coefficients of the first non-zero quantized wavelet coefficient. To encode index information, the index of the first non-zero quantized wavelet coefficient of the last at least two non-zero quantized wavelet coefficients in the coefficient array is encoded. The index parameter of each of the at least two subsequent non-zero quantized wavelet coefficients in the subset of the last at least two non-zero quantized wavelet coefficients is encoded. The index parameter of each of the at least two subsequent non-zero quantized wavelet coefficients is equal to the index difference minus one. The index difference is the position difference between a corresponding subsequent non-zero quantized wavelet coefficient and a previous non-zero quantized wavelet coefficient of the corresponding subsequent non-zero quantized wavelet coefficient in the coefficient array.

[0139] Then, the method proceeds to (S999) and terminates.

[0140] The method (900) may be adjusted appropriately. One or at least two steps in the method (900) may be modified and / or omitted. An additional one or at least two steps may be added. Any suitable order of implementation may be used.

[0141] In the present application, a method for mesh data processing is provided. In the method, a code stream of mesh data is processed according to a format rule. In an example, the code stream includes displacement information and base mesh information of a mesh in a current mesh frame. The displacement information indicates at least two displacements associated with a base mesh and a mesh. The base mesh includes a subset of at least two vertices of the mesh. The format rule specifies that the last non-zero quantized wavelet coefficient of at least two non-zero quantized wavelet coefficients in a coefficient array associated with at least two displacements is determined based on index information included in the code stream. The format rule specifies that a mesh is processed based on at least two displacements indicated by at least two non-zero quantized wavelet coefficients.

[0142] The above techniques may be implemented as computer software using computer readable instructions and physically stored in one or at least two computer readable media. For example, Fig.10 A computer system (1000) suitable for implementing certain aspects of the disclosed subject matter is shown.

[0143] Computer software may be encoded using any suitable machine code or computer language that may be assembled, compiled, linked or similar mechanisms to create code comprising instructions that may be executed by one or at least two computer central processing units (CPUs), graphics processing units (GPUs), etc., directly or through interpretation, microcode execution, etc.

[0144] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, etc.

[0145] Fig.10 The components for the computer system (1000) shown in the example are examples and are not intended to suggest any limitation on the scope of use or functionality of computer software implementing aspects of the present application. The configuration of components should also not be interpreted as having any dependency or requirement on any one or combination of components illustrated in the example aspects of the computer system (1000).

[0146] The computer system (1000) may include certain human-machine interface input devices. Such human-machine interface input devices may be responsive to input from one or at least two human users through, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, taps), visual input (e.g., gestures), olfactory input (not depicted). The human-machine interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a still image camera), and videos (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0147] The input human-machine interface devices may include one or at least two of the following (only one of each is depicted): keyboard (1001), mouse (1002), touchpad (1003), touch screen (1010), data gloves (not shown), joystick (1005), microphone (1006), scanner (1007), camera (1008).

[0148] The computer system (1000) may also include certain human-computer interface output devices. Such human-computer interface output devices may stimulate the senses of one or at least two human users through, for example, tactile output, sound, light, and smell / taste. Such human-computer interface output devices may include tactile output devices (e.g., tactile feedback of a touch screen (1010), a data glove (not shown), or a joystick (1005), but there may also be tactile feedback devices that are not used as input devices), audio output devices (e.g., speakers (1009), headphones (not depicted)), visual output devices, and printers (not depicted). Visual output devices, such as screens (1010), virtual reality glasses (not depicted), holographic displays, and smoke canisters (not depicted). The screens (1010) include CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch screen input capabilities, each with or without tactile feedback capabilities - some of which are capable of outputting two-dimensional visual outputs or more than three-dimensional outputs, such as stereo output.

[0149] The computer system (1000) may also include human-accessible storage devices and their associated media, such as optical media including media (1021) such as CD / DVD ROM / RW (1020) with CD / DVD, thumb drives (1022), removable hard drives or solid-state drives (1023), traditional magnetic media such as tapes and floppy disks (not depicted), dedicated ROM / ASIC / PLD-based devices such as security dongles (not depicted), and the like.

[0150] Those skilled in the art should also understand that the term "computer-readable media" used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other volatile signals.

[0151] The computer system (1000) may also include an interface (1054) to one or at least two communication networks (1055). The network may be, for example, wireless, wired, optical. The network may also be local, wide area, metropolitan, vehicular and industrial, real-time, delay tolerant, etc. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., TV wired or wireless wide area networks including cable TV, satellite TV, and terrestrial broadcast TV, vehicular and industrial networks including CANBus, etc. Some networks typically require an external network interface adapter attached to some common data port or peripheral bus (1049) (e.g., a USB port of the computer system (1000)); other systems are typically integrated into the kernel of the computer system (1000) by attaching to a system bus as described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (1000) can communicate with other entities. Such communications may be one-way receive-only (e.g., broadcast TV), one-way send-only (e.g., CANbus to certain CANbus devices), or two-way, such as to other computer systems using a local area digital network or a wide area digital network. Certain protocols and protocol stacks may be used on each of those networks and network interfaces as described above.

[0152] The above-mentioned human interface device, human-accessible storage device, and network interface may be attached to the kernel (1040) of the computer system (1000).

[0153] The core (1040) may include one or at least two central processing units (CPUs) (1041), a graphics processing unit (GPU) (1042), a dedicated programmable processing unit in the form of a field programmable gate array (FPGA) (1043), a hardware accelerator (1044) for certain tasks, a graphics adapter (1050), etc. These devices, along with a read-only memory (ROM) (1045), a random access memory (1046), an internal mass storage device (1047) such as an internal non-user accessible hard drive, SSD, etc., may be connected via a system bus (1048). In some computer systems, the system bus (1048) may be accessible in the form of one or at least two physical plugs to enable expansion by additional CPUs, GPUs, etc. Peripheral devices may be attached directly to the core's system bus (1048) or to the core's system bus (1048) via a peripheral bus (1049). In an example, a screen (1010) may be connected to a graphics adapter (1050). Architectures for peripheral buses include PCI, USB, etc.

[0154] The CPU (1041), GPU (1042), FPGA (1043) and accelerator (1044) can execute certain instructions, and the combination of these instructions can constitute the above-mentioned computer code. The computer code can be stored in ROM (1045) or RAM (1046). Transition data can also be stored in RAM (1046), while permanent data can be stored in, for example, an internal mass storage device (1047). Fast storage and retrieval of any memory device can be enabled by using a cache memory, which can be closely associated with one or at least two CPUs (1041), GPUs (1042), mass storage devices (1047), ROMs (1045), RAMs (1046), etc.

[0155] The computer readable medium may have thereon computer code for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of this application, or they may be of a type well known and available to those skilled in the art of computer software.

[0156] As an example and not limitation, a computer system (1000) having an architecture, and in particular a kernel (1040), can provide functionality as a result of one or at least two processors (including CPUs, GPUs, FPGAs, accelerators, etc.) executing software embodied in one or at least two tangible computer-readable media. Such computer-readable media can be media associated with a user-accessible mass storage device as described above, as well as certain storage devices of the kernel (1040) having non-volatile properties, such as a kernel internal mass storage device (1047) or ROM (1045). Software implementing various aspects of the present application can be stored in such devices and executed by the kernel (1040). Depending on specific needs, the computer-readable medium may include one or at least two memory devices or chips. The software can cause the kernel (1040) and in particular the processors therein (including CPUs, GPUs, FPGAs, etc.) to perform specific processes or specific parts of specific processes described herein, including defining data structures stored in RAM (1046) and modifying such data structures according to processes defined by the software. Additionally or alternatively, the computer system may provide functionality as a result of hard-wiring logic or otherwise embodied in circuitry (e.g., accelerator (1044)) that may replace software or operate in conjunction with software to perform specific processes or specific portions of specific processes described herein. Where appropriate, references to software may include logic and vice versa. Where appropriate, references to computer-readable media may include circuitry (e.g., an integrated circuit (IC)) storing software for execution, circuitry embodying logic for execution, or both. The present application includes any suitable combination of hardware and software.

[0157] The use of "at least one of" or "one of" in this application is intended to include any one or combination of the referenced elements. For example, reference to at least one of A, B, or C; at least one of A, B, and C; at least one of A, B, and / or C; and at least one of A through C is intended to include only A, only B, only C, or any combination thereof. References to one of A or B and one of A and B are intended to include A or B or (A and B). When applicable, such as when the elements are not mutually exclusive, the use of "one of" does not exclude any combination of the referenced elements.

[0158] Although the present application has described several examples of various aspects, there are changes, permutations, and various alternative equivalents that fall within the scope of the present application. It should be understood that those skilled in the art will be able to design many systems and methods that, although not explicitly shown or described herein, embody the principles of the present application and are therefore within the spirit and scope of the present application.

Claims

1. A device for grid decoding, characterized in that: include: Processing circuitry for: receiving a code stream, the code stream comprising displacement information of a mesh in a current mesh frame and base mesh information, the displacement information indicating a plurality of displacements associated with a base mesh and the mesh, the base mesh comprising a subset of a plurality of vertices of the mesh; Determining, based on index information included in the code stream, a last non-zero quantized wavelet coefficient among a plurality of non-zero quantized wavelet coefficients in the coefficient array associated with the plurality of displacements; and The mesh is reconstructed based on the plurality of displacements indicated by the plurality of non-zero quantized wavelet coefficients.

2. The device according to claim 1, characterized in that The processing circuit is used for: Determine the index of the penultimate non-zero quantized wavelet coefficient based on the index information; determining an index difference between the second to last non-zero quantized wavelet coefficient and the last non-zero quantized wavelet coefficient in the coefficient array based on the index information; and The index of the last non-zero quantized wavelet coefficient in the coefficient array is determined as the sum of the index of the second to last non-zero quantized wavelet coefficient and the index difference.

3. The device according to claim 1, characterized in that The processing circuit is used for: Determine the index of the penultimate non-zero quantized wavelet coefficient based on the index information; Determining an index parameter based on the index information, the index parameter being equal to an index difference minus one, the index difference being a position difference between the second to last non-zero quantized wavelet coefficient and the last non-zero quantized wavelet coefficient in the coefficient array; and The index of the last non-zero quantized wavelet coefficient in the coefficient array is determined based on the index of the second to last non-zero quantized wavelet coefficient and the index parameter.

4. The device according to any one of claims 1 to 3, characterized in that: The index information identifies the second to last non-zero quantized wavelet coefficient in the coefficient array; and The processing circuit is used for: The last non-zero quantized wavelet coefficient in the coefficient array is determined to be zero.

5. The device according to claim 1, characterized in that The processing circuit is used for: Determining, based on the index information, an index of a first non-zero quantized wavelet coefficient of the last at least two non-zero quantized wavelet coefficients in the coefficient array, the last at least two non-zero quantized wavelet coefficients including the first non-zero quantized wavelet coefficient and at least one subsequent non-zero quantized wavelet coefficient; determining an index parameter of each of the at least one subsequent non-zero quantized wavelet coefficients based on the index information, the index parameter of the corresponding subsequent non-zero quantized wavelet coefficient being equal to an index difference minus one, the index difference being a position difference between the corresponding subsequent non-zero quantized wavelet coefficient and a previous non-zero quantized wavelet coefficient of the corresponding subsequent non-zero quantized wavelet coefficient in the coefficient array; and An index of each of the at least one subsequent non-zero quantized wavelet coefficient is determined based on the index of the first non-zero quantized wavelet coefficient and an index parameter of the subsequent non-zero quantized wavelet coefficient.

6. The device according to claim 1, characterized in that The processing circuit is used for: Determine, based on the index information, an index of a first non-zero quantized wavelet coefficient of the last at least two non-zero quantized wavelet coefficients in the coefficient array; Determine, based on the index information, an index parameter of each wavelet coefficient in the last subset of at least two non-zero quantized wavelet coefficients, the last subset of at least two non-zero quantized wavelet coefficients including at least two subsequent non-zero quantized wavelet coefficients of the first non-zero quantized wavelet coefficient, the index parameter of each wavelet coefficient in the plurality of subsequent non-zero quantized wavelet coefficients being equal to an index difference minus one, the index difference being a position difference between a corresponding subsequent non-zero quantized wavelet coefficient and a previous non-zero quantized wavelet coefficient of the corresponding subsequent non-zero quantized wavelet coefficient in the coefficient array; and The index of each wavelet coefficient in the last subset of at least two non-zero quantized wavelet coefficients is determined based on the index of the first non-zero quantized wavelet coefficient and the index parameters of the subsequent non-zero quantized wavelet coefficients.

7. A method for trellis coding, characterized in that: include: determining at least two displacements of a mesh in a current mesh frame, the at least two displacements being associated with a base mesh and the mesh, the base mesh comprising at least two subsets of vertices of the mesh; performing a wavelet transform on the at least two displacements to generate at least two wavelet coefficients; quantizing the at least two wavelet coefficients to generate a coefficient array including at least two non-zero quantized wavelet coefficients; Determining the last non-zero quantized wavelet coefficient in the coefficient array; and The index information is encoded based on the last non-zero quantized wavelet coefficient of the coefficient array in the bitstream.

8. The method according to claim 7, characterized in that: Determining the last non-zero quantized wavelet coefficient comprises: determining the last non-zero quantized wavelet coefficient and the second to last non-zero quantized wavelet coefficient in the coefficient array, and Encoding the index information includes: encoding the index of the second-to-last non-zero quantized wavelet coefficient in the coefficient array, and The index difference between the second to last non-zero quantized wavelet coefficient and the last non-zero quantized wavelet coefficient in the coefficient array is encoded.

9. The method according to claim 7, characterized in that: Determining the last non-zero quantized wavelet coefficient comprises: determining the last non-zero quantized wavelet coefficient and the second to last non-zero quantized wavelet coefficient in the coefficient array; and Encoding the index information includes: encoding the index of the second-to-last non-zero quantized wavelet coefficient in the coefficient array; and An index parameter is encoded, the index parameter being equal to an index difference minus one, the index difference being a position difference between the second to last non-zero quantized wavelet coefficient and the last non-zero quantized wavelet coefficient in the coefficient array.

10. The method according to any one of claims 7 to 9, characterized in that: Determining the last non-zero quantized wavelet coefficient comprises: determining the last non-zero quantized wavelet coefficient to be zero; and determining the second to last non-zero quantized wavelet coefficient in the coefficient array; and Encoding the index information includes: The index of the second to last non-zero quantized wavelet coefficient in the coefficient array is encoded.

11. The method according to claim 7, characterized in that: Determining the last non-zero quantized wavelet coefficient comprises: Determining the last at least two non-zero quantized wavelet coefficients in the coefficient array, the last at least two non-zero quantized wavelet coefficients comprising a first non-zero quantized wavelet coefficient and at least one subsequent non-zero quantized wavelet coefficient; and Encoding the index information further comprises: encoding an index of the first non-zero quantized wavelet coefficient of the last at least two non-zero quantized wavelet coefficients in the coefficient array; and The index parameter of each wavelet coefficient in the at least one subsequent non-zero quantized wavelet coefficient is encoded, and the index parameter of the corresponding subsequent non-zero quantized wavelet coefficient is equal to the index difference minus one, and the index difference is the position difference between the corresponding subsequent non-zero quantized wavelet coefficient in the coefficient array and the previous non-zero quantized wavelet coefficient of the corresponding subsequent non-zero quantized wavelet coefficient.

12. The method according to claim 7, characterized in that: Determining the last non-zero quantized wavelet coefficient comprises: determining last at least two non-zero quantized wavelet coefficients in the coefficient array, the last at least two non-zero quantized wavelet coefficients comprising a first non-zero quantized wavelet coefficient and a subset of the last at least two non-zero quantized wavelet coefficients, the subset of the last at least two non-zero quantized wavelet coefficients comprising at least two subsequent non-zero quantized wavelet coefficients of the first non-zero quantized wavelet coefficient; and Encoding the index information includes: encoding an index of the first non-zero quantized wavelet coefficient of the last at least two non-zero quantized wavelet coefficients in the coefficient array; and The index parameter of each of the at least two subsequent non-zero quantized wavelet coefficients in the subset of the last at least two non-zero quantized wavelet coefficients is encoded, and the index parameter of each of the at least two subsequent non-zero quantized wavelet coefficients is equal to the index difference minus one, and the index difference is the position difference between the corresponding subsequent non-zero quantized wavelet coefficient in the coefficient array and the previous non-zero quantized wavelet coefficient of the corresponding subsequent non-zero quantized wavelet coefficient.

13. A method for processing grid data, characterized in that: include: Process the code stream of the grid data according to the format rules, where: The code stream includes displacement information of a mesh in a current mesh frame and base mesh information, the displacement information indicating at least two displacements associated with a base mesh and the mesh, the base mesh including a subset of at least two vertices of the mesh; and The format rules specify: Determining, based on index information included in the bitstream, a last non-zero quantized wavelet coefficient of at least two non-zero quantized wavelet coefficients in the coefficient array associated with the at least two displacements; and The grid is processed based on the plurality of displacements indicated by the at least two non-zero quantized wavelet coefficients.

14. The method according to claim 13, characterized in that The format rules specify: Determine the index of the penultimate non-zero quantized wavelet coefficient based on the index information; Determining an index difference between the second to last non-zero quantized wavelet coefficient and the last non-zero quantized wavelet coefficient in the coefficient array based on the index information; and The index of the last non-zero quantized wavelet coefficient in the coefficient array is determined as the sum of the index of the second to last non-zero quantized wavelet coefficient and the index difference.

15. The method according to claim 13, characterized in that The format rules specify: Determine the index of the penultimate non-zero quantized wavelet coefficient based on the index information; Determining an index parameter as an index difference minus one based on the index information, the index difference being a position difference between the second to last non-zero quantized wavelet coefficient and the last non-zero quantized wavelet coefficient in the coefficient array; and The index of the last non-zero quantized wavelet coefficient in the coefficient array is determined based on the index of the second to last non-zero quantized wavelet coefficient and the index parameter.

Images