Location compression based on duplicated vertices

Abstract

The method includes determining whether each of the plurality of duplicate vertices is skippable in the mesh. One or more skippable duplicate vertices of the plurality of duplicate vertices in the mesh are excluded to generate an updated mesh. The vertices of the updated mesh are divided into a plurality of vertex groups, the plurality of vertex groups including a first vertex group. A prediction mode of the first vertex group is determined based on at least an estimation error of the vertices in the first vertex group of the updated mesh. Each estimation error indicates a difference between a respective vertex in the first vertex group and a corresponding reference vertex in a reference frame. Prediction information of the vertices in the first vertex group is generated based on the determined prediction mode.

Description

[Technical Field]

[0001] Incorporation by Reference This application claims the benefit of priority to U.S. Patent Application No. 18 / 207,610, entitled "Duplicate Vertices Based Position Compression," filed June 8, 2023, which in turn claims the benefit of priority to U.S. Provisional Application No. 63 / 394,485, entitled "Duplicate Vertices Based Position Compression," filed August 2, 2022. The disclosures of the prior applications are incorporated herein by reference in their entireties.

[0002] The present disclosure includes embodiments relating to mesh processing. [Background technology]

[0003] The background discussion provided herein is intended to generally present the context for the present disclosure. The inventors' work, to the extent that it is described in this background section, and aspects of the description that may not be admitted as prior art at the time of filing, are not admitted expressly or implicitly as prior art to the present disclosure.

[0004] Advances in three-dimensional (3D) capture, modeling, and rendering are facilitating the ubiquitous presence of 3D content across various platforms and devices. Today, it is possible to capture a baby's first steps on one continent while the baby's grandparents on another continent watch (and potentially interact with) the child, enjoying an immersive experience. To achieve this immersion, models are becoming ever more sophisticated, and a significant amount of data is tied to the generation and consumption of these models. 3D meshes are widely used to represent such immersive content. Summary of the Invention [Problem to be solved by the invention]

[0005] Aspects of the present disclosure provide methods and apparatus for mesh processing. In some examples, an apparatus for mesh processing includes a processing circuit. [Means for solving the problem]

[0006] According to one aspect of the present disclosure, a method for mesh processing performed in a video encoder is provided. The method determines whether each of a plurality of duplicate vertices is skippable in a mesh of a current frame. Each of the plurality of duplicate vertices is a vertex of the mesh that is a duplicate of another corresponding vertex in the mesh. Each of the plurality of duplicate vertices and its corresponding vertex have the same reference vertex in a reference frame. One or more skippable duplicate vertices among the plurality of duplicate vertices in the mesh are excluded to generate an updated mesh. The vertices of the updated mesh are divided into a plurality of vertex groups based on predetermined constant integers, and the plurality of vertex groups include a first vertex group. A prediction mode for the first vertex group is determined based on at least an estimation error for the vertices in the first vertex group of the updated mesh. Each estimation error indicates a difference between a respective vertex in the first vertex group and a corresponding reference vertex in the reference frame. Prediction information for the vertices in the first vertex group is generated based on the determined prediction mode for the first vertex group.

[0007] In one example, a first of the plurality of overlapping vertices is determined to be skippable based on a first of the plurality of overlapping vertices and a vertex in the mesh that corresponds to the first overlapping vertex having the same coordinates. In one example, a second of the plurality of overlapping vertices is determined to be non-skippable based on a second of the plurality of overlapping vertices and a vertex in the mesh that corresponds to the second overlapping vertex having different coordinates.

[0008] In some embodiments, the number of non-skippable duplicate vertices of the plurality of duplicate vertices is determined, and further, the vertex index of each non-skippable duplicate vertex of the plurality of duplicate vertices is determined.

[0009] In some embodiments, the coding information is generated based on one of (i) the number of the plurality of duplicate vertices being greater than or equal to a first threshold, (ii) the number of vertices in the mesh being greater than or equal to a second threshold, and (iii) a ratio of the number of the plurality of duplicate vertices to the number of vertices in the mesh being greater than or equal to a third threshold, wherein the coding information indicates (i) a number of non-skippable duplicate vertices among the plurality of duplicate vertices, and (ii) a vertex index for each non-skippable duplicate vertex among the plurality of duplicate vertices.

[0010] In one embodiment, the number of non-skippable duplicate vertices is coded based on one of variable length coding, fixed length coding, a (b+1)-bit fixed length code, and a fixed length code having a code length less than (b+1) bits, where b is a positive integer. In one embodiment, the vertex index of each non-skippable duplicate vertex of the plurality of duplicate vertices is coded based on one of differential coding, variable length coding, Exp-Golomb coding, a (b+1)-bit fixed length code, and a fixed length code having a code length less than (b+1) bits.

[0011] In some embodiments, to determine a prediction mode for the first vertex group, estimation errors for vertices in the first vertex group of the updated mesh are determined. An average neighborhood estimation error for multiple neighboring vertices associated with each vertex in the first vertex group is determined. Each of the multiple neighborhood estimation errors indicates a difference between one of the multiple neighboring vertices of the respective vertex in the first vertex group and a reference vertex in the reference frame corresponding to the one of the multiple neighboring vertices of the respective vertex in the first vertex group. A first cost value is determined based on the estimation errors for the vertices in the first vertex group. A second cost value is determined based on the average neighborhood estimation error associated with the vertices in the first vertex group. A prediction mode for the first vertex group is determined based on a comparison between the first cost value and the second cost value.

[0012] In one example, to determine an average neighborhood estimation error for multiple neighboring vertices, a first neighborhood estimation error associated with a first vertex of the vertices in a first vertex group is determined. The first neighborhood estimation error indicates a difference between a first neighboring vertex of the first vertex and a reference vertex in a reference frame corresponding to the first neighboring vertex of the first vertex. A second neighborhood estimation error associated with the first vertex of the vertices in the first vertex group is determined. The second neighborhood estimation error indicates a difference between a second neighboring vertex of the first vertex and a reference vertex in a reference frame corresponding to the second neighboring vertex of the first vertex. An average neighborhood estimation error of the first neighborhood estimation error and the second neighborhood estimation error is determined, and the average neighborhood estimation error is associated with the first vertex of the vertices in the first vertex group.

[0013] In some embodiments, the first cost value is determined as the sum of the estimation errors of the vertices in the first group of vertices.

[0014] In some embodiments, an estimated difference between the estimated error of each vertex in the first group of vertices and the average neighborhood estimated error associated with the corresponding vertex in the first group of vertices is determined to determine the second cost value, and the second cost value is determined as a sum of the estimated differences associated with the vertices in the first group of vertices.

[0015] In one example, the prediction mode is determined as the first mode based on the first cost value being less than or equal to the second cost value. In another example, the prediction mode is determined as the second mode based on the first cost value being greater than the second cost value.

[0016] In some embodiments, a first prediction residual is generated for each of the vertices in the first vertex group based on the prediction mode being the first mode, the first prediction residual indicating an estimated error for the corresponding vertex in the first vertex group, and a second prediction residual is generated for each of the vertices in the first vertex group based on the prediction mode being the second mode, the second prediction residual indicating an estimated error associated with the corresponding vertex in the first vertex group.

[0017] According to another aspect of the present disclosure, an apparatus is provided, the apparatus including a processing circuit, the processing circuit may be configured to perform any of the described methods for mesh processing.

[0018] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by a computer, cause the computer to perform any of the described methods for mesh processing.

[0019] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings. [Brief explanation of the drawings]

[0020] [Figure 1] FIG. 1 is a schematic diagram of an exemplary block diagram of a communication system (100). [Figure 2] FIG. 2 is a schematic diagram of an exemplary block diagram of a decoder. [Figure 3] FIG. 2 is a schematic diagram of an exemplary block diagram of an encoder. [Figure 4] FIG. 2 is a schematic diagram of vertices of a mesh according to some embodiments of the present disclosure. [Figure 5] FIG. 10 is a schematic diagram of an exemplary vertex position compression, according to some embodiments of the present disclosure. [Figure 6] 1 is a flowchart outlining a process according to some embodiments of the present disclosure. [Figure 7] 10 is a flowchart outlining another process according to some embodiments of the present disclosure. [Figure 8] FIG. 1 is a schematic diagram of a computer system, according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0021] 1 shows a block diagram of a video processing system (100) in some examples. The video processing system (100) is one example of an application for the disclosed subject matter, a video encoder and video decoder in a streaming environment. The disclosed subject matter may be equally applicable to other image and video-enabled applications, including, for example, video conferencing, digital TV, streaming services, and storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0022] The video processing system (100) includes a capture subsystem (113) that can include a video source (101). The video source (101) can include one or more images captured by a camera and / or generated by a computer. For example, a digital camera can create a stream of uncompressed video pictures (102). In one example, the stream of video pictures (102) includes samples taken by the digital camera. The stream of video pictures (102) is shown in bold to emphasize its large amount of data compared to the encoded video data (104) (or coded video bitstream) and can be processed by an electronic device (120) that includes a video encoder (103) coupled to the video source (101). The video encoder (103) can include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in detail below. The encoded video data (104) (or encoded video bitstream), shown with a thin line to emphasize its small amount of data compared to the stream of video pictures (102), may be stored on a streaming server (105) for future use. One or more streaming client subsystems, such as the client subsystems (106) and (108) of FIG. 1, can access the streaming server (105) to retrieve copies (107) and (109) of the encoded video data (104). The client subsystem (106) may include a video decoder (110), for example, within the electronic device (130). The video decoder (110) decodes the input copy of the encoded video data (107) and generates an output stream of video pictures (111), which can be rendered on a display (112) (e.g., a display screen) or other rendering device (not shown).In some streaming systems, the encoded video data 104, 107, and 109 (e.g., video bitstreams) may be encoded according to a particular video coding / compression standard. Examples of such standards include ITU-T Recommendation H.265. In one example, a video coding standard under development is informally known as Versatile Video Coding (VVC). The disclosed subject matter may be used in conjunction with VVC.

[0023] It should be noted that the electronic devices (120) and (130) may include other components (not shown). For example, the electronic device (120) may also include a video decoder (not shown), and the electronic device (130) may also include a video encoder (not shown).

[0024] 2 shows an example block diagram of a video decoder (210). The video decoder (210) may be included in an electronic device (230). The electronic device (230) may include a receiver (231). The receiver (231) may include receiving circuitry, such as a network interface circuit. The video decoder (210) may be used in place of the video decoder (110) in the example of FIG. 1.

[0025] The receiver (231) may receive one or more coded video sequences to be decoded by the video decoder (210). In one embodiment, one coded video sequence is received at a time, and the decoding of each coded video sequence is independent of the decoding of the other coded video sequences. The coded video sequences may be received from a channel (201), which may be a hardware / software link to a storage device that stores the encoded video data. The receiver (231) may also receive the encoded video data along with other data, such as coded audio data and / or auxiliary data streams, which may be forwarded to their respective using entities (not shown). The receiver (231) may separate the coded video sequences from other data. To combat network jitter, a buffer memory (215) may be coupled between the receiver (231) and the entropy decoder / parser (220) (hereinafter, "parser (220)"). In certain applications, the buffer memory (215) is part of the video decoder (210). In other applications, the buffer memory (215) can be external to the video decoder (210) (not shown). In still other applications, there can be a buffer memory (not shown) external to the video decoder (210), for example, to combat network jitter, plus another buffer memory (215) internal to the video decoder (210), for example, to handle playout timing. When the receiver (231) is receiving data from a storage / forwarding device with sufficient bandwidth and controllability or from an isosynchronous network, the buffer memory (215) may not be necessary or can be small.For use over a best-effort packet network such as the Internet, a buffer memory (215) may be required, which may be relatively large, advantageously adaptively sized, and may be implemented at least in part within an operating system or similar element (not shown) external to the video decoder (210).

[0026] The video decoder (210) may include a parser (220) for reconstructing symbols (221) from the coded video sequence. These symbol categories include information used to manage the operation of the video decoder (210) and, potentially, information for controlling a rendering device, such as a rendering device (212) (e.g., a display screen) that is not an integral part of the electronic device (230) but may be coupled to the electronic device (230), as shown in FIG. 2. The rendering device control information may be in the form of a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (220) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may follow a video coding technique or standard and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (220) may extract a set of subgroup parameters for at least one subgroup of pixels within the video decoder based on at least one parameter corresponding to the group from the coded video sequence. The subgroup may include a Group of Pictures (GOP), an image, a tile, a slice, a macroblock, a coding unit (CU), a block, a transform unit (TU), a prediction unit (PU), etc. The parser (220) may also extract information from the coded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0027] The parser (220) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (215) to create symbols (221).

[0028] The reconstruction of the symbols (221) can involve several different units, depending on the type of coded video picture or portion thereof (inter-picture and intra-picture, inter-block and intra-block, etc.), as well as other factors. Which units are involved and how may be controlled by subgroup control information parsed by the parser (220) from the coded video sequence. The flow of such subgroup control information between the parser (220) and the following units is not depicted for clarity.

[0029] In addition to the functional blocks already mentioned, the video decoder (210) may be conceptually subdivided into several functional units, as described below. In an actual implementation operating under commercial constraints, many of these units may interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, a conceptual subdivision into the following functional units is appropriate:

[0030] The first unit is a scalar / inverse transform unit (251), which receives quantized transform coefficients from the parser (220) as well as control information including which transform to use, block size, quantization coefficients, quantization scaling matrices, etc. as symbols (221). The scalar / inverse transform unit (251) can output blocks containing sample values ​​that can be input to an aggregator (255).

[0031] In some cases, the output samples of the scaler / inverse transform unit (251) may relate to intra-coded blocks. Intra-coded blocks are blocks that do not use prediction information from a previously reconstructed picture but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by the intra-picture prediction unit (252). In some cases, the intra-picture prediction unit (252) generates a block of the same size and shape as the block being reconstructed using surrounding, already reconstructed information fetched from the current picture buffer (258). The current picture buffer (258), for example, buffers a partially reconstructed and / or fully reconstructed current picture. The aggregator (255) may add, on a sample-by-sample basis, the prediction information generated by the intra-prediction unit (252) to the output sample information provided by the scaler / inverse transform unit (251).

[0032] In other cases, the output samples of the scalar / inverse transform unit (251) may relate to an inter-coded, potentially motion-compensated block. In such cases, the motion-compensated prediction unit (253) may access a reference picture memory (257) to fetch samples used for prediction. After motion-compensating the fetched samples according to the symbols (221) associated with the block, these samples may be added by the aggregator (255) to the output of the scalar / inverse transform unit (251) (in this case, referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory (257) from which the motion-compensated prediction unit (253) fetches prediction samples may be controlled by motion vectors available to the motion-compensated prediction unit (253) in the form of symbols (221), which may have, for example, X, Y, and reference picture components. Motion compensation may also include interpolation of fetched sample values ​​from the reference picture memory (257) when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, and the like.

[0033] The output samples of the aggregator (255) can be subjected to various loop filtering techniques in the loop filter unit (256). Video compression techniques can include in-loop filter techniques controlled by parameters contained in the coded video sequence (also called the coded video bitstream) and available to the loop filter unit (256) as symbols (221) from the parser (220). Video compression can also be responsive to meta-information obtained during decoding of previous portions (in decoding order) of the coded picture or coded video sequence, and to previously reconstructed loop-filtered sample values.

[0034] The output of the loop filter unit (256) may be a sample stream that may be output to a rendering device (212) and stored in a reference picture memory (257) for use in future inter-picture prediction.

[0035] Once a particular coded picture is fully reconstructed, it may be used as a reference picture for future prediction. For example, once a coded picture corresponding to a current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (220)), the current picture buffer (258) may become part of the reference picture memory (257), and a new current picture buffer may be reallocated before beginning reconstruction of a subsequent coded picture.

[0036] The video decoder (210) can perform decoding operations according to a given video compression technology or standard, such as ITU-T Recommendation H.265. A coded video sequence may conform to the syntax specified by the video compression technology or standard being used, in the sense that the coded video sequence adheres to both the syntax of the video compression technology or standard and the profile documented in the video compression technology or standard. Specifically, a profile may select certain tools from among all tools available in the video compression technology or standard as the only tools available under that profile. Compliance may also require that the complexity of the coded video sequence be within a range defined by the level of the video compression technology or standard. In some cases, the level limits the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level may, in some cases, be further constrained by the specification of a hypothetical reference decoder (HRD) and metadata for HRD buffer management signaled within the coded video sequence.

[0037] In one embodiment, the receiver (231) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the coded video sequence. The additional data may be used by the video decoder (210) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0038] 3 shows an example block diagram of a video encoder (303). The video encoder (303) is included in an electronic device (320). The electronic device (320) includes a transmitter (340) (e.g., a transmitting circuit). The video encoder (303) may be used in place of the video encoder (103) in the example of FIG. 1.

[0039] The video encoder (303) may receive video samples from a video source (301) (which is not part of the electronic device (320) in the example of FIG. 3) that may capture video images to be coded by the video encoder (303). In other examples, the video source (301) is part of the electronic device (320).

[0040] The video source (301) may provide a source video sequence to be coded by the video encoder (303) in the form of a digital video sample stream, which may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media serving system, the video source (301) may be a storage device that stores previously prepared video. In a video conferencing system, the video source (301) may be a camera that captures local image information as a video sequence. The video data may be provided as multiple individual pictures that convey motion when viewed in sequence. The pictures themselves may be organized as a spatial array of pixels, each of which may contain one or more samples, depending on the sampling structure, color space, etc., in use. The following discussion focuses on samples.

[0041] According to one embodiment, the video encoder (303) may code and compress pictures of a source video sequence into a coded video sequence (343) in real time or under any other required time constraints. Ensuring an appropriate coding rate is one function of the controller (350). In some embodiments, the controller (350) controls and is operatively coupled to other functional units, as described below. For clarity, coupling is not shown. Parameters set by the controller (350) may include rate control-related parameters (picture skip, quantizer, lambda value for rate-distortion optimization techniques, ...), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (350) may be configured with other appropriate functions associated with the video encoder (303) optimized for a particular system design.

[0042] In some embodiments, the video encoder (303) is configured to operate in a coding loop. As an overly simplified explanation, in one example, the coding loop can include a source coder (330) (e.g., responsible for creating symbols, such as a symbol stream, based on an input picture to be coded and reference picture(s)) and a (local) decoder (333) embedded in the video encoder (303). The decoder (333) reconstructs the symbols to generate sample data in a manner similar to that used by a (remote) decoder. The reconstructed sample stream (sample data) is input to a reference picture memory (334). Because decoding of the symbol stream produces bit-exact results regardless of the location (local or remote) of the decoder, the contents of the reference picture memory (334) are also bit-exact between the local and remote encoders. In other words, the predictive portion of the encoder "sees" the exact same sample values ​​as the decoder would "see" when using prediction during decoding. This basic principle of reference picture synchronism (and the resulting drift when synchronism cannot be maintained, for example due to channel errors) is also used in several related techniques.

[0043] The operation of the "local" decoder (333) may be the same as the operation of a "remote" decoder, such as the video decoder (210), already described in detail above in conjunction with Figure 2. However, with brief reference also to Figure 2, because symbols are available and the encoding / decoding of symbols into a coded video sequence by the entropy coder (345) and parser (220) may be lossless, the entropy decoding portion of the video decoder (210), including the buffer memory (215) and parser (220), may not be fully implemented in the local decoder (333).

[0044] In one embodiment, decoder technology, with the exception of parsing / entropy decoding, present in the decoder is present in the corresponding encoder in the same or substantially the same functional form. Therefore, the disclosed subject matter focuses on the operation of the decoder. A description of the encoder technology may be omitted, as it is the reverse of the decoder technology described generically. In certain areas, more detailed descriptions are provided below.

[0045] In operation, in some examples, the source coder (330) may perform motion-compensated predictive coding, which predictively codes an input picture with reference to one or more previously coded pictures from a video sequence designated as “reference pictures.” In this manner, the coding engine (332) codes differences between pixel blocks of the input picture and pixel blocks of reference picture(s) that may be selected as predictive reference(s) for the input picture.

[0046] The local video decoder (333) may decode the coded video data of pictures that may be designated as reference pictures based on symbols created by the source coder (330). The operation of the coding engine (332) may advantageously be a lossy process. When the coded video data is decoded by a video decoder (not shown in FIG. 3), the reconstructed video sequence may typically be a replica of the source video sequence with some errors. The local video decoder (333) may replicate the decoding process that may be performed by the video decoder on the reference pictures and store the reconstructed reference pictures in the reference picture memory (334). In this manner, the video encoder (303) may locally store copies of reconstructed reference pictures that have common content with reconstructed reference pictures obtained by a far-end video decoder (without transmission errors).

[0047] The predictor (335) may perform the prediction search for the coding engine (332). That is, for a new picture to be coded, the predictor (335) may search the reference picture memory (334) for sample data (as candidate reference pixel blocks) or some metadata, such as reference picture motion vectors, block shapes, etc., which may serve as suitable prediction references for the new picture. The predictor (335) may operate on sample blocks, pixel block by pixel block, to find suitable prediction references. In some cases, as determined by search results obtained by the predictor (335), the input picture may have prediction references drawn from multiple reference pictures stored in the reference picture memory (334).

[0048] The controller (350) may manage the coding operations of the source coder (330), including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0049] The output of all of the aforementioned functional units may be entropy coded by an entropy coder (345), which converts the symbols produced by the various functional units into a coded video sequence by applying lossless compression to the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0050] The transmitter (340) may buffer the coded video sequence(s) created by the entropy coder (345) in preparation for transmission over a communication channel (360), which may be a hardware / software link to a storage device that stores the encoded video data. The transmitter (340) may merge the coded video data from the video encoder (303) with other data to be transmitted, such as coded audio data and / or an auxiliary data stream (source not shown).

[0051] The controller (350) may manage the operation of the video encoder (303). During coding, the controller (350) may assign several coding picture types to each coded picture, which may affect the coding technique that may be applied to each picture. For example, pictures may often be assigned as one of the following picture types:

[0052] An intra-picture (I-picture) may be one that can be coded and decoded without using any other picture in a sequence as a source of prediction. Some video codecs allow different types of intra-pictures, including, for example, independent decoder refresh ("IDR") pictures. Those skilled in the art are aware of these variations of I-pictures and their respective uses and characteristics.

[0053] A predictive picture (P picture) may be coded and decoded using intra- or inter-prediction, which uses at most one motion vector and reference index to predict the sample values ​​of each block.

[0054] Bidirectionally predicted pictures (B-pictures) may be coded and decoded using intra- or inter-prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, multiple predicted pictures may use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0055] A source picture is generally spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and may be coded block by block. Blocks may be predictively coded with reference to other (already coded) blocks determined by the coding assignment applied to the block's respective picture. For example, blocks of an I-picture may be non-predictively coded or predictively coded with reference to already coded blocks of the same picture (spatial prediction or intra prediction). Pixel blocks of a P-picture may be predictively coded via spatial prediction or via temporal prediction with reference to one previously coded reference picture. Blocks of a B-picture may be predictively coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference pictures.

[0056] The video encoder (303) may perform coding operations according to a predetermined video coding technique or standard, such as ITU-T Recommendation H.265. In doing so, the video encoder (303) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the coded video data may conform to a syntax specified by the video coding technique or standard being used.

[0057] In one embodiment, the transmitter (340) may transmit additional data along with the encoded video. The source coder (330) may include such data as part of the coded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0058] Video may be captured in time sequence as multiple source pictures (video pictures). Intra-picture prediction (often abbreviated as intra-prediction) uses spatial correlation within a given picture, while inter-picture prediction uses correlation (temporal or other) between pictures. In one example, a particular picture being encoded / decoded, called the current picture, is divided into blocks. When a block in the current picture is similar to a reference block in a previously coded and still buffered reference picture within the video, the block in the current picture may be coded by a vector called a motion vector. A motion vector points to a reference block in the reference picture and may have a third dimension that identifies the reference picture if multiple reference pictures are used.

[0059] In some embodiments, bi-prediction techniques may be used in inter-picture prediction. According to bi-prediction techniques, two reference pictures, such as a first reference picture and a second reference picture, are used, both of which are before the decoding order of the current picture in the video (but their display orders may be past and future, respectively). A block in the current picture may be coded by a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. A block may be predicted by a combination of the first reference block and the second reference block.

[0060] Furthermore, merge mode techniques may be used in inter-picture prediction to improve coding efficiency.

[0061] According to some embodiments of the present disclosure, predictions such as inter-picture prediction and intra-picture prediction are performed in units of blocks, such as polygonal or triangular blocks. For example, according to the HEVC standard, pictures in a sequence of video pictures are divided into coding tree units (CTUs) for compression, and the CTUs in a picture have the same size, such as 64x64 pixels, 32x32 pixels, or 16x16 pixels. Generally, a CTU includes three coding tree blocks (CTBs), one luma CTB and two chroma CTBs. Each CTU may be recursively quadtree-decomposed into one or more coding units (CUs). For example, a 64x64 pixel CTU may be divided into one CU of 64x64 pixels, four CUs of 32x32 pixels, or 16 CUs of 16x16 pixels. In one example, each CU is analyzed to determine the prediction type of the CU, such as an inter-prediction type or an intra-prediction type. A CU is divided into one or more prediction units (PUs) according to temporal and / or spatial predictability. Generally, each PU includes one luma prediction block (PB) and two chroma PBs. In one embodiment, prediction operations in coding (encoding / decoding) are performed in units of prediction blocks. Using a luma prediction block as an example of a prediction block, the prediction block includes a matrix of values ​​(e.g., luma values) for pixels of 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 pixels, etc.

[0062] It should be noted that the video encoders (103) and (303) and the video decoders (110) and (210) may be implemented using any suitable technology. In one embodiment, the video encoders (103) and (303) and the video decoders (110) and (210) may be implemented using one or more integrated circuits. In other embodiments, the video encoders (103) and (303) and the video decoders (110) and (210) may be implemented using one or more processors executing software instructions.

[0063] This disclosure includes embodiments related to methods and systems for mesh vertex position compression using duplicate vertices. In some embodiments, each duplicate vertex in a mesh is a duplicate of another corresponding vertex in the mesh of a current frame. Each duplicate vertex and its corresponding vertex may have the same reference vertex in a reference frame.

[0064] A mesh can include several polygons that describe the surface of a volumetric object. Each polygon of a mesh can be defined by the vertices of the corresponding polygon in three-dimensional (3D) space and information about how the vertices are connected, sometimes referred to as connectivity information. In some embodiments, vertex attributes such as color, normals, etc. can be associated with mesh vertices. Attributes (or vertex attributes) can also be associated with the surface of a mesh by utilizing mapping information that parameterizes the mesh with a two-dimensional (2D) attribute map. Such mapping can typically be described by a set of parametric coordinates, called UV coordinates or texture coordinates, associated with the mesh vertices. 2D attribute maps can be used to store high-resolution attribute information such as texture, normals, and displacement. Such information can be used for various purposes, such as texture mapping and shading.

[0065] Dynamic meshes can contain a significant amount of information that changes over time, so they can require large amounts of data. Therefore, efficient compression techniques are needed to store and transmit such content. Mesh compression standards, such as IC, MESHGRID, and FAMC, were previously developed by MPEG to address dynamic meshes with constant connectivity, time-varying geometry, and vertex attributes. However, these standards may not consider time-varying attribute maps and connectivity information. Digital content generation (DCC) tools typically generate such dynamic meshes. However, volumetric acquisition techniques can have difficulty generating constant connectivity dynamic meshes, especially under real-time constraints. This type of content (e.g., constant connectivity dynamic meshes) may not be supported by existing standards. MPEG is planning to develop a new mesh compression standard to directly handle dynamic meshes with time-varying connectivity information and, optionally, time-varying attribute maps. The new mesh compression standard targets lossy and lossless compression for various applications, such as real-time communication, storage, free-viewpoint video, augmented reality (AR), and virtual reality (VR). Features such as random access and scalable / progressive coding can also be considered.

[0066] Mesh geometry information can include vertex connectivity information, 3D coordinates, 2D texture coordinates, etc. Compressing vertex 3D coordinates, sometimes also called vertex positions, can be important because in many cases, compressing vertex 3D coordinates can consume a significant portion of the overall geometry-related data.

[0067] A dynamic mesh sequence M at time instance t can be represented as M(t). M(t) can be referred to as a positionally tracked frame if there is a mapping (or mapping operation) f from the vertex positions in M(t) to the vertex positions at another time instance M(t), where t and t are different time instances. Therefore, M(t) can be referred to as a reference frame, and the corresponding vertex in the reference frame can be referred to as the reference vertex of the vertex in M(t).

[0068] This disclosure proposes methods and / or systems for mesh vertex position compression using overlapping vertices. It should be noted that the methods and / or systems may be applied individually or in any combination. Furthermore, the disclosed methods and systems are not limited to vertex position compression. The disclosed methods and systems may also be applied to, for example, two-dimensional (2D) texture coordinate compression or more general temporal prediction-based schemes.

[0069] For a vertex V in a positionally tracked frame M(t), the neighbors of the vertex may also be vertices connected to V via edges, and these vertices are called the neighbor vertices (or nearby vertices) of V. For example, as shown in FIG. 4, vertex A may have four neighbor vertices, which are C, D, E, and B. Vertex E may have five neighbor vertices, which are A, B, F, H, and D.

[0070] For a positionally tracked frame M(t) and its reference frame M(t), we assume that f is a mapping between the vertex positions in M(t) and M(t). Given a vertex V in M(t), the mapping between the reference vertex f(V) of vertex V and the coded vertex f(V) in M(t) is

number

number

number

number

number

[0071] In one embodiment, for a duplicate vertex V, V and

number

number

[0072] Figure 5 is a schematic diagram of an exemplary vertex position compression device (or apparatus) (500) according to some embodiments of the present disclosure. As shown in Figure 5, the vertex position compression device (500) may include a duplication signaling module (501) configured to determine whether duplicate vertices are skippable. The duplication signaling module (501) may also be configured to signal (transmit) vertex indices of skippable duplicate vertices or vertex indices of non-skippable duplicate vertices. The apparatus (500) may include a vertex grouping module (502) configured to group vertices of a mesh, a position prediction module (504) configured to calculate predicted positions of the vertices, a prediction mode coding module (506) configured to code (or determine) a prediction mode for the position compression, and a prediction residual coding module (508) configured to code position prediction residuals.

[0073] The term module in this disclosure can refer to a software module, a hardware module, or a combination thereof. A software module (e.g., a computer program) may be developed using a computer programming language. A hardware module may be implemented using processing circuits and / or memory. Each module can be implemented using one or more processors (or a processor and memory). Similarly, a processor (or a processor and memory) can be used to implement one or more modules. Furthermore, each module may be part of an entire module that includes the functionality of the module.

[0074] For a positionally tracked frame M(t), duplicate vertices may be identified using a reference frame M(t0). For each duplicate vertex V, it may be determined as either a skippable duplicate or a non-skippable duplicate. In one embodiment, a set S may be represented as the set of skippable duplicates, and a set N S may be represented as the set of non-skippable duplicates.

[0075] In one embodiment, the encoder can signal the vertex indices of the non-skippable duplicates. In one example, the set NS is: NS={d1, d2, ..., d s}, where 0≦d1 <d2<…<d s ≦T−1 and d i is the vertex index of the i-th non-skippable duplicate, s is the number (or count) of non-skippable duplicate vertices, and T is the number (or count) of vertices in M(t). In one example, s ≥ 0 and T ≥ 3. If s = 0, the set NS can be the empty set, and all duplicate vertices are skippable. The encoder uses the number s and each index d for duplicate signaling. i can be coded.

[0076] The number s may be coded based on various coding methods. In one embodiment, the number s may be coded using variable length coding. In one embodiment, the number s may be coded using fixed length coding. In one embodiment, the number s may be coded using a (b+1)-bit fixed length code, where b may be the largest integer less than or equal to log2(T). In one embodiment, the number s may be coded using a fixed length code, where the code length is less than (b+1)-bits and b is the largest integer less than or equal to log2(T). Since s may be coded using more than one fixed length code, the first bit (or last bit) in each fixed length code may be signaled to indicate whether the respective fixed length code is a representation of a new symbol or a continuation of the last (or previous) symbol.

[0077] vertex index d i may be coded based on various coding methods. In one embodiment, the vertex index d i can be coded using differential coding, in which the encoder generates multiple parameters D i(i≧1) can be coded. i is D1=d1(i=1) and D i =d i -d i-1 -1 (i ≥ 2). Therefore, the vertex index of a non-skippable vertex is the coded D i It can be expressed based on (i≧1).

[0078] In one embodiment, D i may be coded using variable length coding. i may be coded using an Exp-Golomb code. i may be coded using fixed length coding. i may be coded using a (B+1)-bit fixed length code, where B is the largest integer less than or equal to log2(T-2). i may be coded using a fixed-length code, where the code length is less than (B+1) bits, and B is the largest integer less than or equal to log2(T-2). i Since , may be coded using more than one fixed-length code, the first bit (or last bit) in each fixed-length code may be signaled to indicate whether the respective fixed-length code is a representation of a new symbol or a continuation of the last (or previous) symbol.

[0079] In some embodiments, the vertex index of a skippable duplicate (or duplicate vertex) may be signaled.

[0080] In some embodiments, the encoder can adaptively turn duplicate signaling on or off. The adaptive decision to turn duplicate signaling on or off can be based on vertex statistics. In one example, the adaptive decision to turn duplicate signaling on or off can be based on the number (or count) of duplicate vertices. If the number of duplicate vertices is greater than or equal to a threshold, duplicate signaling can be turned on. Otherwise, duplicate signaling can be turned off. In one example, the adaptive decision to turn duplicate signaling on or off can be based on the number (or count) of vertices T. If the number of vertices T is greater than or equal to a threshold, duplicate signaling can be turned off. Otherwise, duplicate signaling can be turned on. In one example, the adaptive decision to turn duplicate signaling on or off can be based on the ratio between the number of duplicate vertices and the number of vertices. If the ratio is greater than or equal to a threshold, duplicate signaling can be turned on. Otherwise, duplicate signaling can be turned off.

[0081] In the present disclosure, the vertices of a positionally tracked frame M(t) may be divided into groups, each of which may contain K vertices, where K is a constant. For example, the vertices of M(t) may be divided into multiple groups, each of which may contain 10 vertices (K=10). In one example, when K=1, each group contains one vertex. In yet another example, when K is equal to the number of vertices in frame M(t), all vertices are included in the same group.

[0082] In one embodiment of the present disclosure, if the vertices of M(t) are divided into groups, skippable duplicates (or skippable duplicate vertices) can be skipped (or removed). For example, the vertices of M(t) are divided into groups, and each group contains 10 vertices. If the first 12 vertices of M(t) have two skippable duplicates, a first group of M(t) can be formed based on vertices with indices from 0 to 11, and the two skippable duplicates of the first 12 vertices can be removed.

[0083] In other embodiments, when M(t) is divided into groups, all vertices, including skippable duplicates, may be considered (or included).

[0084] In the present disclosure, temporal prediction may be applied to predict a vertex in a current frame based on a reference vertex in a reference frame. For a vertex V in a positionally tracked frame (or current frame) M(t), the position of the vertex V can be estimated by the position of a reference vertex f(V) in a reference frame (e.g., M(t), where t is a different time instance), where f is a mapping operation between M(t) and the reference frame. In some embodiments, the vertex V and the reference vertex f(V) in the reference frame are co-located. Therefore, the reference vertex can have the same relative position in the reference frame as the vertex in the current frame M(t). When the vertex V is predicted by the reference vertex f(V), the estimation error E can be calculated as the difference between the position of V and the position of f(V) in Equation (7). E=Vf(V) Equation (7)

[0085] Since each vertex in the frame M(t) can have a 3D coordinate, the 3D coordinate components of the estimation error E can be provided based on equation (1). For example, assuming that the subscripts x, y, and z represent 3D coordinates in the xyz space, the 3D coordinate components of the estimation error E can be given by equations (8) to (10). E x =V x -(f(V)) x Formula (8) E y =V y -(f(V)) y Formula (9) E z =V z -(f(V)) z Formula (10)

[0086] The estimated error E of a vertex V can be predicted or determined from the neighboring vertices (or neighboring vertices) of vertex V. For the neighboring vertices (or neighboring vertices) of V, if the neighboring vertices are coded and can be used for prediction, the estimated errors of the neighboring vertices can be applied to predict E.

[0087] V is a set of N neighboring vertices (or neighboring vertices) V1, V2, ..., V N Assume that we have a neighboring vertex V i If , the neighboring vertex V i The estimation error (or neighborhood estimation error) of is E for i=1,2,…,N. i =V i -f(V i ) can be calculated as f(V i ) is the neighboring vertex V in the reference frame. i The reference vertex of E i can also be referred to as the neighborhood estimation error associated with the vertex V, and each E i can be a candidate prediction for E.

[0088] If N>=2, two or more estimated errors E i is available. In one embodiment, the estimation error E i The average estimation error (or average neighborhood estimation error) E0 can be defined as follows in equation (11): E0=(E1+E2+…+E N ) / N formula (11)

[0089] For a group G in frame M(t), the encoder can determine a coding cost C0 and a coding cost C1. C0 may denote the sum of the estimated errors E of all vertices in group G. Once the estimated errors E of each vertex in group G are determined, such as based on equation (1), the coding cost C0 can be determined as the sum of the estimated errors E of the vertices in group G. C1 may denote the sum of the estimated residuals (E-E0) of all vertices in group G. To determine the coding cost C1, the estimated errors E of the neighboring vertices of each vertex in group G are i is E i =V i -f(V i ) and so on. Furthermore, the estimation error E i The average neighborhood estimation error E0 of can be determined based on equation (5). An estimated residual (or estimated difference) (E-E0) can be determined for each vertex in group G. A coding cost C1 can be determined as the sum of the estimated residuals associated with the vertices of group G.

[0090] In this disclosure, a prediction mode may be determined based on the coding cost C0 and the coding cost C1.

[0091] If the coding cost of C0 is less than or equal to C1, prediction mode 0 is applied to group G. Prediction mode 0 indicates that the prediction residual for each vertex in group G is set as the estimated error E of each vertex. Therefore, at the decoder side, each vertex in group G can be reconstructed as V = f(V) + E, where f(V) is the reference vertex in reference frame M(t0) corresponding to each vertex in group G in frame M(t).

[0092] If the coding cost of C0 is greater than C1, prediction mode 1 is applied to group G. Prediction mode 1 indicates that the prediction residual of each vertex in group G is set as (E-E0), where E is the estimation error of each vertex, and E0 is the average estimation error of the estimation errors of its neighboring vertices. Therefore, at the decoder side, each vertex in group G can be reconstructed as V=f(V)+(E-E0), where f(V) is the reference vertex in reference frame M(t0) corresponding to each vertex in group G in frame M(t).

[0093] In this disclosure, a prediction mode of group G may be coded. The prediction mode may be a binary number, such as 0 or 1. In one embodiment, the prediction mode may be coded using entropy coding. In one embodiment, the prediction mode may be coded using arithmetic coding. In one embodiment, the prediction mode may be coded using context-based arithmetic coding, such as context-adaptive binary arithmetic coding (CABAC). In one embodiment, the prediction mode may be coded using spatial context-based arithmetic coding, where the context may be conditional (or based) on a previously coded group of the same frame.

[0094] In one embodiment, if the prediction mode in a reference frame (e.g., M(t0)) has already been coded and is available, the prediction mode of the group in frame M(t) can be coded using a temporal context, such as temporal context-based arithmetic coding. Thus, the prediction mode of the group in frame M(t) can be determined based on the prediction mode of the corresponding (or associated) group in the reference frame based on temporal context-based arithmetic coding. This is because each vertex in the group in frame M(t) has a corresponding reference vertex in the associated group in the reference frame, and a one-to-one association between the positionally tracked group in frame M(t) and the associated group in reference frame M(t0) can also be established. The associated group in the reference frame (e.g., M(t0), where t0 is a time instance different from t) can be referred to as a reference group.

[0095] In one embodiment, a logical operation (or flag) such as XOR (exclusive OR) may be coded to indicate a relationship between the prediction mode of group G and the prediction mode of the reference group of group G. Thus, if group G and the reference group have the same prediction mode, the logical operation XOR may be coded as 0. If group G and the reference group have different prediction modes, the logical operation XOR may be coded as 1.

[0096] In one embodiment, a binary flag (e.g., 0 or 1) may be coded to indicate whether group G and the reference group of group G have the same prediction mode. Thus, if group G and the reference group have the same prediction mode, a binary 1 may be coded. If group G and the reference group have different prediction modes, a binary 0 may be coded.

[0097] In one embodiment, the prediction mode of group G in positionally tracked frame M(t) may be coded using temporal context-based arithmetic coding, and the context of the temporal context-based arithmetic coding may be based on the prediction mode of a reference group of group G.

[0098] In some embodiments, the prediction residual (e.g., E or (E-E)) may be coded using a coding algorithm such as fixed-length coding, exponential Golomb coding, arithmetic coding, etc. In some embodiments, the prediction residual may undergo a compaction transform such as a fast Fourier transform (FFT), a discrete cosine transform (DCT), a discrete sine transform (DST), a discrete wavelet transform (DWT), etc. The output from the compaction transform may be coded using a coding algorithm such as fixed-length coding, exponential Golomb coding, arithmetic coding, etc.

[0099] 6 shows a flowchart outlining a process (600) according to one embodiment of the present disclosure. The process (600) can be used in an encoder, such as a video encoder. In various embodiments, the process (600) is performed by a processing circuit, such as a processing circuit performing the functions of a video encoder (103), a processing circuit performing the functions of a video encoder (303), or the like. In some embodiments, the process (600) is implemented with software instructions, and thus, the processing circuit performs the process (600) when it executes the software instructions. The process starts at (S601) and proceeds to (S610).

[0100] At (S610), it is determined whether each of the plurality of duplicate vertices can be skipped in the mesh of the current frame. Each of the plurality of duplicate vertices is a vertex of the mesh that is a duplicate of another corresponding vertex in the mesh. Each of the plurality of duplicate vertices and its corresponding vertex have the same reference vertex in the reference frame.

[0101] In (S620), one or more skippable duplicate vertices of the plurality of duplicate vertices in the mesh are eliminated to generate an updated mesh.

[0102] At (S630), the vertices of the updated mesh are divided into a plurality of vertex groups based on predetermined constant integers, and the plurality of vertex groups includes a first vertex group.

[0103] At (S640), a prediction mode for the first vertex group is determined based at least on estimation errors of vertices in the first vertex group of the updated mesh, each estimation error indicating a difference between a respective vertex in the first vertex group and a corresponding reference vertex in the reference frame.

[0104] At (S650), prediction information for the vertices in the first vertex group is generated based on the determined prediction mode of the first vertex group.

[0105] In one example, a first of the plurality of overlapping vertices is determined to be skippable based on a first of the plurality of overlapping vertices and a vertex in the mesh that corresponds to the first overlapping vertex having the same coordinates. In one example, a second of the plurality of overlapping vertices is determined to be non-skippable based on a second of the plurality of overlapping vertices and a vertex in the mesh that corresponds to the second overlapping vertex having different coordinates.

[0106] In some embodiments, the number of non-skippable duplicate vertices of the plurality of duplicate vertices is determined, and further, the vertex index of each non-skippable duplicate vertex of the plurality of duplicate vertices is determined.

[0107] In some embodiments, the coding information is generated based on one of (i) the number of the plurality of duplicate vertices being greater than or equal to a first threshold, (ii) the number of vertices in the mesh being greater than or equal to a second threshold, and (iii) a ratio of the number of the plurality of duplicate vertices to the number of vertices in the mesh being greater than or equal to a third threshold, wherein the coding information indicates (i) a number of non-skippable duplicate vertices among the plurality of duplicate vertices, and (ii) a vertex index for each non-skippable duplicate vertex among the plurality of duplicate vertices.

[0108] In one embodiment, the number of non-skippable duplicate vertices is coded based on one of variable length coding, fixed length coding, a (b+1)-bit fixed length code, and a fixed length code having a code length less than (b+1) bits, where b is a positive integer. In one embodiment, the vertex index of each non-skippable duplicate vertex of the plurality of duplicate vertices is coded based on one of differential coding, variable length coding, Exp-Golomb coding, a (b+1)-bit fixed length code, and a fixed length code having a code length less than (b+1) bits.

[0109] In some embodiments, to determine a prediction mode for the first vertex group, estimation errors for vertices in the first vertex group of the updated mesh are determined. An average neighborhood estimation error for multiple neighboring vertices associated with each vertex in the first vertex group is determined. Each of the multiple neighborhood estimation errors indicates a difference between one of the multiple neighboring vertices of the respective vertex in the first vertex group and a reference vertex in the reference frame corresponding to the one of the multiple neighboring vertices of the respective vertex in the first vertex group. A first cost value is determined based on the estimation errors for the vertices in the first vertex group. A second cost value is determined based on the average neighborhood estimation error associated with the vertices in the first vertex group. A prediction mode for the first vertex group is determined based on a comparison between the first cost value and the second cost value.

[0110] In one example, to determine an average neighborhood estimation error for multiple neighboring vertices, a first neighborhood estimation error associated with a first vertex of the vertices in a first vertex group is determined. The first neighborhood estimation error indicates a difference between a first neighboring vertex of the first vertex and a reference vertex in a reference frame corresponding to the first neighboring vertex of the first vertex. A second neighborhood estimation error associated with the first vertex of the vertices in the first vertex group is determined. The second neighborhood estimation error indicates a difference between a second neighboring vertex of the first vertex and a reference vertex in a reference frame corresponding to the second neighboring vertex of the first vertex. An average neighborhood estimation error of the first neighborhood estimation error and the second neighborhood estimation error is determined, and the average neighborhood estimation error is associated with the first vertex of the vertices in the first vertex group.

[0111] In some embodiments, the first cost value is determined as the sum of the estimation errors of the vertices in the first group of vertices.

[0112] In some embodiments, an estimated difference between the estimated error of each vertex in the first group of vertices and the average neighborhood estimated error associated with the corresponding vertex in the first group of vertices is determined to determine the second cost value, and the second cost value is determined as a sum of the estimated differences associated with the vertices in the first group of vertices.

[0113] In one example, the prediction mode is determined as the first mode based on the first cost value being less than or equal to the second cost value. In another example, the prediction mode is determined as the second mode based on the first cost value being greater than the second cost value.

[0114] In some embodiments, a first prediction residual is generated for each of the vertices in the first vertex group based on the prediction mode being the first mode, the first prediction residual indicating an estimated error for the corresponding vertex in the first vertex group, and a second prediction residual is generated for each of the vertices in the first vertex group based on the prediction mode being the second mode, the second prediction residual indicating an estimated error associated with the corresponding vertex in the first vertex group.

[0115] Thereafter, the process proceeds to (S699) and ends.

[0116] Process 600 may be adapted as appropriate. Step(s) of process 600 may be modified and / or omitted. Additional step(s) may be added. Any suitable order of performance may be used.

[0117] 7 shows a flowchart outlining a process (700) according to one embodiment of the present disclosure. The process (700) may be used in a decoder, such as a video decoder. In various embodiments, the process (700) is performed by a processing circuit, such as a processing circuit performing the functions of the video decoder (110), a processing circuit performing the functions of the video decoder (210), or the like. In some embodiments, the process (700) is implemented by software instructions, such that the processing circuit performs the process (700) when it executes the software instructions. The process begins at (S701) and proceeds to (S710).

[0118] At (S710), coding information for a mesh in a current frame is received. The coding information indicates (i) the number of non-skippable duplicate vertices among a plurality of duplicate vertices in the mesh, and (ii) a vertex index of each of the non-skippable duplicate vertices. Each of the plurality of duplicate vertices is a vertex of the mesh that is a duplicate of another corresponding vertex in the mesh. Each of the plurality of duplicate vertices and its corresponding vertex have the same reference vertex in the reference frame.

[0119] In (S720), one or more skippable duplicate vertices of the plurality of duplicate vertices in the mesh are eliminated to generate an updated mesh.

[0120] (S 730 In step (a), the vertices of the updated mesh are divided into a plurality of vertex groups based on predetermined constant integers, and the plurality of vertex groups includes a first vertex group.

[0121] (S 740 ), a prediction mode for a first vertex group of the plurality of vertex groups is determined based on the coding information.

[0122] (S 750 ), the vertices in the first vertex group are reconstructed based on the determined prediction mode.

[0123] Thereafter, the process proceeds to (S799) and ends.

[0124] Process 700 may be adapted as appropriate. Steps of process 700 may be modified and / or omitted. Additional step(s) may be added. Any suitable order of performance may be used.

[0125] The techniques described above may be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 8 illustrates a computer system (800) suitable for implementing certain embodiments of the disclosed subject matter.

[0126] Computer software may be coded using any suitable machine code or computer language that can be subjected to mechanisms such as assembly, compilation, linking, etc. to create code containing instructions that can be executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc. directly, or via interpretation, microcode execution, etc.

[0127] The instructions may be executed on various types of computers or computer components including, for example, personal computers, tablet computers, servers, smartphones, gaming consoles, Internet of Things devices, and the like.

[0128] 8 for computer system (800) are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure, nor should the arrangement of components be interpreted as having a dependency or requirement related to any one or combination of components illustrated in the exemplary embodiment of computer system (800).

[0129] The computer system 800 may include certain human interface input devices. Such human interface input devices may respond to input by one or more human users, for example, via tactile input (such as keystrokes, swipes, or data glove movements), audio input (such as voice or claps), visual input (such as gestures), or olfactory input (not depicted). Human interface devices may also be used to capture certain media not necessarily directly associated with conscious human input, such as audio (such as voice, music, or ambient sounds), images (such as scanned images or photographic images obtained from a still camera), or video (such as two-dimensional video, three-dimensional video, including stereoscopic video).

[0130] The input human interface devices may include one or more (only one of each shown) of a keyboard (801), a mouse (802), a trackpad (803), a touch screen (810), a data glove (not shown), a joystick (805), a microphone (806), a scanner (807), and a camera (808).

[0131] The computer system (800) may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touchscreen (810), data gloves (not shown), or joystick (805), although haptic feedback devices that are not input devices may also exist), audio output devices (e.g., speakers (809), headphones (not shown)), visual output devices (e.g., screens (810), including CRT screens, LCD screens, plasma screens, and OLED screens, with or without touchscreen input capabilities and with or without haptic feedback capabilities, some of which may be capable of outputting two-dimensional visual output or three or more dimensional output via means such as stereoscopic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0132] The computer system (800) may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW (820), including media such as CD / DVD (821), thumb drives (822), removable hard drives or solid state drives (823), traditional magnetic media such as tape and floppy disks (not shown), dedicated ROM / ASIC / PLD-based devices such as security dongles (not shown), etc.

[0133] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter of this disclosure does not encompass transmission media, carrier waves, or other transitory signals.

[0134] The computer system (800) also includes an interface (854) to one or more communications networks (855). The networks may be, for example, wireless, wired, or optical. The networks may further be local, wide-area, metropolitan, vehicular, and industrial, real-time, delay-tolerant, and the like. Examples of networks include local area networks such as Ethernet and wireless LAN; cellular networks including GSM, 3G, 4G, 5G, and LTE; television wired or wireless wide-area digital networks including cable, satellite, and terrestrial broadcast television; and vehicular and industrial networks including CAN Bus. Certain networks generally require an external network interface adapter (e.g., a USB port on the computer system (800)) attached to a particular general-purpose data port or peripheral bus (849). Other networks are generally integrated into the core of the computer system (800) by attachment to a system bus, as described below (e.g., integrating an Ethernet interface into a PC computer system or integrating a cellular network interface into a smartphone computer system). Using any of these networks, the computer system (800) can communicate with other entities. Such communications may be one-way, receive-only (e.g., television broadcasts), one-way transmit-only (e.g., CANBus device to a particular CANBus device), or bidirectional, for example, to other computer systems using local or wide-area digital networks. Specific protocols and protocol stacks may be used with each of these networks and network interfaces, as described above.

[0135] The aforementioned human interface devices, human-accessible storage, and network interfaces may be attached to the core (840) of the computer system (800).

[0136] The core (840) may include one or more central processing units (CPUs) (841), graphics processing units (GPUs) (842), specialized programmable processing units in the form of field programmable gate arrays (FPGAs) (843), task-specific hardware accelerators (844), graphics adapters (850), etc. These devices may be connected via a system bus (848), along with read-only memory (ROM) (845), random access memory (846), internal mass storage (847) such as a non-user-accessible internal hard drive or SSD, etc. In some computer systems, the system bus (848) may be accessible in the form of one or more physical plugs, allowing expansion with additional CPUs, GPUs, etc. Peripheral devices may also be attached directly to the core's system bus (848) or via a peripheral bus (849). In one example, a screen (810) may be connected to the graphics adapter (850). Peripheral bus architectures include PCI, USB, etc.

[0137] The CPU (841), GPU (842), FPGA (843), and accelerator (844) may combine to execute specific instructions that may constitute the computer code described above. The computer code may be stored in ROM (845) or RAM (846). Temporary data may also be stored in RAM (846), while permanent data may be stored, for example, in internal mass storage (847). The use of cache memory, which may be closely associated with one or more CPUs (841), GPUs (842), mass storage (847), ROM (845), RAM (846), etc., may enable fast storage and retrieval from any of the memory devices.

[0138] The computer-readable medium may bear computer code for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0139] By way of example and not limitation, a computer system (800) having the architecture, and specifically the core (840), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media may be user-accessible mass storage, as described above, as well as media associated with the core's (840) specific storage of a non-transitory nature, such as the core's internal mass storage (847) or ROM (845). Software implementing various embodiments of the present disclosure may be stored on such devices and executed by the core (840). Depending on particular needs, the computer-readable media may include one or more memory devices or chips. The software may cause the core (840), and specifically the processor therein (including a CPU, GPU, FPGA, etc.), to perform certain processes or portions of certain processes described herein, including defining data structures stored in RAM (846) and modifying such data structures according to software-defined processes. Additionally or alternatively, a computer system may provide functionality as a result of hardwired or otherwise embodied logic in circuitry (e.g., accelerator (844)) that may operate in place of or together with software to perform particular processes or portions of particular processes described herein. References to software can encompass logic, where appropriate, and vice versa. References to computer-readable media can encompass circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry that embodies logic for execution, or both, where appropriate. The present disclosure encompasses any suitable combination of hardware and software.

[0140] The use of "at least one of" or "one of" in this disclosure is intended to include any one or combination of the listed elements. For example, reference to at least one of A, B, or C, i.e., at least one of A, B, and C; at least one of A, B, and / or C; at least one of A through C, is intended to include A only, B only, C only, or any combination thereof. Reference to one of A or B and one of A and B is intended to include A or B or (A and B). The use of "one of" does not exclude any combination of the listed elements where applicable, such as when the elements are not mutually exclusive.

[0141] While this disclosure describes several exemplary embodiments, there are alterations, permutations, and various substitute equivalents that fall within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within the spirit and scope of the present disclosure. [Explanation of symbols]

[0142] 100 Video Processing System 101 Video Sources 102 Stream 103 Video Encoder 104 encoded video data 105 Streaming Server 106 Client Subsystem 107 Copying Encoded Video Data 108 Client Subsystem 109 Copying Encoded Video Data 110 Video Decoder 111 Output Stream 112 Display 113 Capture Subsystem 120 Electronic Devices 130 Electronic Devices 201 Channel 210 Video Decoder 212 Rendering Devices 215 buffer memory 220 Parser 221 Symbol 230 Electronic Devices 231 Receiver 251 Scaler / Descaler Unit 252 Intra-picture prediction units 253 Motion Compensation Prediction Unit 255 Aggregator 256 Loop Filter Unit 257 Reference Picture Memory 258 picture buffer 301 Video Sources 303 Video Encoder 320 Electronic Devices 330 Source Coder 332 Coding Engine 333 Local Video Decoder 334 Reference Picture Memory 335 Predictor 340 Transmitter 343 coded video sequence 345 Entropy Coder 350 Controller 360 communication channels 500 Vertex Position Compressor 501 Duplicate Signaling Module 502 Vertex Grouping Module 504 Location Prediction Module 506 Prediction Mode Coding Module 508 Prediction Residual Coding Module 600 processes 700 processes 800 Computer Systems 801 keyboard 802 Mouse 803 Trackpad 805 Joystick 806 Microphone 807 Scanner 808 Camera 809 Speaker 810 screens 821 Medium 822 thumb drive 823 Solid State Drive 840 cores 841 Central Processing Unit (CPU) 842 Graphics Processing Unit (GPU) 843 Field Programmable Gate Area (FPGA) 844 Accelerator 845 Read-Only Memory (ROM) 846 Random Access Memory 847 Mass Storage 848 System Bus 849 Peripheral Bus 850 graphics adapter 854 Interface 855 Communication Network

Claims

1. A method of mesh processing performed by a video encoder, the method comprising: determining whether each of a plurality of duplicate vertices can be skipped in a mesh of a current frame, wherein each of the plurality of duplicate vertices is a vertex of the mesh that is a duplicate of another corresponding vertex in the mesh, and each of the plurality of duplicate vertices and its corresponding vertex have the same reference vertex in a reference frame; excluding one or more skippable duplicate vertices from the plurality of duplicate vertices in the mesh to generate an updated mesh; Dividing the vertices of the updated mesh into a plurality of vertex groups based on predetermined constant integers, the plurality of vertex groups including a first vertex group; determining a prediction mode for the first vertex group based at least on estimation errors of vertices in the first vertex group of the updated mesh, each of the estimation errors indicating a difference between a respective vertex in the first vertex group and a corresponding reference vertex in the reference frame; generating prediction information for the vertices in the first vertex group based on the determined prediction mode of the first vertex group; A method comprising:

2. The step of determining whether each of the plurality of overlapping vertices can be skipped comprises: determining, based on a first overlapping vertex of the plurality of overlapping vertices and a corresponding vertex in the mesh having the same coordinates, that the first overlapping vertex of the plurality of overlapping vertices can be skipped; determining that a second overlapping vertex of the plurality of overlapping vertices is non-skippable based on a second overlapping vertex of the plurality of overlapping vertices and a corresponding vertex in the mesh having different coordinates; The method of claim 1 further comprising:

3. determining a number of non-skippable overlapping vertices of the plurality of overlapping vertices; determining a vertex index for each of the non-skippable duplicate vertices of the plurality of duplicate vertices; 3. The method of claim 2, further comprising:

4. (i) the number of the plurality of duplicate vertices is greater than or equal to a first threshold; (ii) the number of the vertices in the mesh is greater than or equal to a second threshold; and (iii) the ratio of the number of the plurality of duplicate vertices to the number of the vertices in the mesh is greater than or equal to a third threshold. generating coding information indicating (i) the number of non-skippable overlapping vertices among the plurality of overlapping vertices, and (ii) the vertex index for each of the non-skippable overlapping vertices among the plurality of overlapping vertices.

4. The method of claim 3, further comprising:

5. the number of non-skippable duplicated vertices is coded based on one of variable length coding, fixed length coding, a (b+1)-bit fixed length code, and a fixed length code having a code length less than (b+1) bits, where b is a positive integer; the vertex index of each of the non-skippable overlapping vertices among the plurality of overlapping vertices is coded based on one of differential coding, the variable length coding, Exp-Golomb coding, the (b+1)-bit fixed length code, and the fixed length code having a code length smaller than (b+1) bits; The method of claim 4.

6. The step of determining the prediction mode for the first vertex group comprises: determining the estimated errors for the vertices in the first group of vertices in the updated mesh; determining an average neighborhood estimation error for a plurality of neighbor vertices associated with each of the vertices in the first vertex group, each of the plurality of average neighborhood estimation errors indicating a difference between one of the plurality of neighbor vertices of the respective vertex in the first vertex group and a reference vertex in the reference frame corresponding to the one of the plurality of neighbor vertices of the respective vertex in the first vertex group; determining a first cost value based on the estimation error of the vertices in the first group of vertices, and determining a second cost value based on the average neighborhood estimation error associated with the vertices in the first group of vertices; determining the prediction mode for the first vertex group based on a comparison of the first cost value and the second cost value; The method of claim 1 further comprising:

7. The step of determining the average neighborhood estimation error for the plurality of neighboring vertices comprises: determining a first neighborhood estimation error associated with a first one of the vertices in the first vertex group, the first neighborhood estimation error indicating a difference between a first neighboring one of the first vertices and a reference vertex in the reference frame that corresponds to the first neighboring one of the first vertices; determining a second neighborhood estimation error associated with the first one of the vertices in the first vertex group, the second neighborhood estimation error indicating a difference between a second neighboring one of the first vertices and a reference vertex in the reference frame that corresponds to the second neighboring one of the first vertices; determining the average neighborhood estimation error of the first neighborhood estimation error and the second neighborhood estimation error associated with the first one of the vertices in the first group of vertices; 7. The method of claim 6, further comprising:

8. The step of determining the first cost value comprises: determining the first cost value as a sum of the estimated errors of the vertices in the first group of vertices; 8. The method of claim 7, further comprising:

9. The step of determining the second cost value comprises: determining an estimated difference between the estimation error for each of the vertices in the first group of vertices and the average neighborhood estimation error associated with the corresponding vertex in the first group of vertices; determining the second cost value as a sum of the estimated differences associated with the vertices in the first group of vertices; 9. The method of claim 8, further comprising:

10. The step of determining the prediction mode comprises: determining the prediction mode as a first mode based on the first cost value being equal to or less than the second cost value; determining the prediction mode as a second mode based on the first cost value being greater than the second cost value; 10. The method of claim 9, further comprising:

11. The step of generating the prediction information comprises: generating a first prediction residual for each of the vertices in the first vertex group based on the prediction mode being the first mode, the first prediction residual indicating the estimation error of the respective vertex in the first vertex group; generating a second prediction residual for each of the vertices in the first vertex group based on the prediction mode being the second mode, the second prediction residual indicating the estimated difference associated with the respective vertex in the first vertex group; 11. The method of claim 10, further comprising:

12. An apparatus for processing meshes, configured to perform the method according to any one of claims 1 to 11.

13. A computer program for causing a computer to carry out the method according to any one of claims 1 to 11.

Images