Method, apparatus, and storage medium for video encoding and decoding

Abstract

A method, apparatus, and storage medium for video encoding and decoding are provided. A method and apparatus including computer code configured to cause one or more processors to obtain encoded video data representing a mesh as volume data of at least one three-dimensional (3D) visual content, determine whether first two prediction values in an initial set of prediction values have a same value, based on determining that the first two prediction values have the same value, reduce the initial set of prediction values to only a single prediction value, and based on the single prediction value, decode at least a portion of the volume data.

Description

[0001] Related cross-references This application claims priority to U.S. Provisional Application No. 63 / 799,088, filed May 2, 2025, and U.S. Application No. 19 / 570,482, filed March 18, 2026, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0002] This disclosure relates to adaptively reducing the set of predicted values ​​and / or modifying the predicted values ​​based on an initial set of predicted values. Specifically, it relates to a method, apparatus, and storage medium for video encoding and decoding. Background Technology

[0003] Advances in 3D capture, modeling, and rendering have facilitated the widespread adoption of 3D content across various platforms and devices. Today, it's possible to capture a baby's first steps on one continent and allow grandparents on other continents to see (and perhaps interact with) the child, enjoying a fully immersive experience. However, to achieve this realism, the models have become increasingly complex, and their creation and use involve massive amounts of data. 3D meshes are widely used to represent this immersive content.

[0004] A mesh consists of multiple polygons that describe the surface of a volumetric object. Each polygon is defined by its vertices in three-dimensional space and information about how these vertices are connected (called connectivity information). Optionally, vertex attributes (such as color, normals, etc.) can be associated with mesh vertices. Attributes can also be associated with the mesh surface using mapping information that parameterizes the mesh using a two-dimensional property map. This mapping is typically described by a set of parametric coordinates (called UV coordinates or texture coordinates) associated with the mesh vertices. The two-dimensional property map is used to store high-resolution attribute information (such as texture, normals, displacements, etc.). This information can be used for various purposes, such as texture mapping and shading.

[0005] Dynamic mesh sequences can require large amounts of data due to the large amount of time-varying information they contain. Therefore, efficient compression techniques are needed to store and transmit this content. The Moving Picture Experts Group (MPEG) has previously developed mesh compression standards IC, MESHGRID, and FAMC to handle dynamic meshes with constant connectivity and time-varying geometry and vertex properties. However, these standards do not consider time-varying property graphs and connectivity information. Digital Content Creation (DCC) tools often generate such dynamic meshes. Therefore, generating dynamic meshes with constant connectivity is challenging for volumetric acquisition techniques, especially under real-time constraints. Existing standards do not support this type of content. MPEG plans to develop a new mesh compression standard to directly handle dynamic meshes with time-varying connectivity information and optional time-varying property graphs. This standard targets lossy and lossless compression for various applications such as real-time communication, storage, free-viewpoint video, augmented reality (AR), and virtual reality (VR). Features such as random access and scalable / progressive coding are also considered.

[0006] Besides connectivity, meshes typically include other attributes such as vertex positions, texture coordinates, normals, and associated texture maps. Connectivity usually constitutes only a small portion of the mesh data, possibly less than 10% or even 1% of the total bitstream, while positional attributes can account for more than half of the bitstream.

[0007] Parallelogram prediction is only suitable for relatively flat surfaces and equidistant surface structures, while cross-surface prediction is extremely challenging even for height-number polygon meshes. Therefore, effective solutions for handling irregular surfaces such as slender triangles are still needed.

[0008] In other words, mesh encoding involves connectivity encoding and attribute value encoding, where attributes can be location, texture coordinates, normals, etc. Attributes can be defined by mesh vertices, corners, and / or faces. Draco is a mature mesh encoding library for popular 2-bit networks, while VVM is a developing mesh encoding standard for polygonal meshes. However, in the current VVM standard, encoding predictor indices can be redundant in some cases, while in others, using a set of predictor indices can avoid this, although the set of predictor indices is generally fixed to contain four predictor indices.

[0009] Therefore, given this, it is desirable to have a technical solution to address these problems in video coding technology. Summary of the Invention

[0010] A method for video decoding is provided, the method being executed by at least one processor and comprising: acquiring encoded video data in which a grid is represented as at least one three-dimensional (3D) visual content; determining whether the first two predicted values ​​in an initial set of predicted values ​​have the same value; reducing the initial set of predicted values ​​to only a single predicted value based on the determination that the first two predicted values ​​have the same value; and decoding at least a portion of the volume data based on the single predicted value.

[0011] A video encoding method is provided, the method being executed by at least one processor and comprising: acquiring video data of volume data in which a grid is represented as at least one three-dimensional (3D) visual content; determining whether the first two predicted values ​​in an initial set of predicted values ​​have the same value; reducing the initial set of predicted values ​​to only a single predicted value based on the determination that the first two predicted values ​​have the same value; and encoding at least a portion of the volume data based on the single predicted value.

[0012] A method for encoding visual media data is provided, the method comprising generating a bitstream of visual media data according to an encoding process, the bitstream of visual media data comprising video data of volume data in which a grid is represented as at least one three-dimensional (3D) visual content, the encoding process comprising: determining whether the first two predicted values ​​in an initial set of predicted values ​​have the same value; reducing the initial set of predicted values ​​to only a single predicted value based on the determination that the first two predicted values ​​have the same value; and encoding at least a portion of the volume data based on the single predicted value, and transmitting the generated bitstream.

[0013] The initial set of predictions represents the predictions for the current face and one or more adjacent faces of the grid.

[0014] The initial set of predictions includes four predictions, and one or more adjacent faces of the grid are composed of three adjacent faces of the grid.

[0015] The order of predictions in the initial set of predictions is determined based on the centroids of one or more adjacent faces of the grid and the distance between those centroids.

[0016] The initial set of predictions includes two predictions, and one or more adjacent faces of the grid are composed of two adjacent faces of the grid.

[0017] The order of predictions in the initial set of predictions is determined based on the centroids of one or more adjacent faces of the grid and the distance between those centroids.

[0018] The predicted face attribute value is at least one of the face group identifier (ID) and the face normal. Attached Figure Description

[0019] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which: Figure 1 This is a block diagram of a communication system according to an embodiment; Figure 2 This is a block diagram of a video encoder and decoder in a video streaming environment according to an embodiment; Figure 3 This is a block diagram of a video decoder according to an embodiment; Figure 4 This is a block diagram of a video encoder according to an embodiment; Figure 5 This is a block diagram of the content streaming processing flow in a video streaming environment according to an embodiment; Figure 6 This is a block diagram of the content streaming processing flow in a video streaming environment according to an embodiment; Figure 7 This is a block diagram of a video encoder and decoder in a video streaming environment according to an embodiment; Figure 8 This is a simplified illustration of mapping three-dimensional mesh segments to two-dimensional charts according to an embodiment; Figure 9 This is a simplified illustration of deriving triangulation in a block with boundary vertices according to an embodiment; Figure 10 This is a simplified flowchart of the features of the prediction pattern according to the embodiments; Figure 11 This is a simplified flowchart of media processing according to an embodiment; Figure 12 This is a simplified flowchart of media processing according to an embodiment; Figure 13 This is a simplified illustration of vertex prediction according to an embodiment; Figure 14 This is a simplified illustration of progressive coding according to an embodiment; Figure 15 These are simplified illustrations of various instances of grid representation according to the embodiments; Figure 16 An example illustration of dual-degree grid coding according to embodiments of this document is shown; Figure 17 Example illustrations of the middle point and forward prediction according to embodiments of this article are shown; Figure 18 Example illustrations of multiple midpoint candidates are shown according to embodiments of this article; Figure 19An example illustration of reflection symmetry according to embodiments of this article is shown; Figure 20 An example illustration of grid traversal according to embodiments of this article is shown; Figure 21 An example flowchart of the sorting of predicted values ​​according to embodiments of this article is shown; Figure 22 , Figure 23 and Figure 24 Comparative example features of lossless general public text conditional features according to embodiments of this document are shown; Figure 25 , Figure 26 and Figure 27 Comparative example features of lossless sequential common text conditional features according to embodiments of this document are shown; Figure 28 Comparative example features regarding lossy public text conditional features are shown according to embodiments of this document; and Figure 29 This is a simplified diagram of computer features according to an embodiment. Detailed Implementation

[0020] The features discussed below can be used individually or in any combination in any order. Furthermore, these embodiments can be implemented using processing circuitry, such as one or more processors or one or more integrated circuits. In one example, one or more processors execute a program stored on a non-transitory computer-readable medium.

[0021] Figure 1 A simplified block diagram of a communication system 100 according to an embodiment of the present disclosure is shown. The communication system 100 may include at least two terminals 102 and 103 interconnected via a network 105. For unidirectional data transmission, the first terminal 103 may encode video data locally for transmission to the other terminal 102 via the network 105. The second terminal 102 may receive the encoded video data from the other terminal via the network 105, decode the encoded data, and display the recovered video data. Unidirectional data transmission is common in applications such as media services.

[0022] Figure 1 A second terminal pair 101 and 104 is shown for supporting bidirectional transmission of encoded video (e.g., during a video conference). For bidirectional data transmission, each terminal 101 and 104 can encode video data acquired locally and transmit it to the other terminal via network 105. Each terminal 101 and 104 can also receive encoded video data transmitted from the other terminal, decode the encoded data, and display the recovered video data on a local display device.

[0023] exist Figure 1 In this disclosure, terminals 101, 102, 103, and 104 may be shown as servers, personal computers, and smartphones, but the principles of this disclosure are not limited thereto. Embodiments of this disclosure are applicable to laptops, tablets, media players, and / or dedicated video conferencing equipment. Network 105 refers to any number of networks, including, for example, wired and / or wireless communication networks, that transmit encoded video data between terminals 101, 102, 103, and 104. Communication network 105 may exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet. For the purposes of this application, unless otherwise stated below, the architecture and topology of network 105 may be irrelevant to the operation of this disclosure.

[0024] As an example of the application of the disclosed topic. Figure 2 The placement of a video encoder and decoder in a streaming environment is illustrated. The disclosed subject matter is equally applicable to other video-enabled applications, including, for example, video conferencing, digital television, and storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0025] The streaming system may include an acquisition subsystem 203, which may include a video source 201, such as a digital camera, that creates, for example, an uncompressed video sample stream 213. This sample stream 213 may be emphasized as a high-data-volume sample stream compared to an encoded video stream and may be processed by an encoder 202 coupled to the video source 201, which may be, for example, the camera described above. The encoder 202 may include hardware, software, or a combination of hardware and software to implement or enforce aspects of the disclosed subject matter as described in more detail below. The encoded video stream 204 may be emphasized as a low-data-volume encoded video stream compared to the sample stream and may be stored in a streaming server 205 for future use. One or more streaming clients 212 and 207 may access the streaming server 205 to retrieve copies 208 and 206 of the encoded video stream 204. Client 212 may include video decoder 211, which decodes an incoming copy of the encoded video stream 208 and creates an output video sample stream 210 that can be displayed on display 209 or other presentation device (not depicted). In some streaming systems, video streams 204, 206, and 208 may be encoded according to certain video encoding / compression standards. Examples of these standards are described above and further below.

[0026] Figure 3 This can be a functional block diagram of a video decoder 300 according to an embodiment of the present invention.

[0027] Receiver 302 can receive one or more encoded video sequences to be decoded by decoder 300; in the same or another embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the others. The encoded video sequences can be received from channel 301, which can be a hardware / software link to a storage device storing the encoded video data. Receiver 302 can receive encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that can be forwarded to their respective user entities (not depicted). Receiver 302 can separate the encoded video sequences from other data. To prevent network jitter, buffer memory 303 can be coupled between receiver 302 and entropy decoder / resolver 304 (hereinafter referred to as "resolver"). Buffer 303 may not be necessary or can be made small when receiver 302 receives data from a store / forward device with sufficient bandwidth and controllability or from an isochronous synchronization network. Buffer 303 may be required for use on packet-switched networks such as the Internet, and this buffer can be relatively large and advantageously have an adaptive size.

[0028] Video decoder 300 may include parser 304 to reconstruct symbols 313 from an entropy-encoded video sequence. These symbols may include information for managing the operation of video decoder 300, and potential information for controlling a presentation device, such as display screen 312, which is not part of the decoder but may be coupled to it. Control information for the presentation device may be in the form of Supplemental Enhancement Information (SEI) messages or fragments of Video Usability Information (VUI) parameter sets (not depicted). Parser 304 may perform parsing / entropy decoding on the received encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow various principles known to those skilled in the art, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. Parser 304 may extract a subset of parameters from the encoded video sequence for use with pixels in the video decoder, based on at least one parameter corresponding to a group. Subgroups can include Group of Pictures (GOP), pictures, tiles, slices, macroblocks, coding units (CU), blocks, transform units (TU), prediction units (PU), etc. The entropy decoder / parser can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0029] The parser 304 can perform entropy decoding / parsing operations on the video sequence received from the buffer 303 to create symbols 313. The parser 304 can receive encoded data and selectively decode specific symbols 313. Furthermore, the parser 304 can determine whether a specific symbol 313 should be provided to the motion compensation prediction unit 306, the scaler / inverse transform unit 305, the intra-frame prediction unit 307, or the loop filter 311.

[0030] Depending on the type of encoded video frames or a subset of encoded video frames (e.g., inter-frame and intra-frame frames, inter-frame and intra-frame blocks) and other factors, the reconstruction of symbol 311 may involve multiple different units. Which units are involved and how they are involved can be controlled by the parser 304 through subgroup control information parsed from the encoded video sequence. For clarity, the flow of such subgroup control information between the parser 304 and the various units described below is not depicted.

[0031] In addition to the functional blocks already mentioned, the decoder 300 can be conceptually subdivided into multiple functional units as described below. In a practical implementation operating under commercial constraints, many of these units interact closely with each other and can be at least partially integrated with one another. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into multiple functional units as described below.

[0032] The first unit is the scaler / inverse transform unit 305. The scaler / inverse transform unit 305 receives quantization transform coefficients as symbols 313 from the parser 304, along with control information including the transform to be used, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit can output blocks containing sample values, which can be input into the aggregator 310.

[0033] In some cases, the output samples of the scaler / inverse transform unit 305 may belong to intra-coded blocks; that is, blocks that do not use prediction information from previously reconstructed images, but can use prediction information from previously reconstructed portions of the current image. Such prediction information can be provided by the intra-picture prediction unit 307. In some cases, the intra-picture prediction unit 307 uses the reconstructed information extracted from the current (partially reconstructed) image memory 309 to generate blocks of the same size and shape as the blocks being reconstructed. In some cases, the aggregator 310 adds the prediction information generated by the intra-picture prediction unit 307 to the output sample information provided by the scaler / inverse transform unit 305 based on each sample.

[0034] In other cases, the output samples of the scaler / inverse transform unit 305 may belong to blocks of inter-frame coding and potential motion compensation. In this case, the motion compensation prediction unit 306 can access the reference image memory 308 to extract samples for prediction. After motion compensation is performed on the extracted samples according to the symbols 313 belonging to the block, these samples can be added by the aggregator 310 to the output of the scaler / inverse transform unit (in this case, referred to as residual samples or residual signals) to generate output sample information. The extraction of prediction samples from the address in the reference image memory by the motion compensation unit can be controlled by motion vectors, which can be available to the motion compensation unit in the form of symbols 313, such as X, Y, and reference image components. Motion compensation may also include interpolation of sample values ​​extracted from the reference image memory, motion vector prediction mechanisms, etc., when using subsample precise motion vectors.

[0035] The output samples of aggregator 310 can be employed by various loop filtering techniques in loop filter unit 311. Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video bitstream, which can be used by loop filter unit 311 as symbols 313 from parser 304; however, video compression may also respond to metadata acquired during decoding of previous (in decoding order) portions of the encoded picture or encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0036] The output of the loop filter unit 311 can be a sample stream, which can be output to the presentation device 312 and stored in the reference image memory 557 for future inter-frame image prediction.

[0037] Once fully reconstructed, certain encoded images can be used as reference images for future predictions. Once the encoded images have been fully reconstructed and the encoded images (via, for example, parser 304) are identified as reference images, the current reference image 309 can become part of the reference image buffer 308, and new current image memory can be reallocated before reconstructing subsequent encoded images begins.

[0038] The video decoder 300 can perform decoding operations according to predetermined video compression techniques described in standards such as ITU-T H.265. The encoded video sequence may conform to the syntax specified by the video compression technique or standard used, in the sense specified in the video compression technique document or standard, and specifically in the configuration file. For compliance, it may also be necessary for the complexity of the encoded video sequence to be within the limits defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum image size, maximum frame rate, maximum reconstruction sampling rate (measured in megasamples per second, for example), maximum reference image size, etc. In some cases, the limitations set by the hierarchy can be further limited by the Hypothetical Reference Decoder (HRD) specification and metadata managed by the HRD buffer, which is represented by signals in the encoded video sequence.

[0039] In one embodiment, receiver 302 may receive additional (redundant) data when receiving encoded video. The additional data may be included as part of the encoded video sequence. The additional data may be used by video decoder 300 to properly decode the data and / or more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0040] Figure 4 This may be a functional block diagram of a video encoder 400 according to an embodiment of the present disclosure.

[0041] Encoder 400 can receive video samples from video source 401 (which is not part of the encoder), which can capture video images that will be encoded by encoder 400.

[0042] Video source 401 provides a source video sequence in the form of a digital video sample stream to be encoded by video encoder 303. This digital video sample stream can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601 YCrCb, RGB, etc.), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, video source 401 can be a storage device storing previously prepared video. In a video conferencing system, video source 401 can be a camera capturing local image information as a video sequence. The video data can be provided as multiple individual pictures, which are given motion when viewed sequentially. The pictures themselves can be constructed as spatial pixel arrays, where each pixel may include one or more samples, depending on the sampling structure, color space, etc., used. Those skilled in the art can easily understand the relationship between pixels and samples. The following focuses on describing samples.

[0043] According to one embodiment, encoder 400 can encode and compress images of a source video sequence into an encoded video sequence 410 in real time or under any other time constraints required by the application. Implementing an appropriate encoding rate is a function of controller 402. The controller controls and is functionally coupled to other functional units described below. For clarity, the coupling is not depicted in the figures. Parameters set by the controller may include rate control-related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. Those skilled in the art will readily recognize other functions of controller 402 that may relate to a video encoder 400 optimized for a particular system design.

[0044] Some video encoders operate in a “coding loop” manner, readily recognized by those skilled in the art. In a very concise description, the coding loop may consist of an encoding portion of encoder 400 (hereinafter referred to as the “source encoder”) (responsible for creating symbols based on the input image to be encoded and a reference image) and a (local) decoder 406 embedded in encoder 400, which reconstructs the symbols to create sample data that a (remote) decoder would also create (since any compression between the symbols and the encoded video bitstream is lossless in the video compression techniques considered in the disclosed subject matter). The reconstructed sample stream is fed into a reference image memory 405. Since the decoding of the symbol stream produces bit-precise results independent of the decoder’s location (local or remote), the contents of the reference image buffer also correspond bit-precisely between the local encoder and the remote encoder. In other words, the reference image samples “seen” by the encoder’s prediction portion are exactly the same as the sample values ​​that the decoder will “see” during the prediction process. This fundamental principle of reference image synchronization (and the drift that occurs, for example, due to channel errors, when synchronization cannot be maintained) is known to those skilled in the art.

[0045] The operation of the "local" decoder 406 can be combined with the above. Figure 3 The operation is the same as the "remote" decoder 300 described in detail. However, a brief reference is provided. Figure 4 Since symbols are available and the entropy encoder 408 and the parser 304 can encode / decode symbols into an encoded video sequence without loss, the entropy decoding portion of the decoder 300, which includes the channel 301, receiver 302, buffer 303 and parser 304, may not be fully implemented in the local decoder 406.

[0046] It can be observed that, apart from the need for parsing / entropy decoding present in the decoder, the decoder technique exists in the corresponding encoder in essentially the same functional form. The description of the encoder technique can be simplified because it is the inverse of the fully described decoder technique. More detailed descriptions are only required in certain areas, and are provided below.

[0047] As part of the operation, the source encoder 403 may perform motion-compensated predictive coding, referencing one or more previously encoded frames from the video sequence designated as "reference frames," to predictively encode the input frame. Thus, the encoding engine 407 encodes the differences between pixel blocks in the input frame and pixel blocks in the reference frame, which can be selected as the prediction reference for the input frame.

[0048] The local video decoder 406 can decode encoded video data of a frame that can be designated as a reference frame, based on symbols created by the source encoder 403. The operation of the encoding engine 407 can advantageously be a lossy process. When the encoded video data can be decoded by the video decoder (… Figure 4 When decoded at (not shown), the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder 406 replicates the decoding process, which can be performed by the video decoder on the reference frame, and allows the reconstructed reference frame to be stored in the reference image memory 405. In this way, the encoder 400 can locally store a copy of the reconstructed reference frame that shares common content (no transmission errors) with the reconstructed reference frame that will be acquired by the remote video decoder.

[0049] Predictor 404 can perform a prediction search against encoding engine 407. That is, for a new frame to be encoded, predictor 404 can search the reference image memory 405 for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can be used as appropriate prediction references for the new image. Predictor 404 can operate pixel-by-pixel based on the sample blocks to find suitable prediction references. In some cases, based on the search results obtained by predictor 404, it can be determined that the input image may have prediction references obtained from multiple reference images stored in reference image memory 405.

[0050] The controller 402 can manage the encoding operations of the source encoder 403, which may be, for example, a video encoder. The operations include, for example, setting parameters and subgroup parameters for encoding video data.

[0051] The outputs of all the aforementioned functional units can be entropy encoded in the entropy encoder 408. The entropy encoder performs lossless compression on the symbols generated by the various functional units using techniques known to those skilled in the art, such as Huffman coding, variable-length coding, and arithmetic coding, thereby converting the symbols into an encoded video sequence.

[0052] Transmitter 409 can buffer the encoded video sequence created by entropy encoder 408, thereby preparing it for transmission via communication channel 411, which may be a hardware / software link to a storage device that will store the encoded video data. Transmitter 409 can combine the encoded video data from source encoder 403 with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0053] The controller 402 can manage the operation of the encoder 400. During encoding, the controller 402 can assign a specific encoded image type to each encoded image, but this may affect the encoding techniques applicable to the corresponding images. For example, images can typically be assigned to one of the following frame types: An intra-frame picture (I-picture) is a picture that can be encoded and decoded without using any other frames in the sequence as a prediction source. Some video codecs allow different types of intra-frame pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art will understand variations of I-pictures and their corresponding applications and characteristics; Predictive images (P-images) can be images that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most one motion vector and reference index to predict sample values ​​for each block. Bidirectional predictive images (B-images) can be images that can be encoded and decoded using intra-frame or inter-frame prediction, which uses at most two motion vectors and a reference index to predict sample values ​​for each block. Similarly, multiple predictive images can use more than two reference images and associated metadata to reconstruct a single block.

[0054] Source images are typically spatially subdivided into multiple sample blocks (e.g., 4×4, 8×8, 4×8, or 16×16 samples), and each block is encoded sequentially. These blocks can be predictively coded with reference to other (already coded) blocks, which are determined by the coding assignments of the corresponding images applied to the blocks. For example, blocks of an I-image can be nonpredictively coded, or blocks of an I-image can be predictively coded (spatial or intra-frame prediction) with reference to already coded blocks of the same image. Pixel blocks of a P-image can be nonpredictively coded with reference to a previously coded reference image via spatial or temporal prediction. Blocks of a B-image can be nonpredictively coded with reference to one or two previously coded reference images via spatial or temporal prediction.

[0055] Encoder 400 (which may be, for example, a video encoder) performs encoding operations according to a predetermined video coding technique or standard, such as ITU-T H.265 Recommendation. In operation, encoder 400 can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data conforms to the syntax specified by the video coding technique or standard used.

[0056] In this embodiment, transmitter 409 may transmit additional data while transmitting encoded video. Source encoder 403 may include such data as part of the encoded video sequence. Additional data may include temporal / spatial / SNR enhancement layers, redundant images and slices, other forms of redundant data, supplementary enhancement information (SEI) messages, fragments of visual usability information (VUI) parameter sets, etc.

[0057] Figure 5 A simplified box flowchart 500 illustrates exemplary viewport-related processing in the Omnidirectional Media Application Format (OMAF), which enables 360-degree virtual reality (VR360) streaming as described in OMAF.

[0058] At acquisition box 501, video data A is acquired. When image data can represent a scene in VR360, video data A may consist of multiple images and audio data from the same moment. At processing box 503, image B from the same moment is processed. i Perform one or more of the following processes: stitching, mapping to the projected image relative to one or more virtual reality (VR) angles or other angles / viewpoints, and packing by region. Additionally, metadata can be created to indicate any such processing information and other details to assist in the transport and rendering process.

[0059] For data D, at image encoding frame 505, the projected image is encoded as data E. i They are then combined into a media file and streamed in a viewport-independent manner; at video encoding frame 504, the video image is encoded into data E. v For example, a single-layer bitstream; for data B a At audio encoding frame 502, audio data can also be encoded into data E. a .

[0060] Data E a E v and E i The entire encoded bitstream F iAnd / or F can be stored on a (Content Delivery Network (CDN) / cloud) server and can typically be transmitted in its entirety, for example, at transmission frame 507 or otherwise to the OMAF player 520, and can be fully decoded by a decoder to present at least the area of ​​the decoded image corresponding to the current viewport to the user at display frame 516, based on various metadata, file playback, and orientation / viewport metadata (e.g., from head / eye tracking frame 508, the angle at which the user is viewing through the VR imaging device, which is related to the viewport specification of the device). A significant feature of VR360 is that only one viewport can be displayed at any given time. This feature can be used to improve the performance of omnidirectional video systems through selective transmission based on the user's viewport (or any other standard, such as recommended viewport timing metadata). For example, viewport-dependent transmission can be achieved through tile-based video encoding according to an exemplary embodiment.

[0061] Similar to the encoding frame described above, the OMAF player 520 according to the exemplary embodiment targets data F' and / or F' i Decapsulating one or more files / fragments in the metadata allows for the reverse processing of one or more aspects of this encoding in a similar manner; decoding audio data E' at audio decoding frame 510. i Decode video data E' at point 513 in the video decoding frame. v Decode image data E' at image decoding frame 514. i To set data B' at audio rendering frame 511 a Audio rendering is performed, and image rendering of data D' is performed at image rendering frame 515, thereby outputting display data A' in VR360 format at display frame 516 based on various metadata (such as orientation / viewport metadata). i and output audio data A' at speaker / headphone frame 512 s Various metadata can influence the data decoding and rendering process based on the various tracks, languages, qualities, and views selected by the user of the OMAF player 520; and it should be understood that the processing order described herein is presented as an exemplary embodiment, and may be implemented in other orders according to other exemplary embodiments.

[0062] Figure 6A simplified box-style content flow processing flowchart 600 is shown for performing view-position and angle-based point cloud data processing (referred to herein as "V-PCC") on (encoded) point cloud data for capturing / generating / (de)encoding / rendering / displaying 6-DOF media. It should be understood that the described features can be used individually or in any combination in any order, and the elements shown, such as those for encoding and decoding, can be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits), and according to an exemplary embodiment, one or more processors can execute a program stored in a non-transitory computer-readable medium.

[0063] Figure 600 illustrates an exemplary embodiment of streaming encoded point cloud data according to V-PCC.

[0064] At volumetric data acquisition box 601, a real-world visual scene or a computer-generated visual scene (or a combination thereof) can be captured by a set of camera devices or synthesized by a computer as volumetric data. This volumetric data (which may have any format) can be converted into a (quantized) point cloud data format through image processing at conversion to point cloud box 602. For example, according to an exemplary embodiment, data in the volumetric data can be converted into points in the point cloud region by region by extracting one or more of the following values ​​from the volumetric data and any associated data to the desired point cloud format. According to an exemplary embodiment, the volumetric data can be a three-dimensional dataset of a two-dimensional image, such as a slice of a two-dimensional projection from which a three-dimensional dataset can be projected. According to an exemplary embodiment, the point cloud data format includes representations of data points in one or more different spaces and can be used to represent volumetric data, and can provide improvements for sampling and data compression, such as with regard to temporal redundancy, and, for example, point cloud data in x, y, z format, representing color values ​​(e.g., RGB, etc.), brightness, intensity, etc. at each point of multiple points in the cloud data, and can be used in conjunction with progressive decoding, polygon meshing, direct rendering, and octree three-dimensional representations of two-dimensional quadtree data.

[0065] At frame 603 projected onto the image, the acquired point cloud data can be projected onto the 2D image and encoded into an image / video image using video-based point cloud coding (V-PCC). The projected point cloud data can consist of attributes, geometry, occupancy map, and other metadata used for point cloud data reconstruction, such as using painter algorithms, ray casting algorithms, (3D) binary space segmentation algorithms, etc.

[0066] On the other hand, at scene generator box 609, the scene generator can generate metadata for rendering and displaying 6-DOF media, for example, based on director's intentions or user preferences. Such 6-DOF media can include 360VR-like scene 3D viewing effects, supporting not only rotational changes on the 3D coordinate axes X, Y, Z, but also additional dimensions for forward / backward, up / down, and left / right movement for virtual experiences based on or at least on point cloud encoded data. Scene description metadata defines one or more scenes composed of encoded point cloud data and other media data (including VR360, light fields, audio, etc.), and as... Figure 6 As shown in the relevant description, it can be provided to one or more cloud servers and / or file / fragment encapsulation / decapsulation processing.

[0067] Following video encoding frame 604 and image encoding frame 605, similar to those described above (and as will be understood, audio encoding may also be as provided above), file / segment encapsulation frame 606 performs processing to combine the encoded point cloud data into a media file for file playback, or to process it into a sequence of initial segments and media fragments for streaming, according to a specific media container file format (e.g., one or more video container formats, and formats that may be used with DASH, etc.), as described in exemplary embodiments. The file container may also include scene description metadata, such as from scene generator frame 1109, into the file or segment.

[0068] According to an exemplary embodiment, the file is encapsulated based on scene description metadata to include at least one view position and one or more angle views at that view position in the 6DoF media, each angle view corresponding to one or more moments, such that the file can be transmitted on request based on user or creator input. Furthermore, according to an exemplary embodiment, fragments of the file may include one or more portions of the file, such as the portion in the 6DoF media indicating a single viewpoint and its corresponding angle at one or more moments; however, these are merely exemplary embodiments and can vary depending on various conditions such as network, user, creator capabilities, and input.

[0069] According to an exemplary embodiment, point cloud data is segmented into multiple two-dimensional / three-dimensional regions, which are independently encoded, for example, at one or more locations in video encoding frame 604 and image encoding frame 605. Each independently encoded point cloud data partition can then be encapsulated as a track in a file and / or segment at file / segment encapsulation frame 606. According to an exemplary embodiment, each point cloud track and / or metadata track may include some useful metadata for viewpoint / viewpoint-related processing.

[0070] According to an exemplary embodiment, metadata useful for viewpoint / viewpoint-related processing (such as metadata included in files and / or fragments encapsulated at file / fragment encapsulation frames) includes one or more of the following: layout information of indexed 2D / 3D partitions; (dynamic) mapping information associating 3D volume partitions with one or more 2D partitions (e.g., any of tiles / tile groups / slices / subpictures); the 3D position of each 3D partition in the 6DoF coordinate system; a representative list of viewpoint / viewpoints; a list of selected viewpoint / viewpoints corresponding to the 3D volume partition; an index of the 2D / 3D partition corresponding to the selected viewpoint / viewpoint; quality (rank) information for each 2D / 3D partition; and rendering information, for example, based on each 2D / 3D partition at each viewpoint / viewpoint. When such metadata is invoked, such as by a user of the V-PCC player or by a content creator targeting a user of the V-PCC player, more efficient processing can be performed on specific portions of the 6DoF media desired by such metadata, allowing the V-PCC player to transmit images of the desired portions of the 6DoF media that are of higher quality than other portions, rather than transmitting unused portions of the media.

[0071] A file or one or more fragments of a file from the file / fragment encapsulation box 606 can be directly transmitted to the V-PCC player 625 or a cloud server (such as the cloud server box 607) using a transmission mechanism (e.g., Dynamic Adaptive Streaming over HTTP (DASH)). At the cloud server box 607, the cloud server can extract one or more tracks and / or one or more specific two-dimensional / three-dimensional partitions from the file and can merge multiple encoded point cloud data into a single data.

[0072] Based on data such as the position / view tracking box 608, if the current viewing position and angle are defined in the 6DoF coordinate system in the client system, the viewpoint / view metadata can be transmitted from the file / fragment encapsulation box 606, or the cloud server box 607 can otherwise process files or fragments already stored in the cloud server; the cloud server can extract appropriate partitions from the stored files and merge them if necessary based on the metadata from the client system with V-PCC player 625, and the extracted data can be transmitted to the client in the form of files or fragments.

[0073] For this type of data, in the file / fragment decapsulation box 615, the file decapsulator processes the file or received fragment, extracts the encoded bitstream, and parses the metadata; then in the video decoding box 610 and the image decoding box 611, the encoded point cloud data is decoded and reconstructed into point cloud data in the point cloud reconstruction box 612. The reconstructed point cloud data can be displayed in the display box 614, and / or first synthesized in the scene compositing box 613 based on the scene description data of the scene generator box 609, based on one or more different scene descriptions.

[0074] In light of the foregoing, such exemplary V-PCC procedures offer advantages over the V-PCC standard, including one or more of the following: the ability to partition descriptions of multiple 2D / 3D regions, the ability to assemble encoded 2D / 3D partitions into a single compliant encoded video stream in a compressed domain, and the ability to extract compliant encoded streams from encoded 2D / 3D partitions of encoded images. Support for such V-PCC systems can be further improved by incorporating containerization into VVC streams to support mechanisms that include metadata carrying one or more of the aforementioned metadata.

[0075] In view of this, and according to the exemplary embodiments further described below, the term "mesh" refers to a combination of one or more polygons used to describe the surface of a volume object. Each polygon is defined by information about its vertices in three-dimensional space and how those vertices are connected (referred to as connectivity information). Optionally, vertex attributes (such as color, normals, etc.) may be associated with mesh vertices. Attributes can also be associated with the surface of the mesh using mapping information that parameterizes the mesh using a two-dimensional property graph. This mapping may be described by a set of parametric coordinates (referred to as UV coordinates or texture coordinates) associated with mesh vertices. The two-dimensional property graph is used to store high-resolution attribute information (such as texture, normals, displacements, etc.). According to exemplary embodiments, this information can be used for purposes such as texture mapping and shading.

[0076] However, dynamic mesh sequences can require large amounts of data due to the large amount of time-varying information they contain. For example, unlike a “static mesh” or “static mesh sequence” where the mesh information may not change between frames, a “dynamic mesh” or “dynamic mesh sequence” indicates the movement of the mesh vertices that changes between frames. Therefore, efficient compression techniques are needed to store and transmit such content. MPEG previously developed mesh compression standards IC, MESHGRID, and FAMC to handle dynamic meshes with constant connectivity and time-varying geometry and vertex properties. However, these standards do not take into account time-varying property graphs and connectivity information. DCC (Digital Content Creation) tools often generate such dynamic meshes. Accordingly, generating constant connectivity dynamic meshes using volumetric acquisition techniques is challenging, especially under real-time constraints. Existing standards do not support this type of content. Based on exemplary embodiments herein, aspects of a novel mesh compression standard for directly processing dynamic meshes with time-varying connectivity information and optionally time-varying property graphs are described, targeting lossy and lossless compression for various applications such as real-time communications, storage, free-viewpoint video, AR, and VR. Features such as random access and scalable / progressive coding were also considered.

[0077] Figure 7 An example framework 700 for dynamic mesh compression is shown, for example, a method based on sampling from a 2D atlas. Each frame of the input mesh 701 can be preprocessed through a series of operations, such as tracing, remeshing, parameterization, and voxelization. It is important to note that these operations may be encoder-only, i.e., not part of the decoding process, and this possibility can be signaled in the metadata via a flag, e.g., 0 indicating encoder-only operation and 1 indicating otherwise. A mesh 702 with a 2D UV atlas is then obtained, where each vertex of the mesh has one or more associated UV coordinates on the 2D atlas. Next, by sampling the 2D atlas, the mesh can be converted into multiple graphs, including a geometry graph and an attribute graph. These 2D graphs can then be encoded using a video / image codec (such as HEVC, VVC, AV1, AVS3, etc.). At the decoder 703, the mesh can be reconstructed from the decoded 2D graphs. Any post-processing and filtering can also be applied to the reconstructed mesh 704. It is important to note that additional metadata can be signaled to the decoder for use in 3D mesh reconstruction. It is important to note that the block boundary information of the boundary vertices (including uv and xyz coordinates) can be predicted, quantized, and entropy-encoded into the bitstream. The quantization step size can be configured at the encoder end to strike a tradeoff between quality and bitrate.

[0078] In some implementations, a 3D mesh can be divided into segments (or patches / blocks). According to an exemplary embodiment, one or more 3D mesh segments can be considered a "3D mesh." Each segment consists of a set of connected vertices associated with its geometric information, attribute information, and connectivity information. For example... Figure 8 As shown in Example 800 of the medium-volume data, the UV parameterization process 802, which maps 3D mesh fragments to 2D blocks (e.g., mapping to box 702 of the aforementioned 2D UV atlas), maps one or more mesh fragments 801 to a 2D block 803 in the 2D UV atlas 804. Each vertex (v) in the mesh fragment n All vertices in the 2D UV atlas will be assigned a set of 2D UV coordinates. Note that the vertices (v...) in the 2D block... n This forms connection components as its three-dimensional corresponding vertices. The geometric information, attribute information, and connectivity information of each vertex can also be inherited from its three-dimensional corresponding vertex. For example, information indicating that vertex v4 is directly connected to vertices v0, v5, v1, and v3 can be provided, and similarly, information about each of the other vertices can also be provided. Furthermore, according to an exemplary embodiment, this two-dimensional textured mesh will further indicate information (such as color information) in a per-facet manner (such as by each triangle's facets, for example, v2, v5, v3 forming a "facet").

[0079] For example, regarding Figure 8 For the features of Example 800, see [link to example]. Figure 9 Example 900 shows that a 3D mesh fragment 801 can also be mapped to multiple independent 2D blocks 901 and 902. In this case, a vertex in the 3D may correspond to multiple vertices in the 2D UV atlas. Figure 9 As shown, the same 3D mesh fragment is mapped onto multiple 2D blocks in the 2D UV atlas, instead of... Figure 8 The mapping is shown on a single block. For example, three-dimensional vertices v1 and v4 each have two two-dimensional correspondences v1 and v4. 1' and v4, v 4' Therefore, a general two-dimensional UV atlas of a three-dimensional mesh can be derived from, for example... Figure 14 The diagram shows a series of blocks, each of which may contain multiple (usually more than or equal to 3) vertices associated with its 3D geometric information, attribute information, and connectivity information.

[0080] Figure 9Example 903 illustrates the derivation of triangulation in a block having boundary vertices B0, B1, B2, B3, B4, B5, B6, and B7. When such information is provided, any triangulation method can be applied to create connectivity between vertices (including boundary vertices and sampled vertices). For example, for each vertex, the two nearest vertices are found. Alternatively, for all vertices, triangles are generated continuously until a minimum number of triangles is reached after a certain number of attempts. As shown in Example 903, there are various regular shapes, repeating triangles, and various oddly shaped triangles (which are typically closest to the boundary vertices) with their own unique dimensions, which may be the same as or different from any other triangle. Connectivity information can also be reconstructed via explicit signaling. According to an exemplary embodiment, if the polygon cannot be recovered via implicit rules, the encoder can signal connectivity information in the bitstream.

[0081] Boundary vertices B0, B1, B2, B3, B4, B5, B6, and B7 are defined in a two-dimensional UV space. Boundary edges can be determined by checking whether an edge appears only in one triangle. According to an exemplary embodiment, the following information about the boundary vertices is important and should be signaled in the bitstream: geometric information, such as three-dimensional XYZ coordinates (even if currently in two-dimensional UV parametric form) and two-dimensional UV coordinates.

[0082] For cases where a boundary vertex in 3D corresponds to multiple vertices in a 2D UV map, such as Figure 9 As shown, the mapping from 3D XUZ to 2D UV can be a one-to-many mapping. Therefore, UV-to-XYZ (or UV2XYZ) indices can be signaled to indicate the mapping relationship. UV2XYZ can be a one-dimensional array of indices used to map each 2D UV vertex to a 3D XYZ vertex.

[0083] According to an exemplary embodiment, in order to efficiently represent mesh signals, a subset of mesh vertices and the connectivity information between them can first be encoded. In the original mesh, since these vertices are sampled from the original mesh, there may be no connections between them. There are multiple ways to signal the connectivity information between these vertices; therefore, this subset is referred to as the base mesh or base vertices.

[0084] According to exemplary embodiments, various dynamic mesh compression methods are implemented as part of the aforementioned edge-based vertex prediction framework. These methods first encode a base mesh and then predict additional vertices based on the edge connectivity information of the base mesh. Note that these methods can be applied individually or in combination in any form.

[0085] For example, consider Figure 10 Example flowchart 1001 for vertex grouping used in prediction patterns. At S101, vertices within the grid are acquired, and at S102, they are divided into different groups for prediction, for example, see [link to flowchart]. Figure 9 In one example, patch / block segmentation is used for partitioning at S104. In another example, each patch / block is partitioned at S105. The decision at S103 to proceed to S104 or S105 can be signaled via flags or other means. In the case of S105, multiple vertices in the same patch / block form a prediction group and share the same prediction mode, while other vertices in the same patch / block may use a different prediction mode. Here, a "prediction mode" can be considered as a specific mode used by the decoder to predict the video content containing the patch. Prediction modes can be broadly categorized into intra-frame prediction modes and inter-frame prediction modes; and within each category, there may be different specific modes selected by the decoder. According to an exemplary embodiment, each group (i.e., a "prediction group") may share the same specific mode (e.g., an angle mode for a specific angle) or the same category of prediction modes (e.g., all intra-frame prediction modes, but predictions can be made at different angles). This grouping at S106 can be allocated across different layers by determining the number of vertices involved in each group. For example, according to an exemplary embodiment, every 64, 32, or 16 vertices within a patch / block, in scan order, will be assigned the same prediction mode, while other vertices may be assigned in a different manner. For each group, the prediction mode can be an intra-frame prediction mode or an inter-frame prediction mode. This can be notified or assigned via signaling. According to example flowchart 1000, if it is determined at S107 that a mesh frame or mesh slice is of intra-frame type (e.g., by checking whether the flag of the mesh frame or mesh slice indicates an intra-frame type), then all vertex groups within that mesh frame or mesh slice should use the intra-frame prediction mode; otherwise, at S108, an intra-frame or inter-frame prediction mode can be used for all vertices in each group.

[0086] Furthermore, for a set of grid vertices using intra-frame prediction mode, the vertices can only be predicted using previously encoded vertices within the same sub-partition of the current grid. According to an exemplary embodiment, this sub-partition can sometimes be the current grid itself; while for a set of grid vertices using inter-frame prediction mode, the vertices can only be predicted using previously encoded vertices from other grid frames. Each of the above pieces of information can be determined and signaled via flags or similar means. The prediction features may appear at S110, and the results of the prediction and signaling notification may appear at S111.

[0087] According to an exemplary embodiment, for each vertex in a set of vertices in example flowchart 1000 and flowchart 1100 below, the predicted residual will be a three-dimensional displacement vector indicating the current vertex's offset relative to its prediction. The residual of a set of vertices needs further compression. In one example, before entropy encoding, the residual of a set of vertices can be transformed and signaled accordingly at S111. The encoding of a set of displacement vectors can be processed using the following methods. For example, in one method, appropriate signaling is given for a set of displacement vectors, a subset of displacement vectors, or cases where all components are zero. In another embodiment, a flag is signaled for each displacement vector to indicate whether the vector has any non-zero components; if not, encoding for all components of that displacement vector can be skipped. Furthermore, in another embodiment, a flag is signaled for each set of displacement vectors to indicate whether the set has any non-zero vectors; if not, encoding for all displacement vectors in that set can be skipped. Furthermore, in another embodiment, a flag is signaled for each component of a set of displacement vectors to indicate whether the components of that set have any non-zero vectors; if not, encoding of the components for all displacement vectors in that set can be skipped. Furthermore, in another embodiment, signaling can be provided for cases where a set of displacement vectors or a component of that set of displacement vectors needs to be transformed; if not, the transformation can be skipped, and quantization / entropy encoding of the set or its components can be performed directly. Furthermore, in another embodiment, a flag is signaled for each set of displacement vectors to indicate whether the set needs to be transformed; if not, transformation encoding of all displacement vectors in that set can be skipped. Furthermore, in another embodiment, a flag is signaled for each component of a set of displacement vectors to indicate whether the components of that set need to be transformed; if not, transformation encoding of the components for all displacement vectors in that set can be skipped. The embodiments described in this paragraph (related to vertex prediction residual processing) can also be combined and implemented in parallel on different patches.

[0088] Figure 11An example flowchart 1100 is shown, in which, at S121, an acquired grid frame can be encoded as a whole data unit, meaning that all vertices or attributes of the grid frame may be interconnected. Alternatively, based on the determination at S122, the grid frame can be divided into smaller, independent sub-partitions at S123, a concept similar to slices or tiles in a two-dimensional video or image. At S124, a prediction type can be assigned to the encoded grid frame or encoded grid sub-partition. Possible prediction types include intra-frame coding and inter-frame coding. For intra-frame coding, at S125, prediction is only allowed based on the reconstructed portion of the same frame or slice. On the other hand, inter-frame prediction at S125 allows prediction based on previously encoded grid frames, in addition to intra-frame grid frame prediction. Furthermore, inter-frame prediction can be further subdivided into more subtypes, such as P-type or B-type. In P-type, only one prediction value can be used for prediction; while in B-type, two prediction values ​​from two previously encoded grid frames can be used to generate a prediction value. Taking a weighted average of two predictions is one example. When a grid frame is encoded as a whole, it can be considered an intra- or inter-frame encoded grid frame. For inter-frame grid frames, the P-type or B-type can be further identified via signaling notification. Alternatively, if the grid frame is encoded with further intra-frame segmentation, a prediction type is assigned to each sub-segment at S124. Each of the above pieces of information can be determined and signaled via flags, etc. Figure 10 Similar to S110 and S111, the predicted feature may appear at S126, and the result of the prediction and signaling notification may appear at S127.

[0089] Therefore, although dynamic mesh sequences may require a large amount of data due to the large amount of information that changes over time, efficient compression techniques are still needed to store and transmit such content. The features described in this paper improve efficiency by allowing the use of either previously decoded vertices in the same mesh frame (intra-frame prediction) or vertices in previously encoded mesh frames (inter-frame prediction).

[0090] Furthermore, an exemplary embodiment may generate the displacement vector of the third layer 1303 of the mesh based on the reconstructed vertices of one or more previous layers (such as the second layer 1302 and the first layer 1301). Assuming the index of the second layer 1302 is T, the predictor for the vertices in the third layer 1303 (T+1) is generated based on the reconstructed vertices of at least the current layer or the second layer 1302. Figure 13Example 1300 in illustrates such a layer-based prediction structure, which illustrates reconstruction-based vertex prediction: progressive vertex prediction using edge-based interpolation, where prediction values are generated based on previously decoded vertices rather than predicted vertices. The first layer 1301 may be a mesh enclosed by a first polygon 1340, whose vertices are the decoded vertices at the boundary and the interpolated vertices on the connecting lines between these decoded vertices. As progressive coding proceeds from the first layer 1301 to the second layer 1302, additional polygons 1341 may be formed by displacement vectors from the interpolated vertices of the first layer to the additional vertices of the second layer 1302, such that the total number of vertices of the second layer 1302 may be greater than the total number of vertices of the first layer 1301. Similarly, when proceeding to the third layer 1303, the additional vertices of the second layer 1302 and the decoded vertices of the first layer 1301 may then be encoded in a manner similar to the role played by decoded vertices when proceeding from the first layer 1301 to the second layer 1303; that is, a plurality of additional polygons may be formed. As Figure 14 shown in example 1400 in, which illustrates such progressive coding, in contrast to Figure 13 , in example 1400, during the process of proceeding from the first layer 1401 to the second layer 1403 and then to the third layer 1403, each additionally formed polygon may lie entirely within the polygon formed by the boundary of the first layer 1401.

[0091] Taking examples 1300 and / or 1400 as instances, in accordance with an exemplary embodiment, referring to Figure 12 the example flowchart 1200 in, since the interpolated vertices on the current layer are prediction values, these prediction values need to be reconstructed before they are used to generate the prediction values of the vertices of the next layer. The specific approach is: encoding the base mesh at S131; performing vertex prediction at S132; then adding the decoded displacement vector of the current layer to the prediction values of the vertices of, for example, layer 1302 at S133. Subsequently, the reconstructed vertices of this layer, together with all decoded vertices of previous layers (for example, checking the additional vertex values of these layers at S134), may be used at S135 to generate and signal the predicted vertices of the next layer 1303. This process may also be summarized as follows: let P[t](Vi) denote the prediction value of vertex Vi on layer t; let R[t](Vi) denote the reconstructed vertex Vi on layer t; let D[t](Vi) denote the displacement vector of vertex Vi on layer t; let f( ) denote a prediction generation function, which specifically may be the average value of two existing vertices. Then, in accordance with an exemplary embodiment, for each layer t, the following holds: P[t](Vi)=f(R[s|s<t](Vj), R[m|m<t](Vk)), where Vj and Vk are reconstructed vertices of previous layers R[t](Vi)=P[t](Vi)+D[t](Vi) Formula (1).

[0092] Then, all vertices in a mesh frame are divided into layer 0 (base mesh), layer 1, layer 2, and so on. The reconstruction of vertices in a certain layer depends on the reconstruction of vertices in previous layers. In the above, within the context of the 3D mesh representation, P, R, and D each represent a 3D vector. D is the decoded displacement vector, for which quantization may or may not be applied.

[0093] According to an exemplary embodiment, vertex prediction using reconstructed vertices may be applicable only to certain layers, such as layers 0 and 1. For other layers, vertex prediction can still use adjacent predicted vertices without adding displacement vectors to them for reconstruction. In this way, these other layers can be processed simultaneously without waiting for the previous layer to complete reconstruction. According to an exemplary embodiment, for each layer, signaling may be used to indicate whether to use reconstruction-based vertex prediction or prediction-based vertex prediction, or signaling may be used to indicate which layers (and subsequent layers) do not use reconstruction-based vertex prediction.

[0094] For vertex prediction, the displacement vectors generated by reconstructing the vertices can be directly quantized without further transformation (such as wavelet transform). For vertex prediction, the displacement vectors generated by other predicted vertices may require transformation, and the transformation coefficients of these displacement vectors can be quantized.

[0095] Therefore, dynamic mesh sequences can require large amounts of data due to the presence of a significant amount of time-varying information. Efficient compression techniques are needed to store and transmit such content. In the aforementioned interpolation-based vertex prediction method framework, a crucial step is the compression of the displacement vectors, which constitutes a major portion of the encoded bitstream and is the focus of this disclosure. The features of this disclosure alleviate this problem by providing such compression methods.

[0096] Furthermore, similar to the other examples mentioned above, even using those embodiments, dynamic mesh sequences may still require a large amount of data due to the presence of a significant amount of time-varying information, thus necessitating efficient compression techniques for storing and transmitting such content. A key advantage of the above-described method framework based on 2D atlas sampling lies in deriving connectivity information from sampled vertices and boundary vertices at the decoding end. This is a major part of the decoding process and the focus of further examples below.

[0097] According to an exemplary embodiment, at the encoding and decoding ends, the connectivity information of the base mesh can be derived from the decoded boundary vertices and the sampled vertices of each block.

[0098] Similar to the above, any triangulation method can be used to create connections between vertices, including boundary vertices and sampled vertices. According to an exemplary embodiment, the connection type can be signaled in a high-level syntax such as a sequence header or slice header.

[0099] As described above, connectivity information can also be reconstructed through explicit signaling for irregular triangular meshes. That is, if a polygon cannot be recovered using implicit rules, the encoder can signal this connectivity information in the bitstream. According to an exemplary embodiment, the overhead of this explicit signaling can be reduced based on the polygon's boundaries.

[0100] According to the embodiment, it is determined that only signaling notifications are sent to the connectivity information between the boundary vertices and the sampling locations, while the connectivity information between the sampling locations is inferred.

[0101] Furthermore, in any embodiment, connectivity information can be signaled via prediction, allowing the difference in inferred connectivity (as a prediction) between different grids to be signaled only in the bitstream.

[0102] It should be noted that, according to the exemplary embodiment, the deduced triangle orientation (e.g., each triangle is deduced in a clockwise or counterclockwise direction) can be signaled for all blocks in the high-level syntax (such as sequence header, slice header, etc.), or it can be fixed by the encoder and decoder (assuming). The deduced triangle orientation can also be signaled separately for each block.

[0103] Furthermore, any reconstructed mesh can have different connectivity than the original mesh. For example, the original mesh can be a triangular mesh, while the reconstructed mesh can be a polygonal mesh (such as a quadrilateral mesh).

[0104] According to an exemplary embodiment, connectivity information for any base vertices can be notified without signaling. Instead, the same algorithm can be used on both the encoder and decoder sides to derive the edges between base vertices. Furthermore, according to an exemplary embodiment, interpolation of predicted vertices for additional mesh vertices can be based on the edges derived from the base mesh.

[0105] According to an exemplary embodiment, a flag can be used to signal whether the connectivity information of the underlying vertex is obtained by signaling notification or by deduction, and the flag can be signaled at different levels of the bitstream (such as at the sequence level, frame level, etc.).

[0106] According to an exemplary embodiment, firstly, the encoder and decoder use the same algorithm to derive the edges between the base vertices. Then, they compare this to the original connection pattern of the base mesh vertices and signal the difference between the derived edges and the actual edges. Therefore, after decoding this difference, the original connection pattern of the base vertices can be recovered.

[0107] In one example, if a derived edge is determined to be incorrect when compared with the original edge, this information can be signaled in the bitstream (by indicating the pair of vertices forming the edge); and if an original edge is not derived, this information can be signaled in the bitstream (by indicating the pair of vertices forming the edge). Furthermore, the connection method of boundary edges and the vertex interpolation involving boundary edges can be handled separately from internal vertices and edges.

[0108] Therefore, the aforementioned technical problems can be effectively improved by one or more of these technical solutions through the exemplary embodiments described herein. For example, since dynamic grid sequences may require a large amount of data due to containing a large amount of information that changes over time, the exemplary embodiments described herein at least provide efficient compression techniques for storing and transmitting such content.

[0109] The above embodiments can be further applied to instance-based mesh coding, where an instance can be a mesh of objects or a portion of an object. For example, Figure 15 Illustrative example 1500 shows a mesh example 1501, in which there are various instances 1502 (a mesh representing a cup), 1503 (a mesh representing a spoon), and 1504 (a mesh representing a plate), which can be separated and encoded separately. Each of instances 1501, 1502, 1503, and 1504 is shown as a bounding box; however, it should be noted that instance 1501 can be considered as being defined by a "mesh-based bounding box," while each of instances 1502, 1503, and 1504 can be considered as being defined by its respective "instance-based bounding box."

[0110] In addition, combined Figure 16As illustrated in Example 1600 and other figures, according to various embodiments, "dual-degree" mesh encoding is a specialized technique designed to efficiently encode the connectivity of polygonal meshes. By applying the principle of duality in mathematics, this method encodes connectivity data by constructing two independent sequences: one sequence characterizing the degrees of vertices, and the other sequence describing the degrees of faces (see Example 1601, which is an exemplary schematic of dual-degree traversal). That is, the encoding process involves traversing both faces and vertices simultaneously. Specifically, the traversal begins with an arbitrary seedface, recording the degree of that face (e.g., F3, which represents the valence of a vertex and the degree of the face). Subsequently, as shown in Example 1601, the degrees of adjacent vertices are recorded (e.g., V5, V5, V5). The algorithm then selects a so-called "pivot vertex," which is identified by having the minimum number of degrees of freedom (i.e., the number of its untraversed neighboring faces). The traversal continues around this pivot vertex, adding new faces and vertices and recording their respective degrees. According to the embodiments, supplementary notation is used to handle special cases involving vertex splitting or merging. While this design highlights the superior efficiency of two-degree mesh encoding in handling polygonal meshes, including those that are highly irregular or in the worst-case scenario, the performance of two-degree encoding depends heavily on the regularity of face degree and vertex valence.

[0111] In terms of location attribute encoding, in addition to connectivity, meshes typically include other attributes such as vertex positions, texture coordinates, normal vectors, and associated texture maps. Therefore, connectivity usually constitutes only a small portion of the mesh data, possibly less than 10% or even 1% of the total bitstream, while location attributes may account for more than half of the bitstream.

[0112] For information on using parallelograms for position prediction, see Example 1602. Among the many attributes, the 3D position of a vertex typically accounts for the majority of the bits required for a geometric attribute. According to the embodiment, predictive coding schemes can be employed for efficient encoding of vertex positions, a prominent example being parallelogram prediction. Parallelogram prediction performs best in quadrilateral meshes within the context of polygonal meshes.

[0113] In other words, Example 1602 illustrates a schematic diagram of "cross-face" and "in-face" parallelogram prediction for vertex encoding of a polygonal mesh, where A, B, and C are three reference locations for both types of parallelogram prediction. According to the embodiment, for a given polygonal mesh, the position (V) of the vertex to be predicted is estimated using three previously encoded vertices (A, B, C) as references, as shown in Example 1602, as follows: - Formula (2), The weighting factors are usually chosen as w1=w3=1, w2= 1. These weights can be further modified to suit different polygon structures. In the context of a polygonal mesh, parallelogram prediction can be categorized into two types: "withinprediction" and "across prediction," as shown in Example 1602. In "withinprediction," all three reference vertices lie within the same face; while in "across prediction," vertex C is taken from the opposite face.

[0114] According to the embodiment, see also Example 1603 regarding multi-parallelogram position prediction. That is, as an extension of parallelogram prediction, multi-parallelogram prediction is introduced to enhance the position prediction effect of triangular meshes. As shown in Example 1603, multi-parallelogram prediction "a" employs the average position derived from two or more parallelogram predictions when feasible. Prediction "b" illustrates an example of bi-parallelogram prediction. The number of available prediction candidates determines how many predictions are used for averaging.

[0115] The proposed methods can be used individually or in combination in any order. Furthermore, these methods can be implemented using processing circuitry (e.g., one or more processors or one or more integrated circuits). While some methods are explicitly applied to triangular meshes, they can also be used with arbitrary polygonal meshes. In this disclosure, a method is proposed that adaptively merges triangular faces into quadrilateral faces according to the traversal order to create in-plane predictions for parallelogram predictions.

[0116] For example, Example 1604 illustrates adaptive parallelogram prediction in the context of a triangular mesh. As shown in Example 1604, following the traversal order starting with pivot vertex A, the next face to be traversed is F0, and the vertex to be encoded is D.

[0117] According to this embodiment and in conjunction with Example 1604, parallelogram prediction uses reference vertices A, B, and C0 to predict D. Although vertices A and B are located on the same face (F0) as D, vertex C0 is selected from the opposite face (F1) that spans edge AB, as shown by C1 in Example 1604. However, in many cases, this prediction may be inaccurate, especially when the face of the quadrilateral formed by vertices A, C, B, and D is concave, or when the position of vertex D is irregular.

[0118] To address this issue, the embodiments described herein provide an alternative reference vertex C1, where C1 is a vertex connected to A across edge A-C1 on the opposite face of F1 (i.e., face F2 in Example 1604). In this case, the encoder can select either C0 or C1 as the reference for predicting D, according to at least one of the following embodiments.

[0119] In one embodiment, for example 1604, the selection criterion for reference C is based on the predicted location P. i =A+BC i The sum of absolute differences (SAD) of the residual vectors between the positions to be encoded and the positions to be encoded is calculated as R. i =C i -P i , where i=0,1.

[0120] In another embodiment, for example 1604, the choice between C0 and C1 is based on the estimated number of bits required to encode the residual, i.e., the bit cost (bitCost(R)). i This cost calculation also includes the number of bits used for the signaling notification adaptive flag.

[0121] In another embodiment, for example 1604, an additional reference vertex C2 is selected from the face opposite F1 across edge B-C0.

[0122] In yet another embodiment, for example 1604, the number of reference vertices used to predict D is expanded to include all neighboring vertices of A and B, which increases the signaling bits.

[0123] In yet another embodiment, for example 1604, the number of reference vertices used to predict D is expanded to include the first N available neighboring vertices of A and B, where N can be specified in the header syntax (such as a frame-level or sequence-level header) or assumed jointly by the encoder and decoder. Assuming the total number of available neighboring vertices is greater than N, the first N are used as candidates in a given order to reduce signaling costs.

[0124] This disclosure further provides signaling notification schemes for adaptive reference selection, including using flag signaling to notify adaptive reference and hiding the adaptive reference flag by modifying the facet.

[0125] According to an embodiment, for signaling adaptive references using flags, a direct approach to indicate which reference vertex C (C0 or C1) is being used is to signal a binary flag for each face. Specifically, each face has one bit to indicate whether vertex C0 (flag value 0) or vertex C1 (flag value 1) is being used. According to an embodiment, these flags are encoded using binary arithmetic coding. Furthermore, to at least further optimize the entropy coding of the adaptive reference flags, the probability of the current adaptive reference flag is estimated using the context of the flags of the faces adjacent to F1.

[0126] To address the issue of hiding the adaptive reference flag by modifying the facet, given that C0 is typically preferred in vertex prediction, signaling the adaptive reference flag incurs significant bit overhead. To solve this problem, the embodiments in this paper employ a simple facet merging algorithm to hide the adaptive reference flag. Specifically, taking a triangular mesh as an example, the facet is always 3. At the encoder end, traversal follows two-degree connectivity encoding, and the facet is modified to 3+i only when i>0, where i represents the corresponding prediction mode. At the decoder end, vertex prediction selects the appropriate reference based on the current facet.

[0127] Furthermore, regarding the hiding of adaptive reference flags by modifying facet degrees, the embodiments described herein can limit each facet to only one reference flag to ensure decodeability. In this way, such embodiments avoid the most common case of signaling notification (i.e., i = 0) and hide other adaptive reference flags within the facet symbols encoded with two-degree connectivity. This increases the number of bits for connectivity encoding but saves bits for position encoding. Moreover, according to embodiments, when more than two reference modes need to be processed, the facet degree can be increased accordingly. Less desirable reference selections will be added at a higher cost. For example, if the list of reference candidates includes C0, C1, and C2, the degree will increase by +0, +1, and +2, respectively.

[0128] Observing Example 1602, among the many attributes, the three-dimensional position of the vertex typically accounts for the majority of the bits required for geometric attributes. To efficiently encode vertex positions, a predictive coding scheme is employed; a prominent example is parallelogram prediction. In the context of polygonal meshes, parallelogram prediction performs best in quadrilateral meshes. Example 1602 illustrates cross-face parallelogram prediction and in-face parallelogram prediction for vertex encoding in polygonal meshes. According to the embodiment, A, B, and C are three reference positions for the parallelogram prediction.

[0129] For example, for a given polygonal mesh, the position (V) of a vertex to be predicted is estimated using three previously encoded vertices (A, B, C) as references, as follows: P= w 1 A + w 2 B + w 3 C, - Formula 3, The weighting factors are typically chosen as w1 = w3 = 1, w2 = -1. These weights can be further modified to suit different polygon structures. In the context of a polygonal mesh, parallelogram prediction can be divided into two types: in-plane prediction and cross-plane prediction, as shown in Example 1602. In in-plane prediction, all three reference vertices lie within the same plane; while in cross-plane prediction, vertex C is taken from the opposite plane.

[0130] However, as mentioned above, there are some problems. For example, parallelogram prediction is only applicable to relatively flat surfaces and equidistant surface structures, while cross-surface prediction is extremely challenging even for height-number polygon meshes. Therefore, there is still a need for effective solutions to handle irregular surfaces such as slender triangles.

[0131] Therefore, the embodiments described herein provide a two-step cross-face prediction. For example, for the position encoding of triangular meshes, there exists a two-step cross-face prediction, such as... Figure 17 Example 1700 shows: (a) a midpoint prediction for midpoint D1 in one case, and (b) a forward prediction in another case.

[0132] like Figure 17 As shown, the new vertex position D will be predicted using encoded positions A, B, and C as adjacent triangles. Unlike predicting D based on a traditional parallelogram utilizing triangle ABC, the embodiment in this paper performs a two-step encoding method: first, the midpoint of edge DA is encoded, and then D is predicted by forward prediction using that midpoint.

[0133] First, according to the embodiment, a conventional parallelogram is performed to estimate the predicted value, as follows: P = w1A + w2B + w3C. – Formula (4).

[0134] First, according to the embodiment, a conventional parallelogram is performed to estimate the predicted value, as follows: D1 = ½ (A + D) – Formula (5).

[0135] That is, decide whether to encode using single-step or two-step prediction. Define and signal control flags as follows: f = |D 1 – P|<|D – P|. – Formula (6).

[0136] If f is false, then derive and encode the residual vector, and complete the encoding of the current position: Δ = D – P - Formula (7).

[0137] Otherwise, if f is true, then derive and encode the first residual vector: Δ 1 = D 1 – P - Formula (7).

[0138] The second step of encoding is a forward prediction based on the fact that D1 is the midpoint of edge DA. Therefore, the predicted value of D (denoted as P1) will satisfy: D1 = ½ (P1 + A) => P1 = 2D1 – A. – Formula (8).

[0139] Then, the second residual vector is derived and encoded: Δ 2 = D – P1 - Formula (9).

[0140] In one embodiment, the two-step prediction method is improved to handle quantized non-negative integer location grids. Essentially, the predicted values ​​are rounded before calculating the residual vector. For example, the first residual vector would be calculated as: Δ 1 = round(D 1 ) – round(P) - Formula (10).

[0141] The second residual vector is similarly calculated: Δ 2 = D-round(P) 1 ) - Formula (11).

[0142] In this two-step prediction, the two residuals Δ1 and Δ2 will be encoded and transmitted in the bitstream.

[0143] According to the embodiment, adaptive prediction is also provided. For example, there are multiple choices for the midpoint, such as the midpoint of AD, the midpoint of BD, or the centroid of ABD. The optimal midpoint may differ depending on the irregular structure of the triangular face. The default prediction mode is mode 0 (default parallelogram), followed by two-step prediction modes 1, 2, and 3, corresponding to the midpoint of AD, the midpoint of BD, and the centroid of the ABD face, as shown below: - Mode 0: D 0 = D ; - Mode 1: D 1 = ½ (A + D) ; - Mode 2: D 2 = ½ (B + D) ; - Mode 3: D 3 = 1 / 3 (A + B + D).

[0144] As expected, mode 0 requires single-step encoding, while the other modes are two-step encoding. Furthermore, modes 1 and 2 use the two-step encoding described above, but mode 3 requires special handling. That is, in the second step, forward prediction is performed based on the centroid D3 and the midpoint of edge AB.

[0145] In this case, the best prediction pattern is written into the bitstream.

[0146] In other examples, combinations of the patterns listed above can be made, without being limited to the entire set of candidates.

[0147] Examples of these patterns are in Figure 18 Example 1800 shows multiple midpoint candidates.

[0148] Lossless mesh compression is also provided. For example, to compress a 3D mesh, mesh compression requires encoding properties such as connectivity, vertex positions, texture coordinates, normals, and related texture maps. Traversal algorithms are the core of mesh compression and are used to encode connectivity. Among them, edge-breakers and double-degree traversal are the most popular algorithms. To compress other properties, predictive coding schemes are used, such as using parallelogram prediction in position and texture coordinate encoding. However, all lossless coding techniques focus on prediction and traversal based on local neighborhood information.

[0149] Furthermore, regarding symmetry in mesh coding, symmetry is an attribute of geometric objects, meaning that an object can be mapped to itself after a certain operation. In Euclidean metric, a set of symmetry transformations is called Euclidean isometric transformations, including reflection, translation, rotation, and their combinations. Among these, reflection symmetry (also known as bilateral symmetry) is the most common form of symmetry in both biological and abiotic worlds. All points and edges in a reflection-symmetric mesh form a one-to-one correspondence through the plane of symmetry. Moreover, many man-made objects are designed according to reflection symmetry, such as... Figure 19 Example 1900 is shown in the example.

[0150] The reflection symmetry plane of the mesh can be detected by simple methods such as principal component analysis (PCA) or by more advanced techniques such as deep learning. Reflection symmetry can be used to create folded meshes [3]. The folded mesh M is divided into non-overlapping regions. Then, a folded tree structure is used to represent symmetric planes, as well as mesh data with both symmetry and asymmetry. While folded meshes have the potential to save bits for symmetric meshes, they are only one representation method, have low encoding efficiency, and cannot handle vertex misalignment issues, thus making them unsuitable for lossless compression.

[0151] Therefore, even with the above techniques, the following problems still exist: attribute encoding and traversal encoding are still limited to the local features of the 3D mesh, making it difficult to fully utilize the global features of the 3D mesh (such as symmetry).

[0152] Therefore, the embodiments also consider the case of fully symmetric meshes. According to the embodiments, a fully symmetric mesh generally refers to a mesh in which all properties have a correspondence through a given plane of symmetry. Given an input fully symmetric mesh... M Its perfectly symmetrical plane divides the mesh into two mirror parts, referred to in this disclosure as the left half M. L and the right half of M R Each vertex v in the left half L Each of the right halves has a unique corresponding vertex v. R The two are mirror images of each other through the symmetry plane p. Furthermore, the connection structure between the left and right meshes is also mirror-symmetric. In this ideal case, we only need to encode one half of the mesh, using it as a mirror image of the other half for prediction, and compensate for the other half in case of quantization errors, prediction errors, or other potential errors.

[0153] It should be noted that the positions require a one-to-one mapping relationship, but not perfect symmetry. That is, from v... L Predict v using mirror image R The resulting residuals can be non-zero, as shown below: - Formula 13, in, .

[0154] According to embodiments, a symmetric encoding method is also provided. For example, this encoding method may include grid segmentation. That is, even if the input grid is perfectly symmetrical, the embodiments will divide it into a left half M. L and the right half of M R Based on the position of the vertex relative to the symmetry plane p, the vertices are classified into a left vertex set V. L Right vertex set V R and the set of vertices V in the plane P .

[0155] Note that in order to separate the left and right parts, it may be necessary to add a V-shaped element. P The new virtual vertices and corresponding virtual edges. For example, see [link to example]. Figure 20 Example 2000 in the document adds a virtual vertex X and the corresponding connection relationships. That is, Example 2001 illustrates a symmetric mesh partitioning according to embodiments of this document by adding virtual vertices and virtual edges (dashed lines).

[0156] It is important to note that in order to achieve lossless reconstruction of symmetric meshes, the decoder needs to know the indices of the vertices in the plane. Although this can be achieved by comparing the distance between the decoded position and the symmetric plane, quantization errors may cause uncertainty for some vertices. Therefore, the embodiments in this paper identify vertices that are misclassified due to quantization errors and notify the decoder of these misclassified vertices.

[0157] Building upon the aforementioned mesh segmentation, a dual traversal feature is also provided. This means that a dual traversal is performed after mesh segmentation. In other words, the next step is to utilize symmetry properties to simultaneously traverse the left and right meshes. For example, in... Figure 20 In Example 2002, the traversal of the right grid is a mirror image of the traversal of the left grid, which illustrates a double traversal of a symmetrical grid.

[0158] Since the connectivity between the left and right grids is the same, only the connectivity on the left side is encoded, which can reduce the connectivity coding rate by half.

[0159] In one embodiment, a vertex in the plane is selected as the starting vertex. This reduces the need to find corresponding vertex pairs between the left and right grids.

[0160] As for position encoding, any position encoding tool can be used to encode the left grid. For the right grid, in the case of a perfectly symmetrical grid, the right position can be accurately predicted from the left position through mirror prediction. No additional signaling is required, thus significantly reducing the position bit rate.

[0161] For near-symmetric positions, i.e., when only a one-to-one mapping relationship exists between the left and right vertices, there is a residual between the symmetric prediction value and the right-side position, as shown below: - Formula 14.

[0162] Therefore, the embodiment encodes this displacement of the right-side grid.

[0163] Signaling notification methods for virtual vertices and virtual edges are also provided. The indices of virtual vertices are in encoded order. All virtual vertex indices are signaled to the decoder.

[0164] Virtual edges are always associated with virtual vertices. Therefore, the implementation only needs (but is not limited to) signaling notification of the number of virtual edges associated with the current virtual vertex, and the corresponding local indexes in clockwise order.

[0165] In one embodiment, to reduce signaling for the virtual vertex index, the embodiment signals the pivot vertex index and its offset in the encoding order. The pivot vertex index is significantly smaller than the encoding order index.

[0166] In one embodiment, the pivot order index is reselected for each subgrid to further reduce signaling costs.

[0167] Symmetric decoding is also provided, as shown below: - Decode the indices of vertices that are misclassified in the plane and name them as a set. ; - Secondly, for virtual vertex sets and virtual edge sets Decode the index; - Perform double traversal and position decoding on the left and right grid sections: The decoded left-hand joins are mirrored and used for the right-hand joins; For each decoded left vertex v L If it is not a planar vertex (by checking the distance to the symmetric plane p and the correction list) If ), then perform symmetric prediction: - Formula 15; Decoding symmetrical displacement To reconstruct the right vertex: - Formula 16; - Remove virtual connection relationships; - Remove dummy vertices and reconnect edges. If two edges connected to a dummy vertex belong to the same line, merge them into one edge. For example, after removing X, edges BX and XC become edge BC.

[0168] The proposed method can be used alone or in any combination in any order, and can be used for arbitrary polygonal meshes and arbitrary traversal algorithms.

[0169] The proposed method is applicable to arbitrary location attribute compression algorithms. Although this example is shown for triangular meshes, in some cases, the method is applicable to arbitrary polygonal meshes.

[0170] The embodiments described herein provide further contributions. For example, for geometry-guided face attribute predictions, the embodiments provided a method for ranking the predictions using distances to the face centroids. For instance, in a three-adjacency configuration, three of the four candidate predictions are set to the face attributes of the three adjacent faces themselves. These three predictions are ranked based on the distances from the centroid of the current face to the centroids of each of the adjacent faces. The face attribute of the closest adjacent face is set as the first of the three predictions, and so on.

[0171] When using a two-adjacent configuration, the embodiments in this paper employ the same logic to sort the predicted values. See, for example, [link to example]. Figure 21Therefore, the first predicted value is set to the face attribute of the neighboring face with a shorter distance between its centroid and the current centroid. This change is logical and consistent with the ordering of predicted values ​​in a three-adjacent configuration. According to one or more embodiments, such as those relating to the above-described features, a method is provided for ordering face attribute predicted values ​​based on the distance between the centroids of a face. For example, when predicting the current attribute value, the embodiment uses the face attributes of its N neighboring faces as N predicted values. These N predicted values ​​are arranged in ascending order of the distance between the centroids of the faces in the predicted value list.

[0172] As an example, let the surface attribute to be predicted be... It has N adjacent faces, whose encoded face attributes are as follows: The centroids of the current face and its N neighboring faces are all usable. Let the centroids (3D vectors) of these faces be respectively... (Regarding the current situation) and (For N adjacent faces).

[0173] First, calculate the distance between the centroid of the front face and the centroids of each adjacent face: -Formula 17; in, It is a distance metric in 3D space, such as, but not limited to, the square of the L2-norm, L1-norm, and L2-norm.

[0174] Then, sort in ascending order The ordering is performed so that neighboring faces whose centroids are closer to the current face in the metric space are ranked higher in the ordered list. The ordered list is then used to sort the predictions, as shown below.

[0175] Let `orderedFacesCentroid` be an ordered list of size N, based on the distances between the centroids of the current face and the centroids of all adjacent faces. Then the ordered list of predicted face attributes is:

[0176] It also provides a method for sorting the predicted face attribute values ​​based on the dihedral angles between faces. For example, when predicting the current attribute value, the face attributes of its N neighboring faces are used as N predicted values. These N predicted values ​​are sorted in the predicted value list according to the order of the dihedral angles between the current face and its neighboring faces.

[0177] Let the surface attribute to be predicted be It has N adjacent faces, whose encoded face attributes are as follows: The dihedral angle between the current face and each of its N adjacent faces can be calculated from the encoded face vertices. Let the dihedral angles between the current face and its N adjacent faces be... .

[0178] The sorting method is as follows: the closer the dihedral angle of the preceding face is to 180°, the higher its rank in the predicted value list. This is particularly useful when predicting the normal of the current face based on the normals of adjacent faces, but it can also be applied to other face properties. Therefore, the calculation... The absolute value of.

[0179] Then, The dihedrals are sorted in ascending order so that adjacent faces with dihedral angles close to 180° rank higher in the ordered list. Note that if the dihedral angle is expressed in other units (such as radians), or if a quantized version of the dihedral angle is used, the equivalent value of 180° in that unit is subtracted. The ordered list is then used to sort the predicted values, as shown below.

[0180] Let `orderedFacesDihedral` be an ordered list of dihedral angles between the current face and its adjacent faces, with a size of N. Then the ordered list of predicted face attribute values ​​for N faces is:

[0181] The above-described technology can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media; or it can be implemented by one or more specially configured hardware processors. For example, Figure 21 A computer system 2100 suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0182] Furthermore, through the embodiments of this paper, a method is also provided for sorting the attribute values ​​of N adjacent faces (as predicted values ​​for the current face attribute) using the face centroid and dihedral angle in combination. Using the definition above... and Here, a joint metric is also defined as... and The linear combinations of are shown below: – Formula 18, in, It is a joint metric, where a and b are scalar constants.

[0183] Then, The predictions are sorted in ascending order so that neighboring faces that are closer to the current face in the joint metric space rank higher in the ordered list. This ordered list is then used to sort the predictions, as shown below.

[0184] set up Given an ordered list of N face attribute predictions based on the distances between the centroids of the current face and the centroids of the adjacent faces, the ordered list of N face attribute predictions is as follows:

[0185] This embodiment also provides a generalized linear face attribute prediction value. When using the available face attributes of N neighboring faces to predict face attribute values, the generalized linear prediction value is a linear combination of the face attributes of the N neighboring faces. Let the face attribute to be predicted be... It has N adjacent faces, whose encoded face attributes are as follows: The linear prediction is then given by the following equation: -Formula 19; in, It is a constant. These are constant weights derived from the geometric structure, as described below.

[0186] The distance between the centroid of the preceding face and the centroids of the N adjacent faces is used to determine the prediction. . The prediction weight is inversely proportional to the distance between the centroids of the faces; that is, faces with closer centroids have higher prediction weights. Therefore: – Formula 20; in, As described in Example 1, the distance between the centroid of the front face and the centroid of the adjacent face, M is a constant that may or may not be equal to C.

[0187] This embodiment also provides further features for deriving the prediction weights in Embodiment 4. As described in Embodiment 2, the dihedral angle between the current face and the adjacent face... Used to determine . and Inversely proportional, meaning that faces with dihedral angles close to 180° have higher prediction weights. Therefore: – Formula 21.

[0188] According to the embodiments herein, further features for deriving the aforementioned prediction weights are provided. As described in Embodiment 3... Used to determine . and Inversely proportional. Therefore, – Formula 22.

[0189] Therefore, this paper provides a feature for sorting face attribute predictions based on an increasing order of distance to the current face centroid, and illustrates and describes the advantages of such a feature with comparative examples. In other words, aspects of the embodiments described herein provide ways to improve predictions for face attributes such as face group IDs, face normals, etc., by utilizing mesh geometry information. First, this paper discloses a method for sorting face attribute predictions using face centroids and / or dihedral angles calculated based on mesh geometry. Second, this paper discloses a generalized linear face attribute prediction method.

[0190] The disclosed embodiments can be applied to arbitrary face attribute prediction and / or other attribute encoding (position, UV, normals, etc.) of arbitrary polygonal meshes, regardless of the traversal algorithm used. These embodiments can be used individually or in combination in any form.

[0191] Furthermore, mesh encoding involves connectivity encoding and attribute value encoding, where attributes can be location, texture coordinates, normals, etc. Attribute values ​​are predicted based on known attributes in the local neighborhood, and the residual between the true and predicted values ​​is encoded. Typically, a set of candidate predicted values ​​is available, and the index of the best predicted value is additionally encoded.

[0192] However, in the current VVM standard, the encoding predictor index may be redundant in some cases, while in others, using a predictor set can avoid this. Furthermore, in the current VVM standard, the predictor set is mostly fixed, consisting of four predictors; it may be feasible to reduce the set size by removing some predictors to optimize encoding performance.

[0193] Therefore, the embodiments described herein have been improved. According to one or more embodiments, a method is provided to combine N candidate predicted values ​​into a single predicted value. For example, among the N candidate predicted values, if any M predicted values ​​have the same value, then that value is used as the unique predicted value, and the other predicted values ​​are discarded.

[0194] According to one or more embodiments, another method is provided for combining N candidate predicted values ​​into a single predicted value. Among the N candidate predicted values, if the index is... If M predicted values ​​have the same value, then that value is used as the unique predicted value, and the other predicted values ​​are discarded.

[0195] According to one or more embodiments, another method is provided for combining N candidate predicted values ​​into a single predicted value. Among the N candidate predicted values, if any M predicted values ​​differ at most by d in each component, then the first predicted value among these M predicted values ​​(the one with the highest ranking in the initial set of predicted values) is used as the unique predicted value, and the other predicted values ​​are discarded.

[0196] According to one or more embodiments, another method is provided for combining N candidate predicted values ​​into a single predicted value. Among the N candidate predicted values, if the index is... If the difference between the M predicted values ​​on each component is at most d, then the first predicted value among these M predicted values ​​(the one with the higher ranking in the initial set of predicted values) is used as the unique predicted value, and the other predicted values ​​are discarded.

[0197] According to one or more embodiments, another method is provided for combining N candidate predicted values ​​into a single predicted value. Among the N candidate predicted values, if the difference between any M predicted values ​​at each component is at most d, a new predicted value is constructed, which is the component median of the M predicted values. This predicted value is then used as the unique predicted value, and the other predicted values ​​are discarded.

[0198] According to one or more embodiments, another method is provided for combining N candidate predicted values ​​into a single predicted value. Among the N candidate predicted values, if the index is... If the difference between the M predicted values ​​on each component is at most d, then a new predicted value is constructed, which is the component wise median of the M predicted values. This predicted value is then used as the unique predicted value, and the other predicted values ​​are discarded.

[0199] According to one or more embodiments, another method is provided for combining N candidate predicted values ​​into a single predicted value. Among the N candidate predicted values, if the difference between any M predicted values ​​on each component is at most d, a new predicted value is constructed, which is the component wise mean of the M predicted values. This predicted value is then used as the unique predicted value, and the other predicted values ​​are discarded.

[0200] According to one or more embodiments, another method is provided for combining N candidate predicted values ​​into a single predicted value. Among the N candidate predicted values, if the index is... If the difference between the M predicted values ​​on each component is at most d, then a new predicted value is constructed, which is the component average of the M predicted values. This predicted value is then used as the unique predicted value, and the other predicted values ​​are discarded.

[0201] According to one or more embodiments, a method is provided to reduce an initial set of predicted values ​​from N candidate predicted values ​​to P predicted values. Among the N candidate predicted values, if M1 predicted values ​​have the same value, the duplicate predicted values ​​are eliminated, and only the first-occurring predicted value is retained. The remaining predicted values ​​are moved up in the predicted value list. If other sets... If the values ​​are the same, repeat the process.

[0202] According to one or more embodiments, a method is provided to reduce an initial set of predicted values ​​from N candidate predicted values ​​to P predicted values. Among the N candidate predicted values, if the index is... of If multiple predicted values ​​have the same value, the duplicate predicted values ​​are eliminated, and only the first predicted value is retained. The remaining predicted values ​​are moved up in the predicted value list. If other sets... If the values ​​are the same, repeat the process.

[0203] According to one or more embodiments, a method is provided to reduce an initial set of predicted values ​​from N candidate predicted values ​​to P predicted values. Among the N candidate predicted values, if the index is... of If the difference between each predicted value and each component is at most d, then only the first predicted value is retained, and the other predicted values ​​are eliminated. The remaining predicted values ​​are moved up in the predicted value list. If other sets... If the values ​​are the same, repeat the process.

[0204] According to one or more embodiments, a method is provided to reduce an initial set of predicted values ​​from N candidate predicted values ​​to P predicted values. Among the N candidate predicted values, if the index is... of If the difference between each predicted value and each component is at most d, then a new predicted value is constructed. The average of the components of each predicted value. Replace the original predicted value with this new predicted value. The first predicted value is selected, and other predicted values ​​are eliminated. The remaining predicted values ​​are moved up in the predicted value list. If other sets... If the values ​​are the same, repeat the process.

[0205] According to one or more embodiments, a method is provided to reduce an initial set of predicted values ​​from N candidate predicted values ​​to P predicted values. Among the N candidate predicted values, if the index is... of If the difference between each predicted value and each component is at most d, then a new predicted value is constructed. The median of each component of the predicted value. Replace the predicted value with this new predicted value. The first predicted value is selected, and other predicted values ​​are eliminated. The remaining predicted values ​​are moved up in the predicted value list. If other sets... If the values ​​are the same, repeat the process.

[0206] According to one or more embodiments, in the above embodiments, it is provided to use different component offsets (instead of a fixed d). Therefore, for an attribute with dimension D, the maximum possible component offset difference between each component value is... .

[0207] According to one or more embodiments, in the above embodiments, a method is provided to set the component offset d as a function of the attribute bit depth. Generally, the higher the bit depth, the higher the offset.

[0208] According to one or more embodiments, in the above embodiments, a method is provided to use the distance between different predicted values ​​(Manhattan distance, Euclidean distance, etc.) instead of fixed or different component offsets.

[0209] In other words, due to the high similarity of face group IDs in the face neighborhood, the predicted attribute values ​​of four candidate faces are very likely to be the same. Therefore, this embodiment provides an improved method that reduces the set of attribute predicted values ​​to a single predicted value in the face group ID encoding if the first two predicted values ​​in the predicted value set are the same. Accordingly, the encoding of the predicted value index is skipped; and according to this embodiment, this change can bring an average encoding gain of up to 15% in face group ID encoding, while the change in the complexity of the encoder and decoder is negligible.

[0210] For example, the predicted face group IDs for three-adjacent and two-adjacent configurations include the face group IDs of each adjacent face group, the average face group ID of paired adjacent face groups, and / or the median of adjacent face group IDs. Face group IDs are highly similar in their local face neighborhoods, so most predicted values ​​tend to be the same. In this case, encoding the predicted value index is redundant.

[0211] Therefore, this embodiment provides a method for checking the features of the first two predicted values ​​in the predicted value set, and if they are the same, reducing the predicted value list to a single predicted value. Then, based on the existing logic in the test model used for the comparison example below, the predicted value index is not encoded. Furthermore, this check is performed only when using three-adjacency and two-adjacency configurations. The slight increase in complexity introduced by checking two predicted values ​​and reducing the predicted value set is compensated by avoiding the encoding of the predicted value index and the bitrate estimation (at the encoder).

[0212] The embodiments described in this paper demonstrate a significant coding gain of up to 15% in face group ID encoding, with an overall coding gain of 0.02%. No coding loss was observed. The results for both general and sequence-by-sequence configurations are summarized below: (Results for lossless general test condition CTC configuration) Figure 22 , Figure 23 and Figure 24 Examples 2200, 2300, and 2400; (results for lossless sequence-by-sequence CTC configuration) Figure 25 , Figure 26 and Figure 27 Examples 2500, 2600, and 2700; (for results with a lossy CTC configuration) Figure 28Example 2800 in the example.

[0213] Therefore, as described above and further elaborated, this embodiment provides a method for calculating a set of predicted values ​​for linear surface attribute predictions. This process generates a set of one to four predicted values ​​for predicting the values ​​of the surface attributes. This set depends on the number of available adjacencies (three, two, one, or none).

[0214] According to this embodiment, for the case of three adjacent nodes, the initial set includes the following four predicted values: 1. Single-way predictor; 2.ap0[0]; 3.ap0[1]; 4.ap0[2]; If the first two predicted values ​​in the set are equal, the set is reduced to contain only the first predicted value.

[0215] According to this embodiment, for the case of two adjacent pairs, the initial set includes the following two predicted values: 1.ap0[0]; 2.ap0[1]; If the first two predicted values ​​in the set are equal, the set is reduced to include only the first predicted value.

[0216] According to this embodiment, for an adjacent case, the initial set includes the following predicted values: 1.ap0[0].

[0217] According to the embodiments described herein, for the case of no adjacency, the initial set includes the following predicted values: 1. Zero predicted value; 2. A maximum of three available cache predictions; If the first two predicted values ​​in the set are equal, the set is reduced to include only the first predicted value.

[0218] According to this embodiment, a method is also provided for calculating a set of predicted values ​​for the predicted properties of a geometrically guided surface. This process generates a set of one to four predicted values ​​for predicting the values ​​of the surface properties. The set depends on the number of available adjacencies (three, two, one, or none).

[0219] According to this embodiment, for the case of three adjacent nodes, the order of predicted values ​​is determined based on the decoding geometry as follows: 1. Calculate the centroid (D0) of the face whose properties are to be decoded (for example, see...). Figure 21 Example 2100 (S2101) 2. Calculate the centroids (DA, DB, DC) of three adjacent faces (see S2102); 3. Calculate the squared Euclidean distance between the centroid of the front face and the centroid of the adjacent face (see S2103): dA = norm2(D0 – DA); dB = norm2(D0 – DB); dC = norm2(D0 – DC); 4. The order of the predicted values ​​is determined as follows (see S2104):

[0220] The initial set includes the following four predicted values: 1. One-way predicted value; 2.ap0[order[0]]; 3.ap0[order[1]]; 4.ap0[order[2]]; If the first two predicted values ​​in the set are equal, the set is reduced to include only the first predicted value.

[0221] According to the embodiments in this paper, for the case of two adjacent values, the order of predicted values ​​is determined based on the decoding geometry as follows: 1. Calculate the centroid (D0) of the face whose properties are to be decoded (e.g., see S2101); 2. Calculate the centroids (DA, DB) of two adjacent faces (see S2102 for example); 3. Calculate the squared Euclidean distance between the centroid of the front face and the centroid of the adjacent face (see S2103 for example): dA = norm2(D0 – DA); dB = norm2(D0 – DB); 4. Determine the order of the predicted values ​​as follows (for example, see S2104):

[0222] The initial set includes the following two predicted values: 2.ap0[order[0]]; 3.ap0[order[1]]; If the first two predicted values ​​in the set are equal, the set is reduced to include only the first predicted value.

[0223] According to the embodiments described herein, for an adjacency case, the initial set includes the following predicted values: 1.ap0[0].

[0224] According to the embodiments in this paper, for the case of no adjacency, the initial set includes the following predicted values: 1. Zero predicted value; 2. A maximum of three cached predictions are available; If the first two predicted values ​​in the set are equal, the set is reduced to include only the first predicted value.

[0225] The above-described technology can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media; or implemented by one or more specially configured hardware processors. For example, Figure 29 A computer system 2900 suitable for implementing certain embodiments of the disclosed subject matter is shown.

[0226] The computer software can be encoded using any suitable machine code or computer language, which can be assembled, compiled, linked, or similarly processed to create code containing instructions that can be executed directly by the computer's central processing unit (CPU), graphics processing unit (GPU), or through interpretation, microcode execution, or other means.

[0227] These instructions can be executed on various types of computers or their components, including, for example, personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0228] Figure 29 The components of the computer system 2900 shown are exemplary in nature and are not intended to impose any limitation on the scope of use or functionality of computer software implementing embodiments of this disclosure. Furthermore, the configuration of the components should not be construed as having any dependencies or requirements relating to any component or combination of components shown in the exemplary aspects of the computer system 2900.

[0229] Computer system 2900 may include certain human-machine interface input devices. Such human-machine interface input devices can respond to input from one or more human users through, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., speech, clapping), visual input (e.g., gestures), and olfactory input (not depicted). Human-machine interface devices can also be used to capture certain media that are not necessarily directly related to human conscious input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, images captured from a still image camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0230] Human-machine interface input devices may include one or more of the following (only one of each is shown): keyboard 2901, mouse 2902, touchpad 2903, touch screen 2910, joystick 2905, microphone 2906, scanner 2908, and camera 2907.

[0231] Computer system 2900 may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback of touchscreen 2910, or joystick 2905, but may also be tactile feedback devices that are not input devices), audio output devices (e.g., speakers 2909, headphones (not depicted)), visual output devices (e.g., screen 2910 including CRT screens, LCD screens, plasma screens, OLED screens, each screen may or may not have touchscreen input functionality, each screen may or may not have tactile feedback functionality, some of which are capable of outputting two-dimensional visual output or more than three-dimensional output through devices such as stereoscopic image output, virtual reality glasses (not depicted), holographic displays, and smoke boxes (not depicted), and printers (not depicted).

[0232] The computer system 2900 may also include user-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW 2920 with media such as CD / DVD 2911, finger drives 2922, removable hard disk drives or solid-state drives 2923, conventional magnetic media such as magnetic tapes and floppy disks (not depicted), devices based on dedicated ROM / ASIC / PLD such as security dongles (not depicted), etc.

[0233] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter currently disclosed does not include transmission media, carrier waves, or other transient signals.

[0234] Computer system 2900 may also include an interface 2999 providing access to one or more communication networks 2998. Network 2998 may be, for example, a wireless network, a wired network, or an optical network. The network may further be a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), an vehicular and industrial network, a real-time network, a latency-tolerant network, etc. Examples of network 2998 include LANs such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., cable or wireless wide area digital networks including cable television, satellite television, and terrestrial broadcast television, vehicular and industrial networks including CANBus, etc. Some networks 2998 typically require an external network interface adapter (e.g., a USB port of computer system 2900) attached to certain general-purpose data ports or peripheral buses (2950 and 2951); other network interfaces are typically integrated into the core of computer system 2900 by attaching to system buses as described below (e.g., an Ethernet interface connected to a PC computer system or a cellular network interface connected to a smartphone computer system). Computer system 2900 can communicate with other entities using any of these networks 2998. Such communication can be one-way receive-only (e.g., broadcast television), one-way transmit-only (e.g., CANBus connected to some CANBus device), or two-way communication (e.g., connecting to other computer systems using a local area network or wide area network digital network). Certain protocols and protocol stacks can be used on each of those networks and network interfaces as described above.

[0235] The aforementioned human-machine interface device, user-accessible storage device, and network interface can be attached to the kernel 2940 of the computer system 2900.

[0236] The core 2940 may include one or more central processing units (CPUs) 2941, graphics processing units (GPUs) 2942, graphics adapters 2917, dedicated programmable processing units 2943 in the form of field-programmable gate arrays (FPGAs), hardware accelerators 2944 for certain tasks, etc. These devices, along with read-only memory (ROM) 2945, random access memory 2946, and internal mass storage 2947 such as internal non-user-accessible hard disk drives (SDs), etc., can be connected via the system bus 2948. In some computer systems, the system bus 2948 can be accessed via one or more physical connectors to allow for expansion with additional CPUs, GPUs, etc. Peripheral devices may be directly attached to the core's system bus 2948 or attached via a peripheral bus 2949. Peripheral bus architectures include PCI, USB, etc.

[0237] The CPU 2941, GPU 2942, FPGA 2943, and accelerator 2944 can execute certain instructions, which, when combined, constitute the aforementioned computer code. This computer code can be stored in ROM 2945 or RAM 2946. Transient data can also be stored in RAM 2946, while permanent data can be stored in internal mass storage 2947. Fast storage and retrieval to any storage device can be achieved using a cache, which can be closely associated with one or more CPUs 2941, GPUs 2942, mass storage 2947, ROM 2945, RAM 2946, etc.

[0238] Computer-readable media may have computer code thereon that performs various computer-implemented operations. The media and computer code may be media and computer code specifically designed and constructed for the purposes of this disclosure, or the media and computer code may be of a type known and available to those skilled in the art of computer software.

[0239] As an example, and not a limitation, a computer system 2900 having an architecture, particularly a kernel 2940, can provide functionality because one or more processors (including CPUs, GPUs, FPGAs, accelerators, etc.) execute software embodied in one or more tangible computer-readable media. Such computer-readable media can be media associated with user-accessible mass storage as described above, as well as some non-transitory memory of the kernel 2940, such as internal kernel mass storage 2947 or ROM 2945. Software implementing embodiments of this disclosure can be stored in such devices and executed by the kernel 2940. Depending on specific needs, the computer-readable media may include one or more storage devices or chips. The software can cause the kernel 2940, particularly the processors therein (including CPUs, GPUs, FPGAs, etc.), to perform specific processes described herein or specific portions of those processes, including defining data structures stored in RAM 2946 and modifying such data structures according to software-defined processes. Additionally or alternatively, due to hard-wired or otherwise embodied logic in the circuitry (e.g., accelerator 2944), the computer system can provide functionality that the circuitry can replace or operate with the software to perform a particular process described herein or to perform a particular portion of the particular process described herein. Where appropriate, the portion referring to the software may include logic, and vice versa. Where appropriate, the portion referring to the computer-readable medium may include circuitry storing software for execution (e.g., an integrated circuit (IC)), circuitry embodying logic for execution, or both. This disclosure includes any suitable combination of hardware and software. Although some exemplary embodiments are described in this disclosure, there are modifications, substitutions, and various equivalent alternatives that fall within the scope of this disclosure. Therefore, it should be appreciated that those skilled in the art will be able to design numerous systems and methods that, while not expressly shown or described herein, embody the principles of this disclosure and thus fall within its spirit and scope.

Claims

1. A method for video decoding, characterized in that, The method includes: Acquire encoded video data that represents a mesh as volume data of at least one three-dimensional visual content; Determine whether the first two predicted values ​​in the initial set of predicted values ​​have the same value; Based on the determination that the first two predicted values ​​have the same value, the initial set of predicted values ​​is reduced to only a single predicted value; and Based on the single predicted value, at least a portion of the volume data is decoded.

2. The method according to claim 1, characterized in that, The initial set of prediction values ​​represents the prediction values ​​of the current face and one or more adjacent faces of the grid.

3. The method according to claim 2, characterized in that, The initial set of predicted values ​​includes four predicted values, and one or more adjacent faces of the grid are composed of three adjacent faces of the grid.

4. The method according to claim 3, characterized in that, The order of predictions in the initial set of predictions is determined based on the centroids of one or more adjacent faces of the grid and the distance between the centroids.

5. The method according to claim 2, characterized in that, The initial set of predicted values ​​includes two predicted values, and one or more adjacent faces of the grid are composed of two adjacent faces of the grid.

6. The method according to claim 5, characterized in that, The order of predictions in the initial set of predictions is determined based on the centroids of one or more adjacent faces of the grid and the distance between the centroids.

7. The method according to claim 1, characterized in that, The predicted face attribute value is at least one of the face group identifier and the face normal.

8. A video encoding method, characterized in that, The method includes: Acquire video data that represents a mesh as volume data of at least one 3D visual content; Determine whether the first two predicted values ​​in the initial set of predicted values ​​have the same value; Based on the determination that the first two predicted values ​​have the same value, the initial set of predicted values ​​is reduced to only a single predicted value; and Based on the single predicted value, at least a portion of the volume data is encoded.

9. The method according to claim 8, characterized in that, The initial set of prediction values ​​represents the prediction values ​​of the current face and one or more adjacent faces of the grid.

10. The method according to claim 9, characterized in that, The initial set of predicted values ​​includes four predicted values, and one or more adjacent faces of the grid are composed of three adjacent faces of the grid.

11. The method according to claim 10, characterized in that, The order of predictions in the initial set of predictions is determined based on the centroids of one or more adjacent faces of the grid and the distance between the centroids.

12. The method according to claim 9, characterized in that, The initial set of predicted values ​​includes two predicted values, and one or more adjacent faces of the grid are composed of two adjacent faces of the grid.

13. The method according to claim 12, characterized in that, The order of predictions in the initial set of predictions is determined based on the centroids of one or more adjacent faces of the grid and the distance between the centroids.

14. The method according to claim 8, characterized in that, The predicted face attribute value is at least one of the face group identifier and the face normal.

15. A method for encoding visual media data, characterized in that, The method includes: The encoding process generates a bitstream of the visual media data, the bitstream of which includes video data representing a grid as volume data of at least one three-dimensional visual content. The encoding process includes: Determine whether the first two predicted values ​​in the initial set of predicted values ​​have the same value; Based on the determination that the first two predicted values ​​have the same value, the initial set of predicted values ​​is reduced to only a single predicted value; and Based on the single predicted value, at least a portion of the volume data is encoded, and the generated bitstream is transmitted.

16. A video decoding apparatus, characterized in that, The device includes: The acquisition module is configured to acquire coded video data that represents a mesh as volume data of at least one three-dimensional visual content; The determination module is configured to determine whether the first two predicted values ​​in the initial set of predicted values ​​have the same value; The reduction module is configured to reduce the initial set of predicted values ​​to a single predicted value based on the determination that the first two predicted values ​​have the same value; and The decoding module is configured to decode at least a portion of the volume data based on the single predicted value.

17. A video encoding apparatus, characterized in that, The device method includes: The acquisition module is configured to acquire video data that represents a mesh as volume data of at least one three-dimensional visual content; The determination module is configured to determine whether the first two predicted values ​​in the initial set of predicted values ​​have the same value; The reduction module is configured to reduce the initial set of predicted values ​​to a single predicted value based on the determination that the first two predicted values ​​have the same value; and An encoding module is configured to encode at least a portion of the volume data based on the single predicted value.

18. A method for storing video streams, characterized in that, The video encoding method according to any one of claims 8 to 14 is used to generate a video stream; and the video stream is stored.

19. A method for transmitting a video stream, characterized in that, The video encoding method according to any one of claims 8 to 14 is used to generate a video stream; and the video stream is transmitted.

20. A computer-readable storage medium storing instructions, characterized in that, When the instructions are executed by at least one processor, they perform the video encoding method of any one of claims 8 to 14 to generate and store the bitstream.

Images