Motion compensation-based neighborhood construction for TriSoup vertex information

Abstract

The encoder and / or decoder may encode visual data based on the motion-compensated point cloud. For example, the encoder and / or decoder may determine one or more symbols of a neighboring configuration of the current edge based at least on the motion-compensated point cloud. The encoder and / or decoder may encode (e.g., arithmetically encode) vertex information of the current edge based on a context (or a probability model). The context (or probability model) may be selected, for example, using a lookup table based on the neighboring configuration. The neighboring configuration may be a reduced configuration that may represent a subset of symbols of the entire neighboring configuration.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 63 / 433,884, filed December 20, 2022. The above-referenced application is incorporated herein by reference in its entirety. [Background technology]

[0002] An object or scene can be described using volumetric visual data consisting of a series of points. The points can be stored in a point cloud format, which includes a collection of points in three-dimensional space. Because point clouds can be very large in data size, transmitting and processing point cloud data can require data compression schemes specifically designed for the unique characteristics of point cloud data.

[0003] The following summary provides a simplified overview of certain features. It is not an extensive overview and is not intended to identify key or critical elements.

[0004] Coding (e.g., encoding, decoding) may be used to compress and decompress point cloud frames or sequences for efficient storage and transmission. The encoder and / or decoder may encode visual data based on the motion-compensated point cloud. For example, the encoder and / or decoder may determine one or more symbols of a neighboring configuration of the current edge based at least on the motion-compensated point cloud. The encoder and / or decoder may entropy code (e.g., arithmetically code) vertex information of the current edge based on a context (or probability model). The context (or probability model) may be selected, for example, using a lookup table based on the neighboring configuration. The neighboring configuration may be a reduced configuration, which may represent a subset of symbols in the entire neighboring configuration. For example, compensated TriSoup vertices may be determined for the current edge and neighboring edges based, for example, on points of the motion-compensated point cloud adjacent to the edges. For example, multiple planes may be used for the reduction. The use of motion-compensated point clouds may add inter-frame correlation and / or improve compression / decompression of the visual data.

[0005] These and other features and advantages are described in more detail below. [Brief explanation of the drawings]

[0006] Certain features are illustrated by way of example, and not by way of limitation, in the accompanying drawings in which like numerals refer to like elements and in which:

[0007] [Figure 1] 1 illustrates an exemplary point cloud encoding system. [Figure 2] 1 shows an exemplary Morton order. [Figure 3] 1 illustrates an exemplary scan order. [Figure 4] 1 shows an exemplary neighborhood of a cuboid with occupied bits already coded. [Figure 5]We present an example of a dynamic reduction function (DR) that can be used in Optimal Binary Coders with Dynamic Update on the Fly (OBUF). [Figure 6] 10 illustrates an exemplary method for encoding cuboid occupancy using dynamic OBUFs. [Figure 7] An example of an occupied rectangular parallelepiped is shown below. [Figure 8A] 1 shows an exemplary cuboid corresponding to a TriSoup node. [Figure 8B] 1 shows an exemplary refinement to the TriSoup model. [Figure 9] An example of voxelization is shown below. [Figure 10A] 1 shows an exemplary rectangular parallelepiped. [Figure 10B] 1 shows an exemplary rectangular parallelepiped. [Figure 11A] Indicates a TriSoup edge that can be used to entropy encode the current TriSoup edge. [Figure 11B] Indicates a TriSoup edge that can be used to entropy encode the current TriSoup edge. [Figure 11C] Indicates a TriSoup edge that can be used to entropy encode the current TriSoup edge. [Figure 12] 1 illustrates an exemplary encoding method. [Figure 13A] 1 illustrates exemplary types of neighbor information. [Figure 13B] 1 illustrates exemplary types of neighbor information. [Figure 13C] 1 illustrates exemplary types of neighbor information. [Figure 14A] 10 shows exemplary point locations in the point cloud adjacent to the current edge E of the TriSoup node. [Figure 14B] 10 shows exemplary point locations in the point cloud adjacent to the current edge E of the TriSoup node. [Figure 15A] 1 illustrates an exemplary occupancy of point neighbors in a motion compensated point cloud. [Figure 15B]1 illustrates an exemplary occupancy of point neighbors in a motion compensated point cloud. [Figure 16] 10 shows an example of a reduction that can be applied to point neighbors. [Figure 17A] 10 shows another example of shrinking that can be applied to point neighbors. [Figure 17B] 10 shows another example of shrinking that can be applied to point neighbors. [Figure 18A] 10 shows an example compensated TriSoup vertex belonging to an edge. [Figure 18B] This shows the quantization function of the positions of the compensated TriSoup vertices belonging to an edge. [Figure 18C] This shows the quantization function of the positions of the compensated TriSoup vertices belonging to an edge. [Figure 19A] 1 illustrates an exemplary method for encoding vertex information for a current edge. [Figure 19B] 10 illustrates an exemplary method for decoding the vertex information of the current edge. [Figure 20] 1 illustrates an exemplary computer system that may use any of the embodiments described herein. [Figure 21] 1 illustrates exemplary elements of a computing device that may be used to implement any of the various devices described herein. DETAILED DESCRIPTION OF THE INVENTION

[0008] The accompanying drawings and description provide examples. It should be understood that the embodiments shown in the drawings and / or description are non-exclusive and that the features shown and described may be practiced in other embodiments. Examples are provided for the operation of a point cloud or point cloud sequence encoding or decoding system. More specifically, the techniques disclosed herein may relate to point cloud compression for use in encoding and / or decoding devices and / or systems.

[0009] At least some visual data may describe an object or scene using a series of points. Each point may include a two-dimensional (x and y) position and one or more optional attributes, such as color. Volumetric visual data may add another positional dimension to this visual data. For example, volumetric visual data may describe an object or scene using a series of points, each of which may include a three-dimensional (x, y, and z) position and one or more optional attributes, such as color, reflectance, timestamp, etc. Volumetric visual data may, for example, provide a more immersive way to experience the visual data than at least some visual data. For example, an object or scene described by volumetric visual data may be viewable from any angle(s), whereas at least some visual data is generally viewable only from the angle at which it was captured or rendered.

[0010] Volumetric visual data can be used in many applications, including augmented reality (AR), virtual reality (VR), and mixed reality (MR). Scattered volumetric visual data can be used in the automotive industry for the representation of three-dimensional (3D) maps (e.g., cartography) or as input to advanced driver assistance systems. For advanced driver assistance systems, volumetric visual data can typically be input into driving decision algorithms. Volumetric visual data can be used to preserve valuable objects in digital form. In applications for preserving cultural heritage, the goal can be to preserve representations of objects that may be threatened by natural disasters. For example, statues, vases, and temples can be scanned in their entirety and stored as volumetric visual data with billions of samples. This use case for volumetric visual data can be particularly relevant for valuable objects in locations where earthquakes, tsunamis, and typhoons occur frequently. Volumetric visual data can take the form of volumetric frames. A volumetric frame can describe an object or scene captured at a specific time instance. Volumetric visual data may take the form of a sequence of volumetric frames (called a volume sequence or volumetric video), which may describe an object or scene captured at multiple different instances in time.

[0011] Volumetric visual data can be stored in various formats. A point cloud is one format for storing volumetric visual data. A point cloud can include a collection of points in 3D space. Each point in the point cloud can include geometric information that can indicate the point's location in 3D space. For example, the geometric information can indicate the point's location in 3D space using, for example, three Cartesian coordinates (x, y, and z) and / or using spherical coordinates (r, phi, theta) (e.g., when acquired by a rotational sensor). The locations of points in a point cloud can be quantified according to spatial precision. The spatial precision can be the same or different in each dimension. A quantization process can generate a grid in 3D space. One or more points residing within each subgrid volume can be mapped to subgrid center coordinates called voxels. A voxel can be considered a 3D extension of a pixel corresponding to a 2D image grid coordinate. A point in a point cloud can include one or more types of attribute information. The attribute information can indicate characteristics of the point's visual appearance. For example, the attribute information may indicate the texture (e.g., color) of the point, the material type of the point, transparency information of the point, reflectance information of the point, a normal vector to the surface of the point, the velocity of the point, the acceleration at the point, a timestamp indicating when the point was captured, or a modality (e.g., running, walking, or flying) indicating how the point was captured. Points in the point cloud may include light field data in the form of multiple view-dependent texture information. The light field data may be another type of arbitrary attribute information.

[0012] The points in the point cloud may describe an object or scene. For example, the points in the point cloud may describe the exterior surface and / or interior structure of the object or scene. The object or scene may be synthetically generated by a computer. The object or scene may be generated from capturing a real-world object or scene. Geometry information of a real-world object or scene may be obtained by 3D scanning and / or photogrammetry. 3D scanning may include different types of scanning, such as laser scanning, structured light scanning, and / or modulated light scanning. 3D scanning may obtain the geometry information. 3D scanning may obtain the geometry information, for example, by moving one or more laser heads, structured light cameras, and / or modulated light cameras relative to the object or scene being scanned. Photogrammetry may obtain the geometry information. Photogrammetry may obtain the geometry information, for example, by triangulating the same features or points in different spatially shifted 2D photographs. Point cloud data may take the form of a point cloud frame. A point cloud frame may describe an object or scene captured at a particular time instance. Point cloud data may take the form of a sequence of point cloud frames, which may be referred to as a point cloud sequence or a point cloud video, which may describe an object or scene captured at multiple different instances of time.

[0013] The data size of a point cloud frame or point cloud sequence may be excessive (e.g., too large) for storage and / or transmission in many applications. For example, a single point cloud may include more than one million points, or even more than one billion points. Each point may include geometric information and one or more types of attribute information. The geometric information for each point may include, for example, three Cartesian coordinates (x, y, and z) and / or spherical coordinates (r, phi, theta), each of which may be represented using at least 10 bits per component or a total of 30 bits. The attribute information for each point may include texture corresponding to multiple (e.g., three) color components (e.g., R, G, and B color components). Each color component may be represented using, for example, 8 to 10 bits per component or a total of 24 to 30 bits. For example, a single point may include at least 54 bits of information, with at least 30 bits of geometric information and at least 24 bits of texture. If a point cloud frame contains 1 million such points, each point cloud frame may require 54 million bits or 54 megabits to represent. For a dynamic point cloud that changes over time, a data rate of 1.32 gigabits per second may be required to transmit (e.g., transmit) the points of a point cloud sequence at a frame rate of 30 frames per second. A raw representation of a point cloud may require a large amount of data, and practical deployment of point cloud-based technologies may require compression techniques that enable the storage and distribution of point clouds at a reasonable cost.

[0014] Encoding may be used to compress and / or reduce the data size of a point cloud frame or a point cloud sequence to provide more efficient storage and / or transmission. Decoding may be used to decompress a compressed point cloud frame or a point cloud sequence for display and / or other forms of consumption (e.g., by a machine learning-based device, a neural network-based device, an artificial intelligence-based device, or other types of machine-based processing algorithms and / or devices). Compression of the point cloud may be lossy (introducing differences to the original data) for distribution to and viewing by an end user, for example, on AR or VR glasses or any other 3D-enabled device. Lossy compression may enable high compression ratios but may imply a trade-off between compression and visual quality perceived by the end user. Other frameworks, such as those for medical applications or autonomous driving, may require lossless compression to avoid, for example, transmission (e.g., transmission) and altering the results of decisions made based on analysis of the decompressed point cloud frames.

[0015] 1 illustrates an exemplary point cloud encoding (e.g., encoding and / or decoding) system 100. The point cloud encoding system 100 may include a source device 102, a transmission medium 104, and a destination device 106. The source device 102 may encode a point cloud sequence 108 into a bitstream 110 for more efficient storage and / or transmission. The source device 102 may store and / or transmit (e.g., transmit) the bitstream 110 to the destination device 106 via the transmission medium 104. The destination device 106 may decode the bitstream 110 to display the point cloud sequence 108 or for other forms of consumption (e.g., further analysis, storage, etc.). The destination device 106 may receive the bitstream 110 from the source device 102 via the storage medium or transmission medium 104. The source device 102 and the destination device 106 may include any number of different devices. The source device 102 and the destination device 106 may include, for example, a cluster of interconnected computer systems acting as a seamless pool of resources (also called a cloud of computers or cloud computing), a server, a desktop computer, a laptop computer, a tablet computer, a smartphone, a wearable device, a television, a camera, a video game console, a set-top box, a video streaming device, a vehicle (e.g., an autonomous vehicle), or a head-mounted display. The head-mounted display may allow a user to view a VR, AR, or MR scene and, for example, adjust the view of the scene based on the user's head movements. The head-mounted display may be connected (tethered) to a processing device (e.g., a server, desktop computer, set-top box, or video game console) or may be completely self-contained.

[0016] The source device 102 may include a point cloud source 112, an encoder 114, and an output interface 116. The source device 102 may include, for example, the point cloud source 112, the encoder 114, and the output interface 116 for encoding the point cloud sequence 108 into a bitstream 110. The point cloud source 112 may provide (or generate) the point cloud sequence 108, for example, from the capture of natural and / or synthetically generated scenes. The synthetically generated scenes may be scenes including computer-generated graphics. The point cloud source 112 may include one or more point cloud capture devices, a point cloud archive containing previously captured natural and / or synthetically generated scenes, a point cloud feed interface for receiving captured natural and / or synthetically generated scenes from a point cloud content provider, and / or a processor for generating the synthesized point cloud scene. The point cloud capture device may include, for example, one or more laser scanning devices, structured light scanning devices, modulated light scanning devices, and / or passive scanning devices.

[0017] A point cloud sequence 108 may include a series of point cloud frames 124 (e.g., the example shown in FIG. 1 ). A point cloud frame may describe an object or scene captured at a particular time instance. The point cloud sequence 108 may achieve the impression of motion by sequentially presenting the point cloud frames 124 of the point cloud sequence 108 using a constant or variable time. A point cloud frame may include a collection of points (e.g., voxels) 126 in 3D space. Each point 126 may include geometric shape information that may indicate the point's location in 3D space. The geometric shape information may indicate the point's location in 3D space using, for example, three Cartesian coordinates (x, y, and z). One or more of the points 126 may include one or more types of attribute information. The attribute information may indicate characteristics of the point's visual appearance. For example, the attribute information may indicate, for example, the texture (e.g., color) of the point, the material type of the point, transparency information of the point, reflectance information of the point, a normal vector relative to the surface of the point, the velocity of the point, the acceleration at the point, a timestamp indicating when the point was captured, a modality indicating how the point was captured (e.g., running, walking, or flying), etc. One or more of the points 126 may include light field data, for example, in the form of multiple view-dependent texture information. The light field data may be any other type of attribute information. The color attribute information of one or more of the points 126 may include a luminance value and two color difference values. The luminance value may represent the luminance (e.g., luma component, Y) of the point. The color difference values ​​may represent the blue and red components (e.g., chroma components, Cb and Cr) of the point, respectively, separate from its brightness. The other color attribute values ​​may be represented, for example, based on a different color scheme (e.g., RGB or monochrome color scheme).

[0018] The encoder 114 may encode the point cloud sequence 108 into a bitstream 110. To encode the point cloud sequence 108, the encoder 114 may use one or more lossless or lossy compression techniques to reduce redundant information in the point cloud sequence 108. To encode the point cloud sequence 108, the encoder 114 may use one or more prediction techniques to reduce redundant information in the point cloud sequence 108. Redundant information is information that can be predicted at the decoder 120 and may not need to be sent (e.g., transmitted) to the decoder 120 for accurate decoding of the point cloud sequence 108. For example, the Motion Picture Expert Group (MPEG) introduced the Geometry-Based Point Cloud Compression (G-PCC) standard (ISO / IEC Standard 23090-9: Geometry-Based Point Cloud Compression). G-PCC specifies encoded bitstream syntax and semantics for transmission and / or storage of compressed point cloud frames, as well as decoder operations for reconstructing compressed point cloud frames from the bitstream. During the standardization of G-PCC, reference software (ISO / IEC Standard 23090-21: Reference Software for G-PCC) was developed to encode the geometric shape and attribute information of a point cloud frame. To encode the geometric shape information of a point cloud frame, the G-PCC reference software encoder may perform voxelization. The G-PCC reference software encoder may perform voxelization, for example, by quantifying the positions of points within a point cloud. Quantifying the positions of points within a point cloud may generate a grid in 3D space. The G-PCC reference software encoder may map points to the center coordinates of sub-grid volumes (e.g., voxels) within which their quantized positions reside. The G-PCC reference software encoder may perform geometric shape analysis using an occupancy tree to compress the geometric shape information. The G-PCC reference software encoder may entropy encode the results of the geometric shape analysis to further compress the geometric shape information.To encode the point cloud attribute information, the G-PCC reference software encoder may use transform tools such as a region-adaptive hierarchical transform (RAHT), a predictive transform, and / or a lifting transform. The lifting transform may be built on top of the predictive transform. The lifting transform may include an additional update / lifting step. The lifting transform and the predictive transform may also be referred to as a predictive / lifting transform or a "pred lift." The encoder 114 may operate in the same or similar manner as the encoder provided by the G-PCC reference software.

[0019] The output interface 116 may be configured to write and / or store the bitstream 110 on the transmission medium 104. The bitstream 110 may be sent (e.g., transmitted) to the destination device 106. Additionally or alternatively, the output interface 116 may be configured to send (e.g., transmit), upload, and / or stream the bitstream 110 to the destination device 106 via the transmission medium 104. The output interface 116 may include a wired and / or wireless transmitter configured to send (e.g., transmit), upload, and / or stream the bitstream 110 according to one or more proprietary, open source, and / or standardized communication protocols. The one or more proprietary, open source, and / or standardized communication protocols may include, for example, Digital Video Broadcasting (DVB) standards, Advanced Television Systems Committee (ATSC) standards, Integrated Services Digital Broadcasting (ISDB) standards, Data Over Cable Service Interface Specification (DOCSIS) standards, 3rd Generation Partnership Project (3GPP®) standards, Institute of Electrical and Electronics Engineers (IEEE) standards, Internet Protocol (IP) standards, Wireless Application Protocol (WAP) standards, and / or any other communication protocols.

[0020] Transmission medium 104 may include wireless, wired, and / or computer-readable media. For example, transmission medium 104 may include one or more wires, cables, air interfaces, optical disks, flash memory, and / or magnetic memory. Additionally or alternatively, transmission medium 104 may include one or more networks (e.g., the Internet) or file servers configured to store and / or transmit (e.g., transmit) encoded video data.

[0021] The destination device 106 may decode the bitstream 110 into a point cloud sequence 108 for display or other consumption. The destination device 106 may include one or more of an input interface 118, a decoder 120, and / or a point cloud display 122. The input interface 118 may be configured to read the bitstream 110 stored on the transmission medium 104. The bitstream 110 may be stored on the transmission medium 104 by the source device 102. Additionally or alternatively, the input interface 118 may be configured to receive, download, and / or stream the bitstream 110 from the source device 102 over the transmission medium 104. The input interface 118 may include a wired and / or wireless receiver configured to receive, download, and / or stream the bitstream 110 according to one or more proprietary, open source, standardized communication protocols, and / or any other communication protocol. Examples of protocols include the Digital Video Broadcasting (DVB) standard, the Advanced Television Systems Committee (ATSC) standard, the Integrated Services Digital Broadcasting (ISDB) standard, the Data Over Cable Service Interface Specification (DOCSIS) standard, the 3rd Generation Partnership Project (3GPP®) standard, the Institute of Electrical and Electronics Engineers (IEEE) standard, the Internet Protocol (IP) standard, and the Wireless Application Protocol (WAP) standard.

[0022] The decoder 120 may decode the point cloud sequence 108 from the encoded bitstream 110. For example, the decoder 120 may operate in the same or similar manner as the decoder provided by the G-PCC reference software. The decoder 120 may decode a point cloud sequence that approximates the point cloud sequence 108. The decoder 120 may decode a point cloud sequence that approximates the point cloud sequence 108 due to, for example, lossy compression of the point cloud sequence 108 by the encoder 114 and / or errors introduced into the encoded bitstream 110 when transmission to, for example, the destination device 106 occurred.

[0023] The point cloud display 122 may display the point cloud sequence 108 to a user. The point cloud display 122 may include, for example, a cathode ray tube (CRT) display, a liquid crystal display (LCD), a plasma display, a light emitting diode (LED) display, a 3D display, a holographic display, a head-mounted display, or any other display device suitable for displaying the point cloud sequence 108.

[0024] The point cloud encoding (e.g., encoding / decoding) system 100 is presented by way of example and not limitation. Point cloud encoding systems other than the point cloud encoding system 100 and / or modified versions of the point cloud encoding system 100 may implement the methods and processes as described herein. For example, the point cloud encoding system 100 may include other components and / or arrangements. The point cloud source 112 may be external to the source device 102, for example. The point cloud display device 122 may be external to the destination device 106, for example, or may be omitted entirely (e.g., if the point cloud sequence 108 is intended for consumption by a machine and / or a storage device). The source device 102 may further comprise, for example, a point cloud decoder. The destination device 106 may comprise, for example, a point cloud encoder. For example, the source device 102 may be configured to further receive an encoded bitstream from the destination device 106. Receiving the encoded bitstream from the destination device 106 may support bidirectional point cloud transmission between the devices.

[0025] As described herein, the encoder may quantify the location of points within a point cloud according to a spatial precision, which may be the same or different in each dimension of the points. The quantization process may generate a grid in 3D space. The encoder may map any point that resides within each subgrid volume to a subgrid center coordinate called a voxel. A voxel may be considered a 3D extension of a pixel that corresponds to a 2D image grid coordinate.

[0026] The encoder may represent or encode the voxelized point cloud. The encoder may represent or encode the voxelized point cloud using, for example, an occupancy tree. For example, the encoder may divide an initial volume or cuboid containing the voxelized point cloud into sub-cuboids. The initial volume or cuboid may be referred to as a bounding box. The cuboid may be, for example, a cube. The encoder may recursively divide each sub-cuboid that contains at least one point of the point cloud. The encoder may not further divide a sub-cuboid that does not contain at least one point of the point cloud. A sub-cuboid that contains at least one point of the point cloud may be referred to as an occupied sub-cuboid. A sub-cuboid that does not contain at least one point of the point cloud may be referred to as an unoccupied sub-cuboid. The encoder may divide an occupied sub-cuboid into, for example, two sub-cuboids (to form a binary tree), four sub-cuboids (to form a quadtree), or eight sub-cuboids (to form an octree). The encoder may split the occupied sub-cuboid to obtain further sub-cuboids. The sub-cuboids may have the same size and shape at a given depth level of the occupancy tree. For example, the sub-cuboids may have the same size and shape at a given depth level of the occupancy tree if the encoder splits the occupied sub-cuboid along a plane that passes through the center of the edges of the sub-cuboid.

[0027] The initial volume or cuboid containing the voxelized point cloud may correspond to the root node of the occupancy tree. Each occupied subcuboid split from the initial volume may correspond to a node (of the root node) at a second level of the occupancy tree. Each occupied subcuboid split from an occupied subcuboid at the second level may correspond to a node at a third level of the occupancy tree (the node off the occupied subcuboid at the second level from which it was split). The occupancy tree structure may continue to be formed in this manner for each recursive splitting iteration, for example, until some maximum depth level of the occupancy tree is reached or until each occupied subcuboid has a volume corresponding to one voxel.

[0028] Each non-leaf node in the occupancy tree may include or be associated with an occupancy word that represents the occupancy state of the cuboid corresponding to the node. For example, a node in the occupancy tree corresponding to a cuboid divided into eight sub-cuboids may include or be associated with a one-byte occupancy word. Each bit (called an occupancy bit) of the one-byte occupancy word may represent or indicate the occupancy of a different one of the eight sub-cuboids. Each occupied sub-cuboid may be represented or indicated by a binary "1" in the one-byte occupancy word. Each unoccupied sub-cuboid may be represented or indicated by a binary "0" in the one-byte occupancy word. Occupied and unoccupied sub-cuboids may be represented or indicated by opposite one-bit binary values ​​in the one-byte occupancy word (e.g., a binary "0" representing or indicating an occupied sub-cuboid and a binary "1" representing or indicating an unoccupied sub-cuboid).

[0029] Each bit of the occupancy word may represent or indicate the occupancy of a different one of the eight sub-rectangles. Each bit of the occupancy word may represent or indicate the occupancy of a different one of the eight sub-rectangles, for example, according to the so-called Morton order. For example, the least significant bit of the occupancy word may represent or indicate the occupancy of, for example, a first sub-rectangle of the eight sub-rectangles, for example, according to Morton order. The second least significant bit of the occupancy word may represent or indicate the occupancy of, for example, a second sub-rectangle of the eight sub-rectangles, for example, according to Morton order, etc.

[0030] 2 shows an exemplary Morton order. More specifically, FIG. 2 shows the Morton order of eight sub-rectangles 202-216 divided from a rectangular prism 200. The sub-rectangles 202-216 may be labeled based on their Morton order, for example, with the child node 202 being first in the Morton order and the child node 216 being last in the Morton order. The Morton order for the sub-rectangles 202-216 may be a local lexicographic order in xyz.

[0031] The voxelized point cloud geometry may be represented by and determined from the initial volumes and occupancy words of the nodes in the occupancy tree. The encoder may send (e.g., transmit) the initial volumes and occupancy words of the nodes in the occupancy tree in a bitstream to a decoder to reconstruct the point cloud. The encoder may entropy encode the occupancy words. For example, the encoder may entropy encode the occupancy words before sending (e.g., transmitting) the initial volumes and occupancy words of the nodes in the occupancy tree. The encoder may encode occupancy bits of the occupancy words of the nodes corresponding to the cuboid. For example, the encoder may encode occupancy bits of the occupancy words of the nodes corresponding to the cuboid based on one or more occupancy bits of the occupancy words of other nodes corresponding to cuboids that are adjacent to or spatially close to the cuboid of the occupancy bits being encoded.

[0032] The encoder and / or decoder may encode (e.g., encode and / or decode) occupied bits of consecutive occupied words in scan order. Scan order may also be referred to as scanning order. For example, the encoder and / or decoder may scan the occupation tree in breadth-first order. All occupied words of nodes at a given depth (e.g., level) in the occupation tree may be scanned. All occupied words of nodes at a given depth (e.g., level) in the occupation tree may be scanned before scanning occupied words of nodes at the next depth (e.g., level). Within a given depth, the encoder and / or decoder may scan occupied words of nodes in Morton order. Within a given node, the encoder and / or decoder may also scan occupied bits of occupied words of the node in Morton order.

[0033] FIG. 3 illustrates an exemplary scan order. FIG. 3 illustrates an example scan order (e.g., breadth-first order as described herein) of an occupancy tree 300. More specifically, FIG. 3 illustrates an exemplary scan order of the first three levels of the occupancy tree 300. In FIG. 3, a cuboid (e.g., cube) 302 corresponding to the root node of the occupancy tree 300 may be divided into eight sub-cuboids (e.g., sub-cubes). Two sub-cuboids 304 and 306 of the eight sub-cuboids may be occupied. The other six sub-cuboids of the eight sub-cuboids may be unoccupied. According to Morton order, the first 8-bit occupancy word (e.g., occW 1,1 ) can be constructed to represent the occupation word of the root node. The first 8 bits of the occupation word (e.g., occW 1,1 ) may represent or indicate the occupation of a subcube of the eight subcubes in the Morton order. For example, the first 8-bit occupation word occW 1,1 The least significant occupancy bit of the first 8-bit occupancy word occW represents or may indicate the occupancy of the first of the 8 sub-cuboids in Morton order. 1,1 The second least significant occupied bit of may represent or indicate the occupancy of the second of the eight sub-cuboids in Morton order.

[0034] Each of the occupied subcubes (e.g., two occupied subcubes 304 and 306) may correspond to a node from the root node of the second-level occupancy tree 300. Each of the occupied subcubes (e.g., two occupied subcubes 304 and 306) may be further divided into eight subcubes. For example, one of the subcubes 308 of the eight subcubes divided from subcube 304 may be occupied, and the other seven subcubes may be unoccupied. Of the eight subcubes divided from subcube 306, three of subcubes 310, 312, and 314 may be occupied, and the other five of the eight subcubes divided from subcube 306 may be unoccupied. Two second 8-bit occupancy words occW 2,1 and occW 2,2may be constructed in this order to represent the occupancy words of the nodes corresponding to sub-cuboid 304 and sub-cuboid 306, respectively.

[0035] Each of the occupied sub-cuboids (e.g., four occupied sub-cuboids 308, 310, 312, and 314) may correspond to a node in the third-level occupancy tree 300. Each of the occupied sub-cuboids (e.g., four occupied sub-cuboids 308, 310, 312, and 314) may be further divided into a total of eight sub-cuboids or 32 sub-cuboids. For example, four third-level 8-bit occupied words occW 3,1 , occW 3,2 , occW 3,3 , and occW 3,4 may be constructed in this order to represent the occupancy words of the nodes corresponding to sub-cuboid 308, the occupancy words of the nodes corresponding to sub-cuboid 310, the occupancy words of the nodes corresponding to sub-cuboid 312, and the occupancy words of the nodes corresponding to sub-cuboid 314, respectively.

[0036] The occupancy words of the exemplary occupancy tree 300 may be entropy encoded (e.g., entropy encoded by an encoder and / or entropy decoded by a decoder) according to, for example, a scan order (e.g., Morton order) described herein. The occupancy words of the exemplary occupancy tree 300 may be entropy encoded (e.g., entropy encoded by an encoder and / or entropy decoded by a decoder) according to, for example, a scan order (e.g., Morton order) described herein. 1,1 ~occW 3,4The scan order described herein may be a breadth-first scan order. The occupied words of all nodes having the same depth (or level) as the current parent node may already be entropy encoded, for example, if the occupied words of the current child node belonging to the current parent node have been entropy encoded. For example, the occupied words of all nodes having the same depth (e.g., level) as the current child node and having a lower Morton order than the current child node may also already be entropy encoded. Some of the already encoded occupied words may be used to entropy encode the occupied words of the current child node. The already encoded occupied words of adjacent parent and child nodes may be used, for example, to entropy encode the occupied words of the current child node. The occupied bits of the occupied word having a lower Morton order than the particular occupied bit may also already be entropy coded and may be used to code the occupied bit of the occupied word of the current child node, for example, when the particular occupied bit of the occupied word of the current child node has been coded (e.g., entropy coded).

[0037] 4 illustrates exemplary neighborhoods of cuboids for entropy encoding the occupancy of a child cuboid. More specifically, FIG. 4 illustrates exemplary neighborhoods of cuboids with already-coded occupancy bits. The neighborhoods of cuboids with already-coded occupancy bits can be used to entropy encode the occupancy bits of a current child cuboid 400. The neighborhoods of cuboids with already-coded occupancy bits can be determined, for example, based on a scan order of an occupancy tree representing the geometry of the cuboids of FIG. 4 as discussed herein. For a current child cuboid, the cuboid neighbors may include one or more of: a cuboid close to the current child cuboid, a cuboid sharing a vertex with the current child cuboid, a cuboid sharing an edge with the current child cuboid, a cuboid sharing a face with the current child cuboid, a parent cuboid close to the current child cuboid, a parent cuboid sharing a vertex with the current child cuboid, a parent cuboid sharing an edge with the current child cuboid, a parent cuboid sharing a face with the current child cuboid, a parent cuboid close to the current parent cuboid, a parent cuboid sharing a vertex with the current parent cuboid, a parent cuboid sharing an edge with the current parent cuboid, a parent cuboid sharing a face with the current parent cuboid, etc. As shown in FIG. 4 , a current child cuboid 400 may belong to a current parent cuboid 402. According to the scan order of the occupancy words and occupancy bits of the nodes of the occupancy tree, the occupancy bits of the four child cuboids 404, 406, 408, and 410 belonging to the same current parent cuboid 402 may have already been coded. The occupancy bits of the child cuboid 412 of the preceding parent cuboid may have already been coded. The occupancy bits of the parent cuboid 414, whose occupancy bits have not yet been coded, may have already been coded. The occupancy bits of the current child cuboid 400 may be coded using the occupancy bits already coded of the cuboids 404, 406, 408, 410, 412, and 414.

[0038] The number (e.g., quantity) of possible occupancy configurations (e.g., sets of one or more occupancy words and / or occupancy bits) for the neighbors of the current child cuboid is 2 Nwhere N is the number (e.g., quantity) of cuboids in the neighborhood of the current child cuboid that have already coded occupancy bits. The neighborhood of the current child cuboid may include tens of cuboids. The neighborhood of the current child cuboid (e.g., tens of cuboids) may include 26 neighboring parent cuboids that share faces, edges, and / or vertices with the parent cuboid of the current child cuboid, and also several neighboring child cuboids that share faces, edges, or vertices with the current child cuboid. The occupancy configuration of the neighborhood of the current child cuboid may be limited to a subset of neighboring cuboids or may have billions of possible occupancy configurations, making its direct use impractical. The encoder and / or decoder may use the occupancy configurations for the neighborhood of the current child cuboid to select a context (e.g., a probability model) from a set of contexts of a binary entropy coder (e.g., a binary arithmetic coder) that can code the occupancy bits of the current child cuboid. Context-based binary entropy coding may be similar to the context-adaptive binary arithmetic coder (CABAC) used in MPEG-H Part 2 (also known as High Efficiency Video Coding (HEVC)).

[0039] The encoder and / or decoder may use several methods to reduce the adjacent occupancy configuration of the current child cuboid to be encoded to a practical number (e.g., quantity) of reduced occupancy configurations. 6 That is, 64 occupancy configurations can be reduced to 9 occupancy configurations. Occupancy configurations can be reduced by using geometric invariants. The occupancy score of the current child cuboid is 2x the occupancy score of the 26 neighboring parent cuboids. 26 A score can be obtained from the occupancy configurations. The score can be further reduced to a ternary occupancy prediction (e.g., "predicted-occupied," "uncertain," or "predicted-unoccupied") by using a score threshold. The number (e.g., quantity) of nearby occupied child cuboids and the number (e.g., quantity) of nearby unoccupied child cuboids can be used instead of the individual occupancies of these child cuboids.

[0040] An encoder and / or decoder using / employing one or more of the methods described herein may reduce the number (e.g., quantity) of possible occupancy configurations for the neighbors of the current child cuboid to a more manageable number (e.g., several thousand). It has been observed that instead of directly associating the reduction in the number (e.g., quantity) of contexts (e.g., probability models) with the reduction in occupancy configurations, another mechanism, namely, OBUF (Optimal Binary Coders with Update on the Fly), may be used. The encoder and / or decoder may implement OBUF to limit the number (e.g., quantity) of contexts to a lower number (e.g., 32 contexts).

[0041] The OBUF may use a limited number (e.g., 32) of contexts (e.g., probability models). The number (e.g., quantity) of contexts in the OBUF may be a fixed number (e.g., a fixed quantity). The contexts used by the OBUF may be ordered and referenced by a context index (e.g., a context index ranging from 0 to 31), and a "1" may be encoded from the lowest to the highest virtual probability. A context index lookup table (LUT) may be initialized at the beginning of the point cloud encoding process. For example, the LUT may initially point to a context having a median virtual probability for encoding a "1" for all inputs. The LUT may initially point to a context having a median virtual probability for encoding a "1" among the limited number (e.g., quantity) of contexts for all inputs. This LUT may take as input the occupancy configuration of neighbors of the current child cuboid and output a context index associated with the occupancy configuration. The LUT may have the same number of entries as the reduced occupancy configuration (e.g., approximately several thousand entries). Encoding the occupancy bit of the current child cuboid may include steps including, for example, determining a reduced occupancy configuration of the current child node based on the value of the encoded occupancy bit of the current child cuboid, obtaining a context index by using the reduced occupancy configuration as an entry into the LUT, encoding the occupancy bit of the current child cuboid by using the context pointed to (or indicated) by the context index, and updating the LUT entry corresponding to the reduced occupancy configuration. For example, if a binary "0" (e.g., indicating that the current child cuboid is unoccupied) is encoded, the LUT entry may be decreased to a lower context index value. For example, if a binary "1" (e.g., indicating that the current child cuboid is occupied) is encoded, the LUT entry may be increased to a higher context index value. The context index update process may be based, for example, on a theoretical model of optimal distribution for hypothetical probabilities associated with a limited number (e.g., quantity) of contexts.This virtual probability may be fixed by the model and may differ from the internal probability of the context that may evolve when, for example, encoding of a bit of data occurs. The evolution of the internal context may follow a well-known process similar to that of CABAC.

[0042] The encoder and / or decoder may implement a “dynamic OBUF” scheme. The “dynamic OBUF” scheme may, for example, enable the encoder and / or decoder to handle a much larger number (e.g., quantity) of occupancy configurations for the neighbors of the current child cuboid than a typical OBUF. Using a larger number (e.g., quantity) of occupancy configurations for the neighbors of the current child cuboid may result in improved compression performance and may keep complexity within a reasonable range. By using an occupancy tree compressed by an OBUF, the encoder and / or decoder may achieve lossless compression performance as good as 1 bit per point (bpp) for encoding dense point cloud geometry. The encoder and / or decoder may implement a dynamic OBUF to further reduce the bitrate, potentially by more than 25%, down to 0.7 bpp.

[0043] The OBUF may not take as input a wide variety of reduced occupancy configurations for the neighbors of the current child cuboid, potentially resulting in the loss of useful correlations. The OBUF may increase the size of the context index LUT to handle a greater variety of occupancy configurations as input than for the neighbors of the current child cuboid. Such an increase may dilute statistics and worsen compression performance. For example, if the LUT has millions of entries and the point cloud has hundreds of thousands of points, most entries may not be visited (e.g., looked up, accessed, etc.). Many entries may be visited only a few times, and their associated context indexes may not be updated enough times to reflect any meaningful correlation between occupancy configuration values ​​and the occupancy probability of the current child cuboid. A dynamic OBUF may be implemented to mitigate the dilution of statistics due to an increase in the number (e.g., quantity) of occupancy configurations for the neighbors of the current child cuboid. This mitigation may be achieved by “dynamic shrinking” of occupancy configurations in the dynamic OBUF.

[0044] A dynamic OBUF may add an additional step of shrinking the neighboring occupancy configurations of the current child cuboid before using the context index LUT, for example. This step may be called dynamic shrinking because it evolves based on, for example, the progress of the point cloud encoding, or more precisely, based on the occupancy configurations already visited (e.g., looked up in the LUT).

[0045] As discussed herein, many possible occupancy configurations for the neighbors of the current child cuboid may potentially be involved, but only a subset may be visited when point cloud encoding occurs. This subset may characterize the type of point cloud. For example, most of the visited occupancy configurations may indicate occupied neighboring cuboids of the current child cuboid, for example, when an AR or VR dense point cloud is encoded. On the other hand, most of the visited occupancy configurations may indicate only a few occupied neighboring cuboids of the current child cuboid, for example, when a sparse point cloud acquired by a sensor is encoded. The role of dynamic shrinking may be, for example, to obtain more accurate correlation based on the most frequently visited occupancy configurations while refraining from (e.g., actively reducing) other occupancy configurations that are much less frequently visited. Dynamic shrinking may be updated on the fly. Dynamic shrinking may be updated on the fly, for example, after each visit of an occupancy configuration (e.g., lookup in a LUT) when occupancy data encoding occurs.

[0046] 5 shows an example of a dynamic shrinking function DR that can be used in a dynamic OBUF. The dynamic shrinking function DR is a function of bit β of the occupancy configuration 500. j can be obtained by masking β=β1...β K It consists of K bits. The size of the mask can be reduced, for example, if the occupancy configuration is visited (e.g., looked up in a LUT) a certain number of times (e.g., a certain number). The initial dynamic reduction function DR 0 is a constant function DR for all occupancy configurations β 0 All bits may be masked for all occupancy configurations so that (β)=0. The dynamic reduction function is the function DR n Updated function from DR n+1 The dynamic shrinking function may evolve into, for example, a function DR n Updated function from DR n+1 The function can evolve to β'=DR n (β)=β1...β kn(β) where k n(β) 510 is the number of unmasked bits (e.g., quantity). 0 The initialization of k(β) may correspond to k0(β)=0, and the natural evolution of the shrinkage function for finer statistics is the increase in the number of unmasked bits (e.g., quantity) k n (β)≦k n+1 (β). The dynamic shrinkage function is k for all occupancy configurations β. n can be completely determined by the value of

[0047] A visit to an occupied configuration (e.g., an instance of a lookup in the LUT) is performed for all dynamically reduced occupied configurations β'=DR n (β) can be tracked by a variable NV(β'). The corresponding number (e.g., quantity) of visits NV(β V ') is, for example, the occupied configuration β V After each instance of encoding the occupancy bits based on , the number (e.g., quantity) of visits NV(β V ') is the threshold th V If it is greater than NV(β V ')>th V Next, the number of unmasked bits (e.g., quantity) k n (β) is β V ' can be increased by 1 for all occupancy configurations that are dynamically reduced to '. This means that the dynamically reduced occupancy configuration β V ' into a dynamically reduced new two-occupancy configuration β defined by the following equation: 0 ' and β 1 ' corresponds to replacing it with '. β 0 '=β V '0=β V 1...β V kn(β) 0, and β 1 '=β V '1=β V 1...β V kn(β) 1 In other words, the number of unmasked bits (e.g., quantity) is DR n (β)=βV ', k for all occupancy configurations β n+1 (β)=k n (β)+1, which is an increase of 1. The number of visits (e.g., quantity) of the new dynamically reduced two-occupancy configuration may be initialized to zero. NV(β 0 ')=NV(β 1 ')=0 (I) At the beginning of encoding, an initial dynamic reduction function DR 0 The initial number (e.g., quantity) of visits may be set as follows: NV(DR 0 (β))=NV(0)=0 The evolution of NVs in dynamically reduced occupancy configurations can be fully defined.

[0048] The corresponding LUT entry LUT[β V '] is β V Two new entries LUT[β 0 '] and LUT[β 1 ']. The corresponding LUT entry LUT[β V '] is, for example, the dynamically reduced occupancy configuration β V ' is dynamically reduced to a new two-occupancy configuration β 0 ' and β 1 ', then β V Two new entries LUT[β 0 '] and LUT[β 1 '], LUT[β 0 ']=LUT[β 1 ']=LUT[β V '] (II) They are then evolved separately. The evolution of the coder index LUT on the dynamically reduced occupancy configuration can be fully defined.

[0049] Reduction Function DR n is the occupancy configuration β'=DR in which the leaf node 530 is reduced.n (β) is a set of growing binary trees T n 520. The initial tree can be modeled by 0=DR 0 There can be a single root node associated with (β). 0 ' and β 1 ' by β V The replacement of the dynamically reduced β 0 ' and β 1 ' and attach two new nodes associated with it, V ' to the leaf node associated with the tree T n It can be used to grow a tree. n+1 can be obtained by this growth. The number of visits (e.g., quantity), NV, and LUT of context indexes are defined on the leaf nodes and can evolve with the growth of the tree through equations (I) and (II).

[0050] A practical implementation of the dynamic OBUF is to use the arrays NV[β'] and LUT[β'] of context indices and the tree T n 520. An alternative to storing the tree is to store an array k of the number of unmasked bits (e.g., quantity). n [β] 510 can be stored.

[0051] A limitation for implementing a dynamic OBUF may be its memory footprint. In some applications, millions of occupied configurations are actually processed, and the approximately 20-bit β i Each bit β i may correspond to the occupancy of the neighboring cubes of the current child cube, or to the set of neighboring cubes of the current child cube.

[0052] The higher (e.g., more significant) bit β i (e.g., β0, β1, etc.) may be the first unmasked bit. i(e.g., β0, β1, etc.) may be, for example, the first bit that is not masked during the evolution of the dynamic shrinkage function DR. i The order of neighbor-based information placed in can affect compression performance. The neighboring information is ordered from higher (e.g., highest) priority to lower priority, and in this order, the bit β i , may be ordered from higher weight to lower weight. The priority may be, from most important to least important, the occupancy of a set of close adjacent child cuboids, then the occupancy of close adjacent child cuboids, then the occupancy of close adjacent parent cuboids, then the occupancy of non-close adjacent child nodes, and finally the occupancy of non-close adjacent parent nodes. Neighboring nodes that share a face with the current child node may also have a higher priority than neighboring nodes that share an edge (but not a face) with the current child node. Neighboring nodes that share an edge with the current child node may have a higher priority than neighboring nodes that share only vertices with the current child node.

[0053] FIG. 6 illustrates an exemplary method for encoding the occupancy of a cuboid using a dynamic OBUF. More specifically, FIG. 6 illustrates an exemplary method for encoding the occupancy bits of a current child cuboid using a dynamic OBUF. One or more steps of FIG. 6 may be performed by an encoder and / or a decoder (e.g., the encoder 114 and / or the decoder 120 of FIG. 1). All or part of the flowchart may be implemented by a coder (e.g., the encoder 114 and / or the decoder 120 of FIG. 1), the exemplary computer system 2000 of FIG. 20, and / or the exemplary computing device 2130 of FIG. 21.

[0054] In step 602, the occupancy configuration (e.g., occupancy configuration β) of the current child cube may be determined. The occupancy configuration (e.g., occupancy configuration β) of the current child cuboid may be determined, for example, based on the occupancy bits of already coded cuboids adjacent to the current child cuboid. In step 604, the occupancy configuration (e.g., occupancy configuration β) may be dynamically reduced. The occupancy configuration may be dynamically reduced, for example, based on a dynamic reduction function DR nFor example, the occupancy configuration β can be dynamically reduced using the reduced occupancy configuration β′=DR n (β). In step 606, the context index may be looked up, for example, in a look-up table (LUT). For example, the encoder and / or decoder may look up the context index LUT[β'] in the LUT of the dynamic OBUF. In step 608, a context (e.g., a probability model) may be selected. For example, the context (e.g., a probability model) pointed to by the context index may be selected. In step 610, the occupancy of the current child cube may be entropy coded. For example, the occupancy bits of the current child cube may be entropy coded (e.g., arithmetic coded), for example, based on the context.

[0055] Although not shown in Figure 6, the encoder and / or decoder may update the reduction function and / or update the context index. For example, the encoder and / or decoder may update the reduction function DR based on the occupancy bits of the current child cuboid. n DR n+1 and / or update the context index LUT[β']. The method of Figure 6 may be repeated for additional or all child cuboids of a parent cuboid corresponding to a node in the occupancy tree in a scan order, such as the scan order discussed herein with respect to Figure 3.

[0056] In general, occupancy trees are lossless compression techniques. Occupancy trees can be adapted to provide lossy compression, for example, by modifying the point cloud on the encoder side (e.g., downsampling, removing points, moving points, etc.). Lossy compression performance may be weak. Lossy compression can be a useful lossless compression technique for dense point clouds.

[0057] One approach to lossy compression for point cloud geometries may be to set the maximum depth of the occupancy tree to stop at a larger volume size (e.g., an N×N×N rectangular parallelepiped (e.g., cube), where N>1) instead of reaching a minimum volume size of one voxel. The geometry of the points belonging to each occupied leaf node associated with the larger volume may then be modeled. This approach may be particularly suitable for dense, smooth point clouds that can be locally modeled by a smooth function such as a plane or a polynomial. The encoding cost may be the cost of the occupancy tree plus the cost of a local model of each occupied leaf node.

[0058] A scheme for modeling the geometry of points belonging to each occupied leaf node associated with a volume size larger than one voxel may use a set of triangles as a local model. The scheme may be called a "TriSoup" scheme. TriSoup is an abbreviation for "triangle soup" because the connections between triangles may not be part of the model. An occupied leaf node in the occupation tree corresponding to a cuboid with a volume greater than one voxel may be referred to as a TriSoup node. An edge belonging to at least one cuboid corresponding to a TriSoup node may be referred to as a TriSoup edge. A TriSoup node stores an existence flag (s) for each TriSoup edge of its corresponding occupied cuboid. k ) TriSoup edge existence flag (s k ) is a TriSoup vertex (V k ) on a TriSoup edge. At most one TriSoup vertex (V k ) can exist on the TriSoup edge. Each vertex (V k ), the TriSoup node corresponding to the occupied cuboid is the vertex along the TriSoup edge (V k ) position (p k ).

[0059] In addition to the occupancy word of the occupancy tree, the encoder may entropy encode the TriSoup vertex presence flag and the position of each TriSoup edge belonging to a TriSoup node in the occupancy tree. The decoder may similarly entropy decode the occupancy word of the occupancy tree, as well as the TriSoup vertex presence flag and the position of each TriSoup edge belonging to a TriSoup node in the occupancy tree.

[0060] FIG. 7 illustrates an example of an occupied cuboid (e.g., cube) 700. More specifically, FIG. 7 illustrates an example of an occupied cuboid (e.g., cube) 700 of size N×N×N (where N>1) corresponding to a TriSoup node in an occupancy tree. The occupied cuboid 700 may include edges (e.g., TriSoup edges 710-721). The TriSoup node corresponding to the occupied cuboid 700 stores an existence flag (s k ). For example, the existence flag for TriSoup edge 714 may indicate that TriSoup vertex V1 is on TriSoup edge 714. The existence flag for TriSoup edge 715 may indicate that TriSoup vertex V2 is on TriSoup edge 715. The existence flag for TriSoup edge 716 may indicate that TriSoup vertex V3 is on TriSoup edge 716. The existence flag for TriSoup edge 717 may indicate that TriSoup vertex V4 is on TriSoup edge 717. The existence flags for the remaining TriSoup edges may each indicate that the TriSoup vertex is not on the corresponding TriSoup edge. The TriSoup node corresponding to occupied cuboid 700 may include the location of each TriSoup vertex that is along one of its TriSoup edges 710-721. More specifically, the TriSoup node corresponding to occupied cuboid 700 may include position p1 of TriSoup vertex V1, position p2 of TriSoup vertex V2, position p3 of TriSoup vertex V3, and position p4 of TriSoup vertex V4.

[0061] 8A shows an exemplary rectangular parallelepiped (e.g., cube) 800 that corresponds to a TriSoup node. The rectangular parallelepiped 800 is a TriSoup vertex V k Within the cuboid 800, the TriSoup triangles may correspond to TriSoup nodes with a number K of TriSoup vertices V k A TriSoup triangle can be constructed from, for example, TriSoup vertices V if there are at least three (K≧3) TriSoup vertices on the TriSoup edges of the rectangular solid 800. k For example, with respect to FIG. 8A, there may be four TriSoup vertices and a TriSoup triangle may be constructed. A TriSoup triangle is constructed from TriSoup vertices V k The main direction can be determined and then the vertex V k can be ordered by rotating around this direction, and the following K TriSoup triangles can be constructed: V1V2C, V2V3C, ..., V K V1C. The main direction may be chosen from among three directions each parallel to an axis in 3D space, for example, to increase or maximize the 2D surface of the triangle if the triangle is projected along the main direction. In doing so, the main direction may be somewhat perpendicular to the local surface defined by the points of the point cloud belonging to the TriSoup node.

[0062] FIG. 8B shows an example fine-tuning for a TriSoup model. The TriSoup model can be fine-tuned by encoding the centroid residual value. The centroid residual value C res can be coded into the bitstream. res For example, use C+C instead of C as the pivot vertex of the triangle. res C+C as the pivot vertices of the triangle. res By using res may be closer to the points in the point cloud than the centroid C, lowering the reconstruction error and res The lower distortion can be achieved at the cost of a small increase in the bit rate required to encode the image.

[0063] FIG. 9 shows an example of voxelization. Voxelization may refer to the reconstruction of a decoded point cloud from a set of TriSoup triangles. Voxelization may be performed by ray tracing for each triangle individually. Voxelization may be performed by ray tracing for each triangle individually, for example, before removing overlapping points between voxelized triangles. As shown in FIG. 9, a ray 900 may be shot parallel to one of three axes in 3D space. The ray 900 may be projected along integer coordinate P start The intersection point P of a ray 900 with a TriSoup triangle 901 belonging to a rectangular parallelepiped (e.g., cube) 902 corresponding to a TriSoup node int (if any) may be rounded to obtain the decoded point. This intersection point P int can be found using, for example, the Moller-Trumbore algorithm.

[0064] Existence flag (s k ) and existence flag (s k ) can indicate the existence of a vertex, the current TriSoup edge position (p k ) can be entropy coded. k ) and position (p k ) may be individually or collectively referred to as vertex information or TriSoup vertex information. k ), and existence flag (s k ) indicates the existence of a vertex, the current TriSoup edge position (p k ) can be entropy coded, for example, based on the already coded existence flags and the positions of the TriSoup edges adjacent to the current TriSoup edge. k ) and existence flag (s k ) can indicate the existence of a vertex, the current TriSoup edge position (p k ) may additionally or alternatively be entropy coded. The existence flag of the current TriSoup edge (s k ) and position (p k) can additionally or alternatively be entropy coded, for example, based on the occupancy of the cuboids adjacent to the current TriSoup edge. Similar to the entropy coding of the occupancy bits of the occupancy tree, the neighbors (neighbor configuration β TS (also called) configuration β TS Obtain the reduced configuration β TS '=DR n (β TS ) can be dynamically reduced to the context index LUT[β TS '] may be obtained from the OBUF LUT. At least a portion of the vertex information of the current TriSoup edge may be entropy coded using the context (e.g., a probability model) pointed to by the context index.

[0065] The TriSoup vertex position (p k ) (if present) can be binarized. TriSoup vertex positions (p k ) (if present) may be binarized to entropy encode at least a portion of the vertex information of the current TriSoup edge, for example, using a binary entropy coder. b The number (e.g., quantity) of TriSoup vertices along a TriSoup edge of length N (p k ) can be set to quantify the length of a TriSoup edge. Nb The quantization interval can be divided evenly. By doing so, the TriSoup vertex position (p k ) can be individually encoded by a dynamic OBUF scheme. b Bit(p k j 、 j=1,...,N b ), and existence flags (s k ) can be represented by the bits corresponding to the neighbor configuration β TS、 OBUF reduction function DR n , and the context index is the nature of the coded bits (e.g., presence flag (sk ), the highest bit (p k 1 ), the second highest bit (p k 2 In practice, there may be several dynamic OBUF schemes, each of which depends on a specific bit of information in the vertex information (e.g., existence flag (s k ) or position bit (p k j )) is exclusive to this site.

[0066] 10A and 10B show exemplary rectangular parallelepipeds. More specifically, FIGS. 10A and 10B show twelve rectangular parallelepipeds 1000-1003, 1010-1013, and 1020-1023, whose volumes intersecting with the current TriSoup edge E are entropy encoded. The current TriSoup edge E may be an edge of the rectangular parallelepipeds 1000-1003. The start point of the current TriSoup edge E may intersect with the rectangular parallelepipeds 1010-1013. The end point of the current TriSoup edge E may intersect with the rectangular parallelepipeds 1020-1023. The adjacent configuration β of the current TriSoup edge E is determined using the occupied bits of one or more of the twelve rectangular parallelepipeds 1000-1003, 1010-1013, and 1020-1023. TS can be determined.

[0067] TriSoup edges may be oriented from start to end according to the orientation of one of the three axes in 3D space along which the edges are parallel. The overall ordering of TriSoup edges may be defined as a lexicographical order over the set (e.g., start, end). Vertex information associated with TriSoup edges may be encoded according to the TriSoup edge order. The causal neighbors of the current TriSoup edge may be obtained from already-encoded TriSoup edges that are adjacent to the current TriSoup edge.

[0068] 11A, 11B, and 11C show TriSoup edges that may be used to entropy encode the current TriSoup edge. 11A, 11B, and 11C show TriSoup edges (E' and E'') that may be used to entropy encode the current edge E. In some instances, five TriSoup edges (E' and E'') may be used to entropy encode the current edge E. The five TriSoup edges are: - an edge E' that is parallel to the current TriSoup edge E and has an end point equal to the start point of the current TriSoup edge E; - four edges E'' that are perpendicular to the current TriSoup edge E and have a start point or end point equal to the start point of the current TriSoup edge E. Depending on the direction of the current TriSoup edge E, any two (in FIG. 11C, direction z), three (in FIG. 11B, direction y), or four (in FIG. 11A, direction x) of the four perpendicular TriSoup edges may already be encoded, and their vertex information may be used to create the neighbor configuration β for the current TriSoup edge E. TS A TriSoup edge E' may already be encoded for each direction of the current TriSoup edge E, and its vertex information may be used to construct an adjacency configuration β of the current TriSoup edge E, independent of its direction. TS can be constructed.

[0069] Adjacency configuration β of the current TriSoup edge E TS can be obtained from one or more of the occupied bits of the cuboid and / or from the vertex information of adjacent already encoded TriSoup edges. For example, the adjacent configuration β of the current TriSoup edge E TS can be obtained from one or more of the 12 occupied bits of the 12 cuboids shown in Figures 10A and 10B, and from the vertex information of up to five adjacent already-encoded TriSoup edges (E' and E'') shown in Figures 11A, 11B, and 11C.

[0070] Performance can be improved by using inter-frame prediction, for example, in video compression. The bit rate required to compress inter-frames can typically be one to two orders of magnitude lower than the bit rate within a frame without inter-frame prediction, by definition. Point cloud data may behave differently because 3D geometry is coded, unlike video coding, where typically only attributes (e.g., color) are coded after projecting the 3D geometry onto a 2D plane (e.g., a camera sensor). Even if the 2D projected attributes are expected to have higher temporal correlation than the underlying 3D geometry, it can be expected that inter-frame prediction between 3D point clouds can provide improved compression capabilities over intra-frame prediction within point clouds alone. Octrees can benefit from inter-frame prediction and geometry compression gains.

[0071] Figure 12 shows an exemplary encoding method. One or more steps of Figure 12 may be performed by an encoder (e.g., encoder 114 of Figure 1). All or part of the flowchart may be implemented by a coder (e.g., encoder 114 of Figure 1 and / or decoder 120 of Figure 1), the exemplary computer system 2000 of Figure 20, and / or the exemplary computing device 2130 of Figure 21.

[0072] The general framework of inter-frame prediction of 3D point clouds may be similar to that of video compression, as shown in FIG. 12 for a coding (e.g., encoding) method. A current frame (e.g., image or point cloud) 1200 may be coded based on an already coded reference frame (e.g., image or point cloud) 1210. A motion search 1220 may be performed from the already coded reference frame 1210 toward the current frame 1200 to obtain a motion vector 1221, which may represent the flow of motion between the two frames 1210 and 1200. The motion vector may be a two-component (or 2D) vector representing the movement from a reference block of pixels to a current block of pixels, for example, in video compression. The motion vector may be a three-component (or 3D) vector representing the movement from a reference set of 3D points to a current set of 3D points, for example, in point cloud compression. The motion vector 1221 may be coded (e.g., entropy coded) into a bitstream 1250 (at step 1225, as shown in FIG. 12). The reference frame 1210 may be motion compensated (at step 1230, as shown in FIG. 12 ) to obtain a motion-compensated frame 1231. Motion compensation may involve moving pixels of the reference image according to a 2D motion vector and / or moving points of the reference point cloud according to a 3D motion vector. The obtained motion-compensated frame 1231 may be closer to the current frame 1200 than the reference frame 1210. The obtained motion-compensated frame 1231 may be closer to the current frame 1200 than the reference frame 1210, for example, in that the color difference and / or point distance between the motion-compensated frame 1231 and the current frame 1200 may be smaller than the color difference and / or point distance between the reference frame 1210 and the current frame 1200. The acquired motion-compensated frame 1231 may be closer to the current frame 1200 than the reference frame 1210, for example, in that the color difference and / or point distance between the motion-compensated frame 1231 and the current frame 1200 may be smaller, on average, than the color difference and / or point distance between the reference frame 1210 and the current frame 1200.In step 1240, inter-frame prediction may be performed to obtain an inter-residual 1241. The inter-residual 1241 may be coded (e.g., entropy coded) into a bitstream 1250 (as shown in FIG. 12 , in step 1245). The inter-residual 1241 may contain more compressible information than the current frame 1200 or a current frame that has undergone an intra-prediction process. The coding (e.g., entropy coding) 1245 may be more efficient to obtain a bitstream 1250 that is smaller in size compared to a bitstream obtained by coding the current frame 1200 that does not benefit from inter-frame prediction.

[0073] For example, in video coding, the inter-residual may be constructed as a pixel-by-pixel color difference between a current block of pixels belonging to a current frame (e.g., image) and a co-located compensated block of pixels belonging to a motion-compensated frame (e.g., image). The inter-residual may be an array of color differences, which may have a small size and therefore can be efficiently compressed.

[0074] There is no such concept for the difference between two sets of points, and the concept of residual difference may not be simply generalized to point clouds, for example, in point cloud compression. For prediction of an octree representing a point cloud, the concept of residual difference may be replaced by conditional entropy coding, and the conditional information for performing the conditional entropy coding may be constructed, for example, based on the motion-compensated point cloud. This can be extended to the framework of dynamic OBUF.

[0075] As described herein, the current occupancy bits of an octree may be coded by a coder (e.g., an entropy coder) selected by the output of a dynamic OBUF LUT of a coder index that may take the neighboring configuration β as input. The neighboring configuration β may be constructed, for example, based on already coded occupancy bits associated with volumes adjacent to the current volume. The current volume may be associated with a current node, whose occupancy may be signaled by a current occupancy bit. The construction of the neighboring configuration β may be extended using inter-frame information. An inter-predictor occupancy bit may be defined relative to the current occupancy bit as a bit representing the presence of at least one point of the motion-compensated point cloud within the current volume. A strong correlation between the current occupancy bits and the inter-predictor occupancy bits may exist, for example, when motion compensation is efficient, because the current compensated point cloud and the motion-compensated point cloud should be close to each other. Using the inter-predictor occupancy bits as bits of the neighboring configuration β may result in improved compression performance of the octree (e.g., by dividing the size of the octree bitstream by a factor of 2).

[0076] The inter-octree motion field may be created from 3D motion vectors associated with 3D prediction units (PUs). The 3D prediction units (PUs) may have volumes that may include at least a portion of one or several volumes (e.g., rectangular parallelepipeds) associated with nodes of the octree. Motion compensation may be performed for each volume based on the 3D motion vectors, for example, to obtain motion-compensated point clouds in one or more current volumes. Inter-predictor occupancy bits may be obtained, for example, based on the presence of at least one point in the motion-compensated point clouds.

[0077] For lossy coding of dense point clouds, the TriSoup method can be more efficient than octree-only approaches, which are primarily lossless methods. Even using inter-frame prediction as described herein, octree-only approaches may not be competitive with TriSoup for lossy coding of point clouds. TriSoup is an enhancement to incomplete octrees, and octree prediction may benefit the overall TriSoup scheme by reducing the bitrate of the octrees on which TriSoup is enhanced. The TriSoup method also uses TriSoup vertex information (e.g., TriSoup vertex presence flags (s k ), TriSoup vertex position (p k ), and centroid residual value (C res )) may not be coded based on any reference frame and therefore may not fully benefit from inter-frame correlation.

[0078] Examples of the present disclosure may encode TriSoup vertex information based on a motion-compensated point cloud. The motion-compensated point cloud may be determined, for example, when encoding of the underlying octree occurs, as discussed herein. The encoder and / or decoder may, for example, determine the neighbor configuration β of the current TriSoup edge E based on at least the motion-compensated point cloud. TS The encoder and / or decoder may determine one or more symbols of the neighbor configuration β TS The encoder and / or decoder may select a context (or probability model) for encoding the vertex information of the current TriSoup edge E based on, for example, the neighborhood configuration β TS The reduced configuration β represents a subset of the symbols in TS '=DR n (β TS ), the encoder and / or decoder may select a context for encoding the vertex information of the current TriSoup edge E. TS or reduced configuration β TSThe context may be selected based on an OBUF lookup table (LUT) that may map ' to an index of the context. The encoder and / or decoder may, for example, encode (e.g., entropy encode, e.g., arithmetic encode) the vertex information of the current TriSoup edge E based on the context.

[0079] The use of motion-compensated point clouds to encode TriSoup vertex information is shown in the TriSoup neighboring configuration β TS Inter-frame correlation may be added to the intra-frame correlation (e.g., adjacent edges and nodes) on which the calculation may be based. Adding inter-frame correlation to intra-frame correlation may lead to a better choice of coder (e.g., entropy coder, e.g., its associated probability or probability model), improving the compression of the TriSoup information and resulting in a reduction in the total number of bits required to represent the geometry of the compressed point cloud.

[0080] 13A-13C illustrate exemplary types of neighbor information. More specifically, FIGS. 13A-13C illustrate three example types of neighbor information that may be used by an encoder and / or decoder to compress TriSoup vertex information associated with a current edge E. FIG. 13A illustrates the occupancy of neighboring octree leaf nodes (e.g., nodes 1300-1303) for a current edge E. FIG. 13B illustrates the occupancy of neighboring octree leaf nodes (e.g., nodes 1300-1303) for a TriSoup vertex V. k ', the TriSoup vertex information (s k’ , p k’ 13C shows a motion compensated point cloud created with points 1310 that may partly belong to some as yet undefined neighborhood of the current edge E.

[0081] The encoder and / or decoder calculates the neighbor configuration β of the current TriSoup edge E. TS The encoder and / or decoder may determine one or more symbols of the neighbor configuration β of the current TriSoup edge E based on, for example, one or more of the three types of neighbor information shown in FIGS. TSThe encoder and / or decoder may determine one or more symbols of the neighbor configuration β TS The encoder and / or decoder may select a context (or probability model) for encoding the vertex information of the current TriSoup edge E based on, for example, the reduction configuration β TS '=DR n (β TS ), we can select a context for encoding the vertex information of the current TriSoup edge E. TS ' is the neighbor configuration β TS The encoder and / or decoder may, for example, represent a subset of the symbols in the neighbor configuration β TS or reduced configuration β TS The encoder and / or decoder may, for example, encode (e.g., entropy encode, e.g., arithmetic encode) the vertex information of the current TriSoup edge E based on the context.

[0082] The occupancy of one or more of the points of the motion compensated point cloud is determined, at least in part, by the neighboring configuration β TS For example, the individual occupancies of adjacent point locations for the current edge E can be used to construct the adjacent configuration β TS The number of such occupied bits can be directly used as the bits constituting the TriSoup neighbor configuration β TS Instead, we combine these occupancies into a neighboring configuration β TS At least one bit of the following may be obtained.

[0083] 14A and 14B show example point locations of a point cloud adjacent to the current edge E of a TriSoup node. The volume 1400 associated with the TriSoup node may be represented in 3D, for example, as shown in FIG. 14B. The TriSoup node may be an occupied leaf node of the underlying octree. The volume 1400 associated with the TriSoup node may be represented in 2D, for example, projected along the z-axis, as shown in FIG. 14A. The volume 1400 may be a 3D cuboid that may project as a square onto the xy plane. For example, in FIG. 14B, the volume 1400 may have a size of 4x4x4 point locations. The current edge E to be encoded may be represented as a vertical edge (along the z-axis) of the volume 1400. In FIG. 14A, potential point locations of the point cloud may be represented by gray dots 1410. The volume 1400 may encompass the 4x4x4 point locations, and the plane of the volume may lie between the point locations. In a 2D projection, there may be multiple closest point locations for the current edge E. For example, in the 2D projection of FIG. 14A , there may be four closest point locations 1420, 1430, 1440, and 1450 for the current edge E. The closest point locations in the 2D projection may correspond to multiple 3D point locations in the 3D representation (e.g., a 3D cuboid). For example, 2D point location 1420 may correspond to several 3D point locations in FIG. 14B , such as point locations 1421-1424. For example, the 3D point neighborhood for the current edge E may be created from the 16 points shown in FIG. 14B.

[0084] The set of point locations constituting the point neighbors of the current edge E of length L (not shown in FIGS. 14A and 14B ) may include a quantity of closest point locations. The set of point locations constituting the point neighbors of the current edge E of length L (not shown in FIGS. 14A and 14B ) may include a certain quantity of closest point locations that have a predetermined distance from the current edge E along each coordinate perpendicular to the current edge E. Coordinates along an axis parallel to the current edge E may be included in the edge interval. The edge interval is the projection of the edge onto the axis. For example, if the axis is perpendicular, no neighboring points are below or above the edge. For example, in FIG. 14 , the set of point locations constituting the point neighbors of the current edge E of length L may include 4*L closest point locations. Of these closest point locations, the distance from the current edge E along each coordinate perpendicular to the current edge E (or in a certain direction) may be less than or equal to half (or not more than half) of the minimum distance between point locations, and their coordinates along an axis parallel to the current edge E may be included in the edge interval. For example, a different set of point locations may constitute point neighbors of the current edge E of length L. For example, the set of point locations may include more or fewer locations, including point locations that have a greater distance from the current edge E.

[0085] FIG. 15A illustrates an exemplary occupancy of point neighbors in a motion-compensated point cloud. More specifically, FIG. 15A illustrates an exemplary occupancy of the point neighbors of FIGS. 14A and 14B in a motion-compensated point cloud. The motion-compensated point cloud may be determined, for example, based on a reference point cloud (as described herein) that better matches (or is "closer") to the points of the current edge E of FIGS. 14A and 14B. For example, in FIG. 15A, only nine of the 16 point locations in the point neighborhood may be occupied by the motion-compensated point cloud. In FIG. 15A, occupied points are illustrated, for example, as filled (gray) points, and unoccupied points are illustrated, for example, as unfilled (white) points.

[0086] FIG. 15B shows another example occupancy of point neighbors within a motion-compensated point cloud. The motion-compensated point cloud may be determined, for example, based on a reference point cloud (as described herein) that better matches (or is "closer") to the point cloud of the current edge E of FIGS. 14A and 14B. The point neighbors of FIG. 15B may include the point neighbors shown in FIGS. 14A and 14B extended with additional point locations. The quantity of point locations may include the product of the extended (e.g., stretched, extended) length of the current edge. For example, the point neighbors of FIG. 15B for the current edge E shown in FIGS. 14A and 14B of length L may include 4*(L+2) nearest point locations. Of these nearest point locations, the distance along each coordinate perpendicular to the current edge E may be less than or equal to half (or not more than half) the minimum distance between the point locations, and coordinates along an axis parallel to the current edge E may be included in the extended edge spacing. For example, the extended edge spacing is extended by adding the minimum distance between the point locations at each end (or extremity) of the edge to the length L. In Figure 15B, of the 24 point locations in the point neighborhood, only 11 locations are occupied by the motion-compensated point cloud. In Figure 15B, occupied points are shown, for example, as filled (gray) points, and unoccupied points are shown, for example, as unfilled (white) points.

[0087] The encoder and / or decoder may determine the neighbor configuration (e.g., neighbor configuration β) of the current TriSoup edge (e.g., TriSoup edge E). TS The encoder and / or decoder may determine one or more symbols of a neighbor configuration (e.g., neighbor configuration β ) of a current TriSoup edge (e.g., TriSoup edge E) based on, for example, the occupancy of one point among the point neighbors of a motion-compensated point cloud (e.g., as shown in FIGS. 15A and 15B). TS ) The encoder and / or decoder may determine one or more symbols of the neighbor configuration β TS The encoder and / or decoder may select a context (or probability model) for encoding the vertex information of the current TriSoup edge E based on, for example, the neighborhood configuration β TS The reduced configuration β represents a subset of the symbols in TS '=DR n (βTS ), the encoder and / or decoder may select a context for encoding the vertex information of the current TriSoup edge E. TS or reduced configuration β TS The encoder and / or decoder may, for example, encode (e.g., entropy encode, e.g., arithmetic encode) the vertex information of the current TriSoup edge E based on the context.

[0088] The quantity (or number) of occupied points in the point neighborhood may be determined by the neighborhood structure (e.g., neighborhood structure β ) for encoding the vertex information of the current edge (e.g., current edge E) even when, for example, the point neighborhood of the current edge is limited (e.g., the point neighborhood of the current edge E is limited to the point locations shown in FIG. 15A or FIG. 15B ). TS ) may be excessive (e.g., too large) to be used directly as a symbol of ). A reduction of the set of occupied bits (e.g., a reduction R) may be used. The reduction (e.g., a reduction R) may be obtained, for example, based on the existence of points in the motion compensated point cloud.

[0089] FIG. 16 illustrates an example of reduction. More specifically, FIG. 16 illustrates an example of reduction (e.g., reduction R) that may be used with respect to point neighbors of the current edge E. For example, FIG. 16 illustrates reduction (e.g., reduction R) used with respect to the example point neighbors shown in FIG. 15B. The reduction (e.g., reduction R) may be used / applied to / for other point neighbors. As shown in FIG. 16 and described with respect to FIG. 16, the reduction (e.g., reduction R) may regroup point locations into multiple planes (e.g., planes 1600-1605) perpendicular to the current edge E. Each plane may include multiple point locations. For the example point neighbors shown in FIG. 16, six planes are defined, each including four point locations. The plane occupancy may be defined by at least one point location belonging to the plane occupied by a point in the motion-compensated point cloud. For example, the plane occupancy may be represented by "1" (occupied) or "0" (unoccupied). For example, in FIG. 16, 24 occupied bits may be reduced to six plane occupied bits. For example, the occupancy of six planes 1600, 1601, 1602, 1603, 1604, and 1605 may be represented by "0", "0", "1", "1", "1", and "1", respectively. Contraction using planes may reduce the number of occupied bits. With reference to the example described herein with respect to FIG. 15A, for a current edge having length L, contraction using planes may reduce the number of occupied bits from 4*L to L. With reference to the example described herein with respect to FIG. 15B, for a current edge having length L, contraction using planes may reduce the number of occupied bits from 4*(L+2) to L+2.

[0090] The encoder and / or decoder may determine the neighbor configuration (e.g., neighbor configuration β) of the current TriSoup edge (e.g., TriSoup edge E). TS The encoder and / or decoder may determine one or more symbols of a neighboring configuration (e.g., neighboring configuration β ) of a current TriSoup edge (e.g., TriSoup edge E) based on, for example, the plane occupancy bits. TS ) The encoder and / or decoder may, for example, determine one or more symbols of a neighboring configuration (e.g., neighboring configuration β TS), the encoder and / or decoder may select a context (e.g., a probability model) for encoding the vertex information of the current TriSoup edge (e.g., TriSoup edge E). TS The reduced configuration β represents a subset of the symbols in TS '=DR n (β TS ), the encoder and / or decoder may select a context for encoding the vertex information of the current TriSoup edge E. TS or reduced configuration β TS The encoder and / or decoder may, for example, encode (e.g., entropy encode, e.g., arithmetic encode) the vertex information of the current TriSoup edge E based on the context.

[0091] 17A and 17B illustrate another example of a reduction (e.g., reduction R). As described with respect to FIGS. 17A and 17B, the compensated TriSoup vertices (e.g., compensated TriSoup vertices V pred,k ) can be determined for the current edge (e.g., current edge E). pred,k ) may be determined for a current edge (e.g., current edge E) based on, for example, points (e.g., point 1700) of the motion compensated point cloud that are adjacent to the current edge (e.g., current edge E). The motion compensated point cloud may be determined based on, for example, a reference point cloud (as described herein) that better matches (or is "closer") to the current frame's point cloud around the current edge (e.g., current edge E). A more compensated TriSoup vertex (e.g., compensated TriSoup vertex V pred,k’ ) may be determined for an adjacent (relative to the current edge) edge (e.g., adjacent edge E′) based on, for example, the motion compensated point cloud. The compensated TriSoup vertex may represent an intersection of the motion compensated point cloud with the edge. The compensated TriSoup vertex may be determined for, for example, the TriSoup vertex V based on which the point cloud of the current frame is encoded.k The method for determining the compensated TriSoup vertices may be the same for both encoding and decoding of the point cloud, e.g., to maintain synchronization between the encoder and decoder.

[0092] The encoder and / or decoder may determine the neighbor configuration (e.g., neighbor configuration β) of the current TriSoup edge (e.g., TriSoup edge E). TS The encoder and / or decoder may determine one or more symbols of a neighbor configuration (e.g., neighbor configuration β ) of a current TriSoup edge (e.g., TriSoup edge E) based on, for example, information associated with the compensated TriSoup vertices. TS ) information associated with the compensated TriSoup vertex may determine one or more symbols of the compensated TriSoup vertex on the edge, for example, the vertex presence flag (s pred,k The information associated with the compensated TriSoup vertex may include the vertex presence flag (s pred,k ), and if it exists, the position of the vertex (p pred,k ) The information associated with the compensated TriSoup vertices may be referred to as compensated TriSoup vertex information. The encoder and / or decoder may, for example, use a neighbor configuration (e.g., neighbor configuration β TS ), the encoder and / or decoder may select a context (e.g., a probability model) for encoding the vertex information of the current TriSoup edge (e.g., TriSoup edge E). For example, the encoder and / or decoder may select, for example, a neighbor configuration β TS The reduced configuration β represents a subset of the symbols in TS '=DR n (β TS ), the encoder and / or decoder may select a context for encoding the vertex information of the current TriSoup edge E. TS or reduced configuration β TSThe encoder and / or decoder may, for example, encode (e.g., entropy encode, e.g., arithmetic encode) the vertex information of the current TriSoup edge E based on the context.

[0093] Adjacent configuration (e.g., adjacent configuration β TS ) is, for example, the existence (e.g., existence flag s k,pred ) for the TriSoup vertices and the compensated TriSoup vertices when the original point cloud and the motion compensated point cloud of the current frame are close to each other. k and s k,pred ) can be highly correlated, which can be advantageous.

[0094] FIG. 18A illustrates an example compensated TriSoup vertex (e.g., compensated TriSoup vertex V) belonging to an edge (e.g., edge E). pred,k 18B and 18C show compensated TriSoup vertices (e.g., compensated TriSoup vertex V) belonging to an edge (e.g., edge E). pred,k ) position (e.g., position p pred,k ) quantization function. As described with respect to FIG. 18B, pred,k ) can be quantized into two values, e.g., "center" and "end". The first value (e.g., "center") corresponds to a position that is far from the start point and far from the end point of the edge (e.g., position p pred,k ) The second value (e.g., "end") may represent a position closer to the start or end of the edge (e.g., position p pred,k) For example, "near" or any synonym thereof may be understood to have a distance from the start point or end point that is less than one-sixth, one-fifth, one-quarter, or one-third of the length of the edge. As another example, "near" or any synonym thereof may be understood to have a distance from the start point or end point that is less than a value greater than zero and less than half the length of the edge. As yet another example, the existence flag (e.g., existence flag s k ) is used to select the entropy coder to encode the TS ) is the position of the compensated TriSoup vertex (e.g., position p k,pred ) into first and second values ​​(e.g., "center" and / or "tail"). This example shows how the quantized value (e.g., center or tail) can be used in conjunction with a presence flag (e.g., presence flag s k,pred ) can be an indicator of the reliability of the TriSoup vertices. k and s k,pred The correlation between the TriSoup vertex and the compensated TriSoup vertex may be stronger, for example, when the quantized value is the first value (e.g., "center"). Two presence flags (e.g., presence flag s) for the TriSoup vertex and the compensated TriSoup vertex may be stronger, for example, when the quantized value is the first value (e.g., "center"). k and s k,pred ) is the correlation between the presence flag (e.g., presence flag s) when the error of the motion-compensated point cloud relative to the original point cloud is higher than when the quantized value is a second value (e.g., "end") k,pred ) may be less likely to change its value, making it stronger.

[0095] As described with respect to FIG. 18C, pred,k ) can be quantized into two values, e.g., "bottom" and "top." The first value (e.g., "bottom") corresponds to a position closer to the start point of the edge than to the end point (e.g., position p pred,k ) The second value (e.g., "top") may represent a position closer to the end point of the edge than the start point (e.g., position p pred,k) on the current edge (e.g., edge E). k ) is used to select the entropy coder to encode the TriSoup neighbor configuration (e.g., neighbor configuration β TS ) is the position of the compensated TriSoup vertex (e.g., position p k,pred ) into first and second values ​​(e.g., “bottom” and / or “top”). This example shows a case where the quantized first value (e.g., “bottom”) is a quantized value for a TriSoup vertex located near the start of an edge (e.g., vertex V k ), while the second quantized value (e.g., “top”) may be an indicator of the TriSoup vertices located near the edge end (e.g., vertex V k ) The quantized value (e.g., "bottom" or "top") can be an index of the TriSoup vertex position (e.g., position p k ) (e.g., position bit p k 1 ) can be strongly correlated with

[0096] FIG. 19A illustrates an exemplary method for encoding vertex information for a current edge. More specifically, FIG. 19A illustrates a flowchart 1900 of exemplary method steps for encoding vertex information for a current edge. One or more steps of the exemplary flowchart 1900 may be performed by an encoder (e.g., encoder 114 as shown in FIG. 1). FIG. 19B illustrates an exemplary method for decoding vertex information for a current edge. More specifically, FIG. 19B illustrates a flowchart 1910 of exemplary method steps for decoding vertex information for a current edge. One or more steps of the exemplary flowchart 1910 may be performed by a decoder (e.g., decoder 120 as shown in FIG. 1). The flowcharts of FIGS. 19A and 19B, in whole or in part, may be implemented by a coder (e.g., encoder 114 of FIG. 1 and / or decoder 120 of FIG. 1), the exemplary computer system 2000 of FIG. 20, and / or the exemplary computing device 2130 of FIG. 21.

[0097] 19A, in step 1902, the encoder may determine one or more symbols of a neighboring configuration of the current edge. The one or more symbols of the neighboring configuration of the current edge may be determined based on, for example, a motion-compensated point cloud.

[0098] More specifically, in step 1902, the encoder may determine one or more symbols of a neighboring configuration for the current edge based on, for example, the occupancy of point locations in the motion-compensated point cloud. For example, each point location may be located a certain distance from the current edge (e.g., edge E) along each coordinate perpendicular to the current edge. The distance may be less than or equal to half (or not more than half) the minimum distance between any two point locations. For example, each point location may include a coordinate along an axis parallel to the current edge. The coordinate may be included in the edge spacing of the current edge. For example, the point locations may include 4×L point locations, where L is the length of the current edge. For example, the point locations may include 4*(L+2) point locations, where L is the length of the current edge.

[0099] More specifically, in step 1902, the encoder may determine one or more symbols of a neighbor configuration of the current edge based on, for example, the reduction used / applied to / for the occupancy of the point locations. For example, the reduction may include determining compensated TriSoup vertex information. For example, the compensated TriSoup vertex information may include a vertex presence flag. For example, the compensated TriSoup vertex information may include a vertex position. The vertex position may indicate one of two positions between the start point of the current edge and the end point of the current edge, for example, if the compensated TriSoup vertex information includes a vertex position. For example, the two positions may include a center position and an end position (e.g., as shown in FIG. 18B). As one example, the end positions may be located on either side of the center position. As another example, the two positions may include an upper position and a lower position (e.g., as shown in FIG. 18C).

[0100] In step 1904, the encoder may select / determine a context / probability model for encoding the vertex information of the current edge. The context / probability model for encoding the vertex information of the current edge may be selected based on, for example, a neighboring configuration. The vertex information of the current edge may include, for example, a vertex presence flag for the current edge. The vertex information of the current edge may include, for example, a vertex position for the current edge. The encoder may select / determine a context / probability model for encoding the vertex information of the current edge based on, for example, a lookup table that maps the neighboring configuration to a context / probability model. The encoder may select / determine a context / probability model for encoding the vertex information of the current edge based on, for example, a lookup table that maps only a subset of the symbols of the neighboring configuration to a context / probability model. For example, the number of symbols in the subset may increase based on, for example, the number of encoded edges having neighboring information including the same subset of symbols. The encoder may update the lookup table to, for example, map a subset of the symbols of the neighboring configuration to a different context / probability model based on the vertex information of the current edge.

[0101] At 1906, the encoder may encode (e.g., entropy encode) the vertex information of the current edge, for example, based on the context / probability model. For example, the vertex information of the current edge may be arithmetically encoded, for example, based on the context / probability model.

[0102] 19B, in step 1912, the decoder may determine one or more symbols of a neighboring configuration of the current edge. The one or more symbols of the neighboring configuration of the current edge may be determined based on, for example, a motion compensated point cloud.

[0103] More specifically, in step 1912, the decoder may determine one or more symbols of a neighboring configuration for the current edge based on, for example, the occupancy of point locations in the motion-compensated point cloud. For example, each point location may be located a certain distance from the current edge (e.g., edge E) along each coordinate perpendicular to the current edge. The distance may be less than or equal to half (or not more than half) the minimum distance between any two point locations. For example, each point location may include a coordinate along an axis parallel to the current edge. The coordinate may be included in the edge spacing of the current edge. For example, the point locations may include 4×L point locations, where L is the length of the current edge. For example, the point locations may include 4*(L+2) point locations, where L is the length of the current edge.

[0104] More specifically, in step 1912, the decoder may determine one or more symbols of the neighbor configuration of the current edge based on, for example, the reduction used / applied to / for the occupancy of the point locations. For example, the reduction may include determining compensated TriSoup vertex information. For example, the compensated TriSoup vertex information may include a vertex presence flag. For example, the compensated TriSoup vertex information may include a vertex position. The vertex position may indicate one of two positions between the start point of the current edge and the end point of the current edge, for example, if the compensated TriSoup vertex information includes a vertex position. For example, the two positions may include a center position and an end position (e.g., as shown in FIG. 18B). As one example, the end positions may be located on either side of the center position. As another example, the two positions may include an upper position and a lower position (e.g., as shown in FIG. 18C).

[0105] In step 1914, the decoder may select / determine a context / probability model for decoding the vertex information of the current edge. The context / probability model for decoding the vertex information of the current edge may be selected based on, for example, a neighboring configuration. The vertex information of the current edge may include, for example, a vertex presence flag for the current edge. The vertex information of the current edge may include, for example, a vertex position for the current edge. The decoder may select / determine a context / probability model for decoding the vertex information of the current edge based on, for example, a lookup table that maps the neighboring configuration to a context / probability model. The decoder may select / determine a context / probability model for decoding the vertex information of the current edge based on, for example, a lookup table that maps only a subset of the symbols of the neighboring configuration to a context / probability model. For example, the number of symbols in the subset may increase based on, for example, the number of decoded edges having neighboring information including the same subset of symbols. The decoder may update the lookup table to, for example, map a subset of the symbols of the neighboring configuration to a different context / probability model based on the vertex information of the current edge.

[0106] At 1916, the decoder may decode (e.g., entropy decode) the vertex information of the current edge, for example, based on the context / probability model. For example, the vertex information of the current edge may be arithmetically decoded, for example, based on the context / probability model.

[0107] 20 illustrates an exemplary computer system that may use any of the embodiments described herein. For example, the exemplary computer system 2000 illustrated in FIG. 20 may implement one or more of the methods described herein. For example, various devices and / or systems described herein (e.g., FIGS. 1, 2, and 3) may be implemented in the form of one or more computer systems 2000. Furthermore, each of the steps of the flowcharts illustrated in this disclosure may be implemented on one or more computer systems 2000.

[0108] Computer system 2000 may include one or more processors, such as processor 2004. Processor 2004 may be a special purpose processor, a general purpose processor, a microprocessor, and / or a digital signal processor. Processor 2004 may be connected to a communications infrastructure 2002 (e.g., a bus or network). Computer system 2000 may also include main memory 2006 (e.g., random access memory (RAM)) and / or secondary memory 2008.

[0109] Secondary memory 2008 may include a hard disk drive 2010 and / or a removable storage drive 2012 (e.g., a magnetic tape drive, an optical disk drive, and / or the like). The removable storage drive 2012 may be read from and / or written to a removable storage unit 2016. The removable storage unit 2016 may include a magnetic tape, an optical disk, and / or the like. The removable storage unit 2016 may be read by and / or written to the removable storage drive 2012. The removable storage unit 2016 may include a computer-usable storage medium having computer software and / or data stored therein.

[0110] Secondary memory 2008 may include other similar means for allowing computer programs or other instructions to be loaded into computer system 2000. Such means may include removable storage unit 2018 and / or interface 2014. Examples of such means may include program cartridges and / or cartridge interfaces (such as video game devices), removable memory chips (such as erasable programmable read-only memory (EPROM) or programmable read-only memory (PROM)), and associated sockets, thumb drives, and USB ports, and / or other removable storage units 2018 and interfaces 2014 that may allow software and / or data to be transferred from removable storage unit 2018 to computer system 2000.

[0111] Computer system 2000 may also include a communications interface 2020. Communications interface 2020 may allow software and data to be transferred between computer system 2000 and external devices. Examples of communications interface 2020 may include a modem, a network interface (e.g., an Ethernet card), a communications port, etc. The software and / or data transferred via communications interface 2020 may be in the form of signals, which may be electronic, electromagnetic, optical, and / or other signals that can be received by communications interface 2020. The signals may be provided to communications interface 2020 via communications path 2022. Communications path 2022 may transmit signals and may be implemented using wire or cable, fiber optics, a phone line, a cellular phone link, an RF link, and / or any other communications channel.

[0112] Computer program medium and / or computer-readable medium may be used to refer to tangible storage media, such as removable storage units 2016 and 2018, or a hard disk installed in hard disk drive 2010. A computer program product may be a means for providing software to computer system 2000. Computer programs (which may also be called computer control logic) may be stored in main memory 2006 and / or secondary memory 2008. Computer programs may be received via communications interface 2020. Such computer programs, when executed, may enable computer system 2000 to implement the present disclosure as discussed herein. In particular, computer programs, when executed, may enable processor 2004 to perform processes of the present disclosure, such as any of the methods described herein. Thus, such computer programs may represent controllers of computer system 2000.

[0113] 21 shows exemplary elements of a computing device that may be used to implement any of the various devices described herein, including, for example, a source device (e.g., 102), an encoder (e.g., 200), a destination device (e.g., 106), a decoder (e.g., 300), and / or any computing device described herein. The computing device 2130 may include one or more processors 2131 that may execute instructions stored on random access memory (RAM) 2133, removable media 2134 (such as a universal serial bus (USB) drive, a compact disc (CD) or digital versatile disc (DVD), or a floppy disk drive), or any other desired storage medium. Instructions may also be stored on an attached (or internal) hard drive 2135. Computing device 2130 may also include a security processor (not shown) that may execute instructions of one or more computer programs to monitor processes running on processor 2131 and any processes requesting access to any hardware and / or software components of computing device 2130 (e.g., ROM 2132, RAM 2133, removable media 2134, hard drive 2135, device controller 2137, network interface 2139, GPS 2141, Bluetooth interface 2142, WiFi interface 2143, etc.). Computing device 2130 may include one or more output devices such as a display 2136 (e.g., a screen, display device, monitor, television, etc.) and may include one or more output device controllers 2137, such as a video processor. There may also be one or more user input devices 2138, such as a remote control, keyboard, mouse, touchscreen, microphone, etc. Computing device 2130 may also include one or more network interfaces, such as network interface 2139, which may be a wired interface, a wireless interface, or a combination of the two.The network interface 2139 may provide an interface through which the computing device 2130 communicates with a network 2140 (e.g., a RAN or any other network). The network interface 2139 may include a modem (e.g., a cable modem), and the external network 2140 may include a communications link, an external network, a home network, a provider's wireless, coaxial, fiber, or hybrid fiber / coaxial distribution system (e.g., a DOCSIS network), or any other desired network. Additionally, the computing device 2130 may include a location detection device such as a global positioning system (GPS) microprocessor 2141, which may be configured to receive and process global positioning signals and, with possible assistance from external servers and antennas, determine the geographic location of the computing device 2130.

[0114] While the example of FIG. 21 may be a hardware configuration, the components shown may be implemented as software. Changes may be made, as desired, to add, remove, combine, divide, etc., components of computing device 2130. Additionally, components may be implemented using basic computing devices and components, and the same components (e.g., processor 2131, ROM storage 2132, display 2136, etc.) may be used to implement any of the other computing devices and components described herein. For example, the various components described herein may be implemented using a computing device having components such as a processor that executes computer-executable instructions stored on a computer-readable medium, as shown in FIG. 21. Some or all of the entities described herein may be software-based and coexist on a common physical platform (e.g., a requesting entity may be a separate software process and program from a dependent entity, both of which may run as software on a common computing device).

[0115] Various features are highlighted below in sets of numbered clauses or paragraphs. These features are not to be construed as limiting the invention or inventive concept, but are provided merely as highlighting some of the features described herein, without implying the importance or relevance of any particular order of such features.

[0116] Clause 1A. A method comprising determining one or more symbols of a current edge neighborhood configuration associated with a video frame or a point cloud frame based on a motion compensated point cloud.

[0117] Clause 1B. The method of Clause 1A, wherein the method further includes selecting a context for encoding (e.g., encoding or decoding) the vertex information of the current edge based on the adjacency configuration. References to Clause 1 herein may refer to one or each of Clause 1A and Clause 1B.

[0118] Clause 1C. The method of Clause 1B, wherein the method further includes encoding (e.g., encoding or decoding) vertex information of the current edge based on the context. References to Clause 1 herein may refer to one or each of Clause 1A, Clause 1B, and Clause 1C.

[0119] Clause 2. The method of any one of clauses 1A, 1B, or 1C, wherein determining further includes determining one or more symbols of the adjacent configuration based on the occupancy of point locations within the motion-compensated point cloud.

[0120] Clause 3. The method of any one of clauses 1A, 1B, 1C, or 2, wherein determining further includes determining one or more symbols of a neighboring configuration based on a reduction in the occupation of point locations within the motion-compensated point cloud.

[0121] Clause 4. The method of any one of clauses 1A-3, wherein determining further comprises determining compensated vertex information for the current edge.

[0122] Clause 5. The method of any one of clauses 1A to 4, wherein the vertex information includes at least one of a vertex presence flag or a vertex position.

[0123] Clause 6. The method of any one of clauses 1A-5, wherein selecting further comprises selecting the context based on an association between the neighboring configuration and the context.

[0124] Clause 7. The method of any one of clauses 1A-6, wherein selecting further comprises selecting the context based on an association between a subset of one or more symbols of the contiguous configuration and the context.

[0125] Clause 8. The method of any one of clauses 1A to 7, further comprising updating associations between a subset of one or more symbols of the adjacent configuration and different contexts based on vertex information of the current edge.

[0126] Clause 9. The method of any one of clauses 1A to 8, wherein the motion compensated point cloud comprises point locations, and the quantity of point locations comprises a product of the lengths of the current edges.

[0127] Clause 10. The method of any one of clauses 1A to 9, wherein the motion compensated point cloud comprises point locations, and the quantity of the point locations comprises a product of the extended lengths of the current edges.

[0128] Clause 11. The method of any one of clauses 1A to 10, wherein the vertex information includes a vertex position, the vertex position indicating one of two positions including a central position and an end position.

[0129] Clause 12. The method of any one of clauses 1A to 11, wherein the vertex information includes a vertex position, and the vertex position indicates one of two positions including an upper position and a lower position.

[0130] Clause 13. A computing device comprising one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform a method according to any one of clauses 1A to 12.

[0131] Clause 14. A system comprising: a first computing device configured to perform the method described in any one of clauses 1A to 12; and a second computing device configured to encode point cloud frames or video frames.

[0132] Clause 15. A computer-readable medium storing instructions that, when executed, cause performance of the method of any one of clauses 1A-12.

[0133] Clause 16A. A method comprising determining one or more symbols of a current edge neighborhood configuration associated with a video frame or a point cloud frame based on occupancy of point locations within a motion compensated point cloud.

[0134] Clause 16B. The method of clause 16A, wherein the method further includes selecting a context based on a neighboring configuration and an association between the neighboring configuration and the context. References herein to clause 16 may refer to one or each of clauses 16A and 16B.

[0135] Clause 16C. The method of clause 16B, wherein the method further includes encoding (e.g., encoding or decoding) the vertex information of the current edge based on the selected context. References herein to clause 16 may refer to one or each of clauses 16A, 16B, and 16C.

[0136] Clause 17. A method according to any one of clauses 16A, 16B or 16C, wherein each of the point locations is located a distance from the current edge and in a direction perpendicular to the current edge, the distance being less than or equal to half the minimum distance between any two point locations.

[0137] Clause 18. The method of any one of clauses 16A, 16B, 16C, or 17, wherein each of the point locations includes a coordinate along an axis parallel to the current side, the coordinate being within the side spacing of the current side.

[0138] Clause 19. The method of any one of clauses 16A to 18, wherein the quantity of point locations comprises a product of the lengths of the current edges.

[0139] Clause 20. The method of any one of clauses 16A to 19, wherein the quantity of point locations comprises a product of the extended lengths of the current edges.

[0140] Clause 21. The method of any one of clauses 16A to 20, wherein determining further comprises determining one or more symbols of a neighboring configuration based on a reduction in the occupation of point locations within the motion compensated point cloud.

[0141] Clause 22. The method of any one of clauses 16A to 21, further comprising determining associations between a subset of one or more symbols of the adjacent configuration and different contexts based on vertex information of the current edge.

[0142] Clause 23. A computing device comprising one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform a method according to any one of clauses 16A to 22.

[0143] Clause 24. A system comprising: a first computing device configured to perform the method described in any one of Clauses 16A to 22; and a second computing device configured to encode point cloud frames or video frames.

[0144] Clause 25. A computer-readable medium storing instructions that, when executed, cause performance of the method of any one of clauses 16A to 22.

[0145] Clause 26A. A method comprising determining one or more symbols of a current edge neighborhood configuration associated with a video frame or a point cloud frame based on a reduction in the occupation of point locations within a motion compensated point cloud.

[0146] Clause 26B. The method of Clause 26A, wherein the method further includes selecting a context for encoding (e.g., encoding or decoding) vertex information for the current edge, including a vertex presence flag and a vertex position, based on the adjacency configuration. References herein to Clause 26 may refer to one or each of Clause 26A and Clause 26B.

[0147] Clause 26C. The method of clause 26B, wherein the method further includes encoding (e.g., encoding or decoding) vertex information of the current edge based on the context. References herein to clause 26 may refer to one or each of clauses 26A, 26B, and 26C.

[0148] Clause 27. The method of any one of clauses 26A, 26B, or 26C, wherein the vertex position indicates one of two positions between the start point of the current edge and the end point of the current edge.

[0149] Clause 28. The method of any one of clauses 26A, 26B, 26C, or 27, wherein the apex position denotes one of two positions including a central position and an end position.

[0150] Clause 29. The method of any one of clauses 26A to 28, wherein the vertex position indicates one of two positions including an upper position and a lower position.

[0151] Clause 30. The method of any one of clauses 26A to 29, wherein determining, selecting, and encoding are performed by a decoder.

[0152] Clause 31. A computing device comprising one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform a method according to any one of clauses 26A to 30.

[0153] Clause 32. A system comprising: a first computing device configured to perform the method described in any one of Clauses 26A to 30; and a second computing device configured to encode point cloud frames or video frames.

[0154] Clause 33. A computer-readable medium storing instructions that, when executed, cause performance of the method of any one of clauses 26A to 30.

[0155] Clause 34A. A method comprising: determining one or more symbols of a current edge neighborhood configuration associated with a video frame or a point cloud frame based on a motion compensated point cloud.

[0156] Clause 34B. The method of clause 34A, wherein the method further includes selecting a context / probability model for encoding (e.g., encoding or decoding) the vertex information of the current edge based on the neighbor configuration. References herein to clause 34 may refer to one or each of clauses 34A and 34B.

[0157] Clause 34C. The method of clause 34B, wherein the method further includes entropy encoding (e.g., encoding or decoding) the vertex information of the current edge based on the context / probability model. References herein to clause 34 may refer to one or each of clauses 34A, 34B, and 34C.

[0158] Clause 35. The method of any one of clauses 34A, 34B, or 34C, wherein determining further comprises determining one or more symbols of a neighboring configuration based on the occupancy of point locations within the motion compensated point cloud.

[0159] Clause 36. The method of any one of clauses 34A, 34B, 34C, or 35, wherein the point locations are each located from the current edge by a distance that is less than or equal to half the minimum distance between any two point locations along each coordinate perpendicular to the current edge E.

[0160] Clause 37. The method of any one of clauses 34A to 36, wherein the point locations each include a coordinate along an axis parallel to the current edge that is included in the edge spacing of the current edge.

[0161] Clause 38. The method of any one of clauses 34A to 37, wherein the point locations include 4*L point locations, where L is the length of the current edge.

[0162] Clause 39. The method of any one of clauses 34A to 38, wherein the point locations include 4*(L+2) point locations, where L is the length of the current side.

[0163] Clause 40. The method of any one of clauses 34A to 39, wherein determining further comprises determining one or more symbols of the adjacent configuration based on a reduction applied to the occupancy of the point location.

[0164] Clause 41. The method of any one of clauses 34A to 40, wherein the reduction includes determining compensated TriSoup vertex information.

[0165] Clause 42. The method of any one of clauses 34A to 41, wherein the compensated TriSoup vertex information includes a vertex presence flag.

[0166] Clause 43. The method of any one of clauses 34A to 42, wherein the compensated TriSoup vertex information includes vertex positions.

[0167] Clause 44. The method of any one of clauses 34A to 43, wherein the vertex position indicates one of two positions between the start point of the current edge and the end point of the current edge.

[0168] Clause 45. The method of any one of clauses 34A to 44, wherein the two locations include a central location and an end location.

[0169] Clause 46. The method of any one of clauses 34A to 45, wherein the two positions include an upper position and a lower position.

[0170] Clause 47. The method of any one of clauses 34A to 46, wherein the vertex information of the current edge includes a vertex presence flag of the current edge.

[0171] Clause 48. The method of any one of clauses 34A to 47, wherein the vertex information of the current edge includes a vertex position of the current edge.

[0172] Clause 49. The method of any one of clauses 34A to 48, wherein selecting further includes selecting a context / probability model for encoding (e.g., encoding or decoding) the vertex information of the current edge based on a lookup table that maps an adjacent configuration to a context / probability model.

[0173] Clause 50. The method of any one of clauses 34A to 49, wherein selecting further includes selecting a context / probability model for encoding (e.g., encoding or decoding) the vertex information of the current edge based on a lookup table that maps only a subset of the symbols of the adjacent configuration to the context / probability model.

[0174] Clause 51. The method of any one of clauses 34A to 50, wherein the number of symbols in a subset increases based on the number of coded edges having neighbor information that includes the same subset of symbols.

[0175] Clause 52. The method of any one of clauses 34A to 51, further comprising updating a lookup table to map a subset of symbols in the adjacent configuration to different context / probability models based on the vertex information of the current edge.

[0176] A computing device may perform a method including multiple operations. The computing device may determine, based on the motion-compensated point cloud, one or more symbols of a neighboring configuration of a current edge associated with a video frame or a point cloud frame. The computing device may select a context for encoding (e.g., encoding or decoding) vertex information of the current edge based on the neighboring configuration. Based on the context, the computing device may encode (e.g., encode or decode) vertex information of the current edge. Based on the occupancy of a point location in the motion-compensated point cloud, the computing device may determine one or more symbols of a neighboring configuration. Based on a reduction in the occupancy of a point location in the motion-compensated point cloud, the computing device may determine one or more symbols of a neighboring configuration. The computing device may determine compensated vertex information of the current edge. The vertex information may include at least one of a vertex presence flag or a vertex position. The computing device may select a context based on an association between the neighboring configuration and the context. The computing device may select a context based on an association between a subset of one or more symbols of the neighboring configuration and the context. Based on the vertex information of the current edge, the computing device may update the association between a subset of one or more symbols of the neighboring configuration and a different context. The motion compensated point cloud may include point locations. The quantification of the point locations may include a product of the current edge lengths. The motion compensated point cloud may include point locations. The quantification of the point locations may include a product of the current edge extended lengths. The vertex information may include a vertex position. The vertex position may indicate one of two positions including a middle position and an end position. The vertex information may include a vertex position. The vertex position may indicate one of two positions including a top position and a bottom position. The computing device may include one or more processors and memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements.The system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode (e.g., encode or decode) the video frames, point cloud frames, or point cloud sequences. A computer-readable medium may store instructions that, when executed, cause the described methods, additional operations, and / or include additional elements.

[0177] A computing device may perform a method including multiple operations. The computing device may determine one or more symbols of a neighboring configuration of a current edge associated with a video frame or a point cloud frame based on the occupancy of point locations in the motion-compensated point cloud. The computing device may select a context based on the neighboring configurations and an association between the neighboring configurations and the context. Based on the selected context, the computing device may encode (e.g., encode or decode) vertex information of the current edge. Each of the point locations may be located a distance from the current edge and in a direction perpendicular to the current edge. The distance may be less than or equal to half the minimum distance between any two point locations. Each of the point locations may include a coordinate along an axis parallel to the current edge. The coordinate may be included in the edge spacing of the current edge. A quantity of the point location may include a product of the lengths of the current edge. A quantity of the point location may include a product of the expanded lengths of the current edge. Based on the contraction of the occupancy of the point locations in the motion-compensated point cloud, the computing device may determine one or more symbols of a neighboring configuration. Based on the vertex information of the current edge, the computing device may determine associations between a subset of one or more symbols of the adjacent configuration and different contexts. The computing device may include one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. A system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode (e.g., encode or decode) video frames, point cloud frames, or point cloud sequences. A computer-readable medium may store instructions that, when executed, cause the described methods to be performed, additional operations, and / or include additional elements.

[0178] A computing device may perform a method including multiple operations. The computing device may determine one or more symbols of a neighboring configuration of a current edge associated with a video frame or a point cloud frame based on a reduction in the occupation of point locations in a motion-compensated point cloud. The computing device may select a context for encoding (e.g., encoding or decoding) vertex information of the current edge based on the neighboring configuration. The vertex information may include a vertex presence flag and a vertex position. Based on the context, the computing device may encode (e.g., encode or decode) the vertex information of the current edge. The vertex position may indicate one of two positions between a start point of the current edge and an end point of the current edge. The vertex position may indicate one of two positions including a middle position and an end position. The vertex position may indicate one of two positions including a top position and a bottom position. The determining, selecting, and encoding (e.g., entropy encoding) may be performed by a decoder. The computing device may include one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. A system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode (e.g., encode or decode) video frames, point cloud frames, or point cloud sequences. A computer-readable medium may store instructions that, when executed, cause the described methods, additional operations, and / or include additional elements.

[0179] A computing device may perform a method including multiple operations. The computing device may determine one or more symbols of a neighboring configuration of a current edge associated with a video frame or a point cloud frame based on the motion-compensated point cloud. The computing device may select a context / probability model for coding (e.g., encoding or decoding) vertex information of the current edge based on the neighboring configuration. The computing device may code (e.g., entropy code) vertex information of the current edge based on the context / probability model. The computing device may determine one or more symbols of the neighboring configuration based on the occupancy of point locations in the motion-compensated point cloud. Each point location may be located from the current edge by a distance along each coordinate perpendicular to the current edge E that may be less than or equal to half the minimum distance between any two point locations. Each point location may include a coordinate along an axis parallel to the current edge that may be included in the edge spacing of the current edge. The point locations may include 4*L point locations, where L is the length of the current edge. The point locations may include 4*(L+2) point locations, where L is the length of the current edge. The computing device may determine one or more symbols of the adjacent configuration based on a reduction applied to the occupancy of the point locations. The reduction may include determining compensated TriSoup vertex information. The compensated TriSoup vertex information may include a vertex presence flag. The compensated TriSoup vertex information may include a vertex position. The vertex position may indicate one of two positions between the start point of the current edge and the end point of the current edge. The two positions may include a middle position and an end position. The two positions may include a top position and a bottom position. The vertex information of the current edge may include a vertex presence flag of the current edge. The vertex information of the current edge may include a vertex position of the current edge. The computing device may select a context / probability model for encoding (e.g., encoding or decoding) the vertex information of the current edge based on a lookup table that maps the adjacent configuration to the context / probability model.The computing device may select a context / probability model for encoding (e.g., encoding or decoding) the vertex information of the current edge based on a lookup table that maps only a subset of the symbols of the neighboring configuration to the context / probability model. The number of symbols in the subset may increase based on the number of encoded edges having neighboring information that includes the same subset of symbols. The computing device may update the lookup table to map a subset of the symbols of the neighboring configuration to a different context / probability model based on the vertex information of the current edge. The computing device may include one or more processors and a memory that stores instructions that, when executed by the one or more processors, cause the computing device to perform the described methods, additional operations, and / or include additional elements. A system may include a first computing device configured to perform the described methods, additional operations, and / or include additional elements, and a second computing device configured to encode (e.g., encode or decode) a video frame, a point cloud frame, or a point cloud sequence. A computer-readable medium may store instructions that, when executed, cause the described methods to be performed, additional operations, and / or include additional elements.

[0180] One or more examples herein may be described as a process, which may be depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, and / or a block diagram. While a flowchart may describe operations as a sequential process, one or more operations may be performed in parallel or simultaneously. The order of operations shown may be rearranged. A process may terminate when its operations are completed, but may have additional steps not shown in the figures. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or the main function.

[0181] The operations described herein may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, program code or code segments (e.g., computer program product) to perform the necessary tasks may be stored on a computer-readable or machine-readable medium. A processor may perform the necessary tasks. Features of the present disclosure may be implemented in hardware using, for example, hardware components such as application-specific integrated circuits (ASICs) and gate arrays. Implementation of hardware state machines to perform the functions described herein will also be apparent to those skilled in the art.

[0182] One or more features described herein may be implemented in computer-usable data and / or computer-executable instructions, such as one or more program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types when executed by a processor or data processing device within a computer. Computer-executable instructions may be stored on one or more computer-readable media, such as hard disks, optical disks, removable storage media, solid-state memory, RAM, etc. The functionality of the program modules may be combined or distributed as desired. Functionality may be implemented in whole or in part in firmware or hardware equivalents, such as integrated circuits, field programmable gate arrays (FPGAs), etc. Certain data structures may be used to more efficiently implement one or more features described herein, and such data structures are contemplated within the scope of the computer-executable instructions and computer-usable data described herein. Computer-readable media may include, but are not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or carrying instructions and / or data. Computer-readable media may include non-transitory media on which data may be stored and which do not include carrier waves and / or transitory electronic signals propagated via wireless or wired connections. Examples of non-transitory media include, but are not limited to, magnetic disks or tapes, optical storage media such as compact disks (CDs) or digital versatile disks (DVDs), flash memory, memory or memory devices. Computer-readable media may store code and / or machine-executable instructions, which may represent procedures, functions, subprograms, programs, routines, subroutines, modules, software packages, classes, or any combination of instructions, data structures, or program statements.A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.

[0183] A non-transitory tangible computer-readable medium may include instructions executable by one or more processors configured to cause the operations described herein. An article of manufacture may include a non-transitory tangible computer-readable machine-accessible medium encoded with instructions for enabling programmable hardware to cause a device (e.g., an encoder, decoder, transmitter, receiver, etc.) to perform the operations described herein. A device, or one or more devices, such as in a system, may include one or more processors, memory, interfaces, and / or the like.

[0184] Communications described herein may be determined, generated, sent, and / or received using any amount of messages, information elements, fields, parameters, values, indications, information, bits, and / or the like. While one or more examples may be described herein using any of the terms / phrases message, information element, field, parameter, value, indication, information, bit, and / or the like, those skilled in the art will understand that such communications may be implemented using any one or more of these terms, including other such terms. For example, one or more parameters, fields, and / or information elements (IEs) may include one or more information objects, values, and / or any other information. An information object may include one or more other objects. At least some (or all) parameters, fields, IEs, and / or the like may be used and may be interchangeable depending on the context. Where meanings or definitions are given, such meanings or definitions are controlling.

[0185] One or more elements of the examples described herein may be implemented as a module. A module may be an element that performs a defined function and / or has a defined interface to other elements. A module may be implemented in hardware, software in combination with hardware, firmware, wetware (e.g., hardware with biological components), or a combination thereof, all of which may be behaviorally equivalent. For example, a module may be implemented as a software routine written in a computer language configured to run on a hardware machine (e.g., C, C++, Fortran, Java, Basic, Matlab, etc.) or Simulink, Stateflow, GNU Octave, or LabVIEW MathScript. Additionally or alternatively, it may be possible to implement a module using physical hardware incorporating discrete or programmable analog, digital, and / or quantum hardware. Examples of programmable hardware include computers, microcontrollers, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and / or complex programmable logic devices (CPLDs). Computers, microcontrollers, and / or microprocessors may be programmed using languages ​​such as assembly, C, C++, etc. FPGAs, ASICs, and CPLDs are often programmed using hardware description languages ​​(HDLs) such as Verilog or Verilog Hardware Description Language (VHDL), which allow for the construction of connections between the less functional internal hardware modules of the programmable device. The techniques described above can be used in combination to achieve a functionally modular result.

[0186] One or more of the operations described herein may be conditional. For example, one or more operations may be performed if certain criteria are met, such as by a computing device, a communication device, an encoder, a decoder, a network, a combination of the above, and / or the like. Exemplary criteria may be based on one or more conditions of device configuration, traffic load, initial system setup, packet size, traffic characteristics, a combination of the above, and / or the like. Various examples may be used when one or more criteria are met. It may be possible to implement any part of the examples described herein in any order and based on any condition.

[0187] While examples are described above, features and / or steps of these examples may be combined, divided, omitted, rearranged, revised, and / or extended in any desired manner. Various changes, modifications, and improvements will readily occur to those skilled in the art. Such changes, modifications, and improvements, although not expressly described herein, are intended to be a part of this specification and are intended to be within the spirit and scope of the description herein. Accordingly, the foregoing description is by way of example only and not by way of limitation.

Claims

1. 1. A method comprising: determining one or more symbols of a current edge neighborhood associated with the video frame based on the motion compensated point cloud; selecting a context for encoding vertex information of the current edge based on the adjacency configuration; and decoding the vertex information of the current edge based on the context.

2. The determining step comprises: The method of claim 1 , further comprising determining the one or more symbols of the neighboring configuration based on occupancy of point locations within the motion compensated point cloud.

3. The determining step comprises: The method of claim 1 or 2, further comprising determining the one or more symbols of the neighboring configuration based on a reduction in occupation of point locations within the motion compensated point cloud.

4. The determining step comprises: The method of any one of claims 1 to 3, further comprising determining compensated vertex information for the current edge.

5. The vertex information is vertex existence flag, or The method of any one of claims 1 to 4, comprising at least one of the vertex positions.

6. The selecting The method of claim 1 , further comprising selecting the context based on an association between the neighboring configuration and the context.

7. The selecting The method of any one of claims 1 to 6, further comprising selecting the context based on an association between the subset of the one or more symbols of the contiguous configuration and the context.

8. 8. The method of claim 1, further comprising updating associations between the subset of the one or more symbols of the adjacent configuration and different contexts based on the vertex information of the current edge.

9. The method of any one of claims 1 to 8, wherein the motion compensated point cloud comprises point locations, and wherein the quantity of point locations comprises a product of the current edge lengths.

10. The method of any one of claims 1 to 9, wherein the motion compensated point cloud comprises point locations, and wherein the quantity of point locations comprises a product of the extended lengths of the current edges.

11. The method of any one of claims 1 to 10, wherein the vertex information comprises a vertex position, the vertex position indicating one of two positions comprising a central position and an end position.

12. The method of any one of claims 1 to 11, wherein the vertex information comprises a vertex position, the vertex position indicating one of two positions comprising a top position and a bottom position.

13. 1. A computing device comprising one or more processors and a memory storing instructions, the instructions, when executed by the one or more processors, causing the computing device to: A computing device configured to perform the method of any one of claims 1 to 12.

14. a first computing device configured to perform the method of any one of claims 1 to 12; a second computing device configured to encode the point cloud frames or the video frames.

15. When executed, A computer readable medium storing instructions causing the performance of the method of any one of claims 1 to 12.

Images