Method and apparatus for decoding geometry of point cloud and improving edge-based vertex refinement

Abstract

A method of decoding geometry of a point cloud from a bitstream is described. The point cloud is represented by a plurality of cubes. The plurality of cubes includes a plurality of occupied cubes modeled by one or more triangles. The one or more triangles have vertices on the sides of the occupied cube. The method includes decoding vertices located on edges of occupied cubes, decoding centroid residuals for each occupied cube, and refining vertices on one or more unique edges shared by two or more occupied cubes, where the centroid residuals are used to refine the vertices.

Description

Methods and apparatus for decoding point cloud geometry and improving edge-based vertex thinning. Technical Field

[0001] This invention generally relates to data compression, and more particularly to methods and apparatus for decoding the geometry (or geometric structure) of point clouds. Embodiments of the invention relate to methods and apparatus for decoding the geometry of point clouds and improving edge-based vertex thinning. Background Technology

[0002] Data compression is used in communications and computer networks to efficiently store, transmit, and reproduce information. For example, point clouds have recently gained attention as a format for representing three-dimensional (3D) data due to their versatility in representing all types of 3D objects or scenes. Therefore, many use cases can be addressed using point clouds, including: • Post-production in film; • Real-time 3D immersive presence or VR / AR (virtual reality / augmented reality) applications; • Free-viewpoint video, such as for watching sports; • Geographic Information Systems, also known as cartography; • Cultural heritage, such as storing scans of rare items in digital form; • Autonomous driving, including 3D mapping of the environment and real-time LiDAR data acquisition (LiDAR: Light Detection and Ranging = a method of measuring distance (range) by illuminating a target with a laser and measuring the reflected light with sensors).

[0003] During reconstruction, visible holes may be generated on the surface of the reconstructed point cloud, leading to lower visual quality and lower compression efficiency. Therefore, the problem to be solved is to reduce, eliminate, or avoid the holes generated on the surface of the reconstructed point cloud. Summary of the Invention

[0004] Therefore, the object of the present invention is to provide a method and apparatus for reducing or eliminating / avoiding holes generated on one or more surfaces of a reconstructed point cloud.

[0005] This invention provides a method for decoding the geometry of a point cloud from a bitstream, the point cloud being represented by a plurality of cuboid volumes, the plurality of cuboid volumes including a plurality of occupied cuboids modeled by one or more triangles having vertices on the edges of the occupied cuboids, the method comprising: decoding the vertices located on the edges of the occupied cuboids, decoding the centroid residual of each occupied cuboid, and refining the vertices on one or more unique edges shared by two or more occupied cuboids, wherein the centroid residual is used to refine the vertices.

[0006] Optionally, decoding vertices located on the edges of the occupied cube includes decoding the position of the vertices.

[0007] Optionally, the method further includes: - for example, constructing at least one triangle using decoded and refined vertices and decoded centroid residuals for an occupied cube, and - reconstructing a point cloud on the constructed triangle, for example, using ray tracing on the constructed triangle in the occupied cube.

[0008] Optionally, reconstructing the point cloud involves using ray tracing on triangles constructed within the occupied cube.

[0009] Optionally, refining the vertices on the unique edge includes using vertex shift information to determine the refined vertices along the unique edge. .

[0010] Optionally, the vertex shift information includes the corresponding shift of the vertex on the edge corresponding to the unique edge.

[0011] Optionally, the method includes obtaining vertex shift information of the unique edge for each occupied cube sharing a unique edge using the decoded centroid residual.

[0012] Optionally, obtaining vertex shift information for a unique edge includes: for each occupied cube sharing a unique edge, determining the vertex shift using the decoded centroid residual of the occupied cube. And shift the vertex corresponding to the unique edge. Mapped to a unique edge.

[0013] Optionally, vertex shifting of the mapped edges involves using the mapping between all the edges of the occupied cube and all unique edges.

[0014] Optionally, the centroid residual vector is set to ,in The representation is defined as follows: The centroid point (or centroid point) of the sphere, where Let represent the average position of the centroid obtained by averaging the coordinates of all vertices of the occupied cube, where Represents the centroid residual of the occupied cube, and determines the vertex shift. This includes determining the centroid residual vector. Displace vertices along the axis_max, which has the maximum value, and along each edge of the cube that is parallel to the axis_max. Set as .

[0015] Optionally, the vertices of edges not parallel to axis_max are shifted. It is set to zero.

[0016] Optionally, obtaining the vertex shift information of a unique edge also includes storing the vertex shifts of the edges corresponding to the unique edge. .

[0017] Optionally, a list L containing at most K elements is used to store the vertex shifts corresponding to the unique edges. Each element represents the vertex shift corresponding to the unique edge. .

[0018] Optionally, K = 4 or K > 4.

[0019] Optionally, list L contains the N vertex shifts of the N edges belonging to the N occupied cubes. .

[0020] Optionally, determine the refined vertices along the unique edge. This includes: using vertex shift information to determine the unique edge vertex shift. and using unique edge vertex shifting and the original vertex The position on the unique edge determines the refined vertex along the unique edge. .

[0021] Optional, unique edge vertex shift It uses the shifting of all vertices corresponding to the unique edge. It is determined by a weighted average.

[0022] Optionally, decoding the vertices includes: decoding the vertex information of the edges of each occupied cube from the bitstream, and constructing a mapping between the edges of all occupied cubes and all unique edges.

[0023] Optionally, decoding the centroid residual of each occupied cube includes: decoding the centroid residual and using the decoded centroid residual of each occupied cube to obtain vertex shift information for the corresponding unique edge.

[0024] Optionally, refining vertices on one or more unique edges includes: determining refined vertices along the unique edge by iterating each unique edge using vertex shift information; mapping the refined vertices along the unique edge to the edges of all occupied cubes based on the mapping between the edges of all occupied cubes and all unique edges; constructing triangles based on the vertices and the centroids within the occupied cubes; and reconstructing the point cloud using ray tracing methods on the constructed triangles.

[0025] Optionally, the coordinates of the centroid of the occupied cube are the average coordinates of all vertices of the occupied cube.

[0026] Optionally, the centroid residual is a one-dimensional 1D residual.

[0027] Optionally, the centroid residual is equal to

[0028] in Centroid residual The normal vector is obtained by averaging the coordinates of the vertices and the centroid of the occupied cube. The centroid is obtained by averaging the coordinates of the vertices of the occupied cube. : 1D signed scalar value for encoding and decoding in a bitstream.

[0029] Optionally, the normal vector Determined by the following formula: and normalization

[0030] Where × is the cross product of two vectors, also known as the vector (cross) product, and the edges yes .

[0031] Optionally, the normal vector Parallel to a specific axis.

[0032] Optionally, a specific axis is a principal axis, which is the axis along which the vertical projection of the occupied cube reveals the largest total 2D surface of the triangle.

[0033] Choose any location It is a binary value, where each bit is encoded.

[0034] Choose any location Encoded by a binary entropy codec such as an arithmetic codec or a context-adaptive binary codec like CABAC.

[0035] Choose any location Binarized to • Indicator The flag f0 indicates whether it is equal to 0. or The symbol, • indicates The flag f1 indicating whether it is equal to 1, and the remainder after encoding / decoding by the expGolomb codec. .

[0036] Choose any location This is achieved by considering all points P in the point cloud that belong to the occupied cube. k It was determined that.

[0037] Choose any location Obtained through the following formula:

[0038] Where S: has a distance d below a predetermined threshold. k Point P k The set, |S|: the number of points belonging to the set, and r k Point P k Average position relative to the centroid of the occupied cube The 1D residual, of which ,as well as .

[0039] The present invention provides a non-transitory computer-readable storage medium including instructions that, when executed by a computer, cause the computer to perform the method of the present invention.

[0040] The present invention provides an apparatus for decoding the geometry of a point cloud from a bitstream, the point cloud being represented by a plurality of cubes, the plurality of cubes including at least one occupied cube modeled by one or more triangles having vertices on the edges of the occupied cube, the apparatus comprising: a decoding module configured to decode the positions of vertices located on the edges of the occupied cube and to decode the centroid drift of each occupied cube; and a thinning module configured to thin the vertices on one or more unique edges shared by two or more occupied cubes, wherein centroid residuals are used to thin the vertices.

[0041] The technical solution provided by the embodiments of the present invention has the following beneficial effects. According to the method of the present invention, artifacts in the reconstructed point cloud are eliminated or reduced without increasing the computational complexity and overhead of the decoder or changing the bitstream.

[0042] It should be understood that the description in this section is not intended to identify key or essential features of embodiments of the invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0043] The accompanying drawings are illustrative and intended to explain the invention, and are not intended to limit the invention to the illustrated embodiments.

[0044] Figure 1 illustrates the TriSoup vertices along the edges of the volume associated with the leaf nodes of the octree representation of the point cloud geometry; Figure 2 illustrates a volume comprising two TriSoup triangles, each with its vertices on an edge of the volume; Figure 3 illustrates the process for selecting triangles to be constructed within the leaf nodes, where Figure 3(a) illustrates the volume of Figure 2 without triangles, Figure 3(b) illustrates the 2D surface of the triangles using the vertical projection of the volume, and Figure 3(c) illustrates the 2D surface of the triangles using the vertical horizontal projection of the volume; Figure 4 illustrates the volume of Figure 2 with two TriSoup triangles constructed under the assumption that the horizontal axis is the principal axis; Figure 5 illustrates the ray tracing of rendering the TriSoup triangles as a decoded point cloud; Figure 6 illustrates the TriSoup geometric representation of the volume comprising four vertices and a centroid; Figure 7 illustrates the ray tracing from the vertices and the mean point. The obtained normal vector; Figure 8 illustrates an example of a 1D residual along a line constructed from the mean point and the normal vector.

[0045] Figure 9 illustrates the visible hole generated on the common surface of two adjacent leaf nodes when the traditional vertex thinning method is applied; Figure 10 illustrates the relationship between the unique edge and the corresponding leaf edge.

[0046] Figure 11 illustrates a flowchart of a method for decoding the geometry of a point cloud from a bit stream according to an embodiment of the present invention; Figure 12 illustrates a flowchart of a method for decoding the geometry of a point cloud from a bit stream according to another embodiment of the present invention.

[0047] Figure 13 illustrates an embodiment for obtaining vertex shift information of a unique edge; Figure 14 illustrates an embodiment for determining a refined vertex along a unique edge; Figure 15 illustrates a block diagram of an apparatus / decoder for decoding geometry of a point cloud from a bitstream according to an embodiment of the present invention; Figure 16 illustrates a data stream according to an embodiment of the present invention; and Figure 17 illustrates a block diagram illustrating an electronic device configured to implement an image processing method according to an embodiment. Detailed Implementation

[0048] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely illustrative. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0049] In this invention, the terms "node," "volume," "sub-volume," and "occupied cube" are used interchangeably. It should be understood that a node is associated with a volume or sub-volume. A node is a specific point on a tree, which can be an internal node or a leaf node. A volume or sub-volume is the bounded physical space represented by a node. In some cases, the term "volume" can be used to refer to the largest bounded space defined as containing a point cloud. For the purpose of constructing a tree structure of interconnected nodes for encoding point cloud data, a volume can be recursively divided into sub-volumes. An "occupied cube" is a volume that includes one or more triangles that can be rendered.

[0050] In this invention, the term "and / or" is intended to cover all possible combinations and sub-combinations of the listed elements, including any one element, any sub-combination, or all elements listed individually, and does not necessarily exclude additional elements.

[0051] In this invention, the phrase “...or at least one of ...” is intended to cover any one or more of the listed elements, including any one of the individually listed elements, any sub-combination, or all elements, without necessarily excluding any additional elements, and without necessarily requiring all elements.

[0052] In this invention, as will be apparent from the context of the described embodiments involving encoding geometric information into / decoding geometric information from a bitstream, the term "coding" refers to "encoding" or "decoding". Similarly, the term "coder" refers to "encoder" or "decoder".

[0053] A point cloud is a collection of points in a three-dimensional coordinate system. These points are typically intended to represent the outer surface of one or more objects. Each point has a position or orientation in the three-dimensional coordinate system. This position can be represented by three coordinates (X, Y, Z), which can be a Cartesian coordinate system or any other coordinate system. These points can have other associated properties, such as color, which in some cases can also be three-component values, such as R, G, B or Y, Cb, Cr. Other associated properties may include transparency, reflectivity, normal vectors, etc., depending on the desired application of the point cloud data.

[0054] Point clouds can be static or dynamic. For example, a detailed scan or drawing of an object or terrain can be static point cloud data. LiDAR-based environmental scans for machine vision purposes can be dynamic because the point cloud at least potentially changes over time, for example, with each successive scan of the volume. Therefore, a dynamic point cloud is a temporally ordered sequence of point clouds.

[0055] As mentioned above, point cloud data can be used in a variety of applications or use cases, including preservation, such as scanning historical or cultural artifacts, mapping, machine vision (e.g., for autonomous or semi-autonomous vehicles), and virtual or augmented reality systems. Dynamic point cloud data used for applications such as machine vision can be drastically different from static point cloud data used for preservation purposes. For example, automotive vision typically involves relatively small-resolution, colorless, and highly dynamic point clouds acquired at high capture frequencies using LiDAR or similar sensors. These point clouds are not intended for human consumption or viewing but for machine object detection / classification in decision-making processes. For instance, a typical LiDAR frame contains approximately tens of thousands of points, while high-quality virtual reality applications require millions of points. It is anticipated that, over time, higher resolution data will be needed as computing speeds increase and new applications or use cases are discovered.

[0056] In other words, a point cloud is a collection of points located in 3D space, optionally with each point having an additional value attached. These additional values ​​are often referred to as point attributes. Therefore, a point cloud can be considered a combination of geometry (the 3D location of each point) and attributes. Attributes can be, for example, three-component color, material properties (such as reflectivity), and / or a two-component normal vector of the surface associated with the point. Point clouds can be captured by various types of devices, such as camera arrays, depth sensors, LiDAR (as mentioned), scanners, or they can be computer-generated (e.g., in film post-production use cases). Depending on the use case, a point cloud can have thousands to billions of points for cartographic applications.

[0057] The raw representation of a point cloud requires a very high number of bits per point, at least a dozen bits per spatial component X, Y, or Z, and optionally more bits for one or more attributes, such as three times the 10 bits for color. Therefore, the practical deployment of point cloud-based applications or use cases requires compression techniques capable of storing and distributing point clouds with a reasonable storage and transmission infrastructure. In other words, while point cloud data is useful, the lack of effective and efficient compression (i.e., encoding and decoding processes) can hinder its adoption and deployment. A particular challenge in encoding and decoding point clouds is the geometry of the point cloud, which is not present in the case of other data compression (such as audio or video), and the sparse distribution of point clouds makes efficient encoding and decoding of point locations even more challenging.

[0058] Until recently, point cloud compression (also known as PCC) had not been solved by the mass market, and there was no standardized point cloud codec available. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11, also known as the Moving Picture Experts Group or MPEG, launched a work project on point cloud compression, which resulted in two standards: • MPEG-I Part 5 (ISO / IEC 23090-5), also known as video-based point cloud compression V-PCC, • MPEG-I Part 9 (ISO / IEC 23090-9), also known as geometry-based point cloud compression G-PCC.

[0059] The first versions of the V-PCC standard and the first versions of the G-PCC standard were completed in 2020 and 2022, respectively.

[0060] The V-PCC encoding method compresses point clouds by performing multiple projections of a 3D object to obtain 2D patches that are packed into an image or, when processing moving point clouds, into video. The image or video is then compressed using an existing image / video codec, allowing the utilization of already deployed image and video solutions. By its very nature, V-PCC is only effective on dense and continuous point clouds because image / video codecs cannot compress non-smooth patches obtained from projections of sparse geometry acquired, such as from LiDAR.

[0061] The G-PCC encoding and decoding method has two schemes for compressing geometry.

[0062] • The first approach is based on an occupancy tree representation of the point cloud geometry, such as an octree, quadtree, or binary tree. In this tree-based structure, the boundary 3D volume of the point cloud is recursively divided into subvolumes. Tree nodes correspond to subvolumes. The decision to further subdivide into subvolumes can be based on the tree's resolution and / or whether a subvolume contains any points. Leaf nodes can have occupancy flags indicating whether their associated subvolume contains points. Segmentation flags can signal whether a node has child nodes, i.e., whether the current volume has been further subdivided into subvolumes. A commonly used tree structure is the octree. In this structure, the volume / subvolumes are cubes, and each division of a subvolume results in eight other subvolumes / subcubes. Another commonly used tree structure is the KD-tree, in which a volume (such as a cube) is recursively bisected by a plane orthogonal to one of its axes. An octree is a special case of a KD-tree where the volume is divided by three planes, each orthogonal to one of the three axes.

[0063] In other words, occupied nodes are segmented until a certain size is reached, and the occupied leaf nodes provide the location of points, typically at the center of these nodes. High levels of compression for dense point clouds can be achieved by using neighbor-based prediction techniques. Sparse point clouds are also addressed by directly encoding the locations of points within nodes with non-minimum sizes, stopping tree construction when only isolated points exist in a node. This stopping technique is also known as Direct Codec Mode (DCM).

[0064] • The second approach is based on a prediction tree, where each node represents the 3D location of a point, and the relationship between nodes is a spatial prediction from parent to child nodes. This method can handle only sparse point clouds and offers advantages such as lower latency and simpler decoding compared to using an occupancy tree. However, the compression performance is only slightly better than the first approach, but the encoding is complex because the optimal predictor needs to be searched intensively from a long list of potential predictors when constructing the prediction tree.

[0065] In both schemes, attribute encoding and decoding (i.e., attribute coding and attribute decoding) are performed after the entire geometry encoding and decoding, resulting in two encoding and decoding processes. Low latency can be achieved by using slices that decompose the 3D space into independently encoded and decoded sub-body segments without the need for prediction between sub-body segments. However, this can severely impact compression performance when using many slices.

[0066] One use case of particular interest is the transport of dynamic AR / VR point clouds, where "dynamic" means that the point cloud evolves over time. Furthermore, AR / VR point clouds are typically native 2D because they represent the surface of objects most of the time. Therefore, AR / VR point clouds are highly connected (also referred to as dense) because points are rarely isolated but have many neighbors. Thus, dense or solid point clouds represent continuous surfaces with a resolution that allows the volumes associated with the points (also referred to as small cubes or voxels) to contact each other without revealing any visible holes on the surface. Such point clouds, as mentioned above, are commonly used in AR / VR environments and are viewed by end users through devices such as TVs, smartphones, or headsets that include AR / VR glasses. Point clouds can be transported to a device or stored locally. Many AR / VR applications use moving point clouds, rather than static ones, that change over time. Therefore, the data volume can be large and requires compression. For example, lossless compression can achieve slightly less than 1 bit (or 1 bpp) per point when an octree representation of the geometry of the aforementioned point cloud is applied. However, this may not be sufficient for real-time transmission, which may involve millions of points per frame at frame rates of up to 50 frames per second (fps), resulting in hundreds of megabytes of data per second.

[0067] Therefore, lossy compression schemes can be used, typically requiring compression sufficient to maintain acceptable visual quality within the bandwidth available in the transmission channel, while ensuring real-time frame transmission. In many applications, bit rates as low as 0.1 bpp have made real-time transmission possible, meaning that point cloud compression can be up to ten times higher with lossy compression compared to lossless encoding / decoding schemes.

[0068] Codecs based on MPEG-I Part 5 (ISO / IEC 23090-5) or V-PCC can achieve such low bit rates by using lossy compression in video codecs that compress 2D frames obtained from the projection of point clouds onto a plane. The geometry is represented by a series of projected patches assembled into frames, where each patch is a small local depth map. However, V-PCC is not universal and is limited to narrow types of point clouds that do not exhibit complex local geometry (such as trees or hair) because the resulting projected depth maps are not smooth enough to be effectively compressed by the video codec.

[0069] On the other hand, pure 3D compression techniques can handle any type of point cloud. For example, G-PCC could provide lossy compression in the future, which would also allow for compression of dense point clouds as well as V-PCC intra-frame coding, while maintaining the versatility of G-PCC to handle any type of point cloud, such as dense point clouds, point clouds obtained through LiDAR, or point clouds representing 3D maps. To implement this G-PCC mechanism, a so-called TriSoup encoding / decoding scheme can be applied on the first layer based on an octree. Currently, the TriSoup encoding / decoding scheme is being discussed in ISO / IEC standardization working group JTC1 / SC29 / WG7. When considering the possibility of obtaining a lossy scheme from G-PCC, there are basically three methods to obtain a lossy scheme through the octree representation used by the 3-PCC codec: • Downsampling + (lossless) coding + resampling • Locally modifying voxels at the encoder end • Locally modeling the point cloud.

[0070] The first approach essentially involves downsampling the entire point cloud to a smaller resolution, losslessly encoding and decoding the downsampled point cloud, and then upsampling it after decoding. Many upsampling schemes exist, such as super-resolution, AI, or learning-based 3D post-processing, which can provide good peak signal-to-noise ratio (PSNR) results when downsampling is not too aggressive (e.g., no more than twice the normal value in each direction). However, even when the metrics show good PSNR, visual quality remains controversial and not well controlled.

[0071] The second method allows the encoder to locally adjust the point cloud, making octree encoding and decoding require a lower bit rate. To this end, the points can be slightly moved to obtain occupancy information that can be better predicted by neighboring nodes, resulting in lossless encoding of the modified octree at a lower bit rate. However, unfortunately, this method only leads to a small reduction in bit rate.

[0072] The third approach is to encode and decode the geometry using a tree (such as an octree) up to a specific resolution, for example, up to N×N×N blocks, where N can be, for example, 4, 8, or 16. This tree is then encoded and decoded using a lossless scheme such as G-PCC. The tree itself does not require a high bit rate because it does not penetrate to the deepest point and has only a small number of leaf nodes compared to the number of points in the point cloud. Then, within each N×N×N block, the point cloud is modeled using a local model. This model can be a mean plane or a set of triangles as mentioned above in the TriSoup encoding and decoding scheme, which will now be described in more detail.

[0073] The TriSoup encoding scheme uses a set of triangles to locally model the point cloud without explicitly providing connection information; hence its name, derived from the term "soup of triangles." As mentioned above, each N×N×N block defines a volume associated with a leaf node, and within each N×N×N block or volume, a set of triangles is used to locally model the point cloud, with the vertices of the triangles encoded along the edges of the volume associated with the leaf node of the tree. Figure 1 illustrates the volume 100 associated with the leaf node, which consists of twelve edges 1001 to 100... 12 The designed cube.

[0074] The portion of the point cloud enclosed by volume 100 is modeled by at least one triangle, which lies between sides 1001 and 100. 12 One of them has at least one vertex. In the example of Figure 1, five vertices 1 to 4 are shown, where vertices 1 to 4 are located on edges 1002, 1001, 1008 and 1007 respectively.

[0075] Vertices located on edges are shared among leaf nodes that share a common edge. This means that each edge belonging to at least one leaf node encodes at most one vertex, thus ensuring the continuity of the model through the leaf nodes. Encoding and decoding TriSoup vertices requires two pieces of information for each edge: • a vertex flag indicating whether a TriSoup vertex exists on the edge, also referred to as the presence flag in this paper, and • the vertex position along the edge if a vertex exists.

[0076] Therefore, the encoded / decoded data includes octree data plus TriSoup data. For example, vertex tags can be encoded / decoded by an adaptive binary arithmetic codec that uses a specific context to encode / decode vertex tags with a length of N=2. s The position of the vertex on the edge is determined by pushing s bits into the bit stream, i.e., by bypassing / non-entropy encoding and decoding s bits to encode and decode with unit precision.

[0077] Figure 2 illustrates a volume 100 associated with a leaf node, comprising two TriSoup triangles 102 and 104, whose respective vertices 1 to 4 lie on the edges (see Figure 1) 1002, 1001, 1008, and 1004 of volume 100. Triangle 102 includes vertices 1, 2, and 3, while triangle 104 includes vertices 1, 3, and 4. Therefore, a triangle can be constructed if at least three vertices exist on the edges of volume 100. Naturally, any other combination of triangles besides the triangles shown in Figure 2 is possible within volume 100 associated with a leaf node. Furthermore, not all vertices of one or more triangles within volume 100 necessarily lie on the edges of volume 100; instead, one or two vertices of a triangle can be located anywhere within volume 100.

[0078] The triangle to be constructed inside volume 100 is based on the following three-step process, including: 1. Determining the dominant direction along one of the three axes.

[0079] 2. Sort the TriSoup vertices according to the dominant direction.

[0080] 3. Construct triangles based on an ordered list of vertices.

[0081] Figure 3 illustrates the process for selecting the triangle to be constructed inside the body 100 associated with the leaf node in Figure 2. Figure 3(a) again illustrates the body 100 of Figure 2, but without the triangle. Figures 3(b) and 3(c) illustrate the process on two axes, namely the vertical axis or z-axis (Figure 3(b)) and the horizontal axis or x-axis (Figure 3(c)).

[0082] As shown in Figure 3(c), the first test along the vertical axis is performed by a vertical projection or cube 100 on the 2D plane and the TriSoup vertices, i.e., testing from the top. Then, the vertices are ordered clockwise relative to the center of projection node 114, which is a square in the illustrated example. As illustrated in Figure 3(c), triangles are constructed according to fixed rules based on the ordered vertices. In the example of Figure 3, four vertices are involved, and triangles 102 and 104 are systematically constructed to include vertices 1, 2, and 3 of the first triangle and vertices 1, 3, and 4 of the second triangle. With only three vertices, the only possible triangle is the one including vertices 1, 2, and 3, and with five vertices, fixed rules can be used to construct triangles including vertices (1, 2, 3), (1, 3, 5), and (4, 5, 1), etc. This can be repeated up to 12 times.

[0083] When viewed from the left side of Figure 3(a), a second test along the horizontal axis is performed by horizontally projecting cube 100 and TriSoup vertices onto the 2D plane, producing projection 116 as shown in Figure 3(b). When the vertices are ordered clockwise relative to the center of projection node 100, triangles 102 and 104 include vertices 1, 2, and 3 of the first triangle and vertices 1, 3, and 4 of the second triangle, as shown in Figure 3(b).

[0084] As can be seen from Figure 3, the vertical projection (Figure 3(c)) shows the largest total 2D surface of the triangle, allowing the principal axis to be chosen as either the vertical axis or the z-axis. The TriSoup triangles to be constructed are obtained according to the order of the vertical projections shown in Figure 3(c), which in turn produces the triangles inside the body depicted in Figure 2. It should be noted that when the horizontal axis is considered as the principal axis, this results in different constructions of the triangles inside body 100 as depicted in Figure 4, where triangles 102 and 104 are constructed according to the vertex order shown in Figure 3(b) when the principal axis is assumed to be the horizontal axis.

[0085] By maximizing the projection surface and appropriately selecting the principal axis, continuous reconstruction of hole-free point clouds can be achieved.

[0086] Rendering of TriSoup triangles is performed via ray tracing, and the set of all points rendered by ray tracing forms the decoded point cloud. Figure 5 illustrates the ray tracing used to render TriSoup triangle 102 of Figure 2, which includes vertices 1, 2, and 3. Rays (ray 118 in Figure 5) are emitted along a direction parallel to the axis (z-axis in Figure 5). The origin of the ray is a point with integer, voxelized coordinates of precision corresponding to the sampling position required for rendering. The intersection 120 of ray 118 and triangle 102 is then voxelized, i.e., rounded to the nearest point at the desired sampling position, and added to the list of rendered points. After applying the TriSoup encoding / decoding scheme to all leaf nodes, i.e., after constructing the triangle and obtaining the intersection via ray tracing, copies of identical points in the list of all rendered points are discarded (i.e., only one voxel is retained among all voxels sharing the same 3D position), thus obtaining a unique set of decoded points.

[0087] Furthermore, as described, for example, in PCT / CN2023 / 077400, a centroid vertex or centroid point of each volume or leaf node 100 can be encoded or decoded to characterize the surface curvature within each volume 100. Figure 6 illustrates the TriSoup geometric representation of a volume 100 including four vertices V1 to V4. The coordinates of the centroid point C are the average coordinates of all (ordered) vertices Vi. By rotating around the centroid point C, the coordinates of the ordered vertices (V1, V2, ..., V4) are obtained. M Construct triangles. Construct the following M triangles: • V1V2C • V2V3C • …• V M-1 V M C• V M The V1C construction maintains the natural symmetry of the model and provides additional degrees of freedom to improve the model's accuracy, namely the position of the centroid C.

[0088] The position of the centroid C can be improved by using the remaining positions in the encoded / decoded bitstream, making the position of centroid C closer to the original point in the point cloud. The centroid can be defined as... ,in This represents the average position of the centroid obtained by averaging the coordinates of all vertices of the occupied cube. This represents the centroid residual of the occupied cube.

[0089] The encoded centroid residual can be a 3D residual. However, it has been observed that 3D residuals are rarely advantageous because they require encoding and decoding many bits, and these bits are not fully compensated for by the model's better accuracy. Therefore, encoding and decoding of 1D residuals is recommended. Figure 7 illustrates the encoding and decoding from ordered vertices Vi and the mean point. The obtained normal vector The centroid residual can be equal to

[0090] in It is a 1D signed scalar value encoded and decoded in the bitstream. Figure 8 illustrates the values ​​along the mean point. and normal vector 1D residual of the constructed line Example. Normal vector. It can be determined by the following formula: and normalization

[0091] Where × is the cross product of two vectors, also known as the vector (cross) product, and the edges yes .

[0092] In the variant, to simplify its calculation, vectors It can be taken as parallel to the axis. Vector It can be taken as parallel to the principal axis, which is a good approximation of the vector calculated above.

[0093] value It can be determined by the encoder, encoded into the bitstream, and obtained by the decoder by decoding the bitstream. Value It can be binarized, and each bit can be encoded using a binary entropy codec, such as an arithmetic codec or a context-adaptive binary codec like CABAC. Value It can be binarized into a • indicator The flag f0 indicates whether it is equal to 0. or The symbol, • indicates The flag f1 indicating whether it is equal to 1, and the remainder after encoding / decoding by the expGolomb codec. .

[0094] value The encoder can be configured by considering all points P in the point cloud belonging to the current leaf node. k To determine this. For each point P k Its relationship with lines distance d k The following formula is derived .

[0095] At this distance d k If the value is below the threshold th (e.g., th = 2), then point P... k Used to calculate values Point P k Relative to the mean point 1D residual r kObtained through scalar product (also known as inner product or dot product) .

[0096] Therefore, value Obtained from the following formula

[0097] Where S is the distance d that makes it possible. k Point P below the threshold th k Let S be a set of points, and |S| be the number of points belonging to that set.

[0098] As described, for example, in PCT / CN2023 / 089073, the triangle modeling in the leaf nodes can be refined to make the reconstructed surface closer to the original surface at the decoder side. After obtaining the decoded vertices and the decoded centroid C at the decoder side, the vertex positions can be directed toward the vectors in the leaf nodes of the convex or concave regions. The orientation is adjusted. On the decoder side, after decoding the vertices of each leaf node from the bitstream, triangles are constructed for each leaf node using the ray tracing method described above to obtain the reconstructed point cloud. More specifically, for each leaf node, the following is performed: if the processed leaf node has more than 3 vertices: The centroid C is determined to construct the modeling surface consisting of triangles formed by the vertices and the centroid C. More specifically, the mean point of the vertices (V0, ..., Vi) in the leaf nodes is determined. unit vector of centroid residual Decoding the magnitude of the centroid residual from the bitstream. The center of mass C is formed by... Obtain, among which From The vector to C, also known as .

[0099] Determine the prominence of the modeled surface in the leaf node; for example, determine the centroid residual C. res Is it large relative to the size of the leaf node?

[0100] If the centroid residual C res The relatively large size of the leaf nodes makes the surface constructed based on the current vertices V0, ..., Vi and the centroid C more likely to have sharp protrusion artifacts in the generated modeled surface. Therefore, among all vertices V0, ..., Vi in the leaf nodes, the vector along all three axes... The vertices on the edges of the axes with the maximum value are refined within the boundaries of the edges in the leaf nodes, along the edges to which they belong, and in a direction that makes the reconstructed surface more natural.

[0101] Otherwise, all vertices V0, ..., Vi in the leaf nodes will not be refined.

[0102] Then, all vertices in the leaf node (including the refined vertex V' and other unrefined vertices V) are used to construct triangles that are combined with the centroid C, and the ray tracing method described above is applied to each triangle to obtain the reconstructed points in the leaf node.

[0103] If the processed leaf node has 3 vertices: There is no mean point in the processed leaf nodes. With the centroid C, only one triangle V1V2V3 can be constructed. Applying the above ray tracing method to the triangle, we obtain the reconstructed points in the leaf nodes.

[0104] If a leaf node has fewer than 3 vertices: Since no triangles were constructed for the leaf nodes, no reconstructed points were generated for the leaf nodes.

[0105] The aforementioned technique for adjusting vertex positions refines the vertices on the edges of leaf nodes sequentially, leaf nodes one by one. The refined vertices in the leaf nodes are then used to reconstruct triangles to obtain the reconstructed point cloud. However, this refinement is based solely on the centroid drift information of the current leaf node. Therefore, in the following cases, vertices on edges belonging to two adjacent leaf nodes may drift along... Different directional shifts by different distances: Case (a): The direction of centroid drift in two adjacent leaf nodes Conversely, or case (b): even if direction Similarly, the centroid drift values ​​of two adjacent leaf nodes also differ significantly.

[0106] This will result in visible holes appearing on the surface of the reconstructed point cloud. Figure 9 illustrates such visible holes generated on the common face 150 of two adjacent leaf nodes. Points 152a and 152b on the edge represent unrefined vertices. and The vertex shift vector (including direction and value) represents the shift of points 152a and 152b to the positions indicated by points 152a1, 152b1 and 152a2, 152b2. The dashed area 154 represents the hole or gap caused by the vertex shift. In Figure 9(a), due to the vertex shift vector... and The two adjacent leaf nodes are in opposite directions, thus creating a hole, which corresponds to case (a); in Figure 9(b), although the vertex shift vector and In the same direction, but with large differences in their values, this will also result in holes, which corresponds to case (b).

[0107] These visible holes can lead to lower visual quality and lower compression efficiency. Therefore, the problem to be solved is to reduce or eliminate or avoid the holes generated by the vertex thinning methods described above.

[0108] TriSoup data is provided when a point cloud is modeled by applying a TriSoup encoding / decoding scheme using a set of triangles for each leaf node or volume as described above. TriSoup data includes, for example, information about the vertices of the corresponding triangles of the volume, also referred to as the occupied cube or leaf node. As outlined above, vertex thinning methods can be applied at the decoder side; however, applying such vertex thinning methods can result in visible holes, and this invention is based on the finding that the reason conventional vertex thinning methods generate unwanted visible holes is that the vertices on the edges of leaf nodes are thinned leaf node by leaf node. Each leaf node has its own 12 edges, so the vertex thinning method in one leaf node is executed independently of the vertex thinning method in another leaf node, even if two leaf nodes share the same edge. For example, since an edge can be shared by up to four leaf nodes, a vertex along one edge of a leaf node can also belong to three other leaf nodes. Therefore, if the vertex is refined when processing the current leaf node, the refined vertex will be different from the unrefined vertex on the same edge belonging to the other three leaf nodes, thus creating a hole due to mismatched vertices on the same edge shared by more than one leaf node.

[0109] Embodiments of the present invention avoid unwanted visible holes by introducing a unique edge-based vertex thinning method for TriSoup encoding and decoding. More specifically, according to an embodiment, if any vertex exists on a unique edge, all unique edges Ue (i.e., edges shared by more than one leaf node) of all leaf nodes are iterated to determine their respective thinned vertices. Since a unique edge can be shared among more than two leaf nodes, for example, it can be shared among up to four leaf nodes, the thinned vertices along the unique edge are no longer determined solely based on the centroid residual information of the currently processed leaf node, but, according to an embodiment, are determined based on the centroid residual information of all leaf nodes intersecting with that unique edge.

[0110] A unique edge Ue can be considered as a skeleton edge that constructs all leaf nodes without using duplicate edges. The relationship between a unique edge Ue and the edges Le of a leaf node is that a unique edge Ue can contain N leaf edges Le, where 1 ≤ N ≤ 4. Figure 10 illustrates the relationship between a unique edge Ue and its corresponding leaf edges Le. For example, as shown in Figure 10(a), when a unique edge Ue is a shared edge between two leaf nodes or is shared by two leaf nodes, the unique edge Ue can contain two leaf edges Le. As shown in Figure 10(b), when four leaf nodes share a unique edge Ue, the unique edge Ue can contain at most four leaf edges Le, and as shown in Figure 10(c), when no two adjacent leaf nodes intersect on the unique edge Ue, the unique edge Ue can also contain only one leaf edge Le.

[0111] Figure 11 illustrates a flowchart of a method for decoding the geometry of a point cloud from a bitstream according to an embodiment of the present invention. The point cloud is represented by a plurality of cubes. The plurality of cubes includes a plurality of occupied cubes modeled by one or more triangles. One or more triangles have vertices on the edges of the occupied cubes. According to an embodiment, as shown in Figure 11, the method includes the following steps: S100: Decoding the vertices or the positions of vertices located on the edges of the occupied cubes.

[0112] S102: Decode the centroid residual of each occupied cube.

[0113] S104: Refine vertices on one or more unique edges shared by two or more occupied cubes. For example, vertices can be refined using the centroid residual decoded for occupied cubes sharing a unique edge.

[0114] Embodiments of the invention will now be described in more detail.

[0115] As described above, by modifying the vertex thinning method so that it is no longer based solely on the centroid residual information of the currently processed leaf node, but rather, according to an embodiment, based on the centroid residual information of all leaf nodes intersecting the unique edge, the refined vertices along the unique edge are determined. This invention addresses the problem associated with artifacts or holes generated in the reconstructed point cloud.

[0116] Figure 12 illustrates a flowchart of a method for decoding the geometry of a point cloud from a bitstream according to another embodiment of the present invention. According to an embodiment, as depicted in Figure 12, the vertex thinning method at the decoder side includes the following steps: S200: Decoding the vertex information of the edges of each leaf node from the bitstream. During this process, a mapping relationship is constructed between the leaf edges Le of all leaf nodes and all unique edges Ue.

[0117] S202: Decode the centroid residual Cres and use the decoded centroid residual Cres of each leaf node to obtain the vertex shift information of the corresponding unique edge Ue.

[0118] S204: Determine the refined vertices along the unique edges by iterating through each unique edge Ue using the obtained vertex shift information. .

[0119] S206: Based on the mapping relationship between the leaf edges Le of all leaf nodes and all unique edges Ue, refine the vertices along the unique edges Ue. Leaf edges Le that are mapped to all leaf nodes.

[0120] S208: Optionally, decode the opposite vertex.

[0121] S210: Construct triangles based on the determined vertices and centroids within the leaf nodes.

[0122] S212: Reconstruct the point cloud using ray tracing on the constructed triangle.

[0123] Now, referring to FIG13, an embodiment for obtaining vertex shift information of the corresponding unique edge Ue (see S202) is described. According to the embodiment, for each leaf node, after decoding the centroid residual at the decoder side, the vertex shift information of the corresponding unique edge Ue is obtained using the decoded centroid residual Cres of the leaf node. According to the embodiment, as depicted in FIG13, obtaining the vertex shift information of the corresponding unique edge for each leaf node includes the following steps: S300: Determining the vertex shift of the leaf edge Le of the leaf node based on the decoded centroid residual. For example, if the centroid residual vector is set to... Then determine the vector The vertex of each leaf edge Le, parallel to axis_max, is shifted along the axis with the maximum value and within the leaf node. Set by the following formula .

[0124] For a leaf edge Le that is not parallel to axis_max within a leaf node, the vertex is shifted. Set to zero. According to other embodiments, for leaf edges Le within a leaf node that are not parallel to the axis_max, no vertex shift is generated or set. .

[0125] S302: Based on the mapping relationship between the leaf edges Le of all leaf nodes and all unique edges Ue, shift the vertices corresponding to the leaf edges Le within the leaf nodes. Mapping to the corresponding unique edge Ue, thus obtaining the vertex shift information of the corresponding unique edge within the leaf node. For example, if the leaf edge Le of the current leaf node is mapped to the i-th unique edge Ue, and the vertex shift of leaf edge Le is... Then the vertex is shifted. The list L is stored or used to determine the vertex shift information of the i-th unique edge Ue. According to an embodiment, a list L is used for each unique edge Ue to determine vertex shift information. This list can contain up to K elements, each element representing the vertex shift of a leaf edge Le. The leaf edge Le has the same start position and the same end position as the unique edge Ue. For example, K = 4 or K > 4.

[0126] After iterating through each leaf node and performing steps S300 and S302 as described above, the vertex shift information for each unique edge is obtained, such as list L.

[0127] Now, referring to Figure 14, we describe the method used to determine the refined vertices along the unique edge. An embodiment is described in FIG14. After obtaining the vertex shift information (e.g., list L) for each unique edge Ue, the following steps are performed for each unique edge: S400: Determine the vertex shift of the unique edge Ue using the stored vertex shift information. According to an embodiment, list L may contain N vertex shifts of N leaf edges Le belonging to N leaf nodes. The vertex shift of the unique edge Ue is determined based on the weighted average of all elements in list L. According to the embodiment, the weighted average is as follows: The weight The following conditions must be met: (1) (2) .

[0128] S402: If the original vertex exists, then shift based on the determined vertex. and the original vertex Determine the refined vertices along each unique edge Ue by locating the unique edge Ue. Refined vertices It can be described by the following formula .

[0129] If the original vertex V does not exist along a particular unique edge, the vertex thinning method will not be applied to that particular unique edge.

[0130] So far, the concept of the invention has been described with reference to embodiments relating to methods for decoding the geometry of a point cloud from a bitstream. According to other embodiments, the invention also provides apparatus for decoding the geometry of a point cloud from a bitstream, such as a decoder operating according to the embodiments described above.

[0131] Figure 15 illustrates a block diagram of an apparatus or decoder 200 for decoding the geometry of a point cloud from a bitstream. The point cloud is represented by a plurality of cubes. The plurality of cubes includes at least one occupied cube modeled by one or more triangles. The one or more triangles have vertices on the edges of the occupied cube. According to an embodiment, the apparatus 200 includes the following modules: a decoding module 202 configured to decode the positions of vertices located on the edges of the occupied cubes and to decode the centroid drift of each occupied cube; and a thinning module 204 configured to thin the vertices on one or more unique edges shared by two or more occupied cubes. For example, the centroid residual decoded for occupied cubes sharing unique edges can be used to thin the vertices.

[0132] Figure 16 illustrates a data stream or bitstream 300 according to an embodiment of the present invention, wherein the geometry of a point cloud is encoded. The point cloud is represented by a plurality of cubes. The plurality of cubes includes at least one occupied cube modeled by one or more triangles. The one or more triangles have vertices on the edges of the occupied cube. For example, the data stream or bitstream 300 may be provided by an encoder 400 for encoding the geometry of the point cloud into the bitstream 300. The data stream 300 is transmitted to a decoder 200 via a wired or wireless transmission medium 302 (such as a cable or radio link), and the decoder 200 decodes the geometry of the point cloud from the data stream 300. Thus, according to an embodiment, as depicted in Figure 16, the data stream 300 includes the encoded positions 304 of the vertices located on the edges of the occupied cubes and the encoded centroid residuals for each occupied cube.

[0133] The present invention also provides electronic devices, computer-readable storage media, and computer program products in the embodiments.

[0134] Although certain aspects of the disclosed concepts have been described in the context of the apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of the corresponding apparatus.

[0135] Figure 17 is a block diagram illustrating an electronic device 600 according to an embodiment of the present invention.

[0136] Electronic devices are intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative examples and are not intended to limit the embodiments of the invention described herein and / or claimed.

[0137] Referring to Figure 17, device 600 includes a computing unit 601 for performing various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 602 or loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data for storing the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0138] The components in device 600 are connected to I / O interface 605, including: input unit 606, such as a keyboard and mouse; output unit 607, such as various types of displays and speakers; storage unit 608, such as a disk and optical disk; and communication unit 609, such as a network card, modem, or wireless transceiver. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0139] The computing unit 601 may be formed of various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the image processing method described above may be performed. In some embodiments, the computing unit 601 may be configured to perform the image processing method in any other suitable manner (e.g., by means of firmware).

[0140] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuits, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, at least one input device, and at least one output device.

[0141] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions and / or operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0142] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0143] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD)) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0144] The systems and technologies described herein can be implemented on computing systems that include back-end components (e.g., as data servers), or middleware components (e.g., application servers), or front-end components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or computing systems that include such back-end components, middleware components, front-end components, or any combination thereof. The components of the system can be interconnected via any form of digital data communication medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0145] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in a cloud computing service system and solves the management difficulties and weak business scalability of traditional physical hosts and Virtual Private Servers (VPS). Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0146] It should be understood that the steps can be reordered, added, or deleted using the various forms of processes shown above. For example, the steps described in this invention can be executed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0147] The specific embodiments described above do not limit the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for decoding the geometry of a point cloud from a bitstream, the point cloud being represented by a plurality of cubes, the plurality of cubes including a plurality of occupied cubes modeled by one or more triangles, the one or more triangles having vertices on the edges of the occupied cubes, the method comprising: The vertices located on the edges of the occupied cubes are decoded, the centroid residual of each occupied cube is decoded, and the vertices on one or more unique edges shared by two or more of the occupied cubes are refined, wherein the centroid residual is used to refine the vertices.

2. The method of claim 1, wherein decoding the vertex located on the edge of the occupied cube includes decoding the position of the vertex.

3. The method according to claim 1 or 2, further comprising: - Construct at least one triangle using the decoded and refined vertices and the decoded centroid residuals, and - Reconstruct the point cloud on the constructed triangle.

4. The method of claim 3, wherein reconstructing the point cloud comprises using ray tracing on the constructed triangles in the occupied cube.

5. The method according to any one of the preceding claims, wherein refining the vertices on the unique edge includes using vertex shift information to determine the refined vertices along the unique edge. 。 6. The method of claim 5, wherein the vertex shift information includes the corresponding shift of the vertex on the edge corresponding to the unique edge.

7. The method according to claim 5 or 6, comprising: For each occupied cube sharing the unique edge, the vertex shift information is obtained using the decoded centroid residual.

8. The method according to claim 7, wherein obtaining the vertex shift information of the unique edge comprises: For each occupied cube sharing the unique edge, the decoded centroid residual is used to determine the vertex shift. And using the mapping between all occupied cube edges and all unique edges, shift the vertex corresponding to the unique edge. Mapped to the unique edge.

9. The method of claim 5, wherein the vertex of the mapped edge is shifted. This includes the mapping between all occupied cube edges and all unique edges.

10. The method according to claim 8 or 9, wherein the centroid residual vector is set as follows: ,in The representation is defined as follows: The center of mass, of which This represents the average position of the centroid obtained by averaging the coordinates of all vertices of the occupied cube, where... The centroid residual of the occupied cube, and the vertex shift are determined. include: Determine the centroid residual vector Displace the vertices within the occupied cube that are parallel to the axis_max along the axis_max where the axis_max has the maximum value. Set as 。 11. The method of claim 10, wherein the vertex of the edge not parallel to the axis_max is shifted. It is set to zero.

12. The method according to any one of claims 6 to 11, wherein obtaining the vertex shift information of the unique edge further comprises storing the vertex shift of the edge corresponding to the unique edge. 。 13. The method of claim 12, wherein a list containing at most K elements is used. To store the vertex shift of the edge corresponding to the unique edge. Each element represents the vertex shift corresponding to the edge of the unique edge. 。 14. The method of claim 13, wherein K = 4 or K > 4.

15. The method of claim 13 or 14, wherein the list L comprises N vertex shifts belonging to the N edges of the N occupied cubes. 。 16. The method according to any one of claims 5 to 15, wherein the refined vertex along the unique edge is determined. include: The vertex shift information is used to determine the shift of the unique edge vertex. and using the unique edge vertex shift and the original vertex The position on the unique edge determines the refined vertex along the unique edge. 。 17. The method of claim 16, wherein the unique edge vertex shift It is to shift all vertices of the edge corresponding to the unique edge. It is determined by a weighted average.

18. The method of claim 1, wherein decoding the vertex comprises: Decode the vertex information of the edges of each occupied cube from the bit stream, and construct a mapping between the edges of all occupied cubes and all unique edges.

19. The method of claim 1, wherein decoding the centroid residual for each occupied cube comprises: The centroid residual is decoded, and the decoded centroid residual of each occupied cube is used to obtain the vertex shift information of the corresponding unique edge.

20. The method of claim 1, wherein refining the vertices on the one or more unique edges comprises: The point cloud is determined by iterating each unique edge using the vertex shift information, mapping the refined vertex along the unique edge to the edges of all occupied cubes based on the mapping relationship between the edges of all occupied cubes and all unique edges, constructing triangles based on the vertices and the centroids within the occupied cubes, and reconstructing the point cloud using ray tracing on the constructed triangles.

21. The method of claim 1, wherein the coordinates of the centroid of the occupied cube are the average coordinates of all vertices of the occupied cube.

22. The method according to any one of the preceding claims, wherein the centroid residual is a one-dimensional 1D residual.

23. The method according to any one of the preceding claims, wherein the centroid residual is equal to in Centroid residual : Normal vector, which is obtained from the average position of the vertex and the centroid of the occupied cube, the centroid being obtained by averaging the coordinates of the vertices of the occupied cube, and : The 1D signed scalar value encoded and decoded in the bit stream.

24. The method of claim 23, wherein the normal vector Determined by the following formula: and normalization Where × is the cross product between two vectors, also known as the vector (cross) product, and the edge yes 。 25. The method of claim 32, wherein the normal vector Parallel to a specific axis.

26. The method of claim 25, wherein the particular axis is a principal axis, which is the axis along which the vertical projection of the occupied cube exhibits the largest total 2D surface of the triangle.

27. The method according to any one of claims 23 to 26, wherein It is a binary value, where each bit is encoded.

28. The method of claim 27, wherein Encoded by a binary entropy codec such as an arithmetic codec or a context-adaptive binary codec like CABAC.

29. The method according to claim 27 or 28, wherein Binarized to • Indicator The flag f0 indicates whether it is equal to 0. or The symbol, • indicates The flag f1 indicating whether it is equal to 1, and the remainder after encoding / decoding by the expGolomb codec. 。 30. The method according to any one of claims 23 to 29, wherein This is achieved by considering all points P of the point cloud that belong to the occupied cube. k It was determined that.

31. The method of claim 30, wherein Obtained through the following formula: Where S: has a distance d below a predetermined threshold. k Point P k The set, |S|: the number of points belonging to the set, and r k Point P k The average position relative to the centroid of the occupied cube The 1D residual, of which ,as well as 。 32. A non-transitory computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of the preceding claims.

33. An apparatus for decoding geometry of a point cloud from a bitstream, the point cloud being represented by a plurality of cubes, the plurality of cubes including at least one occupied cube modeled by one or more triangles having vertices on the edges of the occupied cube, the apparatus comprising: A decoding module is configured to decode vertices located on the edges of the occupied cubes and to decode the centroid drift of each occupied cube, and a refinement module is configured to refine vertices on one or more unique edges shared by two or more of the occupied cubes, wherein the centroid residual is used to refine the vertices.

Images