Method and apparatus for encoding and decoding geometric position information of point cloud and data stream in which geometric position information of point cloud is encoded

Abstract

A method of encoding geometric position information of a point cloud into a bitstream is described. The point cloud is represented by a plurality of cuboid volumes. The plurality of cuboid volumes includes at least one occupied cuboid volume modeled by one or more triangles. The one or more triangles have vertices on the sides of the occupied cuboid volume. The method includes encoding locations of vertices located on edges of occupied cuboid volumes, and encoding a centroid drift for each occupied cuboid volume. Encoding a centroid drift for at least one occupied cuboid volume includes modifying a position of at least one vertex on a side of the occupied cuboid volume, and calculating a centroid drift to be encoded using the modified position of the at least one vertex.

Description

Technical Field

[0001] The present invention generally relates to data compression, and more particularly to methods and apparatus for encoding and decoding geometric position information of a point cloud. Embodiments of the present invention relate to methods and apparatus for encoding / decoding geometric position information of a point cloud using a centroid shift of an occupied cuboid volume (the centroid shift of the occupied cuboid volume is calculated using the modified position of at least one vertex of the occupied cuboid volume), and data or a bitstream in which the centroid shift is encoded. Background Art

[0002] Data compression is used in communications and computer networks to efficiently store, transmit, and reproduce information. For example, point clouds have recently gained significant attention as a representation format for three-dimensional (3D) data due to their rich functionality in representing all types of 3D objects or scenes. Consequently, point clouds can be used to address a wide variety of use cases, including:

[0003] Film post-production,

[0004] Real-time 3D immersive telepresence or VR / AR (virtual reality / augmented reality) applications,

[0005] Free viewpoint videos, e.g. for watching sports,

[0006] Geographic Information Systems, also known as Cartography,

[0007] Cultural heritage, e.g. storing scans of rare objects in digital form,

[0008] Autonomous driving, including 3D mapping of the environment and real-time LiDAR data acquisition (LiDAR: Light Detection and Ranging = a method of measuring distance (odometry) by illuminating a target with laser light and measuring the reflection using a sensor).

[0009] A point cloud is a set of points in a three-dimensional coordinate system. These points are typically used to represent the exterior surface of one or more objects. Each point has a location or position in the three-dimensional coordinate system. This position can be represented by three coordinates (X, Y, Z), which can be Cartesian or any other coordinate system. These points can have other associated attributes, such as color. In some cases, color can also be a three-component value such as R, G, B or Y, Cb, Cr. Other related attributes can include transparency, reflectivity, normal vectors, etc., depending on the desired application of the point cloud data.

[0010] Point clouds can be static or dynamic. For example, a detailed scan or mapping of an object or terrain can be static point cloud data. LiDAR-based environmental scans for machine vision purposes can be dynamic, as the point cloud at least potentially changes over time, e.g., with each successive scan of a volume. Therefore, a dynamic point cloud is a temporal sequence of point clouds.

[0011] As mentioned above, point cloud data can be used in a variety of applications or use cases, including conservation (e.g., scanning historical or cultural objects), mapping, machine vision (e.g., for autonomous or semi-autonomous vehicles), and virtual or augmented reality systems. Dynamic point cloud data for applications such as machine vision can be very different from static point cloud data (e.g., point cloud data used for conservation purposes). For example, automotive vision often involves relatively small-resolution, colorless, highly dynamic point clouds acquired at a high capture frequency using LiDAR or similar sensors. Such point clouds are not intended for human consumption or viewing, but rather for machine object detection / classification in decision-making processes. For example, a typical LiDAR frame contains tens of thousands of points, while high-quality virtual reality applications require millions of points. It is expected that the demand for higher-resolution data will increase over time as computing speeds increase and new applications or use cases are discovered.

[0012] In other words, a point cloud is a set of points located in 3D space, optionally with additional values ​​attached to each point. These additional values ​​are often called point attributes. Therefore, a point cloud can be considered a combination of geometric position information (the 3D position of each point) and attributes. For example, an attribute can be a three-component color, a material property (such as reflectivity), and / or a two-component normal vector of the surface associated with the point. Point clouds can be captured by various types of devices, such as camera arrays, depth sensors, the aforementioned LiDAR, scanners, or they can be generated by computers, such as in film post-production use cases. Depending on the use case, a point cloud can have anywhere from thousands to as many as billions of points in mapping applications.

[0013] The raw representation of point clouds requires a very large number of bits per point, at least a dozen bits per spatial component X, Y, or Z, and optionally more bits for one or more attributes, such as three times 10 bits for color. Therefore, practical deployment of point cloud-based applications or use cases requires compression techniques that can store and distribute point clouds through a reasonable storage and transmission infrastructure. In other words, while point cloud data is useful, the lack of effective and efficient compression (i.e., encoding and decoding processes) may hinder its adoption and deployment. A particular challenge in encoding and decoding point clouds is encoding and decoding the geometric position information of the point cloud, which does not arise in the case of other data compression such as audio or video. The tendency of point clouds to be sparse makes it even more challenging to effectively encode and decode the localization of points.

[0014] Until recently, point cloud compression (also known as PCC) had not received mass market attention and no standardized point cloud codec was available. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11 (also known as the Moving Picture Experts Group or MPEG) started a work item on point cloud compression, which resulted in two standards, namely:

[0015] MPEG-I Part 5 (ISO / IEC 23090-5) also known as Video-based Point Cloud Compression (V-PCC)

[0016] MPEG-I Part 9 (ISO / IEC 23090-9) is also known as Geometric Position Information-based Point Cloud Compression (G-PCC).

[0017] The first versions of the V-PCC standard and the G-PCC standard were finalized in 2020 and 2022, respectively.

[0018] The V-PCC codec method compresses point clouds by performing multiple projections of 3D objects to obtain two-dimensional (2D) patches. When processing moving point clouds, these patches are packed into images or videos. The images or videos are then compressed using existing image / video codecs, taking into account the impact of deployed image and video solutions. By its nature, V-PCC is only effective on dense and continuous point clouds, as image / video codecs cannot compress non-smooth patches obtained from the projection of sparse geometric position information data, such as those obtained by LiDAR.

[0019] The G-PCC coding method has two geometric position information compression schemes.

[0020] The first approach is based on an occupancy tree representation of the geometric position information of the point cloud, for example, through an octree representation, a quadtree representation, or a binary tree representation. In a tree-based structure, the bounding three-dimensional volume of the point cloud is recursively divided into sub-volumes. The nodes of the tree correspond to the sub-volumes. The decision on whether to further divide the sub-volume can be based on the resolution of the tree and / or whether the sub-volume includes any points. The leaf node can have an occupancy flag indicating whether its associated sub-volume contains points. The split flag can indicate whether the node has child nodes, that is, whether the current volume is further divided into sub-volumes. A commonly used tree structure is the octree. In this structure, the volume / sub-volume is a cuboid, and each split of the sub-volume results in eight further sub-volumes / sub-cuboids. Another commonly used tree structure is the KD tree, in which a volume (such as a cuboid) is recursively divided into two by a plane orthogonal to one of the axes. The octree is a special case of the KD tree, in which the volume is divided by three planes, each plane being orthogonal to one of the three axes.

[0021] In other words, the occupied nodes are split until a certain size is reached, and the occupied leaf pattern provides the positions of the points, usually located in the centers of these nodes. By using neighbor-based prediction techniques, a high level of compression can be obtained for dense point clouds. Sparse point clouds can also be processed by directly encoding and decoding the positions of points within nodes with non-minimum sizes, stopping the tree construction when only isolated points exist in the node. This stopping technique is also called direct encoding and decoding mode (DCM).

[0022] The second approach is based on a prediction tree, where each node represents the 3D position of a point, and the relationship between nodes is the spatial prediction from parent to child. Compared to using an occupancy tree, this approach can only handle sparse point clouds and has the advantages of lower latency and simpler decoding. However, compared to the first approach, compression performance is slightly better, but encoding is more complex because the prediction tree construction requires focusing on the best prediction value among a long list of potential prediction values.

[0023] In both schemes, attribute encoding and decoding (i.e., attribute encoding and attribute decoding) are performed after encoding and decoding the complete geometric position information, which again results in a two-pass encoding and decoding process. Low latency can be achieved by using slices that decompose the 3D space into independently encoded and decoded sub-volumes without the need for prediction between sub-volumes. However, this can severely impact compression performance when using multiple slices.

[0024] A particularly interesting use case is the transmission of dynamic AR / VR point clouds, where dynamic refers to the evolution of the point cloud over time. Furthermore, AR / VR point clouds are typically locally 2D, as they often represent the surfaces of objects. Consequently, AR / VR point clouds are highly connected, also known as dense, in the sense that a point is rarely isolated but rather has many neighbors. Consequently, a dense or solid point cloud represents a continuous surface at a resolution such that the volumes associated with the points (also known as cuboids or voxels) touch each other without visual holes in the surface. As mentioned above, such point clouds are often used in AR / VR environments and can be viewed by end users through devices such as televisions, smartphones, or headsets including AR / VR glasses. The point cloud can be transmitted to the device or stored locally. Many AR / VR applications utilize dynamic point clouds, which, in contrast to static point clouds, change over time. Consequently, the data volume can be large, requiring compression. For example, when using an octree representation of the geometric position information of the point cloud, lossless compression down to slightly less than 1 bit per point (or 1 bit per pixel) can be achieved. However, this may not be sufficient for real-time transmission at frame rates up to 50 frames per second, which may involve millions of points per frame, resulting in hundreds of megabytes of data per second.

[0025] Therefore, lossy compression schemes can be used to meet the conventional requirement of maintaining acceptable visual quality by providing compression sufficient to fit the compressed data into the available bandwidth in the transmission channel while maintaining real-time transmission of the frames. In many applications, bit rates as low as 0.1bpp may already allow real-time transmission, which means that with lossy compression, the point cloud can be compressed ten times more than when applying a lossless codec scheme.

[0026] Codecs based on MPEG-1 Part 5 (ISO / IEC 23090-5), or V-PCC, can achieve such low bit rates by using lossy compression with a video codec that compresses 2D frames obtained from the projection of a point cloud onto a plane. The geometric position information is represented by a series of projected patches assembled into the frame, each patch being a small local depth map. However, V-PCC is not general and is limited to narrow types of point clouds that do not exhibit local complex geometric position information, such as trees or hair, because the obtained projected depth maps may not be smooth enough to be effectively compressed by the video codec.

[0027] Pure 3D compression techniques, on the other hand, can process any type of point cloud. For example, G-PCC may in the future provide lossy compression that also allows to compress dense point clouds as well as V-PCC intra while maintaining the versatility of G-PCC, in order to process any type of point cloud, such as dense point clouds, point clouds obtained by LiDAR or point clouds representing 3D maps. In order to implement such a G-PCC mechanism, a so-called Trisoup codec scheme can be applied on the first level based on an octree. Currently, the ISO / IEC standardization working group JTC1 / SC29 / WG7 is discussing the Trisoup codec scheme. When considering the possibility of obtaining a lossy scheme from G-PCC, there are basically three ways to obtain a lossy scheme via the octree representation used by the 3-PCC codec, namely

[0028] Downsampling + (lossless) encoding and decoding + re-upsampling

[0029] Modify voxels locally on the encoder side

[0030] Local simulation point cloud.

[0031] The first approach basically consists of downsampling the entire point cloud to a smaller resolution, losslessly encoding and decoding the downsampled point cloud, and then upsampling after decoding. There are various upsampling schemes, such as super-resolution, artificial intelligence, AI, or learning-based 3D post-processing, which can provide good peak signal-to-noise ratio (PSNR) results when the downsampling is not too aggressive, such as not exceeding a factor of two in each direction. However, even if the metric shows a good PSNR, the visual quality is still controversial and not well controlled.

[0032] The second approach allows the encoder to locally adjust the point cloud so that the encoding and decoding of the octree requires a lower bit rate. To this end, the points can be slightly moved to obtain occupancy information that can be better predicted by neighboring nodes, resulting in lossless encoding of the modified octree at a lower bit rate. Unfortunately, however, this approach only leads to a small bit rate reduction.

[0033] A third approach is to use a tree (e.g., an octree) to encode and decode the geometric position information to a certain resolution, for example, to N×N×N blocks, where N can be, for example, 4, 8, or 16. This tree is then encoded and decoded using a lossless scheme, such as the G-PCC scheme. The tree itself does not require a high bit rate because it does not go to the deepest depth and has only a small number of leaf nodes compared to the number of points in the point cloud. Then, in each N×N×N block, the point cloud is modeled by a local model. Such a model can be an average plane or a set of triangles, as in the Trisoup codec scheme described above, which is now described in more detail.

[0034] The Trisoup encoding scheme locally models a point cloud using a set of triangles without explicitly providing connectivity information - that's why its name comes from the term "soup of triangles". As mentioned above, each NxNxN block defines a volume associated with a leaf node, and within each NxNxN block or volume, the point cloud is locally modeled using a set of triangles, where the vertices of the triangles are encoded along the edges of the volume associated with the leaf nodes of the tree. Figure 1 The volume 100 associated with a leaf node is shown, which is composed of twelve edges 1001 to 100 12 The designed rectangular volume.

[0035] The portion of the point cloud enclosed by volume 100 is modeled by at least one triangle having a plurality of edges 1001 to 100. 12 There is at least one vertex on one of them. Figure 1 In the example of , five vertices 1 to 4 are shown, where vertices 1 to 4 are located on edges 1002 , 1001 , 1008 and 1007 , respectively.

[0036] Vertices lying on edges are shared between those leaf nodes that have common edges, which means that each edge belonging to at least one leaf node encodes and decodes at most one vertex, and by doing so, the continuity of the model is ensured through the leaf nodes. The encoding and decoding of Trisoup vertices requires two pieces of information per edge:

[0037] a vertex flag indicating whether a Trisoup vertex exists on the edge, also referred to here as a presence flag, and

[0038] The position of the vertex along the edge, if it exists.

[0039] Therefore, the codec data includes the octree data plus the trisoup data. For example, the vertex flags can be encoded and decoded by an adaptive binary arithmetic codec that uses a specific context to encode and decode vertex flags, and the length is N=2 s The positions of the vertices on the edge of are encoded and decoded with unit precision by pushing s bits into the bitstream, i.e. by bypassing / not entropy encoding and decoding s bits.

[0040] Figure 2 A volume 100 is shown associated with a leaf node comprising two trisoup triangles 102, 104, each of which has vertices 1 to 4 located at the edges of the volume 100 (see Figure 1 ) 1002, 1001, 1008 and 1004. Triangle 102 includes vertices 1, 2 and 3, while triangle 104 includes vertices 1, 3 and 4. Therefore, in the case where there are at least three vertices on the edge of volume 100, a triangle can be constructed. Naturally, within the volume 100 associated with the leaf node, except Figure 2 Any other triangle combination than the triangle shown is possible.In addition, one or more triangles in volume 100 do not necessarily have all vertices on the edge of volume 100. Instead, one or two vertices of the triangle can be located anywhere in volume 100.

[0041] The triangles to be constructed within volume 100 are based on the following three-step process, including:

[0042] 1. Determine the dominant direction along one of the three axes.

[0043] 2. Sort the trisoup vertices according to the dominant direction.

[0044] 3. Build a triangle based on the ordered list of vertices.

[0045] Figure 3 shows the options for Figure 2 The process of constructing triangles within the volume 100 associated with the leaf node, Figure 3 The leaf node is shown again in (a) without the triangle. Figure 3 (b) and Figure 3 (c) shows the process in two axes, the vertical or z-axis ( Figure 3 (b)) and the horizontal or x-axis ( Figure 3 (c)).

[0046] like Figure 3As shown in (c), the first test along the vertical axis is performed by projecting the volume or cuboid 100 and the trisoup vertices vertically onto the 2D plane, i.e., testing from the top. The vertices are then sorted in clockwise order relative to the center of the projection node 114, which is a square in the example shown. Figure 3 As shown in (c), triangles are constructed according to a fixed rule based on ordered vertices. Figure 3 In the example of , four vertices are involved, and triangles 102, 104 are systematically constructed to include vertices 1, 2, and 3 of the first triangle and vertices 1, 3, and 4 of the second triangle. In the case of only three vertices, the only possible triangle is the one that includes vertices 1, 2, and 3, and in the case of five vertices, a fixed rule can be used to construct triangles that include vertices (1, 2, 3), (1, 3, 5), (4, 5, 1), etc. This can be repeated for up to 12 vertices.

[0047] When from Figure 3 A second test along the horizontal axis is performed by horizontally projecting the cuboid 100 and the trisoup vertices on the 2D plane, yielding Figure 3 (b) shows the projection 116. When the vertices are ordered in clockwise order relative to the center of the projection node 100, the triangles 102, 104 include vertices 1, 2 and 3 of the first triangle and vertices 1, 3 and 4 of the second triangle, as shown in FIG. Figure 3 (b) shown.

[0048] from Figure 3 It can be seen that the vertical projection ( Figure 3 (c)) shows that the total 2D surface of the triangle is maximized, so the principal axis is chosen to be the vertical or z-axis, and from Figure 3 The order of vertical projections shown in (c) yields the trisoup triangles to be constructed, which in turn yields Figure 2 Note that when considering the horizontal axis as the principal axis, this leads to Figure 4 Different configurations of triangles within the volume 100 are shown, Figure 4 A volume 100 is shown in which triangles 102, 104 are arranged in a manner assuming the major axis is horizontal. Figure 3 (b) is constructed in the order of vertices shown.

[0049] By maximizing the projected surface to adequately select the principal axes, a point cloud without holes can be continuously reconstructed.

[0050] Additionally, one centroid vertex of each volume or leaf node 100 is encoded to characterize the surface curvature within each volume 100 . Figure 5The trisoup geometric position information representation of a volume 100 including four vertices V1 to V4 is shown. The centroid vertex C is encoded as the center of gravity C of all vertices V1 to V4. mean The drift value can be encoded as the drift along the surface normal vector The center of mass drift C res . Figure 5 Shows the vector indicating the normal of the triangle surface Optionally, you can also create and signal Figure 5 The face vertices are shown by multiple points.

[0051] Rendering of trisoup triangles is performed by ray tracing, and the set of all rendered points traced by the ray tracing produces a decoded point cloud. Figure 6 Shows rendering including vertices 1, 2 and 3 Figure 2 Ray tracing of the Trisoup triangle 102. Figure 6 Rays like ray 118 in the figure are emitted in a direction parallel to the axis, like Figure 6 The z-axis in . The origin of the ray is an integer, voxelized, precise coordinate point corresponding to the sampling position required for rendering. The intersection 120 of the ray 118 with the triangle 102 is then voxelized, that is, rounded to the nearest point at the desired sampling position, and added to the list of rendering points. After the Trisoup encoding and decoding scheme is applied to all leaf nodes, that is, after the triangles are constructed and the intersection points are obtained by ray tracing, all copies of the same point in the rendering point list are discarded, that is, only one voxel is retained among all voxels sharing the same 3D position, thereby obtaining a set of decoded unique points.

[0052] In the above-described method for trisoup encoding and decoding, vertex determination along an edge within a leaf node is based on points within a predetermined distance from the edge of the node, as described, for example, in WO 2023 / 197122 A1. However, it has been found that if a surface passes through a corner very close to a leaf node, significant visual artifacts may be generated when reconstructing the point cloud. This is illustrated in FIG7 , which shows a leaf node 100 and an original point or surface 130 within a volume 100 for which vertices 1 and 2 are created in the above-described manner. As can be seen in FIG7( a ), as shown at 132 , the point or surface 130 is very close to a corner 134 of the volume 100. For points in region 132 of the original point cloud 130 within the leaf node 100, that is, for points close to corner 134, the above-described trisoup encoding and decoding method can actually generate more than one vertex along more than one edge. For example, instead of generating a single vertex 3, three vertices V1, V2, and V3 are actually generated on the edge around corner 134. In the depicted example, a corresponding vertex is generated on each of the edges that meet at the corner 134 , however, it should be noted that multiple vertices may also be generated on only one or two edges that intersect the corner 134 .

[0053] When the vertices shown in FIG7( a), namely vertices 1, 2, V1, V2, and V3, are used to construct a triangle together with the centroid vertex C (Vc) to obtain a reconstructed surface within the node, undesirable folds are generated in the reconstructed surface, even though there are no folded areas in the original point cloud. FIG7( b) shows the points reconstructed for the volume 100 of FIG7( a) using vertices 1, 2, V1, V2, and V3. It can be seen that for vertices 1 and 2 located at a certain distance from the respective corners of the volume 100, the point cloud is correctly reconstructed, as shown at 130 in FIG7( b). However, the three vertices V1 to V3 generated for the point cloud region 132 near the corner 134 of the volume 100 in FIG7( a) result in undesirable folds in the reconstructed surface, as shown at 136 in FIG7( b). As described above, this results in noticeable visual artifacts in the reconstructed point cloud, as shown in FIG7( b). Summary of the Invention

[0054] It is therefore an object of the present invention to provide methods and devices that allow removing or reducing undesired folds or artifacts in a Trisoup-coded reconstructed surface due to non-optimal vertex positions.

[0055] The present invention provides a method for encoding geometric position information of a point cloud into a bitstream, wherein the point cloud is represented by a plurality of cuboid volumes, the plurality of cuboid volumes including at least one occupied cuboid volume, the occupied cuboid volume being modeled by one or more triangles, the one or more triangles having vertices on edges of the occupied cuboid volume, the method comprising:

[0056] encodes the positions of vertices that lie on the edges of the occupied cuboid volume, and

[0057] Encodes the centroid drift of each occupied cuboid volume,

[0058] The encoding of the centroid drift of at least one occupied cuboid volume comprises:

[0059] - modify the position of at least one vertex on an edge of the occupied cuboid volume, and

[0060] - Using the modified position of at least one vertex to calculate the center of mass shift to be encoded.

[0061] The present invention provides a method for decoding geometric position information of a point cloud from a bit stream, wherein the point cloud is represented by a plurality of cuboid volumes, the plurality of cuboid volumes including at least one occupied cuboid volume, the occupied cuboid volume being modeled by one or more triangles, the one or more triangles having vertices on edges of the occupied cuboid volume, the method comprising:

[0062] decode the positions of vertices that lie on the edges of the occupied cuboid volume, and

[0063] Decode the centroid drift of each occupied cuboid volume,

[0064] For at least one occupied cuboid volume, the method includes:

[0065] - Modify the position of at least one vertex on the edge of the occupied cuboid volume,

[0066] - using the modified position of the at least one vertex to calculate an initial position of the centroid vertex of the at least one occupied cuboid volume, and

[0067] - Use the decoded centroid drift to adjust the initial position of the centroid vertex.

[0068] Optionally, the method further comprises:

[0069] - constructing at least one triangle for at least one occupied cuboid volume using the decoded vertex positions and the adjusted initial position of the centroid vertex, and

[0070] -Reconstructs the point cloud using ray tracing on the constructed triangles in the occupied cuboid volume.

[0071] Optionally, the position of at least one vertex is modified when one or more of the following conditions are met:

[0072] - the distance between at least one vertex and a corner of the occupied cuboid volume is less than or equal to the distance threshold,

[0073] - the total number of vertices in the occupied cuboid volume is greater than the first minimum number,

[0074] - In the occupied cuboid volume, the total number of vertices whose distance to a corner is less than or equal to the distance threshold is greater than the second minimum number.

[0075] Optionally, one or more of the distance threshold, the first minimum number and the second minimum number are signaled in the bitstream or are fixed values ​​known to both the encoding side and the decoding side.

[0076] Optionally, the distance threshold is determined using the width of the occupied cuboid volume and a scale factor.

[0077] Optionally, the distance threshold is determined as follows:

[0078] th=ω*blockWidth

[0079] in:

[0080] th = distance threshold;

[0081] ω = proportionality coefficient, in the range of (0, 1);

[0082] blockWidth = the width of the occupied cuboid volume; and

[0083] *= represents a multiplication operation or a k-bit right shift operation.

[0084] Optionally, the scaling factor and / or k are signaled in the bitstream or are fixed values ​​known to both the encoding and decoding sides.

[0085] Optionally, the distance threshold is the number of vertex sample positions away from the corner.

[0086] Optionally, a geometric position information parameter set GPS is encoded in the bitstream, and the GPS includes one or more of the following:

[0087] - a distance threshold, a first minimum number and a second minimum number,

[0088] - scaling factor and / or k,

[0089] - Number of vertex sampling positions.

[0090] Optionally, modifying the position of at least one vertex comprises: modifying the position of at least one vertex to match positions of corners of the occupied cuboid volume.

[0091] The present invention provides a non-transitory computer-readable storage medium comprising instructions, which, when executed by a computer, cause the computer to perform the method of the present invention.

[0092] The present invention provides an apparatus for encoding geometric position information of a point cloud into a bitstream, wherein the point cloud is represented by a plurality of cuboid volumes, the plurality of cuboid volumes including at least one occupied cuboid volume, the occupied cuboid volume being modeled by one or more triangles, the one or more triangles having vertices on edges of the occupied cuboid volume, the apparatus comprising:

[0093] an encoding module configured to encode positions of vertices located on edges of the occupied cuboid volumes and a centroid shift of each occupied cuboid volume,

[0094] The encoding module is configured to encode the centroid drift of at least one occupied cuboid volume in the following manner:

[0095] - modify the position of at least one vertex on an edge of the occupied cuboid volume, and

[0096] - Using the modified position of at least one vertex to calculate the center of mass shift to be encoded.

[0097] The present invention provides an apparatus for decoding geometric position information of a point cloud from a bit stream, wherein the point cloud is represented by a plurality of cuboid volumes, wherein the plurality of cuboid volumes includes at least one occupied cuboid volume, wherein the occupied cuboid volume is modeled by one or more triangles, wherein the one or more triangles have vertices on the edges of the occupied cuboid volume, and the apparatus comprises:

[0098] A decoding module configured to decode positions of vertices located on edges of the occupied cuboid volumes and a centroid shift of each occupied cuboid volume,

[0099] For at least one occupied cuboid volume, the decoding module is configured to:

[0100] - Modify the position of at least one vertex on the edge of the occupied cuboid volume,

[0101] - using the modified position of the at least one vertex to calculate an initial position of the centroid vertex of the at least one occupied cuboid volume, and

[0102] - Use the decoded centroid drift to adjust the initial position of the centroid vertex.

[0103] The present invention provides a data stream, wherein geometric position information of a point cloud is encoded, the point cloud is represented by a plurality of cuboid volumes, the plurality of cuboid volumes including at least one occupied cuboid volume, the occupied cuboid volume being modeled by one or more triangles, the one or more triangles having vertices on edges of the occupied cuboid volume, the data stream comprising:

[0104] the encoded positions of vertices lying on the edges of the occupied cuboid volume, and

[0105] A centroid shift of each occupied cuboid volume is encoded, the encoded centroid shift of at least one occupied cuboid volume being based on a position of the modified at least one vertex of the at least one occupied cuboid volume.

[0106] The technical solution provided by the embodiments of the present invention has the following beneficial effects: The method of the present invention removes or reduces artifacts in the reconstructed point cloud without significantly increasing the computational complexity and overhead of the encoder / decoder and without making any or only minor changes to the bitstream.

[0107] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood based on the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0108] The drawings are explanatory for the purpose of explaining the invention and are not to be construed as limiting the invention to the embodiments shown.

[0109] Figure 1 showing trisoup vertices along the edges of the volume associated with the leaf nodes of the octree representation of the geometric position information of the point cloud;

[0110] Figure 2 A volume comprising two sets of trisoup triangles is shown, wherein the vertices of the two sets of trisoup triangles are located on the edges of the volume;

[0111] Figure 3 shows the process for selecting triangles to be constructed within a leaf node, where Figure 3 (a) shows the triangle Figure 2 The volume, Figure 3 (b) shows a triangulated 2D surface using the perpendicular projection of the volume, Figure 3 (c) shows a triangulated 2D surface using vertical and horizontal projections of the volume;

[0112] Figure 4 shows a triangle with two sets of trisoup triangles Figure 2 The volume of these two sets of trisoup triangles is constructed under the assumption that the horizontal axis is the principal axis;

[0113] Figure 5 A trisoup geometric position information representation including a centroid vertex to be decoded as a centroid shift value for all vertices is shown;

[0114] Figure 6 Shows ray tracing of rendering trisoup triangles into decoded point clouds;

[0115] FIG7 shows the original points of the point cloud and the vertices generated for a Trisoup node in FIG7( a ), and the points reconstructed using these generated vertices in FIG7( b );

[0116] Figure 8 A flowchart of a method for encoding geometric position information of a point cloud into a bitstream according to an embodiment of the present invention is shown;

[0117] Figure 9 A flowchart of a method for decoding geometric position information of a point cloud from a bit stream according to an embodiment of the present invention is shown;

[0118] Figure 10 shows the data flow according to an embodiment of the present invention;

[0119] Figure 11 The vertices on the six edges that intersect at a node corner of a leaf node are shown;

[0120] Figure 12 According to an embodiment of the present invention, Figure 11 The modification of the position of the vertex that satisfies the condition of moving from its original position to a corner of a leaf node;

[0121] Figure 13 An embodiment of encoding the geometric position information of a point cloud into a bit stream is shown;

[0122] Figure 14 Shows the use of reference Figure 13 An implementation of the described process for decoding a bitstream generated by an encoder;

[0123] Figure 15 A block diagram of an apparatus / encoder for encoding geometric position information of a point cloud into a bitstream according to an embodiment is shown;

[0124] Figure 16 A block diagram showing an apparatus / decoder for decoding geometric position information of a point cloud from a bitstream according to an embodiment;

[0125] Figure 17 A block diagram illustrating an electronic device configured to implement an image processing method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0126] The following description of illustrative embodiments of the present invention is provided with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding and should be regarded as illustrative only. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present invention. In addition, for the sake of clarity and simplicity, descriptions of well-known functions and structures are omitted in the following description.

[0127] In the present invention, the terms "node", "volume", "subvolume" and "occupied cuboid volume" can be used interchangeably. It should be understood that a node is associated with a volume or subvolume. A node is a specific point on a tree, which can be an internal node or a leaf node. A volume or subvolume is a finite physical space represented by a node. In some cases, the term "volume" can be used to refer to the largest bounded space defined to contain a point cloud. In order to build a tree structure of interconnected nodes for encoding and decoding point cloud data, the volume can be recursively divided into subvolumes. An "occupied cuboid volume" is a volume that includes one or more triangles that can be rendered.

[0128] In the present invention, the term "and / or" is intended to cover all possible combinations and subcombinations of the listed elements, including any one, any subcombination or all of the elements listed individually, and does not necessarily exclude additional elements.

[0129] In the present invention, the phrase "at least one... or..." is intended to cover any one or more listed elements, including any one, any subcombination or all of the elements listed individually, without necessarily excluding any additional elements, nor necessarily requiring all elements.

[0130] In this disclosure, the term "codec" refers to either "encoding" or "decoding," as is apparent from the context of implementations involving encoding or decoding geometric position information into or from a bitstream. Similarly, the term "codec" refers to either "encoder" or "decoder."

[0131] When a point cloud is simulated using a set of triangles for each leaf node or volume by applying the Trisoup encoding and decoding scheme in the manner described above, Trisoup data is provided. Trisoup data includes, for example, information about the vertices of the corresponding triangles of the volume, which is also called the occupied cuboid volume or leaf node. However, in the prior art, as described above, points close to the corners of the leaf nodes in the original point cloud may result in more than one vertex being generated along more than one edge, and when these vertices are used together with the centroid vertex to construct triangles to obtain a reconstructed surface within the leaf node, undesirable folds are generated in the reconstructed surface. The present invention is based on the following findings:

[0132] The present invention is based on the inventors' discovery that by modifying the positions of one or more vertices near corners to a common location, such as the location of a volume or leaf node corner, and then using these vertices to calculate a center of mass shift value encoded into the bitstream, undesirable artifacts in the reconstructed point cloud can be removed or significantly reduced. At the decoder, the decoded center of mass shift obtained by the method of the present invention is used to adjust the initial position of the center of mass vertex, which is also determined at the decoder based on the modified vertices, so that multiple vertices created near the corners of the leaf volume are merged into a common location, such as the corner location of the volume. This process removes or reduces undesirable wrinkles or artifacts in the reconstructed point cloud or surface without significantly increasing computational complexity at the decoder or encoder. At the encoder, the original vertices, such as Vertex 1, 2, V1, V2, and V3 (see Figure 7(a)), and the center of mass shift value are encoded into the bitstream. However, the center of mass shift value is obtained by simply replacing vertices V1 through V3 with a single vertex located at, for example, a corner location. Therefore, only a simple shift operation is required at the encoder, and the process for determining the center of mass shift does not need to be modified. Therefore, the method of the present invention does not incur substantial processing complexity and overhead on the encoder side. The same is true for the decoder side, which performs the conventional steps for decoding vertices and centroid drifts from the bitstream. On the decoder side, only simple shift operations are required to modify the position of one or more vertices to a common position, based on which the initial position of the centroid vertex is determined or calculated, as is done in a conventional decoder, and then the initial position is adjusted by the decoded centroid drift. Therefore, also on the decoder side, substantially no additional processing complexity or overhead is required.

[0133] Another advantage of embodiments of the present invention is that when hard-coding the parameters required to determine whether vertex positions need to be moved, the bitstream, which still includes the encoded positions and centroid shift values, remains unchanged, however, the centroid shift values ​​are now determined according to the method of the present invention. According to other embodiments, only a small amount of additional information will be included in the bitstream to signal certain parameters required by the method of the present invention from the encoder side to the decoder side, however, this does not significantly increase the complexity of the bitstream.

[0134] Therefore, the method according to the present invention avoids the above-mentioned artifacts in the reconstructed point cloud without adding a lot of computational complexity and overhead to the encoder / decoder and with no or only minor changes in the bitstream.

[0135] Figure 8A flow chart of a method for encoding geometric position information of a point cloud into a bitstream is shown. The point cloud is represented by a plurality of cuboids. The plurality of cuboid volumes includes at least one occupied cuboid volume, which is modeled by one or more triangles. The one or more triangles have vertices on the edges of the occupied cuboid volume. According to an embodiment, as Figure 8 As shown, the method includes the following steps:

[0136] S100: Encode the positions of vertices located on the edges of the occupied cuboid volume.

[0137] S102: Encode the centroid drift of each occupied cuboid volume.

[0138] Encoding the centroid drift of at least one occupied cuboid volume includes:

[0139] S102a: Modify the position of at least one vertex on the edge of the occupied cuboid volume.

[0140] S102b: Calculate a centroid shift to be encoded using the modified position of the at least one vertex.

[0141] In other words, the center of mass shift to be encoded for the occupied cuboid volume is based on the position of the modified at least one vertex of the occupied cuboid volume.

[0142] Figure 9 A flow chart of a method for decoding geometric position information of a point cloud from a bitstream is shown. The point cloud is represented by a plurality of cuboids. The plurality of cuboid volumes includes at least one occupied cuboid volume, which is modeled by one or more triangles. One or more triangles have vertices on the edges of the occupied cuboid volume. According to an embodiment, as Figure 9 As shown, the method includes the following steps:

[0143] S200: Decode the positions of vertices located on the edges of the occupied cuboid volume.

[0144] S202: Decode the centroid drift of each occupied cuboid volume.

[0145] For at least one occupied cuboid volume

[0146] S204: Modify the position of at least one vertex on the edge of the occupied cuboid volume.

[0147] S206: Calculate an initial position of a centroid vertex of at least one occupied cuboid volume using the modified position of the at least one vertex.

[0148] S208: Use the decoded centroid drift to adjust the initial position of the centroid vertex.

[0149] Figure 10 A data stream 300 according to an embodiment of the present invention is shown, in which geometric position information of a point cloud is encoded. The point cloud is represented by a plurality of cuboids. The plurality of cuboid volumes includes at least one occupied cuboid volume, which is modeled by one or more triangles. One or more triangles have vertices on the edges of the occupied cuboid volume. For example, the data stream or bit stream 300 can be provided by an encoder 400, which performs an inventive method of encoding the geometric position information of the point cloud into the bit stream 300. The data stream 300 is transmitted via a wired or wireless transmission medium 302 (such as a cable or a radio link) to a decoder 500, which decodes the geometric position information of the point cloud from the data stream 300. Therefore, according to an embodiment, as Figure 10 As shown, data stream 300 includes encoded positions 304 of vertices located on edges of occupied cuboid volumes and an encoded centroid shift for each occupied cuboid volume. The encoded centroid shift 306 of at least one occupied cuboid volume is based on the modified position of at least one vertex of at least one occupied cuboid volume.

[0150] Embodiments of the present invention will now be described in more detail.

[0151] As described above, the method of the present invention solves the problem associated with the generation of artifacts in the reconstructed point cloud by modifying the positions of the trisoup vertices near the corner of the leaf pattern to a common position so that there is only one position for multiple vertices near the corner. According to an embodiment of the present invention, the positions of the trisoup vertices near the corner are modified to be positions directly at the corner, that is, the positions of the vertices near the corner are modified so that their positions correspond to the position of the corner. In most cases, using the corner position as the new position of one or more modified vertices may be the most advantageous case because it is located at the center position, however, the present invention is not limited to such embodiments. According to other embodiments, the problem associated with the generation of artifacts in the reconstructed point cloud is solved by modifying the position of one or more vertices to any common position within a distance threshold around the corner.

[0152] In the following, embodiments are described according to which the positions of vertices close to a corner are modified to match the positions of the corners of a leaf node or volume, without limiting the method of the present invention to these embodiments. When performing Trisoup encoding and decoding on multiple volumes or leaf nodes, during the Trisoup vertex derivation process, all vertices that are considered to be sufficiently close to the corners of the leaf node are determined for each volume or leaf node.

[0153] Figure 11 The leaf nodes 100 are shown on the corresponding six edges intersecting at the node corner 134 and the Figure 11 The points in the figure show multiple vertices. Figure 11 A threshold distance th is also shown. For each corner of the node 100, e.g. Figure 11 In the corner 134, the vertex positions along all edges intersecting at the corner 134 after inverse quantization are checked, and more specifically, the number Vn of vertices whose distance to the corner 134 is less than or equal to the distance threshold th is determined, denoted as NUM. Figure 11 In , when the threshold th is applied, among the vertices shown along the six edges, vertex 138 is considered to be close enough to corner 134. Figure 11 In the example of , the number of vertices that are sufficiently close to the corner 134 is 6, ie, NUM=6.

[0154] According to the implementation scheme, the threshold value th can be set as follows:

[0155] Th=ω*blockWidth

[0156] Equation 1

[0157] in:

[0158] th = distance threshold;

[0159] ω = proportionality coefficient, in the range of (0, 1);

[0160] blockWidth = the width of the occupied cuboid volume; and

[0161] *= represents multiplication operation.

[0162] Once the number of vertices that are sufficiently close to the corner 138 is determined, within each leaf node 100, if there are enough vertices Vn (close to the corner) in the current leaf node and if enough of these vertices are close to the corner 134, the positions of the vertices Vn close to the corner are modified to the corresponding positions of the corner 138. These conditions are met when the following conditions apply:

[0163] (1) The number of vertices V in the current node is greater than a minimum value N, which can be signaled in the bitstream, for example using the geometric position information parameter set GPS, or can be fixed and known on the encoder and decoder side. According to an embodiment, N can be equal to 6, which means that the current leaf node or volume includes at least half of the possible vertices. According to an embodiment, this test is applied to determine whether there are enough vertices in the node. If there are not enough vertices in the node, then the collapse behavior is likely not to occur and there is no need to have a process to prevent it. Therefore, according to an embodiment, this test can be applied to speed up processing.

[0164] (2) The number NUM, i.e., the number Vn of vertices close to the corner 134, is greater than a minimum value M, which can also be signaled in the bitstream, such as in GPS, or can be fixed and known on the encoder and decoder sides. According to an embodiment, M can be equal to 4, which means that the corner has at least four vertices that are considered close enough to the corner.

[0165] For example, when considering Figure 11 When looking at the embodiment of , it can be seen that the leaf node 100 (i.e., the current node) includes 14 vertices shown on the edge intersecting the corner 134, thereby satisfying condition (1). In addition, when considering the surroundings of the corner 134, the number of vertices Vn within the distance threshold th is 6, i.e., NUM=6, thereby also satisfying condition (2), which considers not only the close vertices Vn from the current node, but also all the close vertices Vn from the surroundings of the corner 134, such as Figure 11 shown.

[0166] like Figure 12 As shown, this allows the positions of the vertices V1 , V2 , V3 to be moved from their original positions to an angular position 134 .

[0167] Based on such modified vertex positions, on the encoder side, a centroid shift is determined and then encoded into the bitstream. On the decoder side, the initial position of the centroid vertex is determined and then modified according to the centroid shift signaled in the bitstream.

[0168] Therefore, applying the method of the present invention improves the visual quality of the reconstructed point cloud, for example, undesirable wrinkles or other artifacts are removed or reduced when applying the method of the present invention.

[0169] Figure 13 An embodiment of the method of the present invention implemented on the encoder side is shown. For triangular surface reconstruction using the Trisoup encoding and decoding method of point cloud data, on the encoder side, initially, as shown in S300, for each Trisoup node or leaf node 100, a vertex existence flag is determined for each edge, and if the vertex existence flag is true, a quantized vertex position is determined.

[0170] In step S302, all unique Trisoup edges are reordered according to lexicographic order, and then, in step S304, the context of the vertex presence flag and the vertex position is determined using the neighbor information. Following the encoding and decoding order determined in step S302, the vertex presence flag and the quantized vertex position are encoded and decoded into a bitstream, for example, by using context-based adaptive binary arithmetic coding (CABC).

[0171] In step S306, the positions of one or more vertices are modified to correspond to the corner positions. More specifically, by applying the above method, eligible vertices are determined and their positions are replaced with the positions of the corresponding corners.

[0172] In step S308, the vertex coordinates, including the coordinates of the modified and unmodified vertices, are used to calculate the initial position of the leaf node centroid. Using the original points of the point cloud around the calculated initial position of the centroid, the centroid drift value Cres is calculated, quantized and encoded into the bitstream. According to the present invention, since this value depends on the updated vertex position, this is actually the only difference in the bitstream whether the method of the present invention is used or not (and whether signaling is used or not). In other words, only the parameter C res The actual value of is different, that is, the method of the present invention according to the embodiment has the advantage that no modification of the existing bit stream is required.

[0173] According to the embodiment, in step S310, the above-mentioned face vertices can be determined for each leaf node (see Figure 5 ), and the presence of face vertices can be signaled via the bitstream.

[0174] Figure 14 shows the use of the above reference Figure 13 The process described is an embodiment of decoding the bit stream generated by the encoder. More specifically, Figure 14 An embodiment of a method for triangular surface reconstruction in a Trisoup decoding method for point cloud data is shown.

[0175] In step S400 , neighboring information of each edge is determined to be used as context later in a subsequent entropy decoding process.

[0176] In step S402, the vertex existence flag and the quantized vertex position (if the vertex existence flag is true) of each edge are entropy decoded using the neighbor information obtained in step S400.

[0177] At step S404, one or more positions of the vertex are modified according to the above method so that the positions of the qualifying vertices are replaced with the corresponding angular positions of the qualifying vertices based on the decoded and quantized vertex positions.

[0178] In step S406, the quantized centroid drift value of each node from the bitstream is decoded, and the vertex or vertex coordinates of the modified and unmodified vertices are used to calculate the initial position of the centroid vertex of the currently processed leaf node. The initial position of the centroid vertex is adjusted using the quantized drift value decoded from the bitstream from the currently processed leaf node, thereby obtaining a refined centroid vertex C.

[0179] In step S408, information about the existence of face vertices is decoded, and if present, the Trisoup face vertices of the currently processed leaf node are derived.

[0180] In step S410, the vertex coordinates and face vertex coordinates (if any) in each node are sorted, and then triangles are constructed using edge vertices, thinned centroid vertices and face vertices. The edge vertices used include those modified according to the method of the present invention.

[0181] In step S412 , the point cloud is reconstructed by performing ray tracing on the constructed triangles.

[0182] In the embodiments described so far, reference has been made to the modification of the positions of the vertices that are eligible for position replacement according to the above conditions relative to the positions of the corners of the vertices. The present invention is not limited to such embodiments, but rather the vertices that are eligible for position replacement can be modified so as to have a common position, for example when considering Figure 7(a), the positions of vertices V2 and V3 can be modified so as to correspond or match the position of vertex V1 or any other vertex position within a distance threshold from the corner. This type of approach can be used if the corner is not the best choice. For example, three vertices are close to the corner, but two of them are in close proximity and one of them is at the extreme of being close. Then choosing to replace them all with positions between the corner and the farthest vertex may provide better results. Depending on the embodiment, if the corner is not selected, some additional processing may be required to find the best position.

[0183] Furthermore, in the above embodiment, Equation 1 includes a multiplication to find vertices Vn whose distance to the corner is less than or equal to the distance threshold th. However, the present invention is not limited to such embodiments. According to other embodiments, the multiplication in Equation 1 can be replaced by a k-bit right shift operation or a simple integer threshold T that indicates the maximum number of vertex sample positions away from the corner for a vertex to be considered close to the corner. For example, if an edge is sampled at eight possible vertex positions, all vertices that are one and two steps away from the corner can be considered close.

[0184] According to further embodiments, one or more or all of the above parameters ω, k, and T may be signaled in the bitstream, for example, using GPS. According to other embodiments, one or more or all of the above parameters ω, k, and T may be fixed both at the encoder side and at the decoder side, such that they do not need to be signaled in the bitstream.

[0185] Thus far, the concepts of the present invention have been described with reference to embodiments of methods for encoding / decoding geometric position information of a point cloud into / from a bitstream. According to further embodiments, the present invention also provides an apparatus for encoding / decoding geometric position information of a point cloud into / from a bitstream, such as an encoder and / or decoder operating according to the above-described embodiments.

[0186] Figure 15 A block diagram of an apparatus or encoder 400 for encoding geometric position information of a point cloud into a bitstream is shown. The point cloud is represented by a plurality of cuboid volumes. The plurality of cuboid volumes includes at least one occupied cuboid volume, which is modeled by one or more triangles. The one or more triangles have vertices on the edges of the occupied cuboid volume. According to an embodiment, the apparatus 400 includes the following modules:

[0187] The encoding module 402 is configured to encode the positions of the vertices located on the edges of the occupied cuboid volumes and the centroid drift of each occupied cuboid volume.

[0188] The encoding module 402 for encoding the center of mass drift of at least one occupied cuboid volume includes: a modification module 404 for modifying the position of at least one vertex on the edge of the occupied cuboid volume; and a calculation module 406 for calculating the center of mass drift to be encoded using the modified position of at least one vertex.

[0189] Figure 16 A block diagram of an apparatus or decoder 500 for decoding geometric position information of a point cloud from a bitstream is shown. The point cloud is represented by a plurality of cuboid volumes. The plurality of cuboid volumes includes at least one occupied cuboid volume, which is modeled by one or more triangles. The one or more triangles have vertices on the edges of the occupied cuboid volume. According to an embodiment, the apparatus 500 includes the following modules:

[0190] A decoding module 502 is configured to decode positions of vertices located on edges of the occupied cuboid volumes and a centroid shift of each occupied cuboid volume.

[0191] A decoding module 502 for operating on at least one occupied cuboid volume includes: a modification module 504 for modifying the position of at least one vertex on the edge of the occupied cuboid volume; a calculation module 506 for calculating the initial position of the centroid vertex of at least one occupied cuboid volume using the modified position of the at least one vertex; and an adjustment module 508 for adjusting the initial position of the centroid vertex using the decoded centroid drift.

[0192] The present invention further provides, in embodiments, an electronic device, a computer-readable storage medium, and a computer program product.

[0193] Although some aspects of the disclosed concepts have been described in the context of an apparatus, it is clear that these aspects also represent a description of a corresponding method, where a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0194] Figure 17 is a block diagram illustrating an electronic device 600 according to an embodiment of the present invention.

[0195] Electronic device is intended to refer to various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device may also refer to various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are described only as examples and are not intended to limit the implementation of the inventions described and / or claimed herein.

[0196] refer to Figure 17 , the device 600 includes a computing unit 601 for performing various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 602 or loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data for the operation of the storage device 600 may also be stored in the RAM 603. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0197] The components in device 600 are connected to I / O interface 605, including: input unit 606, such as a keyboard and mouse; output unit 607, such as various types of displays and speakers; storage unit 608, such as a magnetic disk and optical disk; and communication unit 609, such as a network card, modem, wireless communication transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices via a computer network (such as the Internet and / or various telecommunication networks).

[0198] The computing unit 601 can be composed of various general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing method can be implemented as a computer software program tangibly presented on a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the above-mentioned image processing method can be performed. In some embodiments, the computing unit 601 can be configured to perform the image processing method in any other suitable manner (e.g., by firmware).

[0199] Various implementations of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include being implemented in one or more computer programs executable and / or interpretable on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor and can receive data and instructions from a storage system, at least one input device, and at least one output device, and can transmit data and instructions to the storage system, at least one input device, and at least one output device.

[0200] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that the program code, when executed by the processor or controller, causes the functions and / or operations specified in the flow chart and / or block diagram to be performed. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0201] In the context of the present invention, a machine-readable medium may be a tangible medium containing or storing a program used by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0202] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user. For example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0203] The systems and techniques described herein can be implemented on a computing system that includes a back-end component (e.g., as a data server), or on a computing system that includes a middleware component (e.g., an application server), or on a computing system that includes a front-end component (e.g., a user computer with a graphical user interface or web browser through which a user can interact with an implementation of the systems and techniques described herein), or on a computer system that includes such back-end components, middleware components, front-end components, or any combination thereof. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0204] A computer system may include a client and a server. The client and server are typically remote from each other and typically interact via a communication network. The client-server relationship is established by computer programs running on the respective computers and establishing a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in a cloud computing service system that addresses the management difficulties and weak business scalability of traditional physical hosts and virtual private servers (VPS). The server may also be a server in a distributed system or a server integrated with a blockchain.

[0205] It should be understood that the steps can be reordered, added, or deleted by using the various forms of the above-described processes. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the intended results of the technical solution of the present invention can be achieved, and this is not limited here.

[0206] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and replacements can be made based on design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the scope of the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for encoding geometric position information of a point cloud into a bitstream, wherein the point cloud is represented by a plurality of cuboid volumes, the plurality of cuboid volumes including at least one occupied cuboid volume, the occupied cuboid volume being modeled by one or more triangles, the one or more triangles having vertices on edges of the occupied cuboid volume, the method comprising: encoding the positions of vertices lying on the edges of the occupied cuboid volume, and Encodes the centroid drift of each occupied cuboid volume, The encoding of the center-of-mass drift of at least one occupied cuboid volume comprises: - modifying the position of at least one vertex on an edge of said occupied cuboid volume, and - using the modified position of said at least one vertex to calculate said centroid shift to be encoded.

2. A method for decoding geometric position information of a point cloud from a bitstream, wherein the point cloud is represented by a plurality of cuboid volumes, the plurality of cuboid volumes including at least one occupied cuboid volume, the occupied cuboid volume being modeled by one or more triangles, the one or more triangles having vertices on edges of the occupied cuboid volume, the method comprising: decoding the positions of vertices located on the edges of the occupied cuboid volume, and Decode the centroid drift of each occupied cuboid volume, For at least one occupied cuboid volume, the method includes: - modifying the position of at least one vertex on an edge of said occupied cuboid volume, - using the modified position of the at least one vertex to calculate an initial position of the centroid vertex of the at least one occupied cuboid volume, and - Using the decoded centroid drift to adjust the initial position of the centroid vertex.

3. The method according to claim 2, further comprising: - constructing at least one triangle for said at least one occupied cuboid volume using the decoded positions of said vertices and the adjusted initial position of said centroid vertex, and - Reconstructing the point cloud using ray tracing on the constructed triangles in the occupied cuboid volume.

4. A method according to any one of the preceding claims, wherein When one or more of the following conditions are met, the position of the at least one vertex is modified: - a distance between said at least one vertex and a corner of said occupied cuboid volume is less than or equal to a distance threshold, - the total number of vertices in the occupied cuboid volume is greater than a first minimum number, - In the occupied cuboid volume, the total number of vertices whose distance to the corner is less than or equal to the distance threshold is greater than a second minimum number.

5. The method according to claim 4, wherein One or more of the distance threshold, the first minimum number, and the second minimum number are signaled in the bitstream or are fixed values ​​known to both the encoding side and the decoding side.

6. The method according to claim 4, wherein: The distance threshold is determined using the width and a scale factor of the occupied cuboid volume.

7. The method according to claim 6, wherein: The distance threshold is determined as follows: th=ω*blockWidth in: th = distance threshold; ω = proportionality coefficient, in the range of (0, 1); blockWidth = the width of the occupied cuboid volume; and *= represents a multiplication operation or a k-bit right shift operation.

8. The method according to claim 7, wherein: The scaling factor and / or k are signaled in the bitstream or are fixed values ​​known to both the encoding side and the decoding side.

9. The method of claim 4 or 5, wherein the distance threshold is the number of vertex sampling positions away from the corner.

10. The method according to any one of claims 4 to 9, wherein the bit stream is encoded with a geometric position information parameter set GPS, wherein the GPS includes one or more of the following: - a distance threshold, a first minimum number and a second minimum number, - scaling factor and / or k, - Number of vertex sampling positions.

11. A method according to any one of the preceding claims, wherein Modifying the position of the at least one vertex includes modifying the position of the at least one vertex to match positions of the corners of the occupied cuboid volume.

12. A non-transitory computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to perform the method of any one of the preceding claims.

13. An apparatus for encoding geometric position information of a point cloud into a bitstream, the point cloud being represented by a plurality of cuboid volumes, the plurality of cuboid volumes including at least one occupied cuboid volume, the occupied cuboid volume being modeled by one or more triangles, the one or more triangles having vertices on edges of the occupied cuboid volume, the apparatus comprising: an encoding module configured to encode positions of vertices located on edges of the occupied cuboid volumes and a centroid shift of each occupied cuboid volume, The encoding module is configured to encode the centroid drift of at least one occupied cuboid volume in the following manner: - modifying the position of at least one vertex on an edge of said occupied cuboid volume, and - using the modified position of said at least one vertex to calculate said centroid shift to be encoded.

14. An apparatus for decoding geometric position information of a point cloud from a bitstream, the point cloud being represented by a plurality of cuboid volumes, the plurality of cuboid volumes including at least one occupied cuboid volume, the occupied cuboid volume being modeled by one or more triangles, the one or more triangles having vertices on edges of the occupied cuboid volume, the apparatus comprising: A decoding module configured to decode the positions of vertices located on the edges of the occupied cuboid volumes and the centroid drift of each occupied cuboid volume, For at least one occupied cuboid volume, the decoding module is configured to: - modifying the position of at least one vertex on an edge of said occupied cuboid volume, - using the modified position of the at least one vertex to calculate an initial position of the centroid vertex of the at least one occupied cuboid volume, and - Use the decoded centroid drift to adjust the initial position of the centroid vertex.

15. A data stream comprising: encoding geometric position information of a point cloud, wherein the point cloud is represented by a plurality of cuboid volumes, the plurality of cuboid volumes including at least one occupied cuboid volume, the occupied cuboid volume being modeled by one or more triangles, the one or more triangles having vertices on edges of the occupied cuboid volume; the data stream comprising: - the encoded positions of vertices lying on the edges of the occupied cuboid volume, and - the encoded centroid shift of each occupied cuboid volume, the encoded centroid shift of at least one occupied cuboid volume being based on the modified position of at least one vertex of the at least one occupied cuboid volume.

Images