Method for encoding and decoding 3D point clouds, encoder and decoder

Abstract

A method for decoding geometry of a 3D point cloud from a bitstream, preferably implemented in a decoder, the method comprising the steps of receiving and decoding a bitstream, wherein the bitstream comprises octree information and vertex information, the octree information comprising information about the octree structure of a volume of the point cloud, the vertex information comprising information about the presence and location of vertices at edges of cubes of leaf nodes of the octree structure; determining triangles by connecting vertices of one cube associated with the leaf nodes of the octree structure; and voxelizing the triangles to determine points of the point cloud, the method further comprising the steps of determining whether additional information included in the bitstream satisfies a predefined condition, the additional information being determined based on the density of the point cloud, preferably a sampling distance d of the point cloud. sampl and if the predefined condition is met, the sampling distance d sampl and expanding at least one triangle along at least one side based on the calculated voxelization vector to perform voxelization.

Description

[Technical Field]

[0001] The present invention relates to a method for decoding a 3D point cloud from a bitstream. It is an object of the present invention to provide a method for encoding a 3D point cloud into a bitstream. It is also an object of the present invention to provide an encoder and decoder, a bitstream encoded by the present invention, and software. In particular, it is an object of the present invention to provide a method for improving the accuracy of the 3D point cloud decoding or reconstruction process, enabling better overall compression performance. [Background technology]

[0002] Point clouds have gained the most attention as a format for representing 3D data due to their ability to represent any type of 3D object or scene. Therefore, many use cases can be solved with point clouds, such as: Film post-production, Real-time 3D immersive remote presentation or virtual reality (VR) / augmented reality (AR) applications, Free viewpoint video (e.g. for sports viewing), Geographic Information Systems (aka cartography), Cultural heritage (scans of rare items stored in digital format), Autonomous driving, including 3D mapping of the environment and real-time laser radar (lidar) data collection.

[0003] A point cloud is a set of points in 3D space, optionally with additional values ​​attached to each point. These additional values ​​are usually called point attributes. A point cloud is thus a combination of geometry (the 3D position of each point) and attributes.

[0004] The attribute may be, for example, a three-component color, a material attribute such as reflectance, and / or a two-component normal vector of a surface associated with the point.

[0005] Point clouds can be captured by many different kinds of devices, such as camera arrays, depth sensors, laser radar, scanners, or they can be generated computer-generated (e.g., during film post-production). Depending on the use case, point clouds can contain thousands or even billions of points for use in drafting applications.

[0006] The original representation of point clouds requires a very high number of bits per point, with each spatial component X, Y, or Z requiring at least 12 bits, and optionally attributes requiring more bits, e.g., color requiring three times 10 bits. Practical deployment of point cloud applications requires compression techniques that can preserve and distribute point clouds via appropriate storage and transmission infrastructure.

[0007] For example, in AR / VR glasses and other devices with 3D capabilities, the compression may be lossy (e.g., video compression) in order to distribute to and enable visualization by the end user. Other use cases, such as medical applications or autonomous driving, require lossless compression so that the compression and transmission does not alter the decision-making results obtained from point cloud analysis.

[0008] To date, point cloud compression (aka PCC) has not yet achieved mass market traction and there are no standardized point cloud codecs available. In 2017, the Standardization Working Group ISO / JCT1 / SC29 / WG11 (also known as the Motion Picture Experts Group or MPEG) started a working project on PCC. This resulted in two standards: MPEG-I Part 5 (ISO / IEC 23090-5) or Video-Based Point Cloud Compression (V-PCC), MPEG-I Part 9 (ISO / IEC 23090-9) or Geometry-Based Point Cloud Compression (G-PCC).

[0009] The first versions of the V-PCC and G-PCC standards were completed at the end of 2020 and were quickly released to the market.

[0010] The V-PCC encoding method compresses point clouds by performing multiple projections of a 3D object to obtain 2D patches that are packaged as images (or as videos when processing displaced point clouds). The resulting images or videos can then be compressed using existing image / video codecs, leveraging already deployed image and video solutions. Essentially, V-PCC only works with dense, continuous point clouds because image / video codecs cannot compress non-smooth patches, such as those obtained from projections of sparse geometry data collected from lidar.

[0011] The G-PCC coding method has two types of geometry compression methods.

[0012] The first method is based on an occupancy tree (octree / quadtree / binary tree) representation of the point cloud geometry. Occupied nodes are split until a certain size is reached, and occupied leaf nodes provide the point locations, usually located at the center of these nodes. A high level of compression can be obtained for dense point clouds by using a neighborhood-based prediction technique. Sparse point clouds are also solved by directly encoding the point locations in nodes with non-minimal size, and stopping the tree construction if a node contains only isolated points; this technique is called Direct Coding Mode (DCM).

[0013] The second method is based on prediction trees, where each node represents the 3D location of a point, and the relationship between nodes is a spatial prediction from parent to child. This method can only handle sparse point clouds, and has the advantages of lower latency and easier decoding than occupancy trees. However, compared to the first occupancy-based method, it offers slightly better compression performance, is more complex to encode, and requires an intensive search for the best predictor (among a long sequence of potential predictors) when constructing the prediction tree.

[0014] In both schemes, attribute encoding (decoding) is performed after geometry encoding (decoding) is completed, resulting in two-pass coding. Therefore, low latency is achieved by using slices, which decompose the 3D space into independently coded subvolumes, without prediction between subvolumes. Using many slices can significantly impact compression performance.

[0015] One important use case is the transmission of dynamic AR / VR point clouds. Dynamic means that the point cloud evolves over time. AR / VR point clouds are also typically localized and 2D, since they represent the surface of an object most of the time. As such, AR / VR point clouds are highly connected (i.e., dense), meaning that points are rarely isolated and have many neighbors.

[0016] A dense (or solid) point cloud represents a continuous surface, and its resolution allows the volumes (small cubes called voxels) associated with the points to touch each other without displaying any visible cavities in the surface.

[0017] This point cloud is typically used in AR / VR environments, where end users view it on devices such as televisions, smartphones, or headphones. It is either transmitted to the device or stored locally. Many AR / VR applications use moving point clouds that change over time, rather than static point clouds. This results in a huge amount of data that must be compressed. Currently, lossless compression of octree representations based on point cloud geometry can achieve just under 1 bit per point (bpp), which may not be sufficient for real-time transmission, which can involve millions of points per frame and frame rates approaching 50 frames per second (fps), resulting in hundreds of megabits of data per second.

[0018] Lossy compression can therefore be used with the usual requirement of maintaining acceptable visual quality while still compressing frames sufficiently to fit within the bandwidth offered by the transmission channel. In many applications, real-time transmission is possible at bit rates as low as 0.1 bpp (10 times higher than lossless coding).

[0019] MPEG-I Part 5 (ISO / IEC 23090-5) or video-based point cloud compression (V-PCC)-based codecs, such as VPCC, can achieve these low bit rates through lossy compression using a video codec that compresses 2D frames obtained from projecting point clouds onto a plane. The geometry is represented by a series of projected patches assembled into a frame, with each patch representing a small local depth map. However, VPCC is not general-purpose and is limited to a narrow type of point cloud that does not exhibit locally complex geometric shapes (e.g., trees or hair). This is because the resulting projected depth maps are not smooth and cannot be efficiently compressed by a video codec.

[0020] Pure 3D compression techniques can handle any type of point cloud. For dense point clouds, it remains an open question whether 3D compression techniques can compete with VPCC (or any projection + image coding scheme). Standardization is moving toward providing an extension (proposed amendment) to GPCC. This extension would provide competitive lossy compression, compressing dense point clouds in the same way as VPCC intra, while preserving the versatility of GPCC and being able to handle any type of point cloud (dense, laser radar, 3D maps). This extension may use the so-called TriSoup (triangle soup) scheme, which is suitable for octrees. The ISO / IEC standardization workgroup JTC1 / SC29 / WG7 is considering TriSoup. For more information on TriSoup coding, see A. DRICOT et al., "Adaptive multi-level triangle soup for geometry-based point cloud coding," presented at the 21st IEEE International Symposium on Multimedia Signal Processing (MMSP) in 2019, Nakagami O., "Report on triangle soup decoding," presented at ISO / IEC JTC1 / SC29-WG11, m52279, presented at ISO / IEC JTC1 / SC29-WG11, 2020, and US10,192,353.

[0021] However, in all lossy compression methods, the quality of the point cloud reconstruction is important. Summary of the Invention

[0022] Therefore, the present invention aims to provide a method for decoding the geometry of a 3D point cloud from a bitstream and a method for encoding a 3D point cloud into a bitstream, said method having improved accuracy.

[0023] This problem is solved by a decoding method according to claim 1, an encoding method according to claim 2, an encoder according to claim 16, a decoder according to claim 17, a bitstream according to claim 18 and software according to claim 19.

[0024] In a first aspect, there is provided a method for decoding 3D point cloud geometry from a bitstream, preferably implemented in a decoder, the method comprising: receiving and decoding a bitstream, the bitstream including octree information and vertex information, the octree information including information about an octree structure of a point cloud volume, and the vertex information including information about the presence and location of vertices at cube edges of leaf nodes of the octree structure; determining triangles by connecting the vertices of one cube associated with a leaf node of the octree structure; voxelizing the triangles to determine points of the point cloud; The method comprises: determining whether or not additional information included in the bitstream satisfies a predefined condition, the additional information being determined based on the density of the point cloud, preferably a sampling distance d sampl and If the predefined condition is met, the sampling distance d sampl and expanding at least one triangle along at least one side based on the calculated voxelization vector to perform voxelization.

[0025] Thus, in a first step, a bitstream is received, the bitstream including information about the octree structure of the decoded point cloud volume. Preferably, the point cloud geometry is GPCC encoded. Thus, decoding from the bitstream provides octree information about the point cloud volume. Furthermore, the bitstream further includes vertex information, the vertex information including information about the presence and position of vertices on cube edges associated with leaf nodes in the octree structure. Thus, decoding from the bitstream provides the vertex information. Here, the bitstream is preferably encoded by the Trisoup encoding scheme in the encoder.

[0026] After decoding the octree and vertex information from the bitstream described in the previous step, the next step to reconstruct the point cloud geometry is to determine triangles for each cube by connecting the vertices at the cube's edges. Thus, the surface of the triangle is determined by the positions of the vertices contained in the bitstream. To reconstruct the point cloud points from the triangles, voxelization is performed using a ray tracing process. In ray tracing, rays are emitted in three directions parallel to one of the three axes. Their origins are integer coordinate points corresponding to the sampling precision required for rendering. Next, the intersection point between the ray and one of the triangles (if any) is determined and added to the list of rendering points, i.e., to the point cloud points. During the voxelization process, the rays sample the surface of the triangle to determine the point cloud points.

[0027] According to the present invention, there are different ways to determine triangles, among which how the triangles are determined based on the additional information contained in the bitstream.

[0028] Method 1. Adaptive Halo: For / during voxelization, the sampling distance d of the point cloud is sampl At least one triangle is extended along at least one side to extend the triangular surface along at least one direction based on d, where the sampling distance is an attribute of the initial point cloud data, and if no points were lost during data collection, the sampling distance correlates to the distance between the actual sampled points of the point cloud in sampling resolution units. samplis set, for example, by a device (e.g., LIDAR) that acquires the points of the point cloud. Therefore, by expanding triangles during voxelization, it is possible to reliably determine additional points in the original point cloud that are ignored during voxelization, thereby improving the accuracy of the voxelization process. Because triangles are sampled with a certain accuracy and sampling resolution, expanding a triangle along at least one edge to expand the triangle's surface allows for capturing points in the point cloud that are outside the triangle. Note that because triangle expansion is based on the sampling distance of the point cloud, the expansion applies to any point cloud, regardless of the sampling distance. Preferably, the expansion is directly proportional to the sampling distance of the point cloud. Therefore, the larger the sampling distance of the point cloud, the greater the expansion of the triangle. Details of the adaptive halo method are described in the dependent claims. Therefore, in many cases, higher accuracy in reconstructing 3D point clouds is achieved, reducing the number of sampling errors during voxelization. Furthermore, the complexity of the encoding and / or decoding algorithms is maintained. However, this method does not perform well with all types of point clouds. In some cases, it may lead to lossy compression results.

[0029] Method 2. Fixed or Other Value Halo: Compared to the adaptive halo, the triangle expansion in this method is based on a fixed value that is independent of the sampling distance of the point cloud. It should be understood that Method 2 may be other schemes for determining triangles, such as not expanding triangles at all.

[0030] Therefore, in accordance with the present invention, additional information included in the bitstream is used to select between adaptive and non-adaptive hello schemes. By introducing such an indicator, each aspect of the present invention can be applied to the appropriate use case, achieving better overall compression performance than solutions that implement only a single aspect.

[0031] Preferably, at least one triangle is extended by more than one side to further expand the surface of the corresponding triangle. Thus, a triangle can be extended by one side, two sides, or all three sides to include points of the original point cloud, said points just beyond the triangle defined by the vertices at the cube sides.

[0032] Preferably, if a cube in a leaf node of the octree structure can contain more than one triangle, each triangle in the cube is expanded along at least one edge for use in voxelization. Therefore, the expansion of the triangular surface can be applied to all triangles in the cube. Alternatively or additionally, in each cube in the octree structure, at least one triangle is expanded along at least one edge for use in voxelization. Alternatively, the expansion of one or more edges of a triangle is applied only to a subset of the leaf nodes in the octree structure. Here, this subset can be determined, for example, by the application, the density of points in the leaf nodes of the point cloud, or requirements for accuracy and decoding speed. More preferably, one or more edges of a triangle are expanded based on a local sampling distance. Therefore, triangles in each subset of leaf nodes can be expanded to achieve locally optimal performance.

[0033] Preferably, the dilation of each side is the same. Thus, to enlarge the surface of the triangle, the triangle is dilated by the same amount in at least two directions. More preferably, the dilation amounts in all three directions are the same. Alternatively, the dilation along at least two directions is different. Thus, treating different directions differently improves decoding accuracy.

[0034] Preferably, each leaf node of the octree structure has the same or different extensions. If more than one edge or each edge of a triangle in one leaf node of the octree structure has a different extension, this may be the same or different in other leaf nodes of the octree structure. Here, the extensions may be selectable in advance and may be determined, for example, by the application, the density of points in the leaf nodes of the point cloud, or requirements for accuracy and decoding speed.

[0035] Preferably,

number

[0036] Preferably,

number

number

[0037] Preferably, the requirement for a convex hull is −ε u_a ≦u, -ε v_a ≦v and -ε w_a ≦w, where ε u_a ,ε v_a ,ε w_a ≥ 0, and u, v, w are the barycentric coordinates of the triangle, where ε u_a , ε v_a , ε w_a At least one of the points is sampled at the sampling distance d sampl Therefore, for different directions, separate convex hull requirements can be provided to separately control the expansion of the triangles considered. u_a ≠ε w_a Alternatively or additionally, ε u_a ≠ε v_a Alternatively or additionally, ε v_a ≠ε w_a Thus, expansion in one or more directions can be selected independently of other directions to determine expansion independently.

[0038] Preferably, the expansion is provided by an adaptive halo parameter, where:

number

[0039] Preferably, the adaptive halo parameter is 1 / 4d sampl More preferably, the adaptive halo parameter is set to 1 / 8d sampl Therefore, by selecting the adaptive halo parameter, you can customize the amount of dilation to obtain optimal results, where the larger this value, the more points are determined during the voxelization process. The preferred range for the adaptive halo parameter is between 0 and d sampl As the sampling distance increases, the adaptive halo parameter also increases, increasing the amount of dilation. Therefore, even if the sampling distance changes, the present invention provides an adaptive solution for dilating triangles, and can ensure that a reasonable number of points are always covered by the dilated triangles.

[0040] Preferably, the adaptive halo parameters are preset. Thus, the encoder and decoder may agree on the adaptive halo parameters, and they are constant for each point cloud generated by the encoder and reconstructed by the decoder. Information about the adaptive halo parameters does not need to be coded into the bitstream.

[0041] Alternatively, the adaptive halo parameters are coded into the bitstream, preferably into the geometry parameter set (GPS) of the bitstream. This can be done once to set the adaptive halo parameters for the subsequently decoded point clouds. Alternatively, each adaptive halo parameter or adaptive halo parameter set can be coded separately for each point cloud.

[0042] Alternatively, the halo parameter also depends on the size of the cubic volume, i.e., the octree level of the current leaf node.

[0043] Preferably, the sampling distance d of the point cloudsampl teeth

number

[0044] Preferably, at least one triangle has a weighted halo parameter ε a_t voxelization along at least one edge based on a_t is ε a_t =ε a * t, where ε ais the sampling distance d of the point cloud sampl The adaptive halo parameter is based on , providing at least one triangular expansion, and t is the corresponding weight associated with the sampling distance, and is preferably set to 2. In some embodiments, t is selected between 1 and 4, more preferably between 1.5 and 2.5. Here, the value of t can be determined using a heuristic method. A heuristic method is an optimization method that attempts to find a globally optimal feasible solution for a particular problem under consideration. Heuristic methods are iterative in nature. At each iteration, a feasible solution for the particular problem is determined. When the heuristic method terminates after a certain period or a certain number of iterations, the output solution is the optimal solution found at that iteration. More preferably, the weight tried at each iteration is an integer selected from the range of 1 to 4. Here, the adaptive halo parameter is less than 1. A weight that is too large can affect the overall accuracy of the TriSoup model. Therefore, an upper limit of 4 can be set. For example, if the adaptive halo parameter is 1 / 4 and it is determined that the best results are obtained by assigning a weight of 2 to the sampling distance, then if the adaptive halo parameter is directly proportional to the sampling distance, the updated adaptive halo parameter may be 1 / 4*2 = 1 / 2. Therefore, providing an appropriate range for setting the weight can further improve the efficiency and accuracy of the overall algorithm. It should be understood that different weights can also be determined for different directions of the triangle.

[0045] Preferably, the additional information is a flag for enabling or disabling a function of the encoding or decoding method, and is preferably one bit. In the simplest case, the additional information can be a flag indicating whether an adaptive hello scheme is enabled or disabled. It will be understood that the additional information may be multiple bits, as long as it can indicate the necessary information according to the present invention.

[0046] Preferably, the side information is encoded into the geometry parameter set (GPS) of the bitstream.

[0047] In another aspect of the present invention, a method for encoding a 3D point cloud into a bitstream is provided, preferably implemented in an encoder. The method for encoding a 3D point cloud comprises: obtaining octree information, the octree information including an octree structure of a volume, the volume including a plurality of cubes; obtaining vertex information from the surface of the point cloud of each cube associated with a leaf node, wherein the vertex information includes information regarding the presence and location of vertices on the edges of the cube; encoding the octree information and the vertex information into a bitstream; and reconstructing geometry data of the point cloud based on the octree information and vertex information obtained in the encoding process. Here, the step of reconstructing the geometry data of the point cloud includes: determining triangles by connecting the vertices of one cube associated with a leaf node of the octree structure; voxelizing the triangles to determine points of the point cloud; The method comprises: determining additional information based on the density of the point cloud, preferably by a sampling distance d sampl and evaluating the encoding the side information into the bitstream; determining whether the additional information satisfies a predefined condition; If the predefined condition is met, the sampling distance d sampl and expanding at least one triangle along at least one side based on the calculated voxelization vector to perform voxelization.

[0048] Therefore, the encoding method generates octree information and vertex information. The additional information is determined and generated based on the density of the point cloud, for example, based on the sampling distance of the point cloud. The density can also be determined by other methods, which will not be described in detail here. The information is coded into a bitstream. Then, the encoder performs a reconstruction step. In this reconstruction step, point cloud geometry information is reconstructed, where the reconstruction step is the same as that of the decoding method described above. The encoder then uses the reconstructed geometry of the point cloud to encode the attributes (color, reflectance, etc.) of the points in the point cloud, for example, using RAHT (Region Adaptive Hierarchical Transform), predictive transform, or lifting transform to encode the attributes of the points in the point cloud.

[0049] Preferably, the geometry of the point cloud is encoded into the bitstream by geometry-based point cloud compression (G-PCC).

[0050] Preferably, the bitstream is an MPEG G-PCC compliant bitstream.

[0051] Preferably, the encoding method is constructed according to the characteristics described above in connection with the decoding method.

[0052] In another aspect of the present invention, there is provided an encoder for encoding a 3D point cloud into a bitstream, the encoder comprising a memory and a processor, wherein instructions are stored in the memory and, when executed by the processor, perform the steps of the encoding method described above.

[0053] In another aspect of the present invention, there is provided a decoder for decoding a 3D point cloud from a bitstream, the decoder comprising a memory and a processor, wherein instructions are stored in the memory and, when executed by the processor, perform the steps of the above decoding method.

[0054] In another aspect of the invention, there is provided a bitstream encoded according to the steps of the encoding method described above.

[0055] In another aspect of the invention, a computer readable storage medium is provided that includes instructions for performing the steps of the method for encoding a 3D point cloud into a bitstream as described above.

[0056] In another aspect of the invention, there is provided a computer-readable storage medium comprising instructions for performing the steps of the method for decoding a 3D point cloud from a bitstream as described above.

[0057] In another aspect of the present invention, a computer-readable storage medium is provided, comprising instructions for performing the steps of the method for encoding a 3D point cloud into a bitstream described above, and further comprising a configuration file indicating a point cloud type indicating a point cloud density. Here, the point cloud type may be, for example, solid, dense, sparse, and scant. However, it should be understood that these types can essentially be distinguished by the point cloud sampling distance described above. In any of the above embodiments, the additional information can also be determined based on the point cloud type (e.g., by obtaining information from a configuration file). [Brief explanation of the drawings]

[0058] The present invention will now be described in more detail with reference to the accompanying drawings. These drawings show the following: [Figure 1a] 1 is a flowchart of a method for decoding 3D point cloud geometry according to the present invention; [Figure 1b] 1 is a simplified flowchart of a decoding method according to the present invention; [Figure 2] Here is an example of generating an octree structure. [Figure 3] The octree shown in Figure 2 is [Figure 4] Here is an example of determining the vertices on the edges of a cube. [Figure 5] Here is an example of generating a triangle. [Figure 6]Here is an example of a vertex on the edge of a cube: [Figure 7] Show that the vertices generate triangles. [Figure 8] An example of determining the order of triangles according to FIG. 7 is shown. [Figure 9] FIG. 1 is a schematic diagram of the voxelization step. [Figure 10] 2D representation of a triangle at a leaf node of an octree. [Figure 11] An example of voxelization of the triangle in Figure 10 is shown. [Figure 12] The barycentric coordinates and definitions of the triangle in Figure 10 are shown below. [Figure 13] A comparison of the vertex triangles and the original point cloud is shown. [Figure 14a] The triangle in FIG. 10 is shown to expand along one direction based on fixed halo parameters in barycentric coordinates. [Figure 14b] The triangle in FIG. 10 is shown expanding along one direction based on the adaptive halo parameters in barycentric coordinates. [Figure 15a] The triangle in Figure 10 shows that it expands along all three directions based on fixed halo parameters. [Figure 15b] The triangle in Figure 10 shows that it expands along all three directions based on the adaptive halo parameters. [Figure 16] We present a representation of a triangle extending in three directions based on the weighted halo parameter εa_t and the sampling distance of the point cloud. [Figure 17a] 1 shows a representation of a triangle extending in three directions based on the sampling distance of the point cloud. 1 shows a representation of a triangle extending in three directions based on the sampling distance of the point cloud. [Figure 17b] Shows a representation of a triangle that extends in three directions by a constant amount. [Figure 17c] A sampling distance of 1 represents a triangle that extends in three directions by a constant amount. [Figure 18] 1 is a schematic flow chart of an encoding method. [Figure 19a]The performance of longdress data based on different halo parameters is shown. [Figure 19b] We show the performance of the house_without_roof data based on different halo parameters. [Figure 19c] We show the performance on ulb_unicorn data based on different halo parameters. DETAILED DESCRIPTION OF THE INVENTION

[0059] Please refer to FIG. 1a, which is a schematic diagram of a method for decoding geometry information of a 3D point cloud from a bitstream.

[0060] A method for decoding 3D point cloud geometry from a bitstream, preferably implemented in a decoder, comprising the steps of: In step S01, a bitstream is received and decoded, where the bitstream includes octree information and vertex information, the octree information includes information about the octree structure of the point cloud volume, and the vertex information includes information about the existence and position of vertices on the cube edges of the leaf nodes of the octree structure; In step S02, a triangle is determined by connecting the vertices of one cube associated with a leaf node of the octree structure; In step S03, the triangle is voxelized to determine the points of the point cloud; determining whether additional information included in the bitstream satisfies a predefined condition, the additional information being determined based on the density of the point cloud, preferably a sampling distance d sampl If the predefined condition is met, at least one triangle is located within the sampling distance d sampl The voxelization is expanded along at least one edge based on

[0061] To determine the octree information, the first step in the geometry encoding process is to build and encode an octree, as shown in Figures 2 and 3. The bounding box is a main volume 100 that contains all points and is associated with a root node 112 (i.e., a single node at the top of the tree 110). This main volume 100 is first divided into eight sub-volumes 102, called octants, with each sub-volume 102 represented by a node 114 in the tree 110. Next, recursive divisions are made in the sub-volumes 104 until a target level is reached, where an octant 106 is occupied by at least one point, indicated by hatching in Figures 2 and 3.

[0062] Each octet (or node) is represented by an occupied byte, which contains one bit for each sub-octet: if a sub-octet is occupied by at least one point, the corresponding bit is set to 1, otherwise it is set to 0. The occupied bytes 118 of all octets are serialized and entropy coded with a binary arithmetic encoder (with wide priority).

[0063] 4 shows an example of a block representation of a 3D surface 210 and a block 220 in TriSoup. Surface 210 intersects block 220; therefore, block 220 is an occupied block; block 220 exists between multiple blocks 200 in 3D space. Within block 220, the closed portion of surface 210 intersects the edges of the block at the six illustrated vertices of polygon 230. If an edge of block 220 contains a vertex, then the edge is said to be selected.

[0064] Figure 5 shows a block 220 in TriSoup, with the surface 210 omitted for clarity, showing an unselected edge 270, a selected edge 260, and the i-th edge 250. Assume that the i-th edge 250 is selected. The vertex v on edge i is i To specify a scalar value, specify the corresponding fraction of the length of side 250.

[0065] As shown in Figures 4 and 5, within each octet 220 at the target level of the octree, trisoup represents the original surface 210 as a set of triangles 245. This surface is then coded and used to obtain the reconstructed (or decoded) point locations. First, the intersections of the octet edges with the surface represented by the original points are estimated by averaging the locations of the points closest to those edges within the octet. Next, the 12 edges of all octets and their associated intersections (if any) are stored as segments and vertices, respectively. Each (unique) segment is then coded as follows: The first bit is arithmetically coded; if the segment is occupied by the vertex, this bit is set to 1; otherwise, it is set to 0. If occupied, the vertex's relative position in the segment is also arithmetically coded.

[0066] As shown in Figure 6, triangle vertices 310 are encoded along edges 320 of the volume associated with leaf nodes 300 of the tree. These vertices 310 at edges 320 are shared between multiple leaf nodes 300 that share a common edge 320. This means that each edge belonging to at least one leaf node encodes at most one vertex. In this way, the leaf nodes ensure the continuity of the model.

[0067] As mentioned before, the encoding of a TriSoup vertex requires two pieces of information for each edge: A vertex flag indicating whether the edge has a TriSoup vertex, and The position of the vertex along the edge, if any.

[0068] Therefore, the encoded data consists of octree data and TriSoup data.

[0069] The vertex flags are encoded by an adaptive binary arithmetic encoder, which encodes the vertex flags using a specific context. Length N=2 sThe position of the vertex on an edge of can be coded in a single precision by squeezing (bypassing / non-entropy coding) s bits into the bitstream.

[0070] Within a leaf node, we construct triangles from the TriSoup vertices if there are at least three vertices 310 on an edge 320 of the leaf node 300. Figure 7 shows the reconstructed triangles 330, 340.

[0071] Of course, other combinations of triangles 330, 340 are possible. Triangle selection involves three steps: 1. Determine the leading direction along one of the three axes, 2. Sort the TriSoup vertices based on the leading direction, 3. Construct triangles based on an ordered vertex list.

[0072] Knowledge of the exact location of the triangle within the current leaf node is not required and can be derived from the vertices.

[0073] Figure 8 is used to illustrate this process. Each of the three axes is tested, with the one that maximizes the total surface area of ​​the triangle being the primary axis. For ease of illustration, only two axis tests are shown in Figure 8.

[0074] The first test (top) along the vertical axis is performed by projecting the cube and TriSoup vertices 310 perpendicularly onto a 2D plane. The vertices 310 are then reordered clockwise relative to the center of the projection node (the square). Then, based on the ordered vertices, triangles 330 and 340 are constructed according to fixed rules. Here, when four vertices are involved, triangles 123 and 134 are systematically constructed. When three vertices are involved, triangle 123 is the only possible triangle. When five vertices are involved, the fixed rule may be to construct triangles 123, 134, and 451. In this way, up to 12 vertices.

[0075] We perform a second test (left side) along the horizontal-vertical axis by projecting the cube and Trisoup vertices horizontally onto a 2D plane.

[0076] Since vertical projection displays the largest 2D total surface of the triangle, the major axis is chosen to be the vertical axis, and we obtain the constructed TriSoup triangles in the order of vertical projection, as shown in the nodes of Figure 8. Note that making the horizontal axis the major axis results in a different construction of the triangles.

[0077] By maximizing the projection plane and selecting the principal axes appropriately, a hole-free point cloud is successively reconstructed.

[0078] Rendering TriSoup triangles as points is achieved by ray tracing. The set of all points rendered by ray tracing constitutes the decoded point cloud.

[0079] In the ray tracing shown in Figure 9, rays are emitted in three directions parallel to the axis. Their origin is a point with integer (voxelized) coordinates with a precision corresponding to the sampling precision required for rendering. Then, the intersection point with one of the Trisoup triangles (the dotted point, if any) is voxelized (= rounded to the nearest point with the required sampling precision) and added to the rendering point list.

[0080] After applying Trisoup to all leaf nodes, i.e., building triangles and obtaining points by ray tracing, we discard copies of the same point in all rendered point lists (i.e., only one voxel remains for all voxels that share the same position and volume) and obtain one set of decoded (only one) points.

[0081] For simplicity, from now on, the following Figures 10-16 depict the 2D volumes (squares) associated with leaf nodes, rather than the 3D volumes (cubes). Note that all methods described in this invention are applicable to 3D spaces.

[0082] Referring to Figure 10, it shows an example of an NxNxN volume, where N=2 s = 8. There are at least three vertices V1, V2, and V3 on the edge 410 of this volume (which is depicted as a square in the figure, but is actually a cube).

[0083] The edges of the leaf nodes are located at positions -0.5 and N-0.5 to ensure continuity of the TriSoup model when passing from one "volume" to an adjacent one. In practice, this means that cube faces are shared between adjacent volumes. This way, the positions of the vertices on an edge do not depend on the cube to which the edge belongs.

[0084] A vertex is a position p along its corresponding edge. k are the quantized positions 400 that are coded into the bitstream. These positions 400 are k can be quantized with a unit step size so that p1=4, p2=2, and p3=2.

[0085] A TriSoup triangle 440 is constructed from vertices V1, V2, and V3, and the triangle soup belonging to a volume models the cloud of points enclosed by the volume.

[0086] The process of recovering points 430 (of the decoded point cloud) from triangles 440 is called voxelization of the triangle. Figure 11 shows the voxelization of the TriSoup triangle of Figure 10. Rays are cast along all integer coordinates 420 (white and black points), and rays that intersect with the triangle generate some decoded points (black points). Here, the origins of the rays have a spacing D, which sets the sampling resolution for voxelization.

[0087] The intersection point of the ray and the triangle is determined using barycentric coordinates, as shown in Figure 12.

number

[0088] Any point P in 3D space can be uniquely represented by its barycentric coordinates relative to any non-degenerate 3D triangle ABC (equivalent to any triangle V1V2V3 from the TriSoup model).

[0089] Any point P in 3D space can be uniquely represented as follows: P=uA+vB+wC where: u+v+w=1 There is a condition:

[0090] The points of the triangle correspond to the convex hull. Therefore, 0≦u,v,w.

[0091]

number

number

number

[0092] The intersection point P of the ray with the unique plane passing through A, B, and C can be found by the following calculation:

number

[0093] This intersection point P belongs to the triangle if and only if 0≦u,v,w.

[0094] As shown in Figure 13, there is a slight shift between the position of the Trisoup triangle V1V2V3 determined by the vertices from the bitstream and the natural position of this triangle 450 in the current volume. This position is natural because the encoder derives the vertex V from the closest point (to the edge) in the original point cloud. k Therefore, the position of vertex V k Voxelized points adjacent to are likely to be points in the point cloud. These points are natural candidates for constructing "natural" triangles that model the point cloud.

[0095] This shift is due to the continuity constraints of the adjacent volumes. As a result, the ray trace contains some points 460 (P in Figure 13) because they do not belong to the Trisoup triangle (compared to triangles 440 determined by the vertices provided by the bitstream as shown in Figure 11). miss ) is missing. The direct consequence is a degradation of the quantization geometry metric and a degradation of the rate-distortion performance of this scheme.

[0096] Therefore, by slightly relaxing the convex hull condition 0 ≤ u,v,w, we can build a "halo" around the TriSoup triangles. This will increase the size of the triangles a little, and the ray trace will intersect the increased-sized triangles and find fewer points P. miss You will miss out.

[0097] Let ε>0 be the halo parameter. As shown in Figure 14a, we relax the condition 0 ≤ u to -ε ≤ u, where u is the centroid weight associated with point A, and increase the size of the triangle along side BC opposite point A, shown in dotted region 470.

[0098] By making the convex hull 0≦u,v,w to be −ε≦u,v,w, the relaxation of the condition can be applied to the three centroid weights u,v and w.

[0099] The resulting halo 480 around triangle 440 is shown in Figure 15a. To a first approximation, the size of the halo is directly proportional to the parameter ε.

[0100] The halo parameters may depend on the weight of each centroid of the triangle, e.g., −ε u ≦u, -ε v ≦v and -ε w ≦w, where ε u , ε v and ε w are the three halo parameters.

[0101] The effect on the voxelization is shown in Figure 17b, where several points P miss is now part of the "halo" and is decoded as a point in the decoded point cloud, and therefore does not leak as in the original algorithm.

[0102] Of course, the halo parameter ε (or ε u , ε v and ε w The value of ε needs to be set to result in a halo of sufficient size. If ε is too small, the halo will be very small and will have little effect, returning us to the problem of leaky points in the prior art. If ε is too large, the halo will be large, affecting the overall accuracy of the Trisoup model. In both cases, the distortion of the decoded point cloud will not be optimal.

[0103] The appropriate value of the halo parameter ε is

number

[0104] However, setting the halo parameters to fixed values ​​has the drawback that the "halo" constructed may not always give the best results.

[0105] To prove that any fixed halo value does not yield the best results for a set of data, we tested the performance of different halo values ​​ε on three test point clouds named "longdress_viewdep_vox12," "house_without_roof_00057_vox12," and "ulb_unicorn_vox13" in MPEG G-PCC. In the test experiments, the halo parameter values ​​used in the G-PCC code were obtained by multiplying ε by 256 to improve calculation accuracy, and were set to 16, 32, 64, and 128, with the corresponding values ​​of ε being 1 / 16, 1 / 8, 1 / 4, and 1 / 2. For each data set, we obtained coding performance using these four halo values ​​ε at the same compression rate r02.

[0106] Figures 19a, b, and c show the relationship between the quality of the decoded point cloud (geometry PSNR) and the halo ε value. A higher PSNR indicates better quality. We observed that different data can achieve maximum PSNR quality with different halo values ​​ε. For example, the optimal halo value for longdress data is 128, the optimal halo value for house_without_roof data is 128, and the optimal halo value for ulb_unicorn data is 32. Therefore, the value of the halo parameter ε may not be constant to achieve optimal compression performance for different datasets.

[0107] As shown in Figure 17c, some edges close to the volume have two natural points (represented by all black points), and the TriSoup triangle V1V2V3 is derived from these points. A shift is observed between the TriSoup triangle V1V2V3 and its natural position in the current volume. In Figure 17c, the sampling distance of the points is 1, and the enlarged triangle obtained with the current fixed halo parameter ε is P missHowever, as the sampling distance increases, the farther the natural points are from the triangle, the more natural points will be missed when using the current fixed halo parameters than when the sampling distance is 1, as shown in Figure 17b. Therefore, in order to reduce the point reconstruction error, a larger halo parameter is required for point data with a larger sampling distance.

[0108] Therefore, by slightly relaxing the convex hull condition 0 ≤ u,v,w, an adaptive "halo" is constructed based on the sampling distance of the point cloud around the TriSoup triangle. This allows the triangle size to increase slightly, so that the ray trace intersects the increased triangle size and encounters fewer points P. miss You will miss out.

[0109] The advantages of the adaptive halo method are as follows: The decoded point cloud has less distortion. In fact, a quantitative metric (BDBR) shows that compared to non-adaptive methods (i.e., fixed halo parameters), our proposed method achieves a bitrate gain of 2.6% (for comparable quality). · The overall algorithm remains the same, so complexity is maintained.

[0110] ε a Let ε > 0 be the adaptive halo parameter determined based on the sampling distance of the point cloud. As shown in Figure 14b, we define the condition 0 ≤ u as ε a Relax u, where u is the centroid weight associated with point A, to increase the size of the triangle along side BC opposite point A shown in dotted region 472.

[0111] Convex hull 0≦u,v,w is -ε a By making u,v,w the condition can be relaxed to three centroid weights u,v and w.

[0112] The obtained halo 482 around the triangle 440 is shown in Figure 15b. To a first approximation, the size of the halo is determined by the adaptive halo parameter ε a It is directly proportional to the sampling distance of the point cloud.

[0113] In one embodiment, the adaptive halo parameters may depend on the weight of each centroid of the triangle, e.g., −ε u_a ≦u, -ε v_a ≦v and -ε w_a ≦w, where ε u_a , ε v_a and ε w_a are the three adaptive halo parameters.

[0114] Figures 17a and 17b show the effect on voxelization. Compared to Figure 17b, when the sampling distance increases and the halo parameter is constant (smaller sampling distances are more appropriate), many leaked points P miss In Fig. 17a, the adaptive halo parameters according to the present invention are applied, so that some leaked points P miss is now part of the halo and so will be decoded as a point in the decoded point cloud, and therefore will not leak as in the original algorithm.

[0115] Referring to FIG. 16, more P miss is now part of the halo. A larger halo is provided by a weighted halo parameter, where the weighting parameter relates not only the sampling distance of the point cloud, but also the weight t to the sampling distance. In Figure 16, the weight t is set to 2. This further improves the accuracy of the 3D point cloud decoding or reconstruction process.

[0116] Of course, the adaptive halo parameter ε a (Instead of this, ε u_a , ε v_a and ε w_a The value of ε ) must be set to give a halo with sufficient size. aIf ε is too small, the halo will be very small and have little effect, returning to the leaky point problem of the prior art. If ε is too large, the halo will be large, affecting the overall accuracy of the Trisoup model. In both cases, the distortion of the decoded point cloud will be suboptimal.

[0117] Halo parameter ε a The appropriate value of

number

[0118] When the sampling distance is fixed, the adaptive halo parameter ε a (Instead of this, ε u_a , ε v_a and ε w_a ) may be a fixed value. In one variant, the halo parameter ε a (Instead of this, ε u_a , ε v_a and ε w_a ) is coded into the bitstream, for example into a Geometry Parameter Set (GPS). a (Instead of this, ε u_a , ε v_a and ε w_a ) further depends on the size of the volume N. In yet another variation, for a set of volumes representing a point cloud, the adaptive halo parameters ε a (Instead of this, ε u_a , ε v_a and ε w_a ) locally.

[0119] Although the adaptive halo method has many advantages, it does not work well with all types of MPEG point cloud datasets. When used directly in MPEG G-PCC software, it leads to an overall loss in compression results in the D2 (point-to-plane distortion) metric, and the overall performance gain in the D1 (point-to-point distortion) metric is not significant (approximately 2%, less than 5%), so a single implementation of the adaptive halo method cannot achieve overall optimal coding performance for all types of MPEG point cloud datasets.

[0120] In particular, the MPEG G-PCC standard classifies AR / VR datasets into four categories: solid, dense, sparse, and sparse. The solid category refers to voxelized point clouds with a continuous surface; the dense category refers to discontinuous voxelized point clouds; the sparse category refers to non-dense (sparser than the dense category); and the sparse category refers to very sparse data. The Trisoup model's adaptive halo method has already been tested on data from all of the above categories. As mentioned above, BDBR has two metrics for evaluating the quality of the reconstructed point clouds: D1 (point-to-point distortion) and D2 (point-to-plane distortion). Detailed experimental results for each category are as follows: · Solid category: The adaptive halo method does not affect the compression efficiency of solid category data in terms of D1 and D2 metrics. Dense Category: According to the D1 metric, the adaptive halo method is very effective in improving the compression efficiency of dense category data (with a 5% gain in practice). According to the D2 metric, this method has little impact on the compression efficiency of dense category data. Sparse and rare categories: The adaptive halo method can slightly improve the compression efficiency of sparse and rare categories in the D1 metric plane, but causes loss in the D2 metric plane.

[0121] Therefore, in the last step S03 of FIG. 1a, by selectively executing the adaptive halo method, better overall performance of encoding is achieved. Referring to FIG. 1b, a simplified flow according to the proposed method is shown. Here, the bitstream at the encoder can include one flag (e.g., adaptive_halo_enabled_flag), and the decoder can determine whether to enable the adaptive halo method based on this flag. Preferably, this flag may be included in the geometry parameter set (GPS) of the bitstream. The above GPS includes parameters that specify features and activation tools used in the encoded geometry information bitstream of the point cloud slice, and the GPS is arranged in the slice header of the geometry information stream. For example, when the flag is set to "true", the adaptive halo method is enabled, and otherwise, the adaptive halo method is not used for trisoup encoding. When not using the adaptive halo method, triangles for voxelization can be extended along at least one side based on a fixed value. As described above, how to set the flag may be based on the category of the point cloud data, which can be evaluated by the sampling distance of the point cloud data. For example, when the sampling distance d of the point cloud satisfies the condition 1 < d < 4 (i.e., dense), the value of the flag is set to true, and otherwise, the value of the flag is set to false.

[0122] In some embodiments, the value (true / false) of the flag adaptive_halo_enabled_flag can be determined by reading from the configuration file of the G-PCC encoder, where data information (including data category) is indicated in the configuration file.

[0123] Referring to FIG. 18, it is a schematic flowchart of a method for encoding a 3D point cloud according to the present invention into a bitstream. This method includes the following steps.

[0124] In step S11, octree information is determined, the octree information including the octree structure of a volume, the volume including a plurality of cubes.

[0125] In step S12, vertex information is obtained from the surface of the point cloud of each cube associated with the leaf node, where the vertex information includes information about the presence and location of the vertex on the cube edge.

[0126] In step S13, the octree information and vertex information are coded into a bitstream.

[0127] In step S14, point cloud data is reconstructed based on the octree information and vertex information acquired in the encoding process described above, where reconstructing the point cloud data includes the following steps 141 to 142.

[0128] In step 141, triangles are determined by connecting the vertices of one cube associated with the leaf nodes of the octree structure;

[0129] In step 142, the triangle is voxelized to determine the points of the point cloud, and additional information is determined based on the density of the point cloud (the sampling distance d sampl (which can be evaluated by), encoding the side information into the bitstream, determining whether the side information satisfies a predefined condition, and if the predefined condition is met, determining whether the sampling distance d sampl Based on this, at least one triangle is expanded along at least one side to perform voxelization.

[0130] Here, steps S11 to S13 relate to Trisoup coding. For details of Trisoup coding, see, for example, A. DRICOT et al., "Adaptive multi-level triangle soup for geometry-based point cloud coding," presented at the 21st IEEE International Symposium on Multimedia Signal Processing (MMSP) in 2019, Nakagami O., "Report on triangle soup decoding," ISO / IEC JTC1 / SC29-WG11, m52279, presented in 2020, and US Pat. No. 10,192,353. In addition to the conventional point cloud encoding, this method further includes a reconstruction step that includes steps identical or similar to those of the decoding method specifically described above with reference to FIG. 1. The reconstructed point cloud is then used for attribute (e.g., color) interpolation, and attributes of points in the point cloud can be encoded based on the reconstructed geometry.

Claims

1. 1. A method for decoding geometry of a 3D point cloud from a bitstream, preferably implemented in a decoder, said method comprising: receiving and decoding a bitstream, the bitstream including octree information and vertex information, the octree information including information about an octree structure of a point cloud volume, and the vertex information including information about the presence and location of vertices at cube edges of leaf nodes of the octree structure; determining triangles by connecting the vertices of one cube associated with a leaf node of the octree structure; voxelizing the triangles to determine points of the point cloud; The method comprises: determining whether additional information included in the bitstream satisfies a predefined condition, the additional information being determined based on the density of the point cloud, preferably a sampling distance d ｓａｍｐｌ and If the predefined condition is met, the sampling distance d ｓａｍｐｌ and expanding at least one triangle along at least one side to perform voxelization based on A method characterized by:

2. 1. A method for encoding a 3D point cloud into a bitstream, preferably implemented in an encoder, said method comprising: obtaining octree information, the octree information including an octree structure of a volume, the volume including a plurality of cubes; obtaining vertex information from the surface of the point cloud of each cube associated with a leaf node, the vertex information including information regarding the presence and location of vertices on the edges of the cube; encoding the octree information and the vertex information into a bitstream; and reconstructing geometry data of the point cloud based on the octree information and vertex information obtained in the encoding process. The step of reconstructing geometry data of the point cloud includes: determining triangles by connecting the vertices of one cube associated with a leaf node of the octree structure; voxelizing the triangles to determine points of the point cloud; The method comprises: determining additional information based on the density of the point cloud, preferably by a sampling distance d ｓａｍｐｌ and evaluating the encoding the side information into the bitstream; determining whether the additional information satisfies a predefined condition; If the predefined condition is met, the sampling distance d ｓａｍｐｌ and expanding at least one triangle along at least one side to perform voxelization based on A method characterized by:

3. The encoding is Trisoup encoding.

3. The method of claim 2.

4. the at least one triangle is expanded by two or three sides to perform voxelization; The method according to any one of claims 1 to 3.

5. Each triangle within the cube is expanded, preferably at least one triangle within each cube of the point cloud having a triangle is expanded; The method according to any one of claims 1 to 4.

6. The expansion is the same for each side or different for at least two sides; The method according to any one of claims 1 to 5.

7. For voxelization [Equation 1] obtaining a voxelization of said points using an algorithm and / or by rounding the coordinates of said points to the nearest integer; The method according to any one of claims 1 to 6.

8. The requirement for a convex hull is −ε ａ ≦u,v,w, and ε ａ >0, u, v, w are the barycentric coordinates of the triangle, and ε ａ is the sampling distance d of the point cloud ｓａｍｐｌ determined based on 8. The method of claim 7.

9. The requirement for a convex hull is −ε ｕ_ａ ≦u, −ε ｖ_ａ ≦v and −ε ｗ_ａ ≦w and ε ｕ_ａ ,ε ｖ_ａ ,ε ｗ_ａ ≧0, u, v, w are the barycentric coordinates of the triangle, and ε ｕ_ａ ≠ε ｗ_ａ and / or ε ｕ_ａ ≠ε ｖ_ａ and / or ε ｖ_ａ ≠ε ｗ_ａ and ε ｕ_ａ , ε ｖ_ａ and ε ｗ_ａ At least one of the sampling distances d ｓａｍｐｌ determined based on 8. The method of claim 7.

10. The extension is provided by halo parameters, and the halo parameters of the extension are d ｓａｍｐｌ / 4, preferably d ｓａｍｐｌ / 8 or less, The method according to any one of claims 1 to 9.

11. the extension is provided by adaptive halo parameters, the extension being preset; The method according to any one of claims 1 to 10.

12. the extension is provided by adaptive halo parameters, the adaptive halo parameters being coded into the bitstream, preferably into a geometry parameter set. The method according to any one of claims 1 to 10.

13. The sampling distance d of the point cloud ｓａｍｐｌ teeth [Equation 2] is determined by N ｌｅａｆ is the number of leaf nodes, and N ｔｏｔａｌ is the number of points in the point cloud, N is the size of the corresponding cube of the leaf node, or the sampling distance d of the point cloud ｓａｍｐｌ is determined by the recursive method, 13. The method according to any one of claims 1 to 12.

14. The at least one triangle has a weighted halo parameter ε ａ_ｔ and performing voxelization along at least one edge based on the weighted halo parameter ε ａ_ｔ is ε ａ_ｔ = ε ａ * determined by t (1<t<4), and ε ａ is the sampling distance d of the point cloud ｓａｍｐｌ and providing an expansion of the at least one triangle, where t is a corresponding weight associated with the sampling distance, and preferably t is set to 2.

14. The method according to any one of claims 1 to 13.

15. The additional information is a flag for enabling or disabling a function of the encoding or decoding method, and is preferably 1 bit.

15. The method according to any one of claims 1 to 14.

16. 1. An encoder for encoding a 3D point cloud into a bitstream, comprising: The encoder includes at least one processor and a memory, and instructions are stored in the memory, and when the instructions are executed by the processor, the steps of the method according to any one of claims 2 to 15 are performed. An encoder characterized by:

17. 1. A decoder for decoding a 3D point cloud from a bitstream, comprising: The decoder includes at least one processor and a memory, and instructions are stored in the memory, and when the instructions are executed by the processor, steps of the method according to claim 1 and any one of claims 3 to 15 that rely on claim 1 are performed. A decoder characterized by:

18. A bitstream encoded by the method according to any one of claims 2 to 15.

19. A computer-readable storage medium containing instructions, which when executed by a processor, cause the steps of the method of any of claims 1 to 15 to be performed. A computer-readable storage medium comprising:

Images