Method and apparatus with entropy continuity for entropy coding in l3c2 lidar point cloud coding

Abstract

A method for entropy decoding according to an embodiment is provided. The method comprises obtaining (131) an indication from a bitstream, wherein the indication indicates that at least one frame has been encoded using a context table; determining (132) the context table from the bitstream; decoding (133) point cloud data from the bitstream based on the context table.

Description

Technical Field

[0001] This application generally relates to point cloud compression, and more particularly to methods and apparatuses that support entropy continuity for entropy encoding and decoding, especially methods and apparatuses that support entropy continuity for entropy encoding and decoding in L3C2 (Low Latency Low Complexity Codec) Lidar point cloud encoding and decoding. Background Technology

[0002] Point clouds and point cloud codecs have recently gained attention as a format for representing 3D data due to their versatility in representing all types of 3D objects or scenes. Therefore, many use cases can be addressed using point clouds, including film post-production, real-time 3D immersive presence or VR / AR applications, free-viewpoint video (e.g., for watching sports), geographic information systems (aka cartography), cultural heritage (storing scans of rare objects in digital form), and autonomous driving (including 3D mapping of the environment and real-time LiDAR data acquisition).

[0003] Until recently, point cloud compression (also known as PCC) had not been solved by the mass market, and there was no standardized point cloud codec available. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11, also known as the Moving Picture Experts Group or MPEG, launched a working project on point cloud compression. This resulted in two standards: [1]MPEG-I Part 5 (ISO / IEC 23090-5) or Video-based point cloud compression (V-PCC). [2] MPEG-I Part 9 (ISO / IEC 23090-9) or geometry-based point cloud compression (G-PCC).

[0004] Both the V-PCC and G-PCC standards completed their first versions at the end of 2020 and will soon be available to the market.

[0005] The concept of improving the compression performance and runtime of point cloud encoding and decoding schemes by refining entropy encoding and decoding methods (e.g., L3C2) is welcome. Summary of the Invention

[0006] A method for entropy decoding according to an embodiment is provided. The method includes obtaining an indication from a bitstream, wherein the indication indicates that at least one frame has been encoded using a context table; determining the context table from the bitstream; and decoding point cloud data from the bitstream based on the context table.

[0007] Furthermore, a method for entropy encoding according to an embodiment is provided. The method includes entropy encoding point cloud data of at least one frame into a bitstream using a context table; generating the bitstream such that the bitstream further includes an indication that at least one frame has been encoded using the context table; and further including information about the context table.

[0008] Furthermore, an apparatus for entropy decoding according to an embodiment is provided. The apparatus includes at least one processor configured to obtain an indication from a bitstream, wherein the indication indicates that at least one frame has been encoded using a context table; determine the context table from the bitstream; and decode point cloud data from the bitstream based on the context table.

[0009] Furthermore, an apparatus for entropy encoding according to an embodiment is provided. The apparatus includes at least one processor configured to entropy encode point cloud data of at least one frame into a bitstream using a context table; generate the bitstream such that the bitstream further includes an indication that at least one frame has been encoded using the context table; and further include information about the context table.

[0010] Furthermore, a computer program product is provided, including instructions that, when executed by one or more processors, perform a method for entropy decoding according to an embodiment. The method includes obtaining an indication from a bitstream, wherein the indication indicates that at least one frame has been encoded using a context table; determining the context table from the bitstream; and decoding point cloud data from the bitstream based on the context table.

[0011] Furthermore, a non-transitory digital storage medium is provided, carrying instructions for executing program code for entropy decoding according to an embodiment. The method includes obtaining an indication from a bitstream, wherein the indication indicates that at least one frame has been encoded using a context table; determining the context table from the bitstream; and decoding point cloud data from the bitstream based on the context table.

[0012] Furthermore, a computer program product is provided, including instructions that, when executed by one or more processors, perform an entropy encoding method according to an embodiment. The method includes entropy encoding point cloud data of at least one frame into a bitstream using a context table; generating the bitstream such that the bitstream further includes an indication that at least one frame has been encoded using the context table; and causing the bitstream to further include information about the context table.

[0013] Furthermore, a non-transitory digital storage medium is provided, carrying instructions for executing program code for entropy encoding according to an embodiment. The method includes entropy encoding point cloud data of at least one frame into a bitstream using a context table; generating the bitstream such that the bitstream further includes an indication that at least one frame has been encoded using the context table; and causing the bitstream to further include information about the context table.

[0014] This disclosure provides an entropy continuity method between frames and a better evolutionary prediction model (compared to the initialization prediction model), which can be reused in the entropy encoding / decoding information of later frames. By doing so, better compression performance and less runtime can be achieved. Attached Figure Description

[0015] Figure 1 A spin lidar head is shown, which includes several spin lasers for probing the environment.

[0016] Figure 2 The elevation angle θ of the spin laser is shown.

[0017] Figure 3 The 2D angle (φ, θ) representation of a point acquired by a spin lidar is shown.

[0018] Figure 4 The points collected in the discrete (φ, θ) representation are shown.

[0019] Figure 5 This shows 3D xyz coordinates and angle-based coordinates (r). 2D , φ, θ) or (r 3D , φ, θ).

[0020] Figure 6 The acquisition order of azimuth angle φ and laser index λ is shown.

[0021] Figure 7 The actual acquired azimuth angle φ and the point in the laser index λ are shown.

[0022] Figure 8 The point is shown at a rough azimuth angle φ c And the sorting in the laser index λ.

[0023] Figure 9 The order is shown in the first dictionary (φ) c The point cloud representation of the difference Δnext of λ).

[0024] Figure 10 An overview of the encoding method for implementing the aforementioned disclosure provided in [3] is shown.

[0025] Figure 11An overview of the decoding method implemented in the aforementioned disclosure provided in [3] is shown.

[0026] Figure 12 A method for entropy coding according to an embodiment is shown.

[0027] Figure 13 A method for entropy decoding according to an embodiment is shown.

[0028] Figure 14 A bitstream structure of a frame having a slice is shown according to at least one exemplary embodiment.

[0029] Figure 15 A bitstream structure of a frame having multiple slices is shown according to at least one exemplary embodiment.

[0030] Figure 16 A context-adaptive entropy decoder according to at least one exemplary embodiment is shown. Detailed Implementation

[0031] First, the principles upon which the embodiments are based are described: A point cloud is a collection of points in 3D space, with optional additional values ​​attached to each point. These additional values ​​are often referred to as point attributes. Therefore, a point cloud is a combination of geometry (the 3D position of each point) and attributes.

[0032] The attributes can be, for example, three-component color, material properties (such as reflectivity), and / or two-component normal vectors of the surface associated with the point.

[0033] Point clouds can be captured by various types of devices, such as camera arrays, depth sensors, LiDAR, and scanners, or they can be generated by computers (e.g., in film post-production). Depending on the use case, a point cloud can have thousands to billions of points for mapping applications.

[0034] The raw representation of point clouds requires a very large number of bits per point, with at least a dozen bits per spatial component X, Y, or Z, and optionally even more bits for attributes(s), such as three times 10 bits for color. Practical deployment of point cloud-based applications necessitates compression techniques capable of storing and distributing point clouds with a reasonable storage and transport infrastructure.

[0035] For distribution and visualization by end users, such as on AR / VR glasses or any other 3D-enabled devices, compression can be lossy (as in video compression). Other use cases do require lossless compression, such as medical applications or autonomous driving, to avoid altering the results of decisions derived from the analysis of compressed and transmitted point clouds.

[0036] The V-PCC encoding / decoding method compresses point clouds by performing multiple projections on 3D objects to obtain 2D tiles packed into an image (or video when processing moving point clouds). The resulting image or video is then compressed using an existing image / video codec, allowing full utilization of already deployed image and video solutions. By its very nature, V-PCC is only efficient on dense and continuous point clouds because image / video codecs cannot compress non-smooth tiles, such as those obtained by projecting sparse geometry data from LiDAR.

[0037] The G-PCC encoding method has two schemes for compressing geometry.

[0038] The first approach is based on an occupancy tree (octree / quadtree / binary tree) representing the geometry of the point cloud. Occupied nodes are split until a certain size is reached, and the occupied leaf nodes provide the location of points, typically at the center of these nodes. High levels of compression of dense point clouds can be achieved by using neighbor-based prediction techniques. Sparse point clouds can also be addressed by directly encoding and decoding the locations of points with non-minimum sizes within nodes, stopping tree construction when only isolated points exist in a node; this technique is called Direct Codec Mode (DCM).

[0039] The second approach is based on a prediction tree, where each node represents the 3D location of a point, and the relationship between nodes is a spatial prediction from parent to child. This method can only solve sparse point clouds and offers the advantages of lower latency and simpler decoding than occupancy-based methods. However, compared to the first occupancy-based method, the compression performance is only slightly better, and the encoding is also complex, focusing on finding the best predictor from a long list of potential predictors when constructing the prediction tree.

[0040] In both schemes, attribute (decoding) encoding is performed after geometry (decoding) encoding, resulting in two encoding / decoding operations. Therefore, low latency is achieved by slicing the 3D space into independently encoded / decoded sub-volumes without requiring prediction between sub-volumes. When using many slices, this significantly impacts compression performance.

[0041] A key use case is the transmission of LiDAR data collected by moving vehicles. This typically requires a simple, low-latency embedded encoder. Simplicity is crucial because the encoder may be deployed on computing units performing other processing in parallel (such as (semi-)autonomous driving), limiting the processing power available to the point cloud encoder. Low latency is also required to allow for rapid transmission from vehicle to cloud, enabling real-time viewing of local traffic based on multi-vehicle data and making sufficiently fast decisions based on traffic information. While using 5G can reduce transmission latency sufficiently, the encoder itself should not introduce excessive latency due to encoding and decoding. Furthermore, compression performance is extremely important because the data stream from millions of vehicles to the cloud is expected to be enormous.

[0042] Combining the simplicity, low latency, and compression performance of encoders and decoders remains a problem that existing point cloud codecs have not yet satisfactorily solved.

[0043] Compression of point clouds acquired by a rotating sensor is considered in the following text.

[0044] Specific priors related to LiDAR data acquisition have been utilized in G-PCC and have resulted in very significant compression gains. The first technique involves the vertical angle (relative to the horizontal ground) of the spin lidar acquisition (e.g., ...). Figure 1 (As depicted): The angle θ is fixed, such as... Figure 2 As shown. In particular, Figure 1 A spin lidar head is shown, which includes several spin lasers for probing the environment. Figure 2 The elevation angle θ of the spin laser is shown.

[0045] In fact, the point cloud acquired by lidar is in spherical coordinates (r 3D The representation in (, φ, θ) is not 3D, but quasi-2D, where r 3D It is the 3D distance from the point to the center of the lidar, φ is the azimuth angle, and θ is the elevation angle.

[0046] G-PCC further developed a second technology that utilizes the regularity sensed by the laser during lidar spin, such as... Figure 3 What is depicted. In particular, Figure 3 The diagram shows a 2D angular (φ, θ) representation of points acquired by a spin lidar. A regular distribution along the azimuth angle φ has been observed in the lidar-acquired data. This regularity is used to obtain a quasi-1D representation of the point cloud, where, until noise, only the radius r... 3D The values ​​belong to a continuous range, while the angles φ and θ only take discrete values.

[0047] Figure 4The points acquired are shown in a discrete (φ, θ) representation. Essentially, the point cloud geometry can be represented on a 2D discrete angular plane (see [reference]). Figure 4 (and the radius value of each point). This quasi-1D property has been utilized in G-PCC in both the occupancy tree and the prediction tree by predicting the position of the current point relative to already encoded points in spherical coordinates using the discrete nature of angles.

[0048] In practice, occupancy trees heavily utilize DCM, and employ context-adaptive entropy codecs to entropy code the direct location of points within a node. Context is obtained through a local transformation from point location to angular (φ, θ) coordinates.

[0049] Figure 5 This shows 3D xyz coordinates and angle-based coordinates (r). 2D , φ, θ) or (r 3D (, φ, θ). Direct diagonal coordinates of the prediction tree (r 2D Encoding and decoding are performed using r2D, where r2D is the projected radius on the horizontal xy plane (see [reference]). Figure 5 Then, it is converted to (x,y,z) and then the xyz residual is encoded and decoded to resolve coordinate transformation errors, laser angle approximations, and noise.

[0050] As explained above, there are two main types of encoding and decoding structures in the existing technology: occupancy tree and prediction tree.

[0051] In the framework of a rotating lidar system, a coarse single-chain representation of the point cloud acquired by lidar has been proposed in the aforementioned disclosure [3].

[0052] In the aforementioned disclosure [3], all lasers are encoded and decoded at once by using the acquisition sequence, such as Figure 6 This is shown in the (φ, λ) plane. Specifically, Figure 6 The acquisition order of azimuth angle φ and laser index λ is shown. Due to the regular rotation of the lidar head and the continuous acquisition of each laser at fixed time intervals, the azimuth distance between two points detected by the same laser is a multiple of the basic azimuth angle shift Δφ.

[0053] Based on prior data collection, we can first encode and decode the coarse representation, rather than directly encoding and decoding the point locations.

[0054] For example, a rough representation of the point cloud geometry (φ) can be used. c , λ) is first used in the coarse azimuth angle φ c The points are sorted in the dictionary order (also known as lexicographical order) of the laser index (or sensing elevation index) λ.

[0055] Figure 7 The diagram shows the actual acquired azimuth angle φ and the point in the laser index λ. Schematally, in the (φ, λ) plane, according to... Figure 7 The sequence of acquisition points is already shown. Due to the regular rotation of the lidar head and the continuous acquisition by each laser at fixed time intervals, the azimuth distance between two points detected by the same laser is a multiple of the basic azimuth shift Δφ. In reality, not all points can be acquired; that is, the laser beam may not be reflected, there may be acquisition noise, and the laser may not be perfectly aligned. Actual data is as follows... Figure 7 As shown.

[0056] Coarse angle φ c It can be obtained simply by quantizing φ as follows: φ c = round (φ / Δφ), And the order index o(P) of point P is obtained by the following formula. o(P) = φ c *N 激光器 + λ Where N 激光器 It represents the number of lasers, where λ is the range [0, N] of the points P that have been acquired. 激光器 The index of the laser in [-1]. The codec monotonically encodes point P in its order o(P), for example, using ascending order. Therefore, according to Figure 8 The described order is used to encode and decode the points. Specifically, Figure 8 The point is shown at a rough azimuth angle φ c And the sorting in the laser index λ.

[0057] plane (φ) c The coarse representation in λ can be encoded and decoded using the following methods. -N points 点 - The first sampling point (φ) c , λ) value - N points between the current point and the next point, arranged lexicographically. 点 -1 consecutive difference Δnext, such as Figure 9 What is depicted.

[0058] In particular, Figure 9 The order is shown in the first dictionary (φ) c The point cloud representation of the difference Δnext of λ. The coarse representation consists of continuous differences Δnext, and the compression of the coarse representation is basically based on the compression of continuous positive values ​​of Δnext.

[0059] A brief description of the complete point cloud codec based on the aforementioned publicly available information. Figure 10 An overview of the encoding method based on the public content in [3] is shown, in particular a brief description of the complete point cloud codec. Specifically, Figure 10 An overview of the encoding method for implementing the aforementioned disclosure provided in [3] is shown.

[0060] First, based on the knowledge of the lidar sensor settings, the encoder converts the xyz point positions into the laser index λ and the coarse angle φ. c and radius r 2D The difference Δnext is determined, and this disclosure is applied to encode Δnext, along with useful information about sensor settings (e.g., Δφ and laser elevation angle), into a bitstream.

[0061] Second, calculate the reconstructed azimuth angle φ. rec It can directly quantize Δφ*φ from the solution of the coarse angle. c Obtain. Optionally, the residual φ can be... res Calculated as difference φ - Δφ*φ c And encode it into a bitstream. Before encoding, the residual can be quantized into Q(φ). res In this case, the reconstructed azimuth angle φ rec Obtained through the following formula φ rec = Δφ*φ c + IQ(Q(φ res )) Where IQ represents the inverse quantization process.

[0062] Optionally, after quantization, the radius r (here, r) 2D The value is also encoded as Q(r). It is dequantized to obtain the reconstructed radius. r rec = IQ(Q(r)).

[0063] Third, the reconstructed azimuth angle φ rec and the reconstructed radius r rec Transform back to xy to obtain estimates of the x and y positions of the point.

[0064] x estim = r rec cos(φ rec ), y estim = r rec sin(φ rec ) Calculate the residual x relative to the original point position xy res and y res xres = x - x estim y res = y - y estim , And it is encoded into a bitstream.

[0065] Fourth, and finally, vertical estimation of z. estim It is obtained from the laser angle θ(λ) by the following formula. z estim = r rec tan(θ(λ)) And calculate the residual z relative to the original point position z. res z res = z - z estim And it is encoded into a bitstream.

[0066] Figure 11 An overview of the associated decoding methods based on the public content of [3] is shown. In particular, Figure 11 An overview of the decoding method implemented in the aforementioned disclosure provided in [3] is shown. The encoding process is straightforward once understood.

[0067] First, the decoder decodes useful information about the sensor setup (e.g., Δφ and laser elevation angle) and the difference Δnext from the bitstream using this disclosure. Then, as explained in the next section, the laser index λ and the coarse angle φ are... c The value is obtained from Δnext.

[0068] Second, calculate the reconstructed azimuth angle φ. rec It can directly quantize Δφ*φ from the solution of the coarse angle. c The azimuth residual can be obtained. Optionally, it can be decoded from the bitstream. The decoded residual can be a quantized version of the residual, Q(φ). res ), and the reconstructed azimuth angle φ rec Obtained through the following formula φ rec = Δφ*φ c + IQ(Q(φ res )) Where IQ represents the inverse quantization process.

[0069] radius r (here r) 2D The radius of the encoder and decoder is also decoded. The radius can be a quantized version of the radius, Q(r). It is dequantized to obtain the reconstructed radius. r rec = IQ(Q(r)).

[0070] Alternatively, the radius can be predicted (e.g., the radius encoded and decoded as described above), and the radius residual can be encoded and decoded instead of the radius.

[0071] Third, the reconstructed azimuth angle φ rec and the reconstructed radius r rec Transform back to xy to obtain estimates of the x and y positions of the point.

[0072] x estim = r rec cos(φ rec ), y estim = r rec sin(φ rec ) Decoding residual x from bitstream res and y res And calculate the horizontal position x of the decoded point. dec and y dec x dec = x estim + x res y dec = y estim + y res .

[0073] Fourth, and finally, vertical estimation of z. estim It is obtained from the laser angle θ(λ) by the following formula. z estim = r rec tan(θ(λ)), Decoding residual z from bitstream res And calculate the vertical position z of the decoded point. dec z dec = z estim + z res .

[0074] In the current L3C2 encoding and decoding scheme, entropy encoding and decoding methods (including...) Figure 10 The entropy codes labeled 101, 102, and 103 in and in Figure 11 The entropy decoding (marked as 111, 112, and 113) is not optimal because it reinitializes the entropy context table at the beginning of encoding and decoding each frame, resulting in significant time consumption for reinitializing the order difference. Quantified residuals (such as Q(φ) resThe entropy context table used for encoding and decoding Q(r) and the residuals of reconstruction in the (x,y,z) domain is undesirable for the low latency requirements of Lidar point cloud encoding and decoding. Furthermore, reinitializing the entropy context table for each frame in the sequence does not utilize the well-predicted probabilistic models generated and updated during the encoding and decoding of previous frames, which degrades compression performance.

[0075] Specific embodiments are described below.

[0076] Figure 12 A method for entropy coding according to an embodiment is shown.

[0077] The method includes: Use a context table to entropy encode (121) the point cloud data of at least one frame into a bit stream; Generate (122) bitstream, such that the bitstream further includes an indication that at least one frame has been encoded using a context table; and such that the bitstream further includes information about the context table.

[0078] According to an embodiment, the indication may, for example, indicate whether the context table of point cloud data that has been used to encode at least one frame is the context table of point cloud data of a previous frame that has been used to encode at least one frame.

[0079] In an embodiment, entropy encoding (121) of point cloud data may include, for example, using a context table to entropy encode the geometric location and / or attribute information of points in the point cloud data.

[0080] In other words, this method can also be used only in the geometric information context, or only in the attribute information context, or both the geometric information context and the attribute information context.

[0081] According to an embodiment, the method may, for example, include generating a (122) bitstream such that the bitstream includes a first flag (e.g., inter_entropy_continuation_enabled_flag) indicating that the context table of point cloud data already used to encode at least one frame is: A context table of point cloud data from previous frames that have already been used to encode at least one frame, or A context table of point cloud data for any previous frame that has not yet been used to encode at least one frame.

[0082] In an embodiment, the method may, for example, include generating a (122) bit stream such that the sequence parameter set portion of the bit stream includes a first flag (e.g., inter_entropy_continuation_enabled_flag).

[0083] Figure 13 A method for entropy decoding according to an embodiment is shown.

[0084] The method includes: Obtain (131) indication from the bitstream, wherein the indication indicates that at least one frame has been encoded using a context table; Determine the context table (132) from the bitstream; Decode (133) point cloud data from the bitstream based on the context table.

[0085] According to an embodiment, the indication indicates whether the context table of point cloud data that has been used to encode at least one frame is the context table of point cloud data of a previous frame that has been used to encode at least one frame.

[0086] In an embodiment, entropy decoding (133) of point cloud data may include, for example, using a context table to entropy decode the geometric location and / or attribute information of points in the point cloud data.

[0087] According to an embodiment, this indication may be, for example, a first flag (e.g., inter_entropy_continuation_enabled_flag) indicating that the context table of point cloud data that has been used to encode at least one frame is: A context table of point cloud data from previous frames that have already been used to encode at least one frame, or A context table of point cloud data for any previous frame that has not yet been used to encode at least one frame.

[0088] In an embodiment, the method may include, for example, obtaining a first flag (e.g., inter_entropy_continuation_enabled_flag) from a sequence parameter set portion of the bitstream.

[0089] According to an embodiment, when the first flag indicates that entropy is continuously enabled, one or more entropy context tables for decoding geometric information can be initialized for example for the first frame, and the generated and updated one or more entropy context tables for encoding and decoding geometric information can be reused for one or more subsequent frames.

[0090] In an embodiment, when the first flag indicates that entropy is continuously disabled, a separate context table can be determined, for example, for each of multiple frames.

[0091] According to an embodiment, when the first flag indicates that entropy continuity is enabled, a second flag (e.g., interPredictionEnabledFlag) can be obtained from the bitstream, which indicates whether the current frame is the first frame of a group of frames in a group of pictures (GOP).

[0092] In an embodiment, when a first flag indicates that entropy continuation is enabled, and when a second flag indicates that the current frame is not the first frame in the image group, a context table that has been used for entropy decoding of previous frames for the current frame can be reused, for example.

[0093] According to an embodiment, the method may include, for example, determining one or more context tables for entropy attribute decoding.

[0094] In an embodiment, the method may include, for example, obtaining a third flag (e.g., interPredictionEnabledAttrFlag) from the bitstream, which indicates whether the context table has been used for entropy attribute encoding.

[0095] According to an embodiment, the third flag may have been obtained, for example, from the header of the attribute data unit of the bitstream.

[0096] In an embodiment, the method may, for example, include employing multiple context tables to perform entropy decoding on at least one of the following: Geometric information generated in the L3C2 encoding / decoding scheme Difference in encoding and decoding order Multiple quantized residuals.

[0097] Specific embodiments are provided below.

[0098] An entropy continuity method is proposed to introduce between point cloud frames in a sequence (e.g., in the current L3C2 point cloud encoding and decoding scheme).

[0099] In the proposed method, a flag is introduced to indicate whether the entropy continuity method is enabled between frames of the sequence (e.g., inter_entropy_continuation_enabled_flag).

[0100] If a flag (e.g., inter_entropy_continuation_enabled_flag) is true, the entropy continuation method between frames of the sequence is enabled, and it initializes only the entropy context table used to encode and decode the geometry of the first frame of the sequence, and reuses the generated and updated entropy context table to encode and decode the geometry of later (e.g., subsequent) frames of the sequence in the L3C2 encoding and decoding scheme.

[0101] Otherwise, if the flag (e.g., inter_entropy_continuation_enabled_flag) is false, the entropy continuation method between frames is disabled, and it initializes the entropy context table used for encoding and decoding the geometry information of all frames, which is the same as in the current L3C2 encoding and decoding scheme.

[0102] For example, the proposed method can be used, for instance, in an inter-prediction framework for L3C2 point cloud encoding and decoding. For example, a suitable framework is described in the aforementioned disclosure [4], which introduces a flag (e.g., inter_prediction_enabled_flag) indicating whether an inter-prediction method for Lidar data sequences is enabled.

[0103] In a preferred embodiment of the proposed method, the proposed entropy continuity method for L3C2 can be enabled, for example, only if the flag inter_prediction_enabled_flag is true.

[0104] On the encoder side, in order to encode each frame, the following operations can be performed, for example: First, the values ​​of flags (including inter_entropy_continuation_enabled_flag, etc.) are set to true or false, and then the flags (including inter_entropy_continuation_enabled_flag, etc.) are encoded into the bitstream.

[0105] Then, before encoding the geometric information in the L3C2 point cloud, it determines the context table of the geometric information that needs to be entropy encoded.

[0106] For example, geometric information includes order difference. Quantified residuals (such as Q(φ) res The residuals of reconstruction in the domains (x,y,z) and (Q(r)).

[0107] The determination of the context table used for geometric information can, for example, be based on the previously described flag inter_entropy_continuation_enabled_flag.

[0108] If the flag `inter_entropy_continuation_enabled_flag` is true, it further considers the flag `interPredictionEnabledFlag`, which indicates whether the current encoded frame is the first frame in a GOP (e.g., a group of images), where a GOP is a unit of sequence of point cloud frames in which inter-frame predictions are implicit. And if the current encoded frame is not the first frame in a GOP (if `interPredictionEnabledFlag` is true), it reuses the generated and updated context tables to entropy-encode the geometry of the current frame. Otherwise, if the flag `interPredictionEnabledFlag` is false (meaning the current encoded frame is the first frame in a GOP), it initializes all context tables used for entropy encoding of the geometry of the current frame. Otherwise, if the flag `inter_entropy_continuation_enabled_flag` is false, it initializes all context tables used for entropy encoding of the geometry of the current frame.

[0109] Then, it entropy-encodes the geometric information based on the defined context table mentioned above, and it also entropy-encodes the attribute information based on the same initialization context table as in the current L3C2 encoding scheme.

[0110] Furthermore, on the decoder side, in order to decode each frame, the following operations can be performed, for example: First, the values ​​of flags (including inter_entropy_continuation_enabled_flag, etc.) are decoded from the bitstream.

[0111] Then, before decoding the geometric information in the L3C2 point cloud encoding and decoding, it determines the context table of the geometric information that needs to be entropy decoded.

[0112] For example, geometric information includes order difference. Quantified residuals (such as Q(φ) res The residuals of reconstruction in the domains (x,y,z) and (Q(r)).

[0113] Furthermore, the determination of the context table used for geometric information is based on the previously described flag inter_entropy_continuation_enabled_flag.

[0114] If the flag `inter_entropy_continuation_enabled_flag` is true, it further considers the flag `interPredictionEnabledFlag`, which indicates whether the current codec frame is the first frame in a Group of Points (GOPs), where a GOP is a unit of sequence of point cloud frames that implicitly contain inter-frame predictions. If the current codec frame is not the first frame in a GOP (and `interPredictionEnabledFlag` is also true), it reuses the generated and updated context tables for entropy decoding of the current frame's geometry; otherwise, if `interPredictionEnabledFlag` is false (meaning the current codec frame is the first frame in a GOP), it initializes all context tables used for entropy decoding of the current frame's geometry. Alternatively, if `inter_entropy_continuation_enabled_flag` is false, it initializes all context tables used for entropy decoding of the current frame's geometry.

[0115] Then, it performs entropy decoding on the geometric information based on the defined context table mentioned above, and it also performs entropy decoding on the attribute information based on the same initialization context table as in the current L3C2 encoding and decoding scheme.

[0116] In a preferred embodiment, the flag inter_prediction_enabled_flag is placed in the GPS portion of the bitstream.

[0117] In a preferred embodiment, the flag inter_entropy_continuation_enabled_flag is placed in the SPS (Sequence Parameter Set) portion of the bitstream.

[0118] In a preferred embodiment, the flag interPredictionEnabledFlag is placed in the header of the geometry data unit.

[0119] In the following text, exemplary embodiments are provided in particular.

[0120] In a first exemplary aspect of a particular embodiment, the flag positions in a bitstream according to the proposed method are described.

[0121] exist Figure 15The diagram illustrates the bitstream structure of a frame with multiple slices, where the SPS (Sequence Parameter Set) contains parameters for signaling information (such as flags) for encoding and decoding the entire sequence, the GPS (Geometric Parameter Set) contains parameters for signaling information (such as flags) for encoding and decoding geometric information, and the APS (Attribute Parameter Set) contains parameters for signaling information (such as flags) for encoding and decoding attribute information. The geometric data unit and attribute data unit are the encoded and decoded bitstreams of the geometric and attribute information of the slice, respectively.

[0122] In a preferred embodiment, the flag `inter_entropy_continuation_enabled_flag` is placed in the SPS (Sequence Parameter Set) section of the bitstream described above. Furthermore, in a preferred embodiment, the flag `interPredictionEnabledFlag` is placed in the header of the geometric data unit.

[0123] In another variant, the flag `interPredictionEnabledFlag` is determined based on whether the current codec slice in the frame is the first slice in the GOP. This is because in many cases, a single frame of LiDAR point cloud contains only one slice. Figure 14 The image shows the bitstream structure of this point cloud frame. The flag inter_entropy_continuation_enabled_flag is also placed in the SPS (Sequence Parameter Set) section, and the flag interPredictionEnabledFlag is placed in the header of the geometric data unit.

[0124] In a second exemplary aspect of a particular embodiment, it is described that entropy continuity for attribute encoding and decoding is determined according to the proposed method.

[0125] In one embodiment, the proposed entropy continuation method is also used for attribute encoding / decoding. Specifically, before encoding / decoding attribute information in L3C2 point cloud encoding / decoding, it determines the context table of attribute information for frames requiring entropy encoding / decoding based on the flags inter_entropy_continuation_enabled_flag and interPredictionEnabledFlag used in geometry encoding / decoding. Only when both flags are true does it reuse the generated and updated context table to entropy-encode / decode the attribute geometry information of the current slice / frame; otherwise, it initializes all context tables used for entropy-encoding / decoding the attribute information of the current slice / frame.

[0126] In another variant, the context table for determining the frame's attribute information is based on the flags `inter_entropy_continuation_enabled_flag` and `interPredictionEnabledAttrFlag`. `inter_entropy_continuation_enabled_flag` controls entropy continuity for both geometry and attribute encoding / decoding, while `interPredictionEnabledAttrFlag` is only used for attribute encoding / decoding. Furthermore, `interPredictionEnabledAttrFlag` provides attribute encoding / decoding with more flexibility to implement or not implement the entropy continuity approach. This flag is then placed in the header of the attribute data unit.

[0127] The following describes context entropy encoding and decoding according to a particular exemplary embodiment.

[0128] This section explains in detail the so-called "context entropy encoding and decoding". Furthermore, the context entropy encoding and decoding process is implemented based on an entropy encoder / decoder. Figure 16 A block diagram of the context-adaptive entropy decoder is provided. The associated encoder is simply obtained by restoring the arrows from the bitstream back to the bitstream.

[0129] First, based on the already decoded information, a selection process is used to choose the context to provide the context index ctxIdx. This has N... ctx The context table of an entry stores the probability associated with the context, and the probability p ctxIdx It is obtained as the ctxIdx entry in the context table. The binary symbol s is obtained by the entropy decoder using probability p. ctxIdx Decode from bitstream.

[0130] When decoding symbol s, it is done by using s and p ctxIdx Update probability p for the update process of entries. ctxIdx This update process is typically performed using an update table. The probability of the update replaces the ctxIdx-th entry in the context table. Then, another symbol can be decoded, and so on.

[0131] The update loop that returns the context table is a bottleneck in the decoding workflow because another symbol can only be decoded after the update is performed.

[0132] Based on specific exemplary embodiments, a method is provided for decoding point information from a bitstream point cloud sequence, wherein the method includes: determining a context table for entropy decoding of point cloud frames based on an entropy continuity method; entropy decoding of the geometric position and attribute information of points from the bitstream; and reconstructing the points in the point cloud based on the decoded geometric and attribute information.

[0133] This method may include, for example, additional features: According to an embodiment, the entropy context table determined based on the entropy continuity method can, for example, be used to encode and decode geometric information generated in the L3C2 encoding and decoding scheme, such as for order differences. Quantified residuals (such as Q(φ) res Encode and decode the residuals of reconstruction in the (x,y,z) domain, including Q(r) and Q(r).

[0134] In an embodiment, the entropy continuation method may, for example, introduce a flag (e.g., inter_entropy_continuation_enabled_flag) indicating whether the entropy continuation method between frames of the sequence is enabled. If the flag (e.g., inter_entropy_continuation_enabled_flag) is true, the entropy continuation method between frames of the sequence is enabled, and it initializes only the entropy context table used for encoding and decoding the geometry of the first frame / slice of the sequence, and reuses the generated and updated entropy context table for encoding and decoding the geometry of later frames of the sequence in the L3C2 encoding / decoding scheme.

[0135] According to a specific implementation, the flag inter_entropy_continuation_enabled_flag is placed in the SPS (Sequence Parameter Set) portion of the bitstream.

[0136] In a particular embodiment, if a flag (e.g., inter_entropy_continuation_enabled_flag) is false, the entropy continuation method between frames can be disabled, and it initializes an entropy context table for encoding and decoding geometric information for all frames.

[0137] According to a specific embodiment, when a flag (e.g., inter_entropy_continuation_enabled_flag) is true, in order to determine whether the current frame is the first frame / slice of the sequence, a flag interPredictionEnabledFlag is introduced to indicate whether the currently encoded / decoded frame / slice is the first frame / slice in the GOP.

[0138] In embodiments, the proposed entropy continuity method can also be used for attribute encoding and decoding, for example. It further determines the context table of attribute information for frames requiring entropy encoding and decoding based on the flags inter_entropy_continuation_enabled_flag and interPredictionEnabledFlag used in geometry encoding and decoding. Only when both flags are true does it reuse the generated and updated context table to entropy-encode and decode the attribute information of the current slice / frame.

[0139] In certain embodiments, the context table for determining the attribute information of a frame can be based, for example, on the flags inter_entropy_continuation_enabled_flag and interPredictionEnabledAttrFlag. The interPredictionEnabledAttrFlag is used only for attribute encoding and decoding, and it can give attribute encoding and decoding greater flexibility to implement or not implement the entropy continuity method.

[0140] More specifically, the flag interPredictionEnabledAttrFlag can be placed, for example, in the header of the attribute data cell.

[0141] abbreviation PCC: Point Cloud Compression V-PCC: Video-based PCC G-PCC: Geometry-based PCC Lidar: Light Detection and Ranging = A method of measuring distance (ranging) by illuminating a target with a laser and using sensors to measure the reflection. References [1] MPEG-I Part 5 (ISO / IEC 23090-5) or Video-based Point Cloud Compression (V-PCC).

[0142] [2] MPEG-I Part 9 (ISO / IEC 23090-9) or geometry-based point cloud compression (G-PCC).

[0143] [3] European Patent Application, Application No. 20306672.5, European Patent Application Publication No. EP 4020816 A1.

[0144] [4]PCT / CN2024 / 075584.

Claims

1. A method for entropy decoding, wherein the method comprises: Obtain an indication (131) from the bitstream, wherein the indication indicates that at least one frame has been encoded using a context table; Determine the context table (132) from the bitstream; Decode (133) point cloud data from the bitstream based on the context table; The indication mentioned therein is a first flag, which indicates that the context table that has been used to encode the point cloud data of the at least one frame is: A context table of point cloud data from previous frames that have already been used to encode the at least one frame, or A context table of point cloud data for any previous frame that has not yet been used to encode the at least one frame; The method includes using the context table to perform entropy decoding on the geometric information generated in the L3C2 encoding / decoding scheme.

2. The method according to claim 1, The indication indicates whether the context table that has been used to encode the point cloud data of the at least one frame is a context table that has been used to encode the point cloud data of a previous frame.

3. The method according to claim 1, Decoding from the bitstream based on the context table (133) includes entropy decoding of the geometric position and / or attribute information of the points in the point cloud data using the context table.

4. The method according to claim 1, The method includes obtaining the first flag from a sequence parameter set portion of the bit stream.

5. The method according to claim 4, in, When the first flag indicates that entropy is continuously enabled, one or more entropy context tables for decoding geometric information are initialized for the first frame, and the generated and updated one or more entropy context tables for encoding and decoding geometric information are reused for one or more subsequent frames.

6. The method according to any one of claims 1 to 5, in, When the first flag indicates that the entropy is continuously disabled, a separate context table is determined for each of the multiple frames.

7. The method according to any one of claims 1 to 5, in, When the first flag indicates that the entropy is continuously enabled, a second flag is obtained from the bitstream, the second flag indicating whether the current frame is the first frame of a group of frames in a group of pictures (GOP).

8. The method according to claim 7, in, When the first flag indicates that entropy continuation is enabled, and when the second flag indicates that the current frame is not the first frame in the image group, the context table that has been used to entropy decode the previous frame of the current frame is reused.

9. The method according to any one of claims 1 to 5, The method includes determining one or more context tables for entropy attribute decoding.

10. The method according to claim 9, The method includes obtaining a third flag from the bitstream, the third flag indicating whether the context table has been used for entropy attribute encoding.

11. The method according to claim 10, The third flag mentioned therein has been obtained from the header of the attribute data unit of the bitstream.

12. The method according to any one of claims 1 to 5, The method further includes using the context table to perform entropy decoding on at least one of the following: Difference in encoding and decoding order Multiple quantized residuals.

13. A method for entropy coding, wherein the method comprises: Use a context table to entropy encode (121) the point cloud data of at least one frame into a bit stream; Generate (122) the bitstream such that the bitstream further includes an indication that the at least one frame has been encoded using the context table; and such that the bitstream further includes information about the context table; The method includes generating (122) the bitstream such that the bitstream includes a first flag indicating that the context table of the point cloud data already used to encode the at least one frame is: A context table of point cloud data from previous frames that have already been used to encode the at least one frame, or A context table of point cloud data for any previous frame that has not yet been used to encode the at least one frame; The method includes using the context table to entropy encode the geometric information generated in the L3C2 encoding / decoding scheme.

14. The method according to claim 13, The indication indicates whether the context table that has been used to encode the point cloud data of the at least one frame is a context table that has been used to encode the point cloud data of a previous frame.

15. The method according to claim 13 or 14, Entropy encoding of the point cloud data (121) includes using the context table to entropy encode the geometric position and / or attribute information of the points in the point cloud data.

16. The method according to claim 13, The method includes generating (122) the bit stream such that a sequence parameter set portion of the bit stream includes the first flag.

17. An apparatus for entropy decoding, wherein the apparatus includes at least one processor, the at least one processor being configured to: Obtain an indication (131) from the bitstream, wherein the indication indicates that at least one frame has been encoded using a context table; Determine the context table (132) from the bitstream; Decode (133) point cloud data from the bitstream based on the context table; The indication mentioned therein is a first flag, which indicates that the context table that has been used to encode the point cloud data of the at least one frame is: A context table of point cloud data from previous frames that have already been used to encode the at least one frame, or A context table of point cloud data for any previous frame that has not yet been used to encode the at least one frame; The at least one processor is further configured to: The context table is used to perform entropy decoding on the geometric information generated in the L3C2 encoding and decoding scheme.

18. An apparatus for entropy coding, wherein the apparatus includes at least one processor, the at least one processor being configured to: Use a context table to entropy encode (121) the point cloud data of at least one frame into a bit stream; Generate (122) the bitstream such that the bitstream further includes an indication that the at least one frame has been encoded using the context table; and such that the bitstream further includes information about the context table; The at least one processor is specifically configured as follows: Generate (122) the bitstream such that the bitstream includes a first flag indicating that the context table of the point cloud data already used to encode the at least one frame is: A context table of point cloud data from previous frames that have already been used to encode the at least one frame, or A context table of point cloud data for any previous frame that has not yet been used to encode the at least one frame; The at least one processor is further configured to: The context table is used to entropy encode the geometric information generated in the L3C2 encoding / decoding scheme.

19. A computer program product comprising instructions that, when executed by one or more processors, perform a method for entropy decoding, wherein the method comprises: Obtain an indication (131) from the bitstream, wherein the indication indicates that at least one frame has been encoded using a context table; Determine the context table (132) from the bitstream; Decode (133) point cloud data from the bitstream based on the context table; The indication mentioned therein is a first flag, which indicates that the context table that has been used to encode the point cloud data of the at least one frame is: A context table of point cloud data from previous frames that have already been used to encode the at least one frame, or A context table of point cloud data for any previous frame that has not yet been used to encode the at least one frame; The method includes using the context table to perform entropy decoding on the geometric information generated in the L3C2 encoding / decoding scheme.

20. A non-transitory digital storage medium carrying instructions for performing program code for an entropy decoding method, wherein the method comprises: Obtain an indication (131) from the bitstream, wherein the indication indicates that at least one frame has been encoded using a context table; Determine the context table (132) from the bitstream; Decode (133) point cloud data from the bitstream based on the context table; The indication mentioned therein is a first flag, which indicates that the context table that has been used to encode the point cloud data of the at least one frame is: A context table of point cloud data from previous frames that have already been used to encode the at least one frame, or A context table of point cloud data for any previous frame that has not yet been used to encode the at least one frame; The method includes using the context table to perform entropy decoding on the geometric information generated in the L3C2 encoding / decoding scheme.

21. A computer program product comprising instructions that, when executed by one or more processors, perform a method for entropy encoding, wherein the method comprises: Use a context table to entropy encode (121) the point cloud data of at least one frame into a bit stream; Generate (122) the bitstream such that the bitstream further includes an indication that the at least one frame has been encoded using the context table; and such that the bitstream further includes information about the context table; The method includes generating (122) the bitstream such that the bitstream includes a first flag indicating that the context table of the point cloud data already used to encode the at least one frame is: A context table of point cloud data from previous frames that have already been used to encode the at least one frame, or A context table of point cloud data for any previous frame that has not yet been used to encode the at least one frame; The method includes using the context table to entropy encode the geometric information generated in the L3C2 encoding / decoding scheme.

22. A non-transitory digital storage medium carrying instructions for performing program code for an entropy coding method, wherein the method includes: Use a context table to entropy encode (121) the point cloud data of at least one frame into a bit stream; Generate (122) the bitstream such that the bitstream further includes an indication that the at least one frame has been encoded using the context table; and such that the bitstream further includes information about the context table; The method includes generating (122) the bitstream such that the bitstream includes a first flag indicating that the context table of the point cloud data already used to encode the at least one frame is: A context table of point cloud data from previous frames that have already been used to encode the at least one frame, or A context table of point cloud data for any previous frame that has not yet been used to encode the at least one frame; The method includes using the context table to entropy encode the geometric information generated in the L3C2 encoding / decoding scheme.

Images