Method and apparatus for encoding / decoding point cloud geometric data captured by a spin sensor head - Patents.com

Abstract

A method and apparatus for encoding / decoding a point cloud representing a physical object, where each point of the point cloud is associated with a residual radius between a radius dependent on the distance of the point to a reference and a predicted radius, the method entropy encoding / decoding the sign of a residual radius associated with a current point of the point cloud based on the sign of a previous entropy encoded / entropy decoded non-zero residual radius associated with a previous point.

Description

Cross-references to related applications

[0001] This application claims priority to European Patent Application No. EP21305460.4, filed April 9, 2021, the contents of which are incorporated and disclosed by reference in their entirety. [Technical field]

[0002] This application relates generally to point cloud compression, and more particularly to a method and apparatus for encoding / decoding point cloud geometric data captured by a spinning sensor head. [Background technology]

[0003] This section is intended to introduce the reader to aspects of the present invention that are described below and / or related to at least one exemplary embodiment of the present application for which protection is sought. It is believed that this discussion will be helpful in providing the reader with background information to aid in understanding the aspects of the present application.

[0004] As a kind of representation format of 3D data, point clouds have recently attracted attention due to their various features in terms of representing any type of physical object or scene. Point clouds can be used for various purposes such as cultural heritage and buildings, for example, by scanning objects such as sculptures or architectural structures in a 3D manner, sharing the spatial arrangement of the objects even if the objects are not transmitted or visited. It is also a way to preserve the knowledge of the objects in case they are destroyed, for example, a temple destroyed by an earthquake. Such point clouds are usually static, colored and large.

[0005] Another use case is topology and cartography: with a 3D representation, maps are not limited to flat surfaces, but can also include relief terrain. Currently, Google Maps is a good example of a 3D map, but it uses a mesh rather than a point cloud. However, a point cloud may be a suitable data format for a 3D map, and such point clouds are usually static, colored and large.

[0006] Virtual reality (VR), augmented reality (AR) and immersive worlds have been a hot topic lately, and many foresee them as the future of 2D flat video. The basic tenet is to immerse the viewer in the environment around them, as opposed to standard television, where the viewer can only see the virtual world in front of them. There are several stages of immersion, depending on the viewer's degree of freedom in the environment. Point clouds are a good candidate format for distributing VR / AR worlds.

[0007] The automotive industry, and especially the predicted autonomous vehicles, are also a sector where point clouds can be used extensively: autonomous vehicles need to "sense" their surrounding environment and make good driving decisions based on the presence and nature of the nearest detected objects and the configuration of the road.

[0008] A point cloud is a set of points in three-dimensional (3D) space, selectably adding additional values ​​to each. These additional values ​​are usually called attributes. Attributes may be, for example, a triplet color, a material property (e.g. reflectance) and / or a two-component surface normal vector associated with the points.

[0009] A point cloud is therefore a combination of geometry (position in 3D space, usually given by 3D Cartesian coordinates x, y and z) and attributes.

[0010] Point clouds can be captured by a variety of devices, such as camera arrays, depth sensors, laser instruments (also known as Light Detection and Ranging, LIDAR), radar, or can be computer generated (e.g., in movie post-production). Depending on the use case, a point cloud may contain thousands or even billions of points for mapping applications. The original representation of the point cloud requires a very high number of bits per point, at least a dozen bits per Cartesian coordinate x,y, and optionally provides more bits for attribute(s) to use for color, e.g., three times 10 bits.

[0011] In many applications, it is crucial to be able to distribute point clouds to end users or store them on servers while maintaining an acceptable (or very good) quality of experience, consuming only a reasonable amount of bitrate or storage space. Efficient compression of these point clouds is a key point for practical use in the distribution chain for many immersive worlds.

[0012] For distribution and visualization by end users, such as in AR / VR glasses or other 3D-enabled devices, the compression can be lossy (e.g. in video compression). Other use cases, such as medical applications or autonomous driving, require indeed lossless compression, so that the outcome of decisions derived from subsequent analysis of the compressed and transmitted point cloud is not altered.

[0013] Until recently, the issue of point cloud compression (aka PCC) has not been addressed in the mass market and there are no standardized point cloud codecs available. In 2017, the Standardization Working Group ISO / JCT1 / SC29 / WG11, also known as the Motion Picture Experts Group or MPEG, started a work item on point cloud compression. This resulted in two standards: MPEG-I Part 5 (ISO / IEC 23090-5) or Video-Based Point Cloud Compression (V-PCC) MPEG-I Part 9 (ISO / IEC 23090-9) or Geometry-Based Point Cloud Compression (G-PCC)

[0014] The V-PCC encoding method compresses the point cloud by performing multiple projections on the 3D object, resulting in 2D patches that are packed into an image (or video when dealing with dynamic point clouds). Conventional image / video codecs are then used to compress the acquired images or videos, thus making the most of already deployed image and video solutions. In essence, V-PCC is only efficient on dense, continuous point clouds, since image / video codecs cannot normally compress non-smooth patches, such as those obtained from projections of sparse geometric data captured by lidar.

[0015] The G-PCC coding method provides two solutions for compressing the captured geometric data.

[0016] The first solution is based on an occupancy tree, locally one of octrees, quadtrees or binary trees, to represent the geometry of the point cloud. Occupied nodes are partitioned into fixed sizes, and occupied leaf nodes provide the 3D location of the points, usually located at the center of these nodes. Occupancy information is carried by occupancy flags, which send signals to inform the occupancy status of each of the node's children. A high level of compression of occupancy flags for dense point clouds is possible by using a neighborhood-based prediction technique. Sparse point clouds can be addressed by directly encoding the location of points other than the minimum size in the node, stopping the tree structure when only isolated points exist in the node, and this technique is called direct coding mode (DCM).

[0017] The second solution is based on a prediction tree, where each node represents the 3D location of one point, and the parent / child relationships between nodes represent the spatial prediction from parent to child. This method can only solve sparse point clouds, and has the advantage of providing lower latency and simpler decoding than occupancy trees. However, compared to the first occupancy-based method, it only provides slightly better compression performance, and the encoding is more complex, as the encoder needs to intensively search for the best predictor (among a long list of potential predictors) when building the prediction tree.

[0018] In these two solutions, the attribute (decoding) coding is performed after the geometry (decoding) coding is completed, so in effect, two codings occur. Thus, the low latency of the joint geometry / attributes is achieved by using slices that divide the 3D space into independently coded subvolumes, without the need for prediction between subvolumes. Using many slices has a significant impact on compression performance.

[0019] The combination of encoder and decoder simplicity, low latency and compression performance requirements is a problem that traditional point cloud codecs have not adequately addressed.

[0020] One important use case is the transmission of sparse geometric data captured by spinning sensor heads (e.g. spinning lidar heads) mounted on moving vehicles. This typically requires a simple, low-latency embedded encoder. Simplicity is required because the encoder may be located on a computing unit for parallel execution of other processing (e.g. (semi)autonomous driving), thus limiting the available processing power of the point cloud encoder. Low latency is also required to enable fast transmission from the vehicle to the cloud, facilitating checking the local traffic in real time based on the collection of multiple vehicles, and making decisions based on the traffic information fast enough. 5G can also be used and the latency can be low enough, but the encoder itself should not introduce excessive latency due to encoding. And the data stream from millions of cars to the cloud can be very large, so compression performance becomes very important.

[0021] Certain a priori related to the sparse geometric data captured by the spin sensor head are used to derive an encoding / decoding method that is highly efficient.

[0022] For example, G-PCC utilizes the elevation angles (relative to the horizontal ground) captured by a spinning sensor head, e.g. those depicted in Fig. 1 and Fig. 2. The spinning sensor head 10 includes a set of sensors 11 (e.g. lasers), where five sensors are shown. The spinning sensor head 10 is able to capture geometric data of a physical object, i.e. the 3D positions of the points of a point cloud, by rotating about a vertical axis z. The geometric data captured by the spinning sensor head is then expressed in spherical coordinates (r 3D ,φ,θ) denoted by r 3D is the distance between point P and the center of the spinning sensor head, φ is the azimuth angle of the sensor head spinning relative to the reference, and θ is the elevation angle for an elevation index k of the spinning sensor head relative to the horizontal reference plane of the sensor (here the y-axis). The elevation index k may be, for example, the elevation angle of sensor k, or the kth sensor position where a single sensor detects each of the successive elevation angles in turn.

[0023] In the geometric data captured by the spin sensor head, we observe a regular distribution along the azimuth angle, as depicted in Figure 3. This regularity is used in G-PCC to obtain a quasi-1D representation of the point cloud, where, up to the noise, we 3D Only the angle φ and θ are in the range of continuous values, and only the angles φ and θ are in the range of discrete values. i ∀i=0~I-1, where I is the number to capture the azimuth angle of the point, and θ k ∀k=0 to K-1, where K is the number of sensors in the spin sensor head 10. Basically, G-PCC uses sparse geometric data captured by the spin sensor head along the 2D discrete angle plane (φ, θ) depicted in Figure 3 and the radius value r 3D Represents.

[0024] In spherical coordinate space, we use the discrete property of angles to predict the location of the current point based on the encoded points; this quasi-1D property has already been exploited in the occupancy and prediction trees in G-PCC.

[0025] More precisely, occupancy trees make extensive use of DCM and use a context-adaptive entropy encoder to entropy-encode the direct positions of points within a node. Then, a local transformation from the point positions to coordinates (φ,θ) and the resulting discrete angular coordinates (φ i ,θ k ) position.

[0026] The quasi-1D properties of this coordinate space (r,φ i ,θ k ), the prediction tree directly encodes a first version of the position of the current point P in spherical coordinates (r,φ,θ), where r is the projection radius in the horizontal xy plane, denoted as r on Figure 4. 2D Then we transform the spherical coordinates (r,φ,θ) into 3D Cartesian coordinates (x,y,z) and encode the xyz residual to address the errors in the coordinate transformation, the approximation of the elevation and azimuth angles, and potential noise.

[0027] As explained above, the captured point cloud geometric data captured by the spin sensor head 10 is advantageously represented in the (r,φ,θ) coordinate space. The quasi-discrete (up to noise) nature of the angles allows the angular coordinates (φ,θ) to be effectively compressed. However, the radius r can take any value depending on the distance of the object to be detected from the spin sensor head 10. Advantageously, the method 100 shown in FIG. 5 obtains the coded residual radius by predicting the radius r of a point of the point cloud from the radii of its neighboring coded points.

[0028] FIG. 5 shows a method 100 for encoding the residual radius in a bitstream according to the prior art.

[0029] In step 110, the prediction radius r pred Select as the prediction of radius r of the current point.

[0030] Prediction radius r pred An example of this is given by equation (7), which will be explained later.

[0031] Prediction radius r pred is the coordinate (r pred ,φ pred ,θ pred ) The prediction mode selection is accomplished by minimizing a rate or rate-distortion based cost function.

[0032] The selected prediction mode is represented by a prediction mode index I sel The prediction mode index I can be identified by the prediction mode index I sel signals in bitstream B each point of the point cloud (or equivalently each leaf node of the G-PCC prediction tree).

[0033] For example, the candidate prediction modes are (r min ,φ0,θ0), where r min is the minimum radius value (provided in the geometric parameter set), and φ0 and θ0 are equal to 0 if the current node (current point P) has no parent node, or are equal to the azimuth and elevation angles of the point associated with the parent node.

[0034] Another candidate prediction mode may be equal to (r0,φ0,θ0), where r0,φ0 and θ0 are the radius, azimuth and elevation angles, respectively, of the point associated with the parent node of the current node.

[0035] Another candidate prediction mode may be equal to a linear prediction of the radius, azimuth and elevation angles (r0,φ0,θ0) associated with the parent node of the current node and the radius, azimuth and elevation angles (r1,φ1,θ1) associated with the grandparent node, for example given by 2*(r0,φ0,θ0)-(r1,φ1,θ1).

[0036] Another candidate prediction mode may be equal to a linear prediction of the radius, azimuth and elevation angles (r0,φ0,θ0) of the point associated with the parent node of the current node, the radius, azimuth and elevation angles (r1,φ1,θ1) of the point associated with the grandparent node and the radius, azimuth and elevation angles (r2,φ2,θ2) of the point associated with the great-grandparent node.

[0037] For example, the candidate prediction modes are given by (r0,φ0,θ0)+(r1,φ1,θ1)-(r2,φ2,θ2).

[0038] In step 130, the residual radius rres=rr pred Calculate.

[0039] In step 140, a first flag f0 is signaled in the bitstream B to obtain the residual radius r res is null or not.

[0040] residual radius r res If x is null, the encoding of the residual radius is completed and the method is repeated to encode the residual radius of the next point in the point cloud.

[0041] Otherwise, in step 150, the magnitude |r res |-1 is coded into the bitstream B, and |.| indicates "absolute value". Typically, the magnitude is coded using a series of flags (each flag indicating whether the magnitude is a particular value or not) and / or an exponential Columbus (exp-Golomb) encoder. In step 160, the positive and negative signs s res is coded into the bitstream, typically using one bit for each sign. The method is then repeated to code the residual radius of the next point in the point cloud.

[0042] FIG. 6 shows a method 200 for decoding the radius from a bitstream according to the prior art.

[0043] In step 210, a first flag f0 is accessed in bitstream B.

[0044] If the first flag f0 indicates that the residual radius of the current point is null, the decoding for the residual radius is complete.

[0045] Otherwise, in step 220, a bitstream B is extracted with magnitude |r res In step 230, the sign s of the residual radius is decoded from the bit stream B. res In step 240, the decoded positive / negative sign s res and the residual radius r with a positive or negative sign from the decoded magnitude res Get the.

[0046] In step 250, the prediction radius r is calculated from the prediction mode as described in FIG. pred Then, we obtain the decoded residual radius rres and the prediction radius r pred By adding dec Finally, the method decodes the residual radius and obtains the decoded radius r of the next point in the point cloud. dec is iterated to obtain

[0047] FIG. 7 shows a point cloud encoder similar to the G-PCC prediction tree based encoder, using the residual radius encoding method 100 of FIG.

[0048] First, the Cartesian coordinates (x, y, z) of the points in the point cloud are transformed into spherical coordinates (r, φ, θ) by (r, φ, θ) = C2A(x, y, z), where r is the projection radius r on the horizontal xy plane. 2D where φ is the azimuth angle and θ is the elevation angle, as shown in FIG.

[0049] The transformation function C2A(.) is given by the following equation: r=round(sqrt(x*x+y*y) / Δr) φ=round(atan2(y,x) / Δφ) round() is the rounding up operation to the nearest integer value, sqrt() is the square root function, and atan2(y,x) is the arctangent applied to y / x. The quantities Δr and Δφ are quantization step sizes.

[0050] The angle θ will hereinafter be used as the elevation angle value, which may be obtained, for example, by the following formula: θ=atan(z / r), atan(.) is the arctangent function. In practice, for example, in G-PCC, θ is k represents the integer value of the elevation angle index k (i.e., the index of the kth elevation angle) of θ, so that the operations provided below to be performed on θ (prediction, residual (solution) coding, etc.) are modified to apply to the elevation angle index.

[0051] Spherical coordinates (r,φ,θ) and predicted spherical coordinates (r pred ,φ pred ,θ pred ) residual spherical coordinates between ( r res, φ res, θ res) is given by the following formula: (r res ,φ res ,θ res )=(r,φ,θ)-(r pred ,φ pred ,θ pred ) (1) (r pred ,φ pred ,θ pred ) are the spherical coordinates obtained from the prediction mode illustrated in FIG.

[0052] The residual spherical coordinates are calculated by the operator Q(.) ( r res, φ res, θ res) Let us denote the quantized residual spherical coordinates Q encoded in the bitstream B. ( r res, φ res, θ res) The quantized residual radius r rescan be encoded into bitstream B using the encoding method 100 of FIG.

[0053] Using the selected prediction mode, the predicted azimuth angle φ pred can be obtained.

[0054] In a variant, an additional signal is used in the bitstream B to signal an integer μ representing the number (positive or negative) of basic step sizes Δφ to be added to the selected prediction mode, thereby predicting the predicted azimuth angle φ. pred φ' pred can be fine-tuned to. φ' pred =φ pred+ μ*Δφ The number μ is signaled at each point in the bitstream B points (ie, at each node of the prediction tree).

[0055] The predicted Cartesian coordinates ( xpred ,y pred ,z pred ) is converted to the decoded spherical coordinates (r dec ,φ dec ,θ dec ) is obtained by performing an inverse transformation on ( xpred ,y pred ,z pred )=A2C(r dec ,φ dec ,θ dec ) (2)

[0056] Here, the spherical coordinates (r dec ,φ dec ,θ dec ) is given by the following formula: (r dec ,φ dec ,θ dec )=(r res,dec ,φ res,dec ,θ res,dec )+(r pred ,φ pred ,θ pred ) (3) The decoded residual spherical coordinates (r res,dec ,φ res,dec ,θ res,dec ) is the quantized residual spherical coordinate Q(r res ,φ res ,θ res ) may be the result of inverse quantization (IQ).

[0057] The spherical coordinates of the decoding (r dec ,φ dec ,θ dec ) is given by r=r dec *Δr x pred =round(r*cos(φ dec *Δφ) y pred =round(r*sin(φ dec *Δφ) z pred =round(tan(θ dec )*r) sin(), cos() and tan() are the sine, cosine and tangent functions that can be approximated using arithmetic with fixed-point precision.

[0058] Residual Cartesian coordinates (x res, y res, z res ) can be quantized (Q), and the quantized residual Cartesian coordinates Q(x res, y res, z res ) may be encoded into bitstream B.

[0059] When the quantization step sizes of x, y, and z are equal to the origin precision (usually 1), lossless coding can be performed on the residual Cartesian coordinates, or when the quantization step sizes are greater than the origin precision (usually quantization step sizes > 1), lossy coding can be performed on the residual Cartesian coordinates.

[0060] The Cartesian coordinates of the decoded data obtained by the decoder (x dec ,y dec ,z dec) is given by the following formula: (x dec ,y dec ,z dec )=(x pred ,y pred ,z pred )+IQ(Q(x res, y res, z res )) (4)

[0061] FIG. 8 shows a point cloud decoder similar to the G-PCC prediction tree based decoder, using the residual radius decoding method 200 of FIG.

[0062] The quantized residual spherical coordinates Q(r res ,φ res ,θ res ) is decoded from bitstream B. The quantized residual spherical coordinates Q(r res ,φ res ,θ res ) is inversely quantized to obtain the decoded residual spherical coordinates (r res,dec ,φ res,dec ,θ res,dec ) is obtained by the decoding method 200 of FIG. res,dec ) can be decoded from bitstream B.

[0063] According to equation (3), the decoded spherical coordinates (r dec ,φ dec ,θ dec ) and Here, (r pred ,φ pred ,θ pred ) are spherical coordinates obtained by using a method similar to the encoding method according to the prediction mode described in FIG. 5. Then, the predicted Cartesian coordinates ( xpred ,y pred ,z pred ) is obtained from equation (2).

[0064] In a variant, the predicted azimuth angle φ obtained by using a prediction mode identified by a prediction mode index accessed in bitstream B predis φ' pred can be fine-tuned to be given by: φ' pred =φ pred +μ*Δφ

[0065] For each point in the point cloud (i.e., each node of the prediction tree), we access a number μ in the bitstream B.

[0066] The quantized residual Cartesian coordinates Q(x res, y res, z res ) is decoded from the bitstream and inverse quantized (IQ) to obtain the inverse quantized Cartesian coordinates IQ(Q(x res, y res, z res )) to obtain the decoded Cartesian coordinates (x dec ,y dec ,z dec ) is given by equation (4).

[0067] European Patent Application No. EP20306672 discloses a single-stranded encoding / decoding method as an alternative solution to the encoding / decoding methods of FIGS.

[0068] In single-strand coding, the 3D Cartesian coordinates of a point in the point cloud are given by a radius value r (r 2D or r 3D ) in the 2D coordinate system (Cφ, λ). The coarse azimuth coordinate Cφ is the azimuth index of the spin of the sensor head, and its discrete values ​​are Cφ i (∀i=0~I-1), and the angle is φ i The coordinate λ is the sensor index, and its discrete values ​​are λ k (∀k=0~K-1), where the radius r belongs to a continuous range of values.

[0069] For each point in the point cloud, a sensor index λ (λ is the sensor index λ) associated with the sensor that captures said point. kObtain one of (∀k = 0 to K - 1), and the azimuth index Cφ of the angle captured by the sensor (Cφ is the discrete angle index Cφ i One of (∀i = 0 to I - 1)), and represents the radius value r of the spherical coordinates of the point.

[0070] Obtain the sensor index λ and the azimuth index Cφ by converting the 3D Cartesian coordinates (x, y, z) indicating the 3D position of the captured point. These 3D Cartesian coordinates (x, y, z) may be the output of the spin sensor head 10.

[0071] For example, the angle φ step is assumed to be the basic azimuth step size between two consecutive detections of the sensor head with a given sensor index λ, and assuming that the arctangent value of y / x returned from the function atan2(y, x) is a value within the interval [0; 2*π], Cφ can be obtained in the following manner, Cφ = round(φ / φ_step), where φ = atan2(y, x). Then, the rotation angle φ i is obtained by the formula. φ i = Cφ i * φ step .

[0072] In this case, the set of discrete angles φ i (0 ≤ i < I) is substantially defined by φ i = i * φ step .

[0073] Also, λ can be determined as the index λ of the sensor, and its elevation angle θ k is closest to the elevation angle θ of the points in the point cloud. k

[0074] Subsequently, in single-strand encoding, the points in the point cloud can be rearranged based on the azimuth index Cφ and the sensor index λ.

[0075] In the transformation, we sort the points by lexicographical order, first based on azimuth angle and then based on sensor index. The ordering index o(P) of a point P is obtained by the following scheme: o(P)=Cφ*K+λ

[0076] In another variant, we sort the points by lexicographical order, first based on the sensor index and then based on the azimuth angle. The order index o(P) of a point P is obtained by the following scheme: o(P)=λ*I+Cφ

[0077] Encoding the reordered points into the bitstream B is done by subtracting the order index difference Δo n where each order index difference represents a difference between the order indexes of two consecutive Pn-1 and Pn (for n=2 to N). Δo n =o(P n )-o(P n-1 )

[0078] The order index o(P1) of the first point P1 can be directly coded into the bitstream B. This corresponds to arbitrarily setting the order index of the imaginary 0th point to 0, i.e., o(P0)=0, and coding Δo1=o(P1)-o(P0)=o(P1).

[0079] The order index o(P1) of the first point and the order difference Δo n If a point P is given, n The order index o(P n ) can be constructed recursively. o(P n )=o(P n-1 )+Δo n

[0080] Then, point P is n The sensor index λ associated with n and azimuth index Cφ n : λ n =o(P n )moduloK (5) φ n =o(P n ) / K (6) Division / K is integer division (also called Euclidean division). Therefore, o(P1) and Δo n is λ n and Cφ n is an alternative representation of.

[0081] On the encoding side, the residual azimuth angle φ res is given by the following formula: φ res =φ-φ pred =φ-Cφ n *φ step (7)

[0082] Here, φ pred is the predicted azimuth angle.

[0083] Then, encoding the reordered points into the bitstream B involves the residuals (Qr res ,Qφ res,res ), given by: (Qr res ,Qφ res,res )=(r,Qφ res )-(r pred ,Qφ res,pred ) (8)

[0084] Here, Qr res is the quantized residual radius, and Qφ res is the quantized residual azimuth angle, and Qφ res,res is the residual of the quantized residual azimuth angle, and Qφ res,pred is the quantized prediction residual azimuth angle. n Elevation index θ n is not predictively coded.

[0085] Quantized residual radius Qr resis a quantizer with residual radius r res =rr pred is obtained by applying pred are the spherical coordinates resulting from the prediction mode illustrated in FIG.

[0086] The residual radius Qr quantized using the encoding method 100 of FIG. res 6. The dequantized residual radius IQr res which allows us to obtain the decoded residual radius rres ,dec is equal to.

[0087] Residual azimuth angle φ res is given by the following formula: φ res =φ-φ pred Here, φ pred is the predicted azimuth angle given by equation (7).

[0088] Quantized residual azimuth angle Qφ res is the quantizer Q with the residual azimuth angle φ res and is encoded into bitstream B.

[0089] Decoded residual azimuth angle φ res,dec is the inverse quantized residual azimuth angle IQφ obtained by applying the inverse quantizer IQ res Then, the decoded residual azimuth angle φ res,dec and the predicted azimuth angle φ given by Eq. (7). pred By adding dec Get (r dec ,φ dec )=(IQr res ,IQφ res )+(r pred ,φ pred ) (9)

[0090] Coding the reordered points in bitstream B involves the use of residual Cartesian coordinates (x res, y res, z res ). (x res, y res, z res )=(x,y,z)-(x pred ,y pred ,z pred )

[0091] where (x, y, z) are the 3D Cartesian coordinates of the permuted point, and (x pred ,y pred ,z pred ) are the predicted Cartesian coordinates obtained by the following method:

number

[0092] Residual Cartesian coordinates (x res, y res, z res ) is the quantized residual Cartesian coordinate Q(x res, y res, z res )

[0093] Lossless coding can be performed on the residual Cartesian coordinates if the quantization step sizes of x, y, z are equal to the origin precision (usually 1), or lossy coding can be performed on the residual Cartesian coordinates if the quantization step sizes are greater than the origin precision (usually the quantization step size is greater than 1).

[0094] At the decoding side, the order index difference Δo n Decode (n=2~N) the current point P n For each order index difference Δo n Decrypt the

[0095] Current point Pn For each, the order index o(P n ) to get the o(P n )=o(P n-1 )+Δo n

[0096] The sensor index λ associated with the sensor capturing the current point n and an azimuth angle φ representing the capture angle of the sensor. n is the order index o(P n ) to obtain (equations (5) and (6)).

[0097] The decoding method 200 of FIG. 6 decodes the quantized residual radius Qr res,dec can be decoded, and the inverse quantizer can be Qr res,dec By applying ,dec get.

[0098] Then, the decoded radius r dec is given by the following formula: r dec =r res,dec +r pred

[0099] r pred are the spherical coordinates obtained from the prediction mode illustrated in FIG.

[0100] Quantized residual azimuth angle Qφ res is decoded from bitstream B. The decoded residual azimuth angle φ res,dec Get the. φ res,dec =IQ(Qφ res )

[0101] The decoded spherical coordinates are then given by: (r dec ,φ dec )=(r res,dec ,φ res,dec )+(r pred ,φpred ) (10)

[0102] The decoded Cartesian coordinates (x dec ,y dec ,z dec ) is given by the following formula: (x dec ,y dec ,z dec )=(x pred ,y pred ,z pred )+IQ(Q(x res, y res, z res )) Here, IQ(Q(x res, y res, z res )) are the dequantized quantized residual Cartesian coordinates decoded from bitstream B.

[0103] In the prior art, angular data of the (Cφ, λ) format can be successfully compressed. However, the radius compression is not only due to the continuity of the radius, but also due to the residual radius r res is much less efficient due to weaknesses in its entropy coding.

[0104] Therefore, most of the bitstream is composed of radius data. In our tests, we have already observed that the radius data occupies 70%-90% of the total bitstream. Therefore, in order to improve the overall compression efficiency of the GPCC prediction tree encoding / decoding method or the single-chain encoding / decoding method, it is necessary to compress the radius data (e.g., the residual radius r res ) issues must be resolved. Summary of the Invention [Problem to be solved by the invention]

[0105] The following presents a simplified summary of at least one example embodiment in part to provide a basic understanding of some aspects of the present application. The summary is not a detailed overview of the example embodiment. It is not intended to identify key or critical elements of the example embodiment. The following summary is merely intended to present some aspects of at least one example embodiment in a simplified form as a prelude to the more detailed description provided elsewhere in the specification. [Means for solving the problem]

[0106] According to a first aspect of the present application, there is provided a method for encoding a point cloud into a bitstream of encoded point cloud data representing a physical object, where each point of the point cloud is associated with a residual radius between a radius dependent on the distance of the point to a reference and a predicted radius, the method comprising entropy coding the sign of a residual radius associated with a current point of the point cloud based on the sign of a previously entropy coded non-zero residual radius associated with a previous point of the point cloud.

[0107] In one exemplary embodiment, each point in the point cloud is associated with coordinates including an azimuth angle indicating the angle of capture by a sensor of the spin sensor head and a sensor index associated with the sensor capturing the point, and points in the point cloud having the same coordinates are adaptively reordered based on points in the point cloud captured at different azimuth coordinates with the same sensor index.

[0108] According to a second aspect of the present application, there is provided a method for decoding a point cloud from a bitstream of encoded point cloud data representing a physical object, where each point of the point cloud is associated with a residual radius between a sum of radii depending on a distance of the point to a reference and a predicted radius, the method comprising entropy decoding a sign of a residual radius associated with a current point of the point cloud based on the sign of a previously entropy decoded non-zero residual radius associated with a previous point of the point cloud.

[0109] In one embodiment, the entropy encoding or decoding of the sign of the residual radius of the current point is context-based entropy encoding or decoding using a context based on the sign of a previously entropy encoded or decoded non-zero residual radius.

[0110] In one embodiment, the current point and the previous point are captured by different sensors on the spin sensor head.

[0111] In one embodiment, the current point and the previous point are captured by the same sensor of the spin sensor head.

[0112] In one embodiment, the points of the point cloud are reordered based on a particular order, and the previous point is the last, previous reordered point associated with the entropy encoded or decoded non-zero residual radius.

[0113] In one embodiment, each point in the point cloud is associated with coordinates including an azimuth angle indicating the angle of capture by a sensor of the spin sensor head and a sensor index associated with the sensor capturing the point, and further derives context from first binary data indicating whether a current point and a previous point in the sorted points were captured at the same azimuth angle.

[0114] In one embodiment, the context is further obtained from the second binary data indicating whether the current point is at least the third point in the reordered points.

[0115] In one embodiment, points of the point cloud are associated with nodes of a tree, where the context is obtained from the third binary data where the predicted radius for obtaining the residual radius associated with the current point may or may not depend on the parent node of the current point.

[0116] In one embodiment, the context is obtained from a context table having as entries at least one of the following information: the first binary data, the second binary data, the third binary data, and a previously entropy coded non-zero residual radial sign.

[0117] According to a third aspect of the present application, there is provided an apparatus for encoding a point cloud into a bitstream of encoded point cloud data representing a physical object, the apparatus comprising one or more processors, the processors being configured to perform the method of the first aspect of the present application.

[0118] According to a fourth aspect of the present application, there is provided an apparatus for decoding points of a point cloud representing a physical object from a bitstream, the apparatus comprising one or more processors, the processors being configured to perform the method of the second aspect of the present application.

[0119] According to a fifth aspect of the present application there is provided a computer program product comprising instructions which, when executed by one or more processors, cause the one or more processors to perform the method of the second aspect of the present application.

[0120] According to a sixth aspect of the present application, there is provided a non-transitory storage medium carrying program code instructions for performing the method of the second aspect of the present application.

[0121] The following description taken in conjunction with the drawings will make apparent at least one specific feature of an exemplary embodiment and at least one other object, advantage, feature and application of said exemplary embodiment. [Brief description of the drawings]

[0122] Reference is now made in an exemplary manner to the drawings, which show exemplary embodiments of the present application. [Figure 1] FIG. 1 is a side view of a sensor head and some parameters according to the prior art. [Diagram 2]FIG. 1 is a top view of a sensor head and some parameters according to the prior art. [Diagram 3] 1 is a regular distribution of data captured by a spin sensor head according to the prior art. [Figure 4] 1 is a representation of a point in 3D space according to the prior art. [Diagram 5] 1 is a prior art method 100 for encoding the residual radius into the bitstream. [Figure 6] 2 is a method 200 for decoding a radius from a bitstream according to the prior art. [Figure 7] A point cloud encoder similar to the prior art G-PCC prediction tree based encoder. [Figure 8] 1 is a point cloud decoder similar to the prior art G-PCC prediction tree based decoder. [Figure 9] This illustrates the natural monotonicity of the radii of the points in the point cloud captured by the spin sensor head. [Figure 10] 3 is a block diagram of steps of a method 300 for encoding a residual radius into a bitstream, in accordance with at least one exemplary embodiment of the present invention. [Figure 11] 4 is a block diagram of steps of a method 400 for decoding a residual radius from a bitstream, in accordance with at least one exemplary embodiment of the present invention. [Figure 12] 2 is an example of the evolution of the radius of the points of a point cloud captured by a spin sensor head according to the prior art; [Figure 13] FIG. 5 is a block diagram of steps of a method 500 for adaptive radial reordering of capture points with the same coordinates (Cφ, λ), in accordance with at least one exemplary embodiment of the present invention. [Figure 14] The method 500 in FIG. [Figure 15] The method 500 in FIG. [Figure 16] 4 is an example of the evolution of the radius of points of a point cloud captured by a spin sensor head according to an exemplary embodiment of the present invention. [Figure 17]FIG. 1 is an exemplary block diagram of a system for implementing aspects and exemplary embodiments.

[0123] Similar reference numbers may be used in different drawings to represent similar parts. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0124] At least one of the exemplary embodiments will be described below with reference to the drawings, in which at least one example of the exemplary embodiments is shown. However, the exemplary embodiments may be embodied in many alternative forms and should not be construed as being limited to the examples described herein. Thus, it should be understood that the exemplary embodiments should not be limited to the particular forms disclosed. Rather, the present disclosure is intended to cover all modifications, equivalents and alternative solutions falling within the spirit and scope of the present application.

[0125] It should be understood that when a drawing is presented in flow chart form, a corresponding apparatus block diagram is also provided. Similarly, it should be understood that when a drawing is presented in block diagram form, a corresponding method / process flow is also provided.

[0126] At least one of these aspects relates generally to point cloud encoding and decoding, and at least one other aspect relates generally to transmission of the generated or encoded bitstream.

[0127] And, the present aspects are not limited to MPEG standards such as MPEG-I Parts 5 or 9 related to point cloud compression, but are applicable to other standards and recommendations, including pre-existing, yet to be developed, and extensions of any such standards and recommendations, including MPEG-I Parts 5 and 9. Unless otherwise indicated or technically excluded, the aspects described herein may be used alone or in combination.

[0128] The present invention relates to the field of encoding and decoding technology, and aims to provide a technical solution for encoding / decoding point cloud data. Because a point cloud is a collection of large amounts of data, a large amount of memory is consumed to store the point cloud, and if the point cloud is not compressed, the point cloud cannot be directly transmitted at the network layer, so compression of the point cloud is required. Therefore, with the increasing use of point clouds in autonomous navigation, patrol inspection, geographic information services, cultural heritage / building protection, 3D immersive communication and interaction, etc., the present invention can be used in many application scenarios.

[0129] The present invention particularly relates to coding / decoding the positive / negative sign of the residual radius, which can reduce the coding cost of the positive / negative sign, and improve the overall coding efficiency of the point cloud geometric data captured by the spin sensor head.

[0130] The invention relates to a method for encoding / decoding a cloud of points representing a physical object, each point of said cloud of points being associated with a residual radius between a radius depending on the distance of said point to a reference and a predicted radius.

[0131] The present invention entropy encodes / decodes the sign of the residual radius associated with a point of the point cloud based on the sign of the previously entropy encoded / decoded non-zero residual radius.

[0132] Usually, the sign of any residual is unpredictable, and if the residual is non-zero, it costs one uncompressed bit to encode it. However, when the geometric data is captured by the spinning sensor head 10, the radii of the captured points follow a natural monotonicity, as shown in FIG. 9. On the left, the sensor beam is shown emanating from the sensor towards the object, here shown diagrammatically with a grey shaded rectangle. Seven points are captured, each corresponding to a certain azimuth angle φ. The diagram on the right shows the evolution of the radii of the seven captured points with respect to the azimuth angle. The diagram clearly shows the monotonicity of the radii of the captured points, i.e. negative (radius decreases as the azimuth angle increases) and positive (radius increases as the azimuth angle increases). Thus, the radius r of the previous points captured by the same sensor is prec If we predict the radius r of the current point by prev shows a constant positive or negative sign over time, and is either a set of negative residual radii or a set of positive residual radii. Of course, this is not always true, but it is statistically significant enough that this property can be used to find the residual radius r res Positive / negative sign of s res It is sufficient to improve the compression of

[0133] Therefore, the previously encoded residual radius r res,prev Positive / negative sign of s res,prev Based on res Positive / negative sign of s res To encode / decode the positive / negative sign s res (one bit lower than each positive and negative sign). The overall coding efficiency of the point cloud geometric data captured by the spin sensor head 10 is then improved.

[0134] FIG. 10 is a block diagram of steps of a method 300 for encoding the residual radius rres into a bitstream B according to at least one example embodiment.

[0135] The method includes: resThe resulting quantized residual radius Qr res For encoding, for example, as described above in FIG. 7 or single-strand encoding.

[0136] In one exemplary embodiment of the method 300, the entropy coding uses a context, ie, the entropy coding is context-based coding.

[0137] In step 310, a previously entropy coded non-zero residual radius r associated with a previous point in the point cloud is calculated. res,prev Positive / negative sign of s res,prev Get the.

[0138] In step 320, the positive / negative sign s res,prev Based on the current pointwise residual radius rres, the sign s res The context Ctx used to encode sign Get the.

[0139] For example, the positive and negative sign s res,prev If is positive, the first context Ctx sign,1 Get the positive or negative sign s res,prev If is negative, the second context Ctx sign,2 Get the.

[0140] In step 330, the context Ctx is encoded using context-based entropy coding. sign Using the residual radius r res Positive / negative sign of s res is encoded into bitstream B.

[0141] In Fig. 10, the residual radius r res The size of is the residual radius r res Positive / negative sign of s res Alternatively, the residual radius r res The residual radius r before the size res Positive / negative sign of s res Encode the following.

[0142] A variant uses a usage context adaptive binary arithmetic encoder (CABAC).

[0143] FIG. 11 illustrates a method for extracting residual radius r from a bitstream B in accordance with at least one example embodiment. res 4 is a block diagram of steps of a method 400 for decoding a

[0144] The method includes: res The decoded residual radius Qr res Decoding of is, for example, as described above in FIG. 8 or single strand decoding.

[0145] In one exemplary embodiment of the method 400, the entropy decoding uses a context, ie, the entropy decoding is context-based decoding.

[0146] In step 410, a previously entropy decoded non-zero residual radius r associated with the previous point is calculated. res,prev Positive / negative sign of s res,prev Get the.

[0147] In step 320, the positive / negative sign s res,prev Based on the residual radius r of the current point res Positive / negative sign of s res The context Ctx used by the context-based entropy decoding to decode sign Get

[0148] In step 420, a context-based entropy coding is used to obtain the context Ctx sign Using the residual radius r res The decoded sign of s res from bitstream B.

[0149] In one exemplary embodiment of step 320 , the current point and the previous point are captured by the same sensor of the spin sensor head 10 .

[0150] In most cases, the exemplary embodiment provides compression gains when building a prediction tree (e.g., as defined in G-PCC) such that points acquired by the same sensor are consecutively coded, or when the encoder builds a prediction tree in which consecutively coded radius residuals have the same sign.

[0151] In one exemplary embodiment of step 320 , the current point and the previous point are captured by different sensors of the spin sensor head 10 .

[0152] The exemplary embodiment further provides compression gains that are independent of the constructed prediction tree.

[0153] In one example implementation of step 320, points of the point cloud captured by the same sensor are sorted based on a specific order, and previous points are sorted by the entropy-encoded non-zero residual radius r res,prev is the last, previously sorted point associated with

[0154] For each sensor index, the sign of the previously encoded / decoded residual radius can be kept in memory. After encoding / decoding the sign of the residual radius and the sensor index of a new point, the sign is also updated. Then, the mechanism calculates the entropy-encoded non-zero residual radius r res,prev may be used to find the last previous point associated with

[0155] In a variant, points captured by the same sensor are reordered based on the coding order given by the predictive tree coding defined in G-PCC.

[0156] In a variant, a prediction tree can be constructed to facilitate taking into account the monotonicity of the radius of the coding order of points captured by the same sensor.

[0157] In a variant, in a single-chain encoding / decoding method, points of the point cloud having the same coordinates (Cφ, λ) can be sorted in order of increasing (or decreasing) radius.

[0158] As can be seen from observing the contents shown in Figure 12, when the radius r naturally increases in the process of object sensing, the evolution of the radius r is smooth. However, when the radius r tends to decrease in the process of sensing, a sawtooth curve is observed. This can be easily explained by the local addition (of multiple captured points) of the same coordinate (Cφ, λ) and the overall decrease in the captured object due to the progression of sensing.

[0159] In a variant, the single-stranded encoding / decoding method adaptively reorders the point cloud points having the same coordinates (Cφ, λ) based on the point cloud points captured by the same sensor index λ (same sensor) but with different coarse coordinates.

[0160] 13 is a block diagram of the steps of an adaptive radius reordering method 500 of capture points with the same coordinates (Cφ, λ). The method 500 is applied only to the encoding side.

[0161] In step 510, as shown in FIG. 14, the maximum radius of the point with coordinates (Cφ, λ) and the coordinate (φ -1 , λ) to the radius of the last point of the previous sorted point associated with the coordinate φ -1 =Cφ-φ step It is.

[0162] In step 520, the minimum radius of points having the same coordinates (Cφ, λ) and the coordinates (φ -1 , λ) to the radius of the last point of the previous sorted point associated with it.

[0163] In step 530, if the second radius jump J2 is higher than the first radius jump J1, the points with the same coordinates (Cφ, λ) are sorted according to the order of decreasing radius, as shown in FIG. 14, otherwise, they are sorted according to the order of increasing radius (step 540), as shown in FIG. 15. In this way, the first point among the points with the same coordinates (Cφ, λ) and the second point among the points with the same coordinates (φ -1 , λ) to the last point of the previous sorted point associated with it, thereby facilitating the gain from the context coding of the positive and negative sign of the radius residual.

[0164] Also, applying the adaptive radius reordering method 500 to the capture points with the same coordinates (Cφ, λ) of FIG. 13 can provide a smoother, more monotonic radius evolution, as shown in FIG. 16.

[0165] When the method 500 is applied to points of a point cloud having the same coordinates (Cφ, λ), a sorted list of points having the same coordinates {..., P -2 ;P -1 ;P;...}, and let P be at least the third point in the list of sorted points. Then, if the radius of the previous point is selected as the radius predictor, it is calculated by subtracting the sign s of the residual radius associated with the current point P. res is the previous point P -1 The sign of the residual radius associated with s res,prec This is a direct consequence of method 500, which is that r(P)-r(P -1 ) and r(P -1 )-r(P -2 ) has the same sign as s res =s res,prec We force the last three points to have decreasing or increasing radii so that

[0166] In one exemplary embodiment of step 320, the binary data D last From the context Ctx sign and obtain the binary data Dlast indicates whether the current point and the previous point in the sorted list of points were captured by the same coarse azimuth angle Cφ.

[0167] The exemplary embodiment uses the sign s of the residual radius associated with the current point captured by the sensor. res Since when is the previously entropy-encoded nonzero residual radius r associated with the previous point captured by the same sensor? res,prev Positive / negative sign of s res,prev , which improves the overall coding efficiency of the point cloud geometric data.

[0168] In one exemplary embodiment of step 320 in combination with the previous exemplary embodiments of step 320, at least a third point P in the list is further included which indicates whether the current point is a sorted point. penult Binary data D penult From the context Ctx sign Get the.

[0169] In one exemplary embodiment of step 320 applied in the G-PCC encoding / decoding method, an integer number μ of previous coded points of the point cloud ( FIG. 7 ) that are signaled or accessed in bitstream B is prev can be stored in memory, and then the stored integer number μ associated with the previous point prev From binary data D penult and / or signal in the bitstream binary information D last Estimate.

[0170] In G-PCC, multiple points can be captured by the same sensor with the same or different azimuth angles. When two points are captured consecutively with respect to the same azimuth angle, the integer quantity μ is equal to 0 for encoding the two consecutively captured points. When a current point and a previous point are captured consecutively with different azimuth angles, the number of integers μ for encoding the current point is equal to 0 and the number of integers for encoding the previous point is equal to 1. Thus, the value of the integer signaled in the bit stream B is the binary data D penult and D. last Indicates similar information.

[0171] For example, μ prev If =0, D penult = 0, otherwise D penult If =1 and μ=0, then D last =0, otherwise Dlast=1.

[0172] In one example embodiment of step 320, for a previously coded point of the point cloud (FIG. 7), a prediction mode index I is signaled or accessed in bitstream B. sel (Figure 5) can be stored in memory and then the predictor mode index I sel,prec , or a prediction mode index I sel from the prediction mode identified in the candidate prediction mode list by sign Get the.

[0173] For example, as explained above, the candidate prediction mode indicates whether the prediction radius is determined by the parent node or not. Then, the predictor mode index I sel,prec From binary data D previous can be determined.

[0174] In one exemplary embodiment of step 320, the context Ctx sign is the first binary data Dlast and the second binary data D penult and the third binary data D previous and the previously entropy-coded non-zero residual radius r res,prev Positive / negative sign of s res,prev The context information can be obtained from a context table ctxTab that has at least one of the following entries:

[0175] for example, Ctx sign =ctxTab[D penult ][D last ][s res,prec ] Or, Ctx sign =ctxTab[D previous ][D penult ][D last ][s res,prec ]

[0176] The present encoding / decoding method is used for encoding / decoding point clouds, which can be used for various purposes, in particular for encoding / decoding the positive / negative sign of the residual radius, which can reduce the encoding of the positive / negative sign and thereby improve the overall encoding efficiency of the point cloud geometric data captured by the spin sensor head.

[0177] FIG. 17 is a schematic block diagram of an exemplary system for implementing aspects and example embodiments.

[0178] System 600 may be implemented in one or more devices and may include the components described below. In various embodiments, system 600 may be configured to implement one or more aspects described herein.

[0179] Examples of devices that may comprise all or part of system 600 include personal computers, laptops, smartphones, tablets, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, connected vehicles and their associated processing systems, head mounted display devices (HMDs, see-through glasses), projectors, "caves" (systems including multiple displays), servers, video encoders, video decoders, post-processors that process output from a video decoder, pre-processors that provide input to a video encoder, web servers, set-top boxes, any other device for processing point clouds, videos or images, or other communications devices. The elements of system 600 may be implemented individually or in combination in a single integrated circuit (IC), multiple ICs and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 600 may be distributed across multiple ICs and / or discrete components. In various embodiments, system 600 may be communicatively coupled to other similar systems or other electronic devices, for example, via a communications bus or dedicated input and / or output ports.

[0180] The system 600 may include at least one processor 610, which is configured to execute instructions loaded therein to implement the aspects described herein. The processor 610 may include embedded memory, input / output interfaces, and various other circuits known in the art. The system 600 may include at least one memory 620 (e.g., a volatile memory device and / or a non-volatile memory device). The system 600 may include a storage device 640, which may include non-volatile memory and / or volatile memory, and may include, but is not limited to, an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a disk drive, and / or an optical disk drive. As a non-limiting example, the storage device 640 may include an internal storage device, an attached storage device, and / or a network-accessible storage device.

[0181] The system 600 may include an encoder / decoder module 630 configured to, for example, process data to provide encoded / decoded point cloud geometry data, and the encoder / decoder module 630 may include its own processor and memory. The encoder / decoder module 630 may represent a module or modules that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include either one or both of an encoding and decoding module. Also, the encoder / decoder module 630 may be implemented as a separate element of the system 600 or may be coupled into the processor 610 as a combination of hardware and software known by those skilled in the art.

[0182] Program code loaded into the processor 610 or the encoder / decoder 630 to perform aspects described herein can be stored in the storage device 640 and then loaded into the memory 620 and executed by the processor 610. According to various embodiments, while performing the processes described herein, one or more of the processor 610, the memory 620, the storage device 640, and the encoder / decoder module 630 can store one or more of various items. Such stored items can include, but are not limited to, point cloud frames, encoded / decoded geometry / attribute videos / images or portions of encoded / decoded geometry / attribute videos / images, bitstreams, matrices, variables, and equations, formulas, intermediate or final results of arithmetic and logical operations.

[0183] In some embodiments, memory within the processor 610 and / or the encoder / decoder module 630 can be used to store instructions and provide working memory for processes performed during encoding or decoding.

[0184] However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 610 or the encoder / decoder module 630) is used for one or more of these functions. The external memory may be the memory 620 and / or the storage device 640, such as dynamic volatile memory and / or non-volatile flash memory. In some embodiments, the external non-volatile flash memory is used to store the television's operating system. In at least one embodiment, a fast external dynamic volatile memory, such as RAM, may be used as working memory for video encoding and decoding operations, such as HEVC (Efficient Video Coding), VVC (Versatile Video Coding), or MPEG-I Part 5 or Part 9 for MPEG-2 Part 2 (also known as ITU-T Recommendation H.262 and ISO / IEC 13818-2, also known as MPEG-2 Video).

[0185] As indicated by block 690, inputs to the elements of system 600 can be provided via various input devices, including, but not limited to, (i) an RF section capable of receiving RF signals transmitted wirelessly, for example, by a broadcast device, (ii) a composite input, (iii) a USB input, and / or (iv) an HDMI input.

[0186] In various embodiments, the input devices of block 690 have corresponding input processing elements associated therewith, as known in the art. For example, the RF section may be associated with the following necessary elements: (i) select the desired frequency (also called signal selection, or limiting the signal frequency band to within a frequency band), (ii) downconvert the selected signal, (iii) select (for example) the signal frequency band by limiting the frequency band again to a narrow frequency band, referred to in certain embodiments as a channel, (iv) demodulate the downconverted and frequency band limited signals, (v) perform debugging, and (vi) demultiplex to select the desired packet stream. The RF section of various embodiments may include one or more elements that perform these functions, such as a frequency selector, a signal selector, a frequency band limiter, a channel selector, a filter, a downconverter, a demodulator, a debugging device, and a demultiplexer. The RF section may include a tuner that performs each of these functions, including downconverting the received signal to a lower frequency (e.g., an intermediate frequency or a frequency near baseband) or to baseband.

[0187] In one set-top box embodiment, the RF section and its associated input processing elements can receive RF signals transmitted over a wired (e.g., cable) medium. The RF section can then perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band.

[0188] Various embodiments may rearrange the order of these (and other) elements, eliminate some of these elements, and / or add other elements that perform similar or different functions.

[0189] Adding elements may be inserting elements between conventional elements, for example, inserting amplifiers and analog to digital converters. In various embodiments, the RF section may include an antenna.

[0190] Additionally, the USB and / or HDMI terminals may include corresponding interface processors, which are used to connect the system 600 to other electronic devices via USB and / or HDMI connections. Note that, when necessary, aspects of the input processing (e.g., Reed-Solomon debug) may be implemented, for example, in a separate input processing IC or in the processor 610. Similarly, when necessary, aspects of the USB or HDMI interface processing may be implemented, for example, in a separate interface IC or in the processor 610. The demodulated, debug, and demultiplexed streams may be provided to various processing elements, including, for example, the processor 610 and the encoder / decoder 630, which may operate in combination with memory and storage elements to process the data streams for display at an output device, when necessary.

[0191] 6. Providing the various elements of system 600 within a monolithic housing, within which suitable connection arrangements 690 can be made, such as for connecting and transmitting data between the various elements via internal buses (including an I2C bus), wiring, and printed circuit boards as are known in the art.

[0192] System 600 may include a communication interface 650 such that communication outlet 650 may communicate with other devices via communication channel 1000. Communication interface 650 may include, but is not limited to, a transceiver that transmits and receives data over communication channel 1000. Communication interface 650 may include, but is not limited to, a modem or a network card, and communication channel 1000 may be implemented, for example, within a wired and / or wireless medium.

[0193] In various embodiments, a Wi-Fi network, such as IEEE 802.11, may be used to stream data to the system 600. The Wi-Fi signal in these embodiments may be received by a communication channel 1000 and communication interface 650 compatible with Wi-Fi communications. The communication channel 1000 in these embodiments may typically be connected to an access point or router that provides access to external networks, including the Internet, to enable streaming applications and other over-the-top communications.

[0194] Other embodiments may provide stream data to the system 600 using a set-top box, which transmits the data via an HDMI connection in the input block 690.

[0195] Other embodiments may also use the RF of input block 690 to provide stream data to system 600 .

[0196] The stream data can be used as a form of signaling information used by the system 600. The signaling information can include a bit stream B, which includes at least a flag f0, a context Ctx sign The coded residual radius r using context-based entropy coding with res Positive / negative sign of s resand information representing the magnitude of the residual radius, the number of points in the point cloud, the coordinates or order o(P1) of the first point in the 2D coordinate system (Cφ, λ) and / or sensor setting parameters, e.g. parameters μ or elevation angle θ associated with the sensor of the spin sensor head 10. k . included.

[0197] It should be noted that various signaling possibilities are possible. For example, in various embodiments, one or more syntax elements, flags, etc. may be used to transmit signaling information to a corresponding decoder.

[0198] System 600 can provide output signals to a variety of output devices including a display 700, speakers 800, and other peripherals 900. In various example embodiments, other peripherals 900 can include one or more of a standalone DVR, a disc player, a stereo system, a lighting system, and other devices based on the output providing capabilities of system 600.

[0199] In various embodiments, control signals can be communicated between system 600 and display 700, speaker 800, or other peripherals 900 using, for example, AV.Link (Audio / Video Link), CEC (Consumer Electronics Control), or other communication protocol signaling that allows control between devices with or without user intervention.

[0200] The output devices are communicatively coupled to the system 600 by corresponding interfaces 660, 670 and 680 via dedicated connections.

[0201] Alternatively, an output device can be connected to system 600 using communication channel 1000 via communication interface 650. Display 700 and speakers 800 can be integrated into a single unit along with other components of system 600 within an electronic device (e.g., a television).

[0202] In various embodiments, the display interface 660 can include a display driver, such as a timing controller (T Con) chip.

[0203] For example, if the RF portion of input 690 is part of a separate set-top box, display 700 and speakers 800 may be selectably separate from one or more of the other components. In various embodiments where display 600 and speakers 700 may be external components, output signals may be provided via dedicated output connections (including, for example, an HDMI port, a USB port, or a COMP output).

[0204] 1-17, various methods are described herein, each of which includes one or more steps or actions to achieve the described method. To the extent that the exact operation of the method does not require a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined.

[0205] Some examples are described with respect to block diagrams and / or operational flow charts. Each block represents a circuit element, module, or portion of code, which includes one or more executable instructions for implementing a specified logical function(s). It should be understood that in other embodiments, the function(s) marked in the block may differ from the order shown. For example, depending on the functions involved, two blocks shown in succession may in fact be executed essentially simultaneously, or the blocks may be executed in the reverse order.

[0206] The embodiments and aspects described herein may be implemented in, for example, a method or process, an apparatus, a computer program, a data stream, a bit stream, or a signal. Even when discussed only in the context of a single type of embodiment (e.g., discussed only as a method), the discussed embodiment features may be implemented in other forms (e.g., an apparatus or a computer program).

[0207] The methods may be implemented, for example, in a processor, which generally refers to a processing device including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors further include communications devices.

[0208] Also, the method may be implemented with instructions executable by a processor, and such instructions (and / or data values ​​generated by the embodiment) may be stored in a computer-readable storage medium. The computer-readable storage medium may be in the form of a computer-readable program product having computer-readable program code embodied in one or more computer-readable media and executable by a computer embodied therein. Given the inherent ability to store information and provide retrieval of information, the computer-readable storage medium used herein may be considered as a non-transitory storage medium. The computer-readable storage medium may be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Note that, although more specific examples are provided below to which the computer-readable storage medium of the present embodiment is applied, as can be easily understood by those skilled in the art, they are merely illustrative and not a detailed list: a portable computer floppy disk, a hard disk, a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any combination of the foregoing.

[0209] The instructions may form an application program tangibly embodied on a processor-readable medium.

[0210] For example, instructions may reside in hardware, firmware, software, or a combination. For example, instructions may be found in an operating system, an independent application, or a combination of both. A processor may thus be characterized as, for example, a device configured to execute a process, and a device having a processor-readable medium (e.g., a storage device) for executing instructions of the process. Also, in addition to or in lieu of instructions, a processor-readable medium may store data values ​​produced by an embodiment.

[0211] The device may be implemented in, for example, suitable hardware, software and firmware. Examples of such devices include personal computers, laptops, smartphones, tablets, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, head mounted display devices (HMD, see-through glasses), projectors, "caves" (systems including multiple displays), servers, video encoders, video decoders, post-processors that process the output from a video decoder, pre-processors that provide input to a video encoder, web servers, set-top boxes, any other device for processing point clouds, videos or images, or other communications devices. It should be noted that the device may be mobile and may be mounted in a moving vehicle.

[0212] The computer software may be implemented by the processor 510, hardware, or a combination of hardware and software. As a non-limiting example, an embodiment may be implemented by one or more integrated circuits. The memory 520 may be of any type suitable for the technology environment and may be implemented using any suitable data storage technology, including, as non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memories, and removable memories. The processor 510 may be of any type suitable for the technology environment and may include, as non-limiting examples, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0213] It will be apparent to those skilled in the art that the embodiments can generate a signal formatted to carry, for example, storable or transmittable information. The information can include, for example, instructions for performing a method or data generated by one of the described embodiments. For example, the signal can be formatted to carry a bit stream of the described embodiments. Such a signal can be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or a baseband signal. Formatting can include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal can be, for example, analog or digital information. As is known, the signal can be transmitted over a variety of different wired or wireless links. The signal can be stored in a processor-readable medium.

[0214] The terms used herein are intended to describe particular embodiments and are not limiting. Unless the context clearly dictates otherwise, the singular forms "a," "an," and "the" as used herein include the plural forms. Furthermore, as used herein, the terms "include / comprise" and / or "including / comprising" may specify the presence of, for example, stated features, integers, steps, operations, elements, and / or components, but cannot preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof. And, when an element is said to be "responsive to" or "connected" to another element, it may be directly responsive to or connected to the other element, or intermediate elements may be present. Conversely, when an element is said to be "directly responsive to" or "directly connected" to another element, intermediate elements are not present.

[0215] For example, in the case of "A / B," "A and / or B," and "at least one of A and B," the use of any one of the symbols / terms " / ," "and / or," and "at least one of" is intended to include only the selection of the first listed option (A), or to include only the selection of the second listed option (B), or to include the selection of both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A and B," such language is intended to include only the selection of the first listed option (A), or to include only the selection of the second listed option (B), or to include only the selection of the third listed option (C), or to include only the selection of the first and second listed options (A and B), or to include only the selection of the first and third listed options (A and C), or to include only the selection of the second and third listed options (B and C), or to include all three options (A, B, and C). As will be apparent to one skilled in the art, this can be expanded to as many items as are listed.

[0216] In this application, various numerical values ​​may be used, and specific values ​​may be used for illustrative purposes, and the aspects described are not limited to these specific values.

[0217] It should be noted that terms such as first, second, etc. may be used to describe various elements in this specification, but these elements are not limited by these terms. These terms are used only to distinguish one element from another element. For example, a first element may be called a second element, and similarly, a second element may be called a first element, without departing from the teachings of this application. No order is implied between the first element and the second element.

[0218] References to "one exemplary embodiment" or "exemplary embodiment" or "one embodiment" or "embodiment" and other variations are often used to indicate that a particular feature, structure, characteristic, etc. (described in conjunction with the embodiment) may be included in at least one embodiment / embodiment. Thus, appearances of the terms "in one exemplary embodiment" or "in an exemplary embodiment" or "in one embodiment" or "in an embodiment" and any other variations appearing in various places throughout this application are not necessarily all referring to the same embodiment.

[0219] Similarly, the present specification often uses references to "according to an exemplary embodiment" or "in an exemplary embodiment" and other variations to indicate that a particular feature, structure, or characteristic (described in combination with an exemplary embodiment) may be included in at least one exemplary embodiment. Thus, the phrases "according to an exemplary embodiment" or "in an exemplary embodiment" appearing in various places in this application do not necessarily refer to the same exemplary embodiment, and separate or alternative exemplary embodiments are not necessarily mutually exclusive of other exemplary embodiments.

[0220] The reference numerals in the drawings appearing in the claims are for explanatory purposes only and do not limit the scope of the claims. Although there is no explicit description, the present embodiment / example and modified examples can be adopted in any combination or partial combination.

[0221] It will be appreciated that when the drawings are presented as flow charts, they also provide a block diagram of the corresponding apparatus. Similarly, when the drawings are presented as block diagrams, they also provide a flow chart of the corresponding method / process.

[0222] Although some of the drawings include arrows on paths to indicate the primary direction of communication, it is understood that communication may occur in the opposite direction to that depicted.

[0223] Various embodiments relate to decoding. As used herein, "decoding" may include, for example, all or part of a process performed on a received point cloud frame (which may include a received bitstream that performs encoding on one or more point cloud frames) to generate a final output suitable for display or further processing in a reconstructed point cloud domain. In various embodiments, this type of process may include one or more of the processes typically performed by a decoder. In various embodiments, for example, this type of process may further include, optionally, a process performed by a decoder of various embodiments described herein.

[0224] As a further example, in one embodiment, "decoding" may refer to entropy decoding, in another embodiment, "decoding" may refer only to differential decoding, and in another embodiment, "decoding" may refer to a combination of entropy decoding and differential decoding. Depending on the context in which it is specifically described, it is clear and recognized that one skilled in the art can well understand whether the term "decoding process" refers specifically to a subset of operations or generally to a broader decoding process.

[0225] All of the various embodiments relate to encoding. Similar to the "decoding" discussion above, "encoding" as used herein may include all or part of a process that is performed, for example, on an input point cloud frame to generate an encoded bitstream. In various embodiments, this type of process includes one or more of the processes typically performed by an encoder. In various embodiments, this type of process includes, or alternatively includes, a process performed by an encoder in various embodiments described herein.

[0226] As a further example, in one embodiment, "encoding" may refer only to entropy encoding, in another embodiment, "encoding" may refer only to differential encoding, and in another embodiment, "encoding" may refer to a combination of differential and entropy encoding. Based on the limited context described, it is clear whether the term "decoding process" refers specifically to a subset of operations or generally to a broader decoding process, and is recognized as being well understood by those skilled in the art.

[0227] Also, reference has been made herein to "determining" various types of information. Determining the information may include, for example, one or more of estimating the information, calculating the information, predicting the information, or retrieving the information from a memory.

[0228] Also, reference has been made herein to "accessing" various types of information, which may include one or more of receiving information, retrieving information (e.g., from a memory or a bitstream), storing information, moving information, copying information, calculating information, determining information, predicting information, or inferring information.

[0229] Also, reference has been made to "receiving" various types of information. Like "access," receiving is a broad term. Receiving information can include, for example, one or more of accessing information or retrieving information (e.g., from a memory or a bitstream). Also, in one or another form, terms of operations such as storing information, processing information, transmitting information, moving information, copying information, deleting information, calculating information, determining information, predicting information, or estimating information are typically associated with "receiving."

[0230] And, the term "signal" as used herein refers to instructing a corresponding decoder to do something. For example, in some embodiments, the decoder uses the signal notification flag f0 to indicate Ctx sign The residual radius r res Positive / negative sign of s res , the magnitude of the residual radius, the number of points in the point cloud, or the coordinates or order o(P1) of the first point in the 2D coordinate system (Cφ, λ), or the sensor configuration parameters, e.g. the parameter μ associated with the sensor k, or the elevation angle θ k In this way, the same parameters can be used at the encoder side and the decoder side. Thus, for example, the encoder can transmit a specific parameter to the decoder (explicit signaling), and the decoder can use the same specific parameter. Conversely, if the decoder has a specific parameter and other parameters, the signaling is used to allow the decoder to know and select the specific parameter without the need to transmit (implicit signaling). By avoiding the transmission of any actual function, each embodiment realizes bit savings. However, the signaling can be accomplished in many ways. For example, in various embodiments, one or more syntax elements, flags, etc. are used to transmit a signal to inform the corresponding decoder of information. Although the verb form of "signal" was mentioned in the previous sentence, the term "signal" can also be used as a noun in this specification.

[0231] Several embodiments have been described above. However, it should be understood that various modifications are possible. For example, elements of different embodiments can be combined, supplemented, modified, or removed to produce other embodiments. Additionally, one of ordinary skill in the art will appreciate that the disclosed structures and processes can be substituted with other structures and processes, which result in embodiments that perform at least essentially the same function(s) in at least essentially the same way(s) and thereby achieve at least essentially the same result(s) as the disclosed embodiments. Accordingly, the present application contemplates these and other embodiments.

Claims

1. 1. A method of encoding a point cloud into a bitstream of encoded point cloud data representing a physical object, wherein each point of the point cloud is associated with a residual radius between a radius dependent on a distance of the point to a reference and a predicted radius, the method comprising entropy coding a sign (s) of a residual radius (rres) associated with a current point of the point cloud based on a sign (sres,prev) of a previously entropy coded non-zero residual radius (rres,prev) associated with a previous point of the point cloud.

2. 2. The method of claim 1, wherein each point of the point cloud is associated with coordinates including an azimuth angle (Cφ) indicating an angle of capture by a sensor of a spin sensor head, and a sensor index (λ) associated with the sensor capturing the point, and wherein point cloud points having the same coordinates (Cφ, λ) are adaptively reordered based on point cloud points captured at different azimuth coordinates (Cφ) with the same sensor index (λ).

3. The method of claim 1, wherein the entropy coding of the sign (sres) of the residual radius (rres) of the current point is context-based entropy coding using a context (Ctxsign) based on the sign (sres,prev) of a previously entropy coded non-zero residual radius (rres,prev).

4. A method according to any one of claims 1 to 3, wherein the current point and the previous point are captured by different sensors of a spin sensor head.

5. A method according to any one of claims 1 to 3, wherein the current point and the previous point are captured by the same sensor of a spin sensor head.

6. The method of claim 3, further comprising reordering the points of the point cloud based on a particular order, the previous point being the last, previous reordered point associated with an entropy coded non-zero residual radius (rres,prev).

7. The method of claim 6, wherein each point of the point cloud is associated with coordinates including an azimuth angle (Cφ) indicating an angle of capture by a sensor of a spin sensor head, and a sensor index (λ) associated with the sensor capturing the point, and the context (Ctxsign) is further obtained from first binary data (Dlast) indicating whether a current point and a previous point among the sorted points were captured at the same azimuth angle.

8. A method as described in claim 6 or 7, wherein the context (Ctxsign) is further obtained from second binary data (Dpenult) indicating whether the current point is at least the third point among the sorted points.

9. A method according to any one of claims 6 to 8, wherein the points of the point cloud are associated with nodes of a tree, and the context (Ctxsign) is further obtained from a third binary data (Dprevious) indicating whether a predicted radius of a residual radius associated with a current point is determined by a parent node of the current point.

10. The context (Ctxsign) is A first binary datum (Dlast); A second binary data (Dpenult); The third binary data (Dprevious); The method according to any one of claims 6 to 9, wherein the information is obtained from a context table (ctxTab) having as entries at least one of the following information: the positive / negative sign (sres,prev) of a previously entropy-coded non-zero residual radius (rres,prev);

11. 1. A method of decoding a point cloud from a bitstream of encoded point cloud data representing a physical object, wherein each point of the point cloud is associated with a residual radius between a radius dependent on a distance of the point to a reference and a predicted radius, the method comprising entropy decoding a sign (s) of a residual radius (rres) associated with a current point of the point cloud based on a sign (sres,prev) of a previously entropy decoded non-zero residual radius (rres,prev) associated with a previous point of the point cloud.

12. 12. The method of claim 11, wherein the entropy decoding of the sign (sres) of the residual radius (rres) of the current point is a context-based entropy decoding using a context (Ctxsign) based on the sign (sres,prev) of a previously entropy decoded non-zero residual radius (rres,prev).

13. the current point and the previous point are captured by different sensors of a spin sensor head; 13. The method according to claim 11 or 12.

14. 13. The method of claim 11 or 12, wherein the current point and the previous point are captured by the same sensor of a spin sensor head.

15. 13. The method of claim 12, further comprising reordering the points of the point cloud based on a particular order, the previous point being the last, previous, reordered point associated with an entropy decoded non-zero residual radius (rres,prev).

16. 16. The method of claim 15, wherein each point of the point cloud is associated with coordinates including an azimuth angle (Cφ) indicating an angle of capture by a sensor of a spin sensor head, and a sensor index (λ) associated with the sensor capturing the point, and the context (Ctxsign) is further derived from first binary data (Dlast) indicating whether a current point and a previous point among the sorted points were captured at the same azimuth angle.

17. 17. The method of claim 15 or 16, wherein the context (Ctxsign) is further derived from second binary data (Dpenult) indicating whether the current point is at least the third point among the reordered points.

18. 18. The method according to any one of claims 15 to 17, wherein the points of the point cloud are associated with nodes of a tree, and the context (Ctxsign) is further obtained from a third binary datum (Dprevious) indicating whether a predicted radius of a residual radius associated with a current point depends on a parent node of the current point.

19. The context (Ctxsign) is A first binary datum (Dlast); A second binary data (Dpenult); The third binary data (Dprevious); The method according to any one of claims 15 to 18, wherein the information is obtained from a context table (ctxTab) having as entries at least one of the following information: the positive / negative sign (sres,prev) of a previously entropy-coded non-zero residual radius (rres,prev); and

20. 1. An apparatus for encoding a point cloud into a bitstream of encoded point cloud data representing a physical object, wherein each point of the point cloud is associated with a residual radius between a radius dependent on a distance of the point to a reference and a predicted radius, the apparatus comprising one or more processors configured to entropy code a sign (s) of a residual radius (rres) associated with a current point of the point cloud based on a sign (sres,prev) of a previously entropy coded non-zero residual radius (rres,prev) associated with a previous point of the point cloud.

21. 1. An apparatus for decoding a point cloud from a bitstream of encoded point cloud data representing a physical object, wherein each point of the point cloud is associated with a residual radius between a radius depending on a distance of the point to a reference and a predicted radius, the apparatus comprising one or more processors configured to perform entropy decoding on a sign (s) of a residual radius (rres) associated with a current point of the point cloud based on a sign (sres,prev) of a previously entropy decoded non-zero residual radius (rres,prev) associated with a previous point of the point cloud.

22. A computer program which, when executed by one or more processors, causes the one or more processors to perform a method for decoding a point cloud from a bitstream of encoded point cloud data representing a physical object, wherein each point of the point cloud is associated with a residual radius between a radius according to a distance of the point to a reference and a predicted radius, the method comprising entropy decoding a sign (sres) of a residual radius (rres) associated with a current point of the point cloud based on a sign (sres,prev) of a previously entropy decoded non-zero residual radius (rres,prev) associated with a previous point of the point cloud.

23. 1. A non-transitory storage medium carrying program code instructions, the program code being used to execute a method for decoding a point cloud from a bitstream of encoded point cloud data representing a physical object, each point of the point cloud being associated with a residual radius between a radius dependent on a distance of the point to a reference and a predicted radius, the method comprising: entropy decoding a sign (s) of a residual radius (rres) associated with a current point of the point cloud based on a sign (sres,prev) of a previously entropy decoded non-zero residual radius (rres,prev) associated with a previous point of the point cloud.

Images