Method and apparatus for encoding / decoding point cloud geometry data using orientation coding mode
The azimuth coding mode with angular interpolation addresses inefficiencies in existing point cloud compression methods by optimizing context selection, enhancing encoding and decoding efficiency while reducing computational complexity.
Patent Information
- Application Number
- JP2023559993
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2021-04-08
- Filing Date
- 2021-10-13
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-10-13
AI Technical Summary
Existing point cloud compression methods, such as MPEG-I Part 5 and Part 9, face inefficiencies in encoding and decoding sparse point clouds due to suboptimal context selection in azimuthal coding modes, leading to non-optimal compression efficiency and increased computational complexity.
An azimuth coding mode that uses angular interpolation to determine the angle φ based on an offset amount and inheritance angle relative to the parent interval for encoding and decoding point clouds, reducing the need for systematic arctangent function calculations while maintaining flexibility and improving compression efficiency.
The proposed method enhances compression efficiency and maintains reasonable computational complexity by optimizing context selection in azimuthal coding, improving encoding and decoding performance for point clouds.
Smart Images

Figure 0007588733000021 
Figure 0007588733000022 
Figure 0007588733000023
Abstract
Description
Cross-references to related applications
[0001] This application claims priority to European Patent Application No. EP21305455.4, filed April 8, 2021, the contents of which are incorporated by reference in their entirety into this disclosure. [Technical field]
[0002] This application relates generally to point cloud compression, and more specifically to a method and apparatus for encoding / decoding point cloud geometry data using an orientation coding mode. [Background technology]
[0003] This section is intended to introduce the reader to various aspects of the art, which may be related to various aspects of at least one exemplary embodiment of the present application that is described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to better understand all aspects of the present application.
[0004] Point clouds have recently gained attention as a format to represent 3D data. It varies in its ability to represent all types of physical objects or scenarios. Point clouds can be used for various purposes, such as cultural heritage / architecture, where objects like sculptures or buildings are scanned in a 3D way to share the spatial configuration of the object without transmitting or accessing it. It is also a way to ensure that knowledge of the object is preserved in case it may be destroyed. For example, a temple destroyed in an earthquake. Usually, this type of point cloud is static, colored and huge.
[0005] Another use case is in topography and cartography, where 3D representations are used to allow maps to be not limited to flat surfaces but to include relief. Google Maps is currently a good example of a 3D map, but it uses meshes rather than point clouds. However, point clouds can be an appropriate data format for 3D maps, and usually this type of point cloud is static, colored, and large.
[0006] Virtual reality, an immersive world, has been a hot topic recently and is seen by many as the future of 2D tablet video. The basic idea is to immerse the viewer in the environment around them, as opposed to standard TV where the viewer can only see the virtual world in front of them. There are several levels of immersion, depending on the degree of freedom the viewer has in their environment. Point clouds are a good format candidate to distribute virtual reality (VR) worlds.
[0007] The automotive industry and autonomous vehicles are also areas where point clouds can be used. Autonomous vehicles should "sense" their environment to make appropriate driving decisions based on the realistic conditions in the vicinity.
[0008] A point cloud is a collection of points located in three-dimensional (3D) space, optionally with additional values attached to each point. These additional values are typically called attributes. An attribute may be, for example, a three-component color, a material property (e.g., reflectance), and / or a two-component surface normal vector associated with the point.
[0009] A point cloud is therefore a combination of geometry (the positions of points in 3D space, expressed in 3D Cartesian coordinates x, y, z) and attributes.
[0010] Point clouds can be captured by various types of devices, such as an array of cameras, depth sensors, lasers (light detection and ranging), or they can be computer-generated (e.g., in film post-production). Depending on the use case, for cartography, a point cloud may contain thousands to billions of points. The original representation of a point cloud requires a very large number of bits per point, at least a dozen bits per Cartesian coordinate x, y, or z, and optionally provides more bits for one or more attributes, e.g., three times the 10 bits for color.
[0011] For many applications, it is important to be able to distribute dynamic point clouds to end users (or store them on a server) while consuming only a reasonable number of bitrates (or storage space to store the application) while maintaining an acceptable (or preferably very good) quality of experience. Efficient compression of these dynamic point clouds is a key point for the practical realization of many immersive world distribution chains.
[0012] For distribution and visualization by end users, e.g., in AR / VR glasses or any other 3D-enabled device, the compression may be lossy (e.g., in the case of video compression), but other use cases, such as medical applications or autonomous driving, require lossless compression in order not to alter the decisions obtained from the analysis of the compressed and transmitted point cloud.
[0013] Until recently, the mass market did not solve the problem of point cloud compression (aka PCC) and there was no standardized point cloud codec. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11, also known as the Moving Picture Experts Group or MPEG, started a work project on point cloud compression. This resulted in two standards: MPEG-I Part 5 (ISO / IEC 23090-5) or Video-Based Point Cloud Compression (V-PCC) MPEG-I Part 9 (ISO / IEC 23090-9) or Geometry-Based Point Cloud Compression (G-PCC)
[0014] The V-PCC coding method compresses point clouds by multiple projections onto a 3D object to obtain 2D patches that are packaged into an image (or video in the case of processing displaced point clouds). It compresses images and videos acquired with existing image / video codecs, making the most of image and video solutions already in place. V-PCC is essentially only efficient for dense and continuous point clouds, since image / video codecs cannot compress non-smooth patches obtained from projections of sparse geometry data captured by laser lidar, for example.
[0015] The G-PCC coding method provides two schemes for compressing the captured sparse geometry data.
[0016] The first approach is based on an occupancy tree (octree) representation of the point cloud geometry. The occupied nodes are split until a certain size is reached, and the occupied leaf nodes provide the 3D location of the points. The leaf nodes are usually represented by a volume, usually a cube with a predefined size at a known 3D location. The center of this volume can define the 3D location of the point. Alternatively, an estimated direct coding mode (IDCM) can directly code the 3D location belonging to the point in the leaf node of the occupancy tree. By directly coding the 3D location of one or more points in a node with a non-minimal size, sparse point clouds can benefit from the estimated direct coding mode (IDCM) by stopping the tree structure when only isolated points exist in the node.
[0017] The second approach is based on a prediction tree, where each node represents the 3D location of a point and the relationship between nodes is spatial prediction from parent to child. This approach only solves sparse point clouds and offers the advantages of lower latency and easier decoding than occupancy trees. However, compared to the first occupancy-based approach, it only improves compression performance slightly, is more complex to encode, and requires a concentrated effort to find the best predictor (among a long sequence of potential predictors) when building the prediction tree.
[0018] In the first scheme, an orientation coding mode is introduced to improve the IDCM applied to the current leaf node. Essentially, the orientation coding mode uses the azimuth angle of the coded point (belonging to the coded leaf node) to provide a set of bits to code the x or y coordinate of a point in the point cloud. This greatly improves the compression of this point relative to the location of the current leaf node. First, based on the x and y coordinates of point P (in the current node) relative to the x and y axes, an orientation coding direction (x or y axis along the horizontal plane) is selected, and relative to this orientation coding direction, the coordinates of point P are context self-adaptive binary entropy coded. For example, x node andy node are the x and y coordinates of the current leaf node position, and |x node |≦|y node If |, then the orientation coding direction is along the x-axis, otherwise along the y-axis. For example, x node andy node may be the x and y coordinates of the center of the current leaf node. (x P ,y P ,z P ) to denote the coordinates of point P. If the chosen orientation coding direction is along the x (or y) axis, we process the following coordinate encoding: First, y P (or x P ) Coordinate bits are coded, Next, x P (or y P ) coordinate bits are context self-adaptively binary entropy coded; Finally, z of point P P The coordinates are entropy coded.
[0019] First encoding y P (or x P ) can be implemented using bypass entropy coding, i.e., y P (or x P) are simply shoved into the bitstream. Alternatively, a context self-adaptive binary entropy coding can be used to first encode y P (or x P ) can be executed.
[0020] On the drawings, angles are explicitly shown as points on a line segment of the support interval. An angle corresponds to a unique point on the line segment, and every point corresponds to a unique azimuth angle. In what follows, the azimuth angle associated with a point is defined between the line joining the origin of the 2D (xy) horizontal plane to this point, and the x-axis on the other side. This corresponds to the classic definition of the azimuth angle φ.
number
[0021] The invention is not limited to such a definition of the azimuth angle, and for example, an alternate axis can be used as the origin of the angle (φ=0).
[0022] FIG. 1 is a block diagram illustrating steps of a prior art method 100 for encoding the x-coordinate of a point using an orientation coding mode where the orientation coding direction is along the x-axis.
[0023] Method 100 is described in detail when the orientation coding direction is along the x-axis. When the orientation coding direction is along the y-axis, an equivalent method can be derived by replacing the x-coordinate and x-axis with the y-coordinate and y-axis, respectively, and reversing the directions.
[0024] In step 101, select (x P ,y P ,z P ) coordinate, select an orientation coding direction (here along the x-axis) and find one point coordinate (here x P ) is context self-adaptive binary entropy coded in bitstream B.
[0025] In step 102, another point coordinate (here, y axis) is calculated along the opposite direction. P ) is bypass entropy coded into bitstream B.
[0026] In step 103, a predicted azimuth angle φ pred This azimuth angle prediction angle φ pred is the azimuth angle φ of the (closest) coded point al and a multiple n of the fundamental azimuth shift Δφ that represents the sensor rotation sampling. The integer n is obtained by multiplying φ by the azimuth angle associated with the center of the current node. pred =φ al +nΔφ can be chosen to obtain as the closest possible angle (among all possible choices of n).
[0027] In step 104, a parent interval (here, the x-interval) along the orientation coding direction (here, the x-axis) is initialized.
[0028] Graphically, as shown in Figure 2, 2D xy In the plane, this interval belongs to a line parallel to the orientation coding direction (here the x-axis) and corresponds to the inverse coded coordinate (here the y P ) (here x-) section has this line (y=y P ) and the 2D of the current node xy This is the point of intersection with the projection onto the plane.
[0029] In step 105, the x interval is divided into two half intervals, a left half interval and a right half interval. left,d and φ right,d Determine the angle φ left,d is the angle associated with the left half interval, and the angle φ right,d is the angle associated with the right half interval.
[0030] In step 106, the predicted angle φ pred and two angles φ left,d and φ right,d Select a context based on.
[0031] For example, consider FIG. 3, which shows an example point whose location in the interval along the x-axis (the selected orientation coding direction) is three bits b d (d=1, 2, 3) and represents the position of this point among the 8 possibilities. The first bit (b1) is equal to 0 if the coding point is one of the four points on the left, and 1 if the coding point is one of the four points on the right. The predicted angle φ pred provides some kind of predicted information about the point location, and the predicted angle φ pred Since b points to the left half interval, statistically, the coding point is in the left half interval (i.e., b d =0) belongs to the right half interval (i.e., b d =1). The angle difference |φ pred -φ left,d )| and |φ pred -φ right,d By comparing | and selecting the context accordingly, we can easily predict the angle φ pred Benefit from the information carried by the |φ pred- φ left,d |<|φ pred -φ right,d If |, select the first context and press b d The above is entropy coded. If not, select the second context and press b d The following is entropy coded:
[0032] In step 107, by using the selected context, x P The most significant remaining bit b d The bit b is then encoded using context self-adaptive binary entropy coding. d x P is the left half interval (b d =0) or right half interval (b d =1).
[0033] In step 108, the x interval is P The coding bit b d If is equal to 0, then x P is in the left half interval, and the x interval is updated to the left half interval. d If is equal to 1, then x P is in the right half interval, and updates this interval to the right half interval.
[0034] Next, the method checks whether the xP encoding is complete. For example, the method checks whether the updated interval size is less than or equal to the minimum x interval size or whether all bits b d is equivalently checked whether it has been encoded.
[0035] x P If coding of z is complete (i.e., for example, the x interval dimension is less than or equal to the minimum x interval dimension), then in step 109, the remaining coordinates z P is encoded into codestream B.
[0036] x P If the coding of xP is not completed, repeat step 105 and select the next bit b d+1 Code the following.
[0037] Figure 4 shows bit b d x P indicates that x is in the right half-interval. Then, in step 105, the right half-interval of the current iteration (depth d-1) is split into two half-intervals in the next iteration (depth d).
[0038] In the current test model of the G-PCC standard, in step 105, a position x that represents (but is not limited to) the middle of two half intervals is selected. left,d and x right,d From two angles φ left,d and φ right,dOne direct way to calculate the two angles is to use the arctangent formula, as shown in Figure 5.
number
[0039] On some platforms, the computational cost of computing the two arctangent functions may be an issue.
[0040] Each b of the point coordinates d In order to avoid calculating two arctangents for coding along the azimuth coding direction of x, an embodiment of the G_PCC test model is left,d and x right,d Use a specific position of x left,d is the leftmost boundary of the x interval, and x right,d is in the middle of the x interval. This means that the updated (half) interval inherits the angle of its parent interval, and the updated angle φ left,d can be obtained. b d-1 If = 0, the angle φ left ,d=φ left,d-1 It is. b d-1 If = 1, then the angle φ left ,d=φ right,d-1 It is.
[0041] In this way, we perform only one arctangent function per iteration (per depth d) to find φ right,d Calculate.
[0042] The angle φ shown in FIG. left,d and φ right,d Therefore, the G_PCC test model embodiment sacrifices compression efficiency because the two angles cannot well represent the middle of the half interval, affecting the simplicity of the calculation.
[0043] To understand why the compression efficiency is not optimal, look back at the example in Figure 6. In a context self-adaptive binary entropy codec (e.g., CABAC), the bit b d When coding, the coding probability naturally evolves to p(b d =1)<0.5, p(b d =1)>0.5. As a general rule, p is the bit b d Assuming that p is the probability of p being 1, then the cost of coding a 1 is −log2(p) and the cost of coding a 0 is −log2(1−p).
[0044] As can be seen from Fig. 5, the angle φ left,d points to the middle of the left half interval, and the angle φ right,d If points to the middle of the right half interval, the context selection is b d =0 or b d = 1, the probability associated with the selected context corresponds to a reduction in coding cost.
[0045] FIG. 7 shows the same example points, but with the angle φ left,d and φ right,d , which is realized in the G-PCC test model, similar to the prior art shown in FIG. 6. In this example, the predicted angle φ pred still refers to the left half interval, and b d =0 is possible d = 1. However, for this particular angle setting, |φ pred -φ left,d |≧|φ pred -φ right,d Since it has |, the context selection selects the second context.
[0046] b d = 0 is likely to be coded in the second context, and the probability of the second context is p(b d =0)=1-p(b d=1) < 0.5 and the context cost -log2(1-p(b d = 1) is high, such an unfavorable context selection is d This affects the cost of coding.
[0047] Therefore, the angle φ for context selection in the current test model (TMC13) left,d and φ right,d is slightly suboptimal, resulting in non-optimal compression efficiency.
[0048] Therefore, the optimal angle φ for improving compression efficiency while maintaining reasonable complexity is left,d and φ right,d It is very necessary to obtain . Essentially, keeping the complexity reasonable means minimizing the computation of mainly complex arctangent functions.
[0049] It is with the above in mind that at least one exemplary embodiment of the present application is designed. Summary of the Invention [Problem to be solved by the invention]
[0050] The following section presents a simplified summary of at least one exemplary embodiment in order to provide a basic understanding of some aspects of the present application. This summary is not a detailed overview of the exemplary embodiment. It is not intended to identify key or critical elements of the embodiment. The following summary presents only some aspects of at least one exemplary embodiment in a simplified form as a prelude to the more detailed description provided elsewhere in the document.
[0051] Briefly, the present invention relates to an orientation coding mode that uses angular interpolation to determine an angle φ based on an offset amount and a successive angle relative to a parent interval for coding the preceding bits. left,d and φ right,d It is decided that:
[0052] The present invention calculates the angle φ left,d Arctangent function of and angle φ right,d This avoids systematically performing the arctangent function of the angle φ. Compared to the orientation coding modes described above, this improves compression efficiency while maintaining reasonable complexity. left,d and φ right,d It also provides flexibility in selection.
[0053] According to a first aspect of the present application, there is provided a method for encoding a point cloud into a bitstream of encoded point cloud data representing a physical object, the method including an orientation coding mode providing a sequence of bits for encoding coordinates of points of the point cloud, the method comprising: - dividing the parent interval to which the point coordinates belong into a left half interval and a right half interval; - interpolating, from among at least one inherited angle associated with a parent interval, a left angle associated with a left half-interval and a right angle associated with a right half-interval; - selecting a context based on a left corner and a right corner; - context self-adaptive binary entropy coding the bits into a bitstream based on the selected context, the coded bits indicating which of two half-intervals the point coordinates belong to.
[0054] According to a second aspect of the present application, there is provided a method for decoding a point cloud from a bitstream of encoded point cloud data representing a physical object, the method including an orientation coding mode providing a sequence of bits for decoding coordinates of points of the point cloud, the method comprising: - dividing the parent interval to which the point coordinates belong into a left half interval and a right half interval; - interpolating, from among at least one inherited angle associated with a parent interval, a left angle associated with a left half-interval and a right angle associated with a right half-interval; - selecting a context based on a left corner and a right corner; - context self-adaptive binary entropy decoding of bits from the bitstream based on a selected context, the decoded bits indicating which of two half-intervals the point coordinate belongs to.
[0055] According to an example embodiment, the left corner is interpolated by adding the first angle and the weighted difference between the second and third angles, and the right corner is interpolated by adding the fourth angle and the weighted difference between the fifth and sixth angles.
[0056] According to an example embodiment, the first, second, third, fourth, fifth, and sixth angles are inherited angles associated with the parent interval.
[0057] According to an example embodiment, the first, third, and fourth angles are all equal to the same inherited angle associated with the parent interval, and the second, fourth, and fifth angles are all equal to a different inherited angle associated with the midpoint of the parent interval, and the angle associated with the midpoint of the left half interval or the right half interval is calculated by performing an arctangent function.
[0058] According to an example embodiment, the first, third and fifth angles are all equal to the same inherited angle, and the second, fourth and sixth angles are all equal to another inherited angle equal to the upper limit of the parent interval, and an arctangent function is performed to calculate the angle associated with the upper limit of the left half interval or the angle associated with the lower limit of the right half interval.
[0059] According to an exemplary embodiment, the angle associated with the upper limit of the left half-interval or the angle associated with the lower limit of the right half-interval is equal to the inherited angle associated with the midpoint of the parent interval.
[0060] According to an exemplary embodiment, to encode the first bit, an angle associated with the upper limit of the left half interval or an angle associated with the lower limit of the right half interval is calculated by performing an arctangent function, and to encode the subsequent bit, the angle associated with the upper limit of the left half interval or the angle associated with the lower limit of the right half interval is equal to the inherited angle associated with the midpoint of the parent interval.
[0061] According to an example embodiment, the inherited angle associated with the midpoint of the parent interval is interpolated from the two inherited angles associated with the parent interval.
[0062] According to an exemplary embodiment, if the dimension of the parent interval is greater than or equal to a determined threshold, the inherited angle associated with the midpoint of the parent interval is calculated by performing an arctangent function, or, otherwise, is interpolated from the two inherited angles associated with the parent interval.
[0063] According to a third aspect of the present application, there is provided an apparatus for encoding a point cloud into a bitstream of encoded point cloud data representing a physical object, the apparatus comprising one or more processors configured to perform a method according to the first aspect of the present application.
[0064] According to a fourth aspect of the present application, there is provided an apparatus for decoding a point cloud from a bitstream of encoded point cloud data representing a physical object, the apparatus comprising one or more processors configured to perform a method according to the second aspect of the present application.
[0065] According to a fifth aspect of the present application, A computer program is provided When the program is executed by one or more processors, it causes the one or more processors to perform a method of encoding a point cloud according to a first aspect of the present application into a bitstream of encoded point cloud data representing a physical object.
[0066] According to a sixth aspect of the present application, there is provided a non-transitory storage medium carrying program code instructions, the instructions being used to execute a method according to the first aspect of the present application for encoding a point cloud into a bitstream of encoded point cloud data representing a physical object.
[0067] According to a seventh aspect of the present application, Provide computer programs When the program is executed by one or more processors, it causes the one or more processors to perform a method for decoding a point cloud from a bitstream of encoded point cloud data representing a physical object according to the second aspect of the present application.
[0068] According to an eighth aspect of the present application, there is provided a non-transitory storage medium carrying program code instructions, the instructions being used to perform a method of decoding a point cloud from a bitstream of encoded point cloud data representing a physical object according to the second aspect of the present application, the method including an orientation coding mode providing a sequence of bits for decoding coordinates of points of the point cloud.
[0069] The specific nature of at least one of the exemplary embodiments, as well as other objects, advantages, features, and applications of at least one of said exemplary embodiments, will become apparent from the following description of examples taken in conjunction with the drawings. [Brief description of the drawings]
[0070] Reference is now made, by way of example, to the drawings illustrating exemplary embodiments of the present application, wherein: [Figure 1] FIG. 2 is a block diagram illustrating method steps for encoding the x-coordinate of a point using an orientation coding mode when the orientation coding direction is along the x-axis according to the prior art. [Diagram 2] FIG. 2 illustrates the definition of an interval along the orientation coding direction according to the prior art; [Diagram 3] FIG. 2 is a schematic diagram of updating the interval in the orientation coding mode of FIG. 1 when a point coordinate is predicted to belong to the left half interval according to the prior art; [Figure 4] FIG. 13 illustrates updating the x-interval if bit bd indicates that xP is in the right half-interval according to the prior art. [Diagram 5] FIG. 2 shows an exemplary embodiment for determining two angles φleft,d and φright,d according to the prior art. [Figure 6] FIG. 2 shows an exemplary embodiment for calculating two angles φleft,d and φright,d according to the prior art. [Figure 7] FIG. 7 illustrates the drawbacks of the exemplary embodiment of FIG. 6 for calculating the two angles φleft,d and φright,d according to the prior art. [Figure 8] FIG. 2 is a block diagram illustrating method steps for encoding the x-coordinate of a point using an orientation coding mode when the orientation coding direction is along the x-axis in accordance with at least one example embodiment. [Figure 9] FIG. 2 illustrates an example embodiment for interpolating two angles φleft,d and φright,d in accordance with at least one example embodiment. [Figure 10] FIG. 2 illustrates an example embodiment for interpolating two angles φleft,d and φright,d in accordance with at least one example embodiment. [Figure 11] FIG. 11 illustrates a variation of the example embodiment of FIG. 10 for interpolating two angles φleft,d and φright,d in accordance with at least one example embodiment. [Figure 12] FIG. 13 illustrates an exemplary embodiment for calculating the middle of an interval. [Figure 13] FIG. 3 is a block diagram illustrating steps of a method 300 for decoding an x-coordinate of a point using an orientation coding mode where the orientation coding direction is along the x-axis in accordance with at least one example embodiment. [Figure 14] 1 is a schematic block diagram of an example of a system in which various aspects and example embodiments may be implemented.Like reference numerals used in different figures may represent like components. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0071] At least one example of the exemplary embodiment will be described below with reference to the drawings, and at least one example of the exemplary embodiment will be illustrated. However, the exemplary embodiment may be implemented in many alternative forms and should not be construed as being limited to the examples set forth herein. It should therefore be understood that the exemplary embodiment is not intended to be limited to the particular forms disclosed. On the contrary, the present application is intended to cover all modifications, equivalents and alternatives falling within the spirit and scope of the present application.
[0072] It should be understood that when a drawing is presented in the form of a flow chart, a block diagram of the corresponding apparatus is also provided. Similarly, it should be understood that when a drawing is presented in the form of a block diagram, a flow chart of the corresponding method / process is also provided.
[0073] At least one of these aspects relates generally to encoding and decoding point clouds, and at least one other aspect relates generally to transmission of the generated or encoded bitstream.
[0074] Additionally, the aspects are not limited to MPEG standards such as MPEG-I Part 5 or Part 9 relating to point cloud compression, etc., but may be applied, for example, to other standards and recommendations, whether existing or developed in the future, and extensions of such standards and recommendations, including MPEG-I Part 5 and Part 9. Unless otherwise indicated or technically excluded, the aspects described in this application may be used alone or in combination.
[0075] The present invention relates to the field of encoding and decoding technology, and aims to provide a technical solution for encoding / decoding point cloud data. Because point cloud is a collection of a large amount of data, a large amount of memory is consumed by the storage point cloud, and the point cloud cannot be directly transmitted in the network layer without compressing the point cloud, so the point cloud needs to be compressed. Therefore, as point cloud is increasingly widely used in fields such as autonomous navigation, real-time patrol inspection, geographic information services, cultural heritage / architecture protection, 3D immersive communication and interaction, the present invention can be used in many application scenarios.
[0076] The present invention relates to an orientation coding mode, which uses angular interpolation to determine an angle φ based on the inherited angle and an offset amount relative to the parent interval for coding the preceding bits. left,d and φ right,d The orientation coding modes disclosed in this disclosure can be used in methods for encoding / decoding point clouds to improve compression efficiency while maintaining reasonable complexity.
[0077] FIG. 8 is a block diagram illustrating steps of a method 200 for encoding the x-coordinate of a point using an orientation coding mode where the orientation coding direction is along the x-axis in accordance with at least one example embodiment.
[0078] Method 200 is an improvement of method 100 of Fig. 1. Select an exemplary point P from the current (leaf) node of the occupancy tree (step 101). The point location along the selected orientation coding direction is represented by bit b d where each bit is an x interval along the x axis. P In the following, the selected orientation coding direction is along the x-axis, but if the orientation coding direction is along the y-axis, the equivalent method 200 can be derived by replacing the x-coordinate and x-axis with the y-coordinate and y-axis, respectively, and reversing the direction. P ) is coded along the reverse direction (here, the y-axis) (step 102). pred(step 103). The x interval along the x axis is initialized (step 104).
[0079] In step 201, the angle φ of the current node node Obtain this angle φ node is already known and can be retrieved from memory or calculated with the formula:
number
[0080] In step 202, the parent interval is divided into two half intervals, a left half interval and a right half interval. The parent interval is the x interval initialized in step 104 or the leading bit bd -1 is the x interval for encoding.
[0081] In step 203, from among at least one inherited angle associated with the parent interval, a left angle φ associated with the left half interval is selected. left,d and the right angle φ associated with the right half interval right,d and are interpolated.
[0082] In step 106, the predicted angle φ pred and two angles φ left,d and φ right,d Select a context based on
[0083] In step 107, by using the selected context, x P The most significant remaining bits for b d into bitstream B. The bits are x P is the left half interval (b d =0) or right half interval (b d =1).
[0084] According to an exemplary embodiment, the context self-adaptive binary entropy coding is context self-adaptive binary arithmetic coding (CABAC).
[0085] In step 108, the parent interval is x P The coded bit b is updated to the half interval to which it belongs (as indicated by the coded bit). d If is equal to 0, then x P is in the left half interval, and the parent interval is updated to the left half interval. d If is equal to 1, then x P is in the right half interval, and the parent interval is updated to the right half interval.
[0086] Next, the method checks whether the xP encoding is complete.
[0087] x P Once the encoding is complete, in step 109, the remaining coordinates z P Encode into bitstream B.
[0088] x P If encoding is not complete, repeat step 202 and add x P The next bit b d+1 Encode the following.
[0089] According to an exemplary embodiment of step 203, a first angle (φ 1,d ) and the second angle (φ 2,d ) and the third angle (φ 3,d ) and the weighted difference between them, the left angle φ left,d and the fourth angle (φ 4,d ) and the fifth angle (φ 5,d ) and the sixth angle (φ 6,d ) and the weighted difference between them, the right angle φ right,d Interpolate.
[0090] This interpolation can be given by the following formula:
number
[0091] Angle φ left,d and φ right,d may not systematically point halfway between the left and right half intervals. The parameter Off left and Off right By freely setting the value of φ left,d and φ right,d Provides flexibility in the selection of
[0092] According to an exemplary embodiment of step 203 shown in FIG. 9, the angle φ in equation (2) 1,d , φ 2,d , φ 3,d , φ 4,d , φ 5,d , φ 6,d are the inherited angles related to the parent interval, and φ 1,d =φ 3,d =φ 6,d =φ 1,d-1 , φ 2,d= φ 4,d =φ 5,d =φ middle,d-1 Then, the angle φ is calculated using the following formula: left,d and φ right,d and are interpolated.
number
[0093] 1st bit b d=1 To encode (first iteration), we use the inheritance angle φ 1,d-1 is the angle φ obtained from step 201 node and the inheritance angle φ middle,d-1 is associated with the midpoint of the initial x interval (step 104).
[0094] The following bit b d To encode (subsequent iterations), the inheritance angle φ 1,d-1 and φ middle,d-1 is associated with the parent interval. The dashed lines in Figure 9 indicate the inheritance chain.
[0095] For each depth, the angle φ is calculated by performing the arctangent function. middle,d can be calculated.
[0096] 1st bit b d=1 To encode (first iteration), we use the parent angle φ middle,0 is calculated as follows:
number
[0097] The following bit b d To encode (subsequent iterations), we perform an arctangent function to find the parent angle φ middle,d-1 Calculate:
number
[0098] In this exemplary embodiment, each bit b d (Equation 4 or 5) to encode the angle φ. Thus, we achieve high encoding performance and a large reduction in the number of arctangent functions required for the prior art. left,d and φ right,d The flexibility of choice and the ability to maintain the benefits it offers.
[0099] According to another exemplary embodiment of step 203 shown in FIG. 10, the angle φ in equation (2) 1,d , φ 2,d , φ 3,d , φ 4,d , φ 5,d , φ 6,d are the inherited angles related to the parent interval, and φ 1,d =φ 3,d =φ 5,d =φ 1,d-1 , φ 2,d =φ 4,d =φ 6,d =φ top,d-1 The angle φ left,d and φ right,d is interpolated using the following formula:
number
[0100] 1st bit b d=1 To encode (first iteration), we use the inheritance angle φ 1,d-1 is the angle φ obtained from step 201 node and the inheritance angle φ top,d-1 is the angle associated with the upper limit of the initial x interval.
[0101] To encode the subsequent bits bd (subsequent iterations), the succession angle φ1,d-1 and φ top,d-1 is associated with the parent interval. The dashed lines in Figure 10 indicate the inheritance chain.
[0102] At a given depth d, the angle φ associated with the upper limit of the left half interval top,d or the angle φ associated with the lower limit of the right half interval low,d is calculated by performing the arctangent function.
[0103] 1st bit b d=1 (1st iteration)) by using the angle φ top,0 is calculated by the following formula:
number
[0104] b d-1 If = 0, the left half interval is the parent interval for the next depth d (next iteration), and the angle φ top,d is calculated by performing the arctangent function.
[0105] The following bit b d To encode (subsequent iterations), we perform an arctangent function to find the angle φ top,d-1 Calculate:
number
[0106] b d-1 If =1, the right half interval is the parent interval for the next depth d, and the angle φ is calculated by performing the arctangent function. low,d Calculate.
[0107] The following bit bd To encode (subsequent iterations), the angle is low,d-1 Calculated by:
number
[0108] This exemplary embodiment is d A single arctangent function needs to be performed to encode φ (Equations 7-10). Thus, while performing the same number of arctangent functions as in the prior art, we achieve high encoding performance and a large number of errors in the angle φ left,d and φ right,d The flexibility of choice and the ability to maintain the benefits it offers.
[0109] In a first variant of the embodiment of FIG. 10 shown in FIG. 11, the angle φ top,d or angle φ low,d is the inherited angle φ relative to the midpoint of the parent interval middle,d-1 is equal to.
[0110] This first variant calculates φ without invoking the arctangent function. middle,d-1 It offers lower complexity since it is not necessary to perform an arctangent function to encode each bit bd, assuming that φ has been determined. It also offers high encoding performance and a large reduction in the angle φ left,d and φ right,d Maintain flexibility in your choices.
[0111] In a second variant of the exemplary embodiment of FIG. 10, the first bit b d=1 To encode the angle φ by Eq. 7 or 9, top,d or angle φ low,d and then use the angle φ to encode the subsequent bit bd. top,d or angle φ low,d is the inherited angle φ relative to the midpoint of the parent interval middle,d-1 is equal to.
[0112] This variant requires two arctangent functions to be performed to code a series of bits, which reduces the complexity of the method and the computational resources required to realize the orientation coding mode, while still providing high coding performance (high interpolation accuracy) and the angle φ left,d and φ right,d Maintain flexibility in your choices.
[0113] In a variant of the first or second variant, the inheritance angle φ middle,d-1 are the two inherited angles φ related to the parent interval. 1,d-1 and φ top,d-1 is obtained by interpolating:
number
[0114] This variant does not require recursive invocation of the arctangent function to perform the encoding of a series of bits, reducing the complexity of the method and thus the computational resources required to realize the orientation coding mode, while still achieving high encoding performance and φ left,d and φ right,d Maintain flexibility in your choices.
[0115] In another variant of the first or second variant, if the dimension of the parent interval is greater than the determined threshold, the inheritance angle φ is calculated by performing an arctangent function (equation 4 or 5). middle,d-1 If the dimension of the parent section is lower than the determined value, the two inheritance angles φ are calculated by Eq. (11). 1,d-1 and φ top,d-1 Inherited angle φ middle,d-1 Interpolate.
[0116] This variant maintains high accuracy for nodes with significant size, for which the interpolation of equation (11) provides results that are too inaccurate, thus compromising compression performance.
[0117] Alternatively, the angle φ 1,d-1 and φ top,d-1 If the absolute value of the difference between is greater than the determined threshold, the inheritance angle φ is calculated by performing the arctangent function (Equation 4 or 5). middle,d-1 otherwise, the two inheritance angles φ 1,d-1 and φ top,d-1 Inherited angle φ middle,d-1 Interpolate.
[0118] This modification maintains the compression performance to maintain the interpolation accuracy, and reduces φ when the section size becomes sufficiently small. middle,d-1 This ensures lower complexity and is therefore advantageous.
[0119] According to an exemplary embodiment of step 203, the parameter Off left offset is the average position of the center of the left half interval left,mid is defined as the difference between the offset amount Offset2 and the parameter Off right is the average position of the center of the right half interval ffset right,mid and an offset amount Offset2:
number
[0120] The basic idea for obtaining a good context selection is to choose an angle φ close to pointing to the center of each half interval. left,d and φ right,d However, the angle φ left,d and φ right,d It has already been observed that optimal compression performance is obtained when and point exactly to the center of the half interval.
[0121] For example, since the location of the points in the left half interval is discrete, the middle of this left half interval can be easily calculated as (φ middle,d-1 -φ 1,d-1) / 2. Instead, the middle of the left half-interval is defined as the average position of all potential points that belong to the left half-interval, as shown in Figure 12.
[0122] Length is s' d Consider a half-interval where , the potential points are the lower bounds of the left half-interval at positions 0, 1, …, s' d -1. Next, the average position of the center of the left half interval, offset right , mid is given by:
number
[0123] Now, using the arithmetic progression formula, we get the following:
number
[0124] To achieve better compression efficiency, the second offset amount, offset2, is set to the average position of the centers of the left half intervals. left,mid It has already been observed that introducing left,d Push a little to the lower limit of the left half interval, and the right angle φ right,d The solution is to push slightly towards the upper limit of the right half-interval.
[0125] Right half interval related parameters Off right The reasoning is similar.
[0126] An exemplary value of offset2 is offset2=2. Also, offset2 may depend on the depth d taken: for the last depth (i.e., used to encode the last bit bd), offset2 may be equal to 2, for the penultimate depth offset2 may be equal to 1, and for other depths offset2 may be equal to 0.
[0127] In methods 200 and 300, several division operations are performed. For example,
number
[0128] Although these divisions may be problematic in practical implementations, in practice, especially in the current G_PCC TMC13 software framework, they can be replaced by shift>> operations. This replacement is feasible because the dimensions of the nodes are powers of two, and therefore the lengths of the intervals are also powers of two.
number
[0129] Equation (2) becomes:
number
[0130] Here, Ld is the length of the section s d is log2 of L d can be obtained recursively by:
number
[0131] Value2Off left is calculated by the following formula:
number
[0132] FIG. 13 is a block diagram illustrating steps of a method 300 for decoding an x-coordinate of a point using an orientation coding mode where the orientation coding direction is along the x-axis in accordance with at least one example embodiment.
[0133] In what follows, the selected orientation coding direction is along the x-axis, but if the orientation coding direction were along the y-axis, an equivalent method 300 could be derived by replacing the x-coordinate and x-axis with the y-coordinate and y-axis, respectively, and reversing the directions.
[0134] The orientation coding direction is the x and y coordinates (x node ,y node Alternatively, the orientation coding direction can be decoded from bitstream B.
[0135] In step 301, a point coordinate (here, y axis) is calculated from bit stream B along the reverse direction (here, y axis). P ) to decrypt the
[0136] In step 103, the predicted azimuth angle φ of point P is pred The x interval along the x axis is initialized (step 104). The azimuth angle φ of the (closest) decoded point is determined. al The multiple n of the base azimuth offset is determined by the decoder using: Alternatively, multiple n is decoded from bitstream B.
[0137] In step 104, a parent interval (here, the x-interval) along the orientation coding direction (here, the x-axis) is initialized.
[0138] In step 201, the angle φ of the current node node is obtained by equation (1). P has already been decoded from the bitstream and nodepos_x is a known parameter of the IDCM.
[0139] In step 202, the parent interval is divided into two half intervals, a left half interval and a right half interval. The parent interval is the x interval initialized in step 104 or the leading bit bd -1 is the x interval for decoding.
[0140] In step 203, from at least one inherited angle associated with the parent interval, a left angle φ associated with the left half interval is calculated. left,d and the right angle φ associated with the right half interval right,d and are interpolated.
[0141] In step 106, the predicted angle φ pred and two angles φ left,d and φ right,d Select a context based on
[0142] In step 302, the selected context is used to P The most significant bit of the remaining b d The bit is decoded by the context self-adaptive binary entropy decoding. P is the left half interval (b d =0) or right half interval (b d =1).
[0143] According to an exemplary embodiment, the context self-adaptive binary entropy decoding is context self-adaptive binary arithmetic coding (CABAC).
[0144] In step 108, the parent interval is x P belongs to (as indicated by the decoded bits).
[0145] Next, this method uses the coordinate x P Check whether the decryption is complete.
[0146] x P Once the decoding of is completed, in step 303, the remaining coordinates z P Decrypt the
[0147] x P If the decryption of x has not been completed, repeat step 202. P The next bit b d+1 Decrypt the
[0148] The exemplary embodiment and variations of the method 200 are applied to the method 300 .
[0149] The encoding / decoding method can be used to encode / decode point clouds, and in particular can be used for multiple purposes, in particular to encode / decode coordinates of points of a point cloud, thereby improving compression efficiency while maintaining reasonable complexity.
[0150] FIG. 14 shows a schematic block diagram of an embodiment of a system according to the present invention.
[0151] System 400 may be implemented as one or more devices and may include various components as described below. In various embodiments, system 400 may be configured to implement one or more aspects described herein.
[0152] Examples of devices that may comprise all or part of system 400 include personal computers, laptop computers, smartphones, tablets, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, connected cars and their associated processing systems, head mounted displays (HMDs, see-through glasses), projectors, "caves" (systems including multiple displays), servers, video encoders, video decoders, post-processors that process output from video decoders, pre-processors that provide input to video encoders, web servers, set-top boxes, and any other devices for processing point clouds, videos, or images, or other communications devices. The elements of system 400 may be implemented singly or in combination on a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 400 may be distributed across multiple ICs and / or discrete components. In various embodiments, system 400 may be communicatively coupled to other similar systems or other electronic devices, for example, via a communications bus or dedicated input and / or output ports.
[0153] The system 400 includes at least one processor 410 configured to execute instructions loaded therein to, for example, implement aspects described herein. The processor 410 may include embedded memory, input / output interfaces, and various other circuits known in the art. The system 400 may include at least one memory 420 (e.g., volatile and / or non-volatile memory devices). The system 400 may include a storage device 440 including non-volatile and / or volatile memory, including, but not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Flash memory, magnetic disk drives, and / or optical disk drives. As non-limiting examples, the storage device 440 may include an internal storage device, an additional storage device, and / or a network-accessible storage device.
[0154] The system 400 may include an encoder / decoder module 430 configured to, for example, process data to provide encoded / decoded point cloud geometry shape data, and the encoder / decoder module 430 may include its own processor and memory. The encoder / decoder module 430 may represent a module(s) included in a device to perform encoding and / or decoding functions. As is known, a device may include either one or both of an encoding and decoding module. Also, the encoder / decoder module 430 may be implemented as a separate element of the system 400 or may be coupled within the processor 410 as a combination of hardware and software known to those skilled in the art.
[0155] Program code loaded into the processor 410 or the encoder / decoder 430 to perform aspects described herein may be stored in the storage device 440 and subsequently loaded into the memory 420 and executed by the processor 410. According to various embodiments, during execution of processes described herein, one or more of the processor 410, the memory 420, the storage device 440, and the encoder / decoder module 430 may store one or more of the following items: point cloud frames, encoded / decoded geometry shapes / attributes videos / images or portions of encoded / decoded geometry shapes / attributes videos / images, bitstreams, matrices, variables, and expressions, formulas, logic for operations or intermediate or final results of operations.
[0156] In some embodiments, memory within the processor 410 and / or the encoder / decoder module 430 may be used to store instructions and provide working memory for processes performed during encoding or decoding.
[0157] However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 410 or the encoder / decoder module 430) is used for one or more of these functions. The external memory may be the memory 420 and / or the storage device 440, e.g., dynamic volatile memory and / or non-volatile flash memory. In some embodiments, the external non-volatile flash memory is used to store, for example, an operating system of a television. In at least one embodiment, a fast external dynamic volatile memory such as RAM may be utilized as working memory for video encoding / decoding and decoding operations, e.g., for MPEG-2 Part 2 (also known as ITU-T Recommendation H.262 and ISO / IEC 13818-2, also known as MPEG-2 Video), HEVC (High Efficiency Video Coding / Decoding), VVC (Versatile Video Coding / Decoding), or MPEG-I Parts 5 or 9.
[0158] As indicated in block 490, inputs can be provided to the elements of system 400 via various input devices. Such input devices include, but are not limited to, (i) an RF section capable of receiving RF signals transmitted wirelessly from a broadcast station or the like, (ii) a composite input terminal, (iii) a USB input terminal, and / or (iv) an HDMI input terminal.
[0159] In various embodiments, the input devices of block 490 have associated and corresponding input processing elements as known in the art. For example, the RF section may be associated with the following required elements: (i) selecting a desired frequency (also called signal selection, or limiting the signal within a frequency band), (ii) downconverting the selected signal, (iii) selecting a signal frequency band by controlling the frequency band back to a narrow frequency band, (for example) referred to as a channel in some embodiments, (iv) demodulating the downconverted and frequency band limited signals, (v) performing error correction, and (vi) demultiplexing to select a desired data packet flow. The RF section of various embodiments includes elements purposely performing these functions, such as frequency selectors, signal selectors, frequency band limiters, channel selectors, filters, downconverters, demodulators, error correction devices, and demultiplexers. The RF section may include a tuner that performs each of these functions, including, for example, downconverting the received signal to a lower frequency (e.g., an intermediate frequency or a frequency near baseband) or to baseband.
[0160] In one set-top box embodiment, the RF section and its associated input processing elements can receive RF signals transmitted over a wired (e.g., cable) medium, after which the RF section can perform frequency selection by filtering, down-converting, and re-filtering to obtain a desired frequency band.
[0161] In various embodiments, the order of these (and other) elements may be rearranged, some of these elements may be eliminated, and / or other elements that perform similar or different functions may be added.
[0162] Adding elements can include inserting elements such as amplifiers and analog-to-digital converters between existing elements, In various embodiments, the RF section includes an antenna.
[0163] Additionally, the USB and / or HDMI terminals may include corresponding interface processors and are used to connect the system 400 to other electronic devices via USB and / or HDMI connections. It should be noted that aspects of the input processing (e.g., Reed-Solomon error correction) may be implemented, for example, in a separate input processing IC or in the processor 410, when necessary. Thus, it should be understood that aspects of the USB or HDMI interface processing may be implemented, when necessary, in a separate interface IC or in the processor 410. Upon demodulation, the error corrected and demultiplexed stream may be provided to various processing elements, including the processor 410 and the encoder / decoder 430, which operates in combination with memory and storage elements, to process the data stream when necessary for display on an output device.
[0164] The various elements of system 400 may be provided within a unitary housing, where the elements may be interconnected and data transmitted between them using a suitable connection layout 490, such as internal buses (including I2C buses), wiring, and printed circuit boards, as are well known in the art.
[0165] System 400 may include a communication interface 450 such that it can communicate with other devices over a communication channel 500. Communication interface 450 includes, but is not limited to, a transceiver configured to transmit and receive data over communication channel 500. Communication interface 450 may include, but is not limited to, a modem or a network card, and communication channel 500 may be implemented within a wired and / or wireless medium, for example.
[0166] In various embodiments, a Wi-Fi network, such as IEEE 802.11, can be used to transmit the data stream to the system 400. The Wi-Fi signal in these embodiments can be received by a communication channel 500 suitable for Wi-Fi communication and a communication interface 450. The communication channel 500 in these embodiments can generally be connected to an access point or router that provides access to external networks, including the Internet, to enable streaming applications and other over-the-top communications.
[0167] Other embodiments may provide streaming data to the system 400 using a set-top box, which carries the data through an HDMI connection in the input block 490 .
[0168] There are other embodiments in which the RF connection of input block 490 is used to provide streaming data to system 400 .
[0169] The streaming data can be used as a form of signaling information used by the system 300. The signaling information includes coded bits (b d ), the orientation coding direction, the point coordinates, the multiple n of the basic orientation shift azimuth angle, the basic orientation shift azimuth angle, the parameter nodepos_x, etc.
[0170] It should be noted that signaling can be achieved in various manners, for example, in various embodiments, one or more syntax elements, flags, etc. can be used to transmit signaling information to a corresponding decoder.
[0171] System 400 can provide output signals to a variety of output devices, including a display 600, speakers 700, and other peripherals 800. In various example embodiments, other peripherals 800 can include one or more of a separate DVR, disc player, stereo system, lighting system, and other devices that provide functionality based on the output of system 400.
[0172] In various embodiments, control signals may be communicated between system 400 and display 600, speaker 700, or other peripherals 800 using AV.Link (Audio / Video Link), CEC (Consumer Electronics Control) or other communication protocol signaling enabling device-to-device control, with or without a user.
[0173] Output devices can be communicatively connected to the system 400 via dedicated connections by corresponding interfaces 460, 470 and 480.
[0174] Alternatively, an output device can be connected to system 400 using communication channel 500 via communication interface 450. Display 600 and speakers 700 can be integrated into a single unit along with other components of system 400 such as an electronic device (e.g., a television).
[0175] In various embodiments, the display interface 460 can include a display driver, such as a timing controller (T Con) chip.
[0176] For example, if the RF portion of input 490 is part of a separate set-top box, then display 600 and speakers 700 may alternatively be separate from one or more of the other components. In various embodiments where display 400 and speakers 700 may be external components, output signals may be provided via dedicated output connections (including, for example, an HDMI port, a USB port, or a COMP output terminal).
[0177] 1-14 illustrate various methods, each of which includes one or more steps or actions to achieve the described method. Unless a specific order of steps or actions is required for the precise operation of the method, the order and / or use of specific steps and / or actions can be modified or combined.
[0178] Although some examples have been described with respect to block diagrams and / or operational flow charts, each block represents a circuit element, a module, or a portion including one or more executable instruction codes for implementing a specified logic function(s). It should be noted that in other embodiments, the function(s) shown in the blocks may not occur in the order shown. For example, two blocks shown one after the other may in fact be executed essentially in parallel, or the blocks may be executed in the reverse order, depending on the functions involved.
[0179] For example, the embodiments and aspects described herein may be implemented in a method or process, an apparatus, a computer program, a data stream, a bit stream, or a signal. Even if discussed only in the context of a single type of embodiment (e.g., discussed only as a method), embodiments of the discussed features may be implemented in other forms (e.g., an apparatus or a computer program).
[0180] The methods may be implemented, for example, in a processor, which generally refers to a processing device including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device, etc. Processors further include communication devices.
[0181] Also, the methods may be implemented with instructions executed by a processor, and such instructions (and / or data values produced by the embodiments) may be stored in a computer-readable storage medium. A computer-readable storage medium may take the form of a computer-readable program product embodied in one or more computer-readable media and having computer-readable program code embodied therein and executable by a computer. Given the inherent ability to store information thereon and retrieve information provided thereby, a computer-readable storage medium as used herein may be considered a non-transitory storage medium. A computer-readable storage medium may be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. In addition, the following provides more specific examples of computer-readable storage media to which the present embodiment can be applied, but it should be understood that these are merely illustrative and not an exhaustive list, as one of ordinary skill in the art would readily recognize: portable computer floppy disks, hard disks, read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0182] The instructions may create an application tangibly embodied on a processor-readable medium.
[0183] For example, instructions may reside in hardware, firmware, software, or a combination thereof. For example, instructions may be found in an operating system, a separate application, or a combination of both. A processor may thus be characterized as, for example, a device configured to perform a process or a device that includes a processor-readable medium (e.g., a storage device) having instructions for performing a process. Additionally, in addition to or in lieu of instructions, the processor-readable medium may store data values produced by an embodiment.
[0184] The devices may be implemented in, for example, suitable hardware, software, and firmware. Examples of such devices include personal computers, laptop computers, smartphones, tablets, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, head mounted displays (HMDs, see-through glasses), projectors, "caves" (systems including multiple displays), servers, video encoders, video decoders, post-processors that process output from video decoders, pre-processors that provide input to video encoders, web servers, set-top boxes, and any other device for processing point clouds, video or images, or other communications devices. It is noted that the devices may be mobile and may be mounted in a moving vehicle.
[0185] The computer software can be implemented in the processor 410, hardware, or a combination of hardware and software. As a non-limiting example, an embodiment can be implemented in one or more integrated circuits. The memory 420 can be of any type that self-adapts to the technological environment and can be implemented in any suitable data storage technology (as a non-limiting example, for example, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memories and removable memories). As a non-limiting example, the processor 410 can be of any type that fits the technological environment and can cover one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.
[0186] As will be apparent to one skilled in the art, the embodiments can generate various signals shaped to carry, for example, storable or transmittable information. The information can include, for example, instructions for performing a method or data generated by one of the described embodiments. For example, the signal can be shaped to carry a bit stream of the described embodiments. The signal can be shaped, for example, into an electromagnetic wave (e.g., a radio frequency portion of the frequency spectrum) or a baseband signal. The shaping can include, for example, encoding the data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal can be, for example, analog or digital information. As is well known, the signal can be transmitted over different wired or wireless links. The signal can be stored in a processor readable medium.
[0187] The terms used herein are used only to describe particular embodiments and are not intended to be limiting. Unless the context clearly dictates otherwise, the singular forms "a," "an," and "the" as used herein include the plural forms. Furthermore, as used herein, the terms "include / comprise" and / or "including / comprising" may indicate the presence of a described feature, integer, step, operation, element, and / or component, etc., but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof. Also, when an element is said to be "responsive to" or "connected" to another element, it may be directly responsive to or connected to the other element, or intermediate elements may be present. Conversely, when an element is said to be "directly responsive to" or "directly connected" to another element, intermediate elements are not present.
[0188] For example, in the case of "A / B," "A and / or B," and "at least one of A and B," the use of any one of the symbols / terms " / ," "and / or," and "at least one of" is intended to cover the selection of the first listed option (A), or the selection of the second listed option (B), or the selection of two options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to cover only the selection of the first listed option (A), or only the selection of the second listed option (B), or only the selection of the third listed option (C), or only the selection of the first and second listed options (A and B), or only the selection of the first and third listed options (A and C), or only the selection of the second and third listed options (B and C), or all three options (A, B, and C). This can be extended to any number of listed items as would be apparent to one of skill in the art.
[0189] Various numerical values may be used in this application, and the specific values are exemplary, and the embodiments described are not limited to these specific values.
[0190] It should be noted that terms such as first, second, etc. may be used to describe various elements in this specification, but are not limited to these terms. These terms are used only to distinguish one element from another element. For example, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element, without departing from the teachings of this application. No ordering is implied between the first element and the second element.
[0191] References to "one exemplary embodiment" or "exemplary embodiment" or "one embodiment" or "embodiment" and other variations are often used to convey that a particular feature, structure, characteristic, etc. (as described in conjunction with the embodiment) is included in at least one embodiment / embodiment. Thus, appearances of the terms "in one exemplary embodiment" or "in an exemplary embodiment" or "in one embodiment" or "in an embodiment" and any other variations appearing in various places in this application are not necessarily all referring to the same embodiment.
[0192] Similarly, references herein to "according to an exemplary embodiment" or "in an exemplary embodiment" and other variations are often used to convey that a particular feature, structure, or characteristic (as described in conjunction with an embodiment) may be included in at least one exemplary embodiment. Thus, the phrases "according to an exemplary embodiment" or "in an exemplary embodiment" appearing in various places in the specification do not necessarily refer to the same exemplary embodiment, nor are separate or alternative exemplary embodiments mutually exclusive of other exemplary embodiments.
[0193] The reference numerals of the drawings appearing in the claims are used for explanation only and do not limit the scope of the claims. Although not explicitly described, the present embodiment / examples and modifications can be used in any combination or partial combination.
[0194] It is understood that when a figure is presented as a flow chart, a corresponding apparatus block diagram is also provided. Similarly, it is understood that when a figure is presented as a block diagram, a flow chart of the corresponding method / process is also provided.
[0195] Some figures include arrows that indicate a primary direction of communication in a communication path, however, it should be understood that communication can occur in a direction opposite to that of the illustrated arrow.
[0196] Various embodiments relate to decoding. As used herein, "decoding" may cover all or part of a process, such as performing a received point cloud frame (which may include a received bitstream encoding one or more point cloud frames) to generate a final output suitable for display or further processing in a reconstructed point cloud domain. In various embodiments, such a process may include one or more of the processes typically performed by a decoder. In various embodiments, for example, such a process may alternatively include a process performed by a decoder of various embodiments described herein.
[0197] As a further example, in one embodiment, "decoding" may refer to only inverse quantization, in one embodiment, "decoding" may refer to entropy decoding, in another embodiment, "decoding" may refer to differential decoding, and in another embodiment, "decoding" may refer to a combination of inverse quantization, entropy decoding, and differential decoding. Depending on the context in which it is specifically described, it is obvious and easy to understand for those skilled in the art whether the term "decoding process" refers specifically to a subset of operations or to a more general decoding process.
[0198] Various embodiments relate to encoding. Similar to the discussion of "decoding" above, "encoding" as used herein can cover all or part of the process of, for example, executing an input point cloud frame to generate an encoding bitstream. In various embodiments, this type of process includes one or more of the processes typically performed by an encoder. In various embodiments, such processes can alternatively include the processes performed by the encoder of each embodiment described herein.
[0199] As a further example, in one embodiment, "encoding" may refer only to quantization, in one embodiment, "encoding" may refer only to entropy coding, in another embodiment, "encoding" may refer only to differential coding, and in another embodiment, "encoding" may refer to a combination of quantization, differential coding, and entropy coding. Depending on the context in which a particular description is made, it will be clear and easy to understand for one of ordinary skill in the art whether the term "encoding process" refers specifically to a subset of operations or to a more general encoding process.
[0200] This application has also referred to "determining" various pieces of information. Determining the information may include one or more of estimating the information, calculating the information, predicting the information, or looking up the information from a memory.
[0201] This application also refers to "accessing" various pieces of information, which may include one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.
[0202] This application has also referred to "receiving" various pieces of information. Like "access," receiving is a broad term. Receiving information can include, for example, one or more of accessing information or retrieving information (e.g., from a memory). In another manner, terms of operations such as storing information, processing information, transmitting information, moving information, copying information, deleting information, calculating information, determining information, predicting information, or estimating information are typically associated with "receiving."
[0203] Moreover, as used herein, the term "signal" refers to a particular signal that a corresponding decoder may instruct, etc. For example, in one embodiment, the encoder may generate coded bits (b d ), azimuth coding direction, point coordinate, basic azimuth shift azimuth angle multiple n, basic azimuth shift azimuth angle, parameter nodepos_x, etc. are notified by transmitting a signal. In this manner, the same parameters can be used at the encoder side and the decoder side in the embodiment. Thus, for example, the encoder can transmit a specific parameter to the decoder (explicit signaling), so that the decoder can use the same specific parameter. Conversely, if the decoder has a specific parameter and other parameters, a signaling that does not require transmission (indirect signaling) is used to inform the decoder and facilitate the selection of the specific parameter. In order to avoid transmitting any actual function, bit saving is realized in various embodiments. It should be appreciated that the signaling can be completed in various ways. For example, in various embodiments, one or more grammatical elements, flags, etc. are used to transmit information to the corresponding decoder. Although the above relates to the verb form of the word "signal", the word "signal" may also be used as a noun in this specification.
[0204] Although several embodiments have been described above, it should be understood that various modifications may be made. For example, elements of different embodiments may be combined, supplemented, modified, or deleted to produce other embodiments. Also, as will be appreciated by those skilled in the art, other structures and processes may be substituted for the disclosed structures and processes, resulting in embodiments that perform essentially the same function(s) in essentially the same way(s) to achieve at least essentially the same result(s) as the disclosed embodiments. Accordingly, these and other embodiments are contemplated by the present application.
Claims
1. 1. A method for encoding a point cloud into a bitstream of encoded point cloud data representing a physical object, the method including an orientation coding mode providing a series of bits for encoding coordinates of points of the point cloud, the method comprising: - dividing the parent interval to which the coordinates of said point belong into a left half-interval and a right half-interval; - from at least one inherited angle associated with said parent interval, the left angle (φ left,d ) and the right angle (φ right,d ) and - Said left corner (φ left,d ) and the right angle (φ right,d ) selecting a context based on Based on the selected context, the bit (b d ) into the bitstream, d ) indicating which of the two half-intervals the coordinates of the point belong to, A method for encoding a point cloud into a bitstream of encoded point cloud data representing a physical object.
2. 1. A method of decoding a point cloud from a bitstream of encoded point cloud data representing a physical object, the method including an orientation coding mode providing a sequence of bits for decoding coordinates of points of the point cloud, the method comprising: - dividing the parent interval to which the coordinates of said point belong into a left half-interval and a right half-interval; - from at least one inherited angle associated with said parent interval, the left angle (φ left,d ) and the right angle (φ right,d ) and - Said left corner (φ left,d ) and the right angle (φ right,d ) selecting a context based on - extracting bits (b d ), subjecting the decoded bits (b d ) indicating which of the two half-intervals the coordinates of the point belong to, A method for decoding a point cloud from a bitstream of encoded point cloud data representing a physical object.
3. The first angle (φ 1,d ), and a second angle (φ 2,d ) and the third angle (φ 3,d ) and the weighted difference between the left angle (φ left,d ) and the fourth angle (φ 4,d ), and the fifth angle (φ 5,d ) and the sixth angle (φ 6,d ) and the weighted difference between the right angle (φ right,d ), The method according to claim 1 or 2.
4. the first angle, the second angle, the third angle, the fourth angle, the fifth angle, and the sixth angle are inherited angles associated with the parent interval; The method according to claim 3.
5. The first angle, the third angle, and the fourth angle are all related to the parent interval by the same inherited angle (φ 1,d-1 ), and the second, fourth and fifth angles are all equal to another inherited angle (φ middle,d-1 ) and the angle (φ middle,d ) to calculate The method according to claim 4.
6. The first angle, the third angle, and the fifth angle are all the same inheritance angle (φ 1,d-1 ), and the second angle, the fourth angle, and the sixth angle are all equal to the upper limit of the parent interval. top,d-1 ) and the angle (φ top,d ) or the angle (φ low,d ) to calculate The method according to claim 4.
7. The angle (φ top,d ) or the angle (φ low,d ) is the inherited angle (φ middle,d-1 ) The method according to claim 6.
8. To encode the first bit, the angle (φ) associated with the upper limit of the left half-interval is calculated by performing an arctangent function: top,d ) or the angle (φ low,d ) and use the angle (φ) associated with the upper limit of the left half-interval to encode the subsequent bits. top,d ) or the angle (φ low,d ) is the inherited angle (φ middle,d-1 ) The method according to claim 6.
9. The inherited angle (φ middle,d-1 ) are the two inherited angles (φ 1,d-1 ) and (φ top,d-1 ), The method of claim 7.
10. If the dimension of the parent interval is greater than or equal to a determined threshold, the inherited angle (φ middle,d-1 ) or, if the dimension of the parent interval is less than a determined threshold, calculate the two inherited angles (φ 1,d-1 ) and (φ top,d-1 ), The method of claim 7.
11. 1. An apparatus for encoding a point cloud into a bitstream of encoded point cloud data representing a physical object, the apparatus comprising: one or more processors configured to implement an orientation coding mode that provides a series of bits for encoding coordinates of points of the point cloud, the orientation coding mode comprising: - dividing the parent interval to which the coordinates of said point belong into a left half-interval and a right half-interval; - from at least one inherited angle associated with said parent interval, the left angle (φ left,d ) and the right angle (φ right,d ) and - Said left corner (φ left,d ) and the right angle (φ right,d ) selecting a context based on Based on the selected context, the bit (b d ) into the bitstream, d ) indicating which of the two half-intervals the coordinates of the point belong to, An apparatus for encoding a point cloud into a bitstream of encoded point cloud data representing a physical object.
12. 1. An apparatus for decoding a point cloud from a bitstream of encoded point cloud data representing a physical object, the apparatus comprising: one or more processors configured to implement an orientation coding mode that provides a series of bits for encoding coordinates of points of the point cloud, the orientation coding mode comprising: - dividing the parent interval to which the coordinates of said point belong into a left half-interval and a right half-interval; - from at least one inherited angle associated with said parent interval, the left angle (φ left,d ) and the right angle (φ right,d ) and - Said left corner (φ left,d ) and the right angle (φ right,d ) selecting a context based on - extracting bits (b d ) by context self-adaptive binary entropy decoding of the decoded bits (b d ) indicating which of the two half-intervals the coordinates of the point belong to, An apparatus for decoding a point cloud from a bitstream of encoded point cloud data representing a physical object.
13. 1. A computer program product, which when executed by one or more processors, causes the one or more processors to perform a method for encoding a point cloud into a bitstream of encoded point cloud data representing a physical object, the method including an orientation coding mode providing a sequence of bits for encoding coordinates of points of the point cloud, the method comprising: - dividing the parent interval to which the coordinates of said point belong into a left half-interval and a right half-interval; - from at least one inherited angle associated with said parent interval, the left angle (φ left,d ) and the right angle (φ right,d ) and - Said left corner (φ left,d ) and the right angle (φ right,d ) selecting a context based on Based on the selected context, the bit (b d ) into the bitstream, d ) indicating which of the two half-intervals the coordinates of the point belong to, Computer program.
14. A non-transitory storage medium carrying program code instructions for performing a method of encoding a point cloud into a bitstream of encoded point cloud data representing a physical object, the method including an orientation coding mode providing a sequence of bits for encoding coordinates of points of the point cloud, the method comprising: - dividing the parent interval to which the coordinates of said point belong into a left half-interval and a right half-interval; - from at least one inherited angle associated with said parent interval, the left angle (φ left,d ) and the right angle (φ right,d ) and - Said left corner (φ left,d ) and the right angle (φ right,d ) selecting a context based on Based on the selected context, the bit (b d ) into the bitstream, d ) indicating which of the two half-intervals the coordinates of the point belong to, Non-transitory storage media.
15. 1. A computer program, which when executed by one or more processors, causes the one or more processors to perform a method for decoding a point cloud from a bitstream of encoded point cloud data representing a physical object, the method including an orientation coding mode providing a sequence of bits for decoding coordinates of points of the point cloud, the method comprising: - dividing the parent interval to which the coordinates of said point belong into a left half-interval and a right half-interval; - from at least one inherited angle associated with said parent interval, the left angle (φ left,d ) and the right angle (φ right,d ) and - Said left corner (φ left,d ) and the right angle (φ right,d ) selecting a context based on - extracting bits (b d ), subjecting the decoded bits (b d ) indicating which of the two half-intervals the coordinates of the point belong to, Computer program.
16. A non-transitory storage medium carrying program code instructions for performing a method of decoding a point cloud from a bitstream of encoded point cloud data representing a physical object, the method including an orientation coding mode that provides a sequence of bits for decoding coordinates of points of the point cloud, the method comprising: - dividing the parent interval to which the coordinates of said point belong into a left half-interval and a right half-interval; - from at least one inherited angle associated with said parent interval, the left angle (φ left,d ) and the right angle (φ right,d ) and - Said left corner (φ left,d ) and the right angle (φ right,d ) selecting a context based on - extracting bits (b d ), subjecting the decoded bits (b d ) indicating which of the two half-intervals the coordinates of the point belong to, Non-transitory storage media.
Citation Information
Patent Citations
System and method for ordered representation and feature extraction for point clouds obtained by detection and ranging sensor
US20200302237A1
Method and apparatus for point cloud compression
US20200394822A1
Coding of laser angles for angular and azimuthal modes in geometry-based point cloud compression
US20210326734A1
