Method and apparatus for encoding / decoding point cloud geometry data sensed by at least one sensor - Patents.com

Abstract

A method and apparatus for encoding / decoding point cloud geometry data sensed by at least one sensor associated with a sensor index, the point cloud geometry data being represented by ordered coarse points occupying several discrete locations of a set of discrete locations in a two-dimensional space. The method includes: 1 ) and a first radius (r 1 ) associated with the first occupied rough point (P 1 ) at least one second occupied rough point (P 2 ) and at least one second radius (r 2 ), and at least one third occupied rough point (P 3 ) and at least one third radius (r 3 ) from the selected prediction radius (r pred ) and selecting (110) the selected prediction radius (r pred ) is the second radius (r 2 ) or the third radius (r 3 ) is equal to pred ) into a bitstream (120).

Description

[Technical field]

[0001] The present application relates generally to point cloud compression, and more particularly to a method and apparatus for encoding / decoding point cloud geometry data sensed by at least one sensor. [Background technology]

[0002] This section is intended to introduce the reader to various aspects of the art, which may be related to various aspects of at least one embodiment of the present application that are described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to better understand all aspects of the present application.

[0003] Point clouds have recently gained attention as a format to represent 3D data. It varies in its ability to represent all types of physical objects or scenarios. Point clouds can be used for various purposes, such as cultural heritage / architecture, to scan objects like sculptures or buildings in a 3D way in order to share the spatial configuration of the object without transmitting or accessing it. It is also a way to ensure the preservation of knowledge of an object in case it may be destroyed. For example, a temple destroyed in an earthquake. Usually, this type of point cloud is static, colored and huge.

[0004] Another use case is in topography and cartography, where a 3D representation is used to allow maps to be not limited to flat surfaces but to include relief. Google Maps is currently a good example of a 3D map, but it uses a mesh rather than a point cloud. However, a point cloud may be an appropriate data format for a 3D map, and usually this type of point cloud is static, colored, and large.

[0005] Virtual reality (VR), augmented reality (AR), and immersive worlds have been a hot topic lately and are seen by many as the future of 2D tablet video. The basic idea is to immerse the viewer in the surrounding environment, but standard TV only allows the viewer to see the virtual world in front of him / her. There are several levels of immersion depending on the viewer's freedom in the environment. Point clouds are a good format candidate to distribute VR / AR worlds.

[0006] The automotive industry, especially the foreseeable field of autonomous vehicles, is also an area where point clouds can be used extensively. Autonomous vehicles should be able to "sense" their environment in order to make better driving decisions based on the presence and nature of detected objects in their nearest vicinity, and on the road layout.

[0007] A point cloud is a collection of points located in three-dimensional (3D) space, optionally with additional values ​​attached to each point. These additional values ​​are typically called attributes. An attribute may be, for example, a three-component color, a material property (e.g., reflectance), and / or a two-component normal vector to a surface associated with the point.

[0008] A point cloud is therefore a combination of geometry data (point positions in 3D space, usually expressed as 3D Cartesian coordinates x, y, z) and attributes.

[0009] Point clouds can be sensed by various types of devices, such as an array of cameras, depth sensors, lasers (also called light detection and ranging, laser radar), radar, or can be computer-generated (e.g., for movie post-production). Depending on the use case, for cartography, a point cloud may contain thousands to billions of points. The original representation of a point cloud requires a very large number of bits per point, at least a dozen bits per Cartesian coordinate x, y, or z, and optionally provides more bits for one or more attributes, e.g., three times the 10 bits for color.

[0010] In many applications it is important to be able to distribute point clouds to end users or store them on a server while maintaining an acceptable (or preferably very good) quality of experience, consuming only a reasonable number of bitrates or storage space. Efficient compression of these point clouds is a key point for the practical realization of many immersive world distribution chains.

[0011] For distribution and visualization by end users, e.g., in AR / VR glasses or any other 3D-enabled device, the compression may be lossy (e.g., in the case of video compression), but there are also use cases that require lossless compression in order not to alter decisions obtained from subsequent analysis of the compressed and transmitted point cloud, such as medical applications or autonomous driving.

[0012] Until recently, the mass market has not solved the problem of point cloud compression (aka PCC), and there are no standardized point cloud decoders available. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11, also known as the Video Image Experts Group or MPEG, started a work project on point cloud compression. This resulted in two standards: MPEG-I Part 5 (ISO / IEC 23090-5) or Video-Based Point Cloud Compression (aka V-PCC) MPEG-I Part 9 (ISO / IEC 23090-9) or Geometry-Based Point Cloud Compression (aka G-PCC)

[0013] The V-PCC coding method compresses point clouds by multiple projections onto a 3D object to obtain 2D patches that are packaged into an image (or video in the case of processing dynamic point clouds). It compresses images and videos captured with existing image / video codecs, making the most of image and video solutions already in place. By its very nature, V-PCC is only efficient with dense and continuous point clouds, since image / video codecs cannot compress non-smooth patches, such as those obtained from the projection of sparse geometry data sensed by laser radar.

[0014] The G-PCC coding method has two schemes for compressing sparse sensed geometry data.

[0015] The first approach is based on an occupancy tree, which is any type of tree, locally octet, quad or binary, representing the point cloud geometry. Occupied nodes (i.e. nodes related to cubes / cuboids that contain at least one point of the point cloud) are split until a certain size is reached, and the 3D location of the points is provided by occupied leaf nodes, usually located at the center of these nodes. The occupancy information is carried by occupancy data (binary data, flags), which send signals to communicate the occupancy state of each of the node's subnodes. By using neighborhood-based prediction techniques, a high level of compression can be obtained for data occupied by dense point clouds. Sparse point clouds can also be solved by directly encoding the location of non-minimal sized points in the node, stopping the tree construction when only isolated points exist in the node. Such techniques are called direct coding mode (DCM).

[0016] The second approach is based on a prediction tree, where each node represents the 3D location of a point, and the parent / child relationships between nodes represent the spatial prediction from parent to child. This approach only solves sparse point clouds, and offers the advantages of lower latency and easier decoding than occupancy trees. However, it offers only a small improvement in compression performance over the first occupancy-based approach, and increases the coding complexity as the encoder needs to intensively search for the best predictor (from a long list of potential predictors) when building the prediction tree.

[0017] In these two schemes, attribute (decoding) encoding is done after geometry (decoding) encoding is completed, so effectively two encodings occur. Thus, joint geometry / attribute low latency is achieved by decomposing the 3D space into slices of sub-volumes that are coded independently, without prediction between the sub-volumes. Using many slices severely impacts compression performance.

[0018] The combination of requirements for encoder and decoder simplicity, low latency and compression performance is a problem that has yet to be satisfactorily solved by existing point cloud codecs.

[0019] An important use case is the transmission of sparse geometry data sensed by at least one sensor mounted on a moving vehicle. In this case, a simple and low-latency embedded encoder is usually required. The requirement for simplicity limits the processing power available to the point cloud encoder, since the encoder may be located in a computing unit that performs other processing in parallel (e.g., (semi-)automated driving). Also, low latency is required to check the local traffic in real time based on the collection of multiple vehicles and to make rapid enough decisions based on the traffic information to enable fast transmission from the car to the cloud. The use of 5G allows the transmission latency to be low enough, but the encoder itself should not introduce too much latency for encoding. And since the flow of data from millions of cars to the cloud is expected to be huge, compression performance is crucial.

[0020] Specific priors related to sparse geometry data sensed by spin laser radar have already been exploited in G-PCC, leading to very significant compression gains.

[0021] First, G-PCC uses the elevation angle (relative to the horizontal ground) sensed by a spinning laser radar head 10, as shown in Figures 1 and 2. The laser radar head 10 includes a collection of sensors 11 (e.g., laser devices), five sensors are shown here. The spinning laser radar head 10 can rotate around a vertical axis z to sense the geometry data of a physical object. The geometry data sensed by the laser radar is then expressed in spherical coordinates (r 3D , φ, θ) and r 3Dis the distance from point P to the center of the laser radar head, φ is the azimuth angle of the self-rotation of the laser radar head relative to the reference object, and θ is the elevation angle of the sensor k of the laser radar head 10 relative to the horizontal reference plane.

[0022] As shown in Fig. 3, a regular distribution along the azimuth angle is observed in the laser radar sensed data. This regularity is used in G-PCC to obtain a quasi-1D representation of the point cloud, and the noise is reduced to a radius of r 3D Only the angle φ and θ can take on only discrete values, and the angles φ and θ can take on only discrete values. i ∀i=0~I-1, where I is the number of azimuth angles for sensing a point, and θj∀i=0~J-1, where J is the number of sensors in the spin laser radar head 10. Essentially, as shown in Figure 3, G-PCC uses sparse geometry data sensed by the laser radar in a two-dimensional (discrete) angular coordinate space (φ, θ) and the radius value r of each point. 3D Represents.

[0023] In spherical coordinate space, the discrete property of angles is used to predict the position of the current point based on already coded points; such quasi-1D property is already exploited in G-PCC in both occupancy and prediction trees.

[0024] More precisely, occupancy trees make extensive use of DCM and use a context self-adaptive entropy encoder to entropy code the direct positions of points in nodes, as well as the local transformation from point positions to angular coordinates (φ, θ), and the discrete angular coordinates (φ i , θ j ) to get the context. The quasi-1D nature of this coordinate space (r 2D , φ i , θ j ), the prediction tree uses the angle coordinate (r 2D , φ, θ) and directly encode the first version of the point's location in r 2D is the projection radius on the horizontal xy plane. Next, the spherical coordinates (r2D , φ, θ) to 3D Cartesian coordinates (x, y, z) and encode the xyz residuals to address errors in the coordinate transformation, approximations in the elevation and azimuth angles, and potential noise.

[0025] G-PCC certainly uses an angular prior to better compress the sparse geometry data sensed by the spinning laser radar, but does not adapt the coding structure to the sensing order. In essence, the occupancy tree needs to be coded to its final depth before outputting the points. This occupancy data is coded according to a so-called breadth priority: first the occupancy data of the root node is coded to indicate which subnodes it is occupied, then the occupancy data of each occupied subnode is coded to indicate which grandchild nodes it is occupied on, and so on, until the leaf nodes are determined and the corresponding points are provided / output to the application or to the attribute coding scheme(s). For prediction trees, the encoder is free to choose the order of the points in the tree, but to obtain good compression performance and optimize the prediction accuracy, G-PCC recommends coding one tree per sensor. This mainly has the same disadvantages as using one coding slice for each sensor, i.e. non-optimal compression performance, since no prediction between sensors is possible and low encoder latency is not provided. To make matters worse, each sensor should have one encoding process, which is unrealistic since the number of core encoding units needs to be equal to the number of sensors.

[0026] In other words, in the framework of a spinning sensor head for sensing sparse geometric data of a point cloud, the prior art does not solve the problem of combining simplicity of encoding and decoding, low latency, and compression performance.

[0027] Also, using spinning sensor heads to sense the sparse geometric data of a point cloud has some drawbacks, and other types of sensor heads can be used.

[0028] The mechanical parts that generate the spin of the spin sensor head are prone to breakage and are expensive. Also, due to its structure, the viewing angle is necessarily 2π. This does not allow sensing of certain areas of interest at high frequencies, for example, sensing the front of the vehicle may be more interesting than sensing the rear. In fact, in most cases, when the sensor is mounted on a vehicle, most of the 2π viewing angle is blocked by the vehicle itself, and the blocked viewing angle does not need to be sensed.

[0029] Recently, new sensors have emerged that allow more flexibility in selecting the area to be sensed. In most recent designs, the sensor can be moved more freely electronically (to avoid fragile mechanical parts) to obtain various sensing paths in the 3D scene, as shown in Figure 5. In Figure 5, a group of four sensors is shown. Although their relative sensing directions (i.e., azimuth and elevation angles) are fixed relative to each other, they generally sense the scene along a programmable sensing path depicted by a dashed line in the two-dimensional angular coordinate (φ, θ) space. Then, points of the point cloud can be sensed regularly along the sensing path. As shown in Figure 6, some sensor heads can also adjust their sensing frequency by increasing their sensing frequency when a region of interest R is detected. Such a region of interest R can be associated, for example, with a close-range object, a moving object, or any object (pedestrian, other vehicle, etc.) that has been previously segmented in the previous frame or dynamically segmented in the sensing period. FIG. 7 shows a schematic of another example of a sensing path (a typical zigzag sensing path) used by a sensor head including two sensors that can increase their sensing frequency if an area of ​​interest (gray shaded points and gray hash points) is detected. Using a zigzag sensing path can be advantageously used to sense a finite (azimuth) angular sector of a 3D scene. Since the sensor can be mounted on a vehicle, the viewport of interest is necessarily limited by the presence of the vehicle itself, which occludes the scene, unless the sensor is located on top of the vehicle. Therefore, a sensor with a limited sensing angular sector is highly visible and easier to integrate into the vehicle.

[0030] As shown in Figure 8, a sensor head including a single sensor may be used to sense multiple positions (two vertical positions in Figure 8), such as using reflection off a mirror that oscillates with rotation (here vertical rotation). In this case, no group of sensors is used, and sensing using groups of multiple sensors is simulated using a single sensor at different angular positions (i.e., having different elevation angles in Figure 8) along a sensing path (here a zigzag sensing path).

[0031] For simplicity, in the following description and claims, a "sensor head" can refer to a group of physical sensors or a set of sensing elevation indexes that simulate a group of sensors, and one skilled in the art will appreciate that a "sensor" can also refer to a sensor at each sensing elevation index location.

[0032] Combining the requirements of encoder and decoder simplicity, low latency, and compression performance for point clouds sensed by any type of sensor is a problem that has not yet been satisfactorily solved by existing point cloud codecs.

[0033] It is with the above in mind that at least one embodiment of the present application is designed. Summary of the Invention [Problem to be solved by the invention]

[0034] The following section presents a simplified summary of at least one embodiment in order to provide a basic understanding of some aspects of the present application. This summary is not a detailed overview of the embodiment. It is not intended to identify key or critical elements of the embodiment. The following summary presents only some aspects of at least one embodiment in a simplified form as a prelude to the more detailed description provided elsewhere in the document. [Means for solving the problem]

[0035] According to a first aspect of the present application, there is provided a method for encoding point cloud geometry data sensed by at least one sensor associated with a sensor index into a bitstream, the point cloud geometry data being represented by ordered coarse points occupying several discrete locations of a set of discrete locations in a two-dimensional space, each occupied coarse point being positioned in the two-dimensional space by a sensor index associated with a sensor sensing a point cloud point associated with the occupied coarse point and a sample index associated with a sensing time at which the point cloud point was sensed, and each occupied coarse point being associated with a radius based on a distance from the point cloud point to a reference object. The method includes the steps of: for a first occupied coarse point having a first sensor index and associated with a first radius, selecting a selected prediction radius from at least one second radius associated with at least one second occupied coarse point having a second sensor index different from the first sensor index and at least one third radius associated with at least one third occupied coarse point having a sensor index equal to the first sensor index; encoding data into the bitstream indicating whether the selected prediction radius is equal to the second radius or the third radius; and predictively encoding a residual radius between the first radius and the selected prediction radius into the bitstream.

[0036] According to a second aspect of the present application, there is provided a method for decoding point cloud geometry data sensed by at least one sensor associated with a sensor index from a bitstream, the point cloud geometry data being represented by ordered coarse points occupying several discrete positions of a set of discrete positions in a two-dimensional space, each occupied coarse point being positioned in the two-dimensional space by a sensor index associated with a sensor sensing a point cloud point associated with the occupied coarse point and a sample index associated with a sensing time at which the point cloud point was sensed, and each occupied coarse point being associated with a radius based on a distance from the point cloud point to a reference object. The method includes the steps of: for a first occupied coarse point having a first sensor index, decoding from the bit stream data indicating whether a selected prediction radius is equal to a second radius or a third radius, where the selected prediction radius is selected from at least one second radius associated with at least one second occupied coarse point having a second sensor index different from the first sensor index and at least one third radius associated with at least one third occupied coarse point having a sensor index equal to the first sensor index; decoding from the bit stream for the first occupied coarse point a residual radius; and obtaining a radius associated with a point of the point cloud to be used for the first occupied coarse point based on the residual radius and the selected prediction radius obtained from the data.

[0037] In one exemplary embodiment, the data includes binary data indicating whether the selected prediction radius is the second radius or the third radius.

[0038] In one exemplary embodiment, the at least one second radius forms a first radius list and the at least one third radius forms a second radius list, and the binary data indicates whether the selected prediction radius belongs to the first radius list or the second radius list.

[0039] In one exemplary embodiment, the data further includes a predictor index, the predictor index indicating which radius in the first radius list or the second radius list the selected prediction radius is equal to.

[0040] In one exemplary embodiment, the selected prediction radius is selected from a list of a single second radius and at least one third radius, and the data includes a predictor index, and if the predictor index is not equal to a predetermined index value, the predictor index indicates which radius in the list the selected prediction radius is equal to, and otherwise the data further includes binary data indicating whether the selected prediction radius is equal to the second radius or the third radius indicated by the predetermined index value in the radius list.

[0041] In one exemplary embodiment, the at least one second radius and the at least one third radius form a single radius list, and the data includes a predictor index indicating which radius in the radius list the selected prediction radius is equal to.

[0042] In one exemplary embodiment, the radii in the single radius list are statistically sorted from the most selected one to the least selected one.

[0043] According to a third aspect of the present application, there is provided an encoded point cloud data bitstream representing point cloud geometry data sensed by at least one sensor associated with a sensor index, the point cloud geometry data being represented by ordered coarse points occupying several discrete positions of a set of discrete positions in a two-dimensional space, each occupied coarse point being positioned in the two-dimensional space by a sensor index associated with a sensor sensing a point cloud point associated with the occupied coarse point and a sample index associated with a sensing time at which the point cloud point was sensed, and each occupied coarse point being associated with a radius based on a distance from the point cloud point to a reference object. The bitstream includes encoded point cloud data representing a residual radius between a radius associated with a first occupied coarse point and a selected predicted radius, and data indicating whether the selected predicted radius is equal to a second radius or a third radius, the first occupied coarse point having a first sensor index in two-dimensional space, the second radius being associated with a second occupied coarse point having a second sensor index in the two-dimensional space different from the first sensor index, and the radius being associated with at least one third occupied coarse point having a sensor index in the two-dimensional space equal to the first sensor index.

[0044] According to a fourth aspect of the present application, there is provided an apparatus for encoding point cloud geometry data sensed by at least one sensor associated with a sensor index into a bitstream, the point cloud geometry data being represented by ordered coarse points occupying several discrete positions of a set of discrete positions in a two-dimensional space, the apparatus comprising one or more processors configured to perform the method of the first aspect of the present application.

[0045] According to a fifth aspect of the present application, there is provided an apparatus for decoding point cloud geometry data sensed by at least one sensor associated with a sensor index from a bitstream, the point cloud geometry data being represented by ordered coarse points occupying several discrete positions of a set of discrete positions in a two-dimensional space, the apparatus comprising one or more processors configured to perform the method of the second aspect of the present application.

[0046] According to a sixth aspect of the present application, there is provided a computer program product comprising instructions, which when executed by one or more processors, cause the one or more processors to perform a method of the first aspect of the present application.

[0047] According to a seventh aspect of the present application, there is provided a non-transitory storage medium carrying program code instructions for performing the method of the first aspect of the present application.

[0048] According to an eighth aspect of the present application there is provided a computer program product comprising instructions, which when executed by one or more processors cause the one or more processors to perform the method of the second aspect of the present application.

[0049] According to a ninth aspect of the present application, there is provided a non-transitory storage medium carrying program code instructions for performing the method of the second aspect of the present application.

[0050] At least one specific feature of the embodiment and at least one other object, advantage, feature and application of the embodiment will become apparent from the following description of the embodiment in combination with the drawings. [Brief description of the drawings]

[0051] Reference is now made, by way of example, to the drawings, which show exemplary embodiments of the present application. [Figure 1] FIG. 1 shows a schematic side view of a sensor head according to the prior art and some of its parameters. [Diagram 2]FIG. 1 shows a schematic plan view of a sensor head according to the prior art and some of its parameters; [Diagram 3] 1 is a schematic diagram showing a regular distribution of data sensed by a spin sensor head according to the prior art; [Figure 4] FIG. 1 is a schematic diagram illustrating a representation of points of a point cloud in 3D space according to the prior art; [Diagram 5] FIG. 1 shows a schematic diagram of an example of a sensor head capable of sensing a real scene along a programmable sensing path according to the prior art; [Figure 6] FIG. 1 is a schematic diagram illustrating an example of a sensor head capable of sensing a real scene along a programmable sensing path based on different sensing frequencies according to the prior art; [Figure 7] FIG. 1 is a schematic diagram illustrating an example of a sensor head capable of sensing a real scene along a programmable zigzag sensing path based on different sensing frequencies according to the prior art; [Figure 8] FIG. 13 is a schematic diagram illustrating a single sensor head capable of sensing a real scene along a programmable zigzag sensing path based on different sensing frequencies. [Figure 9] FIG. 2 is a schematic diagram illustrating an example of ordered coarse points of a coarse representation in accordance with at least one embodiment. [Figure 10] FIG. 2 is a schematic diagram illustrating an example of a coarse point sequence according to an illustrative embodiment. [Figure 11] FIG. 1 shows a schematic representation of ordered coarse points in a two-dimensional coordinate (s, λ) space. [Figure 12] FIG. 2 is a schematic diagram illustrating ordered coarse points of a coarse representation in accordance with at least one embodiment; [Figure 13] FIG. 2 is a schematic diagram illustrating a sensor sensing a point on a road that resembles a horizontal plane. [Figure 14] FIG. 2 is a schematic diagram illustrating a sensor sensing a point on a road having an object. [Figure 15]FIG. 2 is a schematic diagram of the prediction radius when a radial jump from an old object to a new object occurs according to the prior art; [Figure 16] FIG. 2 is a schematic diagram showing the predicted radius from another sensor when a radial jump occurs from an old object to a new object according to the prior art; [Figure 17] FIG. 1 is a schematic block diagram illustrating steps of a method 100 for encoding point cloud geometry data into a bitstream of encoded point cloud data according to at least one embodiment. [Figure 18] FIG. 2 is a schematic block diagram illustrating steps of a method 200 for decoding point cloud geometry data from a bitstream of encoded point cloud data in accordance with at least one embodiment. [Figure 19] FIG. 2 is a schematic diagram illustrating examples of rough points belonging to a neighborhood area of ​​a first occupied rough point according to an exemplary embodiment. [Figure 20] 2A-2C are schematic diagrams illustrating examples of rough points belonging to a neighborhood of a first occupied rough point according to an exemplary embodiment of the methods 100 and 200. [Figure 21] 2A-2C are schematic diagrams illustrating examples of rough points belonging to a neighborhood of a first occupied rough point according to an exemplary embodiment of the methods 100 and 200. [Figure 22] 2A-2C are schematic diagrams illustrating examples of rough points belonging to a neighborhood of a first occupied rough point according to an exemplary embodiment of the methods 100 and 200. [Diagram 23] 2A-2C are schematic diagrams illustrating examples of rough points belonging to a neighborhood of a first occupied rough point according to an exemplary embodiment of the methods 100 and 200. [Figure 24] 2A-2C are schematic diagrams illustrating examples of rough points belonging to a neighborhood of a first occupied rough point according to an exemplary embodiment of the methods 100 and 200. [Diagram 25] 18 is a block diagram showing the steps of a variation of the method 100 of FIG. 17. [Figure 26] 20 is a block diagram showing steps of a variation of the method 200 of FIG. 18. [Figure 27]FIG. 2 shows a schematic diagram of an example of coarse points belonging to the neighborhood of a first occupied coarse point of the variants of the methods 100 and 200. [Figure 28] FIG. 2 illustrates a schematic diagram of predictor index encoding / decoding in accordance with at least one exemplary embodiment. [Figure 29] FIG. 1 is a block diagram illustrating an example of a system for implementing aspects and example embodiments.

[0052] Similar reference numbers may be used in different drawings to represent similar components. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0053] At least one example of the embodiment will be described more fully below with reference to the drawings, in which at least one of the embodiments is illustrated. However, the embodiment may be implemented in many alternative forms and should not be construed as being limited to the examples set forth herein. It should therefore be understood that the embodiment is not intended to be limited to the particular form disclosed. Rather, the present application is intended to cover all modifications, equivalents and alternatives falling within the spirit and scope of the present application.

[0054] On the one hand, it generally relates to point cloud encoding and decoding, on the other hand, it generally relates to the transmission of generated or encoded bitstreams, and on the other hand, it generally relates to the reception / access of decoded bitstreams.

[0055] Additionally, the aspects are not limited to MPEG standards such as MPEG-I Part 5 or Part 9 relating to point cloud compression, etc., but may be applied, for example, to other standards and recommendations, whether existing or developed in the future, and extensions of such standards and recommendations, including MPEG-I Part 5 and Part 9. Unless otherwise indicated or technically excluded, the aspects described in this application may be used alone or in combination.

[0056] The present invention relates to encoding / decoding point cloud geometry data represented by ordered coarse points of a coarse representation occupying several discrete locations of a set of discrete locations in a two-dimensional space.

[0057] For example, in the working group ISO / IEC JTC1 / SC29 / WG7 related to MPEG 3D graphics coding, a new codec called L3C2 (Low Delay Low Complexity Codec) is considered to improve the coding efficiency of laser radar sensed point clouds with respect to the G-PCC codec. The codec L3C2 provides an example of a two-dimensional representation (i.e. a coarse representation) of the points of the point cloud. A description of this code is given in the Working Group's output document N 00167, ISO / IEC JTC1 / SC 29 / WG7, MPEG 3D graphics coding, "Technologies under Consideration in G-PCC", dated 31 August 2021.

[0058] Basically, each sensing point P n For sensing point P n The 3D Cartesian coordinates (x n ,y n ,z n ) to obtain the sensing point P n The sensor index λ associated with the sensor n and the azimuth angle φ, which represents the sensing angle of this sensor. n Then, obtain the azimuth angle φ n and the sensor index λ n Sort the points of the point cloud based on, for example, lexicographical order based on azimuth angle first and then based on sensor index. Then, for point P n The order index o(P n ) is obtained using the following formula: o(P n )=φ n * K+λ n where K is the number of sensors.

[0059] FIG. 9 is a schematic diagram of ordered coarse points in a coarse representation. Five points of the point cloud have already been sensed. Each of these five points is coarsely represented by a coarse point (black point) in the coarse representation: two coarse points P n and P n+1 is the angle φ at time t1 c (φ i ′ s The three coarse points are aligned at an angle φ c +Δφ represent three points of the sensed point cloud. The coarse points that represent points of the sensed point cloud are occupied points, and the coarse points that do not represent points of the sensed point cloud are unoccupied points. Since the points of the point cloud are represented by occupied coarse points in the coarse representation, the order indices associated with the points of the point cloud are also order indices associated with occupied coarse points.

[0060] It is possible to define a coarse representation of point cloud geometry data in the two-dimensional coordinate (φ,λ) space.

[0061] Coarse representations can also be defined for any type of sensor head, including rotating (spinning) or non-rotating sensor heads. The definition is based on a sensing path defined by sensor characteristics in a two-dimensional angular coordinate (φ,θ) space, which includes an azimuth angle coordinate φ and an elevation angle coordinate θ, where the azimuth angle represents the sensing angle of the sensor relative to a reference object, and the elevation angle coordinate θ represents the elevation angle of the sensor relative to a horizontal reference plane. The sensing path is used to sense the points of the point cloud based on ordered coarse points that represent potential locations of the sensed points of the point cloud. Each coarse point is defined based on one sample index s associated with a sensing time along the sensing path and one sensor index λ associated with a sensor.

[0062] In Fig. 10, we use a sensor head that includes two sensors. The sensing paths along which the two sensors are located are represented by dashed lines. For each sample index s (each sensing time), we define two coarse points. The coarse points associated with the first sensor are represented by black shaded points in Fig. 10, and the coarse points associated with the second sensor are represented by black hashed points. Each of the two coarse points belongs to a sensor sensing path (dashed line) defined by sensing path SP. Fig. 11 shows a schematic representation of ordered coarse points in a two-dimensional coordinate (s, λ) space. The arrows in Fig. 10 and Fig. 11 indicate the links between two consecutive ordered coarse points.

[0063] Based on the ordering of the coarse points in the ordered coarse points, we associate an order index o(P) with each coarse point. o(P)=λ+s * K where K is the number of sensors in the sensor set or the number of different positions of a single sensor for the same sample index, and λ is the sensor index of the sensor that sensed point P in the point cloud at sensing time s.

[0064] Figure 12 shows the ordered coarse points of the coarse representation, showing five occupied coarse points (black circles). Two coarse points P n and P n+1 is occupied by two points of the point cloud sensed at sensing time t1 (corresponding to sample index s1), and the three coarse points are occupied by three points of the point cloud sensed at sensing time t2 (corresponding to sample index s2).

[0065] We can then define a coarse representation of the point cloud geometry data in the 2D coordinate (s,λ) space.

[0066] Given the order index o(P1) of the first coarse point occupied by the first sensing point in the point cloud and the order difference Δo, we can recursively reconstruct the order index o(P) of any occupied coarse point occupied by the sensing point P in the point cloud. o(P)=o(P -1 )+Δo

[0067] Encoding / decoding a point cloud geometry involves encoding / decoding the radius of each point in the point cloud. As shown in Figure 4, the radius associated with a point in the point cloud is the radius r 3D The radius r is equal to the projection of 2D In the following, radius refers to the projection of the 3D radius associated with a point of the point cloud.

[0068] In the following, the present invention will be described by considering a coarse representation defined in a two-dimensional coordinate (s, λ) space, but similarly a coarse representation defined in a two-dimensional coordinate (φ, λ) space may be described, where a spin sensor head such as a laser radar head provides a particular coarse representation defined in a two-dimensional coordinate (s, λ) space, and at each sensing time, the sensor of the sensor head detects an object and the sensing point corresponds to the occupied coarse point being represented.

[0069] As described above, the point cloud geometry data is represented by ordered coarse points occupying some discrete positions of a set of discrete positions in a two-dimensional coordinate (s, λ) space. Each occupied coarse point is then located in the two-dimensional coordinate (s, λ) space by a sensor index associated with a sensor sensing a point of the point cloud associated with the occupied coarse point, and a sample index associated with a sensing time at which the point of the point cloud was sensed. In the method discussed below, each occupied coarse point of the point cloud is identified as a first occupied coarse point P1.

[0070] The radius associated with a point of the point cloud is also associated with an occupied coarse point that represents said point cloud point in the coarse representation.

[0071] A radius r1 is associated to a point P of the point cloud represented in the coarse representation by a first occupied coarse point P1. The first occupied coarse point P1 has a sample index s1 and a sensor index λ1 in two-dimensional coordinates (s, λ). Usually, instead of encoding the radius r1 directly, we encode the residual radius r1 after obtaining res Encode the following. r res =r1-r pred where r pred is the prediction radius.

[0072] The coding performance of radius r1 is determined by the prediction radius r, which is determined to limit the dynamics range of the residual radius. pred A smaller dynamics of the residual radius usually requires fewer bits to be coded into the bitstream.

[0073] Typically, the prediction radius r pred is selected from several previously encoded radii associated with points of the point cloud sensed by the same sensor associated with a sensor index equal to the first sensor index λ1. A small change in radius r1 between two consecutive sensing times (e.g., between sample index s1 and sample index s1-1) provides good coding efficiency.

[0074] This is especially true when the spin sensor senses the road (close to a horizontal plane) as shown in FIG. 13. In this case, the prediction radius r pred is the radius r previously encoded for the same sensor λ1 pred,1 is likely to be equal to

[0075] Of course, some objects may be placed on the road, as shown in Fig. 14. In this case, the prediction radius r pred is the previously encoded radius r pred,1 Instead, we use other previously encoded radii r pred、j It is possible that.

[0076] Therefore, the prediction radius r pred is usually chosen from a list of radii, corresponding to previously encoded / decoded radii associated with points of the point cloud sensed by the same sensor λ1. When sensing a new object for the first time, such a list of radii will be chosen to find a good predicted radius r pred cannot be provided, i.e. none of the previously encoded / decoded radii associated with the new object's points belong to the radius list.

[0077] Such a situation occurs when a point P is detected by a new object O sensed by a sensor λ1, as shown in FIG. new Since it is the first point of new A point P belonging to another object O old The predicted point P pred Therefore, the prediction radius r pred is not the first radius r1 associated with point P, but the residual radius r res The dynamics of is large. Coding such residual radii is expensive in terms of bit rate.

[0078] One of the problems to be solved is to better handle the transition from old to new objects, which leads to a radius jump (for the same sensor λ1) and a bad radius prediction. The prediction is improved to reduce the residual radius r res This should result in better compression performance since it can reduce the dynamics of the signal.

[0079] In brief, the present invention provides the following solution to solve this problem: A predicted radius r associated with a point P of the point cloud encoded and represented by a first occupied coarse point P1 having a first sensor index λ1 and a first sample index in a two-dimensional coordinate (s, λ) space is calculated from at least one second radius associated with at least one point sensed by a sensor associated with a sensor index different from the first sensor index λ1 and a sample index less than or equal to the first sample index s1. pred At least one second point of the point cloud is represented in the coarse representation by at least one second occupied coarse point.

[0080] A predicted radius of a sensed point belonging to the new object is selected from radii associated with points sensed by a sensor associated with a sensor index different from the first sensor index, providing a better prediction compared to a prediction obtained from a radius associated with a point sensed by a sensor associated with a sensor index equal to the first sensor index.

[0081] For example, in Figure 16, the predicted point P pred , where the prediction points are sensed by sensors associated with a sensor index different from the first sensor index λ1. pred The residual radius between the radii associated with points P and P shown in Figure 15 is pred has lower dynamics compared to the residual radius between the radii associated with

[0082] Thus, the coding of the residual radius is improved compared to the coding of the residual radius obtained from a predicted radius corresponding to a previously coded / decoded radius associated with a point sensed by the same sensor.

[0083] The prediction radius is associated with points sensed by sensors having sensor indices different from the first sensor index λ1, and these points are also associated with sample indices less than or equal to the first sample index s1, i.e. they belong to the points in the causal neighborhood of point P.

[0084] This allows both the encoder and the decoder to obtain the same prediction radius.

[0085] FIG. 17 is a block diagram illustrating steps of a method 100 for encoding point cloud geometry data into a bitstream of encoded point cloud data in accordance with at least one embodiment.

[0086] A first occupied coarse point P1 is considered in the two-dimensional coordinate (s, λ) space. In the coarse representation, the first occupied coarse point P1 represents a sensing point of the point cloud. In the two-dimensional coordinate (s, λ) space, the first occupied coarse point P1 has a first sample index s1 and a first sensor index λ1, and is associated with a first radius r1.

[0087] In step 110, a selected predicted radius r2 is determined for the first occupied rough point P1 from at least one second radius r2 associated with at least one second occupied rough point P2. pred Each second occupied coarse point P2 has a second sensor index λ2 different from the first sensor index λ1 and a second sample index s2 less than or equal to the first sample index s1.

[0088] Fig. 19 shows a schematic example of coarse points belonging to the causal neighborhood of a first occupied coarse point P1. The unoccupied coarse points are the white points defined by the breaks, while the grey points indicate the occupied coarse points.

[0089] The grey shaded area represents a set of second coarse points with a second sensor index λ2 different from the first sensor index λ1 and a second sample index s2 less than or equal to the first sample index s1. In the coarse representation, the arrows between the coarse points indicate the order of the coarse points. Some second coarse points are occupied (grey points) and others are unoccupied (white points). All grey shaded areas form a causal neighborhood area around the first occupied coarse point, i.e. the occupied second coarse points belonging to this causal neighborhood area can be obtained and / or decoded by the encoding and / or decoding method before processing the first occupied coarse point P1. Note that the coarse point with a sensor index equal to the first sensor index λ1 is not a second coarse point and does not belong to the grey shaded area.

[0090] In step 120, data I pred is encoded into bitstream B. Data I pred is the selected prediction radius r pred Shows.

[0091] In step 130, the first radius r1 and the selected predicted radius r pred By calculating the difference between res get. r res =r1-r pred residual radius r res Encode into bitstream B.

[0092] In one example embodiment of step 110, a selected predicted radius r2 is selected from the radii r2 associated with the plurality of second occupied rough points P2. pred If you select , the selected prediction radius r pred is the residual radius r res corresponds to the radius that minimizes a cost function that represents the trade-off between the bit rate (used to encode

[0093] In one embodiment of step 130, the signal is transmitted to obtain a residual radius rres is equal to 0 or not, and then the residual radius r res It signals binary data indicating the plus or minus sign of , and the residual radius r res Encode and use the expGolomb encoder to get the remainder |r res |-1 can be encoded.

[0094] In one variant, the remainder |Q(r res )|-1 can be encoded by the expGolomb encoder. res ) is the quantized residual radius.

[0095] Without limiting the scope of the present invention, the residual radius r res Any other encoding of .times. ...

[0096] FIG. 18 is a block diagram illustrating steps of a method 200 for decoding point cloud geometry data from a bitstream of encoded point cloud data in accordance with at least one example embodiment.

[0097] The decoding method 200 of FIG. 18 corresponds to the encoding method 100 of FIG.

[0098] Consider a first occupied coarse point P1 in the two-dimensional coordinate (s, λ) space, where the first occupied coarse point P1 has a first sample index s1 and a first sensor index λ1.

[0099] In step 210, data I is extracted from bit stream B. pred Decrypt the data I pred is a predicted radius r2 selected from at least one second radius r2 associated with at least one second occupied rough point P2. pred wherein the at least one second occupied coarse point P2 has a second sensor index λ2 different from the first sensor index λ1 and a second sample index s2 lower than the first sample index s1.

[0100] In step 220, a residual radius r res Decrypt the

[0101] In step 230, the residual radius r res and Data I pred The selected prediction radius r obtained from pred Based on this, we obtain the (decoded) radius r1 associated with the point of the point cloud represented by the first coarse point P1. r1=r res +r pred

[0102] In one embodiment of step 230, the signal is transmitted by decoding to obtain a residual radius r res is equal to 0 or not, and the signal is sent by decoding to obtain the residual radius r res We can signal binary data indicating the plus or minus sign of , and use the expGolomb decoder to extract the remainder |r res Decode |-1 and use the residual radius r res can be decoded.

[0103] In one variant, the remainder ||Q(r res )|-1 can be decoded using an expGolomb decoder, and the residual radius r res is given by: r1=IQ(Q(r res )+r pred Here, IQ(Q(r res )) is the dequantized residual radius.

[0104] Unless the scope of the present invention is limited, the residual radius r res Any other decoding scheme may be used.

[0105] In one exemplary embodiment of the methods 100 and 200 shown in FIG. predmay be equal to a second radius r2 associated with the second occupied rough point P2, e.g., may be equal to a first distance D1 between the second sensor index λ2 and the first sensor index λ1 of the second occupied rough point P2, the distance being λ 2;below and λ 2;above In the exemplary embodiment of FIG. 2;below is equal to 2, and the boundary λ 2;above is equal to 1.

[0106] The definition on the first distance D1 limits the causal neighborhood region of the first occupied rough point P1, so the computational resources for selecting the prediction radius are limited because it ensures a certain correlation between the radii r1 and r2, which tends to disappear when the sensor index difference |λ2-λ1| is large.

[0107] According to this exemplary embodiment of the methods 100 and 200, the second occupied coarse point P2 may be an occupied coarse point having a second sensor index λ2 different from the first sensor index λ1 and a second sample index s2 less than or equal to the first sample index s1. The second occupied coarse point P2 may be an occupied coarse point having a first distance between the second sensor index λ2 and the first sensor index λ1 of the second occupied coarse point P2 less than or equal to λ1. 2;below and λ 2;above These second occupied rough points belong to the grey shaded area in FIG.

[0108] In one exemplary embodiment of the methods 100 and 200 shown in FIG. pred may be equal to a second radius r2 associated with a second occupied coarse point P2, e.g., equal to a second distance D2 between a second sample index s2 of said second occupied coarse point P2 and the first sample index s1 defined by the boundary W. In Fig. 21, such a second occupied coarse point P2 belongs to the grey shaded area and the boundary W is equal to 4.

[0109] The definition of the second distance D2 limits the causal neighborhood area of ​​the first coarse point P1, so that the computational resources for selecting the prediction radius are limited, since it ensures a certain relevance between the radii r1 and r2, which tends to become unrelevant when the sensor index difference s1-s2 is large, since the points P1 and P2 tend to belong to different objects or to different parts of the same object.

[0110] According to this exemplary embodiment of the methods 100 and 200, a second occupied coarse point P2 may be an occupied coarse point having a second sensor index λ2 different from the first sensor index λ1 and a second sample index s2 less than or equal to the first sample index s1. The second occupied coarse point P2 satisfies a condition that a second distance D2 between the second sample index s2 and the first sample index s1 of the second occupied coarse point P2 is defined by a boundary W.

[0111] In one variation shown in FIG. 22, the selected prediction radius r pred may be equal to the second radius r2 associated with the second occupied rough point P2, e.g., the second distance D2 is smallest.

[0112] This variant maximizes the chance of obtaining a good predicted radius, so that the second occupied coarse point P2 considered for use in selecting the predicted radius is the closest occupied coarse point to the first coarse point P1, and therefore the first occupied coarse point P1 and said second occupied coarse point P2 are likely to be associated with points of the same sensing object.

[0113] By this transformation, the second occupied rough point P2 is an occupied rough point having a second sensor index λ2 different from the first sensor index λ1 and a second sample index s2 equal to or smaller than the first sample index s1. The second occupied rough point P2 also satisfies the condition that the second distance D2 is the smallest. In FIG. 22, these second occupied rough points belong to the gray shaded area, and the predicted radius r is calculated by considering the two second occupied rough points P2. pred Select .

[0114] In one exemplary embodiment of the methods 100 and 200 shown in FIG. 23, at least two second occupied rough points P 2,i At least two second radii r associated with 2,i Based on the selected prediction radius r pred can be selected, and at least two second occupied rough points P 2,i The second sample index s2 of the selected prediction radius r pred is the second occupied rough point P 2,i The second radius r associated with 2,i and the second occupied rough point P 2,i is the closest sensor index λ to the first sensor index λ 2,i (Point P in Fig. 23 2,i ), or in one variant, the closest sensor index λ1 (point P in FIG. 23) that is equal to or less than the first sensor index λ1. 2,i-1 ).

[0115] According to this exemplary embodiment of the methods 100 and 200, the second occupied coarse point P2 may be an occupied coarse point having a second sensor index λ2 different from the first sensor index λ1 and a second sample index s2 less than or equal to the first sample index s1. The second occupied coarse point P2 may be an occupied coarse point having a sensor index λ2 closest to the first sensor index λ1. 2,ior in one variant, the closest sensor index less than or equal to the first sensor index λ1.

[0116] This exemplary embodiment and its variants maximize the chance of obtaining a good predicted radius, so that the second occupied coarse point P2 considered to be used for selecting the predicted radius is the closest occupied coarse point to the first coarse point P1 in the two-dimensional coordinate (s, λ) space. Therefore, the first occupied coarse point P1 and the second occupied coarse point P2 are likely to be associated with points of the same sensing object.

[0117] In one variant, the selected prediction radius r pred is the at least two second radii r 2,i may be equal to the average of

[0118] In one exemplary embodiment of the methods 100 and 200 shown in FIG. 24, the selected prediction radius r pred may be selected from at least one second radius r2 associated with the at least one second occupied coarse point P2 and at least one third radius r3 associated with the at least one third occupied coarse point P3, the at least one third occupied coarse point P3 having a sensor index λ3 equal to the first sensor index λ1 and a sample index s3 lower than the first sample index s1.

[0119] This exemplary embodiment is advantageous because it provides a good prediction radius when a new sensing point belongs to a new object or when a new sensing point belongs to a previously sensed object.

[0120] In one exemplary embodiment of the methods 100 and 200, the method further comprises: i ) and a first correction value C1 based on the first sample index s1, and at least one second sample index s2 (or s2, i) having at least one second occupied rough point P2 (or P 2,i ) associated with at least one second radius r2 (or r 2,i ) the selected prediction radius r pred The method may further include the step of modifying

[0121] This example embodiment is advantageous because it corrects for sample index differences that can occur when the second sample index of the second occupied coarse point is strictly lower than the first sample index. r pred,corr =r pred +C1

[0122] In one exemplary embodiment of the methods 100 and 200, the correction value C1 is calculated by multiplying the two occupied rough points P O1 and P O2 and the two occupied rough points P O1 and P O2 are different sample indices s O1 ands O2 , the same sensor index (which may or may not be equal to the first sensor index λ1), and the associated radius r O1 andr O2 has.

number

number

[0123] In one exemplary embodiment of the methods 100 and 200, the sensing index s O,i , radius r O,i and a set of occupied rough points P with the same sensor index O,i We can estimate the derivatives from these occupied rough points P by, for example, the least mean double method. O,i From the equation r=a *We can obtain the regression line of s+b. The derivative is dr / ds≒a.

[0124] In one exemplary embodiment of the methods 100 and 200, the method further comprises: 2,i ) and a second correction value C2 based on the first sensor index λ1, and at least one second sensor index λ2 (or λ 2,i ) having at least one second occupied rough point P2 (or P 2,i ) associated with at least one second radius r2 (or r 2,i ) the selected prediction radius r pred The method may further include the step of modifying

[0125] This example embodiment is advantageous because it corrects for sample index differences that may occur when the second sample index of the second occupied coarse point differs from the first sample index. r pred,corr =r pred +C2

[0126] In one exemplary embodiment of the methods 100 and 200, the correction value C2 is O1 and P O2 The two occupied rough points P O1 and P O2 Each sensor index λ O1 and λ O2 , the same sample index (which may or may not be equal to the first sample index s1), and an associated radius r O1 andr O2 has.

number

number

[0127] In one exemplary embodiment of the methods 100 and 200, the sensor index λ O,i , radius r O , i and a set of occupied rough points P with the same sample index O,i We can estimate the derivatives from these occupied rough points P by, for example, the least mean double method. O、i From the equation r=c * We can obtain the regression line of λ+d. The derivative is dr / ds≒c.

[0128] In one exemplary embodiment of the methods 100 and 200, the selected prediction radius r pred may be obtained as a radial function response to a first sample index s1 and a first sensor index λ1, the radial function being a response of at least two other second occupied rough points (P 2,i ) the second radius r 2,i The at least two second occupied rough points P are obtained by linear regression of 2,i The second sample index s of 2,i and the second sensor index λ 2,i is obtained based on

[0129] For example, the equation r=e * λ+f * The linear function of s+g is 2,i The prediction radius r pred get. r pred =e * λ1+f * s1+g

[0130] FIG. 25 shows a block diagram of the steps of a variation of the method 100 in FIG.

[0131] In step 105, a list of candidate radii L={r2} is obtained for the first occupied coarse point P1. The list of candidate radii L includes at least one second radius r2 associated with at least one second occupied coarse point P2 having a second sensor index λ2 different from the first sensor index λ1 and having a second sample index s2 less than or equal to the first sample index s1.

[0132] The at least one second occupied rough point may be obtained by any embodiment or variant discussed in relation to FIG. 17 or a combination thereof.

[0133] According to FIG. 25, the at least one second occupied rough point P2 further satisfies at least one qualification condition, the at least one qualification condition being that the radius associated with the at least one second occupied rough point P2 is less than or equal to a residual radius r res This indicates that the proposed method may reduce the dynamics of the

[0134] In one embodiment of a variant of the method 100, the radius associated with the occupied rough point selected based on at least one embodiment or variant considered in relation to FIG. 17 or any combination thereof is determined to be equal to or less than the predicted radius r if the at least one occupied rough point does not satisfy the at least one qualification condition. pred The qualification condition is to select an occupied rough point, which may be selected based on at least one embodiment or variant discussed in relation to FIG.

[0135] In step 110, a predicted radius r is selected for the first occupied rough point P1 from the list of candidate radii L. pred Get the.

[0136] In step 120, data I pred is encoded into a bit stream B, and the data I pred is the selected predicted radius r in the list of candidate radii Lpred Represents.

[0137] In step 130, the residual radius r res and encode it into bitstream B.

[0138] In one example embodiment of step 110, a predicted radius r selected from a plurality of radii associated with the plurality of second occupied rough points and / or the plurality of third occupied rough points is selected. pred When selecting, the selected prediction radius r pred is the residual radius r res corresponds to the radius that minimizes a cost function that represents the trade-off between the bit rate (used to encode

[0139] FIG. 26 shows a block diagram of the steps of a variation of the method 200 of FIG.

[0140] Consider a first occupied coarse point P1 in the two-dimensional coordinate (s, λ) space, where the first occupied coarse point P1 has a first sample index s1 and a first sensor index λ1.

[0141] In step 105, a list of candidate radii L={r2} is obtained for the first occupied coarse point P1. The list of candidate radii L includes at least one second radius r2 associated with at least one second occupied coarse point P2 having a second sensor index λ2 different from the first sensor index λ1 and having a second sample index s2 less than or equal to the first sample index s1.

[0142] The at least one second occupied rough point may be obtained based on any embodiment or variant or combination thereof discussed in relation to FIG.

[0143] According to FIG. 26, the at least one second occupied rough point P2 further satisfies at least one qualification condition, the at least one qualification condition being that the radius associated with the at least one second occupied rough point P2 is less than or equal to a residual radius r res This indicates that the proposed method may reduce the dynamics of the

[0144] In one embodiment of a variant of the method 200, the radius associated with the occupied rough point selected based on at least one embodiment or variant considered in relation to FIG. 18 or any combination thereof is determined to be equal to or less than a predicted radius r if the at least one occupied rough point does not satisfy the at least one qualification condition. pred The qualifying condition is one of occupied coarse points that can be selected based on any combination of at least one embodiment or variant or combination thereof discussed in relation to FIG.

[0145] In step 210, data I is extracted from bit stream B. pred Decrypt the data I pred is the selected predicted radius (r pred )

[0146] In step 220, a residual radius r res Decrypt the

[0147] In step 230, the residual radius r res And, Data I pred and the selected predicted radius r obtained from the list of candidate radii L=r2. pred Based on this, we obtain the (decoded) radius r1 associated with the point of the first occupied coarse point (P1) representation.

[0148] In one exemplary embodiment of step 105 shown in FIG. 24, the list of candidate radii L={r2, r3} may further include at least one third radius r3 associated with at least one third occupied coarse point P3, the at least one third occupied coarse point P3 having a third sensor index λ3 equal to the first sensor index λ1 and a third sample index s3 less than or equal to the first sample index s1.

[0149] This exemplary embodiment is advantageous because it provides a good prediction radius when a new sensing point belongs to a new object, and provides a good prediction radius when a new sensing point belongs to a previously sensed object.

[0150] In one example embodiment of the variant of methods 100 and 200 shown in FIG. 27, the qualification condition may be based on a comparison of the second sample index s2 of the second occupied coarse point P2 with the third sample index s3 of the third occupied coarse point P3.

[0151] As can be seen from simple geometry, in the case of a vertical plane of sensing, the residual radius r res The predicted radius of r pred The dynamics for the distance |s pred -s1|, where s pred is the prediction radius r pred is the sample index of the occupied second (or third) coarse point associated with

[0152] In one variation, as shown in FIG. 27, the second occupied coarse point P2 may meet the qualification condition if the second sample index s2 is greater than the third sample index s3.

[0153] In this exemplary embodiment, it is permitted to select the second occupied rough point instead of the third occupied rough point only when the second occupied rough point satisfies the qualification condition. In this case, since the radius of the second occupied rough point is sensed later than the third occupied rough point, it is a better predicted radius than the radius associated with the third occupied rough point.

[0154] In one exemplary embodiment of the variations of methods 100 and 200, the qualification condition may be based on a comparison of the azimuth angles associated with the sensor for sensing the points of the point cloud associated with the first occupied rough point, the second occupied rough point, and the third occupied rough point.

[0155] This exemplary embodiment determines whether it is necessary to select the second or third rough point to predict the first radius r1 by comparing the azimuth angles associated with the occupied rough points. This exemplary embodiment is advantageous because the azimuth angle comparison provides better accuracy than the sample index comparison.

[0156] In one exemplary embodiment of the variations of methods 100 and 200, when the first azimuth angle difference A1 is lower than the second azimuth angle difference A2, the second occupied rough point P2 can satisfy the qualification condition. The first azimuth angle difference A1 is the difference between the first azimuth angle φ1 associated with the first occupied rough point P1 and the second azimuth angle φ2 associated with the second occupied rough point P2. A1 = |φ1 - φ2|

[0157] The second azimuth angle difference A2 is the difference between the first azimuth angle φ1 and the third azimuth angle φ3 associated with the third occupied rough point P3. A2 = |φ1 - φ3|

[0158] For example, if A1 > A2, the best prediction may be the radius associated with the third occupied rough point, and therefore the second occupied rough point is unqualified. Conversely (A1 < A2), the second occupied rough point is qualified.

[0159] In one exemplary embodiment of the methods 100 and 200 variants, if the distance D3 is greater than the threshold th1, the second occupied rough point P2 may satisfy the following qualification condition: D3>th1

[0160] This exemplary embodiment is advantageous because it selects the radius associated with the second occupied coarse point P2 only if the expected gain of a selected prediction radius equal to the radius r2 is unlikely to justify the cost of the additional syntax to signal the use of the radius r2.

[0161] A distance D3 between a second radius r2 associated with the second rough point P2 and a third radius r3 associated with the third rough point P3 may be calculated. D3=|r2-r3|

[0162] In one variant, the third occupied coarse point P3 at the calculated distance D3 may be the closest occupied coarse point, i.e. the third occupied coarse point that minimizes the distance between the third sample index s3 and the first sample index s1 of said third occupied coarse point P3.

[0163] In one exemplary embodiment of the methods 100 and 200 variants, the threshold th1 may be fixed.

[0164] In one variant, the threshold th1 is a function of the previously encoded or decoded residual radius r res The r res is associated with the occupied coarse point whose sensor index is equal to the first sensor index λ1.

[0165] For example, the threshold th1 is set to 1 / 2 the number of previously encoded / decoded residual radii r res Width of |r res It can be obtained from the average value of |

[0166] This variant is advantageous because it adapts the threshold th1 to the average prediction quality of the radius.

[0167] In one exemplary embodiment of the variant of the methods 100 and 200, at least two previously encoded / decoded second occupied rough points P O,1 and P O,2 If the ratio of the gradient of the radius estimated from and the gradient of the sample index is greater than a threshold, the second occupied rough point P2 can satisfy the following qualification condition:

number

[0168] In one variant, to reduce sensitivity to noise, the estimation of the radius and the gradient of the sample index can use two or more previously encoded / decoded second occupied coarse points.

[0169] This exemplary embodiment is advantageous because it selects the radius associated with the second occupied coarse point P2 only if the sensing point associated with said second occupied coarse point P2 does not actually belong to a plane perpendicular to the sensing direction, i.e. if the point actually belongs to a plane perpendicular to the sensing direction, the radii associated with successive sensing points then have a small change as a function of the sample index s, since there is no change (in a first approximation) as a function of the azimuth angle φ. In this case, the predicted radius is equal to the radius associated with the third occupied coarse point P3, which may provide a better coding performance compared to the case where the predicted radius is equal to the radius associated with the second occupied coarse point P2.

[0170] In one exemplary embodiment of the variant of methods 100 and 200, the threshold th2 is the minimum angle α between the sensing direction and the normal to the sensing plane. min may also correspond to th2=tan(α min )

[0171] For example, α min A value such as =20° may be chosen.

[0172] Once at least one second radius r2 has been obtained and determined to be acceptable, the encoder selects the best predictor from the third radius r3 and the at least one acceptable second radius r2. Data I(pred) must be sent to the decoder to inform it which predictor was used.

[0173] In one exemplary embodiment of step 120 or 210, data I pred is the selected prediction radius r pred is equal to the second radius r2 or the third radius r3.

[0174] In one exemplary embodiment of step 120 or 210, data I pred is the selected prediction radius r pred may include binary data b indicating whether the radius is the second radius r2 or the third radius r3.

[0175] For example, if the binary data b is 0, the selected prediction radius r pred indicates the second radius r2, and if the binary data b is 1, the selected predicted radius r pred is the third radius r3.

[0176] In one example embodiment of step 120 or 210, the at least one second radius r2 can form a first radius list L1={r2}, and the at least one third radius r3 can form a second radius list L2={r3}. Then, the binary data b can be a combination of the selected predicted radius r pred It can be indicated whether the radius list belongs to the first radius list or the second radius list.

[0177] In one exemplary embodiment of step 120 or 210, data I predmay further include a predictor index Idx, which is determined by the selected prediction radius r pred is equal to which radius in the first radius list or the second radius list.

[0178] In one example embodiment of step 120 or 210, a single second radius r2 and N p The third radius r 3,i (N p ≧1), the selected prediction radius r pred You can select the data (I pred ) includes a predictor index Idx, where the predictor index Idx is a given index value Idx V If not equal to , the predictor index Idx is the predictor of the selected prediction radius r pred indicates which radius r3 in the list is equal to. Otherwise, the data I pred is the selected prediction radius r pred is equal to the second radius r2 or a given index value Idx in the radius list V Binary data f indicating whether the third radius r3 indicated by HV Further includes:

[0179] According to this exemplary embodiment, the predictor index Idx is a predefined index value Idx V , the predictor index Idx is first encoded, and then the binary data f HV are selectively encoded.

[0180] The index Idx is a sequence of binary data f i It can be encoded in a unified manner by

[0181] For example, the predictor index Idx includes binary data f1 (FIG. 28). If the binary data f1 is equal to 1 (true), the predictor index Idx is equal to 1, and the predictor Idx is the radius r 3,1If f1=0 and f2=1, the predictor index Idx is equal to 2, and Idx is a predetermined index value Idx V (=2), so data I pred is binary data f HV Further includes f HV = 0, the binary data is the radius r 3,2 If f1=0, f2=0 and f3=1, the predictor index Idx is equal to 3 and the radius r 3,3 For example, giving instructions.

[0182] In this way, a given index value Idx V The radius r with the lower index 3,i is given priority. This is advantageous since it has been observed that the second radius associated with the second occupied coarse point is more likely to be the best predictor compared to the third radius associated with the closest third occupied coarse point.

[0183] In the example of FIG. 28, a given index value Idx V Since is equal to 2, the third radius r associated with the third closest occupied rough point 3,1 Only the applicant will be given priority.

[0184] Predictor list 0…N p -1 If the index is in the range of a given index value Idx V is equal to 1 (not 2).

[0185] In one example embodiment of step 120 or 210, the at least one second radius r2 and the at least one third radius r3 may form a single radius list L={r2, r3}, and the data I pred is the selected prediction radius r pred The predictor index indicates which radius in the radius list the predictor belongs to.

[0186] This single radius list may be established by a "competition" between the predicted radii. In practice, the local statistics may be obtained by selecting all predicted radii from previously encoded / decoded points, depending for example on the occupied coarse point position relative to the first occupied coarse point P1.

[0187] In one variation, the radii in a single radius list are statistically sorted from the most selected one to the least selected one.

[0188] The radii in the radius list L are then sorted from the most selected one to the least selected one. This variant allows the use of multiple radii associated with second occupied coarse points and adapts locally to the structure of the point cloud.

[0189] FIG. 29 illustrates an example block diagram of a system for implementing various aspects and example embodiments.

[0190] System 300 may be implemented as one or more devices and may include various components as described below. In various embodiments, system 300 may be configured to implement one or more aspects described herein.

[0191] Examples of devices that may comprise all or part of system 300 include personal computers, laptop computers, smartphones, tablets, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, connected cars and their associated processing systems, head mounted displays (HMDs, see-through glasses), projectors, "caves" (systems including multiple displays), servers, video encoders, video decoders, post-processors that process output from video decoders, pre-processors that provide input to video encoders, web servers, set-top boxes, and any other devices for processing point clouds, videos, or images, or other communications devices. The elements of system 300 may be implemented singly or in combination on a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 300 may be distributed across multiple ICs and / or discrete components. In various embodiments, system 300 may be communicatively coupled to other similar systems or other electronic devices, for example, via a communications bus or dedicated input and / or output ports.

[0192] The system 300 includes at least one processor 310 configured to execute instructions loaded therein to, for example, implement aspects described herein. The processor 310 may include embedded memory, input / output interfaces, and various other circuits known in the art. The system 300 may include at least one memory 320 (e.g., volatile and / or non-volatile memory devices). The system 300 may include a storage device 340 including non-volatile and / or volatile memory, including, but not limited to, Electrically Erasable Programmable Read-Only Memory (EEPROM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Flash memory, magnetic disk drives, and / or optical disk drives. By way of non-limiting example, the storage device 340 may include an internal storage device, an additional storage device, and / or a network-accessible storage device.

[0193] The system 300 may include an encoder / decoder module 330 configured to process data to provide encoded / decoded point cloud geometry shape data, for example, and the encoder / decoder module 330 may include its own processor and memory. The encoder / decoder module 330 may represent a module(s) included in a device to perform encoding and / or decoding functions. As is known, a device may include either one or both of an encoding and decoding module. Also, the encoder / decoder module 330 may be implemented as a separate element of the system 300 or may be coupled within the processor 310 as a combination of hardware and software known to those skilled in the art.

[0194] Program code loaded into the processor 310 or the encoder / decoder 330 to perform aspects described herein may be stored in the storage device 340 and subsequently loaded into the memory 320 and executed by the processor 310. According to various embodiments, during execution of processes described herein, one or more of the processor 310, the memory 320, the storage device 340, and the encoder / decoder module 330 may store one or more of the following items: point cloud frames, encoded / decoded geometry / attribute videos / images or portions of encoded / decoded geometry / attribute videos / images, bitstreams, matrices, variables, and equations, formulas, logic for operations or intermediate or final results of operations.

[0195] In some embodiments, memory within the processor 310 and / or the encoder / decoder module 330 may be used to store instructions and provide working memory for processes performed during encoding or decoding.

[0196] However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 310 or the encoder / decoder module 330) is used for one or more of these functions. The external memory may be the memory 320 and / or the storage device 340, e.g., dynamic volatile memory and / or non-volatile flash memory. In some embodiments, the external non-volatile flash memory is used to store, for example, an operating system of a television. In at least one embodiment, a fast external dynamic volatile memory such as RAM may be utilized as working memory for video encoding / decoding and decoding operations, e.g., for MPEG-2 Part 2 (also known as ITU-T Recommendation H.262 and ISO / IEC 13818-2, also known as MPEG-2 Video), HEVC (High Efficiency Video Coding / Decoding), VVC (Versatile Video Coding / Decoding), or MPEG-I Parts 5 or 9.

[0197] As indicated in block 390, inputs can be provided to the elements of system 300 via various input devices. Such input devices include, but are not limited to, (i) an RF section capable of receiving RF signals transmitted wirelessly from a broadcast station or the like, (ii) a composite input terminal, (iii) a USB input terminal, and / or (iv) an HDMI input terminal.

[0198] In various embodiments, the input devices of block 390 have associated corresponding input processing elements as known in the art. For example, the RF section may be associated with each of the following required elements: (i) selecting a desired frequency (also called signal selection, or limiting the signal within a frequency band), (ii) downconverting the selected signal, (iii) selecting a signal frequency band by controlling the frequency band back to a narrow frequency band, (for example) referred to as a channel in some embodiments, (iv) demodulating the downconverted and frequency band limited signals, (v) performing error correction, and (vi) demultiplexing to select a desired data packet flow. The RF section of various embodiments includes elements purposely performing these functions, such as frequency selectors, signal selectors, frequency band limiters, channel selectors, filters, downconverters, demodulators, error correction devices, and demultiplexers. The RF section may include a tuner performing each of these functions, including, for example, downconverting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency near baseband) or to baseband.

[0199] In one set-top box embodiment, the RF section and its associated input processing elements can receive RF signals transmitted over a wired (e.g., cable) medium, after which the RF section can perform frequency selection by filtering, down-converting, and re-filtering to obtain a desired frequency band.

[0200] In various embodiments, the order of these (and other) elements may be rearranged, some of these elements may be eliminated, and / or other elements that perform similar or different functions may be added.

[0201] Adding elements can include inserting elements such as amplifiers and analog-to-digital converters between existing elements, In various embodiments, the RF section can include an antenna.

[0202] Additionally, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 300 to other electronic devices via USB and / or HDMI connections. It should be noted that aspects of the input processing (e.g., Reed-Solomon error correction) may be implemented, for example, in a separate input processing IC or in the processor 310, when desired. Thus, it should be understood that aspects of the USB or HDMI interface processing may be implemented, when desired, in a separate interface IC or in the processor 310. Upon demodulation, the error corrected and demultiplexed stream may be provided to various processing elements, including the processor 310 and the encoder / decoder 330, which operates in conjunction with memory and storage elements, to process the data stream when desired for display on an output device.

[0203] The various elements of the system 300 may be provided within a unitary housing in which a suitable connection layout 390 (e.g., internal buses known in the art, including I2C buses, wires, and printed circuit boards) may be used to connect the elements together and transmit data between them.

[0204] System 300 may include a communication interface 350 such that it may communicate with other devices over a communication channel 700. Communication interface 350 may include, but is not limited to, a transceiver configured to transmit and receive data over communication channel 700. Communication interface 350 may include, but is not limited to, a modem or a network card, and communication channel 700 may be implemented within a wired and / or wireless medium, for example.

[0205] In various embodiments, a Wi-Fi network, such as IEEE 802.11, may be used to stream data to the system 300. The Wi-Fi signal in these embodiments may be received via a communication channel 700 suitable for Wi-Fi communication and the communication interface 350. The communication channel 700 in these embodiments may typically be connected to an access point or router that provides access to external networks, including the Internet, allowing streaming applications and other over-the-top wireless communications.

[0206] Another embodiment may provide streaming data to the system 300 using a set-top box, which carries the data through an HDMI connection in the input block 390.

[0207] In some embodiments, the RF connection of input block 390 is used to provide streaming data to system 300 .

[0208] Streaming data can be used as a form of signaling information used by the system 300. The signaling information can include information on the number of points, coordinates, and / or sensor configuration parameters, such as a bit stream B and / or a point cloud.

[0209] It should be noted that signaling can be achieved in various manners, for example, in various embodiments, one or more syntax elements, flags, etc. can be used to transmit signaling information to a corresponding decoder.

[0210] System 300 can provide output signals to a variety of output devices, including a display 400, speakers 500, and other peripherals 600. In various example embodiments, other peripherals 600 can include one or more of a separate DVR, a disc player, a stereo system, a lighting system, and other devices that provide functionality based on the output of system 300.

[0211] In various embodiments, control signals may be communicated between system 300 and display 400, speaker 500 or other peripherals 600 using AV.Link (Audio / Video Link), CEC (Consumer Electronics Control) or other communication protocol signaling enabling device-to-device control, with or without a user.

[0212] Output devices can be communicatively connected to the system 300 via dedicated connections by corresponding interfaces 360, 370 and 380.

[0213] Optionally, an output device can be connected to the system 300 using a communication channel 700 via the communication interface 350. The display 400 and the speaker 500 can be integrated into a single unit along with other components of the system 300 such as an electronic device (e.g., a television).

[0214] In various embodiments, the display interface 360 ​​can include a display driver, such as a timing controller (T Con) chip.

[0215] For example, if the RF portion of input 390 is part of a separate set-top box, display 400 and speaker 500 are optionally separate from one or more of the other components. In various embodiments where display 400 and speaker 500 may be external components, the output signals may be provided via dedicated output connections (including, for example, an HDMI port, a USB port, or a COMP output terminal).

[0216] 1-29, various methods are described herein, each of which includes one or more steps or actions to achieve the described method. To the extent that a specific order of steps or actions is required for the precise operation of the method, the order and / or use of specific steps and / or actions can be modified or combined.

[0217] Although some examples have been described with respect to block diagrams and / or operational flow charts, each block represents a circuit element, a module, or a portion including one or more executable instruction codes for implementing a specified logic function(s). It should be noted that in other embodiments, the function(s) shown in the blocks may not occur in the order shown. For example, two blocks shown one after the other may in fact be executed essentially in parallel, or the blocks may be executed in the reverse order, depending on the functions involved.

[0218] For example, the embodiments and aspects described herein may be implemented in a method or process, an apparatus, a computer program, a data stream, a bit stream, or a signal. Even if discussed only in the context of a single type of embodiment (e.g., discussed only as a method), embodiments of the discussed features may be implemented in other forms (e.g., an apparatus or a computer program).

[0219] The methods may be implemented, for example, in a processor, which generally refers to a processing device including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device, etc. Processors further include communication devices.

[0220] Also, the methods may be implemented with instructions executed by a processor, and such instructions (and / or data values ​​produced by the embodiments) may be stored in a computer-readable storage medium. A computer-readable storage medium may take the form of a computer-readable program product having computer-readable program code embodied in and executable by a computer, embodied in one or more computer-readable media. Given the inherent ability to store information thereon and retrieve information provided thereby, a computer-readable storage medium as used herein may be considered a non-transitory storage medium. A computer-readable storage medium may be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. The following provides more specific examples of computer readable storage media to which the present embodiment can be applied, but as one of ordinary skill in the art would readily recognize, it should be understood that these are merely illustrative and not an exhaustive list: portable computer floppy disks, hard disks, read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0221] The instructions may create an application tangibly embodied on a processor-readable medium.

[0222] For example, instructions may reside in hardware, firmware, software, or a combination thereof. For example, instructions may be found in an operating system, a separate application, or a combination of both. A processor may thus be characterized as, for example, a device configured to perform a process or a device that includes a processor-readable medium (e.g., a storage device) having instructions for performing a process. Additionally, in addition to or in lieu of instructions, the processor-readable medium may store data values ​​produced by an embodiment.

[0223] The devices may be implemented in, for example, suitable hardware, software, and firmware. Examples of such devices include personal computers, laptop computers, smartphones, tablets, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, head mounted displays (HMDs, see-through glasses), projectors, "caves" (systems including multiple displays), servers, video encoders, video decoders, post-processors that process output from video decoders, pre-processors that provide input to video encoders, web servers, set-top boxes, and any other device for processing point clouds, video or images, or other communications devices. It is noted that the devices may be mobile and may be mounted in a moving vehicle.

[0224] The computer software may be implemented in the processor 310, hardware, or a combination of hardware and software. As a non-limiting example, an embodiment may be implemented in one or more integrated circuits. The memory 320 may be of any type suitable for the technology environment and may be implemented in any suitable data storage technology (as a non-limiting example, for example, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memories and removable memories). As a non-limiting example, the processor 310 may be of any type suitable for the technology environment and may cover one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0225] As will be apparent to one skilled in the art, the embodiments can generate various signals shaped to carry, for example, storable or transmittable information. The information can include, for example, instructions for performing a method or data generated by one of the described embodiments. For example, the signal can be shaped to carry a bit stream of the described embodiments. The signal can be shaped, for example, into an electromagnetic wave (e.g., a radio frequency portion of the frequency spectrum) or a baseband signal. The shaping can include, for example, encoding the data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal can be, for example, analog or digital information. As is well known, the signal can be transmitted over different wired or wireless links. The signal can be stored in a processor readable medium.

[0226] The terms used herein are used only to describe particular embodiments and are not intended to be limiting. Unless the context clearly dictates otherwise, the singular forms "a," "an," and "the" as used herein include the plural forms. Furthermore, as used herein, the terms "include / comprise" and / or "including / comprising" may indicate the presence of a described feature, integer, step, operation, element, and / or component, etc., but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof. Also, when an element is said to be "responsive to" or "connected" to another element, it may be directly responsive to or connected to the other element, or intermediate elements may be present. Conversely, when an element is said to be "directly responsive to" or "directly connected" to another element, intermediate elements are not present.

[0227] For example, in the case of "A / B," "A and / or B," and "at least one of A and B," the use of any one of the symbols / terms " / ," "and / or," and "at least one of" is intended to cover the selection of the first listed option (A), or the selection of the second listed option (B), or the selection of two options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to cover only the selection of the first listed option (A), or only the selection of the second listed option (B), or only the selection of the third listed option (C), or only the selection of the first and second listed options (A and B), or only the selection of the first and third listed options (A and C), or only the selection of the second and third listed options (B and C), or all three options (A, B, and C). This can be extended to any number of listed items as would be apparent to one of skill in the art.

[0228] Various numerical values ​​may be used in this application, and the specific values ​​are exemplary, and the embodiments described are not limited to these specific values.

[0229] It should be noted that terms such as first, second, etc. may be used to describe various elements in this specification, but are not limited to these terms. These terms are used only to distinguish one element from another element. For example, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element, without departing from the teachings of this application. No ordering between a first element and a second element is implied.

[0230] References to "one example" or "example" or "one embodiment" or "embodiment" and other variations are often used to convey that a particular feature, structure, characteristic, etc. (as described in conjunction with the example / embodiment) is included in at least one example / embodiment. Thus, appearances of the terms "in one example" or "in an example" or "in some examples" or "in one embodiment" or "in an embodiment" and any other variations appearing in various places in this application are not necessarily all referring to the same example.

[0231] Similarly, references herein to "according to an embodiment" or "in an embodiment" and other variations are often used to convey that a particular feature, structure, or characteristic (as described in conjunction with an embodiment) may be included in at least one embodiment. Thus, the phrases "according to an embodiment" or "in an embodiment" appearing in various places in the specification do not necessarily refer to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments.

[0232] The reference numerals of the drawings appearing in the claims are used for explanation only and do not limit the scope of the claims. Although not explicitly described, the present embodiment / examples and modifications can be used in any combination or partial combination.

[0233] It is understood that when a figure is presented as a flow chart, a corresponding apparatus block diagram is also provided. Similarly, it is understood that when a figure is presented as a block diagram, a flow chart of the corresponding method / process is also provided.

[0234] Some figures include arrows that indicate a primary direction of communication in a communication path, however, it should be understood that communication can occur in a direction opposite to that of the illustrated arrow.

[0235] Various embodiments relate to decoding. As used herein, "decoding" may cover all or part of a process, such as performing a received point cloud frame (which may include a received bitstream encoding one or more point cloud frames) to generate a final output suitable for display or further processing in a reconstructed point cloud domain. In various embodiments, such a process may include one or more of the processes typically performed by a decoder. In various embodiments, for example, such a process may alternatively include a process performed by a decoder of various embodiments described herein.

[0236] As a further example, in one embodiment, "decoding" may refer to only inverse quantization, in one embodiment, "decoding" may refer to entropy decoding, in another embodiment, "decoding" may refer to differential decoding, and in another embodiment, "decoding" may refer to a combination of inverse quantization, entropy decoding, and differential decoding. Depending on the context specifically described, it is obvious and easy to understand for one skilled in the art whether the term "decoding process" refers specifically to a subset of operations or to a more general decoding process.

[0237] Various embodiments relate to encoding. As at least in the manner of the above discussion of "decoding," "encoding" as used herein may cover all or part of the process of, for example, executing an input point cloud frame to generate an encoding bitstream. In various embodiments, this type of process includes one or more of the processes typically performed by an encoder. In various embodiments, such processes may include, or may selectively include, the processes performed by the encoder of each embodiment described herein.

[0238] As a further example, in one embodiment, "encoding" may refer only to quantization, in one embodiment, "encoding" may refer only to entropy coding, in another embodiment, "encoding" may refer only to differential coding, and in another embodiment, "encoding" may refer to a combination of quantization, differential coding, and entropy coding. Depending on the context in which a particular description is made, it may be clear and easy to understand for one of ordinary skill in the art whether the term "encoding process" refers specifically to a subset of operations or to a more general encoding process.

[0239] Additionally, the application may refer to "obtaining" various pieces of information. Obtaining information may include one or more of estimating information, calculating information, predicting information, or looking up information from a memory.

[0240] Additionally, the application may refer to "accessing" various pieces of information, which may include one or more of receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0241] The application may also refer to "receiving" various information. Like "access," receiving is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from a memory). In another manner, terms of operations such as storing information, processing information, transmitting information, moving information, copying information, deleting information, calculating information, determining information, predicting information, or estimating information are typically associated with "receiving."

[0242] Moreover, as used herein, the term "signal" specifically refers to instructing a corresponding decoder to do something specific. For example, in some embodiments, an encoder transmits a signal to inform a specific piece of information, such as the number or coordinates of points in a point cloud or sensor setting parameters. In this manner, the same parameters can be used at the encoder side and the decoder side in some embodiments. Thus, for example, the encoder can transmit a specific parameter to the decoder (explicit signaling), so that the decoder can use the same specific parameter. Conversely, if the decoder has a specific parameter and another parameter, signaling that does not require transmission (indirect signaling) can be used to inform the decoder and facilitate the selection of the specific parameter. To avoid transmitting any actual function, bit savings are achieved in various embodiments. It should be appreciated that signaling can be accomplished in various ways. For example, in various embodiments, one or more grammatical elements, flags, etc. are used to transmit information to a corresponding decoder. Although the above relates to the verb form of the word "signal", the word "signal" may also be used as a noun in this specification.

[0243] Although several embodiments have been described above, it should be understood that various modifications may be made. For example, elements of different embodiments may be combined, supplemented, modified, or deleted to produce other embodiments. Also, as will be appreciated by those skilled in the art, other structures and processes may be substituted for the disclosed structures and processes, resulting in embodiments that perform essentially the same function(s) in essentially the same way(s) to achieve at least essentially the same result(s) as the disclosed embodiments. Accordingly, these and other embodiments are contemplated by the present application.

Claims

1. 1. A method of encoding point cloud geometry data sensed by at least one sensor associated with a sensor index into a bitstream, the point cloud geometry data being represented by ordered coarse points occupying several discrete locations of a set of discrete locations in a two-dimensional space, each occupied coarse point being positioned in the two-dimensional space by a sensor index associated with a sensor sensing a point cloud point associated with the occupied coarse point and a sample index associated with a sensing time at which the point cloud point was sensed, each occupied coarse point being associated with a radius based on a distance from the point cloud point to a reference, the method comprising: - the first sensor index (λ １ ) and a first radius (r １ ) associated with the first occupied rough point (P １ ), the first sensor index (λ １ ) and a second sensor index (λ ２ At least one second occupied rough point (P ２ ) associated with at least one second radius (r ２ ), and the first sensor index (λ １ The sensor index (λ) is equal to ３ At least one third occupied rough point (P ３ ) and at least one third radius (r ３ ) from the selected prediction radius (r ｐｒｅｄ ) selecting a - the selected prediction radius (r ｐｒｅｄ ) is the second radius (r ２ ) or the third radius (r ３ ) is equal to ｐｒｅｄ ) into a bitstream; - the first radius (r １ ) and the selected prediction radius (r ｐｒｅｄ ) the residual radius (r ｒｅｓ ) into a bitstream. A method for encoding point cloud geometry data sensed by at least one sensor associated with a sensor index into a bitstream.

2. 1. A method for decoding point cloud geometry data sensed by at least one sensor associated with a sensor index from a bitstream, the point cloud geometry data being represented by ordered coarse points occupying several discrete positions of a set of discrete positions in a two-dimensional space, each occupied coarse point being positioned in the two-dimensional space by a sensor index associated with a sensor sensing a point cloud point associated with the occupied coarse point and a sample index associated with a sensing time at which the point cloud point was sensed, each occupied coarse point being associated with a radius based on a distance from the point cloud point to a reference, the method comprising: - the first sensor index (λ １ ) the first occupied rough point (P １ ), the selected prediction radius (r ｐｒｅｄ ) is the second radius (r ２ ) or the third radius (r ３ ) is equal to ｐｒｅｄ ), wherein the selected prediction radius (r ｐｒｅｄ ) is the first sensor index (λ １ ) and a second sensor index (λ ２ At least one second occupied rough point (P ２ ) associated with at least one second radius (r ２ ), and the first sensor index (λ １ The sensor index (λ) is equal to ３ At least one third occupied rough point (P ３ ) and at least one third radius (r ３ ) and - the first occupied rough point (P １ ), the residual radius (r ｒｅｓ ) and -Residual radius (r ｒｅｓ ) and Data (I ｐｒｅｄ ) is the selected prediction radius (r ｐｒｅｄ ) and the first occupied rough point (P １ ) the radius (r １ ) A method for decoding point cloud geometry data sensed by at least one sensor associated with a sensor index from a bitstream.

3. Data (I ｐｒｅｄ ) is the selected prediction radius (r ｐｒｅｄ ) is the second radius (r ２ ) or the third radius (r ３ ) The method according to claim 1 or 2.

4. The at least one second radius (r ２ ) is the first radius list (L １ = {r ２ }), and forming the at least one third radius (r ３ ) is the second radius list (L ２ = {r ３ }), and the binary data is calculated based on the selected prediction radius (r ｐｒｅｄ ) belongs to the first radius list or the second radius list; The method according to claim 3.

5. Data (I ｐｒｅｄ ) further includes a predictor index (Idx), which is a function of the selected prediction radius (r ｐｒｅｄ ) is equal to which radius in the first radius list or the second radius list; The method according to claim 4.

6. A single second radius (r ２ ) and at least one third radius (r ３ ) list (L = {r ３ }) and select the selected prediction radius from the data (I ｐｒｅｄ ) includes a predictor index (Idx), and the predictor index (Idx) has a predetermined index value (Idx Ｖ ), the predictor index (Idx) is equal to the selected prediction radius (r ｐｒｅｄ ) is which radius (r ３ ) if not, the data (I ｐｒｅｄ ) is the selected prediction radius (r ｐｒｅｄ ) is the second radius (r ２ ) or a given index value (Idx) in the radius list Ｖ ) the third radius (r ３ ) is equal to binary data (f ＨＶ ) The method according to claim 1 or 2.

7. The at least one second radius and the at least one third radius (r ２ , r ３ ) is a single radius list (L = {r ２， r ３ }) and forming data (I ｐｒｅｄ ) is the selected prediction radius (r ｐｒｅｄ ) is equal to a radius in the radius list; The method according to claim 1 or 2.

8. Statistically sorting the radii in a single radius list from the most selected one to the least selected one; The method according to claim 7.

9. 1. An apparatus for encoding point cloud geometry data sensed by at least one sensor associated with a sensor index into a bitstream, the point cloud geometry data being represented by ordered coarse points occupying a number of discrete locations of a set of discrete locations in a two-dimensional space, each occupied coarse point being positioned in the two-dimensional space by a sensor index associated with a sensor sensing a point cloud point associated with the occupied coarse point and a sample index associated with a sensing time at which the point cloud point was sensed, each occupied coarse point being associated with a radius based on a distance from the point cloud point to a reference, the apparatus comprising at least one processor, the at least one processor comprising: - the first sensor index (λ １ ) and a first radius (r １ ) associated with the first occupied rough point (P １ ), the first sensor index (λ １ ) and a second sensor index (λ ２ At least one second occupied rough point (P ２ ) associated with at least one second radius (r ２ ), and the first sensor index (λ １ The sensor index (λ) is equal to ３ At least one third occupied rough point (P ３ ) and at least one third radius (r ３ ) from the selected prediction radius (r ｐｒｅｄ ) and click - the selected prediction radius (r ｐｒｅｄ ) is the second radius (r ２ ) or the third radius (r ３ ) is equal to ｐｒｅｄ ) into a bitstream, - the first radius (r １ ) and the selected prediction radius (r ｐｒｅｄ ) the residual radius (r ｒｅｓ ) into a bitstream; An apparatus for encoding point cloud geometry data sensed by at least one sensor associated with a sensor index into a bitstream.

10. 1. An apparatus for decoding point cloud geometry data sensed by at least one sensor associated with a sensor index from a bitstream, the point cloud geometry data being represented by ordered coarse points occupying a number of discrete positions of a set of discrete positions in a two-dimensional space, each occupied coarse point being positioned in the two-dimensional space by a sensor index associated with a sensor sensing a point cloud point associated with the occupied coarse point and a sample index associated with a sensing time at which the point cloud point was sensed, each occupied coarse point being associated with a radius based on a distance from the point cloud point to a reference object, the apparatus comprising at least one processor, the at least one processor comprising: - the first sensor index (λ １ ) the first occupied rough point (P １ ), the selected prediction radius (r ｐｒｅｄ ) is the second radius (r ２ ) or the third radius (r ３ ) is equal to ｐｒｅｄ ) and decode the selected prediction radius (r ｐｒｅｄ ) is the first sensor index (λ １ ) and a second sensor index (λ ２ At least one second occupied rough point (P ２ ) associated with at least one second radius (r ２ ), and the first sensor index (λ １ The sensor index (λ) is equal to ３ At least one third occupied rough point (P ３ ) and at least one third radius (r ３ ) are selected from - the first occupied rough point (P １ ), the residual radius (r ｒｅｓ ) and -Residual radius (r ｒｅｓ ) and Data (I ｐｒｅｄ ) is the selected prediction radius (r ｐｒｅｄ ) and the first occupied rough point (P １ ) the radius (r １ ) configured to obtain An apparatus for decoding point cloud geometry data sensed by at least one sensor associated with a sensor index from a bitstream.

11. A computer program, when executed by one or more processors, causing the one or more processors to perform a method for encoding point cloud geometry data sensed by at least one sensor associated with a sensor index into a bitstream, the point cloud geometry data being represented by ordered coarse points occupying several discrete locations of a set of discrete locations in a two-dimensional space, each occupied coarse point being positioned in the two-dimensional space by a sensor index associated with a sensor sensing a point cloud point associated with the occupied coarse point and a sample index associated with a sensing time at which the point cloud point was sensed, each occupied coarse point being associated with a radius based on a distance from the point cloud point to a reference, the method comprising: - the first sensor index (λ １ ) and a first radius (r １ ) associated with the first occupied rough point (P １ ), the first sensor index (λ １ ) and a second sensor index (λ ２ At least one second occupied rough point (P ２ ) associated with at least one second radius (r ２ ), and the first sensor index (λ １ The sensor index (λ) is equal to ３ At least one third occupied rough point (P ３ ) and at least one third radius (r ３ ) from the selected prediction radius (r ｐｒｅｄ ) selecting a - the selected prediction radius (r ｐｒｅｄ ) is the second radius (r ２ ) or the third radius (r ３ ) is equal to ｐｒｅｄ ) into a bitstream; - the first radius (r １ ) and the selected prediction radius (r ｐｒｅｄ ) the residual radius (r ｒｅｓ ) into a bitstream. Computer program.

12. 1. A non-transitory storage medium carrying program code instructions for executing a method for encoding into a bitstream point cloud geometry data sensed by at least one sensor associated with a sensor index, the point cloud geometry data being represented by ordered coarse points occupying several discrete locations of a set of discrete locations in a two-dimensional space, each occupied coarse point being positioned in the two-dimensional space by a sensor index associated with a sensor sensing a point cloud point associated with the occupied coarse point and a sample index associated with a sensing time at which the point cloud point was sensed, each occupied coarse point being associated with a radius based on a distance from the point cloud point to a reference, the method comprising: - the first sensor index (λ １ ) and a first radius (r １ ) associated with the first occupied rough point (P １ ), the first sensor index (λ １ ) and a second sensor index (λ ２ At least one second occupied rough point (P ２ ) associated with at least one second radius (r ２ ), and the first sensor index (λ １ The sensor index (λ) is equal to ３ At least one third occupied rough point (P ３ ) and at least one third radius (r ３ ) from the selected prediction radius (r ｐｒｅｄ ) selecting a - the selected prediction radius (r ｐｒｅｄ ) is the second radius (r ２ ) or the third radius (r ３ ) is equal to ｐｒｅｄ ) into a bitstream; - the first radius (r １ ) and the selected prediction radius (r ｐｒｅｄ ) the residual radius (r ｒｅｓ ) into a bitstream. Non-transitory storage media.

13. A computer program, when executed by one or more processors, causing the one or more processors to perform a method of decoding, from a bitstream, point cloud geometry data sensed by at least one sensor associated with a sensor index, the point cloud geometry data being represented by ordered coarse points occupying several discrete positions of a set of discrete positions in a two-dimensional space, each occupied coarse point being located in the two-dimensional space by a sensor index associated with a sensor sensing a point cloud point associated with the occupied coarse point and a sample index associated with a sensing time at which the point cloud point was sensed, each occupied coarse point being associated with a radius based on a distance from the point cloud point to a reference, the method comprising: - the first sensor index (λ １ ) the first occupied rough point (P １ ), the selected prediction radius (r ｐｒｅｄ ) is the second radius (r ２ ) or the third radius (r ３ ) is equal to ｐｒｅｄ ), wherein the selected prediction radius (r ｐｒｅｄ ) is the first sensor index (λ １ ) and a second sensor index (λ ２ At least one second occupied rough point (P ２ ) associated with at least one second radius (r ２ ), and the first sensor index (λ １ The sensor index (λ) is equal to ３ At least one third occupied rough point (P ３ ) and at least one third radius (r ３ ) and - the first occupied rough point (P １ ), the residual radius (r ｒｅｓ ) and -Residual radius (r ｒｅｓ ) and Data (I ｐｒｅｄ ) is the selected prediction radius (r ｐｒｅｄ ) and the first occupied rough point (P １ ) the radius (r １ ) Computer program.

14. 1. A non-transitory storage medium carrying program code instructions for executing a method for decoding point cloud geometry data sensed by at least one sensor associated with a sensor index from a bit stream, the point cloud geometry data being represented by ordered coarse points occupying several discrete locations of a set of discrete locations in a two-dimensional space, each occupied coarse point being positioned in the two-dimensional space by a sensor index associated with a sensor sensing a point cloud point associated with the occupied coarse point and a sample index associated with a sensing time at which the point cloud point was sensed, each occupied coarse point being associated with a radius based on a distance from the point cloud point to a reference, the method comprising: - the first sensor index (λ １ ) the first occupied rough point (P １ ), the selected prediction radius (r ｐｒｅｄ ) is the second radius (r ２ ) or the third radius (r ３ ) is equal to ｐｒｅｄ ), wherein the selected prediction radius (r ｐｒｅｄ ) is the first sensor index (λ １ ) and a second sensor index (λ ２ At least one second occupied rough point (P ２ ) associated with at least one second radius (r ２ ), and the first sensor index (λ １ The sensor index (λ) is equal to ３ At least one third occupied rough point (P ３ ) and at least one third radius (r ３ ) and - the first occupied rough point (P １ ), the residual radius (r ｒｅｓ ) and -Residual radius (r ｒｅｓ ) and Data (I ｐｒｅｄ ) is the selected prediction radius (r ｐｒｅｄ ) and the first occupied rough point (P １ ) the radius (r １ ) Non-transitory storage media.

Images