Method and apparatus for data coding and decoding using hybrid prediction of coarse representations of points of point cloud

Abstract

A method for encoding a point cloud is described. The method includes encoding, for one or more points of the point cloud having a coarse representation, order difference information between a coarse position of a current point and a coarse position of a previous point in a current frame of the point cloud selectively using an intra prediction scheme or an inter prediction scheme.

Description

Technical Field

[0001] This invention generally relates to data compression, and more specifically, to methods and apparatus for encoding / decoding point clouds (e.g., lidar data acquired by a moving vehicle). Embodiments of the invention relate to methods and apparatus for encoding order difference information between the approximate position of a current point and the approximate position of a previous point in a current frame of a point cloud by selectively using intra-frame prediction schemes or inter-frame prediction schemes. Background Technology

[0002] Data compression is used in communications and computer networking to efficiently store, transmit, and copy information. For example, point clouds have recently gained significant attention as a format for representing three-dimensional (3D) data because they can flexibly represent all types of 3D objects or scenes. Therefore, point clouds can address many use cases, including… • Film post-production • Real-time 3D immersive telepresence or VR / AR (virtual reality / augmented reality) applications, • Free viewpoint videos, for example, for sports viewing. Geographic Information System (GIS), also known as mapping, • Cultural heritage, for example, storing scanned copies of rare items in digital form. • Autonomous driving, including 3D mapping of the environment and real-time LiDAR data acquisition (LiDAR: light detection and ranging = a method of measuring distance (range) by illuminating a target with laser light and measuring the reflection with a sensor).

[0003] For example, existing point cloud codecs cannot provide the required encoder and decoder simplicity, low latency, and compression performance for transmitting LiDAR data acquired by mobile vehicles. For encoding and decoding point clouds, such as for transmitting LiDAR data acquired by mobile vehicles, combining encoder and decoder simplicity, low latency, and compression performance is important.

[0004] Therefore, there is a need to provide a solution to the problems in the prior art by combining the simplicity, low latency and compression performance of encoders and decoders, for example, for transmitting lidar data acquired by mobile vehicles. Summary of the Invention

[0005] This disclosure provides a method for encoding point clouds, the method comprising: For one or more points in a point cloud with a coarse representation, an intra-frame prediction scheme or an inter-frame prediction scheme is selectively used to encode the order difference information between the coarse position of the current point and the coarse position of the previous point in the current frame of the point cloud.

[0006] Optionally, the best of the intra-frame prediction scheme and the inter-frame prediction scheme is selected to encode the order difference information.

[0007] Optionally, a rate-distortion optimization (RDO) process can be used to select the best option among the intra-frame prediction scheme and the inter-frame prediction scheme.

[0008] Optionally, the RDO procedure is used to determine the first bit cost of encoding the order difference information using an intra-frame prediction scheme and the second bit cost of encoding the order difference information using an intra-frame prediction scheme.

[0009] Optionally, the first bit cost is compared with the second bit cost to determine whether to use an inter-frame prediction scheme for the current point with a coarse location.

[0010] Optionally, - When the cost of the first bit is higher than the cost of the second bit, an inter-frame prediction scheme is used for the current point with a coarse location, and - When the cost of the first bit is equal to or less than the cost of the second bit, the inter-frame prediction scheme is not used for the current point with a coarse position.

[0011] Optionally, the inter-frame prediction scheme uses the order difference residual obtained from the previous frame through a coarse representation of the reference point cloud.

[0012] Optionally, the order difference residual is obtained by using one or more order differences between one or more successive points in the previous frame and a point, wherein the point is a previous point with a coarse position in the previous frame, and wherein the one or more successive points include one or more points with a coarse position in the previous frame, which are located at a position corresponding to the coarse position of the current point or within a certain range of the coarse position of the current point.

[0013] Optionally, the previous point with a coarse position in the previous frame corresponds to the previous point with a coarse position in the current frame, and the current point with a coarse position in the previous frame corresponds to the current point in the current frame.

[0014] Optionally, one or more order differences used to encode the current point in the current frame are obtained in the following way: - Search in the previous frame for a point with a coarse position that corresponds to the current point with a coarse position in the current frame. - Obtain the order difference between the approximate position of a point in the previous frame and the approximate position of the point preceding it in the previous frame, and - Obtain the order difference residual using the obtained order difference and the order difference between the current point's coarse position and the previous point's coarse position in the current frame.

[0015] Optionally, a coarse location with the same index as the current point in the current frame can be searched in the previous frame.

[0016] Optionally,

[0017] in Point P in the previous frame i The approximate position index in -1, and Current point P in the current frame i The approximate positional index in the text.

[0018] Optionally, a coarse location of a point with a specific laser index and azimuth sampling angle is searched in the previous frame, wherein the specific laser index and azimuth sampling angle are... - Corresponds to the laser index and azimuth sampling angle of the coarse position of the current point in the current frame, or - Within a specific range of the laser index and azimuth sampling angle of the approximate position of the current point in the current frame.

[0019] Alternatively, the order difference residuals are obtained by the following formula:

[0020] in Order difference residuals The difference in the approximate position of the current point Pn in the current frame i. The difference in the approximate position of point P in the previous frame i-1.

[0021] Optionally, if there is no approximate position of a point corresponding to the approximate position of the current point in the current frame, the approximate position of a point located within a predefined range of the location corresponding to the approximate position of the current point in the current frame is searched in the previous frame.

[0022] Optionally, in the previous frame, a point within a predefined range of the order index of the previous point in the current frame is selected as the approximate position of the point.

[0023] Optionally, Select the order index in the previous frame. The order index of the approximate position of the current point in the current frame Points that are roughly represented before or after, and The order difference is set to the order index in the previous frame. The difference in order between the selected point and the points before or after the selected point in the previous frame.

[0024] Optionally, if there is no coarse position of a point corresponding to the coarse position of the current point in the current frame, the inter-frame prediction scheme is not used for the current point in the current frame, but the intra-frame prediction scheme is used to encode the order difference information between the current point and the previous point in the current frame.

[0025] Alternatively, entropy coding can be used to encode the order difference residuals, such as context-based adaptive binary arithmetic codec (CABAC).

[0026] Optionally, the order difference residuals can be encoded into the data stream.

[0027] Optionally, the intra-frame prediction scheme uses the order difference between the coarse position of the current point in the current frame and the coarse position of the previous point.

[0028] Alternatively, entropy coding can be used to encode the order differences, such as context-based adaptive binary arithmetic codec (CABAC).

[0029] Optionally, the order difference can be encoded into the data stream.

[0030] Optionally, the intra-frame prediction scheme uses a first entropy encoder, while the inter-frame prediction scheme uses a second entropy encoder. The first and second entropy encoders are independent encoders that use separate context tables to model the statistical probabilities of the intra-frame prediction scheme and the inter-frame prediction scheme, respectively.

[0031] Optionally, a first flag is provided, indicating whether, for one or more coarse points, - Encode the order difference information using an intra-frame prediction scheme, or - Use an inter-frame prediction scheme to encode the order difference information.

[0032] Optionally, the first flag is encoded into the data stream, such as a bit stream including data representing the point cloud.

[0033] Optionally, intra-frame prediction schemes and inter-frame prediction schemes may be selectively used in response to specific signaling.

[0034] Optionally, the specific signaling includes a second flag indicating whether to selectively use intra-frame prediction schemes and inter-frame prediction schemes.

[0035] Optionally, the second flag is included in the geometric parameter set (GPS) or sequence parameter set (SPS) of the data stream, such as the L3C2 bitstream structure.

[0036] Optionally, the coarse representation for each original point in the point cloud is obtained by quantizing one or more original points using azimuth fundamental shift quantization, wherein the coarse location contains one or more original points.

[0037] This disclosure provides a method for decoding a point cloud from a data stream, the method comprising: Decode signaling from the data stream, wherein the signaling indicates whether, for one or more points of the point cloud with a coarse representation, an intra-frame prediction scheme or an inter-frame prediction scheme is used to decode the order difference information between the coarse position of the current point in the current frame and the coarse position of the previous point.

[0038] Optionally, when signaling indicates the use of an inter-frame prediction scheme, the method includes: - Decode the order difference residual from the data stream. - Using the order difference residual and the order difference between the coarse positions of the first and second points in the previous frame, the order difference between the coarse positions of the current point and the previous point in the current frame is obtained, where the coarse position of the first point corresponds to the coarse position of the current point and the coarse position of the second point corresponds to the coarse position of the previous point. - Use the obtained order difference and the order index of the coarse position of the previous point in the current frame to obtain the order index of the current point in the current frame, and - Construct a rough representation of the point cloud.

[0039] Alternatively, the order difference can be obtained as follows: +

[0040] in Current frame i The difference in order between the approximate position of the current point and the approximate position of the previous point. Previous frame i The difference in order between the approximate positions of the first and second points in -1, and Order difference residuals.

[0041] Alternatively, the approximate index of the current point's position can be obtained as follows: +

[0042] in Current frame i The order index of the approximate position of the current point. Previous frame i The approximate index of the first point in -1. Current frame i The difference in order between the approximate position of the current point and the approximate position of the previous point. Previous frame i The difference in order between the approximate positions of the first and second points in -1, and Order difference residuals.

[0043] Optionally, when signaling indicates the use of an intra-frame prediction scheme, the method includes: - Decode the order difference between the coarse position of the current point in the current frame and the coarse position of the previous point from the data stream. - Use the decoding order difference and the order index of the coarse position of the previous point in the current frame to obtain the coarse position index of the current point in the current frame, and - Construct a rough representation of the point cloud.

[0044] Alternatively, the approximate index of the current point's position can be obtained as follows:

[0045] in Current frame i The order index of the approximate position of the current point. Previous frame i The approximate index of the first point in -1, and Current frame i The difference in order between the approximate position of the current point and the approximate position of the previous point.

[0046] Optionally, based on the current point P n The laser index λ and azimuth sampling angle Ф are obtained by ordering the approximate positions. c At the current point P n plane (Ф) c , λ n The rough representation is constructed in )

[0047]

[0048] in The total number of laser beams.

[0049] Optionally, the signaling includes a flag indicating whether, for one or more points with a coarse representation, - To use an intra-frame prediction scheme to decode order difference information, or - An inter-frame prediction scheme should be used to decode the order difference information.

[0050] This disclosure provides a non-transitory computer-readable storage medium including instructions that, when executed by a computer, cause the computer to perform the methods of this disclosure.

[0051] This disclosure provides an encoder comprising: An encoder module, wherein the encoder module is configured to encode a point cloud according to the method of this disclosure.

[0052] This disclosure provides an encoder comprising: Encoder module, The encoder module is configured to encode point clouds into a data stream, and Specifically, for one or more points in a point cloud with a coarse representation, the encoder module is configured to selectively use an intra-frame prediction scheme or an inter-frame prediction scheme to encode the order difference information between the coarse position of the current point in the current frame and the coarse position of the previous point.

[0053] This disclosure provides a decoder, which includes: Decoder module, The decoder module is configured to decode point clouds according to the method of this disclosure.

[0054] This disclosure provides a decoder, which includes: Decoder module, The decoder module is configured to decode point clouds from the data stream, and Specifically, for one or more points of a point cloud with a coarse representation, the decoder module is configured to decode signaling from the data stream, wherein the signaling indicates whether an intra-frame prediction scheme or an inter-frame prediction scheme is used to decode the order difference information between the coarse position of the current point and the coarse position of the previous point in the current frame of the point cloud.

[0055] This disclosure provides a data stream encoding a point cloud, wherein the data stream includes: The signaling indicates whether, for one or more points in a point cloud with a coarse representation, an intra-frame prediction scheme or an inter-frame prediction scheme should be used to decode the order difference information between the coarse position of the current point and the coarse position of the previous point in the current frame of the point cloud.

[0056] The technical solution provided by the embodiments of the present invention has the following beneficial effects. By introducing hybrid prediction of the coarse representation of the point cloud acquired by lidar, that is, by selectively using intra-frame prediction schemes or inter-frame prediction schemes to encode / decode the difference information between the coarse position of the current point and the coarse position of the previous point in the current frame of the point cloud acquired by lidar, the lidar data encoding and decoding performance is improved.

[0057] It should be understood that the description in this section is not intended to identify key or essential features of embodiments of the invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0058] The accompanying drawings are illustrative and intended to explain the invention, and should not be construed as limiting the invention to the embodiments shown.

[0059] Figure 1 The illustration shows a spin lidar head, which includes several spin lasers for detecting the environment. Figure 2 The diagram shows... Figure 1 The elevation angle of the spin laser relative to the horizontal ground; Figure 3 The diagram illustrates how to pass through Figure 1 The 2D angular representation of a spin lidar or the (Ф, θ) plane and its point P; Figure 4 The figure illustrates the points obtained in the discrete (Ф, θ) representation; Figure 5 The diagram illustrates the 3D coordinates and angle-based coordinates of points used in point clouds; Figure 6 The diagram illustrates the acquisition order in the (Ф, λ) plane associated with the corresponding lasers of the lidar head; Figure 7 The illustration shows points along the azimuth and laser index, obtained through actual acquisition; Figure 8 The diagram illustrates the order of points in a plane defined by a rough azimuth and a laser index; Figure 9 The diagram illustrates the representation of the point cloud by difference Δnext in the first lexicographical order; Figure 10 The diagram illustrates the encoder and encoding method used to encode a coarse single-chain representation of a point cloud acquired by lidar. Figure 11 The diagram illustrates the decoder and decoding method used to decode the coarse single-chain representation of point clouds acquired by lidar; Figures 12(A) and 12(B) illustrate a rough representation of two lidar point cloud frames with a 1-second time difference; Figure 13 The diagram illustrates the decoder and decoding method for decoding a coarse single-chain representation of a point cloud acquired by a lidar using an order difference prediction method. Figure 14 (A) and Figure 14 (B) illustrates an example of dissimilar regions in a coarse representation of two adjacent point cloud frames, where Figure 14 (A) illustrates a rough representation of the previous frame, while Figure 14 (B) The diagram shows a rough representation of the current frame; Figure 15 The illustration shows a flowchart of a method for encoding point clouds acquired by a lidar according to an embodiment; Figure 16 The illustration shows a flowchart of a method for encoding point clouds acquired by lidar according to a further embodiment; Figure 17 A flowchart illustrating an embodiment of the method for encoding a coarse representation disclosed herein is shown; Figure 18 The illustration shows a flowchart of an embodiment for obtaining the corresponding successive order differences in the previous cloud frame; Figure 19 The diagram illustrates a flowchart of a method for decoding point clouds acquired from a LiDAR radar from a data stream; Figure 20 The illustration shows a flowchart of an embodiment of the method disclosed herein for decoding geometric and attribute information of points in a point cloud from a bitstream; Figure 21 A flowchart illustrating another embodiment of the method for decoding a coarse representation disclosed herein is shown; Figure 22 A block diagram of an encoder operating according to an embodiment of the present disclosure is shown; Figure 23 A block diagram of a decoder operating according to an embodiment of the present disclosure is shown; Figure 24 The illustration shows a data flow according to an embodiment of the present disclosure; and Figure 25 A block diagram of an electronic device according to an embodiment of the present disclosure is illustrated. Detailed Implementation

[0060] The illustrative embodiments of this disclosure will now be described with reference to the accompanying drawings, including various details of the embodiments of this disclosure to facilitate understanding and should be considered illustrative only. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Furthermore, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0061] In this disclosure, the term "and / or" is intended to cover all possible combinations and sub-combinations of the listed elements, including any one, any sub-combination, or all of the elements listed individually, without necessarily excluding additional elements.

[0062] In this disclosure, the phrase “...or at least one of ...” is intended to cover any one or more of the listed elements, including any one of the elements listed separately, any sub-combination of elements, or all elements, without necessarily excluding any additional elements and without necessarily requiring all elements.

[0063] In this disclosure, the term "encoder / decoder" means "encoding" or "decoding," as is apparent from the context of the described embodiments. Similarly, the term "encoder / decoder" means "encoder" or "decoder."

[0064] Point cloud codec A point cloud is a set of points in 3D space, each optionally with additional values. These additional values ​​are often referred to as point attributes. Therefore, a point cloud is a combination of geometry (the 3D position of each point) and attributes.

[0065] The attribute can be, for example, a three-component color, a material property (such as reflectivity), and / or a two-component normal vector of the surface associated with that point.

[0066] Point clouds can be captured by various types of devices, such as cameras, depth sensors, LiDAR, and scanner arrays, or they can be computer-generated, for example, in film post-production. Depending on the use case, for mapping applications, a point cloud can have thousands to billions of points.

[0067] The raw representation of point clouds requires a very large number of bits per point, with each spatial component X, Y, or Z requiring at least a dozen bits, and optionally even more bits for one or more attributes, such as three times the 10 bits required for color. Practical deployment of point cloud-based applications requires compression techniques that enable the storage and distribution of point clouds with a reasonable storage and transmission infrastructure.

[0068] Compression can be lossy, as in video compression, for distribution to end users and for their visualization, such as in AR / VR glasses or any other device with 3D capabilities. Other use cases do require lossless compression, such as medical applications or autonomous driving, to avoid altering decision-making outcomes derived from the analysis of compressed and transmitted point clouds.

[0069] Point cloud compression in MPEG Until recently, point cloud compression (also known as PCC) has only recently gained mainstream market attention, and there were no standardized point cloud codecs available. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11, also known as the Moving Picture Experts Group or MPEG, launched a working project on point cloud compression, which has resulted in two standards: • MPEG-I Part 5 (ISO / IEC 23090-5), also known as video-based point cloud compression (V-PCC); • MPEG-I Part 9 (ISO / IEC 23090-9), also known as geometry-based point cloud compression (G-PCC).

[0070] The first versions of the V-PCC and G-PCC standards were completed in 2020 and 2022, respectively.

[0071] When processing moving point clouds, the V-PCC encoding / decoding method compresses the point cloud by performing multiple projections onto the 3D object to obtain two-dimensional (2D) patches that are then packed into an image or video. The image or video is then compressed using an existing image / video codec, allowing full utilization of deployed image and video solutions. By its very nature, V-PCC is only efficient on dense and continuous point clouds because if non-smooth patches are obtained from projections of sparse geometry data acquired, such as from LiDAR, then image / video codecs cannot compress them.

[0072] The G-PCC encoding and decoding method has two schemes for geometric compression.

[0073] The first approach is based on an occupancy tree representation of point cloud geometry, such as an octree, quadtree, or binary tree representation. Occupied nodes are split until a certain size is reached, and the occupied leaf nodes provide the locations of points, typically located at the center of these nodes. High levels of compression can be achieved for dense point clouds by using neighbor-based prediction techniques. Sparse point clouds can also be addressed by directly encoding and decoding the locations of points within nodes with non-minimum sizes, by stopping tree construction when only isolated points exist in a node; this technique is called Direct Codec (DCM).

[0074] The second approach is based on a prediction tree, where each node represents the 3D location of a point and the relationship between nodes is a spatial prediction from parent to child nodes. Compared to using an occupancy tree, this method can address only sparse point clouds and offers the advantages of lower latency and simpler decoding. However, compared to the first approach, while offering slightly better compression performance, the encoding is complex due to the need to densely search for the best predictor among a large number of potential predictors when constructing the prediction tree.

[0075] In both schemes, attribute encoding / decoding (e.g., attribute coding and attribute decoding) is performed after the complete geometry has been encoded / decoded, resulting in two encoding / decoding processes. Low latency can be achieved by slicing the 3D space into independently encoded / decoded sub-volumes without requiring prediction between sub-volumes. However, this severely impacts compression performance when using many slices.

[0076] A key use case is the transmission of LiDAR data acquired by moving vehicles. This typically requires a simple, low-latency embedded encoder. Simplicity is crucial because the encoder is likely to be deployed on computing units performing other processing in parallel, such as autonomous or semi-autonomous driving, thus limiting the processing power available to the point cloud encoder. Low latency is also required to allow for rapid transmission from the vehicle to the cloud, enabling real-time views of local traffic based on multiple vehicles and sufficiently rapid decisions based on traffic information. While transmission latency can be reduced sufficiently low by using, for example, 5G-compliant (fifth-generation cellular network technology) radio links, it is expected that the encoder itself will not introduce significant latency due to encoding and decoding. Furthermore, compression performance is also critical, as the data stream from potentially millions of vehicles to the cloud is expected to be very heavy.

[0077] The simplicity, low latency, and compression performance of combining encoders and decoders remain unsatisfactory problems that existing point cloud codecs have yet to solve.

[0078] Compression of point clouds acquired by rotating sensors In G-PCC, certain methods related to acquiring lidar data have been utilized, resulting in significant gains in compression. The first technique involves acquiring the vertical angle relative to the horizontal ground from a spin lidar. Figure 1 The illustration shows a spin lidar head 100, which includes several spin lasers 102a-102e that detect the environment by emitting corresponding laser beams 104a-104e. As schematically illustrated at 106, the spin lidar head 100 rotates or spins about a rotation axis 108. Figure 2 The illustration shows a spin laser (such as...) Figure 1The laser 102a) is at an elevation angle θ relative to the horizontal ground 110. According to some method, the elevation angle θ is fixed.

[0079] By utilizing a second technology, G-PCC goes even further, taking advantage of the regularity of laser sensing during lidar spin. Figure 3 The illustration shows a 2D angle (Ф,θ) representation or a (Ф,θ) plane 112 and a point P on plane 112 acquired by a spin lidar 100. A regular distribution of point P along the azimuth angle Ф can be observed in the lidar-acquired data. This regularity is used to obtain a quasi-1D representation of the point cloud, where, within the noise range, only the radius r... 3D It belongs to the continuous value range, while the angles Ф and θ only take discrete values.

[0080] Basically, the point cloud geometry on the 2D discrete angular plane 112 can be represented together with the radius value of each point. Figure 4 The diagram illustrates a point P acquired on a discrete (Ф,θ) representation 112, and predicts the location of the current point relative to an encoded point in spherical coordinates by using the discrete properties of angles Ф and θ. This quasi-1D property has been utilized in both the occupancy tree and prediction tree of G-PCC.

[0081] Occupation tree intensively uses DCM and performs entropy encoding and decoding on the direct location of point P within a node using a context-adaptive entropy codec. The context can be obtained from the local transformation of the point location to the angle (Ф,θ) coordinates.

[0082] Prediction tree direct diagonal coordinates (r) 2D The coordinates (x, y, z) are encoded and decoded, and then converted into (x, y, z) coordinates. The corresponding x, y, and z coordinate residuals are then encoded and decoded to resolve errors in coordinate transformation, laser angle approximation, and noise. Figure 5 The diagram illustrates the 3D (x, y, z) coordinates and angle-based coordinates (r) of points used in a point cloud. 2D ,Ф,θ) or (r 3D ,Ф,θ), where r 3D It is the radius or distance from the lidar head or the laser of the lidar head to point P, while r 2D It is the radius of the projection onto the horizontal (x,y) plane 112.

[0083] Therefore, conventionally speaking, there are two main types of encoding and decoding structures: occupancy tree and prediction tree.

[0084] Roughly expressed Within the framework of a rotating lidar system, a coarse single-chain representation of the point cloud acquired by the lidar can be used, as described in more detail, for example, in EP4020816A1 (EP application number 20306672.5). When using this method, all lasers are encoded and decoded at once using the order in which they are acquired. Figure 6 Arrow 114 illustrates the acquisition sequence in plane 116, defined by the azimuth angle Ф and the laser index λ associated with the corresponding lasers 102a-102e of the lidar head 100. Due to the regular rotation of the lidar head 100 and the continuous acquisition of each laser 102a-102e at fixed time intervals, two points detected by the same laser (such as...) Figure 6 The azimuth distance between P1 and P2 is a multiple of the basic azimuth shift ΔФ.

[0085] Instead of directly encoding and decoding the points, a coarse representation can be encoded and decoded. For example, a coarse representation (Фc,λ) of the point cloud geometry can be used to sort the points using a dictionary order (also known as lexicographical order), first by the coarse azimuth Фc, and then by the laser index λ or the sensed elevation index θ. Figure 7 The illustration shows point P along the azimuth angle Ф and laser index λ, obtained through actual acquisition. Schematic, point P is located in the (Ф,λ) plane... Figure 7 The acquisition is performed in the sequence 114 shown. Due to the regular rotation of the lidar head 100 and the continuous acquisition by each laser 102a-102e at fixed time intervals, the azimuth distance between two points P1 and P2 detected by the same laser is a multiple of the basic azimuth shift ΔФ. In reality, not all points are acquired; that is, the laser beam may not be reflected, there is acquisition noise, and the laser may not be perfectly aligned. Figure 7 The diagram illustrates point P obtained through actual data acquisition, i.e., the actual data. The approximate angle Фc is simply obtained by quantizing Ф as follows. Фc = round(Ф / ΔФ), And the order index o(P) of point P is obtained by the following formula. , Where N laser λ is the number of lasers and λ is the index of the laser at point P, located in [0, N]. laser In [-1], the codec monotonically encodes point P according to the order o(P), for example, using ascending order. Therefore, Figure 7 P point in Figure 8 The sequence 118 depicted in the figure is encoded and decoded, which illustrates the order 118 of points P in a plane 116 defined by the coarse azimuth angle Фc and the laser index λ.

[0086] The coarse representation in the plane (Фc,λ) can be encoded and decoded in the following way. • Number of points N points , • The value of (Фc,λ) of the first obtained point. • N between the current point and the next point, sorted lexicographically points -1 successive difference Δnext.

[0087] Figure 9 The diagram illustrates the representation of the point cloud by the difference Δnext in the first lexicographical order (Фc,λ). As can be seen, the coarse representation includes successive differences Δnext, and the compression of the coarse representation is essentially based on the compression of successive positive values ​​Δnext.

[0088] Point cloud codec Figure 10 The diagram illustrates an overview of the encoder and encoding method described above for encoding a coarse single-chain representation of a point cloud acquired by lidar.

[0089] Encoder 200 receives the x, y, z points. Encoder 200 converts the x, y, z points into a laser index λ, a coarse angle Фc, and a radius r. 2D For example, knowledge of the settings of the lidar sensor 100 is used. The difference Δnext is determined, and then the encoder 200 encodes the difference Δnext 204 into a data stream or bit stream 206 (e.g., via entropy encoding). Moreover, useful information about the lidar sensor settings, such as ΔФ and the laser elevation angle, can also be encoded into the bit stream 206.

[0090] Encoder 200 determines or calculates the reconstructed azimuth angle Фrec, for example, directly from the dequantization of the coarse angle Фc. Calculation. Optionally, the residuals Фres can be calculated as differences. And, for example, it is encoded 208 times into bitstream 206 via entropy coding. Before encoding 208, the residual Фres can be quantized 210 times into Q(Фres). In this case, the reconstructed azimuth angle Фrec is obtained by the following formula.

[0091] Where IQ represents the inverse quantization process 212.

[0092] radius r (here r) 2D The ) is also encoded and decoded 208 into bitstream 206, optionally after quantization 210 to Q(r). It is then inversely quantized 212 to obtain the reconstructed radius. r rec = IQ(Q(r)).

[0093] The reconstructed azimuth angle Фrec and the reconstructed radius r rec Convert back to 214 x,y coordinates to obtain estimates of the x and y locations of the point:

[0094] .

[0095] The residual x relative to the original point x,y res and y res Obtained or calculated as x res = x - x estim y res = y - y estim , And, for example, it is encoded into a bitstream of 206 via entropy coding 216.

[0096] Vertical estimation z estim The laser angle θ(λ) is obtained from the following formula. z estim = r rec tan(θ(λ)), And the residual z relative to the original point z res Obtained or calculated by the following formula z res = z - z estim It is then encoded into bitstream 206 using 216 bits.

[0097] Figure 11 The document provides an overview of the decoder and decoding method used to decode a coarse single-chain representation of a point cloud acquired by LiDAR, as described above.

[0098] Decoder 300 receives bitstream 302 and decodes 304 difference Δnext from bitstream 302, for example, through entropy decoding. Furthermore, useful information about the sensor settings, such as ΔФ and laser elevation angle, can be decoded from bitstream 302. Then, the values ​​of 306 laser index λ and coarse angle Фc are obtained from Δnext.

[0099] Obtain or calculate the reconstructed azimuth angle Фrec. It can be directly dequantized from the coarse angle Фc. Alternatively, the 308-bit azimuth residual Фres can be decoded from bitstream 302. The decoded residual Фres can be a quantized version of the residual Фres, Q(Фres), and the reconstructed azimuth angle Фrec is obtained by the following formula. , Where IQ represents the inverse quantization process 310.

[0100] radius r (here r) 2D The bitstream is also decoded from bitstream 302 to 308. The encoded / decoded radius can be a quantized version of the radius, Q(r). It is then inversely quantized to obtain the reconstructed radius. r rec = IQ(Q(r)).

[0101] Alternatively, the radius can be predicted, for example, by pre-encoded radius, and the radius residual can be encoded and decoded instead of the radius.

[0102] The reconstructed azimuth angle Фrec and the reconstructed radius r rec Transform back to 312 x,y coordinates to obtain estimates of the x and y locations of that point:

[0103] .

[0104] Decoding 314 residuals from bitstream 302 res and y res The decoded horizontal location x of that point is calculated or obtained using the following formula. dec and y dec x dec = x estim + x y dec = y estim + y res .

[0105] Vertical estimation z estim It is obtained from the laser angle θ(λ) by the following formula. z estim = r rec tan(θ(λ)), And decode 314 residual z from bitstream 302. res The decoded vertical location z of the point is calculated or obtained using the following formula. dec z dec = z estim + z res .

[0106] In practical applications, the vertical angle range of LiDAR scanning can reach approximately 25° or more, for example, from -12.5° to +12.5°. The vertical angle range of the LiDAR laser beam can be divided into three parts: a higher laser beam, a central or intermediate laser beam, and a lower laser beam. Different parts of the laser beam can detect objects at different distances from or distinct from the vehicle. The higher laser beam can detect distant and tall objects, such as buildings or trees. The intermediate laser beam can detect nearby objects, such as cars, people, or bicycles in front. The lower laser beam can detect objects very close to the vehicle, typically the road in front of it. When the LiDAR device is operated on a moving vehicle, objects detected by the higher and lower laser beams do not suddenly disappear (e.g., within two adjacent LiDAR scan frames or a time interval of 0.1 seconds), thus maintaining the order of detection between two adjacent point cloud frames. They can be correlated, and their differences It can be smaller than Therefore, when encoding and decoding a coarse representation of the current lidar data frame (see...), Figure 10 )Will Encoding and decoding to bitstream saves time The number of bits directly encoded and decoded into the bitstream.

[0107] Figure 12 illustrates a rough representation of two lidar point cloud frames with a 1-second time difference. Figure 12(A) shows a rough representation of the lidar point cloud frame captured at time t, and Figure 12(B) shows a rough representation of the lidar point cloud frame captured at time t+1s (1 second). Figure 12 illustrates the intermediate point 140 captured by the intermediate laser beam, the higher point 142 captured by the higher laser beam, and the point 144 captured by the lower laser beam. As can be seen, some of the points 140 (see point P) in the plane (Фc,λ) can change slightly within 1 second due to object movement or the movement of vehicles ahead, and these slight changes can be found by comparing Figure 12(A) and Figure 12(B). Considering the slight changes of point 140 within 1 second, it can be understood that the rough representation of adjacent frames captured within 0.1 seconds is much smaller, especially for the points detected by the higher and lower laser beams. Therefore, the coarse representation information of the previously adjacent lidar point clouds can be used to predict the coarse representation of the lidar point cloud frame being encoded and decoded.

[0108] To reduce the temporal redundancy of coarsely represented information between successive lidar data frames, and thus further improve the compression performance of lidar data sequences, in order to optimize the current lidar point cloud frame (the first frame)... i Encoding and decoding are performed using the previous lidar point cloud frame (the (frame)). i -1) Two consecutive points P in frame) n With Pn-1 Differences in order between To predict the corresponding continuous point P in the current frame n With P n-1 Differences in order between The predicted order difference residuals Encode to / decode from bitstream. Used for point P. n The predicted order difference residuals can be obtained by the following formula. .

[0109] For each point, an order difference prediction method is used to encode / decode the current point cloud frame.

[0110] exist Figure 13 An overview of a decoder and decoding method for decoding a coarse single-chain representation of a point cloud acquired by lidar using an order difference prediction method is shown, as described, for example, in international application PCT / CN2024 / 075584.

[0111] Decoder 300 decodes point P of the LiDAR point cloud frame from bitstream 302 as follows. n : • Obtain two consecutive points P from the previous frame. n With P n-1 Compared with previously encoded / decoded points in the current frame P n-1 order Differences in the order of predictions between ; • Use, for example, context-based adaptive binary arithmetic codec (CABAC) to decode the order difference residuals predicted by 304 from bitstream 302. ; • Use the predicted order difference residuals by the following formula Difference in order from the previous frame Get P in the current frame n Order difference

[0112] + .

[0113] The obtained P is used by the following formula. n Order difference The order of previously encoded and decoded points in the current frame Get the order of the currently encoded / decoded points in the current frame.

[0114] + .

[0115] By using point P n The order of calculation of the laser index λ and the azimuth sampling angle Фc is used to construct a rough representation of 306 at point P. n In the plane (Фc, λn), the laser index λ and the azimuth sampling angle Фc can be calculated using the following formula. ,

[0116] in It represents the total number of laser beams in a lidar device.

[0117] Radius residual r res azimuth residual Фres and residual x res y res and z res According to the above reference Figure 11 The described process is decoded to reconstruct the points in the current frame. The geometric information (x, y, z).

[0118] After decoding the geometry and attribute information of the points in the current frame, the process continues to decode the next point in the current frame until the last point in the current frame is reached.

[0119] In the coarse representation of LiDAR point clouds, when compressing LiDAR data using a low-latency, low-complexity LiDAR codec (L3C2), one or more dissimilar regions with point occupancy may appear in the coarse representation due to moving objects in the captured scene or due to capture noise. For example, in LiDAR point cloud capture, when a LiDAR device scans a scene with moving objects, some regions may be located at the same position in two adjacent or consecutive frames, but there may be significant differences between the two adjacent point cloud frames. This can result in one or more dissimilar regions in the coarse representation of two adjacent point cloud frames when using the low-latency, low-complexity LiDAR codec (L3C2).

[0120] Figure 14 The illustration shows an example of dissimilar regions in a rough representation of two adjacent point cloud frames. Figure 14 (A) illustrates a rough representation of the previous frame, while Figure 14 (B) The diagram shows a rough representation of the current frame. (e.g.) Figure 14 As shown, points (such as P1 and P2) (see...) Figure 14 One or more approximate locations of (A) may appear in the previous frame but not at the corresponding locations in the current frame. Furthermore, points (such as P3 and P4 (see...)) may also appear in the previous frame but not at the corresponding locations in the current frame. Figure 14One or more coarse positions of (B) may appear in the current frame but not at the corresponding positions in the previous frame. This will cause the order difference residual to be large in such regions, when using the above reference. Figure 13 The process described, i.e., when applying a coarse representation of inter-frame prediction, reduces compression efficiency.

[0121] Embodiments of this disclosure improve compression performance for inter-frame prediction used in coarse representations, for example, in L3C2.

[0122] Figure 15 The illustration shows a flowchart of a method for encoding point clouds acquired by a lidar according to an embodiment. The method includes the following: S400: For one or more points with a coarse representation in the point cloud acquired by the lidar, an intra-frame prediction scheme or an inter-frame prediction scheme is selectively used to encode the order difference information between the coarse position of the current point and the coarse position of the previous point in the current frame of the point cloud acquired by the lidar.

[0123] Figure 16 The illustration shows a flowchart of a method for encoding point clouds acquired by a lidar according to a further embodiment. The method includes the following: S402: Receives data acquired by the lidar; S400: For one or more points with a coarse representation in the point cloud acquired by the lidar, an intra-frame prediction scheme or an inter-frame prediction scheme is selectively used to encode the order difference information between the coarse position of the current point and the coarse position of the previous point in the current frame of the point cloud acquired by the lidar.

[0124] Optionally, the method includes including one or more signaling messages (such as flags) into the data stream or bit stream, S404. This signaling indicates whether, for one or more points with a coarse representation, an intra-frame prediction scheme or an inter-frame prediction scheme should be used to decode the order difference information.

[0125] Therefore, embodiments of this disclosure introduce a hybrid method for inter-frame prediction of coarse representations, such as in the L3C2 scheme. According to embodiments, for each point with a coarse representation, a hybrid method of inter-frame prediction and intra-frame prediction is used. The inter-frame prediction method encodes and decodes order differences with reference to previous frames, while the intra-frame method directly encodes and decodes order differences from the current frame to encode and decode order difference information between the coarse position of the current point and the coarse position of the previous point in the current frame. According to embodiments, on the encoder side, the optimal method between the intra-frame method and the inter-frame prediction method is selected to better encode and decode the order difference information for each point. For example, a rate-distortion optimization (RDO) process or method can be used to determine the optimal scheme among the inter-frame prediction scheme and the intra-frame prediction scheme.

[0126] According to an embodiment, a flag is used for each point with a coarse representation, for example... Isinterflag It also indicates whether to use inter-frame prediction or intra-frame methods for points with coarse representations. For example, if the flag... Isinterflag If true, then use the inter-frame prediction method for that point; otherwise, if the flag is true... Isinterflag If false, then the intra-frame method is used for that point. (Flag) Isinterflag It can be encoded into / decoded from the bitstream at the encoder / decoder.

[0127] According to an embodiment, the inventive method uses a flag indicating whether a coarsely represented inter-frame prediction is used in the L3C2 scheme (e.g., InterPredictionEnabled ) is enabled / disabled. If the flag is InterPredictionEnabled If true, then the coarse-representation inter-frame prediction hybrid method is enabled; otherwise, the inventive method is disabled. This flag can be included in the geometric parameter set (GPS) or sequence parameter set (SPS) of the L3C2 bitstream structure.

[0128] In the low-latency, low-complexity lidar codec (L3C2), each point with a coarse representation is iterated in a point cloud frame. First, the coarse representation information (such as the successive difference Δnext) is encoded / decoded, then the residual information of r and Ф is encoded / decoded, and then the residual in the (x,y,z) domain is encoded / decoded, as shown in the reference above. Figure 10 As described. Finally, the attribute information of the current point is encoded / decoded. After the iteration of point geometry and attribute encoding / decoding is completed, the process continues to the next iteration to encode / decode the next point according to the above process. The inventive method exists in encoding / decoding a coarse representation of each point using intra-frame prediction or inter-frame prediction. Figure 17 A flowchart illustrating an embodiment of the method for encoding a coarse representation according to this disclosure is shown. For each point having a coarse representation, the following steps are performed.

[0129] S410: Obtain the current frame being encoded or decoded, for example, in the [frame number]. i In the frame, the current point Compared to the previous point (If the preceding point exists) the order difference between .

[0130] S412: By referring to the previous cloud frame (e.g., the ( ) i -1) The order difference value between successive points in a frame is used to obtain the order difference. Inter-frame prediction residual .

[0131] S414: Determine the optimal prediction method by using the RDO procedure, i.e., directly predicting the current point in the current frame. Intra-frame methods for encoding and decoding order differences and for the current point Order difference It is the best method for inter-frame prediction in encoding and decoding.

[0132] S416: If the inter-frame method is selected, i.e., determined to be the optimal method, then the flag will be set. Isinterflag Set to true.

[0133] S418: Mark Isinterflag Encode into a bitstream.

[0134] S420: According to the markings Isinterflag For the current point Encode the order difference information; if the flag Isinterflag If true, then use an entropy encoder to calculate the inter-frame prediction residuals. Encode into a bitstream.

[0135] S422: If the intra-frame method is selected, i.e., determined to be the optimal method, then the flag will be set. Isinterflag Set to false, where the flag indicates whether to use the inter-frame prediction method for the current point.

[0136] S424: Mark Isinterflag Encode into a bitstream.

[0137] S426: According to the markings Isinterflag For the current point Encode the order difference information, if the flag Isinterflag If it is false, then use the entropy encoder to set the current point. Compared to the previous point (If the preceding point exists) the order difference between Encode into a bitstream.

[0138] According to the embodiment, the RDO process used on the encoder side above estimates the sequence difference. Bit cost of entropy coding and inter-frame prediction residuals Bit cost of entropy coding Compare these two costs ( and This determines whether to use the coarse inter-frame prediction scheme based on the current point's location. For example, when the cost of the first bit is higher than the cost of the second bit, the coarse inter-frame prediction scheme based on the current point's location is used, while when the cost of the first bit is equal to or lower than the cost of the second bit, the coarse inter-frame prediction scheme based on the current point's location is not used.

[0139] According to the embodiment, the cloud frame at the previous point is referenced to obtain information including the current point. The order difference between a point P whose approximate position is close to or at the same position, and a point that precedes P in the previous frame, is used to obtain the order difference residual of the inter-frame prediction of the approximate position of that point. . Figure 18 The diagram illustrates the method for obtaining the successive order differences of points in the previous cloud frame to apply to points in the current frame. A flowchart of an embodiment for encoding and decoding a coarse representation. For each point with a coarse representation, the following steps are performed.

[0140] S440: Search for the approximate location of point P in the previous frame, which has a similar approximate location to the currently encoded / decoded point in the current frame. Same order index ,Right now , in Point P in the previous frame i The approximate position index in -1, and Current point In the current frame i The approximate positional index in the text.

[0141] S442: Determine whether the approximate location of the above point P exists in the previous point cloud frame.

[0142] S444: If the approximate position of point P exists in the previous point cloud frame, then obtain the previous frame (the ( i -1) The order difference between the approximate position of point P in frame P and the approximate position of its predecessor. .

[0143] If the approximate location of the point P being described does not exist in the previous point cloud frame: S446a: According to one embodiment, the inter-frame prediction method for the current point is terminated, and optionally an intra-frame prediction method is used, or S446b: According to another embodiment, a coarse location of point P in the previous frame is searched, which has an index close to one of the points currently being encoded / decoded. .

[0144] For example, the coarse position of point P in the previous frame can be used, which has the order index of the coarse position of the currently encoded / decoded point in the current frame. Previous or next order index ,Right now or

[0145] in Point P in the previous frame i The approximate position index in -1, and Current point P in the current frame i The approximate positional index in the text.

[0146] Then the order difference Set to have an ordered index The difference in order between the approximate position of point P and the approximate position of the point preceding or following point P (if any).

[0147] S448: Use the order difference obtained in the previous frame by the following formula and P in the current frame n Order difference Obtaining order difference residuals

[0148] .

[0149] In another embodiment, S440 may search for the approximate location of point P in the previous frame, which has a laser index λ and azimuth sampling angle Ф that are the same as or close to the approximate location of the currently encoded / decoded point in the current frame. c .

[0150] Figure 19 The flowchart illustrates a method for decoding point clouds acquired from a LiDAR radar from a data stream. This method includes the following: S500: Decodes signaling from the data stream.

[0151] The signaling indicates whether to use an intra-frame prediction scheme or an inter-frame prediction scheme to decode the order difference information between the coarse position of the current point and the coarse position of the previous point in the current frame of the point cloud acquired by the lidar, for one or more points with a coarse representation of the point cloud.

[0152] Figure 20 A flowchart illustrating an embodiment of the method of this disclosure is shown, which decodes the geometric and attribute information of a point (P) in a point cloud from a bitstream by iterating over each point having a coarse representation. For each point, the following steps are performed: S510: Decoding the plane of point clouds (Ф) c ,λ n The information in the plane (Ф) is a rough representation, including the order information of the first point and the order difference information between subsequent points. c ,λ nThe decoding of the coarse representation information in the present disclosure uses the hybrid method to perform inter-frame prediction on the coarse representation of the point. That is, the decoding S410 selectively uses an inter-frame or intra-frame prediction scheme to obtain the order difference information between the coarse position of the current point and the coarse position of the previous point in the current frame.

[0153] S512: Using a plane (Ф) c ,λ n The decoded rough representation of the decoding radius residual, azimuth residual, and (x,y,z) coordinate residual is used to obtain the reconstructed geometric information of the point.

[0154] S514: Use the obtained geometric information of the point to decode the attribute information of the point.

[0155] According to an embodiment, for inter-frame prediction of points with a coarse representation, a previous point cloud frame is referenced to obtain points closer to the current encoding / decoding. The difference in succession order between a point P at the same position as the approximate location and a point that is the previous point of P in the previous frame.

[0156] According to an embodiment, when using the flags mentioned above (e.g., Isinterflag When it is, it is decoded from the bitstream. If the decoded flags Isinterflag If true, then use an entropy decoder to decode the inter-frame prediction residuals from the bitstream. Otherwise, if the decoded flag Isinterflag If the result is false, then use another entropy decoder to decode the current point in the current frame from the bitstream. Compared to the previous point (If the preceding point exists) the order difference between .

[0157] According to the embodiment, in order to obtain the successive order difference corresponding to the previous point cloud frame to adjust the point in the current frame The coarse representation is encoded and decoded, and the coarse position of point P in the previous frame is searched, which has an index of the order of the coarse position of the point being encoded and decoded. Same order index .

[0158] According to an embodiment, the two entropy decoders are two independent decoders, each with a separate context table designed for each entropy decoder.

[0159] Figure 21 A flowchart illustrating another embodiment of the method for decoding a coarse representation disclosed herein is shown. For each point having a coarse location, the following steps are performed.

[0160] S520: Decode flags from bitstream Isinterflag This flag indicates whether an inter-frame prediction method is used for the current point.

[0161] S522: Identify the mark Isinterflag Is it true?

[0162] According to the sign Isinterflag Decode the current point from the bitstream. Information on the order differences.

[0163] If the logo Isinterflag True: S524: Decode inter-frame prediction residuals from the bitstream using an entropy decoder. .

[0164] S526: Residuals based on predicted order differences Difference from the order referenced in the previous frame Get the current frame Order difference ,as follows + .

[0165] S528: Based on the obtained formula below Order difference The order of previously encoded and decoded points in the current frame Get the order of the currently encoded / decoded points in the current frame.

[0166] + .

[0167] S530: By using point P n The order of laser index λ and azimuth sampling angle Ф is determined or calculated. c To construct point P n In plane (Ф) c , λ n A rough representation in () is, for example, as follows: ,

[0168] in It represents the total number of laser beams in a lidar device.

[0169] If the logo Isinterflag False: S532: Decode the current point of the current frame using the entropy decoder bitstream. Compared to the previous point (If the previous point exists) the order difference between them .

[0170] S534: Obtained by using the following formula Order difference The order of previously encoded and decoded points in the current frame Get the order of the currently encoded / decoded points in the current frame.

[0171] .

[0172] S530: By using point P n The order of laser index λ and azimuth sampling angle Ф is determined or calculated. c To construct point P n In plane (Ф) c , λ n A rough representation in () is, for example, as follows:

[0173] , in It represents the total number of laser beams in a lidar device.

[0174] According to an embodiment, based on the flag Isinterflag Whether it is false is determined by using an entropy encoder (e.g., CABAC) to check for order differences. Or order difference residuals Encoding or decoding is performed. In this embodiment, it is used to address order differences. Encode and decode ( Isinterflag = Fake The entropy encoder is independent of the residuals used for order difference. Encode and decode ( Isinterflag = True The two entropy encoders use separate context tables, which allows for better statistical probability modeling for each use case.

[0175] Further embodiments The concept of the invention has now been described with reference to aspects and embodiments related to encoding and decoding methods. According to further embodiments, this disclosure also provides encoders and decoders.

[0176] Figure 22 The diagram illustrates a block diagram of an encoder operating according to an embodiment of the present disclosure. Encoder 400 includes: 402: an encoder module configured to encode point clouds acquired by a LiDAR into a data stream.

[0177] For one or more points with a coarse representation in the point cloud acquired by the lidar, the encoder module is configured to selectively use an intra-frame prediction scheme or an inter-frame prediction scheme to encode the order difference information between the coarse position of the current point and the coarse position of the previous point in the current frame of the point cloud acquired by the lidar.

[0178] Figure 23 A block diagram of a decoder operating according to an embodiment of the present disclosure is illustrated. Decoder 500 includes: 502: Decoder module, configured to decode point clouds acquired by LiDAR from the data stream.

[0179] For one or more points with a coarse representation of a point cloud acquired by a lidar, the decoder module is configured to decode signaling from a data stream, wherein the signaling indicates whether an intra-frame prediction scheme or an inter-frame prediction scheme is used to decode the order difference information between the coarse position of the current point and the coarse position of the previous point in the current frame of the point cloud acquired by the lidar, for one or more points with a coarse representation of a point cloud acquired by the lidar.

[0180] Figure 24 The illustration shows a data stream 700 according to an embodiment of the present disclosure, in which a point cloud acquired by a lidar is encoded. Generally, the data stream may encode information representing the point cloud acquired by the lidar and / or information associated with encoding / decoding the point cloud acquired by the lidar. For example, the data stream or bit stream 700 may be provided by an encoder 400 performing the inventive method for encoding a picture into the data stream 700. The data stream 700 is transmitted via a wired or wireless transmission medium 702 (such as a cable or radio link) to a decoder 500, and the decoder 500 decodes the point cloud acquired by the lidar from the data stream 700. The data stream includes signaling 704 indicating whether, for one or more points of the point cloud acquired by the lidar with a coarse representation, an intra-frame prediction scheme or an inter-frame prediction scheme should be used to decode the order difference information between the coarse position of the current point and the coarse position of the previous point in the current frame of the point cloud acquired by the lidar.

[0181] Although some aspects of the disclosed concepts have been described in the context of the apparatus, it is clear that these aspects also represent descriptions of the corresponding methods, where boxes or devices correspond to method steps or features of method steps. Similarly, aspects described in the context of method steps also represent descriptions of corresponding boxes, items, or features of the corresponding apparatus.

[0182] Figure 18This is a block diagram illustrating an electronic device 900 according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are described by way of example only and are not intended to limit the implementations of the present disclosure described herein and / or claimed. Device 900 includes a computing unit 901 to perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 902 or loaded from storage unit 908 into random access memory (RAM) 903. Various programs and data for storing the operation of device 900 may also be stored in RAM 903. The computing unit 901, ROM 902, and RAM 903 are connected to each other via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0183] Components in device 900 are connected to I / O interface 905, including: input unit 906, such as a keyboard and mouse; output unit 907, such as various types of displays and speakers; storage unit 908, such as a disk and optical disk; and communication unit 909, such as a network card, modem, wireless communication transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices via computer networks (such as the Internet and / or various telecommunications networks). Computing unit 901 may consist of various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 901 performs the various methods and processes described above, such as image processing methods. For example, in some embodiments, image processing methods may be implemented as computer software programs tangibly implemented on a machine-readable medium (such as storage unit 908). In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by computing unit 901, one or more steps of the image processing method described above may be performed. In some embodiments, computing unit 901 may be configured to execute the image processing method in any other suitable manner (e.g., by means of firmware).

[0184] Various implementations of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may be implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, and the programmable processor may be a special-purpose or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and capable of transferring data and instructions to the storage system, at least one input device, and at least one output device.

[0185] Program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when the processor or controller executes the program code, it causes the functions and / or operations specified in the flowcharts and / or block diagrams to be performed. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0186] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program used by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0187] To provide interaction with the user, the systems and techniques described herein can be implemented on a computer with a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD)) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0188] The systems and technologies described herein can be implemented on a computing system including backend components (e.g., as data servers), or on a computing system including middleware components (e.g., application servers), or on a computing system including frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or on a computer system including such backend components, middleware components, frontend components, or any combination thereof. The components of the system can be interconnected via digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0189] Computer systems can include clients and servers. Clients and servers are generally remote to each other and typically interact via communication networks. The client-server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in a cloud computing service system and addresses the management difficulties and weak business scalability of traditional physical hosts and virtual private servers (VPS). A server can also be a server in a distributed system or a server combined with blockchain technology.

[0190] It should be understood that steps can be reordered, added, or deleted using the various forms of processes shown above. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution of this disclosure can be achieved, and no limitation is imposed herein.

[0191] The specific embodiments mentioned above do not limit the scope of protection of this disclosure. Those skilled in the art will understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure or this invention should be included within the scope of protection of this disclosure or this invention.

Claims

1. A method for encoding point clouds, the method comprising: For one or more points in the point cloud with a coarse representation, an intra-frame prediction scheme or an inter-frame prediction scheme is selectively used to encode the order difference information between the coarse position of the current point and the coarse position of the previous point in the current frame of the point cloud.

2. The method of claim 1, wherein the optimal scheme among the intra-frame prediction scheme and the inter-frame prediction scheme is selected to encode the order difference information.

3. The method of claim 1 or 2, wherein a rate-distortion optimization (RDO) process is used to select the optimal scheme among the intra-prediction scheme and the inter-prediction scheme.

4. The method of claim 3, wherein the RDO process is used to determine a first bit cost for encoding the order difference information using the intra-prediction scheme and a second bit cost for encoding the order difference information using the intra-prediction scheme.

5. The method of claim 4, wherein the first bit cost is compared with the second bit cost to determine whether the inter-frame prediction scheme is used for the current point having a coarse location.

6. The method of claim 5, wherein - When the cost of the first bit is higher than the cost of the second bit, the inter-frame prediction scheme is used for the current point with a coarse location, and - When the first bit cost is equal to or lower than the second bit cost, the inter-frame prediction scheme is not used for the current point with a coarse position.

7. The method of any of the preceding claims, wherein the inter-frame prediction scheme uses a sequence difference residual obtained by referring to the previous frame of the coarse representation of the point cloud.

8. The method of claim 7, wherein the order difference residual is obtained by using one or more order differences between one or more successive points in the previous frame and a point, wherein the point is the previous point in the previous frame having a coarse position, and wherein the one or more successive points include one or more points in the previous frame having a coarse position, the one or more points being located at a position corresponding to the coarse position of the current point or within a certain range from the coarse position of the current point.

9. The method of claim 8, wherein the previous point having a coarse position in the previous frame corresponds to the previous point having a coarse position in the current frame, and wherein the current point having a coarse position in the previous frame corresponds to the current point in the current frame.

10. The method of any one of claims 7 to 9, wherein the one or more order differences used to encode the current point in the current frame are obtained by: - Search in the previous frame for a point with a coarse position that corresponds to the current point with a coarse position in the current frame. - Obtain the order difference between the approximate position of the point in the previous frame and the approximate position of the point preceding the point in the previous frame, and - The order difference residual is obtained using the obtained order difference and the order difference between the coarse position of the current point and the coarse position of the previous point in the current frame.

11. The method of claim 10, wherein a coarse position with the same order index as the current point in the current frame is searched in the previous frame.

12. The method of claim 10, wherein in Point P is in the previous frame i The approximate position index in -1, and The current point P is in the current frame i The approximate positional index in the text.

13. The method of claim 10, wherein a coarse location of a point having a specific laser index and azimuth sampling angle is searched in the previous frame, wherein the specific laser index and azimuth sampling angle... - Corresponds to the laser index and azimuth sampling angle of the approximate position of the current point in the current frame, or - Within a specific range of the laser index and the azimuth sampling angle of the approximate position of the current point in the current frame.

14. The method of any one of claims 10 to 13, wherein the order difference residual is obtained by the following formula: in Order difference residuals The difference in the approximate position of the current point Pn in the current frame i The difference in the approximate position of point P in the previous frame i-1.

15. The method according to any one of claims 10 to 14, wherein, If there is no approximate position of a point in the previous frame that corresponds to the approximate position of the current point in the current frame, search for approximate positions of points located within a predefined range of locations corresponding to the approximate position of the current point in the current frame in the previous frame.

16. The method of claim 15, wherein the approximate position of a point is selected within a predefined range of the order index of the approximate position of the previous point in the current frame.

17. The method of claim 15 or 16, wherein Select the order index in the previous frame The order index of the approximate position of the current point in the current frame Points with approximate locations before or after, and The order difference is set to have the order index in the previous frame. The difference in order between the selected point and the points before or after the selected point in the previous frame.

18. The method according to any one of claims 10 to 14, wherein, When there is no approximate position of a point in the previous frame that corresponds to the approximate position of the current point in the current frame, the inter-frame prediction scheme is not used for the current point in the current frame, but the intra-frame prediction scheme is used to encode the order difference information between the current point and the previous point in the current frame.

19. The method of any one of claims 7 to 17, wherein the order difference residual is encoded using entropy coding, such as context-based adaptive binary arithmetic codec (CABAC).

20. The method of claim 19, wherein the order difference residual is encoded into a data stream.

21. The method of any of the preceding claims, wherein the intra-frame prediction scheme uses the order difference between the coarse position of the current point in the current frame and the coarse position of the previous point.

22. The method of claim 21, wherein the order difference is encoded using the entropy coding, such as context-based adaptive binary arithmetic codec (CABAC).

23. The method of claim 22, wherein the order difference is encoded into a data stream.

24. The method of any one of claims 19 to 23, wherein the intra-frame prediction scheme uses a first entropy encoder and the inter-frame prediction scheme uses a second entropy encoder, the first entropy encoder and the second entropy encoder being independent encoders that use separate context tables to model the statistical probabilities of the intra-frame prediction scheme and the inter-frame prediction scheme, respectively.

25. The method as claimed in any of the preceding claims, wherein a first mark is provided, the first mark indicating whether, for the one or more coarse points, - Encode the order difference information using the intra-frame prediction scheme, or - The order difference information is encoded using the inter-frame prediction scheme.

26. The method of claim 25, wherein the first flag is encoded into a data stream, such as a bit stream representing data of the point cloud.

27. The method of any of the preceding claims, wherein the intra-frame prediction scheme and the inter-frame prediction scheme are selectively used in response to specific signaling.

28. The method of claim 27, wherein the specific signaling includes a second flag indicating whether the intra-frame prediction scheme and the inter-frame prediction scheme are to be selectively used.

29. The method of claim 28, wherein the second flag is included in the geometric parameter set (GPS) or sequence parameter set (SPS) of the data stream, such as the L3C2 bitstream structure.

30. The method of any of the preceding claims, wherein the coarse representation for each original point in the point cloud is obtained by quantizing one or more original points using azimuth fundamental shift, and wherein the coarse position contains one or more original points.

31. A method for decoding a point cloud from a data stream, the method comprising: Decode signaling from the data stream, wherein the signaling indicates whether, for one or more points of the point cloud with a coarse representation, an intra-frame prediction scheme or an inter-frame prediction scheme is used to decode the order difference information between the coarse position of the current point and the coarse position of the previous point in the current frame of the point cloud.

32. The method of claim 31, wherein, When the signaling indicates the use of the inter-frame prediction scheme, the method includes: - Decode the order difference residual from the data stream. - The order difference between the coarse position of the current point and the coarse position of the previous point in the current frame is obtained using the order difference residual and the order difference between the coarse positions of the first point and the second point in the previous frame, wherein the coarse position of the first point corresponds to the coarse position of the current point and the coarse position of the second point corresponds to the coarse position of the previous point. - Use the obtained order difference and the order index of the coarse position of the previous point in the current frame to obtain the order index of the current point in the current frame, and - Construct the rough representation of the point cloud.

33. The method of claim 32, wherein the order difference is obtained as follows: + in The current frame i The order difference between the approximate position of the current point and the approximate position of the previous point, as described in the text. The previous frame i The difference in order between the approximate positions of the first and second points in -1, and Order difference residuals.

34. The method of claim 32 or 33, wherein the order index of the coarse position of the current point is obtained as follows: + in The current frame i The order index of the approximate position of the current point, as described in the text. The previous frame i The approximate index of the first point in -1. The current frame i The order difference between the approximate position of the current point and the approximate position of the previous point, as described in the text. The previous frame i The difference in order between the approximate positions of the first and second points in -1, and Order difference residuals.

35. The method according to any one of claims 31 to 34, wherein, When the signaling indicates the use of the intra-frame prediction scheme, the method includes: - Decode the order difference between the coarse position of the current point and the coarse position of the previous point in the current frame from the data stream. - Obtain the coarse index of the current point in the current frame using the decoding order difference and the order index of the coarse position of the previous point in the current frame, and - Construct the rough representation of the point cloud.

36. The method of claim 35, wherein the order index of the coarse position of the current point is obtained as follows: in The current frame i The order index of the approximate position of the current point, as described in the text. The previous frame i The approximate position index of the first point in -1, and The current frame i The difference in order between the approximate position of the current point and the approximate position of the previous point.

37. The method of any one of claims 32 to 36, wherein the method is based on the current point P. n The order of the approximate positions is used to obtain the laser index λ and the azimuth sampling angle Ф. c At the current point P n plane (Ф) c , λ n The rough representation is constructed in ) in The total number of laser beams.

38. The method of any one of claims 31 to 37, wherein the signaling includes a flag indicating whether, for the one or more points having a coarse representation, - To use the intra-frame prediction scheme to decode the order difference information, or - The inter-frame prediction scheme is to be used to decode the order difference information.

39. A non-transitory computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of the preceding claims.

40. An encoder, the encoder comprising: An encoder module, wherein the encoder module is configured to encode a point cloud using the method according to any one of claims 1 to 30.

41. An encoder, the encoder comprising: Encoder module, The encoder module is configured to encode point clouds into a data stream, and Wherein, for one or more points of the point cloud with a coarse representation, the encoder module is configured to selectively use an intra-frame prediction scheme or an inter-frame prediction scheme to encode the order difference information between the coarse position of the current point and the coarse position of the previous point in the current frame of the point cloud.

42. A decoder, the decoder comprising: A decoder module, wherein the decoder module is configured to decode a point cloud according to any one of claims 31 to 38.

43. A decoder, the decoder comprising: Decoder module, The decoder module is configured to decode the point cloud from the data stream, and Wherein, for one or more points of the point cloud having a coarse representation, the decoder module is configured to decode signaling from the data stream, wherein the signaling indicates whether an intra-frame prediction scheme or an inter-frame prediction scheme is used to decode the order difference information between the coarse position of the current point and the coarse position of the previous point in the current frame of the point cloud for one or more points of the point cloud having a coarse representation.

44. A data stream encoding a point cloud, wherein the data stream comprises: The signaling indicates whether, for one or more points of the point cloud with a coarse representation, an intra-frame prediction scheme or an inter-frame prediction scheme should be used to decode the order difference information between the coarse position of the current point and the coarse position of the previous point in the current frame of the point cloud.

Images