Encoding method, decoding method, encoder, decoder and storage medium

Abstract

Disclosed in the embodiments of the present application are an encoding method, a decoding method, an encoder, a decoder and a storage medium. The decoding method comprises: determining reconstructed geometric data of at least one decoded scale of the current point cloud; inputting the reconstructed geometric data of the at least one decoded scale into a neural network model, and outputting an estimation probability of an i-th scale of the current point cloud by means of the neural network model; and on the basis of the estimation probability of the i-th scale, decoding a geometric sub-bitstream of the i-th scale in a bitstream, so as to determine the reconstructed geometric data of the i-th scale of the current point cloud, wherein the value of i is greater than 1 and less than or equal to N, and N represents the number of geometric decoding scales of the current point cloud. In this way, the accuracy of an estimation probability of a neural network can be improved, thereby improving the compression performance of a LiDAR point cloud.

Description

Encoding / decoding methods, encoders, decoders, and storage media Technical Field

[0001] This application relates to the field of point cloud encoding and decoding technology, and in particular to an encoding and decoding method, encoder, decoder and storage medium. Background Technology

[0002] With the booming development of emerging technologies such as augmented reality, virtual reality, autonomous driving, and robotics, point cloud data has become one of the main data forms due to its concise representation of three-dimensional space. However, point cloud data is massive, and directly storing point cloud data consumes a lot of memory and is not conducive to transmission. Therefore, high-performance point cloud compression technology is essential.

[0003] In recent years, neural networks and deep learning technologies have been widely applied in the field of point cloud geometry compression. However, considering the sampling characteristics of LiDAR, LiDAR point clouds are very sparse in spatial distribution, making it difficult for neural networks to extract effective neighborhood context information. This results in inaccurate prediction probabilities from the neural networks, reducing the compression performance of LiDAR point clouds.

[0004] Summary of the Invention

[0005] This application provides an encoding / decoding method, encoder, decoder, and storage medium that can improve the accuracy of probability estimation by neural networks, thereby improving the compression performance of lidar point clouds.

[0006] The technical solution of this application embodiment can be implemented as follows:

[0007] In a first aspect, embodiments of this application provide a decoding method applied to a decoder, the method comprising:

[0008] Determine at least one decoded scale of reconstructed geometry for the current point cloud;

[0009] Input at least one decoded scale of reconstructed geometric data into a neural network model, and output the estimated probability of the i-th scale of the current point cloud through the neural network model;

[0010] The geometric sub-bitstream at the i-th scale is decoded based on the estimated probability at the i-th scale to determine the reconstructed geometric data at the i-th scale of the current point cloud; where i is greater than 1 and less than or equal to N, and N represents the number of geometric decoding scales of the current point cloud.

[0011] Secondly, embodiments of this application provide an encoding method applied to an encoder, the method comprising:

[0012] Determine the geometry of at least one encoded scale of the current point cloud;

[0013] Input geometric data of at least one encoded scale into a neural network model, and output the estimated probability of the i-th scale of the current point cloud through the neural network model;

[0014] The geometric data of the current point cloud at the i-th scale is encoded based on the estimated probability of the i-th scale to generate a geometric sub-bitstream at the i-th scale in the bitstream; where i is greater than 1 and less than or equal to N, and N represents the number of geometric encoding scales of the current point cloud.

[0015] Thirdly, embodiments of this application provide an encoder, which includes a first determining unit and an encoding unit, wherein:

[0016] The first determining unit is configured to determine geometric data of at least one encoded scale of the current point cloud; and to input the geometric data of at least one encoded scale into a neural network model, and output the estimated probability of the i-th scale of the current point cloud through the neural network model.

[0017] The encoding unit is configured to encode the geometric data of the current point cloud at the i-th scale based on the estimated probability of the i-th scale, and generate the geometric sub-bit stream at the i-th scale in the bit stream; where i takes the value greater than 1 and less than or equal to N, and N represents the number of geometric encoding scales of the current point cloud.

[0018] Fourthly, embodiments of this application provide an encoder, which includes a first memory and a first processor, wherein:

[0019] A first memory for storing computer programs that can run on a first processor;

[0020] A first processor is configured to execute the encoding method described in the second aspect when running a computer program.

[0021] Fifthly, embodiments of this application provide a decoder, which includes a second determining unit and a decoding unit, wherein:

[0022] The second determining unit is configured to determine at least one decoded scale of reconstructed geometric data of the current point cloud; and to input the reconstructed geometric data of at least one decoded scale into a neural network model, and output the estimated probability of the i-th scale of the current point cloud through the neural network model.

[0023] The decoding unit is configured to decode the geometric sub-bitstream at the i-th scale in the bitstream based on the estimated probability at the i-th scale, and determine the reconstructed geometric data at the i-th scale of the current point cloud; where i is greater than 1 and less than or equal to N, and N represents the number of geometric decoding scales of the current point cloud.

[0024] Sixthly, embodiments of this application provide a decoder, which includes a second memory and a second processor, wherein:

[0025] The second memory is used to store computer programs that can run on the second processor;

[0026] The second processor is used to execute the decoding method as described in the first aspect when running a computer program.

[0027] In a seventh aspect, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the decoding method as described in the first aspect or the encoding method as described in the second aspect.

[0028] Eighthly, embodiments of this application provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement the decoding method as described in the first aspect or the encoding method as described in the second aspect.

[0029] In a ninth aspect, embodiments of this application provide a computer-readable storage medium having a bitstream stored thereon, the bitstream being generated by performing the steps of the encoding method as described in the second aspect.

[0030] This application provides an encoding / decoding method, an encoder, a decoder, and a storage medium. At the encoding end, at least one encoded scale of geometric data of the current point cloud is determined; the at least one encoded scale of geometric data is input into a neural network model, and the neural network model outputs an estimated probability of the i-th scale of the current point cloud; the i-th scale of the geometric data of the current point cloud is encoded according to the estimated probability of the i-th scale to generate a geometric sub-bitstream of the i-th scale in the bitstream. At the decoding end, at least one decoded scale of reconstructed geometric data of the current point cloud is determined; the at least one decoded scale of reconstructed geometric data is input into a neural network model, and the neural network model outputs an estimated probability of the i-th scale of the current point cloud; the i-th scale of the geometric sub-bitstream in the bitstream is decoded according to the estimated probability of the i-th scale to determine the reconstructed geometric data of the i-th scale of the current point cloud. In this way, whether at the encoding or decoding end, geometric data from at least one previously encoded / decoded scale can be input into the neural network model to obtain the estimated probability of the current scale. Then, the estimated probability is used to encode and decode the geometric data at the current scale, and so on, from the lowest scale to the highest scale, thereby realizing the encoding and decoding of the current point cloud. This neural network model can improve the accuracy of the estimated probability, thereby improving the compression efficiency of the LiDAR point cloud and thus enhancing the encoding and decoding performance of the point cloud. Attached Figure Description

[0031] Figure 1A is a schematic diagram of a three-dimensional point cloud image;

[0032] Figure 1B is a magnified view of a portion of a three-dimensional point cloud image;

[0033] Figure 2A is a schematic diagram of six viewing angles for a point cloud image;

[0034] Figure 2B is a schematic diagram of a data storage format corresponding to a point cloud image;

[0035] Figure 3 is a schematic diagram of a network architecture for point cloud encoding and decoding provided in an embodiment of this application;

[0036] Figure 4 is a schematic diagram of the frame of an encoder provided in an embodiment of this application;

[0037] Figure 5 is a schematic diagram of the framework of a decoder provided in an embodiment of this application;

[0038] Figure 6 is a schematic flowchart of a decoding method provided in an embodiment of this application;

[0039] Figure 7 is a schematic diagram of the composition structure of a neural network model provided in an embodiment of this application;

[0040] Figure 8 is a schematic diagram of the composition structure of a neural network model provided in an embodiment of this application;

[0041] Figure 9 is a schematic flowchart of a decoding method provided in an embodiment of this application;

[0042] Figure 10 is a schematic diagram of the composition structure of a scale embedding module provided in an embodiment of this application;

[0043] Figure 11 is a schematic diagram of the composition structure of a feature enhancement module provided in an embodiment of this application;

[0044] Figure 12 is a schematic diagram of the composition structure of a dual-view conversion module provided in an embodiment of this application;

[0045] Figure 13 is a schematic diagram of the composition structure of a dual-view sparse attention module provided in an embodiment of this application;

[0046] Figure 14 is a schematic diagram of the composition structure of a neural network model provided in an embodiment of this application;

[0047] Figure 15 is a detailed flowchart of a decoding method provided in an embodiment of this application;

[0048] Figure 16 is a flowchart illustrating an encoding method provided in an embodiment of this application;

[0049] Figure 17 is a detailed flowchart of an encoding method provided in an embodiment of this application;

[0050] Figure 18 is a schematic diagram of the encoding principle of a lidar point cloud geometric compression according to an embodiment of this application;

[0051] Figure 19 is a schematic diagram of the decoding principle of a lidar point cloud geometric compression according to an embodiment of this application;

[0052] Figure 20 is a schematic diagram of the composition structure of an encoder provided in an embodiment of this application;

[0053] Figure 21 is a schematic diagram of the hardware structure of an encoder provided in an embodiment of this application;

[0054] Figure 22 is a schematic diagram of the composition structure of a decoder provided in an embodiment of this application;

[0055] Figure 23 is a schematic diagram of the hardware structure of a decoder provided in an embodiment of this application;

[0056] Figure 24 is a schematic diagram of the composition structure of an encoding and decoding system provided in an embodiment of this application. Detailed Implementation

[0057] In order to gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of this application.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0059] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0060] It should also be noted that the terms "first, second, and third" used in the embodiments of this application are only used to distinguish similar objects and do not represent a specific order of objects. It is understood that "first, second, and third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0061] Point cloud is a three-dimensional representation of an object's surface. It can be acquired using acquisition devices such as photoelectric radar, lidar, laser scanner, and multi-view camera.

[0062] A point cloud is a set of randomly distributed discrete points in three-dimensional space that represent the spatial structure and surface properties of a three-dimensional object or scene. These points contain geometric information representing spatial location and attribute information representing the texture of the point cloud. Figure 1A shows a three-dimensional point cloud image, and Figure 1B shows a magnified view of a portion of the three-dimensional point cloud image. It can be seen that the surface of the point cloud is composed of densely distributed points.

[0063] Two-dimensional images contain information at each pixel, and their distribution is regular, so there's no need to record their position information separately. However, the distribution of points in a point cloud in three-dimensional space is random and irregular, so it's necessary to record the position of each point in space to fully represent objects in three-dimensional space. Similar to two-dimensional images, each location during the acquisition process has corresponding attribute information, typically including color and reflectance information. Color information reflects the object's color and is usually represented by RGB; reflectance information reflects the object's surface material and is usually represented by reflectionance. Point cloud data typically consists of geometric information (x, y, z) representing three-dimensional spatial position information, and attribute information such as color information (r, g, b) and reflectance information. For example, reflectance information can be one-dimensional reflectance information (r); color information can be information in any color space, or it can be three-dimensional color information, such as RGB information. Here, R represents red (Red, R), G represents green (Green, G), and B represents blue (Blue, B). For example, color information can be luminance and chromaticity (YCbCr, YUV) information. Here, Y represents luminance (Luma), Cb(U) represents blue color difference, and Cr(V) represents red color difference.

[0064] For example, a point cloud obtained based on laser measurement principles may contain points whose three-dimensional coordinates and reflectance information are included. Similarly, a point cloud obtained based on photogrammetry principles may contain points whose three-dimensional coordinates and three-dimensional color information are included. Furthermore, a point cloud obtained by combining laser measurement and photogrammetry principles may contain points whose three-dimensional coordinates, reflectance information, and three-dimensional color information are included.

[0065] Figures 2A and 2B show a point cloud image and its corresponding data storage format. Figure 2A provides six viewing dimensions of the point cloud image, while Figure 2B consists of a file header and a data section. The header information includes the data format, data representation type, total number of points in the point cloud, and the content represented by the point cloud. For example, the point cloud file format is ".ply", represented by ASCII code, with a total of 207242 points. Each point has three-dimensional coordinate information (x, y, z) and three-dimensional color information (r, g, b).

[0066] Point clouds can be categorized according to the acquisition method:

[0067] Static point cloud: This means that the object is stationary and the device used to acquire the point cloud is also stationary.

[0068] Dynamic point cloud: The object is in motion, but the device acquiring the point cloud is stationary;

[0069] Dynamic point cloud acquisition: The device acquiring the point cloud is in motion.

[0070] For example, point clouds can be divided into two main categories based on their uses:

[0071] Category 1: Machine-perceived point cloud, which can be used in scenarios such as autonomous navigation systems, real-time inspection systems, geographic information systems, visual sorting robots, and disaster relief robots;

[0072] Category 2: Human eye-perceived point clouds, which can be used in point cloud application scenarios such as digital cultural heritage, free-viewpoint broadcasting, 3D immersive communication, and 3D immersive interaction.

[0073] Point cloud acquisition primarily utilizes the following methods: computer generation, 3D laser scanning, and 3D photogrammetry. Computers can generate point clouds of virtual 3D objects and scenes; 3D laser scanning can acquire point clouds of static real-world 3D objects or scenes, with a capacity of millions of point clouds per second; 3D photogrammetry can acquire point clouds of dynamic real-world 3D objects or scenes, with a capacity of tens of millions of point clouds per second. These point cloud acquisition technologies have reduced the cost and time required for point cloud data acquisition and improved data accuracy, further advancing the practical applications of point clouds. The continuous industrialization of point cloud data acquisition methods has made the acquisition of massive amounts of point cloud data possible. However, with the growth of application demands, the processing of massive amounts of 3D point cloud data has encountered bottlenecks in storage space and transmission bandwidth.

[0074] For example, taking a point cloud video with a frame rate of 30 frames per second (fps) as an example, each frame contains 700,000 points, and each point contains coordinate information xyz (float) and color information RGB (uchar). Therefore, the data size of a 10-second point cloud video is approximately 0.7 million × (4 bytes × 3 + 1 byte × 3) × 30 fps × 10 s = 3.15 GB, where 1 byte is 10 bits. Correspondingly, a 10-second 1280×720 two-dimensional video with a YUV sampling format of 4:2:0 and a frame rate of 30 fps has a data size of approximately 1280×720 × 12 bits × 30 fps × 10 s ≈ 0.39 GB, and a 10-second two-view three-dimensional video has a data size of approximately 0.39 × 2 = 0.78 GB. It is evident that the data size of a point cloud video far exceeds that of two-dimensional and three-dimensional videos of the same duration. Therefore, in order to better achieve data management, save server storage space, and reduce the transmission traffic and transmission time between the server and the client, point cloud compression has become a key issue in promoting the development of the point cloud industry.

[0075] In other words, point cloud is a type of 3D data, referring to a set of vectors in a 3D coordinate system. These vectors are typically represented by (x, y, z) 3D coordinates and can also represent information such as color, material, and reflection intensity. It is generally used as a representation of 3D objects or scenes. With the rapid development of emerging technologies such as augmented reality, virtual reality, autonomous driving, and robotics, point cloud data has become one of their primary data forms due to its concise representation of 3D space. However, point cloud data is massive, and directly storing it consumes a lot of memory and is inefficient for transmission. Currently, there is insufficient bandwidth to transmit point clouds directly at the network layer; therefore, high-performance point cloud compression technology is essential.

[0076] Among related technologies, the representative point cloud compression algorithms are two schemes developed by the Moving Picture Experts Group (MPEG): Video-based Point Cloud Compression (V-PCC) and Geometry-based Point Cloud Compression (G-PCC). G-PCC primarily uses octree models and / or triangular surface models for geometric compression, while V-PCC mainly achieves compression through 3D-to-2D projection and video compression.

[0077] In recent years, neural networks and deep learning technologies have been widely applied to point cloud geometry compression techniques. These can be categorized into volumetric model compression techniques based on 3D Convolutional Neural Networks (3D CNNs), compression techniques that directly apply point coordinate sets using neural networks based on Multi-Layer Perceptrons (MLPs), and compression techniques that use MLPs or 3D CNNs for probability estimation and entropy encoding of octree node symbols. These methods all demonstrate superior compression performance compared to traditional methods. However, when considering the specific type of input point cloud data, it becomes apparent that the performance gain of neural network-based point cloud compression techniques on sparse LiDAR point clouds is far less than its performance gain on dense object point clouds. This is because, due to the sampling characteristics of LiDAR, LiDAR point clouds are spatially very sparse, making it difficult for existing neural networks to extract effective neighborhood context information. This leads to inaccurate prediction probabilities from the neural network, thus reducing the compression performance of LiDAR point clouds.

[0078] Based on this, embodiments of this application provide an encoding method that determines at least one encoded scale of geometric data of the current point cloud; inputs the at least one encoded scale of geometric data into a neural network model, and outputs an estimated probability of the i-th scale of the current point cloud through the neural network model; and encodes the i-th scale of geometric data of the current point cloud according to the estimated probability of the i-th scale to generate a geometric sub-bitstream of the i-th scale in the bitstream. Embodiments of this application also provide a decoding method that determines at least one decoded scale of reconstructed geometric data of the current point cloud; inputs at least one decoded scale of reconstructed geometric data into a neural network model, and outputs an estimated probability of the i-th scale of the current point cloud through the neural network model; and decodes the i-th scale of geometric sub-bitstream in the bitstream according to the estimated probability of the i-th scale to determine the reconstructed geometric data of the i-th scale of the current point cloud; wherein, i is greater than 1 and less than or equal to N. In this way, whether at the encoding or decoding end, geometric data from at least one previously encoded / decoded scale can be input into the neural network model to obtain the estimated probability of the current scale. Then, the estimated probability is used to encode and decode the geometric data at the current scale, and so on, from the lowest scale to the highest scale, thereby realizing the encoding and decoding of the current point cloud. This neural network model can improve the accuracy of the estimated probability, thereby improving the compression efficiency of the LiDAR point cloud and thus enhancing the encoding and decoding performance of the point cloud.

[0079] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0080] In one embodiment of this application, a network architecture for a point cloud encoding / decoding system including a decoding method and an encoding method is provided. Figure 3 is a schematic diagram of a point cloud encoding / decoding network architecture provided in this embodiment. As shown in Figure 3, the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, wherein the electronic devices 13 to 1N can perform video interaction through the communication network 01. The electronic devices can be various types of devices with point cloud encoding / decoding capabilities, such as mobile phones, tablets, personal computers, personal digital assistants, navigators, digital phones, video phones, televisions, sensing devices, servers, etc., and this embodiment does not impose any limitations. The decoder or encoder in this embodiment can be one of the aforementioned electronic devices.

[0081] The electronic device in this application embodiment has point cloud encoding and decoding functions, and generally may include a point cloud encoder (i.e., encoder) and a point cloud decoder (i.e. decoder).

[0082] Figure 4 is a schematic diagram of the encoder framework provided in an embodiment of this application. As shown in Figure 4, the encoder framework includes a data partitioning module 401, a high-level syntax module 402, a geometric data module 403, an intra-frame geometric coding module 404, an intra-frame geometric decoding module 405, an inter-frame geometric coding module 406, an inter-frame geometric decoding module 407, and a data multiplexing module 408.

[0083] As shown in Figure 4, for the input point cloud data (x,y,z,r,g,b) / (x,y,z,r), the data is first divided by the data separation module 401. It can be divided into syntax elements / metadata using High Level Synthesis (HLS) and geometric data, which are then input into the high level syntax module 402 and the geometric data module 403, respectively. The geometric data mainly refers to the geometric coordinate information (x,y,z). For the high-level syntax module 402, the high-level syntax elements are encoded, and the resulting encoded bits are input to the data multiplexing module 408. For geometric data, if intra-frame encoding is performed by the intra-frame geometry coding module 404, the geometric data can be intra-frame encoded, and the resulting encoded bits are input to the data multiplexing module 408. At the same time, intra-frame decoding is also required by the intra-frame geometry decoding module 405 to obtain the intra-frame reconstructed geometry. Then, if inter-frame encoding is performed by the inter-frame geometry coding module 406, the geometric data can be inter-frame encoded, and the resulting encoded bits are input to the data multiplexing module 408. At the same time, intra-frame decoding is also required by the inter-frame geometry decoding module 407 to obtain the inter-frame reconstructed geometry. The data multiplexing module 408 can generate a bitstream, such as the geometric bitstream of the input point cloud.

[0084] Figure 5 is a schematic diagram of the framework of a decoder provided in an embodiment of this application. As shown in Figure 5, the decoder framework includes a bitstream demultiplexing module 501, a high-level syntax module 502, a geometric bitstream module 503, an intra-frame geometric decoding module 504, an inter-frame geometric decoding module 505, and a geometric reconstruction module 506.

[0085] As shown in Figure 5, for an encoded point cloud bitstream, such as 0b00101…, the bitstream is first separated by the bitstream demultiplexing module 501 to obtain a geometric bitstream and a high-level syntax bitstream. The high-level syntax bitstream is then decoded by the high-level syntax module 502 to obtain HLS syntax elements / metadata. These HLS syntax elements can then guide the geometric bitstream module 503 in decoding the geometric bitstream. During the geometric bitstream decoding process, if intra-frame decoding is performed by the intra-frame geometric decoding module 504, the reconstructed geometric point cloud within the intra-frame can be output by the geometric reconstruction module 506. If inter-frame decoding is performed by the inter-frame geometric decoding module 505, the reconstructed geometric point cloud between frames can be output by the geometric reconstruction module 506, thus achieving geometric reconstruction of the input point cloud data.

[0086] In another embodiment of this application, Figure 6 is a schematic flowchart of a decoding method provided in an embodiment of this application. As shown in Figure 6, the method may include:

[0087] S601, determine at least one decoded scale of reconstructed geometry data for the current point cloud.

[0088] It should be noted that the decoding method in this application embodiment is applied to a decoder, and this decoding method can refer to a LiDAR point cloud compression method based on a neural network. Here, considering the sparsity of LiDAR point clouds and the scanning characteristics of LiDAR, the existing neural network structure is improved, thereby improving the accuracy of the neural network's probability estimation and thus improving the compression performance of LiDAR point clouds.

[0089] It should also be noted that, in the embodiments of this application, the input to the neural network model involves reconstructed geometric data at at least one decoded scale of the current point cloud. Here, reconstructed geometric data mainly refers to geometric information at the decoded scale. As the input to the neural network model, this geometric information can refer to geometric position information (or "geometric coordinate information").

[0090] S602, input at least one decoded scale of reconstructed geometric data into the neural network model, and output the estimated probability of the i-th scale of the current point cloud through the neural network model.

[0091] It should be noted that, in the embodiments of this application, at least one decoded scale can refer to decoded scales prior to the i-th scale. Furthermore, inputting the reconstructed geometric data of at least one decoded scale into the neural network model can mean inputting the geometric position information of this at least one decoded scale into the neural network model. Through feature extraction and feature concatenation within the neural network model, an estimated probability of the geometric occupancy information at the i-th scale can be output. Here, i is a positive integer, and the value of i is greater than 1 and less than or equal to N, where N represents the number of geometric decoding scales of the current point cloud.

[0092] It should also be noted that, in the embodiments of this application, for reconstructed geometric data of at least one decoded scale, the number of decoded scales input to the neural network model is related to the value of i, which will be described in detail below in several cases.

[0093] In one possible implementation, the value of i is equal to 2, in which case it can be determined that at least one decoded scale of reconstructed geometry data includes: reconstructed geometry data of the first scale.

[0094] In other words, in this embodiment of the application, if the value of i is equal to 2, then the reconstructed geometric data of the first scale can be input into the neural network model, the neural network model can output the estimated probability of the second scale of the current point cloud, and then the estimated probability can be used to decode and obtain the reconstructed geometric data of the second scale.

[0095] In another possible implementation, i is equal to 3, in which case it can be determined that at least one decoded scale of reconstructed geometry data includes: reconstructed geometry data at scale 1 and reconstructed geometry data at scale 2.

[0096] In other words, in this embodiment of the application, if the value of i is equal to 3, then the reconstructed geometric data of the first scale and the reconstructed geometric data of the second scale can be input into the neural network model, the neural network model outputs the estimated probability of the third scale of the current point cloud, and then the estimated probability is used to decode and obtain the reconstructed geometric data of the third scale.

[0097] In another possible implementation, the value of i is greater than 3. In this case, it can be determined that the reconstructed geometric data at at least one decoded scale includes: the reconstructed geometric data at the (i-3)th scale, the reconstructed geometric data at the (i-2)th scale, and the reconstructed geometric data at the (i-1)th scale.

[0098] In other words, in the embodiments of this application, if the value of i is greater than 3, for example, if the value of i is equal to 4, 5, 6, 7, etc., then the reconstructed geometric data of the (i-3)th scale, the reconstructed geometric data of the (i-2)th scale, and the reconstructed geometric data of the (i-1)th scale can be input into the neural network model, the estimated probability of the current point cloud at the i-th scale can be output by the neural network model, and then the reconstructed geometric data at the i-th scale can be obtained by decoding using the estimated probability.

[0099] For example, if the value of i is greater than 3, the reconstructed geometric position information of the three adjacent scales (e.g., scale i-1, scale i-2, and scale i-3) that have been decoded before the i-th scale can be obtained. Then, the reconstructed geometric position information of these three adjacent scales (e.g., scale i-1, scale i-2, and scale i-3) is input into the neural network model, which can output the estimated probability of the geometric occupancy information of the i-th scale. Then, the reconstructed geometric data (specifically, the reconstructed geometric occupancy information) of the i-th scale can be decoded using this estimated probability.

[0100] Understandably, in the embodiments of this application, the value of i is not infinitely large; for example, i ≤ N, where N represents the geometric decoding scale number of the current point cloud. In some embodiments, the method may further include: parsing the geometric decoding scale number of the current point cloud in the bitstream.

[0101] It should be noted that, in this embodiment of the application, for the current point cloud, step-by-step downsampling at the encoding end can generate multi-scale sparse tensors, where each scale of sparse tensor can be regarded as a layer. Therefore, the number of geometric decoding scales can also be called the "number of geometric decoding layers". For the decoding end, the number of geometric decoding scales can be determined by parsing the geometric bitstream.

[0102] It is also understood that, in the embodiments of this application, the decoding end may also have one or more candidate neural network models.

[0103] In one possible implementation, if the decoding end includes only one candidate neural network model, then that candidate neural network model can be determined as the neural network model used by the current point cloud.

[0104] In another possible implementation, if the decoding end includes multiple candidate neural network models, then in some embodiments the method may further include: parsing model index parameters in the bitstream; and determining the neural network model based on the model index parameters.

[0105] It should be noted that, in the embodiments of this application, the model index parameters in the bitstream are first parsed, and then the neural network model used by the current point cloud can be determined by combining multiple candidate neural network models at the decoding end.

[0106] In another possible implementation, if the decoding end includes multiple candidate neural network models, then to improve performance, in practical applications, different neural network models are typically further divided for different scale ranges. For example, one neural network model is used for high-scale geometric decoding, and another neural network model is used for low-scale geometric decoding. In some embodiments, the method may further include: parsing a first model index parameter and a second model index parameter in the bitstream; determining a first neural network model applied to the low-scale range based on the first model index parameter; and determining a second neural network model applied to the high-scale range based on the second model index parameter. In this way, different neural network models can be used for different scale ranges, thereby achieving better compression performance.

[0107] Taking the current point cloud as an example, which uses the same neural network model, the structure of the neural network model will be explained below.

[0108] In one possible implementation, Figure 7 is a schematic diagram of the composition structure of a neural network model provided in an embodiment of this application. As shown in Figure 7, the neural network model may include a scale embedding module 701, at least one feature enhancement module 702, at least one dual-view transformation module 703, a transposed convolution module 704, and a classification module 705. Specifically, for reconstructed geometric data of at least one decoded scale input to the neural network model, the data can sequentially pass through the scale embedding module 701, at least one feature enhancement module 702, at least one dual-view transformation module 703, transposed convolution module 704, and classification module 705, ultimately outputting the estimated probability of the i-th scale.

[0109] In some embodiments, based on the neural network model shown in Figure 7, inputting reconstructed geometric data at at least one decoded scale into the neural network model, and outputting the estimated probability of the i-th scale of the current point cloud through the neural network model, may include:

[0110] The scale embedding module 701 performs feature extraction and feature concatenation on the reconstructed geometric data of at least one decoded scale, and outputs a first sparse tensor, which includes a first geometric component and a first feature component.

[0111] The first sparse tensor is enhanced with positional features by at least one feature enhancement module 702, and a second sparse tensor is output, the second sparse tensor including a first geometric component and a second feature component.

[0112] The second sparse tensor is subjected to coordinate system transformation and feature splicing by at least one dual-view transformation module 703, and a third sparse tensor is output, which includes a first geometric component and a third feature component.

[0113] The third sparse tensor is transposed and convolved by the transposed convolution module 704 to output the fourth sparse tensor, which includes the second geometric component and the fourth feature component corresponding to the i-th scale.

[0114] The classification module 705 performs a high-dimensional feature transformation on the fourth sparse tensor, and outputs a one-dimensional estimated probability of the i-th scale.

[0115] In another possible implementation, at least one feature enhancement module 702 may include a first feature enhancement module and a second feature enhancement module, and at least one dual-view transformation module 703 may include a first dual-view transformation module and a second dual-view transformation module.

[0116] Figure 8 is a schematic diagram of the composition structure of a neural network model provided in an embodiment of this application. As shown in Figure 8, the neural network model may include a scale embedding module 701, a first feature enhancement module 702-1, a first dual-view transformation module 703-1, a second feature enhancement module 702-2, a transposed convolution module 704, a second dual-view transformation module 703-2, and a classification module 705. Specifically, for reconstructed geometric data of at least one decoded scale input to the neural network model, the data can sequentially pass through the scale embedding module 701, the first feature enhancement module 702-1, the first dual-view transformation module 703-1, the second feature enhancement module 702-2, the transposed convolution module 704, the second dual-view transformation module 703-2, and the classification module 705, ultimately outputting the estimated probability of the i-th scale.

[0117] In some embodiments, based on the neural network model shown in Figure 8, at least one decoded scale of reconstructed geometric data is input into the neural network model, and the estimated probability of the i-th scale of the current point cloud is output through the neural network model. Referring to Figure 9, the method may include:

[0118] S901, the scale embedding module 701 performs feature extraction and feature splicing on the reconstructed geometric data of at least one decoded scale, and outputs a first sparse tensor, which includes a first geometric component and a first feature component.

[0119] S902, the first sparse tensor is enhanced by the first feature enhancement module 702-1 to perform position feature enhancement, and the second sparse tensor is output. The second sparse tensor includes a first geometric component and a second feature component.

[0120] S903 performs coordinate system transformation and feature splicing on the second sparse tensor through the first dual-view transformation module 703-1, and outputs a third sparse tensor, which includes a first geometric component and a third feature component.

[0121] S904, the third sparse tensor is enhanced with positional features by the second feature enhancement module 702-2, and the fifth sparse tensor is output. The fifth sparse tensor includes the first geometric component and the fifth feature component.

[0122] S905, the fifth sparse tensor is transposed and convolved by the transposed convolution module 704 to output the sixth sparse tensor, which includes the second geometric component and the sixth feature component corresponding to the i-th scale.

[0123] S906 performs coordinate system transformation and feature splicing on the sixth sparse tensor through the second dual-view transformation module 703-2, and outputs the seventh sparse tensor, which includes the second geometric component and the seventh feature component.

[0124] S907 performs high-dimensional feature transformation on the seventh sparse tensor through classification module 705, and outputs a one-dimensional estimated probability of the i-th scale.

[0125] It should be noted that during the processing of the neural network model, the operations of modules such as the scale embedding module 701, the first feature enhancement module 702-1, the first dual-view transformation module 703-1, and the second feature enhancement module 702-2 only process the feature components, while the geometric components remain unchanged. After the operation of the transpose convolution module 704, the first geometric component of the decoded scale can be converted into the second geometric component of the current scale (i.e., the i-th scale), and the feature components will also change accordingly. Then, the operation of the second dual-view transformation module 703-2 still only processes the changed feature components, while the second geometric component remains unchanged. Finally, the estimated probability of the current scale is output through the classification module 705 (also called the "classifier" or "classifier module").

[0126] It should also be noted that, in the embodiments of this application, the scale embedding module 701 can be called the ancestor sparse tensor embedding module, which is used to extract and concatenate features from the decoded ancestor sparse tensor from a lower scale, and finally output the first sparse tensor. The first sparse tensor may include a first geometric component (represented by C) and a first feature component (represented by F).

[0127] In one possible implementation, Figure 10 is a schematic diagram of the composition structure of a scale embedding module provided in an embodiment of this application. As shown in Figure 10, the scale embedding module 701 may include a first residual network module 1001, a first transposed convolutional layer 1002, a first feature splicing layer 1003, a second residual network module 1004, a second transposed convolutional layer 1005, and a second feature splicing layer 1006.

[0128] In some embodiments, when the value of i is equal to 2, feature extraction and feature splicing are performed on the reconstructed geometric data of at least one decoded scale by the scale embedding module 701 to output a first sparse tensor, which may include: directly determining the first geometric component and the first feature component in the first sparse tensor based on the reconstructed geometric data of the first scale.

[0129] It should be noted that in this embodiment of the application, when the value of i is equal to 2, the sub-modules in the scale embedding module 701 are not enabled. The first geometric component and the first feature component in the first sparse tensor can be determined directly based on the first scale reconstructed geometric data after the first scale is input into the scale embedding module.

[0130] In some embodiments, when the value of i is equal to 3, feature extraction and feature concatenation are performed on the reconstructed geometric data of at least one decoded scale through the scale embedding module 701 to output a first sparse tensor. This may include: performing feature extraction and feature concatenation on the reconstructed geometric data of the first scale and the reconstructed geometric data of the second scale through the second residual network module, the second transposed convolutional layer and the second feature concatenation layer in the scale embedding module to output a first geometric component and a first feature component in the first sparse tensor.

[0131] It should be noted that in this embodiment, when the value of i is equal to 3, some sub-modules in the scale embedding module 701 are not enabled, for example, only the second residual network module 1004, the second transposed convolutional layer 1005, and the second feature concatenation layer 1006 are enabled. In this way, the reconstructed geometric data of the first scale is input into the second residual network module 1004 for feature extraction, and the corresponding output features are mapped to the coordinates of the second scale through the second transposed convolutional layer 1005; then, the second feature concatenation layer 1006 is used to concatenate the mapped output features with the reconstructed geometric data of the second scale, and finally outputs a first sparse tensor with ancestor embedding, wherein the first sparse tensor includes a first geometric component C and a first feature component F.

[0132] In some embodiments, when the value of i is greater than 3, feature extraction and feature concatenation are performed on the reconstructed geometric data of at least one decoded scale by the scale embedding module 701 to output a first sparse tensor. This may include: performing feature extraction and feature concatenation on the reconstructed geometric data of the (i-3)th scale, the reconstructed geometric data of the (i-2)th scale, and the reconstructed geometric data of the (i-1)th scale by the scale embedding module 701 to output a first geometric component and a first feature component in the first sparse tensor.

[0133] It should be noted that in this embodiment, when the value of i is greater than 3, all sub-modules in the scale embedding module 701 are enabled. At this time, the reconstructed geometric data at the (i-3)th scale is the input of the first residual network module 1001, and the output of the first residual network module 1001 is the input of the first transposed convolutional layer 1002. The input of the first feature splicing layer 1003 includes the output of the first transposed convolutional layer 1002 and the reconstructed geometric data at the (i-2)th scale. The output of the first feature splicing layer 1003 is the input of the second residual network module 1004, and the output of the second residual network module 1004 is the input of the second transposed convolutional layer 1005. The input of the second feature splicing layer 1006 includes the output of the second transposed convolutional layer 1005 and the reconstructed geometric data at the (i-1)th scale. The output of the second feature splicing layer 1006 is the first feature component F in the first sparse tensor.

[0134] Accordingly, in some embodiments, based on the scale embedding module shown in FIG10, feature extraction and feature concatenation are performed on the reconstructed geometric data at the (i-3)th scale, the (i-2)th scale, and the (i-1)th scale through the scale embedding module 701, and the first geometric component and the first feature component in the first sparse tensor are output, which may include:

[0135] The first residual network module 1001 extracts features from the reconstructed geometric data at the (i-3)th scale to determine the first output feature, and the first transposed convolutional layer 1002 performs scale mapping on the first output feature to determine the second output feature at the (i-2)th scale.

[0136] The first feature stitching layer 1003 stitches the reconstructed geometric data at the (i-2)th scale with the second output feature to determine the third output feature;

[0137] The second residual network module 1004 extracts features from the third output feature to determine the fourth output feature, and the second transposed convolutional layer 1005 performs scale mapping on the fourth output feature to determine the fifth output feature at the (i-1)th scale.

[0138] The second feature splicing layer 1006 splices the reconstructed geometric data at the (i-1)th scale with the fifth output feature to determine the first feature component in the first sparse tensor, and the first geometric component in the first sparse tensor is the geometric component corresponding to the (i-1)th scale.

[0139] It should also be noted that the scale embedding module (i.e., the ancestor sparse tensor embedding module) includes, in sequence: a ResNet module (i.e., the first residual network module 1001), a transposed convolutional layer (i.e., the first transposed convolutional layer 1002), a feature concatenation layer (i.e., the first feature concatenation layer 1003), a ResNet module (i.e., the second residual network module 1004), a transposed convolutional layer (i.e., the second transposed convolutional layer 1005), and a feature concatenation layer (i.e., the second feature concatenation layer 1006).

[0140] In other words, in this embodiment of the application, the sparse tensor reconstruction geometry data from the decoded ancestor at a lower scale is first determined, such as the reconstruction geometry data at the (i-3)th scale (which can be scaled). i-3 Reconstructed geometric data at the (i-2)th scale (which can be represented by Scale) i-2 (represented) and the reconstructed geometric data at the (i-1)th scale (which can be represented by Scale) i-1 (representation); then Scale i-3The first output features are extracted by inputting them into the first residual network module 1001 and then mapped to the scale layer by the first transposed convolutional layer 1002. i-2 The coordinates; then the second output feature is compared with the scale. i-2 The reconstructed geometric data are stitched together and sent again to the next stacked second residual network module 1004 and second transposed convolutional layer 1005, where the obtained fifth output feature is compared with the scale. i-1 After the reconstructed geometric data is stitched together, the final output is a first sparse tensor with ancestor embedding, which may include a first geometric component C and a first feature component F.

[0141] Additionally, it should be noted that the reconstructed geometric data used in the neural network model in this application embodiment can refer to reconstructed geometric information, such as geometric position information (or "geometric coordinate information").

[0142] In one possible implementation, Figure 11 is a schematic diagram of the composition structure of a feature enhancement module provided in an embodiment of this application. As shown in Figure 11, each feature enhancement module may include a multilayer perceptron module 1101 and a third feature stitching layer 1102.

[0143] Accordingly, in some embodiments, the first sparse tensor is augmented with positional features by at least one feature enhancement module to output the second sparse tensor, which may include:

[0144] The first geometric component in the first sparse tensor is transformed by the multilayer perceptron module 1101 to determine the sixth output feature.

[0145] The sixth output feature is concatenated with the first feature component in the first sparse tensor by the third feature concatenation layer 1102 to determine the second feature component in the second sparse tensor, while the first geometric component in the second sparse tensor remains unchanged.

[0146] It should be noted that, in the embodiments of this application, the feature enhancement module here may refer to the location feature enhancement module. A multilayer perceptron (MLP) is a feedforward artificial neural network model that maps multiple input datasets to a single output dataset; for example, the first geometric component in the first sparse tensor is input into the multilayer perceptron module 1101 to generate a sixth output feature of a high-dimensional location.

[0147] For example, a feature enhancement module (such as a first feature enhancement module or a second feature enhancement module) may sequentially include: an MLP layer (i.e., a multilayer perceptron module 1101) and a feature concatenation layer (i.e., a third feature concatenation layer 1102). That is, taking the first feature enhancement module 702-1 in Figure 8 as an example, the first geometric component C in the first sparse tensor may be input into the multilayer perceptron module 1101 to generate a sixth output feature at a high-dimensional position, and then concatenated with the first feature component F in the first sparse tensor to obtain the second feature component F0 of the second sparse tensor. The geometric component remains unchanged, that is, the output second sparse tensor includes: the first geometric component C and the second feature component F0.

[0148] In one possible implementation, Figure 12 is a schematic diagram of the composition structure of a dual-view transformation module provided in an embodiment of this application. As shown in Figure 12, each dual-view transformation module may include a dual-view sparse attention module 1201, a third residual network module 1202, and a neighbor point attention module 1203.

[0149] Accordingly, in some embodiments, the third sparse tensor is output by performing coordinate system transformation and feature concatenation on the second sparse tensor through at least one dual-view transformation module, which may include:

[0150] The second sparse tensor is subjected to coordinate system transformation and feature splicing by the dual-view sparse attention module 1201 to determine the seventh output feature;

[0151] The seventh output feature is extracted by the third residual network module 1202 to determine the eighth output feature;

[0152] The eighth output feature is extracted by the neighbor point attention module 1203 to determine the third feature component in the third sparse tensor, while the first geometric component in the third sparse tensor remains unchanged.

[0153] It should be noted that, in the embodiments of this application, the dual-view transformation module here may refer to the dual-view sparse transformer module, which is used to realize feature transformation and feature stitching under dual-view views in Cartesian coordinate system and spherical coordinate system.

[0154] For example, a dual-view transformation module (e.g., a first dual-view transformation module or a second dual-view transformation module) sequentially includes: a dual-view sparse attention layer (i.e., dual-view sparse attention module 1201), a ResNet module (i.e., a third residual network module 1202), and a neighbor point attention layer (i.e., neighbor point attention module 1203). That is, taking the first dual-view transformation module 703-1 in Figure 8 as an example, firstly, the second sparse tensor is subjected to coordinate system transformation and feature concatenation by the dual-view sparse attention module 1201 to obtain the seventh output feature; then, the seventh output feature is input into the third residual network module 1202 for feature extraction to obtain the eighth output feature; then, the eighth output feature is input into the neighbor point attention module 1203 for feature extraction to obtain the third feature component F3 in the third sparse tensor, where the geometric component remains unchanged, that is, the output third sparse tensor includes: the first geometric component C and the third feature component F3.

[0155] In one possible implementation, Figure 13 is a schematic diagram of the composition structure of a dual-view sparse attention module provided in an embodiment of this application. As shown in Figure 13, the dual-view sparse attention module may include a coordinate system transformation module 1301, a first-view attention layer 1302, a second-view attention layer 1303, and a fourth feature splicing layer 1304.

[0156] Accordingly, the second sparse tensor is subjected to coordinate system transformation and feature concatenation through a dual-view sparse attention module to determine the seventh output feature, which may include:

[0157] The coordinate system transformation module 1301 performs coordinate system transformation on the first geometric component in the first coordinate system to determine the first geometric component in the second coordinate system; and based on the first geometric component in the first coordinate system and the first geometric component in the second coordinate system, the second feature component in the second sparse tensor is split into a first view feature component and a second view feature component.

[0158] The first view feature component is extracted by the first view attention layer 1302 to determine the ninth output feature, and the second view feature component is extracted by the second view attention layer 1303 to determine the tenth output feature.

[0159] The ninth and tenth output features are concatenated by the fourth feature concatenation layer 1304 to determine the seventh output feature.

[0160] It should be noted that in the embodiments of this application, the first coordinate system can be a Cartesian coordinate system, and the second coordinate system can be a spherical coordinate system. For example, assuming the Cartesian coordinate system of the first geometric component is C1 = {x, y, z}, and it is transformed into a spherical coordinate system C2 = {ρ, θ, φ}, the calculation formula for the coordinate system transformation is as follows:

[0161] Then, the first geometric components in the Cartesian coordinate system and the spherical coordinate system can be divided into non-overlapping windows. Taking the spherical coordinate system as an example, the window is represented as follows:

[0162] Where Δρ, Δθ, and Δφ are the window sizes for the three axes. This indicates the modulo operation.

[0163] Here, the second eigencomponent F in the second sparse tensor o ∈R N×d Decomposed into spherical view feature components and Cartesian view feature components For the spherical view portion, i.e., the aforementioned first view feature component F1, it is linearly transformed into query, key, and value vectors Q1, K1, and V1. Then, using the first-view attention layer 1302, the attention is calculated as follows:

[0164] Similarly, for the Cartesian view portion, i.e., the aforementioned second-view feature component F2, it is linearly transformed into query, key, and value vectors Q2, K2, and V2. Then, using the second-view attention layer 1303, the attention is calculated as follows:

[0165] It should be noted that the first view feature component F1 and the second view feature component F2 are calculated in parallel using the same process described above. Finally, the output features of these two Attn(·) are concatenated together by the fourth feature concatenation layer 1304 to form the final output feature F4 of the dual-view sparse attention module. During this process, the first geometric component remains unchanged.

[0166] It is also worth noting that compared with the single-view perspective in related technologies, the dual-view perspective can contain more information. The Cartesian coordinate system and the spherical coordinate system can correspond to different perspectives. Therefore, more information can be obtained by using the dual-view perspective and input into the neural network model, which makes it easier for the neural network model to reason, and thus helps the neural network model to output a better estimated probability.

[0167] It is also understood that, in the embodiments of this application, for the neural network model, the reconstructed geometric data input to the neural network model is the geometric information of the point cloud, such as geometric position information or geometric coordinate information; the output of the neural network model is the estimated probability of geometric occupancy information. Therefore, the neural network model may also include a geometric code conversion module.

[0168] In one possible implementation, based on the neural network model shown in Figure 8, Figure 14 is a schematic diagram of the composition structure of a neural network model provided in an embodiment of this application. As shown in Figure 14, the neural network model may include a geometric code conversion module 1401, a scale embedding module 701, a first feature enhancement module 702-1, a first dual-view transformation module 703-1, a second feature enhancement module 702-2, a transposed convolution module 704, a second dual-view transformation module 703-2, and a classification module 705.

[0169] In this embodiment, for reconstructed geometric data at at least one decoded scale input to the neural network model, the data can first pass through a geometric code conversion module 1401 to convert the reconstructed geometric information expressed by the point cloud from geometric coordinates into occupancy codes, i.e., geometric occupancy information. Then, the data sequentially passes through a scale embedding module 701, a first feature enhancement module 702-1, a first dual-view transformation module 703-1, a second feature enhancement module 702-2, a transposed convolution module 704, a second dual-view transformation module 703-2, and a classification module 705 to finally output the estimated probability of the geometric occupancy information at the i-th scale.

[0170] In other words, in this embodiment, at least one decoded scale of reconstructed geometric data is input into a neural network model. First, the geometric code conversion module 1401 converts the reconstructed geometric information expressed in the point cloud into geometric occupancy information. Then, the scale embedding module 701 performs feature extraction and feature concatenation to output a first sparse tensor, which includes a first geometric component and a first feature component. Next, the first feature enhancement module 702-1 enhances the positional features of the first sparse tensor to output a second sparse tensor, which includes a first geometric component and a second feature component. Finally, the first dual-view transformation module 703-1 performs coordinate system transformation and feature concatenation on the second sparse tensor to output a third sparse tensor, which includes a first geometric component and a second feature component. The first sparse tensor consists of a geometric component and a third feature component. Then, the second feature enhancement module 702-2 enhances the positional features of the third sparse tensor, outputting a fifth sparse tensor, which includes the first geometric component and the fifth feature component. Next, the transpose convolution module 704 performs transpose convolution on the fifth sparse tensor, outputting a sixth sparse tensor, which includes the second geometric component and the sixth feature component corresponding to the i-th scale. Then, the second dual-view transformation module 703-2 performs coordinate system transformation and feature concatenation on the sixth sparse tensor, outputting a seventh sparse tensor, which includes the second geometric component and the seventh feature component. Finally, the classification module 705 performs high-dimensional feature transformation on the seventh sparse tensor, outputting a one-dimensional estimated probability for the i-th scale.

[0171] S603, decode the geometric sub-bitstream at the i-th scale in the bitstream according to the estimated probability at the i-th scale, and determine the reconstructed geometric data at the i-th scale of the current point cloud.

[0172] In this embodiment, i is a positive integer. For each scale, there exists a corresponding geometric sub-bitstream in the bitstream. For example, for the i-th scale, the bitstream includes a geometric sub-bitstream of the i-th scale. Here, the value of i is greater than 1 and less than or equal to N, where N represents the geometric decoding scale of the current point cloud.

[0173] Thus, after obtaining the estimated probability at scale i, the geometric sub-bitstream at scale i in the bitstream can be decoded based on the estimated probability at scale i to determine the reconstructed geometric data at scale i of the current point cloud. It should be noted that the reconstructed geometric data obtained by decoding here can refer to geometric occupancy information, used to characterize whether a point in the current scale is occupied.

[0174] In some embodiments, when the value of i is equal to 1, the method may further include: decoding the geometric sub-bitstream of the first scale in the bitstream to determine the reconstructed geometric data of the first scale.

[0175] In other words, in this embodiment of the application, for the first scale, the geometric sub-bitstream of the first scale can be directly parsed in the bitstream to obtain the reconstructed geometric data of the first scale.

[0176] In some embodiments, when the value of i is equal to N, the method may further include: determining the reconstructed geometric data at the Nth scale; and determining the reconstructed geometric data at the Nth scale as the geometrically reconstructed point cloud of the current point cloud.

[0177] In other words, in this embodiment of the application, as i increases, decoding is performed from low scale to high scale. When decoding reaches the highest scale, the reconstructed geometric data of the Nth scale can be obtained, which is the geometric reconstructed point cloud of the current point cloud.

[0178] For example, Figure 15 is a detailed flowchart of a decoding method provided in an embodiment of this application. As shown in Figure 15, decoding begins from the lowest scale, i.e., the first scale, and continues until the highest scale is reached. Specifically, the geometric sub-bitstream of the first scale is first decoded... 1 Decoding is performed to obtain the reconstructed geometric data at the first scale; then, the reconstructed geometric data at the first scale is input into a neural network model to obtain the estimated probability at the second scale; this estimated probability is then used to refine the geometric sub-bitstream at the second scale. 2 Decoding is performed to obtain the reconstructed geometric data at the second scale; similarly, the estimated probabilities based on the neural network model are used to generate the geometric sub-bitstream at the i-th scale. i Decoding is performed to obtain the reconstructed geometric data at scale i. The input to the neural network model consists of the reconstructed geometric data at scales i-3, i-2, and i-1, and the output is the estimated probability of the geometric occupancy information at scale i. This process continues until i equals N, where the estimated probability from the neural network model is used to determine the geometric sub-bitstream at scale N. N Decoding is performed, and the reconstructed geometric data at the Nth scale obtained at this time is the geometrically reconstructed point cloud of the current point cloud.

[0179] This embodiment provides a decoding method that determines reconstructed geometric data at at least one decoded scale of the current point cloud; inputs the reconstructed geometric data at at least one decoded scale into a neural network model, and outputs an estimated probability of the i-th scale of the current point cloud through the neural network model; decodes the geometric sub-bitstream of the i-th scale in the bitstream based on the estimated probability of the i-th scale, and determines the reconstructed geometric data of the i-th scale of the current point cloud. In this way, by inputting geometric data from at least one decoded scale prior to the current scale into the neural network model to obtain the estimated probability of the current scale, and then using this estimated probability to decode the geometric data of the current scale, and so on, from the lowest scale to the highest scale, the decoding of the current point cloud is achieved. This neural network model improves the accuracy of the estimated probability, thereby improving the compression efficiency of the LiDAR point cloud and ultimately enhancing the decoding performance of the point cloud.

[0180] In another embodiment of this application, FIG16 is a flowchart illustrating an encoding method provided in an embodiment of this application. As shown in FIG16, the method may include:

[0181] S1601, determine the geometry of at least one encoded scale of the current point cloud.

[0182] It should be noted that the encoding method in this application embodiment is applied to an encoder, and this encoding method can refer to a LiDAR point cloud compression method based on a neural network. Here, considering the sparsity of LiDAR point clouds and the scanning characteristics of LiDAR, the existing neural network structure is improved, thereby improving the accuracy of the neural network's probability estimation and thus improving the compression performance of LiDAR point clouds.

[0183] It should also be noted that, in the embodiments of this application, the input of the neural network model involves geometric data of at least one encoded scale of the current point cloud. Here, geometric data mainly refers to geometric information of the encoded scale. As the input of the neural network model, the geometric information here can refer to geometric position information (or "geometric coordinate information").

[0184] S1602, input at least one encoded scale of geometric data into the neural network model, and output the estimated probability of the i-th scale of the current point cloud through the neural network model.

[0185] It should be noted that, in the embodiments of this application, at least one encoded scale can refer to encoded scales prior to the i-th scale. Furthermore, inputting the geometric data of at least one encoded scale into the neural network model can mean inputting the geometric position information of this at least one encoded scale into the neural network model, and through feature extraction and feature concatenation within the neural network model, outputting an estimated probability of the geometric occupancy information at the i-th scale. Here, i is a positive integer, and the value of i is greater than 1 and less than or equal to N, where N represents the number of geometric encoding scales of the current point cloud.

[0186] It should also be noted that, in the embodiments of this application, for geometric data of at least one encoded scale, the number of encoded scales input to the neural network model is related to the value of i, which will be described in detail below in several cases.

[0187] In one possible implementation, the value of i is equal to 2, in which case it can be determined that at least one encoded scale of geometric data includes: geometric data of the first scale.

[0188] In other words, in this embodiment of the application, if the value of i is equal to 2, then the geometric data of the first scale can be input into the neural network model, the neural network model can output the estimated probability of the second scale of the current point cloud, and then the estimated probability can be used to encode the geometric data of the second scale.

[0189] In another possible implementation, i is equal to 3, in which case at least one encoded scale of geometric data can be determined, including geometric data of the first scale and geometric data of the second scale.

[0190] In other words, in this embodiment of the application, if the value of i is equal to 3, then the geometric data of the first scale and the geometric data of the second scale can be input into the neural network model, the estimated probability of the third scale of the current point cloud can be output by the neural network model, and then the estimated probability can be used to encode the geometric data of the third scale.

[0191] In another possible implementation, the value of i is greater than 3. In this case, it can be determined that the geometric data of at least one encoded scale includes: geometric data of scale i-3, geometric data of scale i-2, and geometric data of scale i-1.

[0192] In other words, in the embodiments of this application, if the value of i is greater than 3, for example, if the value of i is equal to 4, 5, 6, 7, etc., then the geometric data of the (i-3)th scale, the geometric data of the (i-2)th scale, and the geometric data of the (i-1)th scale can be input into the neural network model, the estimated probability of the current point cloud at the i-th scale can be output by the neural network model, and then the estimated probability can be used to encode the geometric data at the i-th scale.

[0193] For example, if the value of i is greater than 3, the geometric position information of the three adjacent scales (e.g., the (i-1), (i-2), and (i-3) scales that have been encoded before the i-th scale can be obtained. Then, the geometric position information of the three adjacent scales (e.g., the (i-1), (i-2), and (i-3) scales) is input into the neural network model, which can output the estimated probability of the geometric occupancy information of the i-th scale. Then, the estimated probability is used to encode the geometric data (specifically the geometric occupancy information) of the i-th scale.

[0194] Understandably, in this embodiment of the application, since the current point cloud contains a large number of points, it is necessary to downsample the current point cloud to achieve better compression performance. In some embodiments, the method may further include: downsampling the current point cloud to determine geometric data at multiple scales.

[0195] In this embodiment, the geometric data at each scale can be viewed as a sparse tensor. That is, this embodiment can progressively downsample the input current point cloud to generate multi-scale sparse tensors, i.e., geometric data at multiple scales. The number of generated scales is related to the number of downsampling operations. Based on this, starting from the lowest-scale tensor, point cloud geometric compression is performed progressively until the highest scale is reached. In other words, the number of geometric encoding scales of the current point cloud is related to the number of downsampling operations.

[0196] In one possible implementation, the method may further include: downsampling the current point cloud using a pooling layer with a preset step size to determine geometric data at multiple scales. For example, the preset step size may be set to 2×2×2.

[0197] In other words, in this embodiment, a pooling layer with a stride of 2×2×2 can be used for step-by-step downsampling to organize the input LiDAR point cloud into a multi-scale sparse tensor. That is, the output results of the step-by-step downsampling process are formed into a set, namely {Scale i |i=1,2,3,...,N}, which contains point cloud geometric information for each scale.

[0198] It is also understandable that after obtaining geometric data at multiple scales, the number of downsampling operations (or, in other words, the number of geometric coding scales of the current point cloud) can be written into the bitstream. Therefore, in some embodiments, the method may further include: encoding the geometric coding scale of the current point cloud and writing the obtained encoded bits into the bitstream.

[0199] It should be noted that, in the embodiments of this application, for the current point cloud, downsampling is performed at the encoding end to generate multi-scale sparse tensors, where each scale of sparse tensor can be regarded as a layer, so the number of geometric encoding scales can also be called the "number of geometric encoding layers".

[0200] It should also be noted that in this embodiment, the value of i is not infinitely large; for example, i ≤ N, where N represents the geometric coding scale number of the current point cloud. For the encoder, the geometric coding scale number can be written into the geometric bitstream so that the decoder can determine the corresponding geometric decoding scale number by parsing the bitstream.

[0201] It is also understood that, in the embodiments of this application, the encoding end may also contain one or more candidate neural network models.

[0202] In one possible implementation, if the encoding end includes only one candidate neural network model, then that candidate neural network model can be determined as the neural network model used by the current point cloud.

[0203] In another possible implementation, if the encoding end includes multiple candidate neural network models, then in some embodiments the method may further include: determining the model index parameters of the neural network models; encoding the model index parameters; and writing the obtained encoded bits into the bitstream.

[0204] It should be noted that, in this embodiment, after determining the neural network model used for the current point cloud from multiple candidate neural network models, the corresponding model index parameters can be determined based on the neural network model, and then written into the geometric bitstream. In this way, at the decoding end, by parsing the model index parameters in the bitstream and combining them with multiple candidate neural network models, the neural network model used for the current point cloud can be determined.

[0205] In another possible implementation, if the encoding end includes multiple candidate neural network models, then to improve performance, in practical applications, different neural network models are usually further divided for different scale ranges. For example, one neural network model is used for high-scale geometric coding, and another neural network model is used for low-scale geometric coding. In some embodiments, the method may further include: determining a first neural network model applied to the low-scale range and a second neural network model applied to the high-scale range; determining corresponding first model index parameters based on the first neural network model, and determining corresponding second model index parameters based on the second neural network model; encoding the first model index parameters and the second model index parameters, and writing the obtained encoded bits into the bitstream.

[0206] In this embodiment, the first model index parameter indicates a first neural network model applied to a low-scale range, and the second model index parameter indicates a second neural network model applied to a high-scale range. Thus, different neural network models can be used for different scale ranges, resulting in better compression performance.

[0207] Taking the current point cloud as an example, which uses the same neural network model, the structure of the neural network model will be explained below.

[0208] In one possible implementation, as shown in Figure 7 above, the neural network model may include a scale embedding module 701, at least one feature enhancement module 702, at least one dual-view transformation module 703, a transposed convolution module 704, and a classification module 705. Specifically, for geometric data of at least one encoded scale input to the neural network model, the data can be sequentially processed through the scale embedding module 701, at least one feature enhancement module 702, at least one dual-view transformation module 703, transposed convolution module 704, and classification module 705, ultimately outputting an estimated probability for the i-th scale.

[0209] In some embodiments, based on the neural network model shown in Figure 7, inputting geometric data of at least one encoded scale into the neural network model, and outputting the estimated probability of the i-th scale of the current point cloud through the neural network model, may include:

[0210] The scale embedding module 701 performs feature extraction and feature concatenation on geometric data of at least one encoded scale, and outputs a first sparse tensor, which includes a first geometric component and a first feature component.

[0211] The first sparse tensor is enhanced with positional features by at least one feature enhancement module 702, and a second sparse tensor is output, the second sparse tensor including a first geometric component and a second feature component.

[0212] The second sparse tensor is subjected to coordinate system transformation and feature splicing by at least one dual-view transformation module 703, and a third sparse tensor is output, which includes a first geometric component and a third feature component.

[0213] The third sparse tensor is transposed and convolved by the transposed convolution module 704 to output the fourth sparse tensor, which includes the second geometric component and the fourth feature component corresponding to the i-th scale.

[0214] The classification module 705 performs a high-dimensional feature transformation on the fourth sparse tensor, and outputs a one-dimensional estimated probability of the i-th scale.

[0215] In another possible implementation, at least one feature enhancement module 702 may include a first feature enhancement module and a second feature enhancement module, and at least one dual-view transformation module 703 may include a first dual-view transformation module and a second dual-view transformation module.

[0216] In this embodiment of the application, as shown in Figure 8 above, the neural network model may include a scale embedding module 701, a first feature enhancement module 702-1, a first dual-view transformation module 703-1, a second feature enhancement module 702-2, a transposed convolution module 704, a second dual-view transformation module 703-2, and a classification module 705. Specifically, for geometric data of at least one encoded scale input to the neural network model, the data can sequentially pass through the scale embedding module 701, the first feature enhancement module 702-1, the first dual-view transformation module 703-1, the second feature enhancement module 702-2, the transposed convolution module 704, the second dual-view transformation module 703-2, and the classification module 705, ultimately outputting the estimated probability of the i-th scale.

[0217] In some embodiments, based on the neural network model shown in Figure 8, inputting geometric data of at least one encoded scale into the neural network model, and outputting the estimated probability of the i-th scale of the current point cloud through the neural network model, may include:

[0218] The scale embedding module 701 performs feature extraction and feature concatenation on geometric data of at least one encoded scale, and outputs a first sparse tensor, which includes a first geometric component and a first feature component.

[0219] The first sparse tensor is enhanced by the first feature enhancement module 702-1 to perform position feature enhancement, and the second sparse tensor is output. The second sparse tensor includes a first geometric component and a second feature component.

[0220] The first dual-view transformation module 703-1 performs coordinate system transformation and feature splicing on the second sparse tensor to output a third sparse tensor, which includes a first geometric component and a third feature component.

[0221] The third sparse tensor is enhanced with positional features by the second feature enhancement module 702-2, and a fifth sparse tensor is output. The fifth sparse tensor includes a first geometric component and a fifth feature component.

[0222] The fifth sparse tensor is transposed and convolved by the transposed convolution module 704 to output the sixth sparse tensor, which includes the second geometric component and the sixth feature component corresponding to the i-th scale.

[0223] The sixth sparse tensor is transformed by coordinate system transformation and feature splicing through the second dual-view transformation module 703-2, and the seventh sparse tensor is output. The seventh sparse tensor includes the second geometric component and the seventh feature component.

[0224] The classification module 705 performs a high-dimensional feature transformation on the seventh sparse tensor, and outputs a one-dimensional estimated probability of the i-th scale.

[0225] It should be noted that during the processing of the neural network model, the operations of modules such as the scale embedding module 701, the first feature enhancement module 702-1, the first dual-view transformation module 703-1, and the second feature enhancement module 702-2 only process the feature components, while the geometric components remain unchanged. After the operation of the transpose convolution module 704, the first geometric component of the encoded scale can be converted into the second geometric component of the current scale (i.e., the i-th scale), and the feature components will also change accordingly. Then, the operation of the second dual-view transformation module 703-2 still only processes the transformed feature components, while the second geometric component remains unchanged. Finally, the estimated probability of the current scale is output through the classification module 705 (also called the "classifier" or "classifier module").

[0226] It should also be noted that, in the embodiments of this application, the scale embedding module 701 can be called the ancestor sparse tensor embedding module, which is used to extract and concatenate features from the encoded ancestor sparse tensor from a lower scale, and finally output the first sparse tensor. The first sparse tensor may include a first geometric component (represented by C) and a first feature component (represented by F).

[0227] In one possible implementation, as shown in FIG10 above, the scale embedding module 701 may include a first residual network module 1001, a first transposed convolutional layer 1002, a first feature splicing layer 1003, a second residual network module 1004, a second transposed convolutional layer 1005, and a second feature splicing layer 1006.

[0228] In some embodiments, when the value of i is equal to 2, feature extraction and feature concatenation are performed on the geometric data of at least one encoded scale by the scale embedding module 701 to output a first sparse tensor, which may include: directly determining the first geometric component and the first feature component in the first sparse tensor based on the geometric data of the first scale.

[0229] It should be noted that in this embodiment of the application, when the value of i is equal to 2, the sub-modules in the scale embedding module 701 are not enabled. The first geometric component and the first feature component in the first sparse tensor can be determined directly based on the geometric data of the first scale after the first scale geometric data is input into the scale embedding module.

[0230] In some embodiments, when the value of i is equal to 3, feature extraction and feature concatenation are performed on geometric data of at least one encoded scale by the scale embedding module 701 to output a first sparse tensor. This may include: performing feature extraction and feature concatenation on geometric data of the first scale and geometric data of the second scale by the second residual network module, the second transposed convolutional layer and the second feature concatenation layer in the scale embedding module to output a first geometric component and a first feature component in the first sparse tensor.

[0231] It should be noted that in this embodiment, when the value of i is equal to 3, some sub-modules in the scale embedding module 701 are not enabled, for example, only the second residual network module 1004, the second transposed convolutional layer 1005, and the second feature concatenation layer 1006 are enabled. In this way, the geometric data of the first scale is input into the second residual network module 1004 for feature extraction, and the corresponding output features are mapped to the coordinates of the second scale through the second transposed convolutional layer 1005; then the second feature concatenation layer 1006 is used to concatenate the mapped output features with the geometric data of the second scale, and finally outputs a first sparse tensor with ancestor embedding, wherein the first sparse tensor includes a first geometric component C and a first feature component F.

[0232] In some embodiments, when the value of i is greater than 3, feature extraction and feature concatenation are performed on geometric data of at least one encoded scale by the scale embedding module 701 to output a first sparse tensor. This may include: performing feature extraction and feature concatenation on geometric data of the (i-3)th scale, the (i-2)th scale, and the (i-1)th scale by the scale embedding module 701 to output a first geometric component and a first feature component in the first sparse tensor.

[0233] It should be noted that in this embodiment, when the value of i is greater than 3, all sub-modules in the scale embedding module 701 are enabled. At this time, the geometric data of the (i-3)th scale is the input of the first residual network module 1001, and the output of the first residual network module 1001 is the input of the first transposed convolutional layer 1002. The input of the first feature splicing layer 1003 includes the output of the first transposed convolutional layer 1002 and the geometric data of the (i-2)th scale. The output of the first feature splicing layer 1003 is the input of the second residual network module 1004, and the output of the second residual network module 1004 is the input of the second transposed convolutional layer 1005. The input of the second feature splicing layer 1006 includes the output of the second transposed convolutional layer 1005 and the geometric data of the (i-1)th scale. The output of the second feature splicing layer 1006 is the first feature component F in the first sparse tensor.

[0234] Accordingly, in some embodiments, based on the scale embedding module shown in FIG10, feature extraction and feature concatenation are performed on the geometric data at the (i-3)th scale, the (i-2)th scale, and the (i-1)th scale through the scale embedding module 701, and the first geometric component and the first feature component in the first sparse tensor are output, which may include:

[0235] The first residual network module 1001 extracts features from the geometric data at the (i-3)th scale to determine the first output feature, and the first transposed convolutional layer 1002 performs scale mapping on the first output feature to determine the second output feature at the (i-2)th scale.

[0236] The first feature splicing layer 1003 splices the geometric data at the (i-2)th scale with the second output feature to determine the third output feature;

[0237] The second residual network module 1004 extracts features from the third output feature to determine the fourth output feature, and the second transposed convolutional layer 1005 performs scale mapping on the fourth output feature to determine the fifth output feature at the (i-1)th scale.

[0238] The second feature splicing layer 1006 performs feature splicing on the geometric data at the (i-1)th scale and the fifth output feature to determine the first feature component in the first sparse tensor, and the first geometric component in the first sparse tensor is the geometric component corresponding to the (i-1)th scale.

[0239] It should also be noted that the scale embedding module (i.e., the ancestor sparse tensor embedding module) includes, in sequence: a ResNet module (i.e., the first residual network module 1001), a transposed convolutional layer (i.e., the first transposed convolutional layer 1002), a feature concatenation layer (i.e., the first feature concatenation layer 1003), a ResNet module (i.e., the second residual network module 1004), a transposed convolutional layer (i.e., the second transposed convolutional layer 1005), and a feature concatenation layer (i.e., the second feature concatenation layer 1006).

[0240] In other words, in the embodiments of this application, the sparse tensor geometric data of the encoded ancestor from a lower scale is first determined, such as the geometric data at the (i-3)th scale (which can be scaled). i-3 (representation), geometric data at the (i-2)th scale (which can be represented by Scale) i-2 (representation) and geometric data at the (i-1)th scale (which can be represented by Scale) i-1 (representation); then Scale i-3 The first output features are input into the first residual network module 1001 for feature extraction, and then passed through the first transposed convolutional layer 1002 to map the first output features to a scaled matrix. i-2The coordinates; then the second output feature is compared with the scale. i-2 The geometric data is concatenated and sent again to the next stacked second residual network module 1004 and second transposed convolutional layer 1005, where the resulting fifth output feature is compared with the scale. i-1 After the geometric data is concatenated, the final output is a first sparse tensor with ancestor embedding, which may include a first geometric component C and a first feature component F.

[0241] Additionally, it should be noted that the geometric data used in the neural network model in this application embodiment can refer to geometric information, such as geometric position information (or "geometric coordinate information").

[0242] In one possible implementation, as shown in Figure 11 above, each feature enhancement module may include a multilayer perceptron module 1101 and a third feature concatenation layer 1102. Accordingly, in some embodiments, performing positional feature enhancement on the first sparse tensor through at least one feature enhancement module to output a second sparse tensor may include:

[0243] The first geometric component in the first sparse tensor is transformed by the multilayer perceptron module 1101 to determine the sixth output feature.

[0244] The sixth output feature is concatenated with the first feature component in the first sparse tensor by the third feature concatenation layer 1102 to determine the second feature component in the second sparse tensor, while the first geometric component in the second sparse tensor remains unchanged.

[0245] It should be noted that, in the embodiments of this application, the feature enhancement module here may refer to the location feature enhancement module. A multilayer perceptron (MLP) is a feedforward artificial neural network model that maps multiple input datasets to a single output dataset; for example, the first geometric component in the first sparse tensor is input into the multilayer perceptron module 1101 to generate a sixth output feature of a high-dimensional location.

[0246] For example, a feature enhancement module (such as a first feature enhancement module or a second feature enhancement module) may sequentially include: an MLP layer (i.e., a multilayer perceptron module 1101) and a feature concatenation layer (i.e., a third feature concatenation layer 1102). That is, taking the first feature enhancement module 702-1 in Figure 8 as an example, the first geometric component C in the first sparse tensor may be input into the multilayer perceptron module 1101 to generate a sixth output feature at a high-dimensional position, and then concatenated with the first feature component F in the first sparse tensor to obtain the second feature component F0 of the second sparse tensor. The geometric component remains unchanged, that is, the output second sparse tensor includes: the first geometric component C and the second feature component F0.

[0247] In one possible implementation, as shown in Figure 12 above, each dual-view transformation module may include a dual-view sparse attention module 1201, a third residual network module 1202, and a neighbor point attention module 1203. Accordingly, in some embodiments, the third sparse tensor is output by performing coordinate system transformation and feature concatenation on the second sparse tensor through at least one dual-view transformation module, which may include:

[0248] The second sparse tensor is subjected to coordinate system transformation and feature splicing by the dual-view sparse attention module 1201 to determine the seventh output feature;

[0249] The seventh output feature is extracted by the third residual network module 1202 to determine the eighth output feature;

[0250] The eighth output feature is extracted by the neighbor point attention module 1203 to determine the third feature component in the third sparse tensor, while the first geometric component in the third sparse tensor remains unchanged.

[0251] It should be noted that, in the embodiments of this application, the dual-view transformation module here may refer to the dual-view sparse transformer module, which is used to realize feature transformation and feature stitching under dual-view views in Cartesian coordinate system and spherical coordinate system.

[0252] For example, a dual-view transformation module (e.g., a first dual-view transformation module or a second dual-view transformation module) sequentially includes: a dual-view sparse attention layer (i.e., dual-view sparse attention module 1201), a ResNet module (i.e., a third residual network module 1202), and a neighbor point attention layer (i.e., neighbor point attention module 1203). That is, taking the first dual-view transformation module 703-1 in Figure 8 as an example, firstly, the second sparse tensor is subjected to coordinate system transformation and feature concatenation by the dual-view sparse attention module 1201 to obtain the seventh output feature; then, the seventh output feature is input into the third residual network module 1202 for feature extraction to obtain the eighth output feature; then, the eighth output feature is input into the neighbor point attention module 1203 for feature extraction to obtain the third feature component F3 in the third sparse tensor, where the geometric component remains unchanged, that is, the output third sparse tensor includes: the first geometric component C and the third feature component F3.

[0253] In one possible implementation, as shown in Figure 13 above, the dual-view sparse attention module may include a coordinate system transformation module 1301, a first-view attention layer 1302, a second-view attention layer 1303, and a fourth feature concatenation layer 1304. Accordingly, determining the seventh output feature by performing coordinate system transformation and feature concatenation on the second sparse tensor through the dual-view sparse attention module may include:

[0254] The coordinate system transformation module 1301 performs coordinate system transformation on the first geometric component in the first coordinate system to determine the first geometric component in the second coordinate system; and based on the first geometric component in the first coordinate system and the first geometric component in the second coordinate system, the second feature component in the second sparse tensor is split into a first view feature component and a second view feature component.

[0255] The first view feature component is extracted by the first view attention layer 1302 to determine the ninth output feature, and the second view feature component is extracted by the second view attention layer 1303 to determine the tenth output feature.

[0256] The ninth and tenth output features are concatenated by the fourth feature concatenation layer 1304 to determine the seventh output feature.

[0257] It should be noted that in the embodiments of this application, the first coordinate system can be a Cartesian coordinate system, and the second coordinate system can be a spherical coordinate system. For example, assuming that the Cartesian coordinate system of the first geometric component is C1 = {x, y, z}, it is transformed into a spherical coordinate system C2 = {ρ, θ, φ}. The calculation of the coordinate system transformation is as shown in the aforementioned equation (1).

[0258] Then the first geometric components in the Cartesian coordinate system and the spherical coordinate system can be divided into non-overlapping windows. Here, the spherical coordinate system is used as an example, and the window is represented as shown in the aforementioned equation (2).

[0259] Here, the second eigencomponent F in the second sparse tensor o ∈R N×d Decomposed into spherical view feature components and Cartesian view feature components For the spherical view portion, i.e., the aforementioned first view feature component F1, it is linearly transformed into query, key, and value vectors Q1, K1, and V1. Then, using the first view attention layer 1302, the attention is calculated using the aforementioned equation (3).

[0260] Similarly, for the Cartesian view part, i.e., the aforementioned second view feature component F2, it is linearly transformed into query, key, and value vectors Q2, K2, and V2. Then, using the second-view attention layer 1303, the attention is calculated using Equation (4) as described above.

[0261] It should be noted that the first view feature component F1 and the second view feature component F2 are calculated in parallel using the same process described above. Finally, the output features of these two Attn(·) are concatenated together by the fourth feature concatenation layer 1304 to form the final output feature F4 of the dual-view sparse attention module. During this process, the first geometric component remains unchanged.

[0262] It is also worth noting that compared with the single-view perspective in related technologies, the dual-view perspective can contain more information. The Cartesian coordinate system and the spherical coordinate system can correspond to different perspectives. Therefore, more information can be obtained by using the dual-view perspective and input into the neural network model, which makes it easier for the neural network model to reason, and thus helps the neural network model to output a better estimated probability.

[0263] It is also understood that, in the embodiments of this application, for the neural network model, the geometric data input to the neural network model is the geometric information of the point cloud, such as geometric position information or geometric coordinate information; the output of the neural network model is the estimated probability of the geometric occupancy information. Therefore, the neural network model may also include a geometric code conversion module.

[0264] In one possible implementation, as shown in Figure 14 above, the neural network model may include a geometric code conversion module 1401, a scale embedding module 701, a first feature enhancement module 702-1, a first dual-view transformation module 703-1, a second feature enhancement module 702-2, a transposed convolution module 704, a second dual-view transformation module 703-2, and a classification module 705.

[0265] In this embodiment, for geometric data of at least one encoded scale input to the neural network model, the geometric code conversion module 1401 first converts the geometric information expressed by the point cloud from geometric coordinates into occupancy codes, i.e., geometric occupancy information. Then, the data sequentially passes through the scale embedding module 701, the first feature enhancement module 702-1, the first dual-view transformation module 703-1, the second feature enhancement module 702-2, the transposed convolution module 704, the second dual-view transformation module 703-2, and the classification module 705, and finally outputs the estimated probability of the geometric occupancy information of the i-th scale.

[0266] In other words, in this embodiment, at least one encoded geometric data scale is input into the neural network model. First, the geometric code conversion module 1401 converts the geometric information expressed by the point cloud into geometric occupancy information. Then, the scale embedding module 701 performs feature extraction and feature concatenation to output a first sparse tensor, which includes a first geometric component and a first feature component. Next, the first feature enhancement module 702-1 enhances the positional features of the first sparse tensor to output a second sparse tensor, which includes a first geometric component and a second feature component. Finally, the first dual-view transformation module 703-1 performs coordinate system transformation and feature concatenation on the second sparse tensor to output a third sparse tensor, which includes a first geometric component and a second feature component. The first sparse tensor consists of a first geometric component and a third feature component. Then, the second feature enhancement module 702-2 enhances the positional features of the third sparse tensor, outputting a fifth sparse tensor, which includes the first geometric component and the fifth feature component. Next, the transpose convolution module 704 performs transpose convolution on the fifth sparse tensor, outputting a sixth sparse tensor, which includes the second geometric component and the sixth feature component corresponding to the i-th scale. Then, the second dual-view transformation module 703-2 performs coordinate system transformation and feature concatenation on the sixth sparse tensor, outputting a seventh sparse tensor, which includes the second geometric component and the seventh feature component. Finally, the classification module 705 performs high-dimensional feature transformation on the seventh sparse tensor, outputting a one-dimensional estimated probability for the i-th scale.

[0267] S1603, Encode the geometric data of the current point cloud at the i-th scale according to the estimated probability of the i-th scale, and generate the geometric sub-code stream at the i-th scale in the code stream.

[0268] In this embodiment, i is a positive integer. For each scale, the geometric data of each scale is encoded, and a corresponding geometric sub-bitstream exists in the bitstream. For example, for the i-th scale, the bitstream includes a geometric sub-bitstream of the i-th scale. Here, the value of i is greater than 1 and less than or equal to N, where N represents the number of geometric encoding scales of the current point cloud.

[0269] Thus, after obtaining the estimated probability at scale i, the geometric data at scale i of the current point cloud can be encoded based on the estimated probability at scale i, generating a geometric sub-bitstream at scale i in the bitstream. It should be noted that the encoded geometric data here can refer to geometric occupancy information, used to characterize whether a point at the current scale is occupied.

[0270] In some embodiments, when the value of i is equal to 1, the method may further include: encoding the geometric data of the first scale to generate a geometric sub-bitstream of the first scale in the bitstream.

[0271] In other words, in this embodiment of the application, for the first scale, the geometric data of the first scale can be directly losslessly encoded to generate the geometric sub-codestream of the first scale.

[0272] It should also be noted that in the embodiments of this application, as i increases, encoding is performed from low scale to high scale. When the highest scale is reached, i.e., when i equals N, it means that all N scales of the current point cloud have been encoded.

[0273] For example, Figure 17 is a detailed flowchart of an encoding method provided in an embodiment of this application. As shown in Figure 17, encoding begins from the lowest scale, i.e., the first scale, and continues until the highest scale is reached. Specifically, the geometric data at the first scale is first encoded to generate a geometric sub-bitstream. 1 Then, by inputting the geometric data at the first scale into the neural network model, an estimated probability at the second scale can be obtained. This estimated probability is then used to encode the geometric data at the second scale, generating a geometric sub-bitstream. 2 Similarly, the geometric data at the i-th scale is encoded based on the estimated probability from the neural network model to generate a geometric sub-bitstream. i The neural network model takes geometric data at scales i-3, i-2, and i-1 as input and outputs an estimated probability of the geometric occupancy information at scale i. This process continues until i equals N, where the estimated probability from the neural network model is used to encode the geometric data at scale N, generating a geometric sub-bitstream. N This means that the current point cloud encoding is complete.

[0274] This embodiment provides an encoding method that determines at least one encoded scale of geometric data for the current point cloud; inputs the geometric data of at least one encoded scale into a neural network model, and outputs an estimated probability of the i-th scale of the current point cloud through the neural network model; encodes the geometric data of the i-th scale of the current point cloud according to the estimated probability of the i-th scale, generating a geometric sub-bitstream of the i-th scale in the bitstream. In this way, by inputting the geometric data of at least one encoded scale prior to the current scale into the neural network model to obtain the estimated probability of the current scale, and then using this estimated probability to encode the geometric data of the current scale, and so on, from the lowest scale to the highest scale, the encoding of the current point cloud is achieved. This neural network model can improve the accuracy of the estimated probability, thereby improving the compression efficiency of the LiDAR point cloud and ultimately enhancing the encoding performance of the point cloud.

[0275] In another embodiment of this application, based on the encoding and decoding methods described in the foregoing embodiments, a neural network-based geometric compression method for lidar point clouds is provided. Taking lidar point clouds as an example, Figure 18 is a schematic diagram of the encoding principle of lidar point cloud geometric compression provided in an embodiment of this application, and Figure 19 is a schematic diagram of the decoding principle of lidar point cloud geometric compression provided in an embodiment of this application.

[0276] At the encoding end, as shown in Figure 18, the input LiDAR point cloud is first organized into a multi-scale sparse tensor {Scale} through progressive downsampling (pooling layers with a stride of 2×2×2). i |i=1,2,3,...,N}, which contains point cloud geometric information for each scale; then {Scale i The input |i=1,2,3,...,N} is an encoder that estimates probabilities based on a neural network model, and the output is the corresponding bitstream. i |i=1,2,3,...,N}, where the input to the neural network model is the sparse tensor of the previous scale (e.g., the encoded scale). i At that time, the input to the neural network model is Scale i-1 ).

[0277] At the decoding end, as shown in Figure 19, the bitstream is sequentially input (i=1 to i=N) into the decoder based on the probability estimation of the neural network model, and the output is the decoded point cloud of the corresponding scale, that is, the reconstructed geometric data of the corresponding scale. The input of the neural network model is the sparse tensor of the previous scale (such as the decoded bitstream). i At that time, the input to the neural network model is Scale i-1 This process is repeated sequentially from the lowest scale to the highest scale to decode the lidar point cloud.

[0278] In this embodiment of the application, the steps for the neural network model are as follows: the sparse tensor is sequentially input into the ancestor sparse tensor embedding module, the position feature enhancement module, the dual-view sparse transformer module, the position feature enhancement module, the transposed convolution module, the dual-view sparse transformer module, and the classifier module, and finally the geometric probability information is output.

[0279] In one possible implementation, the neural network model can include:

[0280] (1) Geometric code conversion module, which converts the geometric information expressed by the point cloud from geometric coordinates into occupancy codes;

[0281] (2) Ancestor sparse tensor embedding module, which consists of a ResNet residual network module, a transposed convolutional layer, a feature concatenation layer, a ResNet residual network module, a transposed convolutional layer, and a feature concatenation layer. Based on the decoded ancestor sparse tensor from a lower scale (Scale). i-3 Scale i-2 and Scale i-1 Scale i-3 The input is fed into a ResNet residual network module for feature extraction, and the output features are mapped to a scaled matrix using transposed convolutional layers. i-2 The coordinates; then the output features are compared with the scale. i-2 The geometric information is concatenated and then sent again to the next stacked ResNet residual network module and transposed convolutional layer, where the output features are compared with the scale. i-1 After splicing the geometric information, the final output is a first sparse tensor with ancestor embedding. The first sparse tensor can include two parts: geometric component C and feature component F.

[0282] (2) The location feature enhancement module consists of a multilayer perceptron (MLP) module and a feature concatenation layer. The geometric component C of the first sparse tensor can be input into the MLP module to generate a high-dimensional location feature vector. This vector is then concatenated with the original input feature component F to form the output feature tensor F of the second sparse tensor. o The geometric component C of the second sparse tensor remains unchanged.

[0283] (3) A dual-view sparse transformer module, which consists of a dual-view sparse attention layer, a ResNet residual network module, and a neighbor point attention layer. Specifically, for the dual-view sparse attention layer, the second sparse tensor can be scaled. i The Cartesian coordinate system C1 = {x, y, z} is transformed into the spherical coordinate system C2 = {ρ, θ, φ} through coordinate transformation, as shown in the aforementioned equation (1).

[0284] Then, the Cartesian coordinates and spherical coordinates are divided into non-overlapping windows. Taking spherical coordinates as an example, the window is shown in equation (2) above. Second sparse tensor Scale i eigencomponents F o ∈R N×d Feature components decomposed into spherical views Feature components corresponding to the Cartesian view For the spherical part, F1, it is linearly transformed into query, key, and value vectors Q1, K1, and V1. The attention is then calculated using the aforementioned equation (3). For the Cartesian part, F2, it can also be calculated in parallel using the same process described above. Finally, the outputs of these two Attn(·) are concatenated as the output of the dual-view sparse attention layer. In this way, based on the different perspectives corresponding to the two coordinate systems, more information can be input into the neural network model, thereby helping the neural network model to better predict probabilities and achieve better compression performance.

[0285] In the embodiments of this application, the specific implementation of the aforementioned embodiments is described in detail through the above embodiments. It can be seen from this that, according to the technical solution of the aforementioned embodiments, the relevant technical framework is improved in view of the sparsity of LiDAR point clouds and the scanning characteristics of LiDAR, so as to improve the accuracy of probability estimation of neural network models, thereby improving the compression performance of LiDAR point clouds and thus improving the encoding and decoding efficiency of point clouds.

[0286] In another embodiment of this application, based on the same inventive concept as the foregoing embodiments, FIG20 is a schematic diagram of the composition structure of an encoder provided in an embodiment of this application. As shown in FIG20, the encoder 200 includes a first determining unit 2001 and an encoding unit 2002, wherein:

[0287] The first determining unit 2001 is configured to determine geometric data of at least one encoded scale of the current point cloud; and to input the geometric data of at least one encoded scale into a neural network model, and output the estimated probability of the i-th scale of the current point cloud through the neural network model.

[0288] The encoding unit 2002 is configured to encode the geometric data of the current point cloud at the i-th scale according to the estimated probability of the i-th scale, and generate the geometric sub-code stream at the i-th scale in the code stream; where i takes the value greater than 1 and less than or equal to N, and N represents the number of geometric encoding scales of the current point cloud.

[0289] In some embodiments, referring to FIG20, encoder 200 may further include downsampling unit 2003, configured to downsample the current point cloud to determine geometric data at multiple scales; wherein the number of geometric coding scales of the current point cloud is related to the number of downsampling times.

[0290] In some embodiments, the downsampling unit 2003 is further configured to downsample the current point cloud using a pooling layer with a preset step size to determine geometric data at multiple scales.

[0291] In some embodiments, the encoding unit 2002 is further configured to encode the geometric encoding scale number of the current point cloud and write the obtained encoded bits into the bit stream.

[0292] In some embodiments, the encoding unit 2002 is further configured to encode the geometric data of the first scale when the value of i is equal to 1, thereby generating a geometric sub-bitstream of the first scale in the bitstream.

[0293] In some embodiments, the first determining unit 2001 is further configured to determine, when the value of i is equal to 2, that the geometric data of at least one encoded scale includes: geometric data of a first scale; when the value of i is equal to 3, that the geometric data of at least one encoded scale includes: geometric data of a first scale and geometric data of a second scale; and when the value of i is greater than 3, that the geometric data of at least one encoded scale includes: geometric data of a (i-3)th scale, geometric data of a (i-2)th scale, and geometric data of a (i-1)th scale.

[0294] In some embodiments, the neural network model includes a scale embedding module, at least one feature enhancement module, at least one dual-view transformation module, a transposed convolution module, and a classification module. Referring to FIG20, the encoder 200 may further include a first processing unit 2004, configured to perform feature extraction and feature concatenation on geometric data of at least one encoded scale through the scale embedding module, and output a first sparse tensor, the first sparse tensor including a first geometric component and a first feature component; perform position feature enhancement on the first sparse tensor through at least one feature enhancement module, and output a second sparse tensor, the second sparse tensor including a first geometric component and a second feature component; perform coordinate system transformation and feature concatenation on the second sparse tensor through at least one dual-view transformation module, and output a third sparse tensor, the third sparse tensor including a first geometric component and a third feature component; perform transposed convolution processing on the third sparse tensor through the transposed convolution module, and output a fourth sparse tensor, the fourth sparse tensor including a second geometric component and a fourth feature component corresponding to the i-th scale; and perform high-dimensional feature transformation on the fourth sparse tensor through the classification module, and output a one-dimensional estimated probability of the i-th scale.

[0295] In some embodiments, at least one feature enhancement module includes a first feature enhancement module and a second feature enhancement module, and at least one dual-view transformation module includes a first dual-view transformation module and a second dual-view transformation module; the first processing unit 2004 is further configured to perform feature extraction and feature concatenation on geometric data of at least one encoded scale through a scale embedding module, and output a first sparse tensor, the first sparse tensor including a first geometric component and a first feature component; perform position feature enhancement on the first sparse tensor through the first feature enhancement module, and output a second sparse tensor, the second sparse tensor including a first geometric component and a second feature component; and perform coordinate system transformation and feature concatenation on the second sparse tensor through the first dual-view transformation module, and output a third sparse tensor. The sparse tensor consists of a third sparse tensor, which includes a first geometric component and a third feature component. A second feature enhancement module enhances the positional features of the third sparse tensor, outputting a fifth sparse tensor, which also includes a first geometric component and a fifth feature component. A transpose convolution module performs transpose convolution on the fifth sparse tensor, outputting a sixth sparse tensor, which includes a second geometric component and a sixth feature component corresponding to the i-th scale. A second dual-view transformation module performs coordinate system transformation and feature concatenation on the sixth sparse tensor, outputting a seventh sparse tensor, which includes a second geometric component and a seventh feature component. Finally, a classification module performs high-dimensional feature transformation on the seventh sparse tensor, outputting a one-dimensional estimated probability for the i-th scale.

[0296] In some embodiments, the scale embedding module includes a first residual network module, a first transposed convolutional layer, a first feature concatenation layer, a second residual network module, a second transposed convolutional layer, and a second feature concatenation layer; the first processing unit 2004 is further configured to, when the value of i is equal to 2, directly determine the first geometric component and the first feature component in the first sparse tensor based on the geometric data of the first scale; when the value of i is equal to 3, perform feature extraction and feature concatenation on the geometric data of the first scale and the geometric data of the second scale through the second residual network module, the second transposed convolutional layer, and the second feature concatenation layer in the scale embedding module, and output the first geometric component and the first feature component in the first sparse tensor; when the value of i is greater than 3, perform feature extraction and feature concatenation on the geometric data of the (i-3)th scale, the (i-2)th scale, and the (i-1)th scale through the scale embedding module, and output the first geometric component and the first feature component in the first sparse tensor.

[0297] In some embodiments, the first processing unit 2004 is further configured to: extract features from the geometric data at the (i-3)th scale using a first residual network module to determine a first output feature; perform scale mapping on the first output feature using a first transposed convolutional layer to determine a second output feature at the (i-2)th scale; perform feature concatenation on the geometric data at the (i-2)th scale and the second output feature using a first feature concatenation layer to determine a third output feature; extract features from the third output feature using a second residual network module to determine a fourth output feature; perform scale mapping on the fourth output feature using a second transposed convolutional layer to determine a fifth output feature at the (i-1)th scale; and perform feature concatenation on the geometric data at the (i-1)th scale and the fifth output feature using a second feature concatenation layer to determine a first feature component in the first sparse tensor, wherein the first geometric component in the first sparse tensor is the geometric component corresponding to the (i-1)th scale.

[0298] In some embodiments, the feature enhancement module includes a multilayer perceptron module and a third feature concatenation layer; the first processing unit 2004 is further configured to perform feature transformation on the first geometric component in the first sparse tensor through the multilayer perceptron module to determine a sixth output feature; and to perform feature concatenation between the sixth output feature and the first feature component in the first sparse tensor through the third feature concatenation layer to determine a second feature component in the second sparse tensor, wherein the first geometric component in the second sparse tensor remains unchanged.

[0299] In some embodiments, the dual-view transformation module includes a dual-view sparse attention module, a third residual network module, and a neighbor point attention module; the first processing unit 2004 is further configured to perform coordinate system transformation and feature concatenation on the second sparse tensor through the dual-view sparse attention module to determine a seventh output feature; to extract features from the seventh output feature through the third residual network module to determine an eighth output feature; and to extract features from the eighth output feature through the neighbor point attention module to determine a third feature component in the third sparse tensor, while the first geometric component in the third sparse tensor remains unchanged.

[0300] In some embodiments, the dual-view sparse attention module includes a coordinate system transformation module, a first-view attention layer, a second-view attention layer, and a fourth feature concatenation layer; the first processing unit 2004 is further configured to: perform coordinate system transformation on the first geometric component in the first coordinate system through the coordinate system transformation module to determine the first geometric component in the second coordinate system; split the second feature component in the second sparse tensor into a first-view feature component and a second-view feature component based on the first geometric component in the first coordinate system and the first geometric component in the second coordinate system; extract features from the first-view feature component through the first-view attention layer to determine the ninth output feature, and extract features from the second-view feature component through the second-view attention layer to determine the tenth output feature; and concatenate the ninth output feature and the tenth output feature through the fourth feature concatenation layer to determine the seventh output feature.

[0301] In some embodiments, the first determining unit 2001 is further configured to determine the model index parameters of the neural network model; the encoding unit 2002 is further configured to encode the model index parameters and write the obtained encoded bits into the code stream.

[0302] Understandably, in the embodiments of this application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular one. Furthermore, the components in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional module.

[0303] In another embodiment of this application, FIG21 is a schematic diagram of the hardware structure of an encoder provided in an embodiment of this application. As shown in FIG21, the encoder 200 may include: a first communication interface 2101, a first memory 2102, and a first processor 2103; the various components are coupled together through a first bus system 2104. It is understood that the first bus system 2104 is used to realize the connection and communication between these components. In addition to a data bus, the first bus system 2104 also includes a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as the first bus system 2104 in FIG21.

[0304] The first communication interface 2101 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0305] The first memory 2102 is used to store computer programs that can run on the first processor 2103;

[0306] The first processor 2103 is configured to, when running the computer program, perform:

[0307] Determine at least one encoded scale of geometric data for the current point cloud; input the geometric data of at least one encoded scale into a neural network model, and output the estimated probability of the i-th scale of the current point cloud through the neural network model; encode the geometric data of the i-th scale of the current point cloud according to the estimated probability of the i-th scale to generate the geometric sub-bitstream of the i-th scale in the bitstream; where i takes the value greater than 1 and less than or equal to N, and N represents the number of geometric encoding scales of the current point cloud.

[0308] It is understood that the first memory 2102 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The first memory 2102 of the system and method described in this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0309] The first processor 2103 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the first processor 2103 or by instructions in software form. The first processor 2103 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the first memory 2102. The first processor 2103 reads the information in the first memory 2102 and completes the steps of the above method in conjunction with its hardware.

[0310] It is understood that the embodiments described in this application can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or combinations thereof. For software implementation, the technology described in this application can be implemented through modules (e.g., procedures, functions, etc.) that perform the functions described in this application. Software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0311] Alternatively, as another embodiment, the first processor 2103 is further configured to perform the method described in any of the foregoing embodiments when running the computer program.

[0312] This embodiment provides an encoder in which geometric data from at least one encoded scale prior to the current scale are input into a neural network model to obtain an estimated probability for the current scale. This estimated probability is then used to encode the geometric data at the current scale, and so on, from the lowest scale to the highest scale, thereby achieving the encoding of the current point cloud. In this way, the accuracy of the estimated probability can be improved through the neural network model, thereby improving the compression efficiency of the LiDAR point cloud and thus enhancing the encoding performance of the point cloud.

[0313] In another embodiment of this application, based on the same inventive concept as the foregoing embodiments, FIG22 is a schematic diagram of the composition structure of a decoder provided in an embodiment of this application. As shown in FIG22, the decoder 220 includes a second determining unit 2201 and a decoding unit 2202, wherein:

[0314] The second determining unit 2201 is configured to determine at least one decoded scale of reconstructed geometric data of the current point cloud; and to input the reconstructed geometric data of at least one decoded scale into a neural network model, and output the estimated probability of the i-th scale of the current point cloud through the neural network model.

[0315] Decoding unit 2202 is configured to decode the geometric sub-bitstream of the i-th scale in the bitstream according to the estimated probability of the i-th scale, and determine the reconstructed geometric data of the i-th scale of the current point cloud; where i is greater than 1 and less than or equal to N, and N represents the number of geometric decoding scales of the current point cloud.

[0316] In some embodiments, the second determining unit 2201 is further configured to determine, when the value of i is equal to 2, that the reconstructed geometric data of at least one decoded scale includes: reconstructed geometric data of the first scale; when the value of i is equal to 3, that the reconstructed geometric data of at least one decoded scale includes: reconstructed geometric data of the first scale and reconstructed geometric data of the second scale; and when the value of i is greater than 3, that the reconstructed geometric data of at least one decoded scale includes: reconstructed geometric data of the (i-3)th scale, reconstructed geometric data of the (i-2)th scale, and reconstructed geometric data of the (i-1)th scale.

[0317] In some embodiments, the decoding unit 2202 is further configured to decode the geometric sub-bitstream of the first scale in the bitstream when the value of i is equal to 1, and determine the reconstructed geometric data of the first scale.

[0318] In some embodiments, the neural network model includes a scale embedding module, at least one feature enhancement module, at least one dual-view transformation module, a transposed convolution module, and a classification module. Referring to FIG22, the decoder 220 may further include a second processing unit 2203, configured to perform feature extraction and feature concatenation on reconstructed geometric data of at least one decoded scale through the scale embedding module, and output a first sparse tensor, the first sparse tensor including a first geometric component and a first feature component; perform position feature enhancement on the first sparse tensor through at least one feature enhancement module, and output a second sparse tensor, the second sparse tensor including a first geometric component and a second feature component; perform coordinate system transformation and feature concatenation on the second sparse tensor through at least one dual-view transformation module, and output a third sparse tensor, the third sparse tensor including a first geometric component and a third feature component; perform transposed convolution processing on the third sparse tensor through the transposed convolution module, and output a fourth sparse tensor, the fourth sparse tensor including a second geometric component and a fourth feature component corresponding to the i-th scale; and perform high-dimensional feature transformation on the fourth sparse tensor through the classification module, and output a one-dimensional estimated probability of the i-th scale.

[0319] In some embodiments, at least one feature enhancement module includes a first feature enhancement module and a second feature enhancement module, and at least one dual-view transformation module includes a first dual-view transformation module and a second dual-view transformation module; the second processing unit 2203 is further configured to perform feature extraction and feature stitching on reconstructed geometric data at at least one decoded scale through a scale embedding module, and output a first sparse tensor, the first sparse tensor including a first geometric component and a first feature component; perform position feature enhancement on the first sparse tensor through the first feature enhancement module, and output a second sparse tensor, the second sparse tensor including a first geometric component and a second feature component; and perform coordinate system transformation and feature stitching on the second sparse tensor through the first dual-view transformation module, and output a third... The sparse tensor consists of a third sparse tensor, which includes a first geometric component and a third feature component. A second feature enhancement module enhances the positional features of the third sparse tensor, outputting a fifth sparse tensor, which also includes a first geometric component and a fifth feature component. A transpose convolution module performs transpose convolution on the fifth sparse tensor, outputting a sixth sparse tensor, which includes a second geometric component and a sixth feature component corresponding to the i-th scale. A second dual-view transformation module performs coordinate system transformation and feature concatenation on the sixth sparse tensor, outputting a seventh sparse tensor, which includes a second geometric component and a seventh feature component. Finally, a classification module performs high-dimensional feature transformation on the seventh sparse tensor, outputting a one-dimensional estimated probability for the i-th scale.

[0320] In some embodiments, the scale embedding module includes a first residual network module, a first transposed convolutional layer, a first feature concatenation layer, a second residual network module, a second transposed convolutional layer, and a second feature concatenation layer; the second processing unit 2203 is further configured to, when the value of i is equal to 2, directly determine the first geometric component and the first feature component in the first sparse tensor based on the reconstructed geometric data of the first scale; when the value of i is equal to 3, perform feature extraction and feature concatenation on the reconstructed geometric data of the first scale and the reconstructed geometric data of the second scale through the second residual network module, the second transposed convolutional layer, and the second feature concatenation layer in the scale embedding module, and output the first geometric component and the first feature component in the first sparse tensor; when the value of i is greater than 3, perform feature extraction and feature concatenation on the reconstructed geometric data of the (i-3)th scale, the (i-2)th scale, and the (i-1)th scale through the scale embedding module, and output the first geometric component and the first feature component in the first sparse tensor.

[0321] In some embodiments, the second processing unit 2203 is further configured to: extract features from the reconstructed geometric data at the (i-3)th scale using a first residual network module to determine a first output feature; perform scale mapping on the first output feature using a first transposed convolutional layer to determine a second output feature at the (i-2)th scale; perform feature concatenation on the reconstructed geometric data at the (i-2)th scale and the second output feature using a first feature concatenation layer to determine a third output feature; extract features from the third output feature using a second residual network module to determine a fourth output feature; perform scale mapping on the fourth output feature using a second transposed convolutional layer to determine a fifth output feature at the (i-1)th scale; and perform feature concatenation on the reconstructed geometric data at the (i-1)th scale and the fifth output feature using a second feature concatenation layer to determine a first feature component in a first sparse tensor, wherein the first geometric component in the first sparse tensor is the geometric component corresponding to the (i-1)th scale.

[0322] In some embodiments, the feature enhancement module includes a multilayer perceptron module and a third feature concatenation layer; the second processing unit 2203 is further configured to perform feature transformation on the first geometric component in the first sparse tensor through the multilayer perceptron module to determine a sixth output feature; and to perform feature concatenation on the sixth output feature and the first feature component in the first sparse tensor through the third feature concatenation layer to determine a second feature component in the second sparse tensor, while the first geometric component in the second sparse tensor remains unchanged.

[0323] In some embodiments, the dual-view transformation module includes a dual-view sparse attention module, a third residual network module, and a neighbor point attention module; the second processing unit 2203 is further configured to perform coordinate system transformation and feature concatenation on the second sparse tensor through the dual-view sparse attention module to determine a seventh output feature; to extract features from the seventh output feature through the third residual network module to determine an eighth output feature; and to extract features from the eighth output feature through the neighbor point attention module to determine a third feature component in the third sparse tensor, while the first geometric component in the third sparse tensor remains unchanged.

[0324] In some embodiments, the dual-view sparse attention module includes a coordinate system transformation module, a first-view attention layer, a second-view attention layer, and a fourth feature concatenation layer; the second processing unit 2203 is further configured to perform coordinate system transformation on the first geometric component in the first coordinate system through the coordinate system transformation module to determine the first geometric component in the second coordinate system; based on the first geometric component in the first coordinate system and the first geometric component in the second coordinate system, split the second feature component in the second sparse tensor into a first-view feature component and a second-view feature component; perform feature extraction on the first-view feature component through the first-view attention layer to determine the ninth output feature, and perform feature extraction on the second-view feature component through the second-view attention layer to determine the tenth output feature; and perform feature concatenation on the ninth output feature and the tenth output feature through the fourth feature concatenation layer to determine the seventh output feature.

[0325] In some embodiments, the decoding unit 2202 is further configured to parse the model index parameters in the bitstream; the second determining unit 2201 is further configured to determine the neural network model based on the model index parameters.

[0326] In some embodiments, the decoding unit 2202 is further configured to parse the geometric decoding scale number of the current point cloud in the bit stream.

[0327] In some embodiments, the second determining unit 2201 is further configured to determine the reconstructed geometric data at the Nth scale when the value of i is equal to N; and to determine the reconstructed geometric data at the Nth scale as the geometrically reconstructed point cloud of the current point cloud.

[0328] Understandably, in the embodiments of this application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular one. Furthermore, the components in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional module.

[0329] In another embodiment of this application, FIG23 is a schematic diagram of the hardware structure of a decoder provided in an embodiment of this application. As shown in FIG23, the decoder 220 may include: a second communication interface 2301, a second memory 2302, and a second processor 2303; the various components are coupled together through a second bus system 2304. It is understood that the second bus system 2304 is used to realize the connection and communication between these components. In addition to a data bus, the second bus system 2304 also includes a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as the second bus system 2304 in FIG23.

[0330] The second communication interface 2301 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0331] The second memory 2302 is used to store computer programs that can run on the second processor 2303;

[0332] The second processor 2303 is configured to, when running the computer program, perform:

[0333] Determine at least one decoded scale of reconstructed geometric data for the current point cloud; input the at least one decoded scale of reconstructed geometric data into a neural network model, and output the estimated probability of the i-th scale of the current point cloud through the neural network model; decode the geometric sub-bitstream of the i-th scale in the bitstream according to the estimated probability of the i-th scale, and determine the reconstructed geometric data of the i-th scale of the current point cloud; where i takes the value greater than 1 and less than or equal to N, and N represents the number of geometric decoding scales of the current point cloud.

[0334] Alternatively, as another embodiment, the second processor 2303 is also configured to perform the method described in any of the foregoing embodiments when running the computer program.

[0335] It is understood that the second memory 2302 has similar hardware functions to the first memory 2102, and the second processor 2303 has similar hardware functions to the first processor 2103; these will not be described in detail here.

[0336] This embodiment provides a decoder in which reconstructed geometric data from at least one previously decoded scale is input into a neural network model to obtain an estimated probability for the current scale. This estimated probability is then used to decode the reconstructed geometric data at the current scale, and so on, from the lowest scale to the highest scale, thereby achieving the decoding of the current point cloud. In this way, the accuracy of the estimated probability can be improved through the neural network model, thereby improving the compression efficiency of the LiDAR point cloud and thus enhancing the decoding performance of the point cloud.

[0337] In another embodiment of this application, FIG24 is a schematic diagram of the composition structure of an encoding and decoding system provided in an embodiment of this application. As shown in FIG24, the encoding and decoding system 240 may include an encoder 2401 and a decoder 2402.

[0338] In this embodiment, encoder 2401 can be any of the encoders described in the foregoing embodiments, and decoder 2402 can be any of the decoders described in the foregoing embodiments.

[0339] In some embodiments, this application also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the method as described in any of the foregoing embodiments. Specifically, when executed by a first processor, the computer program implements the encoding method as described in any of the foregoing embodiments, or when executed by a second processor, it implements the decoding method as described in any of the foregoing embodiments.

[0340] In some embodiments, this application also provides a computer program product, including a computer program or instructions. When executed by a processor, the computer program or instructions implement the method as described in any of the foregoing embodiments. Specifically, when executed by a first processor, the computer program or instructions implement the encoding method as described in any of the foregoing embodiments, or when executed by a second processor, they implement the decoding method as described in any of the foregoing embodiments.

[0341] In some embodiments, this application also provides a computer program that, when executed by a processor, implements the method as described in any of the foregoing embodiments. Specifically, when executed by a first processor, the computer program or instructions implement the encoding method as described in any of the foregoing embodiments, or when executed by a second processor, implement the decoding method as described in any of the foregoing embodiments.

[0342] In some embodiments, this application also provides a computer-readable storage medium storing a bitstream thereon. The bitstream is generated by performing the steps of the encoding method as described in any of the foregoing embodiments.

[0343] In this embodiment, the information to be encoded in the encoding method may include at least one of the following: model index parameters corresponding to the neural network model, geometric data at multiple scales, and the number of geometric encoding scales. Here, this information to be encoded is encoded and written into the bitstream.

[0344] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0345] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0346] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0347] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0348] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0349] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0350] It should be noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0351] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0352] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0353] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0354] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0355] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims. Industrial applicability

[0356] In this embodiment, at the encoding end, at least one encoded scale of geometric data of the current point cloud is determined; the at least one encoded scale of geometric data is input into a neural network model, and the neural network model outputs an estimated probability of the i-th scale of the current point cloud; the geometric data of the i-th scale of the current point cloud is encoded according to the estimated probability of the i-th scale to generate a geometric sub-bitstream of the i-th scale in the bitstream. At the decoding end, at least one decoded scale of reconstructed geometric data of the current point cloud is determined; the at least one decoded scale of reconstructed geometric data is input into a neural network model, and the neural network model outputs an estimated probability of the i-th scale of the current point cloud; the geometric sub-bitstream of the i-th scale in the bitstream is decoded according to the estimated probability of the i-th scale to determine the reconstructed geometric data of the i-th scale of the current point cloud. In this way, whether at the encoding or decoding end, geometric data from at least one previously encoded / decoded scale can be input into the neural network model to obtain the estimated probability of the current scale. Then, the estimated probability is used to encode and decode the geometric data at the current scale, and so on, from the lowest scale to the highest scale, thereby realizing the encoding and decoding of the current point cloud. In this way, the accuracy of the estimated probability can be improved through the neural network model, thereby improving the compression efficiency of the LiDAR point cloud and thus improving the encoding and decoding performance of the point cloud.

Claims

1. A decoding method applied to a decoder, the method comprising: Determine at least one decoded scale of reconstructed geometry for the current point cloud; The reconstructed geometric data at the at least one decoded scale is input into a neural network model, and the estimated probability of the i-th scale of the current point cloud is output through the neural network model. The geometric sub-bitstream at the i-th scale is decoded based on the estimated probability of the i-th scale to determine the reconstructed geometric data of the current point cloud at the i-th scale; where i is greater than 1 and less than or equal to N, and N represents the number of geometric decoding scales of the current point cloud.

2. The method according to claim 1, wherein, The determination of at least one decoded scale of reconstructed geometric data for the current point cloud includes: When the value of i is equal to 2, the reconstructed geometric data of the at least one decoded scale is determined to include: the reconstructed geometric data of the first scale; When the value of i is equal to 3, the reconstructed geometric data of the at least one decoded scale is determined to include: the reconstructed geometric data of the first scale and the reconstructed geometric data of the second scale; When the value of i is greater than 3, the reconstructed geometric data of the at least one decoded scale includes: the reconstructed geometric data of the (i-3)th scale, the reconstructed geometric data of the (i-2)th scale, and the reconstructed geometric data of the (i-1)th scale.

3. The method according to claim 2, wherein, When the value of i is equal to 1, the method further includes: Decode the geometric sub-stream of the first scale in the bitstream to determine the reconstructed geometric data of the first scale.

4. The method according to claim 2, wherein, The neural network model includes a scale embedding module, at least one feature enhancement module, at least one dual-view transformation module, a transposed convolution module, and a classification module; The step of inputting the reconstructed geometric data of at least one decoded scale into a neural network model, and outputting the estimated probability of the i-th scale of the current point cloud through the neural network model, includes: The scale embedding module performs feature extraction and feature concatenation on the reconstructed geometric data of the at least one decoded scale, and outputs a first sparse tensor, which includes a first geometric component and a first feature component. The first sparse tensor is enhanced with positional features by the at least one feature enhancement module, and a second sparse tensor is output, wherein the second sparse tensor includes the first geometric component and the second feature component. The second sparse tensor is subjected to coordinate system transformation and feature stitching by the at least one dual-view transformation module to output a third sparse tensor, which includes the first geometric component and the third feature component. The transposed convolution module is used to perform transposed convolution processing on the third sparse tensor to output a fourth sparse tensor, which includes the second geometric component and the fourth feature component corresponding to the i-th scale. The classification module performs a high-dimensional feature transformation on the fourth sparse tensor and outputs a one-dimensional estimated probability of the i-th scale.

5. The method according to claim 4, wherein, The at least one feature enhancement module includes a first feature enhancement module and a second feature enhancement module, and the at least one dual-view transformation module includes a first dual-view transformation module and a second dual-view transformation module; The step of inputting the reconstructed geometric data of at least one decoded scale into a neural network model, and outputting the estimated probability of the i-th scale of the current point cloud through the neural network model, includes: The scale embedding module performs feature extraction and feature concatenation on the reconstructed geometric data of the at least one decoded scale, and outputs a first sparse tensor, which includes a first geometric component and a first feature component. The first sparse tensor is enhanced by the first feature enhancement module to perform position feature enhancement, and a second sparse tensor is output. The second sparse tensor includes the first geometric component and the second feature component. The first dual-view transformation module performs coordinate system transformation and feature stitching on the second sparse tensor to output a third sparse tensor, which includes the first geometric component and the third feature component. The third sparse tensor is enhanced with positional features by the second feature enhancement module, and a fifth sparse tensor is output, which includes the first geometric component and the fifth feature component. The transposed convolution module is used to perform transposed convolution processing on the fifth sparse tensor to output a sixth sparse tensor, which includes the second geometric component and the sixth feature component corresponding to the i-th scale. The second dual-view transformation module performs coordinate system transformation and feature stitching on the sixth sparse tensor to output a seventh sparse tensor, which includes the second geometric component and the seventh feature component. The classification module performs a high-dimensional feature transformation on the seventh sparse tensor and outputs a one-dimensional estimated probability of the i-th scale.

6. The method according to claim 4 or 5, wherein, The scale embedding module includes a first residual network module, a first transposed convolutional layer, a first feature concatenation layer, a second residual network module, a second transposed convolutional layer, and a second feature concatenation layer; The step of extracting and concatenating features from the reconstructed geometric data at at least one decoded scale using the scale embedding module, and outputting a first sparse tensor, includes: When the value of i is equal to 2, the first geometric component and the first feature component in the first sparse tensor are directly determined based on the reconstructed geometric data of the first scale. When the value of i is equal to 3, the reconstructed geometric data of the first scale and the reconstructed geometric data of the second scale are processed by the second residual network module, the second transposed convolutional layer and the second feature splicing layer in the scale embedding module to extract and splice features, and output the first geometric component and the first feature component in the first sparse tensor. When the value of i is greater than 3, the scale embedding module performs feature extraction and feature splicing on the reconstructed geometric data at the (i-3)th scale, the (i-2)th scale, and the (i-1)th scale, and outputs the first geometric component and the first feature component in the first sparse tensor.

7. The method according to claim 6, wherein, The step of extracting and concatenating features from the reconstructed geometric data at the (i-3)th, (i-2)th, and (i-1)th scales using the scale embedding module, and outputting the first geometric component and the first feature component in the first sparse tensor, includes: The first residual network module extracts features from the reconstructed geometric data at the (i-3)th scale to determine the first output feature, and the first transposed convolutional layer performs scale mapping on the first output feature to determine the second output feature at the (i-2)th scale. The first feature stitching layer stitches the reconstructed geometric data at the (i-2)th scale with the second output feature to determine the third output feature; The second residual network module extracts features from the third output feature to determine the fourth output feature, and the second transposed convolutional layer performs scale mapping on the fourth output feature to determine the fifth output feature at the (i-1)th scale. The reconstructed geometric data at the (i-1)th scale is concatenated with the fifth output feature by the second feature concatenation layer to determine the first feature component in the first sparse tensor, and the first geometric component in the first sparse tensor is the geometric component corresponding to the (i-1)th scale.

8. The method according to claim 4 or 5, wherein, The feature enhancement module includes a multilayer perceptron module and a third feature stitching layer; The step of enhancing the positional features of the first sparse tensor through the at least one feature enhancement module and outputting the second sparse tensor includes: The first geometric component in the first sparse tensor is transformed by the multilayer perceptron module to determine the sixth output feature. The sixth output feature is concatenated with the first feature component in the first sparse tensor by the third feature concatenation layer to determine the second feature component in the second sparse tensor, while the first geometric component in the second sparse tensor remains unchanged.

9. The method according to claim 4 or 5, wherein, The dual-view transformation module includes a dual-view sparse attention module, a third residual network module, and a neighbor point attention module. The step of performing coordinate system transformation and feature concatenation on the second sparse tensor through at least one dual-view transformation module to output a third sparse tensor includes: The second sparse tensor is subjected to coordinate system transformation and feature concatenation by the dual-view sparse attention module to determine the seventh output feature. The third residual network module is used to extract features from the seventh output feature to determine the eighth output feature; The neighbor point attention module extracts features from the eighth output feature to determine the third feature component in the third sparse tensor, while the first geometric component in the third sparse tensor remains unchanged.

10. The method according to claim 9, wherein, The dual-view sparse attention module includes a coordinate system transformation module, a first-view attention layer, a second-view attention layer, and a fourth feature splicing layer; The step of performing coordinate system transformation and feature concatenation on the second sparse tensor through the dual-view sparse attention module to determine the seventh output feature includes: The coordinate system transformation module is used to transform the first geometric component in the first coordinate system to determine the first geometric component in the second coordinate system. Based on the first geometric component in the first coordinate system and the first geometric component in the second coordinate system, the second feature component in the second sparse tensor is split into a first view feature component and a second view feature component. The first view feature component is extracted by the first view attention layer to determine the ninth output feature, and the second view feature component is extracted by the second view attention layer to determine the tenth output feature. The seventh output feature is determined by concatenating the ninth and tenth output features using the fourth feature concatenation layer.

11. The method according to any one of claims 1 to 10, wherein, The method further includes: Parse the model index parameters in the bitstream; The neural network model is determined based on the model index parameters.

12. The method according to any one of claims 1 to 10, wherein, The method further includes: The number of geometric decoding scales of the current point cloud described in the parsed bitstream.

13. The method according to any one of claims 1 to 10, wherein, The method further includes: When the value of i is equal to N, determine the reconstructed geometric data at the Nth scale; The reconstructed geometric data at the Nth scale is determined as the geometrically reconstructed point cloud of the current point cloud.

14. An encoding method applied to an encoder, the method comprising: Determine the geometry of at least one encoded scale of the current point cloud; The geometric data of at least one encoded scale is input into a neural network model, and the estimated probability of the i-th scale of the current point cloud is output by the neural network model. The geometric data of the current point cloud at the i-th scale is encoded based on the estimated probability of the i-th scale to generate a geometric sub-bitstream at the i-th scale in the bitstream; where i is greater than 1 and less than or equal to N, and N represents the number of geometric encoding scales of the current point cloud.

15. The method according to claim 14, wherein, The method further includes: The current point cloud is downsampled to determine geometric data at multiple scales; wherein the number of geometric encoding scales of the current point cloud is related to the number of downsampling operations.

16. The method according to claim 15, wherein, The method further includes: The current point cloud is downsampled using a pooling layer with a preset step size to determine the geometric data at multiple scales.

17. The method of claim 14, wherein, The method further includes: The geometric coding scale of the current point cloud is encoded, and the resulting encoded bits are written into the bitstream.

18. The method according to claim 14, wherein, When the value of i is equal to 1, the method further includes: The geometric data at the first scale is encoded to generate the geometric sub-stream at the first scale in the bitstream.

19. The method of claim 14, wherein, The determination of at least one encoded scale of geometric data for the current point cloud includes: When the value of i is equal to 2, the geometric data of the at least one encoded scale is determined to include: the geometric data of the first scale; When the value of i is equal to 3, the geometric data of the at least one encoded scale is determined to include: geometric data of the first scale and geometric data of the second scale; When the value of i is greater than 3, the geometric data of the at least one encoded scale includes: geometric data of the (i-3)th scale, geometric data of the (i-2)th scale, and geometric data of the (i-1)th scale.

20. The method according to claim 19, wherein, The neural network model includes a scale embedding module, at least one feature enhancement module, at least one dual-view transformation module, a transposed convolution module, and a classification module; The step of inputting the geometric data of at least one encoded scale into a neural network model, and outputting the estimated probability of the i-th scale of the current point cloud through the neural network model, includes: The scale embedding module performs feature extraction and feature concatenation on the geometric data of the at least one encoded scale, and outputs a first sparse tensor, which includes a first geometric component and a first feature component. The first sparse tensor is enhanced with positional features by the at least one feature enhancement module, and a second sparse tensor is output, wherein the second sparse tensor includes the first geometric component and the second feature component. The second sparse tensor is subjected to coordinate system transformation and feature stitching by the at least one dual-view transformation module to output a third sparse tensor, which includes the first geometric component and the third feature component. The transposed convolution module is used to perform transposed convolution processing on the third sparse tensor to output a fourth sparse tensor, which includes the second geometric component and the fourth feature component corresponding to the i-th scale. The classification module performs a high-dimensional feature transformation on the fourth sparse tensor and outputs a one-dimensional estimated probability of the i-th scale.

21. The method according to claim 20, wherein, The at least one feature enhancement module includes a first feature enhancement module and a second feature enhancement module, and the at least one dual-view transformation module includes a first dual-view transformation module and a second dual-view transformation module; The step of inputting the geometric data of at least one encoded scale into a neural network model, and outputting the estimated probability of the i-th scale of the current point cloud through the neural network model, includes: The scale embedding module performs feature extraction and feature concatenation on the geometric data of the at least one encoded scale, and outputs a first sparse tensor, which includes a first geometric component and a first feature component. The first sparse tensor is enhanced by the first feature enhancement module to perform position feature enhancement, and a second sparse tensor is output. The second sparse tensor includes the first geometric component and the second feature component. The first dual-view transformation module performs coordinate system transformation and feature stitching on the second sparse tensor to output a third sparse tensor, which includes the first geometric component and the third feature component. The third sparse tensor is enhanced with positional features by the second feature enhancement module, resulting in a fifth sparse tensor. The sparse tensor includes the first geometric component and the fifth eigencomponent; The transposed convolution module is used to perform transposed convolution processing on the fifth sparse tensor to output a sixth sparse tensor, which includes the second geometric component and the sixth feature component corresponding to the i-th scale. The second dual-view transformation module performs coordinate system transformation and feature stitching on the sixth sparse tensor to output a seventh sparse tensor, which includes the second geometric component and the seventh feature component. The classification module performs a high-dimensional feature transformation on the seventh sparse tensor and outputs a one-dimensional estimated probability of the i-th scale.

22. The method according to claim 20 or 21, wherein, The scale embedding module includes a first residual network module, a first transposed convolutional layer, a first feature concatenation layer, a second residual network module, a second transposed convolutional layer, and a second feature concatenation layer; The step of extracting and concatenating features from the geometric data of the at least one encoded scale through the scale embedding module to output a first sparse tensor includes: When the value of i is equal to 2, the first geometric component and the first feature component in the first sparse tensor are directly determined based on the geometric data of the first scale. When the value of i is equal to 3, the geometric data of the first scale and the geometric data of the second scale are extracted and concatenated through the second residual network module, the second transposed convolutional layer and the second feature concatenation layer in the scale embedding module, and the first geometric component and the first feature component in the first sparse tensor are output. When the value of i is greater than 3, the scale embedding module performs feature extraction and feature concatenation on the geometric data of the (i-3)th scale, the (i-2)th scale, and the (i-1)th scale, and outputs the first geometric component and the first feature component in the first sparse tensor.

23. The method according to claim 22, wherein, The step of extracting and concatenating features from the geometric data at the (i-3)th, (i-2)th, and (i-1)th scales using the scale embedding module, and outputting the first geometric component and the first feature component in the first sparse tensor, includes: The first residual network module extracts features from the geometric data at the (i-3)th scale to determine the first output feature, and the first transposed convolutional layer performs scale mapping on the first output feature to determine the second output feature at the (i-2)th scale. The first feature stitching layer stitches the geometric data at the (i-2)th scale with the second output feature to determine the third output feature; The second residual network module extracts features from the third output feature to determine the fourth output feature, and the second transposed convolutional layer performs scale mapping on the fourth output feature to determine the fifth output feature at the (i-1)th scale. The second feature splicing layer performs feature splicing on the geometric data at the (i-1)th scale and the fifth output feature to determine the first feature component in the first sparse tensor, and the first geometric component in the first sparse tensor is the geometric component corresponding to the (i-1)th scale.

24. The method according to claim 20 or 21, wherein, The feature enhancement module includes a multilayer perceptron module and a third feature stitching layer; The step of enhancing the positional features of the first sparse tensor through the at least one feature enhancement module and outputting the second sparse tensor includes: The first geometric component in the first sparse tensor is transformed by the multilayer perceptron module to determine the sixth output feature. The sixth output feature is concatenated with the first feature component in the first sparse tensor by the third feature concatenation layer to determine the second feature component in the second sparse tensor, while the first geometric component in the second sparse tensor remains unchanged.

25. The method according to claim 20 or 21, wherein, The dual-view transformation module includes a dual-view sparse attention module, a third residual network module, and a neighbor point attention module. The step of performing coordinate system transformation and feature concatenation on the second sparse tensor through at least one dual-view transformation module to output a third sparse tensor includes: The second sparse tensor is subjected to coordinate system transformation and feature concatenation by the dual-view sparse attention module to determine the seventh output feature. The third residual network module is used to extract features from the seventh output feature to determine the eighth output feature; The neighbor point attention module extracts features from the eighth output feature to determine the third feature component in the third sparse tensor, while the first geometric component in the third sparse tensor remains unchanged.

26. The method of claim 25, wherein, The dual-view sparse attention module includes a coordinate system transformation module, a first-view attention layer, a second-view attention layer, and a fourth feature splicing layer; The step of performing coordinate system transformation and feature concatenation on the second sparse tensor through the dual-view sparse attention module to determine the seventh output feature includes: The coordinate system transformation module is used to transform the first geometric component in the first coordinate system to determine the first geometric component in the second coordinate system. Based on the first geometric component in the first coordinate system and the first geometric component in the second coordinate system, the second feature component in the second sparse tensor is split into a first view feature component and a second view feature component. The first view feature component is extracted by the first view attention layer to determine the ninth output feature, and the second view feature component is extracted by the second view attention layer to determine the tenth output feature. The seventh output feature is determined by concatenating the ninth and tenth output features using the fourth feature concatenation layer.

27. The method according to any one of claims 14 to 26, wherein, The method further includes: Determine the model index parameters of the neural network model; The model index parameters are encoded, and the resulting encoded bits are written into the bitstream.

28. An encoder, the encoder comprising a first determining unit and an encoding unit, wherein: The first determining unit is configured to determine at least one encoded scale of geometric data of the current point cloud; And input the geometric data of the at least one encoded scale into the neural network model, and output the estimated probability of the i-th scale of the current point cloud through the neural network model; The encoding unit is configured to encode the geometric data of the current point cloud at the i-th scale according to the estimated probability of the i-th scale, and generate a geometric sub-code stream at the i-th scale in the code stream; wherein the value of i is greater than 1 and less than or equal to N, and N represents the number of geometric encoding scales of the current point cloud.

29. An encoder, the encoder comprising a first memory and a first processor, wherein: The first memory is used to store computer programs that can run on the first processor; The first processor is configured to perform the method as described in any one of claims 14 to 27 when running the computer program.

30. A decoder, the decoder comprising a second determining unit and a decoding unit, wherein: The second determining unit is configured to determine at least one decoded scale of reconstructed geometric data of the current point cloud; and to input the at least one decoded scale of reconstructed geometric data into a neural network model, and output the estimated probability of the i-th scale of the current point cloud through the neural network model; The decoding unit is configured to decode the geometric sub-bitstream at the i-th scale in the bitstream according to the estimated probability of the i-th scale, and determine the reconstructed geometric data of the current point cloud at the i-th scale; wherein the value of i is greater than 1 and less than or equal to N, and N represents the number of geometric decoding scales of the current point cloud.

31. A decoder, the decoder comprising a second memory and a second processor, wherein: The second memory is used to store computer programs that can run on the second processor; The second processor is configured to perform the method as described in any one of claims 1 to 13 when running the computer program.

32. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 13, or the method as described in any one of claims 14 to 27.

33. A computer-readable storage medium having a bitstream stored thereon, wherein, The bitstream is generated by performing the steps of the encoding method as described in any one of claims 14 to 27.

Images