A method and apparatus for geometry recovery of compressed point clouds

Abstract

The application relates to a compressed point cloud geometry recovery method and device, which simultaneously extracts and fuses deep and shallow geometric point cloud features, can comprehensively understand the geometric shape of the point cloud, introduces a candidate anchor point based on a generative sparse convolution, can adaptively generate anchor points according to the geometric features of the point cloud itself, and can more accurately locate the key area of geometry recovery, and most importantly, uses deep geometric point cloud features to predict a continuous three-dimensional offset vector, guides the adaptive projection of the candidate anchor point, breaks the limitation of traditional upsampling that can only perform static point filling at fixed discrete positions, dynamically couples the offset amount predicted based on the deep geometric point cloud features and the projection depth of the shallow geometric point cloud features, adaptively locates the newly added points in the continuous space through the offset amount, the recovered point cloud can more smoothly and naturally fill the geometric loss, and the geometric precision and visual quality of the recovered point cloud are significantly improved.

Description

Technical Field

[0001] This application relates to the field of point cloud restoration technology, and in particular to a method and apparatus for geometric restoration of compressed point clouds. Background Technology

[0002] Point clouds, as a representation of discrete 3D point sets, can intuitively depict the spatial geometric topology of objects or scenes and integrate multi-dimensional attribute information such as color and reflectivity. They are currently the core data carrier for 3D perception and content modeling, playing an irreplaceable role in key areas such as autonomous driving environmental perception, virtual reality interaction, and 3D scene reconstruction. With the rapid popularization of devices such as LiDAR, RGB-D cameras, and 3D scanners, the scale of point cloud data acquisition has exploded. Their high-dimensionality results in massive raw data volumes, placing enormous pressure on real-time transmission bandwidth and long-term storage resources, significantly increasing communication latency and drastically raising data management costs. Against this backdrop, efficient point cloud compression technology has become an indispensable supporting link in the entire 3D application chain, aiming to balance the contradiction between data compression rate and information fidelity.

[0003] The G-PCC standard, developed by the MPEG international standardization organization, is a widely adopted general point cloud compression scheme. It maps continuous spatial point clouds to discrete integer voxel grids through voxelization, and combines octree occupancy bitstream coding, attribute transformation, and entropy coding to achieve data compression, exhibiting good cross-platform compatibility and engineering deployment flexibility. However, in lossy compression mode, the spatial quantization operation during voxelization and the hierarchical simplification mechanism of octree coding inevitably introduce geometric information loss, resulting in surface roughening, blurred details, and local structural distortion in the decoded point cloud. This geometric distortion severely restricts the accuracy of subsequent 3D analysis tasks. Therefore, it is urgent to develop a technical solution that can achieve high-fidelity restoration of the geometric information of decoded point clouds while strictly maintaining G-PCC standard bitstream compatibility or minimizing modifications to the decoding link.

[0004] Existing compressed point cloud geometric reconstruction techniques mainly fall into two categories: traditional optimization methods and deep learning methods. Traditional methods rely on manually designed energy optimization functions for iterative solutions, resulting in low computational efficiency and difficulty in adapting to complex geometric shapes. Deep learning methods attempt to model the reconstruction process using neural networks, with mainstream approaches employing a fragmented processing paradigm of first performing discrete upsampling to generate candidate points, and then superimposing coordinate offsets on fixed-position points. This type of method has an inherent flaw: its discrete upsampling operation can only perform static point filling at preset integer voxel grid positions, and cannot dynamically adjust the spatial distribution of candidate points according to the local geometric features of the point cloud. This leads to the generated point set exhibiting non-uniformity in continuous three-dimensional space, making it difficult to accurately match the surface continuity and detailed features of the original point cloud, ultimately limiting the quality of geometric reconstruction. Summary of the Invention

[0005] To achieve the above objective, a first aspect of the present invention provides a method for geometric restoration of compressed point clouds, the method comprising: Obtain the first point cloud; wherein, the first point cloud is a point cloud decoded by G-PCC; Extract the deep geometric point cloud features of the first point cloud, and extract the shallow geometric point cloud features of the first point cloud. The deep geometric point cloud features and the shallow geometric point cloud features are concatenated to obtain geometric point cloud fusion features, and the geometric point cloud fusion features are excited based on generative sparse convolution to obtain multiple candidate anchor points on a discrete integer voxel grid. Based on the deep geometric point cloud features, predict the continuous three-dimensional offset vector of the first point cloud, and guide the adaptive projection of the multiple candidate anchor points on the discrete integer voxel grid based on the continuous three-dimensional offset vector to obtain the candidate point set; The second point cloud is recovered from the set of candidate points.

[0006] The geometric recovery method for compressed point clouds provided in the first aspect of this application has at least the following beneficial effects: First, this method simultaneously extracts and fuses deep and shallow geometric point cloud features. Deep features provide overall information, while shallow features focus on local details. This fusion of multi-level features enables a comprehensive understanding of the point cloud's geometric morphology, avoiding local optima or global distortion that may result from a single feature. Second, it introduces candidate anchor points based on generative sparse convolution, which can adaptively generate anchor points according to the point cloud's own geometric features, thereby more accurately locating key areas for geometric restoration and improving the relevance of the restoration. Third, it uses deep geometric point cloud features to predict continuous 3D offset vectors and guides candidate anchor points to perform adaptive projection, completely breaking the limitation of traditional upsampling which can only perform static point filling at fixed discrete positions. It deeply couples the dynamic offset predicted based on deep geometric point cloud features with the projection of shallow geometric point cloud features, guiding the newly added points to adaptively locate in continuous space through the offset. The restored point cloud can fill geometric gaps more smoothly and naturally, significantly improving the geometric accuracy and visual quality of the restored point cloud.

[0007] A second aspect of this application provides a geometric recovery apparatus for compressed point clouds, the apparatus comprising: A point cloud acquisition unit is used to acquire a first point cloud; wherein the first point cloud is a point cloud after G-PCC lossy compression; The feature extraction unit is used to extract the deep geometric point cloud features of the first point cloud and the shallow geometric point cloud features of the first point cloud. Anchor point extraction unit is used to concatenate the deep geometric point cloud features and the shallow geometric point cloud features to obtain geometric point cloud fusion features, and to excite the geometric point cloud fusion features based on generative sparse convolution to obtain multiple candidate anchor points on a discrete integer voxel grid. The point set acquisition unit is used to predict the continuous three-dimensional offset vector of the first point cloud based on the deep geometric point cloud features, and guide the adaptive projection of the multiple candidate anchor points on the discrete integer voxel grid based on the continuous three-dimensional offset vector to obtain the candidate point set. A point cloud recovery unit is used to recover a second point cloud from the candidate point set.

[0008] To achieve the above objectives, a third aspect of the present invention provides an electronic device, comprising: at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, the instructions being executed by the at least one control processor to enable the at least one control processor to perform the above-described geometric restoration method for compressed point clouds.

[0009] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the above-described geometric restoration method for compressed point clouds.

[0010] It is understood that the beneficial effects of the second to fourth aspects compared with the related technologies are the same as the beneficial effects of the first aspect compared with the related technologies. Please refer to the relevant description in the first aspect above, which will not be repeated here. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of a geometric restoration method for compressed point clouds provided in an embodiment of this application; Figure 2 This is a schematic diagram of the point cloud recovery model provided in the embodiments of this application; Figure 3 This is a schematic diagram of the feature extraction and offset prediction module and the deformable sparse projection upsampling module provided in the embodiments of this application; Figure 4 yes Figure 3 A schematic diagram of the reparameterized Inception block; Figure 5 This is a structural diagram of a geometric restoration device for compressed point clouds provided in an embodiment of this application; Figure 6 This is a schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0014] like Figure 1 As shown in the embodiments of this application, a geometric restoration method for compressed point clouds is provided, the method comprising: Step S110: Obtain the first point cloud; wherein, the first point cloud is the point cloud decoded by G-PCC; Step S130: Extract the deep geometric point cloud features of the first point cloud and extract the shallow geometric point cloud features of the first point cloud. Step S150: Concatenate the deep geometric point cloud features and the shallow geometric point cloud features to obtain the geometric point cloud fusion features, and excite the geometric point cloud fusion features based on generative sparse convolution to obtain multiple candidate anchor points on the discrete integer voxel grid. Step S170: Predict the continuous three-dimensional offset vector of the first point cloud based on the features of the deep geometric point cloud, and guide the adaptive projection of multiple candidate anchor points on the discrete integer voxel grid based on the continuous three-dimensional offset vector to obtain the candidate point set. Step S190: Recover the second point cloud from the candidate point set.

[0015] For ease of understanding, the following explains some key terms in this embodiment: The first point cloud specifically refers to the point cloud (the set of coordinates reconstructed by G-PCC) after decoding using the G-PCC (Geometry-based Point Cloud Compression) standard. G-PCC is an international standard widely used for point cloud data compression. Its decoding process may introduce geometric distortion, resulting in deviations in local details of the decoded point cloud.

[0016] Generative sparse convolution is a convolutional operation specifically designed for processing sparse data, such as sparse voxel meshes derived from voxelization of point clouds. In this method, generative sparse convolution is used to generate candidate anchor points from fused features of geometric point clouds.

[0017] Discrete integer voxel grids are mesh structures formed by dividing three-dimensional space into a series of tiny cubic elements with integer coordinates. Point cloud data can typically be mapped onto this discrete integer voxel grid, where each voxel represents whether a point exists within it. The grid provides the foundation for the discrete representation and processing of point clouds.

[0018] Candidate anchor points refer to a set of potential point locations obtained by generative sparse convolution on a discrete integer voxel grid. These anchor points serve as the starting point for the geometric reconstruction process, representing regions where geometric details may exist. Subsequent fine-tuning will be based on these anchor points to generate the final reconstructed point cloud.

[0019] A candidate point set refers to a series of potential points with continuous floating-point coordinates, obtained through adaptive projection operations. This point set is the source of the final second point cloud, which contains richer geometric details and more precise spatial location information than the first point cloud.

[0020] The second point cloud refers to the point cloud that is finally recovered from the candidate point set after screening and optimization.

[0021] First, there are several ways to obtain the first point cloud in step S110. For example, it can receive the G-PCC decoded point cloud data stream from a remote server via a network interface, or read a pre-decoded and saved point cloud file from a local storage device. In some scenarios, the compressed bitstream can also be decoded in real time by directly calling the G-PCC decoder, and the decoding result can be used as the first point cloud.

[0022] Step S130 extracts the deep geometric point cloud features and the shallow geometric point cloud features of the first point cloud. Extracting these features is a crucial step in understanding the geometric structure of the point cloud. Deep geometric point cloud features can be extracted based on feature pyramids or U-net networks. Different scales of geometric point cloud features are extracted through layer-by-layer downsampling, followed by layer-by-layer upsampling and concatenation with the geometric point cloud features extracted in the downsampling stage, resulting in multi-scale deep geometric point cloud features. Shallow geometric point cloud features can be extracted using sparse convolution.

[0023] Step S150 concatenates the deep and shallow geometric point cloud features to obtain a fused geometric point cloud feature. This fused feature is then used to generate multiple candidate anchor points on a discrete integer voxel grid based on generative sparse convolution. The initial point positions of the candidate anchor points are generated on the discrete integer voxel grid. Feature concatenation can be achieved by concatenating feature vectors, combining deep and shallow features into a longer feature vector. When generating candidate anchor points, a sparse convolutional network can be designed, whose output layer directly predicts which voxel positions on the preset discrete integer voxel grid might contain new geometric points. For example, the network can output a binary activation value for each voxel, indicating whether the voxel is selected as a candidate anchor point.

[0024] Step S170 predicts the continuous three-dimensional offset vector of the first point cloud based on the features of the deep geometric point cloud, and guides the adaptive projection of multiple candidate anchor points on the discrete integer voxel grid based on the continuous three-dimensional offset vector to obtain the candidate point set.

[0025] Predicting continuous 3D offset vectors aims to remove anchor points from an integer grid to return them to a true continuous surface. This prediction can be achieved using a separate regression network or a multilayer perceptron (MLP). The network takes deep geometric point cloud features as input and outputs a 3D floating-point offset for each point or voxel position. During the adaptive projection stage, the integer coordinates of each candidate anchor point can be directly added to the predicted continuous 3D offset vector to obtain candidate points with floating-point coordinates. This direct addition method transforms discrete anchor point positions into more refined continuous spatial positions.

[0026] Step S190 reconstructs the second point cloud from the candidate point set. Various strategies can be employed to reconstruct the second point cloud. For example, all candidate points obtained through adaptive projection can be directly used as the second point cloud. Alternatively, a distance threshold can be set based on the distance between each candidate point and its nearest point in the first point cloud, retaining only candidate points with a distance less than the threshold. Another approach is to perform cluster analysis on the candidate point set and select representative points from each cluster as the second point cloud. All these methods can generate the final reconstructed point cloud from the candidate point set.

[0027] The following example will provide a more detailed explanation of the above technical solution: Suppose we need to perform geometric reconstruction on a point cloud of a building model obtained by lossy compression and decoding using G-PCC.

[0028] First, the point cloud of the building model after G-PCC decoding is regarded as the first point cloud. Its deep geometric point cloud features and shallow geometric point cloud features are extracted. The deep geometric point cloud features can provide an understanding of the overall shape of the building, while the shallow geometric point cloud features focus on local details.

[0029] Subsequently, the extracted deep geometric point cloud features and shallow geometric point cloud features are concatenated to form a more comprehensive geometric point cloud fusion feature.

[0030] The geometric point cloud fusion features are then input into a generative sparse convolutional network. Based on the geometric point cloud fusion features, the network activates multiple candidate anchor points on a predefined discrete integer voxel grid. For example, in areas such as window frames and door edges of architectural models where details may be missing, the network will determine that these areas need to be supplemented with points based on the fusion features and generate candidate anchor points at the corresponding voxel positions. These candidate anchor points represent potential locations that the system believes may need to be filled or corrected for geometric details.

[0031] Based on this, the extracted deep geometric point cloud features are used to predict the continuous three-dimensional offset vectors of the first point cloud. For example, for a recessed wall area in an architectural model, the deep geometric point cloud features may indicate that the entire area needs to protrude outward by a certain distance. The prediction network will generate a three-dimensional offset vector for each voxel or point in the area, accurately indicating the direction and distance it should move. Subsequently, these continuous three-dimensional offset vectors are used to guide the candidate anchor points on the discrete integer voxel grid for adaptive projection. Specifically, the integer coordinates of each candidate anchor point are added to its corresponding predicted three-dimensional offset vector, thereby accurately moving these discrete anchor points to continuous floating-point coordinate positions, forming a candidate point set containing a large number of potential new points. For example, a candidate anchor point located at voxel (10, 20, 30) will be projected to floating-point coordinates (10.1, 19.95, 30.2) if the predicted three-dimensional offset vector is (0.1, -0.05, 0.2).

[0032] Finally, the candidate point set can be initially screened to remove some obviously outliers or redundant points. Then, the remaining candidate points are merged with the first point cloud to form the final second point cloud. Compared with the first point cloud, the window edges of the building model in the second point cloud will be sharper and the wall surfaces will be smoother, effectively solving the geometric distortion problem caused by lossy compression of G-PCC.

[0033] In contrast, the method in this embodiment achieves technological advancement through the following key points: First, this method simultaneously extracts and fuses deep geometric point cloud features and shallow geometric point cloud features. Deep features provide overall information, while shallow features focus on local details. This fusion of multi-level features enables the system to fully understand the geometric shape of the point cloud and avoids local optima or global distortion that may be caused by a single feature. Secondly, this method introduces candidate anchor points based on generative sparse convolution, which can adaptively generate anchor points according to the geometric features of the point cloud itself, thereby more accurately locating the key areas of geometric restoration and improving the targeting of restoration. Importantly, this method uses deep geometric point cloud features to predict continuous 3D offset vectors and guides candidate anchor points to perform adaptive projection. This method completely breaks the limitation of traditional upsampling, which can only perform static point filling at fixed discrete positions. It deeply couples the dynamic offset predicted based on deep geometric point cloud features with the projection depth of shallow geometric point cloud features. By using the offset to guide the newly added points to perform adaptive positioning in continuous space, the recovered point cloud can fill geometric gaps more smoothly and naturally, significantly improving the geometric accuracy and visual quality of the recovered point cloud.

[0034] In summary, the geometric restoration method of this embodiment effectively solves the geometric distortion problem of G-PCC lossy compressed point clouds by integrating multi-level geometric features, adaptively generating candidate anchor points, and predicting continuous 3D offset vectors and guiding adaptive projection. The method can generate high-fidelity restored point clouds. Compared with existing technologies, it has achieved significant improvements in the accuracy and flexibility of point cloud geometric restoration, providing a more reliable data foundation for the fields of 3D perception and content modeling.

[0035] In some embodiments of this application, step S140, which involves adaptively projecting multiple candidate anchor points onto a discrete integer voxel mesh based on a continuous three-dimensional offset vector to obtain a candidate point set, includes: Step S410: Summate the three-dimensional coordinates of each candidate anchor point with the corresponding three-dimensional offset vector in the continuous three-dimensional offset vector to obtain continuous floating-point coordinates, and use the continuous floating-point coordinates as the candidate point set; Step S150, recovering the second point cloud from the candidate point set, includes: Step S510: Calculate the spatial contribution weight of each floating-point coordinate in the continuous floating-point coordinates to the integer grid points in the corresponding neighborhood, according to the distance decay rule of trilinear interpolation. Step S520: Based on the spatial contribution weight of floating-point coordinates to integer grid points and the shallow geometric point cloud features corresponding to the floating-point coordinates, the aggregated features of integer grid points are obtained. Step S530: Based on the aggregation features of each integer grid point, predict the occupancy probability of each integer grid point, and assign the occupancy probability of each integer grid point to the corresponding floating-point coordinate; Step S540: Based on the occupancy probability, select multiple floating-point coordinates from the continuous floating-point coordinates to form a second point cloud.

[0036] Step S410 refines the candidate anchor points on the discrete integer voxel grid using predicted continuous 3D offset vectors, changing their positions from integer coordinates to more precise floating-point coordinates. This adjustment allows the geometric position of the point cloud to break free from the limitations of the voxel grid and more closely approximate the continuous geometric information of the first point cloud. This is achieved by adding the 3D coordinates of each candidate anchor point on the discrete integer voxel grid component-by-component to its corresponding continuous 3D offset vector, resulting in new 3D floating-point coordinates. These floating-point coordinates collectively constitute the candidate point set.

[0037] Step S510 calculates the spatial contribution weight of each continuous floating-point coordinate to its corresponding neighborhood integer grid points according to the distance decay rule of trilinear interpolation. This step aims to quantify the influence of each continuous floating-point coordinate on its surrounding discrete integer grid points. Trilinear interpolation is a commonly used interpolation method that assigns corresponding weights based on the distance between the floating-point coordinate and its eight surrounding integer grid points; the closer the distance, the greater the contribution weight. This helps map information in continuous space onto discrete grids, preparing for subsequent feature aggregation and occupancy probability prediction. One implementation is to calculate the normalized distance from the floating-point coordinate to each of the eight integer grid points, and then calculate eight weights based on these distances, ensuring that the sum of all weights is 1. Another implementation is to use a Gaussian kernel function or other smooth kernel functions to calculate the distance decay weights. The parameters of the kernel function can be adjusted according to the density of the point cloud or the required recovery accuracy to more flexibly control the range and intensity of the spatial contribution.

[0038] The purpose of step S520 is to effectively aggregate the shallow geometric feature information carried by continuous floating-point coordinates onto these integer grid points through their spatial contribution weights to surrounding integer grid points. In this way, each integer grid point can gather the feature information of all floating-point coordinates in its neighborhood, forming a more representative aggregated feature, providing rich context for subsequent occupancy probability prediction. One implementation is to iterate through all floating-point coordinates in its neighborhood for each integer grid point, multiplying the shallow geometric point cloud feature of each floating-point coordinate by its spatial contribution weight to the current integer grid point, and then summing all these weighted features to obtain the aggregated feature of the integer grid point. Another implementation is to use an attention mechanism, allowing floating-point coordinates to consider not only distance but also the similarity of the features themselves when contributing features, thus more intelligently aggregating features.

[0039] Step S530 utilizes aggregated features to determine the probability that each integer grid point is occupied by the point cloud, and maps this probability (occupancy probability) back to the floating-point coordinates in its neighborhood. This provides crucial confidence information for subsequently selecting high-quality second point clouds. One implementation is to input the aggregated features of each integer grid point into a small neural network, which outputs a value between 0 and 1, representing the occupancy probability of the grid point; then, this occupancy probability is assigned to all floating-point coordinates that contribute to the integer grid point. Another implementation is to use conditional random fields or graph neural networks to model the spatial relationships between integer grid points, thus considering neighborhood information when predicting the occupancy probability, making the prediction results more robust and consistent.

[0040] Step S540 filters out points considered to be truly existing based on the occupancy probability associated with each floating-point coordinate by setting a threshold or using a sorting selection method, thereby removing noise and redundancy and obtaining a high-quality reconstructed point cloud. One implementation is to set a preset occupancy probability threshold. All floating-point coordinates with an occupancy probability higher than the threshold are retained to form the second point cloud. Another implementation is to sort all floating-point coordinates according to their occupancy probability from high to low, and then select the top N floating-point coordinates as the second point cloud, where N can be a preset number of reconstructed point clouds, or dynamically determined through other methods.

[0041] In this embodiment, firstly, by summing the 3D coordinates of candidate anchor points on a discrete integer voxel grid with the predicted continuous 3D offset vector, these anchor points are precisely projected from the discrete grid space to the continuous floating-point coordinate space, thus forming a candidate point set containing more geometric details. This process makes the geometric position of the point cloud no longer limited by the granularity of the voxel grid, laying the foundation for subsequent fine-grained reconstruction. Subsequently, to evaluate the authenticity of these continuous floating-point coordinates, this embodiment uses the distance decay rule of trilinear interpolation to calculate the spatial contribution weight of each floating-point coordinate to its surrounding neighboring integer grid points. This weight calculation method ensures that closer floating-point coordinates have a greater impact on the grid points, thereby effectively mapping information in the continuous space to the discrete grid. Based on this, each floating-point coordinate is further analyzed. The shallow geometric point cloud features of point coordinates are weighted and aggregated onto the corresponding integer grid points, forming aggregated features for each integer grid point. These aggregated features integrate the geometric information and spatial distribution of floating-point coordinates in the neighborhood, providing a comprehensive basis for determining the occupancy status of grid points. Next, based on the aggregated features of each integer grid point, the probability of a grid point being occupied by the point cloud is predicted. This occupancy probability reflects the likelihood of a real point cloud existing at the grid point. To associate this probability information with continuous floating-point coordinates, the occupancy probability of each integer grid point is assigned to all contributing floating-point coordinates. Finally, based on these assigned occupancy probabilities, floating-point coordinates with higher confidence are selected from the continuous floating-point coordinates to form the final second point cloud. Through this series of steps, the embodiment can effectively remove noise and redundancy from the initially generated candidate point set, recovering a second point cloud with higher geometric accuracy and better quality, solving the problem that projection may lead to poor point cloud quality.

[0042] In some embodiments of this application, before step S190, which involves recovering the second point cloud from the candidate point set, the method further includes: Step S1810: Extract the context representation vector from the first point cloud; Step S1820: Predict the target recovery quantity based on the context representation vector; Step S540, which involves selecting multiple floating-point coordinates from continuous floating-point coordinates to form a second point cloud based on the occupancy probability, includes: Step S5410: Sort the continuous floating-point coordinates according to the occupancy probability to obtain the sorting result; Step S5420: Based on the sorting results and the number of target recoveries, select multiple floating-point coordinates to form a second point cloud.

[0043] Step S1810 acquires a compact representation describing the overall or local features of the first point cloud, which includes high-level semantic information such as the structure, density, and distribution of the point cloud. Its purpose is to provide global or local perceptual information for subsequent target recovery, enabling the recovery process to better adapt to the characteristics and needs of different point clouds. One implementation is to aggregate the deep feature maps or fused feature maps output by the encoder into a fixed-dimensional vector through global pooling (such as max pooling or average pooling). Another implementation is to process the original coordinates or low-level features of the first point cloud using a separate neural network module (such as a multilayer perceptron, MLP) to extract representative contextual information.

[0044] Step S1820 refers to determining the number of second point clouds that are ultimately desired to be recovered from the candidate point set. The purpose of predicting the number is to provide a clear quantitative target for the point cloud recovery process, ensuring that the recovered point cloud has the expected density or size, thereby improving the quality and controllability of the recovered point cloud. One implementation is to input the extracted context representation vector into a regression network (such as one or more fully connected layers), and the network outputs a scalar value, which is the target recovery number. Another implementation is to input the context representation vector into a classification network, and the network outputs the probability distribution of different point count intervals. Then, the median of the interval with the highest probability is selected as the target recovery number, or a discrete preset number of points is directly output.

[0045] Step S5410 prioritizes each floating-point coordinate based on its occupancy probability. A higher occupancy probability indicates that the floating-point coordinate is more likely to belong to the actual geometric surface. Sorting facilitates the selection of the most likely points, ensuring the quality of the recovered point cloud. One implementation method is to use standard sorting algorithms such as quicksort or mergesort to arrange all consecutive floating-point coordinates according to their corresponding occupancy probabilities from highest to lowest. Another implementation method is to use partial sorting algorithms, such as selection sort or heapsort, to find only the top K points with the highest occupancy probabilities, without needing to completely sort all points, thus improving efficiency.

[0046] Step S5420 is a crucial step in determining the composition of the second point cloud. By combining the ranking results and the target recovery number (the desired number of points), the most suitable points can be accurately selected from a large number of candidate points to form a high-quality second point cloud. One implementation is to directly select the N floating-point coordinates with the highest occupancy probability from the ranking results, where N is the predicted target recovery number. Another implementation is to combine a dynamic threshold adjustment mechanism with the selection of the top N points. For example, if the occupancy probability of the top N points is very low, the value of N can be adjusted appropriately or additional quality assessment can be introduced to avoid recovering low-quality points.

[0047] This embodiment extracts a context representation vector from the first point cloud. This vector captures the overall geometric structure and density information of the first point cloud. Based on this context representation vector, a target recovery quantity can be predicted, i.e., the number of points in the second point cloud to be recovered. Simultaneously, after generating continuous floating-point coordinates and calculating their occupancy probabilities, these continuous floating-point coordinates are sorted according to their occupancy probabilities, resulting in a sorting result that reflects the geometric confidence of the points. Finally, combining the sorting result with the pre-predicted target recovery quantity, a specified number of points with the highest occupancy probabilities can be accurately selected from the sorted continuous floating-point coordinates to form the second point cloud. This mechanism ensures that the recovered point cloud not only has high geometric accuracy, but its quantity and density can also be adaptively adjusted according to the characteristics of the input point cloud or preset requirements. This effectively solves the problem of uncontrolled point cloud quantity and unstable quality that may occur when selecting points solely based on occupancy probabilities, significantly improving the robustness and accuracy of point cloud geometric recovery.

[0048] In some embodiments of this application, step S130, extracting the deep geometric point cloud features of the first point cloud, includes: S310 is an encoder-decoder architecture that conforms to the feature pyramid, based on sparse convolution. Step S320: Extract deep geometric point cloud features of the first point cloud according to the encoder-decoder architecture.

[0049] Specifically, sparse convolution (3D) is a convolution operation specifically designed for processing sparse data. When point cloud data is converted into a voxel mesh representation, most voxels in the voxel mesh are empty due to the sparsity of the point cloud. Sparse convolution only computes on non-empty voxels and their neighborhoods, thus significantly reducing computational cost and memory consumption, while effectively capturing local features in 3D space.

[0050] A feature pyramid is a multi-scale feature representation structure that extracts features at different resolution levels, enabling the network to simultaneously capture both fine local details and macro-global structure of a point cloud. Lower-level feature pyramids typically contain high-resolution detail information, while higher-level feature pyramids contain low-resolution but more semantic contextual information. This structure can be constructed through a series of downsampling operations to generate feature maps at different scales.

[0051] The encoder-decoder architecture is a neural network structure used for feature extraction and reconstruction tasks. The encoder part typically consists of a series of sparse convolutional layers and downsampling layers, which are used to progressively extract high-level abstract features of the input point cloud and reduce the spatial size of the feature map. The decoder part consists of a series of sparse deconvolutional layers and sparse convolutional layers, which are used to progressively restore the abstract features extracted by the encoder to the original spatial resolution and generate the required deep geometric point cloud features.

[0052] Skip connections are typically introduced between the encoder and decoder to directly pass features from different levels in the encoder to the corresponding levels in the decoder. This helps the decoder to better recover detailed information and avoids losing important spatial information during downsampling.

[0053] This embodiment organically combines sparse convolution, feature pyramids, and an encoder-decoder architecture to form an efficient and robust deep geometric point cloud feature extraction mechanism. The encoder-decoder architecture leverages the characteristics of sparse convolution to perform efficient feature learning directly on sparse point cloud data, avoiding computational redundancy and information loss caused by converting sparse point clouds into dense representations. Simultaneously, the embedded feature pyramid structure enables the network to understand the geometric information of the point cloud at multiple scales. The encoder progressively extracts deep features with global contextual information, while the decoder utilizes these deep features and combines them with multi-scale detail information from the encoder to progressively recover and refine the final deep geometric point cloud features. This multi-scale, sparse-aware feature extraction method can more comprehensively and accurately capture the geometric structure of the first point cloud, providing high-quality input for the prediction of subsequent continuous 3D offset vectors, thereby effectively improving the accuracy of point cloud geometric reconstruction.

[0054] In some embodiments of this application, step S320 extracts deep geometric point cloud features of the first point cloud according to the encoder-decoder architecture, including: Step S3211: Extract the first intermediate geometric point cloud features from the first point cloud based on sparse convolution; Step S3212: Extract the second intermediate geometric point cloud features from the first intermediate geometric point cloud features based on two cascaded reparameterized Inception blocks; Step S3213: Based on sparse convolution, the second intermediate geometric point cloud features are downsampled to obtain the third intermediate geometric point cloud features; Step S3214: Extract the fourth intermediate geometric point cloud features from the third intermediate geometric point cloud features based on two cascaded reparameterized Inception blocks. Step S3215: Based on the sparse convolution downsampling of the fourth intermediate geometric point cloud features, the fifth intermediate geometric point cloud features are obtained; Step S3216: Extract the sixth intermediate geometric point cloud features from the fifth intermediate geometric point cloud features based on two cascaded reparameterized Inception blocks. Step S3217: Extract the seventh intermediate geometric point cloud features from the sixth intermediate geometric point cloud features based on sparse convolution. Step S3218: Extract the eighth intermediate geometric point cloud features from the seventh intermediate geometric point cloud features based on two cascaded reparameterized Inception blocks. Step S3219: Extract the ninth intermediate geometric point cloud features from the eighth intermediate geometric point cloud features based on sparse deconvolution. Step S3220: Extract the tenth intermediate geometric point cloud feature from the ninth intermediate geometric point cloud feature and the fourth intermediate geometric point cloud feature based on two cascaded reparameterized Inception blocks. Step S3221: Extract the eleventh intermediate geometric point cloud features from the tenth intermediate geometric point cloud features based on sparse deconvolution. Step S3222: Based on two cascaded reparameterized Inception blocks, deep geometric point cloud features are extracted from the eleventh intermediate geometric point cloud features and the second intermediate geometric point cloud features.

[0055] The reparameterized Inception block includes a first branch and a second branch. The first branch includes cascaded reparameterized convolutions, activation functions, reparameterized convolutions, activation functions, and a concatenation block. The second branch includes cascaded 1×1×1 sparse convolutions, activation functions, reparameterized convolutions, activation functions, and a 1×1×1 sparse convolution. The output of the second branch is connected to the input of the concatenation block. The output feature of the concatenation block is added to the input feature of the reparameterized Inception block to obtain the output feature of the reparameterized Inception block. Reparameterized convolutions include bottleneck sequence branches, 1×1×1 sparse convolution branches, and 3×3×3 sparse convolution branches; bottleneck sequence branches include concatenated 1×1×1 sparse convolutions, 3×3×3 sparse convolutions, and 1×1×1 sparse convolutions. When training with an encoder-decoder architecture, the process of obtaining output features from the bottleneck sequence branch includes: (1); in, This is a sparse convolution operation. For output features, For input features, Let be the weight tensor of the three sparse convolutions in the bottleneck sequence branch. For the corresponding bias term, and The weights and biases are for the 1×1×1 sparse convolution branch and the 3×3×3 sparse convolution branch, respectively.

[0056] The above-mentioned deep geometric point cloud feature extraction process is a multi-stage, multi-scale encoder-decoder structure. Sparse convolution is used to handle the sparsity of point cloud data, avoid invalid calculations on empty voxels, and thus improve computational efficiency.

[0057] The reparameterized Inception block aims to capture feature information at different scales. It processes the input in parallel through a multi-branch structure and then concatenates the results to enhance feature representation. The reparameterized Inception block further optimizes the structure of the Inception block, making it flexible with multiple branches during training and equivalent to a single convolutional layer during inference, thereby reducing computation and memory usage. The internal structure of the Inception block can be adjusted according to specific needs. For example, in addition to 1x1x1 and 3x3x3 convolutions, 5x5x5 convolutions or pooling operations can be introduced.

[0058] The reparameterized Inception block is a highly efficient feature extraction module designed to enhance the model's expressive power during training by leveraging a multi-branch structure, while effectively functioning as a single convolutional layer during inference to improve inference speed. The first and second branches capture features through different convolutional paths and fuse them via a concatenation block. Adding the output features of the concatenation block to the original input features (residual connection) helps alleviate the vanishing gradient problem and allows the network to learn residual mappings, thereby improving training stability and model performance. Activation functions can employ non-linear functions such as ReLU, LeakyReLU, and Swish to introduce non-linearity. The concatenation block can connect features from different branches along the channel dimension or employ more complex fusion strategies, such as attention mechanisms.

[0059] Reparameterized convolution is the core component of the reparameterized Inception block. It provides richer feature learning paths during training through a multi-branch structure, while during inference it can be equivalent to a single convolutional kernel. The bottleneck sequence branch reduces the number of parameters while increasing the network's depth and non-linear expressive power by first reducing dimensionality (1x1x1), then convolution (3x3x3), and finally increasing dimensionality (1x1x1). The 1x1x1 sparse convolution branch is used to capture local features, while the 3x3x3 sparse convolution branch is used to capture features with a larger receptive field. The kernel size and number of channels in the bottleneck sequence branch can be adjusted according to the model capacity and computational resources. In addition to the above branches, other kernel sizes can be introduced, such as the 5x5x5 sparse convolution branch, to further enhance the multi-scale feature capture capability.

[0060] Formula (1) above details the computation of reparameterized convolution during the training phase. It processes the input features through four parallel paths: a bottleneck sequence branch, a 1x1x1 sparse convolution branch, a 3x3x3 sparse convolution branch, and a directly connected identity mapping. The outputs of these branches are summed to form the final output features. This multi-branch structure allows the network to explore different feature transformation paths during training, thereby learning more robust and richer feature representations. During training, optimizers such as stochastic gradient descent or Adam can be used to update the weight tensors and bias terms. The loss function can combine various loss terms such as point cloud reconstruction error and feature loss to guide the model to learn effective features.

[0061] This embodiment employs the aforementioned encoder-decoder architecture, combined with multi-cascaded reparameterized Inception blocks, to extract deep geometric point cloud features. This effectively captures multi-scale, multi-level geometric information from point cloud data. The process begins with preliminary feature extraction of the first point cloud through sparse convolution, yielding the first intermediate geometric point cloud features. Subsequently, a series of sparse convolution downsampling operations are alternated with cascaded reparameterized Inception blocks to gradually extract deeper and more abstract geometric features. During the training phase, the reparameterized Inception blocks utilize their multi-branch structure (including the first branch, second branch, bottleneck sequence branch, 1×1×1 sparse convolution branch, 3×3×3 sparse convolution branch, and identity mapping) to efficiently capture multi-scale contextual information, thereby enhancing feature expressiveness. After the encoder path is completed, the decoder path performs upsampling through sparse deconvolution, and simultaneously... By utilizing cascaded reparameterized Inception blocks and combining skip connections (e.g., from the fourth intermediate geometric point cloud feature to the ninth intermediate geometric point cloud feature, and from the second intermediate geometric point cloud feature to the eleventh intermediate geometric point cloud feature), features are progressively refined, ultimately yielding high-quality deep geometric point cloud features. These skip connections help preserve fine-grained information that may be lost during downsampling. The multi-branch summation structure of the reparameterized convolution during training enables the network to learn more complex and robust feature maps. This finely designed multi-scale, reparameterized feature extraction mechanism ensures that the obtained deep geometric point cloud features have high discriminativeness and rich spatial information, thus providing a solid foundation for the accurate prediction of subsequent continuous 3D offset vectors. This, in turn, more accurately guides the adaptive projection of candidate anchor points, ultimately significantly improving the geometric accuracy and detail integrity of the second point cloud recovered from the candidate point set.

[0062] In some embodiments of this application, when the encoder-decoder architecture training is complete, the bottleneck sequence branch is compressed into a 3×3×3 convolutional kernel; The convolution kernel includes: (2); (3); in, For convolution kernel, This is the corresponding equivalent bias.

[0063] Specifically, after the training phase of the encoder-decoder architecture is complete and the model parameters are fixed, to optimize inference performance, the weight tensors of the three cascaded sparse convolutions in the bottleneck sequence branch can be equivalently merged into a single 3x3x3 convolution kernel using mathematical kernel fusion techniques. This "compression" operation specifically refers to equivalently merging multiple cascaded convolution operations and their corresponding parameters (weights and biases) into a single convolution operation. For the bottleneck sequence branch, which consists of a cascaded 1x1x1 sparse convolution, a 3x3x3 sparse convolution, and a 1x1x1 sparse convolution, these three cascaded convolution operations can be fused into an equivalent 3x3x3 convolution kernel through mathematical equivalence transformation. This fusion can be achieved by merging the weights of multiple convolution kernels through matrix multiplication and adjusting the bias terms accordingly. Another approach is to utilize model optimization tools provided by deep learning frameworks, which can automatically identify and perform such parameter fusion operations, while the bias terms corresponding to these weight tensors are also processed through appropriate mathematical operations.

[0064] This embodiment reparameterizes the convolution kernel of the bottleneck sequence branch after the encoder-decoder architecture is trained. This effectively fuses the complex structure, which was originally composed of multiple cascaded sparse convolutions, into a single 3x3x3 sparse convolution kernel. Mathematically, this fusion operation maintains the same feature extraction capability as the original multi-layer structure, but in actual computation, only one convolution operation is required. This significantly reduces the computational load and memory access during the inference stage. In this way, the extraction process of deep geometric point cloud features is accelerated, effectively solving the problem of low efficiency of complex models during inference during the training stage. This optimization not only improves the running efficiency of the entire point cloud geometry recovery method, but also makes the method more suitable for application scenarios with high real-time requirements, such as point cloud processing in autonomous driving or augmented reality, thereby improving the practicality and deployment flexibility of the point cloud geometry recovery method.

[0065] In some embodiments of this application, step S130, extracting the shallow geometric point cloud features of the first point cloud, includes: Step S330: Extract the twelfth intermediate geometric point cloud features from the first point cloud based on sparse convolution; Step S340: Extract shallow geometric point cloud features from the twelfth intermediate geometric point cloud features based on the reparameterized Inception block.

[0066] This embodiment first extracts the twelfth intermediate geometric point cloud features from the first point cloud based on sparse convolution. This step utilizes the characteristics of sparse convolution to directly act on the sparse first point cloud data, efficiently capturing its local geometric structure and generating preliminary intermediate features with spatial correspondence. Sparse convolution avoids ineffective computation on a large number of empty voxels, ensuring the efficiency of feature extraction. Subsequently, shallow geometric point cloud features are extracted from the twelfth intermediate geometric point cloud features based on the reparameterized Inception block. The reparameterized Inception block, through its multi-branch structure, can capture local geometric information at different scales from the twelfth intermediate geometric point cloud features, thereby generating... Richer and more discriminative shallow features are achieved through the Inception block's reparameterization technique during the inference phase, which merges multi-branch structures into a single convolutional layer. This significantly improves feature extraction efficiency without sacrificing performance. Through this two-stage feature extraction process, this embodiment can effectively separate and refine high-quality shallow geometric point cloud features from the first point cloud. These features contain fine local details of the point cloud, providing crucial information for accurate point cloud reconstruction. This method complements deep feature extraction, which captures the global structure while shallow features focus on local details, together providing comprehensive information support for the geometric reconstruction of the point cloud.

[0067] like Figures 2 to 4 To facilitate understanding by those skilled in the art, this application also provides an embodiment of a method for geometric restoration of compressed point clouds, which includes the following steps: like Figure 2 As shown, in step S210, a point cloud restoration model is constructed; The model mainly includes the following parts: (1) Feature extraction and offset prediction module, mainly composed of an encoder-decoder architecture based on sparse convolution.

[0068] (2) The deformable sparse projection upsampling module mainly consists of a shallow structure feature extraction branch, a feature splicing unit, a generative sparse convolution unit, an offset prediction unit composed of a multilayer perceptron, and a differentiable projection unit.

[0069] (3) Scale prediction module; (4) Probability prediction module; It is important to note that Figure 2 The terms "location quantization scale," "G-PCC reconstructed coordinates," and "restored coordinates" are descriptions of the corresponding features of the process, and are not network modules based on deep learning.

[0070] Step S220: Extract multi-scale deep geometric point cloud features from the first point cloud (i.e., the lossy point cloud after G-PCC decoding) according to the feature extraction and offset prediction module. The module deeply mines the multi-scale geometric context information of the input point cloud by introducing a 3D sparse convolution encoder-decoder architecture with reparameterized Inception blocks. Its core goal is to accurately estimate the dynamic offset for subsequent deformation operations.

[0071] In the encoder-decoder architecture, the encoder expands the receptive field layer by layer through sparse convolution with a stride of 2, extracting geometric context from local to global. The decoder gradually restores the resolution through sparse deconvolution and skip connections, and fuses shallow detail features with deep semantic features.

[0072] To achieve a balance between recovery accuracy and inference efficiency, a reparameterized Inception block was designed in the feature extraction and offset prediction modules. The reparameterized Inception block combines the multi-branch parallel design concept of Inception with structural reparameterization technology, and specifically reconstructs the structure for sparse convolution.

[0073] During the training phase, the reparameterized Inception block employs a four-branch parallel topology to enhance the expressive power of multi-scale features: 1) Bottleneck sequence branches consisting of concatenated sparse convolutions of 1×1×1, 3×3×3, and 1×1×1; 2) Local topological branch (i.e., 3×3×3 sparse convolution branch); 3) Channel mapping branch (i.e., 1×1×1 sparse convolution branch); 4) Identity mapping residual branch.

[0074] During the inference phase, the aforementioned multi-branch structure can be mathematically equivalently folded into a single-path 3×3×3 sparse convolution, thereby significantly reducing memory access overhead and computational latency while maintaining strong representational capabilities. To achieve structural folding during the inference phase, an equivalent fusion derivation can be performed using the associative law and distributive law of linear operators. For the bottleneck sequence branch, since the cascaded sparse convolutions do not contain nonlinear activation functions, these three consecutive linear affine transformations can be equivalently compressed into a single 3×3×3 convolution kernel. With equivalent bias The specific formulas are the above formulas (2) and (3). For the identity mapping branch, it is constructed as a 3×3×3 tensor with a center weight of 1 and the rest of the positions of 0. Before inference, this method can merge the weights and biases of the above four branches into the equivalent parameters of a single branch by explicitly adding them mathematically equivalently. Finally, in the inference stage, the model only needs to perform a single-path sparse convolution operation using the fused parameters. This perfectly inherits the performance gain of multi-branch topology while avoiding additional computational burden.

[0075] Step S230: Extract shallow geometric point cloud features based on the deformable sparse projection upsampling module, fuse shallow geometric point cloud features and deep geometric point cloud features, and obtain candidate point set; The deformable sparse projection upsampling module is the core execution module. It aims to break the physical limitation that traditional point cloud upsampling can only generate nodes within a fixed discrete grid. Through feature decoupling and continuous coordinate correction mechanisms, it achieves integrated recovery of adaptive point cloud density completion and nonlinear coordinate correction in a unified computation flow.

[0076] The shallow structure feature extraction branch contains a sparse convolutional layer and a reparameterized Inception block that are sequentially connected. The input of the shallow structure feature extraction branch receives the first point cloud, and the output is connected to the first input of the feature concatenation unit.

[0077] The second input of the feature stitching unit receives deep geometric point cloud features from the feature extraction and offset prediction module, and its output is connected to the generative sparse convolution unit.

[0078] The output of the generative sparse convolutional unit is coupled to the offset prediction unit and the differentiable projection unit, which are composed of multilayer perceptrons, respectively.

[0079] The output of the offset prediction unit is connected to the other input of the differentiable projection unit.

[0080] Differentiable projection units output corrected high-fidelity continuous point cloud features to obtain candidate point sets.

[0081] The following is the execution flow of the deformable sparse projection upsampling module: First, the first point cloud is input to the shallow structure feature extraction branch, and then passes through the sparse convolutional layer for basic feature mapping. It then enters the reparameterized Inception block for local topological encoding. The extracted shallow geometric point cloud features serve as the pure feature base, which fully preserves the boundary contour, surface details and local topological information of the first point cloud, and does not directly participate in the macroscopic spatial deformation calculation.

[0082] Meanwhile, the deep geometric point cloud features output by the feature extraction and offset prediction module, carrying the degradation rules of the compressed structure, enter this module and simultaneously enter two parallel processing streams: on the one hand, the deep geometric point cloud features are concatenated with the aforementioned shallow geometric point cloud features in the channel dimension at the feature splicing unit, and the fused joint features are then fed into the generative sparse convolution unit, which generates high-density candidate anchor points in the discrete integer grid space; on the other hand, the deep geometric point cloud features are directly diverted to the offset prediction unit.

[0083] As a dedicated geometric deformation solver, the offset prediction unit regresses a fine, continuous 3D offset vector based on the rich deep geometric context in the deep geometric point cloud features. This vector aims to guide the generated candidate anchor points to break through the adsorption effect of the regular discrete mesh and approach the real and continuous physical position of the object surface.

[0084] To ensure that the generated features are accurately mapped to the corrected continuous sub-voxel coordinates and that the projection of discrete features onto continuous coordinates is end-to-end differentiable, a mathematical model for feature aggregation and weight allocation based on the trilinear interpolation principle is constructed within the differentiable projection unit. Let the network predict a certain continuous floating-point coordinate as... (That is, the sum of the three-dimensional coordinates of the initial anchor point and the three-dimensional offset vector), which corresponds to the shallow geometric point cloud features to be mapped. The floating-point coordinates For any integer grid point in its 8-neighborhood within its three-dimensional physical space Spatial contribution weight This can be dynamically calculated using the linear distance attenuation along each axis, and its mathematical expression is: (4); (5); Formulas (4) and (5) construct a smooth kernel function in three-dimensional space, ensuring that grid points that are closer together are assigned a larger weight and that the sum is strictly 1.

[0085] Subsequently, based on this dynamic weight, the features in the continuous coordinate system are resampled and aggregated onto the discrete target grid. For any integer grid point... The aggregated features output after aggregation are calculated as follows: (6); in, This indicates that the projection influence range covers the grid points. A set of floating-point coordinates.

[0086] Since the coordinate offset components in the weight allocation formula do not undergo any step function or nonlinear truncation, they are continuous and differentiable within their domain. Therefore, the overall reconstruction error of the network can be directly backpropagated to the floating-point coordinates through the partial derivatives of the weights with respect to the coordinate components, according to the chain rule of calculus. superior.

[0087] This embodiment overcomes the gradient discontinuity problem caused by point cloud meshing operations in the two-stage recovery scheme of upsampling + coordinate offset. It constructs a continuous geometric gradient backpropagation path, enabling the network to simultaneously receive joint gradient backpropagation from occupancy probability supervision (such as binary classification cross-entropy loss) and geometric space reconstruction supervision (such as chamfer distance loss) in a single training phase. This allows for accurate filling of structural holes while completing high-fidelity coordinate inverse correction, achieving true integrated joint optimization.

[0088] Step S240: Predict the target recovery quantity based on the scale prediction module; In G-PCC lossy compression, the combined effect of position quantization scale and octree node pruning at the bottom layer leads to a strong non-uniformity in the density degradation of the decoded point cloud in different local regions and compression levels.

[0089] To accurately recover the lost geometric structure at the decoding end, the scale prediction module analyzes the local topological sparsity of the first point cloud and its potential three-dimensional spatial distribution characteristics, and directly predicts the target recovery quantity before compression encoding (i.e., the cardinality of the original lossless point cloud). ).

[0090] In terms of specific composition and connection relationships, the module takes the first point cloud after compression and decoding or its extracted shallow geometric point cloud features as the input end, and internally it is sequentially connected to the multilayer perceptron, the global pooling layer and the cascaded fully connected network layer.

[0091] First, the first point cloud is input into a lightweight multilayer perceptron for feature upscaling. Then, combined with a global aggregation operation, the complex local spatial topological features are compressed into a global density context representation vector.

[0092] The vector is nonlinearly mapped to a scalar output through a cascaded fully connected network. The scalar represents the optimal number of physical points that the current local block should achieve after geometric restoration.

[0093] To avoid gradient conflicts between the point count regression task and the coordinate generation task in the same computational graph, this embodiment employs a decoupled, independent optimization strategy for the modules during the training phase. The actual number of points in the first point cloud before encoding is used as the true label, and smoothing loss is used for separate supervised training. This decoupled training mode ensures that the network can focus highly on learning the sparsity degradation patterns of the point cloud, thereby obtaining extremely high-precision cardinality estimation results. This provides crucial data-driven objective constraints for the subsequent denoising and selection of modules.

[0094] Step S250: Recover the second point cloud based on the probability prediction module; After processing with the deformable sparse projection upsampling model, the geometric coordinates of the first point cloud in local space are significantly recovered. However, the generation process inevitably introduces a certain number of discrete noise points and redundant artifacts around the real surface. Traditional methods usually rely on manually set fixed probability thresholds for hard filtering, which is difficult to adapt to differences in compression levels and the degree of local deformation.

[0095] To address the aforementioned issues, this example designs a probability prediction module. This module mainly includes a classification head composed of sparse convolutional layers, a global sorting unit, and a truncation and filtering unit. The input of the classification head is connected to the output of the deformable sparse projection upsampling module, and its output is connected to the global sorting unit and the truncation and filtering unit in sequence. The other control input of the truncation and filtering unit is communicatively connected to the output of the fully connected network of the aforementioned scale prediction module.

[0096] The module first uses the classification head to perform a geometric reliability assessment on all candidate points generated by upsampling, and maps the high-dimensional geometric fusion feature of each candidate point to an occupancy probability score with a value between [0,1] that represents its belonging to the real physical surface.

[0097] During the joint training phase, the prediction of probability scores is strictly supervised by the binary cross-entropy loss. During the inference phase, the global sorting unit first performs a global descending sort on all candidate points in the current local block according to their occupied probability scores in memory, so that the central surface points with extremely high confidence are arranged at the beginning of the sequence, while potential outlier noise points are arranged at the end. Subsequently, the truncation and screening unit uses the number of predicted encoding points N output by the scale prediction module as a dynamic truncation benchmark, and accurately truncates the top N candidate points with the highest probability from the above descending probability sequence as the final effective point cloud. The low-confidence candidate points ranked after N are judged as redundant artifacts and are directly removed and released at the physical memory level, thereby realizing fully data-driven density adaptive recovery.

[0098] The geometric restoration method for compressed point clouds provided in this embodiment has at least the following beneficial effects; This embodiment takes the compressed and decoded lossy point cloud as input. First, it introduces a feature extraction and offset prediction module with a structure reparameterization mechanism to efficiently obtain the deep geometric context without increasing inference overhead. Then, it uses a deformable sparse projection upsampling module to predict the continuous three-dimensional offset vector, completely abandoning the fragmented paradigm of traditional methods that first perform discrete upsampling and then superimpose coordinate offsets on the generated points. The predicted three-dimensional offset vector is directly applied to the initially excited candidate anchor points, and a sub-voxel-level high-fidelity coordinate correction is achieved in one go through a differentiable projection mechanism. Finally, the scale prediction and probability prediction mechanisms are combined to accurately estimate the cardinality of the original point cloud before compression. Based on the point-by-point occupancy probability of the candidate anchor points, global descending sorting and adaptive Top-N truncation screening are performed to completely eliminate redundant artifacts and output the final high-quality geometrically restored point cloud.

[0099] The following are experimental examples corresponding to this embodiment; 1. Training strategy and parameter configuration; For training the model in the above embodiment, training samples were constructed using content from the ShapeNet dataset. During the data preparation phase, the first point cloud was spatially partitioned to achieve data augmentation. Subsequently, the training data was compressed using the G-PCC reference TMC13 (octree mode) at five default geometrically lossy compression configurations (R1-R5 levels), generating G-PCC damaged point cloud samples for network optimization. In terms of loss function design, the model's scale prediction module was trained separately using an independent L1 loss to effectively decouple heterogeneous tasks and avoid mutual constraints between different optimization objectives. For the other parts of the model, a joint loss function consisting of binary cross-entropy loss and chamfer distance loss was constructed for end-to-end optimization, with hyperparameter weights set to 1 and 0.1, respectively.

[0100] For optimization, the Adam optimizer was used, with the initial learning rate of the point cloud recovery model set to 4×10^-4. A step decay strategy was adopted, where the learning rate automatically decayed to half its current value every 6000 steps. Independent training was performed for different compression bitrates, with a batch size of 8, and each model underwent a maximum of 30,000 iterations of training.

[0101] 2. Analysis of Experimental Results To quantify the overall compression and transmission efficiency, this experiment used the Bjontegaard Delta Rate (BD-Rate) metric to statistically evaluate the objective recovery performance, validating the results on various test sets. A negative BD-Rate with a larger absolute value indicates greater bandwidth savings while maintaining the same geometric reconstruction quality. Test data shows that the average BD-Rate of this method reached -77.17% and -77.03% in the D1 (point-to-point) and D2 (point-to-area) dimensions, respectively. This rate-distortion performance demonstrates that this embodiment can achieve high-fidelity geometric recovery at an equivalent bandwidth cost, improving the compression and transmission efficiency of point cloud data.

[0102] Table 1 shows the BD-Rate performance of the method on different test data.

[0103] like Figure 5 As shown in one embodiment of this application, a geometric restoration apparatus for compressed point clouds is provided, the apparatus comprising: The point cloud acquisition unit 1001 is used to acquire a first point cloud; wherein, the first point cloud is a point cloud after G-PCC lossy compression; The feature extraction unit 1002 is used to extract the deep geometric point cloud features of the first point cloud and the shallow geometric point cloud features of the first point cloud. Anchor point extraction unit 1003 is used to concatenate deep geometric point cloud features and shallow geometric point cloud features to obtain geometric point cloud fusion features, and to excite the geometric point cloud fusion features based on generative sparse convolution to obtain multiple candidate anchor points on discrete integer voxel grids. The point set acquisition unit 1004 is used to predict the continuous three-dimensional offset vector of the first point cloud based on the features of the deep geometric point cloud, and guide the adaptive projection of multiple candidate anchor points on the discrete integer voxel grid based on the continuous three-dimensional offset vector to obtain the candidate point set. The point cloud recovery unit 1005 is used to recover the second point cloud from the candidate point set.

[0104] It should be noted that the geometric restoration device for compressed point clouds provided in this embodiment is based on the same inventive concept as the geometric restoration method for compressed point clouds described above. Therefore, the content of the geometric restoration method for compressed point clouds described above is also applicable to the content of the geometric restoration device for compressed point clouds in this embodiment, and will not be repeated here.

[0105] like Figure 6 This application also provides an electronic device, which includes: At least one memory; At least one processor; At least one program; The program is stored in memory, and the processor executes at least one program to implement the geometric restoration method for compressed point clouds described above in this disclosure.

[0106] Electronic devices can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0107] The electronic devices according to embodiments of this application will now be described in detail.

[0108] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention. The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to perform a geometric restoration method for compressed point clouds according to an embodiment of the present invention.

[0109] The input / output interface 1800 is used to implement information input and output. The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900); The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0110] This invention also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described geometric reconstruction method for compressed point clouds.

[0111] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0112] The embodiments described in this invention are intended to more clearly illustrate the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0113] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present invention, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0114] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0115] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0116] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0117] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0119] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0120] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0121] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0122] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

Claims

1. A method for geometric reconstruction of compressed point clouds, characterized in that, The method includes: Obtain the first point cloud; wherein, the first point cloud is a point cloud decoded by G-PCC; Extract the deep geometric point cloud features of the first point cloud, and extract the shallow geometric point cloud features of the first point cloud. The deep geometric point cloud features and the shallow geometric point cloud features are concatenated to obtain geometric point cloud fusion features, and the geometric point cloud fusion features are excited based on generative sparse convolution to obtain multiple candidate anchor points on a discrete integer voxel grid. Based on the deep geometric point cloud features, predict the continuous three-dimensional offset vector of the first point cloud, and guide the adaptive projection of the multiple candidate anchor points on the discrete integer voxel grid based on the continuous three-dimensional offset vector to obtain the candidate point set; The second point cloud is recovered from the set of candidate points.

2. The geometric restoration method for compressed point clouds according to claim 1, characterized in that, The adaptive projection of the plurality of candidate anchor points onto a discrete integer voxel grid guided by the continuous three-dimensional offset vector to obtain a candidate point set includes: The three-dimensional coordinates of each candidate anchor point are summed with the corresponding three-dimensional offset vector in the continuous three-dimensional offset vector to obtain continuous floating-point coordinates, and the continuous floating-point coordinates are used as the candidate point set; The process of recovering the second point cloud from the candidate point set includes: Based on the distance decay rule of trilinear interpolation, the spatial contribution weight of each floating-point coordinate in the continuous floating-point coordinates to the integer grid points in the corresponding neighborhood is calculated. The aggregated features of the integer grid points are obtained by weighting the spatial contribution weight of the floating-point coordinates to the integer grid points and the shallow geometric point cloud features corresponding to the floating-point coordinates. Based on the aggregation features of each integer grid point, predict the occupancy probability of each integer grid point, and assign the occupancy probability of each integer grid point to each corresponding floating-point coordinate. Based on the occupancy probability, multiple floating-point coordinates are selected from the continuous floating-point coordinates to form a second point cloud.

3. The geometric restoration method for compressed point clouds according to claim 2, characterized in that, Before recovering the second point cloud from the candidate point set, the method further includes: Extract the context representation vector from the first point cloud; Predict the target recovery quantity based on the context representation vector; The step of selecting multiple floating-point coordinates from the continuous floating-point coordinates to form a second point cloud based on the occupancy probability includes: The continuous floating-point coordinates are sorted according to the occupancy probability to obtain the sorting result; Based on the sorting results and the number of target recoveries, multiple floating-point coordinates are selected to form a second point cloud.

4. The geometric restoration method for compressed point clouds according to claim 1, characterized in that, The extraction of deep geometric point cloud features from the first point cloud includes: Construct an encoder-decoder architecture that conforms to the feature pyramid based on sparse convolution; The deep geometric point cloud features of the first point cloud are extracted based on the encoder-decoder architecture.

5. The geometric restoration method for compressed point clouds according to claim 4, characterized in that, The step of extracting deep geometric point cloud features from the first point cloud based on the encoder-decoder architecture includes: The first intermediate geometric point cloud features are extracted from the first point cloud based on sparse convolution. The second intermediate geometric point cloud features are extracted from the first intermediate geometric point cloud features based on two cascaded reparameterized Inception blocks. The third intermediate geometric point cloud feature is obtained by downsampling the second intermediate geometric point cloud feature using sparse convolution; The fourth intermediate geometric point cloud feature is extracted from the third intermediate geometric point cloud feature based on two cascaded reparameterized Inception blocks. The fifth intermediate geometric point cloud feature is obtained by downsampling the fourth intermediate geometric point cloud feature using sparse convolution; The sixth intermediate geometric point cloud feature is extracted from the fifth intermediate geometric point cloud feature based on two cascaded reparameterized Inception blocks. The seventh intermediate geometric point cloud feature is extracted from the sixth intermediate geometric point cloud feature based on sparse convolution; The eighth intermediate geometric point cloud feature is extracted from the seventh intermediate geometric point cloud feature based on two cascaded reparameterized Inception blocks. The ninth intermediate geometric point cloud feature is extracted from the eighth intermediate geometric point cloud feature based on sparse deconvolution; The tenth intermediate geometric point cloud feature is extracted from the ninth intermediate geometric point cloud feature and the fourth intermediate geometric point cloud feature based on two cascaded reparameterized Inception blocks. The eleventh intermediate geometric point cloud features are extracted from the tenth intermediate geometric point cloud features based on sparse deconvolution. Deep geometric point cloud features are extracted from the eleventh intermediate geometric point cloud features and the second intermediate geometric point cloud features based on two cascaded reparameterized Inception blocks. The reparameterized Inception block includes a first branch and a second branch. The first branch includes cascaded reparameterized convolutions, activation functions, reparameterized convolutions, activation functions, and a concatenation block. The second branch includes cascaded 1×1×1 sparse convolutions, activation functions, reparameterized convolutions, activation functions, and 1×1×1 sparse convolutions. The output of the second branch is connected to the input of the concatenation block. The feature obtained by adding the output feature of the concatenation block to the input feature of the reparameterized Inception block is used as the output feature of the reparameterized Inception block. The reparameterized convolution includes a bottleneck sequence branch, a 1×1×1 sparse convolution branch, and a 3×3×3 sparse convolution branch; the bottleneck sequence branch includes a concatenated 1×1×1 sparse convolution, a 3×3×3 sparse convolution, and a 1×1×1 sparse convolution. When training with the encoder-decoder architecture, the process by which the bottleneck sequence branch obtains output features includes: ； in, This is a sparse convolution operation. For output features, For input features, Let be the weight tensor of the three sparse convolutions in the bottleneck sequence branch. For the corresponding bias term, and The weights and biases are for the 1×1×1 sparse convolution branch and the 3×3×3 sparse convolution branch, respectively.

6. The geometric restoration method for compressed point clouds according to claim 5, characterized in that, Once the encoder-decoder architecture has been trained, the bottleneck sequence branch is compressed into a 3×3×3 convolutional kernel; The convolution kernel includes: ； ； in, The convolution kernel, This is the corresponding equivalent bias.

7. The geometric restoration method for compressed point clouds according to claim 4, characterized in that, The extraction of shallow geometric point cloud features from the first point cloud includes: Based on sparse convolution, the twelfth intermediate geometric point cloud features are extracted from the first point cloud; Shallow geometric point cloud features are extracted from the twelfth intermediate geometric point cloud features based on the reparameterized Inception block.

8. A geometric reconstruction device for compressed point clouds, characterized in that, The device includes: A point cloud acquisition unit is used to acquire a first point cloud; wherein the first point cloud is a point cloud after G-PCC lossy compression; The feature extraction unit is used to extract the deep geometric point cloud features of the first point cloud and the shallow geometric point cloud features of the first point cloud. Anchor point extraction unit is used to concatenate the deep geometric point cloud features and the shallow geometric point cloud features to obtain geometric point cloud fusion features, and to excite the geometric point cloud fusion features based on generative sparse convolution to obtain multiple candidate anchor points on a discrete integer voxel grid. The point set acquisition unit is used to predict the continuous three-dimensional offset vector of the first point cloud based on the deep geometric point cloud features, and guide the adaptive projection of the multiple candidate anchor points on the discrete integer voxel grid based on the continuous three-dimensional offset vector to obtain the candidate point set. A point cloud recovery unit is used to recover a second point cloud from the candidate point set.

9. An electronic device, characterized in that, include: At least one control processor and a memory for communicatively connecting to the at least one control processor; The memory stores instructions executable by the at least one control processor, which, when executed, enable the at least one control processor to perform a geometric recovery method for compressed point clouds as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform a geometric restoration method for compressed point clouds as described in any one of claims 1 to 7.

Images