Optimization method, system and equipment for compressed point cloud data and medium

Abstract

The invention discloses an optimization method and system for compressed point cloud data, equipment and a medium. The method comprises the following steps: acquiring first data and second data; extracting local geometric features and global structural features from the first data; adapting a compression distortion type of the first data according to the coding parameter; based on the local geometric features, determining a deviation range of each discrete point in the first data and a point missing region in the first data; restoring the first data by compressing a restoration network corresponding to the distortion type based on the global structure feature, the deviation range and a point missing region in the first data to obtain second data; the first data and the second data are merged to obtain optimized data, distortion types under different compression strengths can be adapted through the coding parameters, and targeted restoration and optimization of the compressed point cloud data are realized in combination with local and global features, so that the geometric fidelity and visual quality of the compressed point cloud data are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of point cloud data processing, in particular to a post-compression point cloud data optimization method, system, device and medium. BACKGROUND

[0002] As a three-dimensional data form for accurately describing the geometric shape and surface characteristics of an object, point cloud contains spatial position and attribute information such as color and reflection intensity, and is widely used in the field of computer vision. However, the amount of point cloud data is huge, and a minute can produce hundreds of megabytes of original data, which brings great pressure to transmission, storage and processing. Therefore, the Moving Pictures Experts Group (MPEG) of the International Organization for Standardization proposes a geometry-based point cloud compression (G-PCC) standard, which realizes efficient compression through octree geometry coding and attribute prediction coding, and effectively promotes the popularization of point cloud technology.

[0003] However, G-PCC compression can introduce geometric distortion, and the existing post-compression point cloud data optimization method uses a fixed multiple (such as 4 times, 8 times) to restore the number of points, which has obvious limitations. Moreover, the loss mode of geometric information caused by G-PCC compression is different for point clouds with different content characteristics, and the loss proportion of the number of points under different compression configurations also has great differences. The existing technology is difficult to adapt to the unique distortion mode of different point clouds, resulting in poor recovery effect when facing point clouds with large differences in content characteristics, and further affecting the reconstruction quality of the post-compression point cloud and the performance of downstream tasks. SUMMARY

[0004] The following is a summary of the subject matter described in detail in this document. This summary is not intended to limit the scope of protection of the claims.

[0005] The main purpose of the embodiments of the present disclosure is to provide a post-compression point cloud data optimization method, system, device and storage medium, which can adapt to different distortion types under different compression intensities through coding parameter adaptation, and realize targeted repair combined with local and global features, optimize post-compression point cloud data, and further improve the geometric fidelity and visual quality of post-compression point cloud data.

[0006] The first aspect of the embodiments of the present application provides a post-compression point cloud data optimization method, which comprises: obtaining first data and second data; the first data is a discrete point set obtained by decoding after G-PCC processing of initial point cloud data; the second data is an encoding parameter used when the initial point cloud data is processed by G-PCC; extracting local geometric features and global structure features from the first data; adapt a compression distortion type of the first data according to the encoding parameter; determine a deviation range of each discrete point in the first data and a point missing area in the first data based on the local geometric feature; repair the first data based on the global structure feature, the deviation range and the point missing area in the first data through a repair network corresponding to the compression distortion type, to obtain second data; merge the first data and the second data to obtain optimized data.

[0007] In some embodiments of the present application, the local geometric feature and the global structure feature are extracted from the first data, including: perform sparse convolution processing on the first data to obtain third data and basic local structure information; perform nonlinear transformation on the third data to obtain nonlinear transformed features; perform multi-scale convolution processing on the nonlinear transformed features to obtain the local geometric feature; generate the global structure feature based on the basic local structure information and the local geometric feature.

[0008] In some embodiments of the present application, the first data is repaired based on the global structure feature, the deviation range and the point missing area in the first data through a repair network corresponding to the compression distortion type, to obtain second data, including: adapt a corresponding repair network in a database according to the compression distortion type; the database contains a mapping relationship between the compression distortion type and the repair network, and the model weights of each repair network in the database are different; repair the first data according to the deviation range through the repair network to obtain fourth data; repair the fourth data according to the global structure feature through the repair network to obtain the second data.

[0009] In some embodiments of the present application, the repair network is trained through the following steps: obtain initial point cloud training data; compress the initial point cloud training data using multiple compression quantization scales to obtain compressed point cloud training data corresponding to each compression quantization scale; train the initialized repair network based on the compressed point cloud training data corresponding to each compression quantization scale according to a binary cross entropy loss function, to obtain a trained repair network.

[0010] In some embodiments of the present application, the merging the first data and the second data to obtain optimized data comprises: merging the first data and the second data to obtain merged data; predicting, by a probability prediction network, a confidence of each discrete point in the merged data; performing pruning processing on the merged data according to the confidence of each discrete point in the merged data to obtain the optimized data.

[0011] In some embodiments of the present application, the predicting, by a probability prediction network, a confidence of each discrete point in the merged data comprises: performing sparse convolution processing on the merged data to extract feature information of the merged data; inputting the feature information into a second activation function to perform nonlinear transformation to obtain a feature of each discrete point in the merged data; performing multi-scale convolution comprehensive processing on the feature of each discrete point in the merged data by the probability prediction network based on an attention mechanism to obtain the confidence of each discrete point in the merged data.

[0012] In some embodiments of the present application, the performing pruning processing on the merged data according to the confidence of each discrete point in the merged data to obtain the optimized data comprises: removing discrete points in the merged data whose confidence is less than a preset confidence threshold to obtain the optimized data.

[0013] A first aspect of an embodiment of the present application provides an optimization method for compressed point cloud data. The method comprises: obtaining first data and second data; extracting local geometric features and global structure features from the first data; adapting a compression distortion type of the first data according to an encoding parameter; determining a deviation range of each discrete point in the first data and a point missing area in the first data based on the local geometric features; repairing the first data based on the global structure features, the deviation range and the point missing area in the first data by a repair network corresponding to the compression distortion type to obtain the second data; and merging the first data and the second data to obtain optimized data. The method can adapt to distortion types under different compression intensities by adapting the encoding parameter, and realize targeted repair by combining local and global features, thereby optimizing the compressed point cloud data and further improving the geometric fidelity and visual quality of the compressed point cloud data.

[0014] To achieve the above object, a second aspect of an embodiment of the present application provides an optimization system for compressed point cloud data. The system comprises: The acquisition module is configured to acquire first data and second data; the first data is a discrete point set obtained by decoding after G-PCC processing of initial point cloud data; and the second data is an encoding parameter used in the G-PCC processing of the initial point cloud data. The extraction module is configured to extract local geometric features and global structure features from the first data. The adaptation module is configured to adapt a compression distortion type of the first data according to the encoding parameter. The determination module is configured to determine a deviation range of each discrete point in the first data and a point missing area in the first data based on the local geometric features. The repair module is configured to repair the first data based on the global structure features, the deviation range, and the point missing area in the first data by using a repair network corresponding to the compression distortion type, to obtain the second data. The merging module is configured to merge the first data and the second data to obtain optimized data.

[0015] To achieve the above object, a third aspect of the embodiment of the present application provides an electronic device, comprising: at least one control processor and a memory connected in communication with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to perform the above-mentioned optimization method for compressed point cloud data.

[0016] To achieve the above object, a fourth aspect of the embodiment of the present application provides a computer readable storage medium, which stores computer executable instructions for causing a computer to perform the above-mentioned optimization method for compressed point cloud data.

[0017] It can be understood that the beneficial effects of the above-mentioned second aspect to fourth aspect and related technology are the same as the beneficial effects of the above-mentioned first aspect and related technology, which can be referred to the related description in the first aspect and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0018] The above and / or additional aspects and advantages of the present application will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which: Figure 1 is a flowchart of an optimization method for compressed point cloud data provided by the embodiment of the present application; Figure 2 is an application diagram provided by the embodiment of the present application; Figure 3This is a schematic diagram of the feature extraction module provided in an embodiment of this application; Figure 4 This is a schematic diagram of the network structure of a feature coding unit provided in an embodiment of this application; Figure 5 This is a schematic diagram of the network structure of the coordinate reconstruction module provided in the embodiments of this application; Figure 6 This is a schematic diagram of the network structure of the probability prediction module provided in the embodiments of this application; Figure 7 This is a schematic diagram of the structure of an optimization system for compressed point cloud data provided in an embodiment of this application; Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0019] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0020] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0021] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0022] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0023] Point clouds are a data format that represents the geometric information of an object as a set of three-dimensional coordinate points. Each point contains not only spatial location information (X, Y, Z coordinates) but may also carry attribute information such as color and reflectivity. As an important three-dimensional data representation method, point cloud data can accurately describe the geometric shape and surface features of an object. In computer vision, with 700,000 points and a frame rate of 10fps, hundreds of megabytes of raw point cloud data can be generated in just one minute. Such a massive amount of data not only poses challenges for real-time transmission but also puts enormous pressure on data storage and processing, severely restricting the widespread application of point cloud technology. Image data also has a large data volume.

[0024] To address this issue, the international standards organization MPEG proposed the G-PCC (Geometry-based Point Cloud Compression) standard. G-PCC employs an octree-based geometric structure coding scheme, recursively dividing the three-dimensional space into regular cubic grids to organize point cloud data into a hierarchical tree structure. This structured representation effectively captures the spatial distribution characteristics of point clouds, achieving efficient geometric information compression. Furthermore, G-PCC also utilizes techniques such as attribute information predictive coding and entropy coding to further improve compression efficiency.

[0025] While G-PCC excels in compression ratio, its compression process inevitably introduces distortion. The loss of geometric information primarily manifests as: geometric structure distortion (spatiotemporal shift of points), loss of detail features (high-frequency geometric details are easily lost during compression), and uneven point distribution (the reconstructed point cloud may exhibit localized sparseness or density). These compression distortions not only affect the visual quality of point cloud data but also degrade the performance of downstream tasks such as object detection and scene segmentation. Therefore, how to improve reconstruction quality while maintaining a high compression ratio has become a crucial research topic in the field of point cloud compression.

[0026] Based on this, embodiments of this application provide a method, system, electronic device, and medium for optimizing compressed point cloud data. The aim is to adapt the encoding parameters to the distortion types under different compression intensities, combine local and global features to achieve targeted repair, optimize the compressed point cloud data, and thereby improve the geometric fidelity and visual quality of the compressed point cloud data.

[0027] The optimization method, system, electronic device, and medium for compressed point cloud data provided in this application are specifically described through the following embodiments. First, the optimization method for compressed point cloud data in this application embodiment is described.

[0028] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0029] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0030] The optimization method for compressed point cloud data provided in this application relates to the field of point cloud data processing technology. This optimization method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the optimization method for compressed point cloud data, but is not limited to the above forms.

[0031] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0032] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0033] Therefore, referring to Figure 1 This application provides a method for optimizing compressed point cloud data. This method is applied to a central controller, which can be a server, an electronic device, or a mobile terminal, etc. There are no specific limitations here. The method includes the following steps S110 to S160.

[0034] Step S110: Obtain first data and second data; the first data is the set of discrete points obtained by decoding the initial point cloud data after G-PCC processing; the second data is the encoding parameters used when performing G-PCC processing on the initial point cloud data. Step S120: Extract local geometric features and global structural features from the first data; Step S130: Adapt the compression distortion type of the first data according to the encoding parameters; Step S140: Based on local geometric features, determine the deviation range of each discrete point in the first data and the missing point region in the first data; Step S150: Using the repair network corresponding to the compression distortion type, the first data is repaired based on global structural features, deviation range, and missing midpoint regions of the first data to obtain the second data; Step S160: Merge the first data and the second data to obtain optimized data.

[0035] In this step, the first data is the set of discrete points obtained by decoding the initial point cloud data after compression using the geometry-based point cloud compression (G-PCC) standard. This set not only contains the spatial coordinates of each discrete point, but also covers relevant features that characterize the validity of the coordinates. The second data is the encoding parameters used in the G-PCC compression process. The core of these parameters includes the position quantization scale parameter, which reflects the compression intensity. This parameter is directly related to the degree and type of subsequent point cloud distortion, providing a key reference for accurate repair and ensuring that the repair process is adapted to the compression configuration.

[0036] Furthermore, local geometric features and global structural features are extracted from the first data. Taking advantage of the sparsity of the first data, sparse convolution is used as the core processing unit for feature extraction. Specifically, sparse convolution operations are first used to compress the data scale while capturing basic local structural relationships between discrete points. Then, a preset activation function is used to perform a nonlinear transformation on the processed data to enhance the discriminative and expressive power of the features. Next, multi-scale convolution processing is used to mine fine geometric relationships of the point cloud at different scales, forming local geometric features that can accurately represent local details. Finally, based on the extracted basic local structural information and local geometric features, global information is aggregated to integrate the overall spatial distribution pattern and structural relationships of the point cloud, generating global structural features that reflect the overall morphology of the point cloud.

[0037] Furthermore, based on the encoding parameters in the second data, namely the position quantization scale parameter, the distortion type is adapted. Specifically, when the position quantization scale parameter value is large, point loss is more likely to occur during G-PCC compression, thus the compression distortion type of the first data is determined to be point loss-dominant. When the position quantization scale parameter value is small, the main impact of compression is the spatial position shift of points, thus the distortion type is determined to be point shift-dominant. Through the direct correlation between parameters and distortion types, targeted pre-adaptation for repair is achieved.

[0038] Furthermore, based on local geometric features, by analyzing the spatial positional relationship and distance distribution between discrete points and their surrounding neighboring points, the deviation range of each discrete point relative to the ideal position is accurately calculated, clarifying the specific degree of point offset. Specifically, firstly, when calculating the offset range of discrete points, local geometric features already cover key information such as the spatial positional coordinate relationship and relative distance distribution patterns between discrete points and their surrounding neighboring points. This information originates from feature mining of different ranges of local regions by multi-scale convolution, including both the fine positional connection relationship of nearby neighboring points and the morphological trend of slightly larger local structures. Based on this, by analyzing the local geometric structure (such as local planes, surfaces, or contours) formed by the discrete point and several neighboring points, and utilizing the spatial distribution patterns of neighboring points (such as fitting local surface equations and inferring local contour trends), the ideal position of the discrete point can be accurately inferred. Then, by comparing the actual coordinates of the discrete point with the ideal position, and combining the distance distribution threshold between neighboring points (determined by the density features in the local geometric features), the coordinate difference and offset direction of the two in three-dimensional space can be calculated. This allows for the quantification of the deviation range of the discrete point, clarifying the specific degree of point offset (such as slight offset, moderate offset, or severe offset), and providing accurate numerical references for subsequent targeted coordinate correction.

[0039] Furthermore, when identifying missing point regions, the local density distribution and structural continuity features contained in the local geometric features play a crucial role: in a normal, undistorted point cloud, the discrete point density follows a relatively uniform distribution pattern, and the structure exhibits coherence and consistency (such as continuous curved surfaces and complete edges); while point disappearance caused by G-PCC compression manifests as a sudden drop in local point density below the normal threshold, or the appearance of broken or disjointed segments in a previously coherent local structure. By statistically analyzing the density distribution data in the local geometric features, abnormal regions with abrupt density changes can be detected; simultaneously, combined with structural continuity features, segments with broken local structures and disjointed shapes can be identified. By integrating both, the specific range of the missing point region (such as boundary coordinates, region size, and shape) can be accurately located, clarifying the connection relationship between the missing region and the surrounding normal structure. This ensures that subsequent missing point supplementation can seamlessly integrate with the surrounding structure, avoiding structural conflicts after repair.

[0040] Furthermore, based on the compression distortion type and the positional quantization scale parameter, model weights suitable for the current compression scenario are selected from multiple pre-trained sub-networks. Specifically, firstly, based on the determined compression distortion type (point offset dominant or point vanishing dominant), the classification range of the pre-trained sub-networks is locked, and a set of sub-networks specifically trained for this distortion type is selected from the database, excluding sub-networks adapted to other distortion types to avoid cross-type weight misuse that could lead to repair failure. Then, the specific value of the positional quantization scale parameter is used for further precise selection. Since this parameter directly reflects the compression intensity and distortion degree (the larger the value, the more severe the distortion and the higher the proportion of point loss), sub-networks whose weights perfectly match the compression intensity corresponding to the parameter value are selected from the locked set of sub-networks. This ensures that the selected model weights can accurately fit the distortion type, compression intensity, and proportion of point loss in the current compression scenario, providing highly adaptable parameter support for subsequent targeted repairs (point offset correction or point vanishing supplementation), and avoiding the shortcomings of traditional single-weight models that are difficult to adapt to diverse compression scenarios.

[0041] For example, when both are point-disappearance dominant types, if the position quantization scale parameter is large, choose a subnetwork with weights that adapt to high-intensity distortion and focus on supplementing a large number of missing points; if the parameter is medium, choose a subnetwork with weights that balance the amount of supplementation and structural coherence.

[0042] The differences in the construction logic and weights of multiple pre-trained sub-networks stem from the comprehensive coverage of G-PCC compression scenarios. During the training phase, they have been trained in a targeted manner based on multiple position quantization scale parameters (i.e. compression quantization scales) for different compression distortion types (point offset dominant type, point vanishing dominant type). The model weights of each sub-network have been specifically optimized to adapt to specific distortion modes and compression intensities.

[0043] For example, in the pre-trained sub-network for point offset-dominated scenarios (when the corresponding quantization scale parameter is small), the weights focus on enhancing the accuracy of discrete point coordinate correction, which can accurately match the slight offset repair needs under small-scale compression. In the pre-trained sub-network for point disappearance-dominated scenarios (when the corresponding quantization scale parameter is large), the weights focus on optimizing the rationality and structural fit of missing point supplementation, adapting to the dense missing point repair needs under large-scale compression. Moreover, under the same distortion type, different weights correspond to different quantization scale parameters at different locations to adapt to different distortion levels.

[0044] Specifically, this embodiment provides a training process for a repair network with various model weights, and the steps are as follows: Obtain initial point cloud training data; Multiple compression quantization scales are used to compress the initial point cloud training data to obtain the compressed point cloud training data corresponding to each compression quantization scale. Based on the binary cross-entropy loss function, the initial repair network is trained using the compressed point cloud training data corresponding to each compression quantization scale, resulting in the trained repair network.

[0045] In this embodiment, the training process of the repair network focuses on adapting to different compression scenarios and improving generalization ability. It is carried out step by step by combining point cloud compression characteristics and deep learning training logic. Specifically, it includes: First, in the initial point cloud training data acquisition stage, point cloud data (including entity object point clouds, dense scene point clouds and sparse sampled point clouds) provided by the International Organization for Standardization are selected as training samples to lay the data foundation for training a repair network that adapts to diverse scenarios and ensure that the network can play a role in the subsequent repair tasks of different types of point clouds.

[0046] Furthermore, multiple compression quantization scales are employed to compress the initial point cloud training data. For example, based on the octree geometric coding scheme of the G-PCC compression standard, multiple position quantization scales (i.e., compression quantization scales) with different values ​​are used to batch compress the initial point cloud training data. Different compression quantization scales correspond to different compression intensities. When the scale value is large, the compressed data exhibits distortion characteristics primarily characterized by point vanishing; when the scale value is small, the distortion is mainly characterized by point shifting, and the proportion of point loss also differs at different scales. This generates multiple sets of compressed point cloud training data with different distortion types and degrees, simulating diverse scenarios of G-PCC compression in real-world applications, allowing the network to fully learn the repair patterns of various distortion modes during training.

[0047] Furthermore, network training is conducted based on the binary cross-entropy loss function. Specifically, the compressed point cloud training data corresponding to each compression and quantization scale is input into the initialized inpainting network. The binary cross-entropy loss function is used as the optimization objective. By measuring the difference between the network's inpainting output and the original point cloud data (true labels), clear guidance is provided for adjusting network parameters. During training, an adaptive moment estimation optimizer is used for parameter updates, where hyperparameters... Set to 0.9 Set to 0.999, initial learning rate set to Furthermore, a learning rate decay strategy is employed, reducing the learning rate to half its original value every 4000 training steps, with each model trained for a maximum of 20,000 steps to ensure full network convergence. Through continuous iterative training, the repair network gradually learns the repair logic for distortion problems such as point shift and point vanishing under different compression and quantization scales, ultimately resulting in a trained repair network that can accurately adapt to various compression scenarios and achieve stable repair effects.

[0048] Furthermore, the first data and the repaired data are merged using coordinate overlay. Specifically, the geometrically accurate and undistorted discrete points in the first data are retained, while supplementary points from the repaired data are incorporated to form optimized data containing both the original valid points and the repaired supplementary points. This ensures that the optimized data not only continues the core structural information of the original point cloud but also fills in the geometric gaps caused by compression, laying the foundation for subsequent confidence assessment and pruning.

[0049] In one embodiment, such as Figure 2 As shown, by applying optimization methods to compressed point cloud data, taking an end-to-end deep learning network for restoring geometric information of G-PCC compressed point clouds as an example, its overall network structure includes four parts: feature extraction module, coordinate reconstruction module, probability prediction module, and data pruning module, thus forming a complete multi-stage processing flow. Specifically, its multi-stage processing flow includes first acquiring the target bitstream, decoding the target bitstream through a G-PCC decoder to obtain the point cloud geometric position information, i.e., the G-PCC reconstructed point cloud coordinates, and then inputting the G-PCC compressed point cloud geometric position information (G-PCC reconstructed point cloud coordinates) into the feature extraction module. This module effectively extracts the local geometric features and global structural information of the point cloud through multi-level feature learning. Then, the coordinate reconstruction module, combined with the encoding parameter information parsed from the bitstream, accurately repairs the point offset and point disappearance problems caused by the point cloud compression process, and generates a dense candidate point set to fill in the missing geometric information. Finally, the probability prediction module evaluates the confidence level of each candidate point, and the data pruning module removes low-quality points to output a high-quality reconstructed point cloud (enhanced point cloud coordinates).

[0050] Furthermore, considering the inherent sparsity of point cloud data, sparse convolution is chosen as the basic unit for feature extraction in the overall network architecture. In the design of the feature extraction module, taking into account the sparsity of point cloud data, a U-Net network structure based on sparse convolution is adopted, such as... Figure 3 As shown, the encoder performs progressive downsampling through feature encoding units with four different input channels. Each feature encoding unit has a 3×3×3 kernel size, and the number of channels gradually increases from 32 to 256, effectively extracting multi-scale features from the point cloud. The downsampling operation is implemented with a convolution stride of 2, and the index information of the point cloud is recorded at each downsampling stage to support the subsequent upsampling process. The decoder then uses sparse deconvolution and skip connections to achieve feature upsampling and fusion, gradually restoring the spatial details of the point cloud.

[0051] Furthermore, such as Figure 4 The diagram shows the network structure of the feature encoding unit. Specifically, the input to the feature encoding unit is downsampled through sparse convolutional layers with a kernel size of 3×3×3 to compress the data scale and extract basic local structural information. Then, it passes through the ReLU activation function to enhance feature representation. Finally, the features are fed into multi-scale convolutional layers for deeper feature mining, enabling the network to simultaneously capture both fine local features and global structural features of the point cloud, providing rich feature representations for subsequent reconstruction tasks. The multi-scale convolutional layers include sparse convolutional layers with different kernel sizes, activation functions, and concatenation layers.

[0052] Furthermore, such as Figure 5 The diagram shows the network structure of the coordinate reconstruction module. Specifically, the input to the coordinate reconstruction module includes multi-scale features output from the feature extraction module and position quantization scale parameters (encoding parameters) parsed from the bitstream. Different position quantization scales significantly affect the compression effect of the point cloud. When the position quantization scale value is large, the point cloud exhibits obvious point vanishing, while a small position quantization scale value mainly manifests as point offset. Therefore, an adaptive weight selection mechanism is used to select the most suitable model weights from four pre-trained sub-networks based on the input position quantization scale value. These four sub-networks optimize the model for the preset "position quantization scale" in the G-PCC reference software, thereby achieving targeted reconstruction of point clouds with different compression intensities.

[0053] Specifically, during the model training phase of each sub-network, a portion of point cloud data provided by the International Organization for Standardization (ISO) was used as training samples. The specific training process included: firstly, the point cloud data in the training set was compressed using different quantization scales to generate compressed point cloud samples with varying degrees of distortion. For example, an octree-based compression method was used, compressing the point clouds by setting multiple different quantization scales to generate diverse training data. Secondly, regarding the loss function, the binary cross-entropy (BCE) loss was chosen as the optimization objective, which helps to effectively evaluate the model's predictive performance during training. Finally, model optimization employed the Adaptive Moment Estimation Optimizer (Adam), where the hyperparameters... and The values ​​were set to 0.9 and 0.999 respectively. During training, the initial learning rate was set to... A learning rate decay strategy is adopted, which reduces the learning rate to half of its original value every 4,000 training steps, and each model is trained for a maximum of 20,000 steps to ensure full convergence.

[0054] Furthermore, such as Figure 6 The diagram shows the network structure of the probability prediction module. Specifically, the goal of the probability prediction module is to evaluate the quality of the candidate point set output by the coordinate reconstruction module and predict the probability value of each candidate point. These predicted probability values ​​are then fed into a pruning unit, where low-confidence points are removed through thresholding and other operations, ultimately outputting an optimized reconstructed point cloud. The input to the probability prediction module is the sum of the coordinates of the candidate point set output by the coordinate reconstruction module and the coordinates of the G-PCC reconstructed point cloud. The input data is first processed through sparse convolutional layers for feature extraction, followed by a non-linear transformation using the ReLU activation function. Then, the features are processed through a sub-attention mechanism based on k-nearest neighbor search to predict their probabilistic features based on the 32 nearest neighbors of the center point. Finally, the features are processed through multi-scale convolutional layers to obtain the final probability prediction result.

[0055] Furthermore, to comprehensively evaluate the performance of this method, systematic tests were conducted on various types of point cloud data in this embodiment, including solid object point clouds, dense scene point clouds, and sparsely sampled point clouds. During the experiment, the G-PCC reference software was first used to compress the various test point clouds at different quantization scales to generate compressed point cloud samples with different degrees of distortion. These samples were then input into the method for geometric information recovery.

[0056] Specifically, the point-to-point peak signal-to-noise ratio (PSNR D1) is used as an objective quality evaluation index to quantify and compare the geometric accuracy of the point cloud before and after restoration. As shown in Table 1, on different test sequences, the method in this embodiment improves the PSNR D1 by an average of 1.9215 dB, significantly improving the visual quality and geometric fidelity of the compressed point cloud. This result fully demonstrates that the proposed method has good restoration effect and versatility under various point cloud types and compression intensities. Experimental results show that the proposed method can effectively address the geometric information loss problem caused by G-PCC compression, exhibiting superior performance in preserving point cloud structure and detailed features, and is suitable for various application scenarios requiring high-quality reconstruction of compressed point clouds.

[0057] Table 1

[0058] Therefore, this embodiment introduces a probabilistic prediction module into the point cloud upsampling framework to adaptively determine the target number of points in the reconstructed point cloud, and incorporates encoding parameters as model input, effectively improving the matching accuracy of the reconstruction results in terms of density and spatial distribution. The probabilistic prediction module possesses good versatility and compatibility, serving as a plug-and-play component that can be seamlessly integrated into different point cloud upsampling methods and compression-reconstruction processes. By inputting encoding parameters into the model, it can dynamically infer the optimal number of reconstructed points based on the characteristics of the input point cloud and the differences in its encoding / compression parameters, thereby generating reconstruction results that better meet actual needs in terms of density and spatial distribution, achieving accurate and adaptive reconstruction effects. Moreover, through joint training on different types of point cloud data and various encoding parameters, this embodiment demonstrates excellent generalization performance and reconstruction quality under diverse data and parameter configurations.

[0059] In some embodiments, extracting local geometric features and global structural features from the first data in step S120 includes the following steps S210 to S240: Step S210: Perform sparse convolution processing on the first data to obtain the third data and basic local structure information; Step S220: Perform a nonlinear transformation on the third data to obtain the nonlinearly transformed features; Step S230: Perform multi-scale convolution processing on the features after nonlinear transformation to obtain local geometric features; Step S240: Generate global structural features based on basic local structural information and local geometric features.

[0060] In this embodiment, sparse convolution is used for core processing to address the inherent sparsity of the first data (the set of discrete point data after G-PCC decoding). Specifically, by setting a convolution kernel adapted to the spatial distribution of the point cloud, convolution operations are performed on the spatial coordinates and associated features of the discrete points. This avoids invalid computation of empty voxels, improves processing efficiency, and compresses the redundant scale of the data to obtain intermediate feature data (i.e., the third data) with optimized dimensions. Simultaneously, during the convolution process, the proximity relationships, local density distribution, and basic structural association information between discrete points are captured to form basic local structural information, providing underlying data support for subsequent feature deepening and global integration.

[0061] Furthermore, the third data is input into a preset activation function for nonlinear transformation. Specifically, a nonlinear mapping relationship is introduced through the activation function to break the limitations of linear processing, enhance the discriminative power of effective features in the third data, suppress invalid and redundant information, and enable features to accurately capture the complex relationships of local geometric shapes in the point cloud (such as implicit features like surface curvature and edge contours), ultimately outputting more representative nonlinearly transformed features.

[0062] Furthermore, multi-scale convolution processing is performed on the features after nonlinear transformation. Specifically, by setting convolution kernels of different scales and a hierarchical processing flow, local geometric relationships of different ranges are captured, including: small-scale convolution kernels focus on fine details in the near-field region around discrete points (such as positional deviations of adjacent points, small structural protrusions, etc.), and large-scale convolution kernels cover local region relationships of a wider range (such as the overall trend of local surfaces, the connection relationship between adjacent regions, etc.). Then, by fusing and integrating the features extracted at multiple scales, local geometric features that can comprehensively represent the local details and structural relationships of the point cloud are formed, ensuring that point offset and missing point problems can be accurately located during subsequent restoration.

[0063] Furthermore, based on the basic local structural information and local geometric features, global information aggregation processing is performed. Specifically, by globally traversing and integrating the basic local structural information and local geometric features of all discrete points, the spatial distribution pattern, overall morphological features, and cross-regional structural relationships of the point cloud are extracted (such as the overall outline of the solid point cloud, the global distribution trend of the sparse point cloud, etc.). This eliminates the limitations of local features and generates global structural features that can reflect the overall structural attributes of the point cloud, providing a key basis for ensuring the consistency of the overall structure of the point cloud during subsequent repair processes.

[0064] In some embodiments, in step S150, the first data is repaired using a repair network corresponding to the compression distortion type, based on global structural features, deviation range, and missing midpoint regions of the first data, to obtain the second data, including the following steps S310 to S330: Step S310: Based on the compression distortion type, adapt the corresponding repair network in the database; the database contains the mapping relationship between compression distortion types and repair networks, and the model weights of each repair network in the database are different; Step S320: Repair the first data according to the deviation range by repairing the network to obtain the fourth data; Step S330: By repairing the network and based on the global structural features, the fourth data is repaired to obtain the second data.

[0065] In this embodiment, the precise invocation of the repair network is achieved based on the coordinated adaptation of compression distortion type and encoding parameters. Specifically, a multi-dimensional mapping relationship between compression distortion type (point offset dominant, point vanishing dominant), encoding parameters (with the core being the position quantization scale parameter), and repair networks is pre-established in the database. Each repair network is a sub-network pre-trained for different compression scenarios, and the differences in their model weights stem from adapting to different distortion modes and compression intensities.

[0066] For example, for the repair network dominated by point offset (when the quantization scale of the corresponding position is small), the weights are optimized to enhance the accuracy of coordinate correction; for the repair network dominated by point disappearance (when the quantization scale of the corresponding position is large), the weights focus on the rationality of the missing point supplementation.

[0067] Furthermore, during the adaptation process, the mapping range is first locked based on the type of compression distortion, and then the filtering is refined by combining the position quantization scale parameter in the encoding parameters. Finally, the repair network that is fully matched with the distortion features and compression intensity of the current first data is selected from the database, providing model support for targeted repair.

[0068] Furthermore, the adapted repair network invokes a preset coordinate correction algorithm, using the deviation range of each discrete point as the core basis, and combining the model weights adapted by the repair network (optimized coordinate correction parameters) to calculate the reasonable correction amount for each offset point. Specifically, by analyzing the deviation direction and value of the discrete point, as well as its spatial relationship with surrounding valid points, the coordinates of the offset points are adjusted point by point, so that each discrete point returns to an ideal position that conforms to the local geometric features.

[0069] Furthermore, based on global structural features, the repair network first performs precise location verification of the missing point regions in the fourth data (combining the global structure to determine the boundaries, range, and connection relationship of the missing regions with the overall structure); then, based on the adapted model weights (optimized missing point generation parameters), it generates supplementary points in the missing regions that conform to the global structural rules.

[0070] Specifically, the spatial location and density distribution of the supplementary points are referenced to the overall shape and spatial distribution trend in the global structural features to ensure that the supplementary points are seamlessly connected with the surrounding repaired valid points and do not damage the overall structural integrity of the point cloud. Finally, the overall data after repair is checked for structural consistency, and possible conflict points are eliminated. The final output is the second data (repaired point cloud data) with completely corrected point offsets, sufficient filling of missing points and coherent overall structure.

[0071] In some embodiments, the first data and the second data are merged in step S160 to obtain optimized data, including the following steps S410 to S430: Step S410: Merge the first data and the second data to obtain merged data; Step S420: Predict the confidence level of each discrete point in the merged data using a probabilistic prediction network; Step S430: Based on the confidence level of each discrete point in the merged data, the merged data is pruned to obtain optimized data.

[0072] In this embodiment, a coordinate overlay method is used to achieve deep fusion of the first and second data, generating merged data that comprehensively covers valid information. The first data is the original discrete point set after G-PCC decoding, containing valid points that are undistorted or only slightly distorted. The second data is the repaired point set processed by the repair network, covering offset points corrected by coordinates and supplementary points filling missing areas. During merging, point cloud spatial coordinates are used as the core basis, and the discrete points of the two types of data are overlaid and integrated at the coordinate level. This process fully preserves valid points in the first data with accurate geometric positions and high consistency with the overall structure, while also incorporating the corrected and supplementary points from the second data. Simultaneously, redundant points with completely overlapping coordinates are automatically filtered out, ultimately forming merged data containing both original valid information and repaired supplementary information, providing a comprehensive point cloud data foundation for subsequent confidence assessment.

[0073] Furthermore, a probabilistic prediction network is used to quantitatively evaluate the quality of each discrete point in the merged data, accurately outputting the confidence score of each point. Specifically, the probabilistic prediction network takes the merged data (i.e., the coordinate superposition result of the first and second data) as input. First, a sparse convolutional layer is used to extract features from the merged data, adapting to the sparsity characteristics of point cloud data, avoiding the waste of computational resources caused by empty voxels, and efficiently capturing the basic features of discrete points. Then, the extracted features are input into an activation function for nonlinear transformation to enhance the discriminative power of effective features and suppress invalid and redundant information. Next, a sub-attention mechanism based on nearest neighbor search is used to obtain the feature information of several neighboring points around each discrete point. The spatial correlation and structural fit of the neighboring points are combined to assist in the confidence score prediction of the point, improving the accuracy of the evaluation. Finally, a multi-scale convolutional layer is used to comprehensively process the features, integrating the feature correlation information at different scales, and outputting the confidence score of each discrete point. The confidence score directly reflects the geometric position accuracy of the point and its fit with the overall structure of the point cloud. High confidence scores correspond to high-quality points, and low confidence scores correspond to low-quality points (such as improperly repaired supplementary points or points with severe original distortion that have not been fully corrected).

[0074] Furthermore, based on the confidence level of each discrete point in the merged data, the merged data is pruned to obtain optimized data, including: removing discrete points in the merged data whose confidence level is less than a preset confidence threshold to obtain optimized data.

[0075] Specifically, precise cropping of merged data is achieved through confidence threshold filtering, ultimately yielding optimized data. First, a confidence threshold is pre-set based on the accuracy requirements of the point cloud application scenario (this threshold has been verified through extensive experiments to balance point cloud density and geometric fidelity). Then, the confidence level of each discrete point in the merged data is compared one by one with the pre-set confidence threshold. Low-quality discrete points with confidence levels below the threshold are removed; these typically include invalid supplementary points generated during the repair process that do not conform to local geometric features or global structure, and points in the original decoded data that are severely distorted and not fully corrected. High-quality discrete points with confidence levels above or equal to the threshold are retained; these include valid, undistorted points in the first data set, and correction and supplementary points in the second data set with excellent repair results. After cropping, the output optimized data ensures both the geometric accuracy and detail integrity of the point cloud while avoiding interference from redundant low-quality points. The overall structure is coherent, and the point distribution is reasonable, providing high-quality data support for downstream tasks such as object detection and scene segmentation.

[0076] In some embodiments, in step S420, the confidence level of each discrete point in the merged data is predicted using a probabilistic prediction network, including the following steps S510 to S530: Step S510: Perform sparse convolution processing on the merged data to extract the feature information of the merged data; Step S520: Input the feature information into the second activation function for nonlinear transformation to obtain the features of each discrete point in the merged data; Step S530: Based on the attention mechanism, the features of each discrete point in the merged data are processed by multi-scale convolution through a probabilistic prediction network to obtain the confidence level of each discrete point in the merged data.

[0077] In this embodiment, sparse convolution is used for feature extraction to address the sparsity of the merged data (the result of superimposing the coordinates of the first and second data), ensuring efficient and accurate capture of core information. The merged data includes original valid points, corrected points, and supplementary points, and its spatial distribution still exhibits the inherent sparsity of point cloud data. Using sparse convolution avoids the ineffective computation on empty voxels inherent in traditional convolution, significantly improving processing efficiency.

[0078] Specifically, by setting convolution kernels adapted to the spatial structure of point clouds, convolution operations are performed on the spatial coordinates of each discrete point in the merged data and its positional association with surrounding points. While compressing the redundant dimensions of the data, initial feature information that can reflect the basic geometric attributes and local correlation features of discrete points is efficiently extracted, laying the underlying data foundation for subsequent feature deepening and confidence prediction.

[0079] Furthermore, the extracted feature information is input into a second activation function (such as ReLU) for nonlinear transformation, enhancing the expressive power and discriminative power of the features. The initial feature information extracted linearly is insufficient to characterize the complex relationships of local geometric morphology in point clouds and the subtle differences in the quality of discrete points. By introducing a nonlinear mapping relationship through the second activation function, the limitations of linear processing can be overcome, thereby amplifying the differences in effective features (such as the fit between discrete points and surrounding points, and features related to positional accuracy) and suppressing invalid redundant information (such as irrelevant features generated by noise interference). This allows the features of each discrete point output after transformation to more accurately represent the geometric quality attributes of that point, providing highly discriminative feature support for subsequent accurate confidence assessment.

[0080] Furthermore, by combining attention mechanisms with multi-scale convolutional processing, accurate quantification of discrete point confidence is achieved. Specifically, firstly, based on an attention mechanism (a sub-attention mechanism based on nearest neighbor search), several neighboring points are matched for each discrete point in the merged data. By mining features such as spatial correlation and structural fit between the current point and its neighbors, the quality of the current point is assessed. Specifically, if the geometric connection between the current point and its neighbors is natural and conforms to local structural patterns, its feature weight is increased; otherwise, it is decreased, thereby enhancing the specificity of confidence prediction.

[0081] Furthermore, the features of each discrete point are processed hierarchically through multi-scale convolutional layers in the probabilistic prediction network. Small-scale convolutional kernels focus on the local detail matching degree between the current point and its neighboring points, while large-scale convolutional kernels focus on the fit between the current point and the global structure of the point cloud. By integrating feature association information at different scales, the geometric position accuracy and consistency with the overall structure of the discrete point are comprehensively evaluated. Finally, the confidence score of each discrete point is output through network comprehensive calculation. This confidence score directly quantifies the quality level of the discrete point, providing a clear judgment basis for subsequent pruning processing.

[0082] like Figure 7 As shown in some embodiments of this application, an optimization system for compressed point cloud data is provided. The system includes an acquisition module 710, an extraction module 720, an adaptation module 730, a determination module 740, a repair module 750, and a merging module 760. Specifically: The acquisition module 710 is used to acquire first data and second data; the first data is a set of discrete points obtained by decoding the initial point cloud data after G-PCC processing; the second data is the encoding parameters used when performing G-PCC processing on the initial point cloud data. Extraction module 720 is used to extract local geometric features and global structural features from the first data; The adaptation module 730 is used to adapt the compression distortion type of the first data according to the encoding parameters; The determination module 740 is used to determine the deviation range of each discrete point in the first data and the missing point region in the first data based on local geometric features; Repair module 750 is used to repair the first data by using a repair network corresponding to the compression distortion type, based on global structural features, deviation range and missing regions of the first data, to obtain the second data; The merging module 760 is used to merge the first data and the second data to obtain optimized data.

[0083] It should be noted that the optimization system for compressed point cloud data provided in this embodiment and the optimization method for compressed point cloud data described above are based on the same inventive concept. Therefore, the relevant content of the optimization method for compressed point cloud data described above also applies to the content of the optimization system for compressed point cloud data. Therefore, it will not be repeated here.

[0084] In this embodiment, the system acquires first data and second data; extracts local geometric features and global structural features from the first data; adapts the compression distortion type of the first data according to the encoding parameters; determines the deviation range of each discrete point in the first data and the missing point region in the first data based on the local geometric features; repairs the first data using a repair network corresponding to the compression distortion type, based on the global structural features, the deviation range of each discrete point in the first data, and the missing point region in the first data, to obtain the second data; and merges the first data and the second data to obtain optimized data. In this way, it is possible to adapt the distortion type under different compression intensities by using encoding parameters, combine local and global features to achieve targeted repair, optimize the compressed point cloud data, and thus improve the geometric fidelity and visual quality of the compressed point cloud data.

[0085] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method for optimizing compressed point cloud data.

[0086] like Figure 8 , Figure 8 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device includes: At least one battery; At least one memory; At least one processor; At least one program; The program is stored in memory, and the processor executes at least one program to implement the above-described method for optimizing compressed point cloud data according to the present disclosure.

[0087] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0088] The electronic devices according to embodiments of this application will now be described in detail.

[0089] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this disclosure. The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to perform an optimization method for compressed point cloud data according to an embodiment of this disclosure.

[0090] The input / output interface 1800 is used to implement information input and output. The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900); The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0091] This disclosure also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described method for optimizing compressed point cloud data.

[0092] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0093] The embodiments described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided by this disclosure. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by this disclosure are also applicable to similar technical problems.

[0094] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this disclosure, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0095] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0096] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any related variations, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0097] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0098] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0099] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0100] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0101] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0102] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

[0103] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. A method for optimizing compressed point cloud data, characterized in that, The method includes: Acquire first data and second data; the first data is a set of discrete points obtained by decoding the initial point cloud data after G-PCC processing; the second data is the encoding parameters used when performing G-PCC processing on the initial point cloud data; Extract local geometric features and global structural features from the first data; Based on the encoding parameters, the compression distortion type of the first data is adapted; Based on the local geometric features, determine the deviation range of each discrete point in the first data and the missing point region in the first data. By using the repair network corresponding to the compression distortion type, the first data is repaired based on the global structural features, the deviation range, and the missing midpoint region of the first data, to obtain the second data; The first data and the second data are combined to obtain optimized data.

2. The method for optimizing compressed point cloud data according to claim 1, characterized in that, The extraction of local geometric features and global structural features from the first data includes: The first data is subjected to sparse convolution processing to obtain the third data and basic local structure information; The third data is subjected to a nonlinear transformation to obtain the nonlinearly transformed features. The local geometric features are obtained by performing multi-scale convolution on the features after the nonlinear transformation. Based on the basic local structural information and the local geometric features, the global structural features are generated.

3. The method for optimizing compressed point cloud data according to claim 1, characterized in that, The method involves using a repair network corresponding to the compression distortion type to repair the first data based on the global structural features, the deviation range, and the missing midpoint region of the first data, to obtain the second data, including: Based on the compression distortion type, a corresponding repair network is adapted in the database; the database contains the mapping relationship between the compression distortion type and the repair network, and the model weights of each repair network in the database are different; The first data is repaired using the repair network according to the deviation range to obtain the fourth data; The repair network repairs the fourth data based on the global structural features to obtain the second data.

4. The method for optimizing compressed point cloud data according to claim 1, characterized in that, The repair network is trained through the following steps: Obtain initial point cloud training data; The initial point cloud training data is compressed using multiple compression quantization scales to obtain compressed point cloud training data corresponding to each compression quantization scale. Based on the binary cross-entropy loss function, the initial repair network is trained using the compressed point cloud training data corresponding to each compression quantization scale, resulting in the trained repair network.

5. The method for optimizing compressed point cloud data according to claim 1, characterized in that, The step of merging the first data and the second data to obtain optimized data includes: The first data and the second data are merged to obtain the merged data; The confidence level of each discrete point in the merged data is predicted using a probabilistic prediction network. Based on the confidence level of each discrete point in the merged data, the merged data is pruned to obtain the optimized data.

6. The method for optimizing compressed point cloud data according to claim 5, characterized in that, The step of predicting the confidence level of each discrete point in the merged data using a probabilistic prediction network includes: The merged data is subjected to sparse convolution processing to extract its feature information; The feature information is input into the second activation function for nonlinear transformation to obtain the features of each discrete point in the merged data. Based on the attention mechanism, the features of each discrete point in the merged data are processed by multi-scale convolution through the probability prediction network to obtain the confidence level of each discrete point in the merged data.

7. The method for optimizing compressed point cloud data according to claim 5, characterized in that, The step of pruning the merged data based on the confidence level of each discrete point in the merged data to obtain the optimized data includes: removing discrete points in the merged data whose confidence level is less than a preset confidence threshold to obtain the optimized data.

8. A system for optimizing compressed point cloud data, characterized in that, The system includes: The acquisition module is used to acquire first data and second data; the first data is a set of discrete points obtained by decoding the initial point cloud data after G-PCC processing; the second data is the encoding parameters used when performing G-PCC processing on the initial point cloud data. The extraction module is used to extract local geometric features and global structural features from the first data; An adaptation module is used to adapt the compression distortion type of the first data according to the encoding parameters; The determination module is used to determine the deviation range of each discrete point in the first data and the missing point region in the first data based on the local geometric features. The repair module is used to repair the first data based on the global structural features, the deviation range, and the missing midpoint region of the first data through the repair network corresponding to the compression distortion type, so as to obtain the second data. The merging module is used to merge the first data and the second data to obtain optimized data.

9. An electronic device, characterized in that, It includes at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, which, when executed by the at least one control processor, enable the at least one control processor to perform an optimization method for compressed point cloud data as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform a method for optimizing compressed point cloud data as described in any one of claims 1 to 7.

Images