Data compression method, electronic device, storage medium, and computer program product

Abstract

The application provides a data compression method, an electronic device, a storage medium and a computer program product; the method comprises the following steps: obtaining an anchor point set of a target three-dimensional scene; querying a general feature pool of the target three-dimensional scene based on the positions of the anchor points in the anchor point set to obtain general features of the anchor points; inputting the general features of the anchor points into a Gaussian compression model to obtain a rendered image of the target three-dimensional scene; training the general feature pool, the Gaussian compression model and attribute features of the anchor points based on the rendered image and a supervised image of the target three-dimensional scene to obtain a trained general feature pool, a trained Gaussian compression model and attribute parameters of the anchor points; and compressing the trained general feature pool, the trained Gaussian compression model and the attribute parameters of the anchor points. According to the application, the storage and transmission overhead of three-dimensional Gaussian can be reduced.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data compression method, electronic device, storage medium, and computer program product. Background Technology

[0002] In the fields of computer graphics and 3D vision, novel perspective synthesis is one of the core tasks of 3D scene reconstruction. Its goal is to generate images of any unknown perspective of a scene using image data from a limited number of known perspectives. It is a key technological foundation for realizing applications such as virtual reality, augmented reality, and immersive roaming.

[0003] In recent years, 3D Gaussian Splatting (3DGS) technology has become an important method for supporting the synthesis of new perspectives due to its efficient scene representation and differentiable rendering capabilities. 3D Gaussian technology fits scenes using three-dimensional Gaussian primitives (such as 3D Gaussian spheres) to achieve real-time rendering of images from new perspectives, demonstrating significant advantages in both reconstruction accuracy and rendering efficiency.

[0004] To ensure the accuracy of scene reconstruction and the quality of new perspective synthesis, 3D scenes often require a large number of 3D Gaussian primitives. Each 3D Gaussian primitive needs to independently store multiple sets of attribute parameters, resulting in a huge overhead in storage and transmission for 3D Gaussian technology, which limits the practical application of 3D Gaussian technology. Summary of the Invention

[0005] To address the related technical problems, embodiments of this application provide a data compression method, an electronic device, a storage medium, and a computer program product.

[0006] The technical solution of this application embodiment is implemented as follows: This application provides a data compression method, the method comprising: Obtain the set of anchor points for the target 3D scene; Based on the position of each anchor point in the anchor point set, the general feature pool of the target 3D scene is queried to obtain the general features of each anchor point; wherein, the general feature pool is used to store the general features of the target 3D scene; The common features of each anchor point are input into the Gaussian compression model to obtain the rendered image of the target 3D scene; Based on the rendered image and the supervised image of the target 3D scene, the general feature pool, the Gaussian compression model and the attribute features of each anchor point are trained to obtain the trained general feature pool, Gaussian compression model and attribute parameters of each anchor point. The trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point are compressed.

[0007] In the above scheme, the attribute parameters of each anchor point include characteristic features; the Gaussian compression model includes a prediction network and a compensation network; the step of inputting the general features of each anchor point into the Gaussian compression model to obtain the rendered image of the target 3D scene includes: The common features of each anchor point are input into the prediction network to obtain the first parameters of each three-dimensional Gaussian element. The characteristic features of each anchor point are input into the compensation network to obtain the second parameters of each three-dimensional Gaussian element. The first parameter and the second parameter are fused together to obtain the rendering parameters of each three-dimensional Gaussian primitive; The rendered image of the target 3D scene is obtained based on the rendering parameters of each 3D Gaussian primitive.

[0008] In the above scheme, the general feature pool includes multiple three-dimensional hash maps and multiple two-dimensional hash maps, with different hash maps corresponding to different resolution grid structures; the three-dimensional hash maps are used to store three-dimensional features, and the two-dimensional hash maps are used to store two-dimensional features; the step of querying the general feature pool of the target three-dimensional scene based on the position of each anchor point in the anchor point set to obtain the general features of each anchor point includes: Based on the position of each anchor point in the anchor point set, each three-dimensional hash mapping table and each two-dimensional hash mapping table are queried to obtain the general characteristics of each anchor point.

[0009] In the above scheme, the step of querying each three-dimensional hash map and each two-dimensional hash map based on the position of each anchor point in the anchor point set to obtain the common characteristics of each anchor point includes: Based on the position of each anchor point in the anchor point set, query each three-dimensional hash mapping table to obtain the three-dimensional features found in each three-dimensional hash mapping table; Based on the position of each anchor point in the anchor point set, query each two-dimensional hash mapping table to obtain the two-dimensional features found in each two-dimensional hash mapping table; The three-dimensional features retrieved from each three-dimensional hash map table and the two-dimensional features retrieved from each two-dimensional hash map table are concatenated to obtain the common features of each anchor point.

[0010] The method in the above scheme further includes: In each training round, the downsampling scale parameter corresponding to each training round is determined based on the iteration round number; wherein, the iteration round number is negatively correlated with the downsampling scale parameter. The scene image of the target 3D scene is downsampled according to the downsampling scale parameter to obtain the supervision image for each training round; wherein, as the number of iterations increases, the image resolution of the supervision image continuously improves.

[0011] In the above scheme, the step of training the general feature pool, the Gaussian compression model, and the attribute features of each anchor point based on the rendered image and the supervised image of the target 3D scene to obtain the trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point includes: Based on the rendered image of the target 3D scene and the supervised image, the 3D reconstruction loss is determined; Based on the attribute parameters of each anchor point and the general feature pool, the bitrate constraint loss is determined. The masking loss is determined based on the masking weights of each 3D Gaussian primitive; wherein the masking weights represent the renderability of the corresponding 3D Gaussian primitive. Based on the 3D reconstruction loss, the bitrate constraint loss, and the masking loss, the general feature pool, the Gaussian compression model, and the attribute parameters of each anchor point are trained to obtain the trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point.

[0012] In the above scheme, the compression of the trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point includes: Determine the probability distribution of the attribute parameters of each anchor point; The attribute parameters of each anchor point are entropy encoded based on the probability distribution.

[0013] This application provides an electronic device, including: a processor and a memory for storing computer programs capable of running on the processor; The processor is used to execute the data compression method provided in the embodiments of this application when running a computer program.

[0014] This application provides a storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the data compression method provided in this application.

[0015] This application provides a computer program product, including a computer program or computer executable instructions, which, when executed by a processor, implements the data compression method provided in this application.

[0016] The data compression method, electronic device, storage medium, and computer program product provided in this application decouple the anchor point features of the target 3D scene by constructing a general feature pool for storing general features of the target 3D scene. The features of each anchor point are split into general features and specific features. The general features of each anchor point can be obtained by querying the general feature pool based on the location of each anchor point. Therefore, each anchor point does not need to store the general part of the features, but only the differentiated part, reducing the data size of each anchor point. After obtaining the general features of each anchor point, the general features of each anchor point are input into a Gaussian compression model. The Gaussian compression model is used to generate a rendered image of the target 3D scene. Based on the rendered image of the target 3D scene and the supervised image, supervised training is performed on the general feature pool, the Gaussian compression model, and the attribute features of each anchor point. After training, the optimized general feature pool, Gaussian compression model, and attribute parameters of each anchor point are compressed. By decoupling the anchor point features that originally needed to be explicitly stored in the bitstream in the target 3D scene reconstruction data into a shared general feature pool and specific features, the redundant storage of similar features between anchor points is effectively reduced, the storage and transmission overhead of 3D Gaussian is reduced, and the application scope of 3D Gaussian technology is expanded. Attached Figure Description

[0017] Figure 1 This is a schematic flowchart of a data compression method provided in an embodiment of this application; Figure 2 This is a schematic diagram illustrating an application example of a data compression method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a data compression device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the application will now be described in further detail with reference to the accompanying drawings.

[0019] 3D reconstruction technology is a core technology in fields such as computer vision and virtual reality. Among them, novel perspective synthesis is a core research direction in 3D reconstruction technology. The goal of novel perspective synthesis is to generate rendered images of a 3D scene from an unknown perspective, based on image data of a limited number of known perspectives, through 3D scene modeling and image rendering.

[0020] 3D Gaussian is a key technology for 3D reconstruction and new perspective synthesis. 3D Gaussian fits a 3D scene with a large number of 3D Gaussian primitives possessing multi-dimensional attributes such as position, scale, rotation, transparency, and color. Combined with a differentiable rendering method, it enables real-time rendering of 3D scenes, improving the efficiency and accuracy of 3D scene reconstruction and rendering.

[0021] However, 3D Gaussian scene representation methods require storing a large number of attribute parameters of 3D Gaussian primitives, resulting in high storage overhead and transmission costs, which limits the application of 3D Gaussian technology in practical scenarios. In view of this, 3D Gaussian compression technology has emerged. 3D Gaussian compression reduces the storage overhead and transmission costs of 3D Gaussian primitives while preserving the quality of the rendered image through pruning, quantization, encoding, or structural reorganization.

[0022] 3D Gaussian compression technology introduces an anchor point structure to achieve a compact representation of 3D scenes and simplify the data structure of 3D scenes. Anchor points are the basic units for 3D scene reconstruction. Each anchor point has four attribute parameters: position, feature, scale, and offset. Using the attribute parameters of the anchor points, 3D Gaussian primitives surrounding the anchor points can be generated. These primitives then adaptively fit the 3D scene, achieving 3D scene rendering.

[0023] In 3D scenes, anchor points that are spatially adjacent share similar scene features and have a large number of shareable features. However, current 3D Gaussian compression schemes store the corresponding high-dimensional features independently for each anchor point, failing to utilize the spatial continuity of 3D scenes. This results in severe feature redundancy between neighboring anchor points, leading to a waste of storage space.

[0024] Based on this, this application provides a data compression scheme. By constructing a general feature pool for storing common features of the target 3D scene, the anchor point features of the target 3D scene are decoupled. The features of each anchor point are split into general features and specific features. The general features of each anchor point can be obtained by querying the general feature pool based on the location of each anchor point. Therefore, each anchor point does not need to store the general part of the features, but only the differentiated part, reducing the data size of each anchor point. After obtaining the general features of each anchor point, the general features of each anchor point are input into a Gaussian compression model. The Gaussian compression model is used to generate a rendered image of the target 3D scene. Based on the rendered image of the target 3D scene and the supervision image, supervised training is performed on the general feature pool, the Gaussian compression model, and the attribute features of each anchor point. After training, the optimized general feature pool, Gaussian compression model, and attribute parameters of each anchor point are obtained. Then, the optimized general feature pool, Gaussian compression model, and attribute parameters of each anchor point are compressed to achieve compressed storage of the target 3D scene reconstruction data. By decoupling the anchor point features in the target 3D scene reconstruction data, which originally needed to be explicitly stored in the bitstream, into general features and specific features provided by a shared general feature pool, the redundant storage of similar features between anchor points is effectively reduced, thus reducing the size of the compressed bitstream.

[0025] The data compression scheme provided by embodiments of this application is described below with reference to the accompanying drawings. First, embodiments of this application provide a data compression method, which can be applied to compression systems or electronic devices. The electronic device can be a terminal device or a server. The data compression method provided by embodiments of this application will be described below using the electronic device as the execution subject.

[0026] like Figure 1 As shown, the data compression method provided in this application includes the following steps: Step 101: Obtain the set of anchor points for the target 3D scene; Step 102: Based on the position of each anchor point in the anchor point set, query the general feature pool of the target 3D scene to obtain the general features of each anchor point. The general feature pool is used to store the general features of the target 3D scene. Step 103: Input the common features of each anchor point into the Gaussian compression model to obtain the rendered image of the target 3D scene; Step 104: Based on the rendered image and supervised image of the target 3D scene, train the general feature pool, Gaussian compression model and attribute features of each anchor point to obtain the trained general feature pool, Gaussian compression model and attribute parameters of each anchor point. Step 105: Compress the trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point.

[0027] In this embodiment, the target 3D scene can be any 3D scene to be subjected to 3D Gaussian compression. The anchor point set of the target 3D scene includes all anchor points of the target 3D scene. Anchor points are structured nodes that characterize the spatial structure of the target 3D scene and are used to generate a set of 3D Gaussian primitives.

[0028] Each anchor point has attribute parameters such as position, characteristic features, scale, and offset.

[0029] Location is used to determine the spatial position of an anchor point within the target 3D scene. For example, the location attribute of an anchor point includes its spatial coordinates within the target 3D scene.

[0030] Characteristic features are used to provide differentiated scene features between different anchor points. For example, characteristic features represent local details specific to the target 3D scene, such as geometric deformations and appearance differences.

[0031] The scale is used to characterize the local spatial extent covered by the anchor point, providing scale constraints for the three-dimensional Gaussian primitives.

[0032] Offset is used to provide the position offset of a 3D Gaussian element. The spatial position of a set of 3D Gaussian elements corresponding to the anchor point can be determined by the offset attribute of the anchor point and the position of the anchor point.

[0033] In this embodiment, the high-dimensional features of the anchor point are split into two parts: general features and specific features. Each anchor point only has corresponding specific features, and the global general features are no longer stored as storage objects for anchor point encoding, thereby reducing the repeated storage of similar features between anchor points and saving storage space.

[0034] In practical applications, electronic devices can determine the set of anchor points for a target 3D scene based on multi-view images of the scene. For example, the electronic device acquires multi-view images of the target 3D scene, performs 3D reconstruction processing on these images, and generates point cloud data of the target 3D scene. The point cloud data includes basic information such as the spatial location, structure, and texture of the target 3D scene. The electronic device then optimizes the point cloud data, such as by performing sparse sampling and clustering filtering, to obtain the set of anchor points for the target 3D scene.

[0035] It is understandable that among the multiple attribute parameters of an anchor point, apart from the positional attribute parameter, the other attribute parameters are learnable. That is, the anchor point's characteristic features, scale, and offset are learnable attribute parameters. By jointly optimizing these learnable attribute parameters, these attribute parameters become more accurate, thereby improving image rendering quality. At the same time, these attribute parameters are optimized towards a more compact expression, reducing the redundancy of overall attribute parameters and bitrate, and lowering storage and transmission costs.

[0036] In practical applications, for the learnable attribute parameters among the multiple attribute parameters of anchor points, electronic devices can initialize the learnable attribute parameters of each anchor point in a random manner to complete the initial assignment of the learnable attribute parameters. For example, the dimension of the characteristic features of each anchor point can be set to 25, and the dimensions of the scale and offset of each anchor point can be set to 3.

[0037] To enable shared common features among all anchor points, this embodiment constructs a common feature pool for the target 3D scene, achieving centralized management of the common features of each anchor point. This reduces the probability of duplicate storage of common features of the target 3D scene and decreases feature redundancy. The common feature pool stores the common features of the target 3D scene. All anchor points in the target 3D scene share this common feature pool.

[0038] In some implementations, the electronic device can employ a hash mapping architecture to construct a general feature pool for the target 3D scene. For example, the general feature pool includes multiple 3D hash feature tables and multiple 2D hash mapping tables. The electronic device divides the target 3D scene into multiple 3D mesh structures and constructs a 3D hash mapping table corresponding to each 3D mesh structure using 3D coordinates as indexes. The 3D general features of the 3D mesh structure are stored in the corresponding 3D hash mapping table. Then, the electronic device projects the 3D mesh structure of the target 3D scene onto a 2D plane to obtain a 2D mesh structure, and constructs a 2D hash mapping table corresponding to the 2D mesh structure using 2D coordinates as indexes. The 2D general features of the 2D mesh structure are stored in the corresponding 2D hash mapping table. After constructing the 3D and 2D hash mapping tables, the electronic device integrates the 3D hash mapping table storing the 3D general features with the 2D hash mapping table storing the 2D general features to construct the general feature pool.

[0039] When querying the common features of each anchor point, the electronic device maps the location (such as spatial coordinates) of each anchor point to a hash index. This hash index is then used to query both a three-dimensional hash mapping table and a two-dimensional hash mapping table to obtain the common features of each anchor point. By constructing a feature pool through hash mapping, fast querying of common features can be achieved.

[0040] In some implementations, the general feature pool includes 3D hash maps corresponding to 3D mesh structures at various resolutions, and 2D hash maps corresponding to 2D mesh structures at various resolutions. For example, the general feature pool includes 13 3D hash maps at different resolutions and 4 2D hash maps at different resolutions. Resolution can be understood as the granularity of the mesh structure. The lower the resolution, the coarser the 3D spatial division of the mesh structure, and the general features stored in the corresponding hash map focus on representing the basic outline of the 3D scene; the higher the resolution, the finer the 3D spatial division of the mesh structure, and the general features stored in the corresponding hash map focus on representing the local details of the 3D scene.

[0041] Based on the position of each anchor point in the anchor point set, the electronic device queries each 3D hash map table and each 2D hash map table to obtain the 3D features of each anchor point corresponding to different resolution 3D mesh structures and the 2D features of different resolution 2D mesh structures. Then, the electronic device concatenates the retrieved 3D features with the 2D features to obtain the common features of each anchor point.

[0042] For example, the electronic device queries a 3D hash map corresponding to each resolution based on the spatial coordinates of each anchor point in the anchor point set, obtaining the 3D features found in each 3D hash map. It also queries 2D hash maps of different resolutions, obtaining the 2D features found in each 2D hash map. Then, the electronic device concatenates the 3D features found in each 3D hash map and the 2D features found in each 2D hash map to obtain the common features for each anchor point.

[0043] By fusing features corresponding to mesh structures of different resolutions, the overall consistency and local details of the 3D scene can be taken into account. This allows the queried general features to provide richer and more adaptable scene features, providing accurate feature support for the subsequently generated 3D scene rendering images.

[0044] To reduce the storage space occupied by the general feature pool, some implementations store only the vertex features of the mesh structure. For example, for a 3D mesh structure, each 3D mesh structure has 8 vertices. For a 2D mesh structure, each 2D mesh structure has 4 vertices. The general feature pool stores the vertex features of each 3D mesh structure and the vertex features corresponding to each 2D mesh structure. When querying the general features of each anchor point, the electronic device determines the 3D mesh structure and the 2D mesh structure to which the spatial coordinates of each anchor point belong.

[0045] Taking any anchor point as an example, for the 3D mesh structure to which the anchor point belongs, the electronic device determines the 8 vertex features of the 3D mesh structure in the general feature pool. Then, based on the relative position of the anchor point in the 3D mesh structure, it performs interpolation processing on the vertex features of the 3D mesh structure to obtain the 3D feature of the anchor point. For the 2D mesh structure to which the anchor point belongs, the electronic device determines the 4 vertex features of the 2D mesh structure in the general feature pool. Then, based on the relative position of the anchor point in the 2D mesh structure, it performs interpolation processing on the vertex features of the 2D mesh structure to obtain the 2D feature of the anchor point. The electronic device then concatenates the 3D and 2D features of the anchor point to obtain the general feature of the anchor point.

[0046] By storing only vertex features, there is no need to independently store the corresponding general features for all anchor points in the target 3D scene, effectively reducing the storage space occupied by the general feature pool.

[0047] After obtaining the common features of each anchor point, in step 103 above, the electronic device inputs the common features of each anchor point into the Gaussian compression model to be trained, uses the Gaussian compression model to generate rendering parameters of each three-dimensional Gaussian primitive, and generates a rendered image of the target three-dimensional scene based on the rendering parameters of each three-dimensional Gaussian primitive.

[0048] Gaussian compression models are machine learning paradigms used to compress 3D scene data. For example, a Gaussian compression model includes a prediction network and a rendering network. The prediction network predicts rendering parameters for 3D Gaussian primitives. The rendering network generates a rendered image of the 3D scene. An electronic device inputs common features from multiple anchor points into the prediction network of the Gaussian compression model, using the prediction network to generate rendering parameters for multiple 3D Gaussian primitives. Then, the electronic device uses the rendering network of the Gaussian compression model to generate a rendered image of the target 3D scene based on these rendering parameters.

[0049] 3D Gaussian primitives, as the fundamental rendering units for 3D scene reconstruction, are the basis of 3D scene rendering. The Gaussian compression model renders each 3D Gaussian primitive using its rendering parameters, resulting in individual 3D Gaussian primitives. By superimposing multiple 3D Gaussian primitives, a rendered image of the target 3D scene is obtained, restoring its geometric structure, texture, color, and lighting details.

[0050] Because general features lack local details at anchor points, some implementations of Gaussian compression models include a compensation network to improve rendering quality and accuracy. In this case, the rendering parameters include a first parameter and a second parameter. The prediction network outputs the first parameter, and the compensation network outputs the second parameter.

[0051] The first parameter provides the overall outline, spatial structure, and approximate lighting and shadow distribution of each 3D Gaussian primitive, and is the fundamental rendering parameter for the 3D Gaussian primitive. For example, the first parameter includes one or more (one or more can be understood as at least one) of the following fundamental rendering parameters: position, scale, rotation, transparency, and color.

[0052] The second parameter is used to provide local offsets, texture and lighting compensation for each 3D Gaussian primitive, and is a compensation parameter for the 3D Gaussian primitive. For example, the second parameter includes one or more compensation parameters such as geometric deformation and appearance supplement values.

[0053] For example, after the electronic device queries the general feature pool of the target 3D scene to find the general features of each anchor point, it inputs the general features of each anchor point into the prediction network of the Gaussian compression model, and uses the prediction network to predict the basic rendering parameters of each 3D Gaussian primitive, thus obtaining the first parameters of each 3D Gaussian primitive. Correspondingly, the electronic device inputs the characteristic features of each anchor point into the compensation network of the Gaussian compression model, and uses the compensation network to compensate for the local details of each 3D Gaussian primitive, thus obtaining the second parameters of each 3D Gaussian primitive.

[0054] After obtaining the first and second parameters of each 3D Gaussian primitive, the electronic device fuses the first and second parameters of each 3D Gaussian primitive to obtain the rendering parameters of each 3D Gaussian primitive. Then, the electronic device inputs the rendering parameters of each 3D Gaussian primitive into the rendering network. The rendering network obtains the rendered image of the target 3D scene based on the rendering parameters of multiple 3D Gaussian primitives. Through the superposition and fusion of the first and second parameters, the rendering parameters achieve both the stability and consistency of the overall structure of the target 3D scene and compensate for the loss of local details in the target 3D scene, thereby improving the overall quality of the rendered image.

[0055] After obtaining the rendered image of the target 3D scene, in step 104 above, the electronic device performs multiple rounds of training on the constructed general feature pool, Gaussian compression model, and attribute parameters of each anchor point based on the rendered image and the supervised image of the target 3D scene.

[0056] Supervised images are obtained from real-world images of the target 3D scene. For example, electronic devices use real-world images of the target 3D scene as supervised images for training, jointly training a general feature pool, a Gaussian compression model, and the attribute parameters of each anchor point.

[0057] To improve training effectiveness, some implementations divide the training process into two phases. When the number of training epochs is less than a preset number, the training process is in the first phase. In the first phase, the electronic device disables the compensation network of the Gaussian compression model and renders only using the first parameters generated by the prediction network as rendering parameters, obtaining a coarse-grained (e.g., first-grained) 3D Gaussian primitives-formed rendered image. In the first phase, the Gaussian compression model learns general features from the general feature pool alone to reconstruct the basic ensemble structure of the target 3D scene.

[0058] When the number of training epochs is greater than or equal to the number of predictions, the electronic device confirms that the training process has entered the second phase. In the second phase, the electronic device activates the compensation network of the Gaussian compression model. The first parameters generated by the prediction network and the second parameters generated by the compensation network are used together as rendering parameters to obtain a rendered image formed by fine-grained (e.g., second-grained) 3D Gaussian primitives. The accuracy of the second-grained model is greater than that of the first. In the second phase, the Gaussian compression model reconstructs the differential details of the target 3D scene by learning the characteristic features of anchor points.

[0059] In some implementations, a progressive training method can be used during training to dynamically adjust the resolution of the supervision images, thereby improving the training stability and reconstruction accuracy of the Gaussian compression model. In each training iteration, the electronic device determines the downsampling scale parameter corresponding to each training iteration, and then downsamples the scene image of the target 3D scene according to the downsampling scale parameter to obtain the supervision image for each training iteration.

[0060] The downsampling scale parameter controls the scaling ratio or sampling step size of the scene image. The downsampling scale parameter for each training round is negatively correlated with the number of iterations. That is, as the number of iterations increases during training, the downsampling scale parameter decreases, thereby continuously improving the resolution of the supervised image obtained through downsampling.

[0061] For example, the electronic device can employ a cosine function-based scheduler to dynamically generate the downsampling scale factor for each training round. This scheduler calculates the downsampling scale factor for each training round using a cosine trigonometric function, and the downsampling scale factor decreases according to the cosine curve as the number of iterations increases. The electronic device inputs the iteration number for each round into the scheduler to obtain the downsampling scale factor for each training round output by the scheduler. Then, the electronic device uses the downsampling scale factor to downsample the scene image, obtaining the supervision image for each training round. In this way, the resolution of the supervision image during training increases from low to high under the control of the downsampling scale factor, and the rate of change of resolution is proportional to the rate of change of the cosine function, thus achieving a continuous and smooth transition in resolution.

[0062] In this approach, the supervisory images are used at low resolution during the initial training phase. These low-resolution images allow the Gaussian compression model to converge quickly, prioritizing the learning of the basic geometric structure of the target 3D scene. As the number of iterations increases, the resolution of the supervisory images gradually increases, prompting the Gaussian compression model to learn the local details of the target 3D scene. This progressive resolution training method reduces geometric errors in 3D scene reconstruction caused by overfitting to high-frequency signals in multi-view scene images during training, balancing training efficiency and reconstruction accuracy, and achieving simultaneous optimization of global structure and local details.

[0063] To achieve better training results, some implementations employ a multi-loss joint constraint strategy in electronic devices. This strategy involves co-optimizing 3D reconstruction loss, bitrate constraint loss, and masking loss to complete end-to-end iterative training of the general feature pool, Gaussian compression model, and anchor point attribute parameters.

[0064] During training, the electronic device determines the 3D reconstruction loss based on the rendered image and the supervised image of the target 3D scene. For example, the electronic device compares the rendered image generated by the Gaussian compression model with the supervised image, calculates the L2 loss between the rendered and supervised images, and determines the L2 loss as the 3D reconstruction loss. The 3D reconstruction loss is used to optimize the 3D scene reconstruction quality by addressing the discrepancy between the rendered and supervised images. Under the influence of the 3D reconstruction loss, the difference between the rendered and supervised images is continuously reduced, improving the quality and effect of the 3D scene reconstruction.

[0065] Simultaneously, the electronic device determines the bitrate constraint loss based on the learnable attribute parameters of each anchor point and the general feature pool. For example, the electronic device calculates the information entropy of the attribute parameters (such as characteristic features, scale, and offset) of each anchor point, and also calculates the information entropy of the general feature pool (such as multiple three-dimensional hash maps and multiple two-dimensional hash maps). The bitrate constraint loss is then determined based on the information entropy of the attribute parameters of each anchor point and the general feature pool. This bitrate constraint loss is used to optimize the learnable attribute parameters of the anchor points and the general feature pool. Under the influence of the bitrate loss, the attribute parameters of the anchor points and the hash maps of the general feature pool continuously tend towards a more concentrated and smooth probability distribution, thereby reducing data storage redundancy and improving data compression efficiency.

[0066] Furthermore, the electronic device calculates masking loss based on the masking weights corresponding to each 3D Gaussian primitive. Masking weights represent the renderability of the corresponding 3D Gaussian primitive. A larger masking weight indicates a higher renderability, while a smaller masking weight indicates a lower renderability. Masking loss is used to optimize the number of 3D Gaussian primitives. Under the influence of masking loss, 3D Gaussian primitives with masking losses approaching zero are eliminated, thereby reducing the number of 3D Gaussian primitives in 3D scene reconstruction and decreasing the rendering overhead and data storage requirements.

[0067] After determining the 3D reconstruction loss, bitrate constraint loss, and masking loss, the electronic device jointly trains the general feature pool, Gaussian compression model, and attribute parameters of each anchor point based on these losses. This yields the trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point. For example, the electronic device weights and fuses the 3D reconstruction loss, bitrate constraint loss, and masking loss to obtain a joint loss. Backpropagation is then performed based on this joint loss to update the hash map in the general feature pool, the network parameters of the Gaussian compression model, and the learnable attribute parameters of each anchor point. Through multiple iterative optimizations, the trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point are obtained.

[0068] For example, the joint loss can be calculated using formula (1): Formula (1) in, Indicates joint loss; The loss in 3D reconstruction can be determined by comparing the rendered image with the supervised image. Indicates the bitrate constraint loss; The bitrate loss corresponding to the learnable attribute parameters (such as characteristic features, scale, offset, etc.) of each anchor point can be determined by the information entropy of the learnable attribute parameters. The code rate loss corresponding to the hash map of the general feature pool can be determined by the information entropy of the hash map; Indicates masking loss; This represents the weight corresponding to the bitrate constraint loss; This represents the weight corresponding to the masking loss.

[0069] After obtaining the trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point, in step 105, the electronic device compresses the trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point.

[0070] Because different data have different characteristics, electronic devices can use different compression methods to compress the trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point.

[0071] In some implementations, an adaptive entropy encoding method is used for compression, targeting the learnable attribute parameters such as the characteristic features, scale, and offset of each anchor point. Specifically, the electronic device determines the probability distribution of the attribute parameters of each anchor point, and based on this probability distribution, performs entropy encoding on the attribute parameters of each anchor point to obtain the encoded attribute features of each anchor point.

[0072] For example, for any attribute parameter among the characteristic features, scale, and offset of each anchor point, the electronic device inputs the position of each anchor point into the entropy estimation model. The entropy estimation model estimates the probability distribution corresponding to the attribute parameter of each anchor point, obtaining the mean, variance, and adaptive quantization step size of that attribute parameter output by the entropy estimation model. Then, the electronic device uses the adaptive quantization step size output by the entropy estimation model to quantize that attribute parameter of each anchor point, mapping the continuous floating-point attribute parameter to integer data. After quantization, the electronic device performs entropy encoding on the quantized attribute parameter based on the mean and variance output by the entropy estimation model, obtaining the encoded bitstream of that attribute parameter.

[0073] Understandably, entropy estimation models are used to estimate the probability distribution of attribute parameters. In practical applications, electronic devices can use pre-trained entropy estimation models to estimate the probabilities of attribute parameters. Alternatively, electronic devices can train the entropy estimation model together with a general feature pool, a Gaussian compression model, and the attribute parameters of each anchor point, achieving end-to-end training and optimization.

[0074] Since the position parameters of each anchor point are fixed, non-learnable parameters, and are characterized by uniform distribution and low redundancy, a different encoding method can be used to compress these position parameters compared to learnable attribute parameters such as characteristic features, scale, and offset. For example, electronic devices can use differential encoding to compress the position parameters of each anchor point, thereby achieving unbiased compression of the position parameters.

[0075] For hash maps of general feature pools (such as three-dimensional hash maps and two-dimensional hash maps), electronic devices can use Huffman coding to compress the hash maps of general feature pools, thereby reducing the redundancy of hash maps and achieving lightweight storage of general feature pools.

[0076] For model data such as Gaussian compression models and entropy estimation models, electronic devices can use run-length encoding to compress the model data, thereby achieving effective compression of the model data.

[0077] In this embodiment, a differentiated encoding strategy is employed to compress different types of data based on their characteristics, thereby improving the overall compression effect. By compressing the trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point, the storage space occupied by the target 3D scene reconstruction-related data can be reduced, achieving effective compression of 3D scene data.

[0078] The data compression method provided in this application embodiment is further illustrated by an application example. For instance, each anchor point in the target 3D scene includes position, scale, offset, and characteristic features. The dimension of the characteristic features of each anchor point can be 25. The general feature pool uses a hash mapping table corresponding to multiple resolutions, including 13 3D hash mapping tables and 4 2D hash mapping tables. Each hash mapping table can be numbered from low to high resolution according to the corresponding mesh structure, realizing multi-resolution storage of general features. The dimension of the general feature of each item in the hash mapping table can be 4.

[0079] like Figure 2 As shown, the prediction network of the Gaussian compression model can employ multi-layer perceptrons (MLPs). The compensation network of the Gaussian compression model can employ a multi-head decoder (MGRD). The backbone of the MGRD is implemented using a two-layer MLP and a ReLU activation function. The output of the MGRD includes a first output head and a second output head. The first output head is used to output geometric deformation; the second output head is used to output appearance compensation. The first and second output heads can each be implemented using linear mapping layers.

[0080] During training, the electronic device queries the general feature pool using the spatial location of each anchor point to obtain the general features of each anchor point. Then, the electronic device inputs the general features of each anchor point into the prediction network of the Gaussian compression model, and uses the prediction network to output the first parameters of the coarse-grained 3D Gaussian primitives. The first parameters include position, scale, rotation, opacity, and color. When the number of training iterations (i.e., training epochs) is less than a preset number, the electronic device inputs the first parameters into the rendering network of the Gaussian compression model to obtain the rendered image corresponding to the first granularity of the target 3D scene. Training is then performed based on the rendered image corresponding to the first granularity and the supervision image to reconstruct the basic ensemble structure of the target 3D scene.

[0081] When the number of training iterations is greater than or equal to a preset number, the electronic device inputs the general features of each anchor point into the prediction network of the Gaussian compression model to obtain the first parameters of the coarse-grained 3D Gaussian primitives. Simultaneously, it inputs the characteristic features of each anchor point into the compensation network of the Gaussian compression model to obtain the second parameters of the fine-grained 3D Gaussian primitives. The second parameters include geometric deformation and appearance compensation. Geometric deformation includes scale deformation (denoted as Δs) and rotation deformation (denoted as Δr), while appearance compensation includes transparency compensation (denoted as Δα) and color compensation (denoted as Δc). The electronic device fuses (e.g., stitches) the first and second parameters of the 3D Gaussian primitives to obtain the rendering parameters of the 3D Gaussian primitives. Then, the electronic device inputs the rendering parameters of the 3D Gaussian primitives into the rendering network of the Gaussian compression model to obtain the rendered image corresponding to the second granularity of the target 3D scene. Training is then performed based on the rendered image corresponding to the second granularity and the supervised image to reconstruct the local details of the target 3D scene.

[0082] The supervised images used in each training round are obtained by adaptively sampling the scene images of the target 3D scene using a cosine function scheduler. The electronic device inputs the training round number or training duration into the cosine function scheduler to obtain the downsampling scale factor output by the cosine function scheduler. The electronic device uses the downsampling scale factor (i.e., The scene image of the target 3D scene is downsampled (DS) to obtain the supervision image for the corresponding round. Through multiple rounds of training (e.g., 30,000 times), the trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point are obtained.

[0083] After training, the electronic device encodes the 3D reconstruction data, including the general feature pool, Gaussian compression model, and attribute parameters of each anchor point, to obtain compressed 3D reconstruction data, achieving lightweight storage of the 3D reconstruction data. In the decoding stage, the electronic device first decodes the position parameters of each anchor point, the model data, and the general feature pool. Then, it inputs the position parameters of each anchor point into the entropy estimation model to determine the probability distribution of the scale, offset, and characteristic features of each anchor point. Based on the probability distribution of the scale, offset, and characteristic features, it performs entropy decoding on the compressed scale, offset, and characteristic features to obtain the attribute parameters such as the scale, offset, and characteristic features of each anchor point. Then, the electronic device queries the general feature pool to obtain general features based on the position of each anchor point, and inputs the general features and characteristic features of each anchor point into the Gaussian compression model to obtain a new perspective image of the target 3D scene.

[0084] In this embodiment, addressing the issue of anchor point features occupying a large proportion in 3D Gaussian compression, a predictive Gaussian compression framework is proposed, combined with a progressive training strategy to improve the performance of 3D Gaussian compression. First, a universal feature pool is constructed using multi-resolution 3D and 2D mesh structures in the target 3D space, combined with hash mapping technology. Each anchor point shares this universal feature pool. By querying the universal feature pool, the universal features of each anchor point can be obtained, thereby enabling coarse-grained prediction of the rendering parameters of 3D Gaussian primitives. Simultaneously, the geometric deformation and appearance residuals of 3D Gaussian primitives are predicted using the characteristic features of each anchor point (i.e., appearance compensation). These geometric deformations and appearance residuals are then used to correct the coarse-grained rendering parameters, resulting in fine-grained rendering parameters. By decoupling the anchor point features, which originally needed to be explicitly stored in the bitstream of the 3D reconstructed data, into a shared universal feature pool and specific features (i.e., characteristic features), the storage of similar features between anchor points is effectively eliminated, reducing the size of the compressed bitstream.

[0085] In this embodiment, to adapt to the predictive feature decoupling characteristics and achieve a coarse-to-fine training process, a progressive training strategy is adopted. Specifically, the training process is divided into two stages. In the first stage, the compensation branch (i.e., the compensation network) is closed, and only general features are used to coarsely fit the geometric structure of the 3D Gaussian primitives. In the second stage, the compensation branch is enabled to optimize local details, achieving phased optimization of the geometric structure and local details of the 3D Gaussian primitives. Simultaneously, the resolution of the supervised images used during training is continuously improved, thereby suppressing overfitting of general features to high-frequency signals from multi-view images, improving training effectiveness, and raising the upper limit of scene reconstruction quality.

[0086] To implement the data compression method provided in the embodiments of this application, the embodiments of this application also provide a data compression apparatus, such as... Figure 3 As shown, the device includes: The acquisition module 31 is used to acquire the set of anchor points of the target 3D scene; The query module 32 is used to query the general feature pool of the target 3D scene based on the position of each anchor point in the anchor point set, and obtain the general features of each anchor point; wherein, the general feature pool is used to store the general features of the target 3D scene; The reconstruction module 33 is used to input the common features of each anchor point into the Gaussian compression model to obtain the rendered image of the target three-dimensional scene; Training module 34 is used to train the general feature pool, the Gaussian compression model and the attribute features of each anchor point based on the rendered image and the supervised image of the target 3D scene, so as to obtain the trained general feature pool, Gaussian compression model and attribute parameters of each anchor point. Compression model 35 is used to compress the trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point.

[0087] In some optional implementations, the attribute parameters of each anchor point include characteristic features; the Gaussian compression model includes a prediction network and a compensation network; the reconstruction module 33 is specifically used for: The common features of each anchor point are input into the prediction network to obtain the first parameters of each three-dimensional Gaussian element. The characteristic features of each anchor point are input into the compensation network to obtain the second parameters of each three-dimensional Gaussian element. The first parameter and the second parameter are fused together to obtain the rendering parameters of each three-dimensional Gaussian primitive; The rendered image of the target 3D scene is obtained based on the rendering parameters of each 3D Gaussian primitive.

[0088] In some optional implementations, the general feature pool includes multiple three-dimensional hash maps and multiple two-dimensional hash maps, with different hash maps corresponding to grid structures of different resolutions; the three-dimensional hash maps are used to store three-dimensional features, and the two-dimensional hash maps are used to store two-dimensional features; the query module 32 is specifically used for: Based on the position of each anchor point in the anchor point set, each three-dimensional hash mapping table and each two-dimensional hash mapping table are queried to obtain the general characteristics of each anchor point.

[0089] In some optional implementations, the query module 32 is specifically used for: Based on the position of each anchor point in the anchor point set, query each three-dimensional hash mapping table to obtain the three-dimensional features found in each three-dimensional hash mapping table; Based on the position of each anchor point in the anchor point set, query each two-dimensional hash mapping table to obtain the two-dimensional features found in each two-dimensional hash mapping table; The three-dimensional features retrieved from each three-dimensional hash map table and the two-dimensional features retrieved from each two-dimensional hash map table are concatenated to obtain the common features of each anchor point.

[0090] In some alternative implementations, the training module 34 is further configured to: In each training round, the downsampling scale parameter corresponding to each training round is determined based on the iteration round number; wherein, the iteration round number is negatively correlated with the downsampling scale parameter. The scene image of the target 3D scene is downsampled according to the downsampling scale parameter to obtain the supervision image for each training round; wherein, as the number of iterations increases, the image resolution of the supervision image continuously improves.

[0091] In some alternative implementations, the training module 34 is specifically used for: Based on the rendered image of the target 3D scene and the supervised image, the 3D reconstruction loss is determined; Based on the attribute parameters of each anchor point and the general feature pool, the bitrate constraint loss is determined. The masking loss is determined based on the masking weights of each 3D Gaussian primitive; wherein the masking weights represent the renderability of the corresponding 3D Gaussian primitive. Based on the 3D reconstruction loss, the bitrate constraint loss, and the masking loss, the general feature pool, the Gaussian compression model, and the attribute parameters of each anchor point are trained to obtain the trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point.

[0092] In some alternative implementations, the compression model 35 is specifically used for: Determine the probability distribution of the attribute parameters of each anchor point; The attribute parameters of each anchor point are entropy encoded based on the probability distribution.

[0093] In practical applications, the acquisition module 31 can be implemented by the processor in the data compression device in conjunction with the communication interface, and the query module 32, reconstruction module 33, training module 34 and compression model 35 can be implemented by the processor in the data compression device.

[0094] It should be noted that the data compression device provided in the above embodiments is only illustrated by the division of the above-described program units. In practical applications, the above processing can be assigned to different program units as needed, that is, the internal structure of the device can be divided into different program units to complete all or part of the processing described above. In addition, the data compression device and the data compression method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0095] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, the embodiments of this application also provide an electronic device, such as... Figure 4 As shown, the electronic device includes: The communication interface 401 enables information exchange with other electronic devices (such as terminal devices); The processor 402 is connected to the communication interface 401 to enable information interaction with other electronic devices and to execute the methods provided by one or more of the above-mentioned technical solutions when running a computer program; The computer program is stored in memory 403.

[0096] Specifically, the processor 402 is used for: Obtain the set of anchor points of the target 3D scene through communication interface 401; Based on the position of each anchor point in the anchor point set, the general feature pool of the target 3D scene is queried to obtain the general features of each anchor point; wherein, the general feature pool is used to store the general features of the target 3D scene; The common features of each anchor point are input into the Gaussian compression model to obtain the rendered image of the target 3D scene; Based on the rendered image and the supervised image of the target 3D scene, the general feature pool, the Gaussian compression model and the attribute features of each anchor point are trained to obtain the trained general feature pool, Gaussian compression model and attribute parameters of each anchor point. The trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point are compressed.

[0097] In some optional implementations, the attribute parameters of each anchor point include characteristic features; the Gaussian compression model includes a prediction network and a compensation network; the processor 402 is specifically used for: The common features of each anchor point are input into the prediction network to obtain the first parameters of each three-dimensional Gaussian element. The characteristic features of each anchor point are input into the compensation network to obtain the second parameters of each three-dimensional Gaussian element. The first parameter and the second parameter are fused together to obtain the rendering parameters of each three-dimensional Gaussian primitive; The rendered image of the target 3D scene is obtained based on the rendering parameters of each 3D Gaussian primitive.

[0098] In some optional implementations, the general feature pool includes multiple three-dimensional hash maps and multiple two-dimensional hash maps, with different hash maps corresponding to grid structures of different resolutions; the three-dimensional hash maps are used to store three-dimensional features, and the two-dimensional hash maps are used to store two-dimensional features; the processor 402 is specifically used for: Based on the position of each anchor point in the anchor point set, each three-dimensional hash mapping table and each two-dimensional hash mapping table are queried to obtain the general characteristics of each anchor point.

[0099] In some alternative implementations, the processor 402 is specifically used for: Based on the position of each anchor point in the anchor point set, query each three-dimensional hash mapping table to obtain the three-dimensional features found in each three-dimensional hash mapping table; Based on the position of each anchor point in the anchor point set, query each two-dimensional hash mapping table to obtain the two-dimensional features found in each two-dimensional hash mapping table; The three-dimensional features retrieved from each three-dimensional hash map table and the two-dimensional features retrieved from each two-dimensional hash map table are concatenated to obtain the common features of each anchor point.

[0100] In some alternative implementations, the processor 402 is further configured to: In each training round, the downsampling scale parameter corresponding to each training round is determined based on the iteration round number; wherein, the iteration round number is negatively correlated with the downsampling scale parameter. The scene image of the target 3D scene is downsampled according to the downsampling scale parameter to obtain the supervision image for each training round; wherein, as the number of iterations increases, the image resolution of the supervision image continuously improves.

[0101] In some alternative implementations, the processor 402 is specifically used for: Based on the rendered image of the target 3D scene and the supervised image, the 3D reconstruction loss is determined; Based on the attribute parameters of each anchor point and the general feature pool, the bitrate constraint loss is determined. The masking loss is determined based on the masking weights of each 3D Gaussian primitive; wherein the masking weights represent the renderability of the corresponding 3D Gaussian primitive. Based on the 3D reconstruction loss, the bitrate constraint loss, and the masking loss, the general feature pool, the Gaussian compression model, and the attribute parameters of each anchor point are trained to obtain the trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point.

[0102] In some alternative implementations, the processor 402 is specifically used for: Determine the probability distribution of the attribute parameters of each anchor point; The attribute parameters of each anchor point are entropy encoded based on the probability distribution.

[0103] It should be noted that the specific processing procedures of the processor 402 and the communication interface 401 can be understood with reference to the data compression method described above.

[0104] Of course, in practical applications, the various components in an electronic device are coupled together through the bus system memory 404. It can be understood that the bus system memory 404 is used to realize the connection and communication between these components. In addition to the data bus, the bus system memory 404 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 4 The various buses are all labeled as Bus System Memory 404.

[0105] The memory 403 in this embodiment is used to store various types of data to support the operation of the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0106] The methods disclosed in the embodiments of this application can be applied to the processor 402, or implemented by the processor 402. The processor 402 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 402 or by instructions in the form of software. The processor 402 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 402 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the memory 403. The processor 402 reads the information in the memory 403 and completes the steps of the aforementioned method in combination with its hardware.

[0107] In an exemplary embodiment, the electronic device may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0108] It is understood that the memory 403 in this embodiment can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0109] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 403 storing a computer program, which can be executed by a processor 402 of an electronic device to complete the steps described in the aforementioned saturation adjustment method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0110] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by a processor 402 of an electronic device to perform the steps described in the aforementioned saturation adjustment method.

[0111] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0112] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0113] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A data compression method, characterized in that, The method includes: Obtain the set of anchor points for the target 3D scene; Based on the position of each anchor point in the anchor point set, the general feature pool of the target 3D scene is queried to obtain the general features of each anchor point; wherein, the general feature pool is used to store the general features of the target 3D scene; The common features of each anchor point are input into the Gaussian compression model to obtain the rendered image of the target 3D scene; Based on the rendered image and the supervised image of the target 3D scene, the general feature pool, the Gaussian compression model and the attribute features of each anchor point are trained to obtain the trained general feature pool, Gaussian compression model and attribute parameters of each anchor point. The trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point are compressed.

2. The method according to claim 1, characterized in that, The attribute parameters of each anchor point include characteristic features; the Gaussian compression model includes a prediction network and a compensation network; the step of inputting the general features of each anchor point into the Gaussian compression model to obtain the rendered image of the target 3D scene includes: The common features of each anchor point are input into the prediction network to obtain the first parameters of each three-dimensional Gaussian element. The characteristic features of each anchor point are input into the compensation network to obtain the second parameters of each three-dimensional Gaussian element. The first parameter and the second parameter are fused together to obtain the rendering parameters of each three-dimensional Gaussian primitive; The rendered image of the target 3D scene is obtained based on the rendering parameters of each 3D Gaussian primitive.

3. The method according to claim 1, characterized in that, The general feature pool includes multiple three-dimensional hash maps and multiple two-dimensional hash maps, with different hash maps corresponding to grid structures of different resolutions; the three-dimensional hash maps are used to store three-dimensional features, and the two-dimensional hash maps are used to store two-dimensional features. The step of querying the general feature pool of the target 3D scene based on the position of each anchor point in the anchor point set to obtain the general features of each anchor point includes: Based on the position of each anchor point in the anchor point set, each three-dimensional hash mapping table and each two-dimensional hash mapping table are queried to obtain the general characteristics of each anchor point.

4. The method according to claim 3, characterized in that, Based on the position of each anchor point in the anchor point set, each three-dimensional hash mapping table and each two-dimensional hash mapping table are queried to obtain the common characteristics of each anchor point, including: Based on the position of each anchor point in the anchor point set, query each three-dimensional hash mapping table to obtain the three-dimensional features found in each three-dimensional hash mapping table; Based on the position of each anchor point in the anchor point set, query each two-dimensional hash mapping table to obtain the two-dimensional features found in each two-dimensional hash mapping table; The three-dimensional features retrieved from each three-dimensional hash map table and the two-dimensional features retrieved from each two-dimensional hash map table are concatenated to obtain the common features of each anchor point.

5. The method according to claim 1, characterized in that, The method further includes: In each training round, the downsampling scale parameter corresponding to each training round is determined based on the iteration round number; wherein, the iteration round number is negatively correlated with the downsampling scale parameter. The scene image of the target 3D scene is downsampled according to the downsampling scale parameter to obtain the supervision image for each training round; wherein, as the number of iterations increases, the image resolution of the supervision image continuously improves.

6. The method according to claim 1, characterized in that, The process involves training the general feature pool, the Gaussian compression model, and the attribute features of each anchor point based on the rendered image and the supervised image of the target 3D scene, to obtain the trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point, including: Based on the rendered image of the target 3D scene and the supervised image, the 3D reconstruction loss is determined; Based on the attribute parameters of each anchor point and the general feature pool, the bitrate constraint loss is determined. The masking loss is determined based on the masking weights of each 3D Gaussian primitive; wherein the masking weights represent the renderability of the corresponding 3D Gaussian primitive. Based on the 3D reconstruction loss, the bitrate constraint loss, and the masking loss, the general feature pool, the Gaussian compression model, and the attribute parameters of each anchor point are trained to obtain the trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point.

7. The method according to claim 1, characterized in that, The compression of the trained general feature pool, Gaussian compression model, and attribute parameters of each anchor point includes: Determine the probability distribution of the attribute parameters of each anchor point; The attribute parameters of each anchor point are entropy encoded based on the probability distribution.

8. An electronic device, characterized in that, include: Processor and memory used to store computer programs that can run on the processor; When the processor is used to run a computer program, it executes the steps of the method according to any one of claims 1 to 7.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images