A 4D Gaussian sputtering compression method based on UV mapping

Abstract

The application discloses a 4D Gaussian sputtering compression method based on UV mapping, relates to the technical field of video basic stream processing, and mainly comprises the following steps: converting the three-dimensional coordinates of the center point of each 3D Gaussian sputtering basic unit in a 4D Gaussian sputtering sequence into spherical coordinates through coordinate conversion, and mapping and constructing the azimuth and polar angle in the spherical coordinates into a two-dimensional UV grid representation; through multi-round iterative sampling, selecting the 3D Gaussian sputtering basic unit with the maximum radial distance in each round grid unit as high-priority data to construct a multi-layer UV mapping; collecting all the 3D Gaussian sputtering basic units that are not selected after iterative sampling and reordering them as detail supplement data; and splitting the attribute data of the 3D Gaussian sputtering basic unit into corresponding bit components based on the channel number of a video codec and encapsulating the attribute data into YUV format data to input the video codec for compression. The application can adapt to the input requirements of a standard video codec without training.

Description

Technical Field

[0001] This invention relates to the field of video basic stream processing technology, and specifically to a 4D Gaussian sputtering compression method based on UV mapping. Background Technology

[0002] While 4D Gaussian Splatting technology has demonstrated significant advantages in virtual reality, telepresence, and other fields due to its high-fidelity modeling and real-time rendering capabilities for dynamic scenes, its large-scale application remains limited by high storage and transmission costs. Existing solutions have significant shortcomings in addressing this challenge: First, compression methods for static 3D Gaussian sputtering cannot effectively handle the inherent temporal redundancy and inter-frame correlation in dynamic scenes, lacking the ability to model temporal consistency. Second, a few compression schemes extended to 4D scenes, due to the unstructured nature of Gaussian primitives, struggle to directly calculate inter-frame motion and residuals, often requiring indirect estimation of motion information from rendered images. This results in strong coupling between compression performance and specific rendering pipelines, poor generalization ability, and reliance on complex neural network training, significantly increasing computational overhead and reducing compatibility with general-purpose hardware platforms. Furthermore, existing standard video codecs (such as AVC and HEVC) are designed for structured 2D image data and cannot directly process unstructured Gaussian sequences, lacking effective adaptation mechanisms, leading to complex and inefficient compression processes. These issues collectively make it difficult for existing methods to balance compression efficiency, reconstruction quality, and hardware compatibility, severely restricting the large-scale application of 4D Gaussian sputtering technology in bandwidth-constrained scenarios. Summary of the Invention

[0003] To balance compression efficiency, reconstruction quality, and hardware compatibility, and to better promote the large-scale application of 4D Gaussian sputtering technology, this invention proposes a 4D Gaussian sputtering compression method based on UV mapping, comprising the following steps:

[0004] S1: The three-dimensional coordinates of the center point of each 3D Gaussian sputtering basic unit in the 4D Gaussian sputtering sequence are converted into spherical coordinates through coordinate transformation, and the azimuth and polar angles in the spherical coordinates are mapped and constructed into a two-dimensional UV representation;

[0005] S2: By performing multiple rounds of iterative sampling on the mesh corresponding to the two-dimensional UV representation, the 3D Gaussian sputtering basic unit with the largest radial distance within the mesh cell in each round is selected as high-priority data to construct a multi-layer UV mapping;

[0006] S3: Collect all unselected 3D Gaussian sputtering basic units after iterative sampling, and reorder the data of each fixed U-axis coordinate point along the V-axis direction according to the polar angle order in the corresponding spherical coordinates, as supplementary detailed data;

[0007] S4: Based on the number of video codec channels, the attribute data of multi-layer UV mapping and reordered 3D Gaussian sputtering basic units are split into corresponding bit components and encapsulated into YUV format data.

[0008] S5: Input YUV format data into the video codec and compress the YUV format data under the bitrate priority allocation with the goal of prioritizing the reconstruction quality of high-priority data.

[0009] This invention transforms unstructured 4D Gaussian sputtering sequences into a structured two-dimensional representation with spatial locality and temporal consistency by introducing a time-consistent spherical projection and UV mapping mechanism, laying a solid foundation for subsequent compression operations. An iterative sampling strategy based on radial distance intelligently identifies and prioritizes Gaussian primitives representing key scene structures, forming a high-priority data layer. The ordered organization of residual primitives effectively ensures the spatial coherence of detailed information. Together, these two mechanisms enable differentiated processing of visually important information at limited bitrates. Based on video codec channel number splitting and data encapsulation, it can adapt to the input requirements of standard video codecs, achieving efficient compression directly using mature codecs such as AVC and HEVC without any neural network training, while ensuring full compatibility of the encoding and decoding process with mainstream hardware decoders.

[0010] Furthermore, in step S1, the coordinate transformation specifically involves:

[0011] Obtain the three-dimensional rectangular coordinates of the center point of each 3D Gaussian sputtering basic unit, and convert the three-dimensional rectangular coordinates into spherical coordinates that include radial distance, azimuth angle and polar angle;

[0012] The mapping is specifically as follows:

[0013] Based on the preset UV mapping resolution parameters M and N, the azimuth and polar angles are divided respectively. The azimuth and polar angles are mapped onto the two-dimensional UV coordinate grid through rounding operations to generate a two-dimensional UV representation with time consistency.

[0014] Furthermore, in step S3, the specific process of reordering includes:

[0015] The UV coordinate distribution of all unselected 3D Gaussian sputtering basic units was analyzed, and based on statistical regularity, it was determined that the uniformity of the distribution along the U-axis in the two-dimensional UV was higher than that along the V-axis.

[0016] Using the U-axis as the reference dimension, all 3D Gaussian sputtering basic units under each fixed U-coordinate value are arranged in ascending or descending order along the V-axis according to the size of their polar angle in spherical coordinates.

[0017] Furthermore, the video codec is an AVC or HEVC standard encoder.

[0018] Furthermore, in step S4, the splitting specifically involves splitting each 16-bit Gaussian attribute data into two components: a high 8-bit component and a low 8-bit component.

[0019] Furthermore, in step S4, the YUV format used for encapsulation is YUV4:4:4 pixel format.

[0020] Furthermore, in step S5, the bitrate priority allocation is achieved by configuring the encoder's quantization parameters or constant bitrate factor, allocating lower quantization parameters or better bitrate control strategies to high-priority data.

[0021] Furthermore, step S2 also includes the step of updating the effective 3D Gaussian sputtering basic unit set of the current mesh after each round of sampling, and the sampling in subsequent rounds is based on the dataset updated in the previous round.

[0022] Compared with the prior art, the present invention has at least the following beneficial effects:

[0023] (1) The present invention proposes a 4D Gaussian sputtering compression method based on UV mapping. By introducing a time-consistent spherical projection and UV mapping mechanism, the unstructured 4D Gaussian sputtering sequence is converted into a structured two-dimensional representation with spatial locality and time consistency, laying a solid foundation for subsequent compression operations.

[0024] (2) The iterative sampling strategy based on radial distance can intelligently identify and prioritize the retention of Gaussian primitives that represent key structures of the scene, forming a high-priority data layer. The ordered organization of residual primitives effectively ensures the spatial coherence of detailed information. The two work together to achieve differentiated processing of visually important information under limited bit rate.

[0025] (3) By splitting the 16-bit attribute data into 8-bit components and encapsulating them in YUV4:4:4 format, it can adapt to the input requirements of standard video codecs. It can directly use mature codecs such as AVC and HEVC to achieve efficient compression without any neural network training, while ensuring that the encoding and decoding process is fully compatible with mainstream hardware decoders. Attached Figure Description

[0026] Figure 1 This is a step diagram of a 4D Gaussian sputtering compression method based on UV mapping;

[0027] Figure 2 This is a schematic diagram of a 4D Gaussian sputtering compression method based on UV mapping;

[0028] Figure 3This is a schematic diagram of UV projection. Detailed Implementation

[0029] The following are specific embodiments of the present invention, which are described in conjunction with the accompanying drawings. However, the present invention is not limited to these embodiments.

[0030] In the field of 4D dynamic scene representation and compression, the core dilemma of existing technologies lies in their inability to effectively handle the inherent unstructured characteristics of Gaussian splatting data. Traditional compression methods for static 3D Gaussian splatting cannot model temporal redundancy in dynamic scenes, while a few solutions extended to 4D have to rely on rendering images to indirectly estimate inter-frame motion information due to the lack of regular spatial structure in Gaussian primitives. This indirect processing method results in compression performance being tightly coupled with specific rendering pipelines, leading to poor versatility, and often requiring computationally intensive neural network training, which increases implementation complexity and reduces compatibility with standard hardware decoding platforms. More importantly, existing international standard video codecs (such as AVC and HEVC) are designed for structured 2D image data and cannot directly process unstructured Gaussian sequence data, making it difficult to balance compression efficiency, reconstruction quality, and hardware adaptability in 4D Gaussian splatting technology, severely restricting its large-scale application in bandwidth-constrained scenarios. These technical bottlenecks are precisely the key problems that this solution aims to solve. Therefore, as Figure 1 As shown, this invention proposes a 4D Gaussian sputtering compression method based on UV mapping, including the following steps:

[0031] S1: The three-dimensional coordinates of the center point of each 3D Gaussian sputtering basic unit in the 4D Gaussian sputtering sequence are converted into spherical coordinates through coordinate transformation, and the azimuth and polar angles in the spherical coordinates are mapped and constructed into a two-dimensional UV representation;

[0032] S2: By performing multiple rounds of iterative sampling on the mesh corresponding to the two-dimensional UV representation, the 3D Gaussian sputtering basic unit with the largest radial distance within the mesh cell in each round is selected as high-priority data to construct a multi-layer UV mapping;

[0033] S3: Collect all unselected 3D Gaussian sputtering basic units after iterative sampling, and reorder the data of each fixed U-axis coordinate point along the V-axis direction according to the polar angle order in the corresponding spherical coordinates, as supplementary detailed data;

[0034] S4: Based on the number of video codec channels, the attribute data of multi-layer UV mapping and reordered 3D Gaussian sputtering basic units are split into corresponding bit components and encapsulated into YUV format data.

[0035] S5: Input YUV format data into the video codec and compress the YUV format data under the bitrate priority allocation with the goal of prioritizing the reconstruction quality of high-priority data.

[0036] Specifically, such as Figure 2 and Figure 3 As shown, before compression, the 4D Gaussian sputtering sequence data to be compressed needs to be prepared. This sequence can be output by any 4D Gaussian sputtering reconstruction model (such as a model trained on the N3DV dataset) or generative model (such as L4GM), and is essentially a time-varying set of 3D Gaussian sputtering basic units. Each 3D Gaussian sputtering basic unit (hereinafter referred to as "Gaussian unit") contains a series of attribute parameters, mainly including the three-dimensional position coordinates of its center point. The parameters include rotation parameters (usually represented by quaternions), 3D scale parameters, transparency (alpha), and spherical harmonics (SH) coefficients representing color. To adapt to the compression process proposed in this invention, key parameters need to be pre-set: the resolution parameters M and N of the UV mapping (i.e., the number of grids in the U and V directions), which are typically set to M=512 and N=512 depending on the target compression accuracy and scene complexity; the number of iterations K for sampling, serving as a control parameter for redundancy removal intensity, preferably K=6 to achieve a balance between efficiency and effect; and the video codec configuration, which can preset two modes to adapt to different needs: a lossless compression mode (e.g., using an HEVC encoder, configured with a constant quantization parameter CRF=0) and a visually lossless high compression mode (e.g., CRF=30).

[0037] To transform unstructured, scattered 3D Gaussian primitives into a structured 2D representation with spatial locality and temporal consistency, facilitating subsequent encoding and decoding using standard video codecs, this invention first traverses each frame of the 4D Gaussian sputtering sequence and obtains the 3D Cartesian coordinates of the center point of each Gaussian primitive within that frame. Next, using the standard spherical coordinate transformation formula, the rectangular coordinates of each point are converted to spherical coordinates. , where radial distance azimuth polar angle Each primitive has a spherical coordinate representation based on the current frame's camera coordinate system (or world coordinate system).

[0038] Next, the crucial UV mapping is performed. Based on preset resolution parameters M and N (e.g., 512×512), the azimuth angle is determined. and polar angle The value range is divided evenly. Specifically, the range of values ​​is divided into... from Mapping to U coordinates ,Will From mapping to V coordinates Rounding down consecutive integers... Discretize and map to a unique two-dimensional grid coordinate system. Above. This mapping strategy ensures that in dynamic scenes, the Gaussian elements of the same object surface are mapped to the same or adjacent cells in the UV mesh at the same time in different frames, thus establishing crucial temporal consistency and solving the inherent problem of difficulty in inter-frame motion estimation for unstructured data.

[0039] After UV mapping, all Gaussian primitives are assigned to specific cells in the 2D UV mesh. However, directly compressing all Gaussian primitives within each cell results in significant redundancy. To address this, this invention proposes an iterative sampling mechanism with redundancy optimization. This process operates on the generated 2D UV mesh, performing K rounds (e.g., 6 rounds) of sampling. At the start of each round, the algorithm traverses every cell in the 2D UV mesh. For cells containing multiple Gaussian primitives, the radial distance of each primitive within the cell is calculated. The Gaussian primitive with the largest radial distance in that cell is selected and retained, while other Gaussian primitives within the same cell are discarded. The underlying principle of this selection method is that, along a ray direction originating from a common viewpoint, the primitive with the largest radial distance is most likely located on the object surface and is a key viewpoint constituting the visible structure of the scene, thus requiring higher priority.

[0040] After each round of sampling, the process doesn't end; instead, it immediately updates the set of valid Gaussian elements in the current 2D UV mesh—that is, removing the Gaussian elements discarded in this round. Subsequent rounds of sampling are based on the partially optimized dataset updated in the previous round. This iterative update mechanism allows the sampling process to progress layer by layer, selecting the most representative Gaussian elements from different perspectives (based on the updated data distribution) in each round. After K rounds of iteration, the representative Gaussian elements retained in each round are integrated to form K layers (e.g., 6 layers) of UV mapping. These K layers of mapping together constitute high-priority data (2D UV maps) representing the core framework of the scene.

[0041] Iterative sampling aims to retain the most critical information, but the discarded Gaussian primitives are not useless; they also contain rich details, such as the internal structure of the object and subtle surface undulations. Directly discarding them would lead to a decrease in reconstruction quality. Therefore, they need to be effectively organized by collecting all primitives that have not been selected in all K rounds of iterative sampling to form a residual set.

[0042] However, the distribution of Gaussian elements in the residual set in the two-dimensional UV space is sparse and disordered. Direct processing results in weak spatial correlation, which is detrimental to compression. Therefore, this invention proposes an ordered organization scheme based on the distribution characteristics of the two-dimensional UV space. By analyzing the two-dimensional UV coordinate distribution of the residual Gaussian elements, a key statistical law is observed: due to the azimuth angle... Due to their continuity, the distribution of Gaussian elements along the U-axis is generally more uniform than that along the V-axis. Therefore, the U-axis is chosen as the baseline dimension for sorting. The specific reordering process is as follows:

[0043] For each fixed U-coordinate value Collect all The residual Gaussian elements are then sorted according to their original, messy V-coordinates, and instead of their original polar angles in spherical coordinates. The sizes of the Gausky units are arranged in ascending (or descending) order along the V-axis. This operation is crucial, as it utilizes the original geometric information (polar angle) of the Gausky units in 3D space. This process forces the introduction of spatial order consistency into the residual data on the two-dimensional UV plane, reorganizing scattered residual points into ordered "lines" along the V-axis. This organized data can serve as residual map data, maintaining spatial coherence and creating favorable conditions for subsequent video encoders to utilize intra-frame prediction compression.

[0044] Having completed the aforementioned steps, we have obtained structured high-priority data and structured supplementary detail data. The next step is to adapt this data to a standard video codec for compression. Since standard codecs (such as AVC and HEVC) typically process 8-bit deep YUV format images, while Gaussian attribute data (such as position and color) are mostly 16-bit or 32-bit floating-point numbers, quantization and format conversion are necessary.

[0045] First, the attribute data is split and quantized. Taking 16-bit attribute data as an example, each 16-bit value is considered to consist of two 8-bit bytes: a most significant byte (MSB) and a least significant byte (LSB). The MSB and LSB of all these attributes (including position, rotation, scale, transparency, SH coefficients, etc.) are extracted and recombined. Specifically, an image plane can be constructed for the MSBs of the same attribute for all Gaussian pixels, and similarly, another image plane can be constructed for the LSBs. Next, when encapsulating the pixel format, the YUV4:4:4 format is selected. This format does not downsample the chroma signal, which can completely preserve the complete spatial resolution of the Y, U, and V channels, thereby avoiding the loss of Gaussian attribute information (especially geometric information) due to chroma subsampling, and maximizing the reconstruction accuracy after compression. At this point, the structured Gaussian data has been converted into a series of 8-bit deep, multi-channel YUV4:4:4 image sequences.

[0046] Finally, the encapsulated YUV image sequence is directly input into a standard video codec for compression. During this process, to achieve the goal of "prioritizing high-priority data," a bitrate priority allocation strategy needs to be implemented. One direct approach is to utilize the encoder's bitrate control function. For example, during encoding, the quantization parameter (QP) of image regions representing high-priority data (such as the first K frames) can be set to a lower value (meaning finer quantization and higher bitrate allocation), while the QP of image regions representing supplementary detail data can be set to a relatively higher value (coarser quantization and lower bitrate allocation). Another approach is to use a constant bitrate factor (CRF) mode, adjusting the quality of different parts by setting different CRF values. In this way, with a limited total bitrate, the encoder will automatically allocate more bits to high-priority data, thereby ensuring the reconstruction quality of critical scene structures.

[0047] The subsequent decoding is the reverse of the encoding process. The receiving end uses a standard video decoder (any decoding chip or software that supports HEVC or AVC) to decode the bitstream and restore the YUV 4:4:4 image sequence. Then, the reverse operation is performed: the data from the MSB and LSB channels are recombined to recover the 16-bit original Gaussian attribute data; according to the two-dimensional UV coordinate mapping relationship, the image data is re-parsed into a structured two-dimensional UV mapping and residual mapping; finally, the high-priority data and detail supplementary data are merged to reconstruct a complete 4D Gaussian sputtering sequence that can be used for real-time rendering.

[0048] Through the above specific implementation methods, it can be seen that the present invention, by introducing a time-consistent spherical projection and UV mapping mechanism, successfully transforms unstructured 4D Gaussian sputtering sequences into a structured two-dimensional representation with spatial locality and temporal consistency, laying a solid foundation for subsequent compression. The radial distance-based iterative sampling strategy intelligently identifies and prioritizes the retention of Gaussian primitives representing key structures of the scene, forming a high-priority data layer. The ordered organization of residual primitives effectively ensures the spatial coherence of detailed information. Together, these two approaches achieve differentiated processing of visually important information under limited bitrates.

[0049] By splitting 16-bit attribute data into 8-bit components and encapsulating them in YUV4:4:4 format, this solution can adapt to the input requirements of standard video codecs. It can directly utilize mature codecs such as AVC and HEVC to achieve efficient compression without any neural network training, while ensuring that the encoding and decoding process is fully compatible with mainstream hardware decoders.

[0050] In summary, the 4D Gaussian sputtering compression method based on UV mapping proposed in this invention effectively solves the compression problem caused by the unstructured nature of 4D Gaussian sputtering data. It achieves a good balance between compression efficiency, reconstruction quality, hardware compatibility and deployment cost, and provides reliable technical support for the large-scale application of dynamic scenes in bandwidth-constrained environments such as mobile devices and real-time transmission.

[0051] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0052] Furthermore, in this invention, descriptions involving terms such as "first," "second," and "a" are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0053] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0054] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

Claims

1. A 4D Gaussian sputter compression method based on UV mapping, characterized in that, The method comprises the steps of: S1: converting the three-dimensional coordinates of the center point of each 3D Gaussian sputtering basic unit in the 4D Gaussian sputtering sequence into spherical coordinates through coordinate conversion, and mapping and constructing the azimuth and polar angle in the spherical coordinates into a two-dimensional UV representation; S2: performing multiple rounds of iterative sampling on the grid corresponding to the two-dimensional UV representation, selecting the 3D Gaussian sputtering basic unit with the largest radial distance in each grid cell as high-priority data in each round of sampling, and forming a layer of UV mapping for the 3D Gaussian sputtering basic unit reserved in each round of sampling, and constructing multiple layers of UV mapping through multiple rounds of sampling; S3: collecting all unselected 3D Gaussian sputtering basic units after iterative sampling, and reordering each fixed U-axis coordinate point data along the V-axis direction according to the polar angle in the corresponding spherical coordinates as supplementary data; S4: based on the number of video codec channels, the multi-layer UV mapping and the attribute data of the reordered 3D Gaussian sputtering basic units are split into corresponding bit components and packaged into YUV format data, wherein the attribute data includes the three-dimensional position information, rotation parameter, scale parameter, transparency and spherical harmonic function coefficient of the center point of each 3D Gaussian sputtering basic unit; S5: inputting the YUV format data into the video codec, and performing data compression under the code rate priority allocation with the goal of prioritizing the reconstruction quality of high-priority data.

2. A 4D Gaussian sputter compression method based on UV mapping as claimed in claim 1, wherein, In the S1 step, the coordinate conversion is specifically: obtaining the three-dimensional rectangular coordinates of the center point of each 3D Gaussian sputtering basic unit, and converting the three-dimensional rectangular coordinates into spherical coordinates containing radial distance, azimuth and polar angle; The mapping is specifically: according to the preset UV mapping resolution parameters M and N, the azimuth and polar angle are divided respectively, and the azimuth and polar angle are mapped to the two-dimensional UV coordinate grid through the integer operation to generate a two-dimensional UV representation with time consistency.

3. A 4D Gaussian sputter compression method based on UV mapping as defined in claim 1, wherein, In the S3 step, the specific process of reordering includes: analyzing the UV coordinate distribution of all unselected 3D Gaussian sputtering basic units, and determining that the distribution uniformity along the U-axis direction in the two-dimensional UV is higher than the distribution uniformity along the V-axis direction based on statistical rules; taking the U-axis as the reference dimension, arranging all 3D Gaussian sputtering basic units with a fixed U coordinate value along the V-axis direction in ascending or descending order according to the polar angle in the spherical coordinates.

4. The 4D Gaussian sputter compression method based on UV mapping of claim 1, wherein, The video codec is an AVC or HEVC standard encoder.

5. A 4D Gaussian sputter compression method based on UV mapping as defined in claim 4, wherein, In the S4 step, the splitting is specifically: splitting each 16-bit Gaussian attribute data into high 8 bits and low 8 bits.

6. A 4D Gaussian sputter compression method based on UV mapping as defined in claim 4, wherein, In the S4 step, the YUV format used for packaging is YUV4:4:4 pixel format.

7. The 4D Gaussian sputter compression method based on UV mapping of claim 1, wherein, In the S5 step, the code rate priority allocation is realized by configuring the quantization parameter or constant code rate factor of the encoder, and lower quantization parameter or better code rate control strategy is allocated for high-priority data.

8. The 4D Gaussian sputter compression method based on UV mapping of claim 1, wherein, In the S2 step, it also includes the step of updating the effective 3D Gaussian sputtering basic unit set of the current grid after each round of sampling, and the sampling of the subsequent rounds is based on the updated data set of the previous round.

Images