4D Gaussian sputtering compression method based on UV mapping

Abstract

The invention discloses a 4D Gaussian sputtering compression method based on UV mapping, and relates to the technical field of video elementary stream processing, and the method mainly comprises the steps: converting a central point three-dimensional coordinate of each 3D Gaussian sputtering elementary unit in a 4D Gaussian sputtering sequence into a spherical coordinate through coordinate conversion, mapping an azimuth angle and a polar angle in the spherical coordinates and constructing a two-dimensional UV grid representation; through multiple rounds of iterative sampling, selecting a 3D Gaussian sputtering basic unit with the maximum radial distance in each round of grid units as high-priority data to construct multilayer UV mapping; collecting all unselected 3D Gaussian sputtering basic units after iterative sampling, and reordering the unselected 3D Gaussian sputtering basic units as detail supplementary data; and splitting the attribute data of the 3D Gaussian sputtering basic unit into corresponding digit components based on the channel number of the video codec, packaging the corresponding digit components into YUV format data, and inputting the YUV format data into the video codec for compression. According to the invention, the input requirement of a standard video codec can be adapted without training.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video elementary stream processing, and particularly relates to a 4D Gaussian Splatting compression method based on UV mapping. BACKGROUND

[0002] Although the 4D Gaussian Splatting technology has significant advantages in virtual reality, remote presentation and other fields due to its high-fidelity modeling and real-time rendering capability for dynamic scenes, its large-scale application is still limited by high storage and transmission costs. The existing technical solutions have obvious deficiencies in dealing with this challenge: first, the compression method for static 3D Gaussian Splatting cannot effectively handle the temporal redundancy and inter-frame correlation inherent in dynamic scenes, and lacks the ability to model temporal consistency; second, a few compression schemes extended to 4D scenes are difficult to directly calculate inter-frame motion and residuals due to the unstructured nature of Gaussian primitives, often need to indirectly estimate motion information by using rendered images, resulting in strong coupling between compression performance and specific rendering pipeline, poor generalization ability, and usually relying on complex neural network training, which not only greatly increases the computational overhead, but also reduces the compatibility with general hardware platforms. In addition, existing standard video codecs (such as AVC, HEVC) are designed for structured 2D image data and cannot directly process unstructured Gaussian sequences, lacking effective adaptation mechanisms, resulting in complex and inefficient compression processes. These problems together make it difficult for existing methods to balance compression efficiency, reconstruction quality and hardware compatibility, severely restricting the large-scale application of 4D Gaussian Splatting technology in bandwidth-limited scenarios. SUMMARY

[0003] In order to balance compression efficiency, reconstruction quality and hardware compatibility, and better promote the large-scale application of 4D Gaussian Splatting technology, the present application proposes a 4D Gaussian Splatting compression method based on UV mapping, including the following steps: S1: converting the three-dimensional coordinates of the center point of each 3D Gaussian Splatting basic unit in the 4D Gaussian Splatting sequence into spherical coordinates through coordinate transformation, and mapping and constructing the azimuth and polar angle in the spherical coordinates into two-dimensional UV representation; S2: selecting the 3D Gaussian Splatting basic unit with the largest radial distance in each grid cell as high-priority data by performing multiple rounds of iterative sampling on the grid corresponding to the two-dimensional UV representation, and constructing a multi-layer UV map; S3: collecting all 3D Gaussian Splatting basic units that are not selected after iterative sampling, and reordering each fixed U-axis coordinate point data along the V-axis direction according to the polar angle order under its corresponding spherical coordinates as detail supplement data; S4: split the multi-layer UV mapping and the attribute data of the reordered 3D Gaussian sputtering basic units based on the number of video codec channels into corresponding bit components and package as YUV format data; S5: input the YUV format data into the video codec to perform data compression under code rate priority allocation with the goal of prioritizing the reconstruction quality of high priority data.

[0004] The application converts the unstructured 4D Gaussian sputtering sequence into a structured two-dimensional representation with spatial locality and temporal consistency by introducing a time-consistent spherical projection and UV mapping mechanism, laying a solid foundation for subsequent compression operations. The iterative sampling strategy based on radial distance can intelligently identify and prioritize high-priority data layers that represent key structures of the scene. The ordered organization of residual cells effectively guarantees the spatial coherence of detailed information, and the two work together to achieve differentiated processing of visually important information under limited code rate. Splitting and packaging data based on the number of video codec channels can adapt to the input requirements of standard video codecs, allowing efficient compression using mature codecs such as AVC and HEVC without any neural network training, while ensuring complete compatibility with mainstream hardware decoders during the encoding process.

[0005] Further, in the S1 step, the coordinate conversion is specifically: Obtain the three-dimensional rectangular coordinates of the center point of each 3D Gaussian sputtering basic unit, and convert the three-dimensional rectangular coordinates into spherical coordinates containing radial distance, azimuth angle and polar angle; The mapping is specifically: According to the preset UV mapping resolution parameters M and N, the azimuth angle and the polar angle are divided respectively, and the azimuth angle and the polar angle are mapped to the two-dimensional UV coordinate grid through the integer operation to generate a two-dimensional UV representation with temporal consistency.

[0006] Further, in the S3 step, the specific process of the reordering includes: Analyze the UV coordinate distribution of all unselected 3D Gaussian sputtering basic units, and determine that the distribution uniformity along the U-axis direction in the two-dimensional UV is higher than the distribution uniformity along the V-axis direction based on statistical rules; Taking the U-axis as the reference dimension, for all 3D Gaussian sputtering basic units with a fixed U coordinate value, arrange them in ascending or descending order along the V-axis direction according to the polar angle in their spherical coordinates.

[0007] Further, the video codec is an AVC or HEVC standard encoder.

[0008] Further, in the S4 step, the splitting is specifically: split each 16-bit Gaussian attribute data into high 8-bit and low 8-bit components.

[0009] Further, in the S4 step, the YUV format used for encapsulation is YUV4:4:4 pixel format.

[0010] Further, in the S5 step, the code rate priority allocation is achieved by configuring the quantization parameter or constant code rate factor of the encoder, and lower quantization parameter or better code rate control strategy is allocated to high priority data.

[0011] Further, in the S2 step, it also includes the step of updating the effective 3D Gaussian spatter basic unit set of the current grid after each round of sampling, and the sampling of the subsequent round is based on the updated data set of the previous round.

[0012] Compared with the prior art, the present application has at least the following beneficial effects: (1) The 4D Gaussian spatter compression method based on UV mapping proposed in the present application converts the unstructured 4D Gaussian spatter sequence into a structured two-dimensional representation with spatial locality and temporal consistency by introducing a time-consistent spherical projection and UV mapping mechanism, laying a solid foundation for subsequent compression operations; (2) The iterative sampling strategy based on radial distance can intelligently identify and preferentially retain Gaussian cells representing key structures of the scene, forming a high-priority data layer, while the ordered organization of residual cells effectively guarantees the spatial coherence of detailed information, and both of them cooperatively realize differentiated processing of visually important information under limited code rate; (3) By splitting 16-bit attribute data into 8-bit components and encapsulating them in YUV4:4:4 format, the input requirements of standard video codecs can be adapted, and mature codecs such as AVC and HEVC can be directly used for efficient compression without any neural network training, while ensuring that the encoding process is fully compatible with mainstream hardware decoders. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is a flowchart of a 4D Gaussian spatter compression method based on UV mapping; Figure 2 is a schematic diagram of a 4D Gaussian spatter compression method based on UV mapping; Figure 3 is a UV projection schematic diagram. DETAILED DESCRIPTION

[0014] The following is a specific embodiment of the present application combined with the drawings, which further describes the technical solutions of the present application, but the present application is not limited to these embodiments.

[0015] In the field of 4D dynamic scene representation and compression, the core dilemma faced by existing technical solutions is the difficulty in effectively handling the unstructured nature of Gaussian Splatting data. Traditional compression methods for static 3D Gaussian Splatting cannot model the temporal redundancy in dynamic scenes, while a few solutions extended to 4D rely on indirect estimation of inter-frame motion information from rendered images due to the lack of regular spatial structure in Gaussian primitives. This indirect processing approach results in poor generalization and often requires computationally intensive neural network training, increasing implementation complexity and reducing compatibility with standard hardware decoding platforms. More importantly, existing international standard video codecs (such as AVC, HEVC) are designed for structured two-dimensional image data and cannot directly handle unstructured Gaussian sequence data, making it difficult to balance compression efficiency, reconstruction quality, and hardware adaptability for 4D Gaussian Splatting technology, which severely restricts its large-scale application in bandwidth-constrained scenarios. This series of technical bottlenecks is the key problem that the present solution aims to solve. Therefore, as shown in Figure 1 , the present application proposes a 4D Gaussian Splatting compression method based on UV mapping, including the following steps: S1: Convert the three-dimensional coordinates of the center point of each 3D Gaussian Splatting basic unit in the 4D Gaussian Splatting sequence to spherical coordinates through coordinate transformation, and map and construct the azimuth and polar angle in the spherical coordinates as two-dimensional UV representation; S2: Perform multi-round iterative sampling on the grid corresponding to the two-dimensional UV representation, and select the 3D Gaussian Splatting basic unit with the largest radial distance in each round grid cell as high-priority data to construct multi-layer UV mapping; S3: Collect all unselected 3D Gaussian Splatting basic units after iterative sampling, and reorder each fixed U-axis coordinate point data along the V-axis direction according to the polar angle order under its corresponding spherical coordinates as detail supplement data; S4: Based on the number of video codec channels, split the multi-layer UV mapping and the attribute data of the reordered 3D Gaussian Splatting basic units into corresponding bit components and encapsulate them as YUV format data; S5: Input the YUV format data into the video codec to perform data compression under the code rate priority allocation with the goal of prioritizing the reconstruction quality of high-priority data.

[0016] Specifically, as shown in Figure 2 and Figure 3As shown, before compression, the 4D Gaussian sputtering sequence data to be compressed needs to be prepared. This sequence can be output by any 4D Gaussian sputtering reconstruction model (such as a model trained on the N3DV dataset) or generative model (such as L4GM), and is essentially a time-varying set of 3D Gaussian sputtering basic units. Each 3D Gaussian sputtering basic unit (hereinafter referred to as "Gaussian unit") contains a series of attribute parameters, mainly including the three-dimensional position coordinates of its center point. The parameters include rotation parameters (usually represented by quaternions), 3D scale parameters, transparency (alpha), and spherical harmonics (SH) coefficients representing color. To adapt to the compression process proposed in this invention, key parameters need to be pre-set: the resolution parameters M and N of the UV mapping (i.e., the number of grids in the U and V directions), which are typically set to M=512 and N=512 depending on the target compression accuracy and scene complexity; the number of iterations K for sampling, serving as a control parameter for redundancy removal intensity, preferably K=6 to achieve a balance between efficiency and effect; and the video codec configuration, which can preset two modes to adapt to different needs: a lossless compression mode (e.g., using an HEVC encoder, configured with a constant quantization parameter CRF=0) and a visually lossless high compression mode (e.g., CRF=30).

[0017] To transform unstructured, scattered 3D Gaussian primitives into a structured 2D representation with spatial locality and temporal consistency, facilitating subsequent encoding and decoding using standard video codecs, this invention first traverses each frame of the 4D Gaussian sputtering sequence and obtains the 3D Cartesian coordinates of the center point of each Gaussian primitive within that frame. Next, using the standard spherical coordinate transformation formula, the rectangular coordinates of each point are converted to spherical coordinates. , where radial distance azimuth polar angle Each primitive has a spherical coordinate representation based on the current frame's camera coordinate system (or world coordinate system).

[0018] Next, the crucial UV mapping is performed. Based on preset resolution parameters M and N (e.g., 512×512), the azimuth angle is determined. and polar angle The value range is divided evenly. Specifically, the range of values ​​is divided into... from Mapping to U coordinates ,Will From mapping to V coordinates Rounding down consecutive integers... Discretize and map to a unique two-dimensional grid coordinate system. The mapping strategy ensures that in a dynamic scene, the corresponding Gaussian primitives of the same object surface at the same time in different frames will be mapped into the same or adjacent cells in the UV grid, thereby establishing crucial temporal consistency and solving the inherent difficulty of inter-frame motion estimation for unstructured data.

[0019] After UV mapping, all Gaussian primitives are assigned to specific cells in the two-dimensional UV grid. However, directly compressing all Gaussian primitives in each cell will generate a large amount of redundancy. Therefore, the present application proposes an iterative sampling mechanism for redundancy optimization. The process takes the generated two-dimensional UV grid as the operating object and performs K rounds (such as 6 rounds) of sampling. At the beginning of each round of sampling, the algorithm traverses each cell in the two-dimensional UV grid. For cells containing multiple Gaussian primitives, the radial distance of each primitive in the cell is calculated. The Gaussian primitive with the largest radial distance in the cell is selected and retained, and the other Gaussian primitives in the same cell are removed. The deep principle of this selection method is that in a ray direction from the common viewpoint, the primitive with the largest radial distance is most likely to be located on the object surface and is the key viewpoint for constructing the visible structure of the scene, so it needs to be given higher priority.

[0020] After each round of sampling is completed, the current set of valid Gaussian primitives of the two-dimensional UV grid is updated - that is, the Gaussian primitives removed in this round are removed from the grid. The sampling of the subsequent rounds is based on the partially optimized data set updated after the previous round. This iterative updating mechanism enables the sampling process to progress layer by layer, selecting the most representative Gaussian primitive in each round from a different perspective (based on the updated data distribution). After K rounds of iteration, the representative Gaussian primitives retained in each round are integrated to form K layers (such as 6 layers) of UV mapping. The K-layer mapping collectively constitutes high-priority data (2D UV maps) representing the core framework of the scene.

[0021] Iterative sampling aims to retain the most critical information, but the removed Gaussian primitives are not useless, as they also contain rich detail information such as internal structure and surface micro-relief, and direct disposal will result in a decrease in reconstruction quality. Therefore, it is necessary to effectively organize them by collecting all primitives that are not selected in all K rounds of iterative sampling to form a residual set.

[0022] However, the distribution of Gaussian primitives in the residual set in the two-dimensional UV space is sparse and disorganized, and if directly processed, the spatial correlation is weak, which is not conducive to compression. Therefore, the present application also proposes an ordered organization scheme based on the characteristics of two-dimensional UV distribution. By analyzing the two-dimensional UV coordinate distribution of the residual Gaussian primitives, a key statistical law is observed: due to the azimuthal symmetry of the Gaussian primitive distribution, the radial distance of the Gaussian primitive in the same cell is approximately equal to the radial distance of the Gaussian primitive in the same cell. With continuity, the distribution of Gaussian primitives in the U-axis direction is usually more uniform than in the V-axis direction. Based on this, the U-axis is chosen as the reference dimension for sorting. The specific reordering process is as follows: For each fixed U coordinate value , all the residual Gaussian primitives are collected. Then, instead of being sorted by their original, chaotic V coordinates, these Gaussian primitives are arranged in ascending (or descending) order along the V-axis direction according to the size of their original polar angle in spherical coordinates. This operation is very important, as it utilizes the original geometric information (polar angle ) of the Gaussian primitives in 3D space to enforce spatial order consistency in the two-dimensional UV plane, reorganizing the scattered residual points into a series of ordered "lines" along the V-axis direction. The data organized in this way can be used as residual map, which maintains spatial coherence and creates favorable conditions for subsequent video encoders to utilize intra-frame prediction compression.

[0023] After the above steps, structured high-priority data and structured residual map data have been obtained, and the next step is to adapt these data to standard video codecs for compression. Since standard codecs (such as AVC, HEVC) usually process 8-bit depth YUV format images, while Gaussian attribute data (such as position, color, etc.) are mostly 16-bit or 32-bit floating-point numbers, quantization and format conversion are required.

[0024] First, attribute data splitting and quantization. Taking 16-bit attribute data as an example, each 16-bit value is considered to be composed of two 8-bit bytes: a most significant byte (MSB) and a least significant byte (LSB). The MSB and LSB of all attributes (including position, rotation, scale, transparency, SH coefficients, etc.) are extracted and reorganized. Specifically, an image plane can be formed for the MSB of all Gaussian primitives of the same attribute, and another image plane can be formed for the LSB. Next, when packaging the pixel format, the YUV4:4:4 format is selected. This format does not downsample the chroma signal, and can fully preserve the complete spatial resolution of the Y, U, and V channels, thereby avoiding the loss of Gaussian attribute information (especially geometric information) caused by chroma subsampling, and maximizing the reconstruction accuracy after compression. At this point, the structured Gaussian data has been converted into a series of 8-bit depth, multi-channel YUV4:4:4 image sequences.

[0025] Finally, the encapsulated YUV image sequence is directly input to a standard video codec for compression. In this process, in order to achieve the goal of "prioritizing high priority data", a code rate priority allocation strategy needs to be implemented. A direct implementation is to use the code rate control function of the encoder. For example, during encoding, the quantization parameter (QP) of the image area representing high priority data (such as the first K frames) can be set to a lower value (meaning finer quantization, higher code rate allocation), while the QP of the image area representing detail supplement data is set to a relatively high value (coarser quantization, lower code rate allocation). Another implementation can also use the constant rate factor (CRF) mode, and adjust the quality of different parts by setting different CRF values. In this way, under the condition of limited total code rate, the encoder will automatically allocate more bits to high priority data, thereby ensuring the reconstruction quality of the key structures of the scene.

[0026] As for the subsequent decoding, it is the inverse process of encoding, and the receiving end uses a standard video decoder (any decoding chip or software supporting HEVC, AVC) to decode the code stream and restore the YUV4:4:4 image sequence. Subsequently, the reverse operation is performed: the data of the MSB and LSB channels are recombined to restore the original 16-bit Gaussian attribute data; according to the two-dimensional UV coordinate mapping relationship, the image data is reanalyzed into a structured two-dimensional UV map and a residual map; finally, the high priority data and the detail supplement data are merged to reconstruct a complete 4D Gaussian sputtering sequence that can be used for real-time rendering.

[0027] From the above specific embodiments, it can be seen that by introducing a time-consistent spherical projection and UV mapping mechanism, the present application successfully converts the unstructured 4D Gaussian sputtering sequence into a structured two-dimensional representation with spatial locality and temporal consistency, laying a solid foundation for subsequent compression. The iterative sampling strategy based on radial distance can intelligently identify and prioritize high priority data layers that represent key structures of the scene, while the ordered organization of residual cells effectively guarantees the spatial coherence of detail information, both of which cooperatively achieve differentiated processing of visually important information under limited code rate.

[0028] By splitting the 16-bit attribute data into 8-bit components and encapsulating them in YUV4:4:4 format, this scheme can adapt to the input requirements of standard video codecs, and directly utilize mature codecs such as AVC, HEVC for efficient compression without any neural network training, while ensuring that the encoding and decoding process is fully compatible with mainstream hardware decoders.

[0029] In conclusion, the 4D Gaussian sputtering compression method based on UV mapping provided by the application effectively solves the compression problem of 4D Gaussian sputtering data caused by unstructured characteristics, and achieves a good balance between compression efficiency, reconstruction quality, hardware compatibility and deployment cost, thereby providing reliable technical support for the large-scale application of dynamic scenes in bandwidth-limited environments such as mobile devices and real-time transmission.

[0030] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the application are only used to explain the relative positional relationship, movement condition, etc. between components in a certain posture (as shown in the drawings), and if the certain posture changes, the directional indications will also change accordingly.

[0031] In addition, the descriptions such as "first", "second", "one" and the like in the present application are only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise explicitly specified and limited.

[0032] In the present application, unless otherwise explicitly specified and limited, the terms "connection", "fixation" and the like should be understood in a broad sense, for example, "fixation" can be fixed connection, or detachable connection, or integral; can be mechanical connection, or electrical connection; can be directly connected, or indirectly connected through an intermediate medium; can be the internal communication of two elements or the interaction relationship between two elements, unless otherwise explicitly limited. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0033] In addition, the technical solutions of each embodiment of the present application can be combined with each other, but it must be based on the fact that a person skilled in the art can realize it, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor is it within the scope of protection required by the present application.

Claims

1. A 4D Gaussian sputtering compression method based on UV mapping, characterized in that, Including the following steps: S1: The three-dimensional coordinates of the center point of each 3D Gaussian sputtering basic unit in the 4D Gaussian sputtering sequence are converted into spherical coordinates through coordinate transformation, and the azimuth and polar angles in the spherical coordinates are mapped and constructed into a two-dimensional UV representation; S2: By performing multiple rounds of iterative sampling on the mesh corresponding to the two-dimensional UV representation, the 3D Gaussian sputtering basic unit with the largest radial distance within the mesh cell in each round is selected as high-priority data to construct a multi-layer UV mapping; S3: Collect all unselected 3D Gaussian sputtering basic units after iterative sampling, and reorder the data of each fixed U-axis coordinate point along the V-axis direction according to the polar angle order in the corresponding spherical coordinates, as supplementary detailed data; S4: Based on the number of video codec channels, the attribute data of multi-layer UV mapping and reordered 3D Gaussian sputtering basic units are split into corresponding bit components and encapsulated into YUV format data. S5: Input YUV format data into the video codec and compress the YUV format data under the bitrate priority allocation with the goal of prioritizing the reconstruction quality of high-priority data.

2. The 4D Gaussian sputtering compression method based on UV mapping as described in claim 1, characterized in that, In step S1, the coordinate transformation specifically involves: Obtain the three-dimensional rectangular coordinates of the center point of each 3D Gaussian sputtering basic unit, and convert the three-dimensional rectangular coordinates into spherical coordinates that include radial distance, azimuth angle and polar angle; The mapping is specifically as follows: Based on the preset UV mapping resolution parameters M and N, the azimuth and polar angles are divided respectively. The azimuth and polar angles are mapped onto the two-dimensional UV coordinate grid through rounding operations to generate a two-dimensional UV representation with time consistency.

3. The 4D Gaussian sputtering compression method based on UV mapping as described in claim 1, characterized in that, In step S3, the specific process of reordering includes: The UV coordinate distribution of all unselected 3D Gaussian sputtering basic units was analyzed, and based on statistical regularity, it was determined that the uniformity of the distribution along the U-axis in the two-dimensional UV was higher than that along the V-axis. Using the U-axis as the reference dimension, all 3D Gaussian sputtering basic units under each fixed U-coordinate value are arranged in ascending or descending order along the V-axis according to the size of their polar angle in spherical coordinates.

4. The 4D Gaussian sputtering compression method based on UV mapping as described in claim 1, characterized in that, The video codec is an AVC or HEVC standard encoder.

5. The 4D Gaussian sputtering compression method based on UV mapping as described in claim 4, characterized in that, In step S4, the splitting specifically involves splitting each 16-bit Gaussian attribute data into two components: a high 8-bit component and a low 8-bit component.

6. The 4D Gaussian sputtering compression method based on UV mapping as described in claim 4, characterized in that, In step S4, the YUV format used for encapsulation is YUV4:4:4 pixel format.

7. The 4D Gaussian sputtering compression method based on UV mapping as described in claim 1, characterized in that, In step S5, the bit rate priority allocation is achieved by configuring the encoder's quantization parameters or constant bit rate factor, allocating lower quantization parameters or better bit rate control strategies to high-priority data.

8. The 4D Gaussian sputtering compression method based on UV mapping as described in claim 1, characterized in that, Step S2 further includes the step of updating the set of valid 3D Gaussian sputtering basic units of the current mesh after each round of sampling, and the sampling of subsequent rounds is based on the dataset updated in the previous round.

Images