Three-dimensional Gaussian compression method and system based on just perceptible distortion

Abstract

The invention relates to the technical field of 3D Gaussian splashing, in particular to a three-dimensional Gaussian compression method and system based on just perceptible distortion, and the method comprises the steps: collecting images, integrating and building a data set, and inputting the data set into a JND model to obtain a corresponding JND image; inputting the data set and the corresponding JND image into an anchor point-based 3DGS compression method model for training, sequentially obtaining the opacity and the JND score accumulation value of the anchor point, designing a loss function, and optimizing the attribute of the anchor point; judging whether the anchor point is pruned or not according to the opacity and the JND score accumulated value, and completing training of a 3DGS compression method model based on the anchor point; and the trained anchor point-based 3DGS compression method model outputs a rendered image. Extreme compression is achieved on the premise that human visual perception is not affected based on just perceptible differences, and an effective technical scheme is provided for storage and transmission of three-dimensional visual data.

Description

Technical Field

[0001] This invention relates to the field of 3D Gaussian splashing technology, specifically to a three-dimensional Gaussian compression method and system based on just-perceptible distortion. Background Technology

[0002] 3D Gaussian Spaltting (3DGS) has become the preferred tool for 3D scene reconstruction due to its low latency and accurate reconstruction capabilities, replacing the previous Neural Radiance Field (NERF) method. This technique represents the scene using Gaussian primitives, whose parameters such as mean, covariance, color, and opacity can be directly optimized through a differentiable pipeline, enabling real-time high-fidelity rendering on graphics processing units (GPUs). However, the massive number of Gaussian primitives results in extremely high scene storage requirements, prompting researchers to conduct research on compression techniques for 3DGS.

[0003] Existing compression techniques typically reduce the number of Gaussian elements through pruning or quantize Gaussian attributes using codebooks. However, both neglect the inherent redundancy in Gaussian structural relationships and fail to consider compression based on the Human Visual System (HVS). For example, Scaffold-GS (Structured 3D Gaussian for Adaptive View Rendering) proposes generating Gaussian elements based on anchor points to reduce structural redundancy and converting anchor attributes into Gaussian attributes using a multilayer perceptron. However, it only processes each anchor point independently, without considering the connection between anchor attributes and the structured feature network. Hash Grid Assisted Context Compression (HAC) constructs a learning-based structured compact hash grid, querying the hash grid by anchor position to obtain interpolated hash features to predict the distribution of anchor attribute values. However, it only uses hash features and does not consider the internal features of anchor attributes. HAC++ further optimizes entropy coding based on HAC, mines information inside anchors to improve contextual accuracy, and uses a masking strategy to prune invalid Gaussian units and anchors. While it reduces redundant information in the model as much as possible, it does not take into account perceptual quality redundancy. Summary of the Invention

[0004] To address the problem that while existing compression techniques can achieve compression to some extent, they still retain a large amount of redundant information and fail to consider compression based on the human visual system, the present invention aims to provide a three-dimensional Gaussian compression method based on just-perceptible distortion. The specific technical solution adopted is as follows:

[0005] Collect images and integrate them to create a dataset. Input the dataset into the JND model to obtain the corresponding JND image.

[0006] The dataset and corresponding JND images are input into the anchor-based 3DGS compression method model for training. The opacity of the anchor points and the cumulative JND score are obtained in sequence. A loss function is designed to optimize the anchor point properties.

[0007] The anchor point should be pruned based on a combination of opacity and cumulative JND score, thus completing the training of the anchor point-based 3DGS compression method model.

[0008] The trained anchor-based 3DGS compression method model is passed through Output rendered image.

[0009] Preferably, the anchor point attributes include features, scale, and offset.

[0010] Preferably, in the anchor-based 3DGS compression method model, minimum perceptible information is introduced to optimize the compression strategy and parameter training.

[0011] Preferably, the dataset and corresponding JND images are input into the anchor-based 3DGS compression method model for training, thereby obtaining the anchor opacity and cumulative JND score, and designing a loss function to optimize anchor attributes, including:

[0012] The JND image is converted into Gaussian elements for analysis based on the anchor-based 3DGS compression method model. The Gaussian opacity of the Gaussian elements and the JND importance score are integrated to determine the opacity of the anchor points and the cumulative value of the JND score.

[0013] The initial rendered images of the dataset are obtained by using an anchor-based 3DGS compression method model. A loss function is designed by combining the dataset and the corresponding JND images to optimize the learning direction of anchor attributes.

[0014] Preferably, the Gaussian opacity of the Gaussian unit and the JND importance score correspond to the determination of the anchor point's opacity and the cumulative JND score, including:

[0015] Evaluate the Gaussian opacity of each Gaussian pixel corresponding to the JND image and backpropagate it to the anchor point to determine the cumulative opacity value of the anchor point;

[0016] The JND image is input into the rasterization module. The JND importance score of the Gaussian pixel is obtained based on the intersection point of the individual Gaussian pixel and the pixel in the JND image and the corresponding pixel value. The score is then propagated backward to the anchor point through offset to generate the JND score of the anchor point. The corresponding cumulative JND score value is then determined.

[0017] Preferably, the initial rendered images of the dataset are obtained through an anchor-based 3DGS compression method model. A loss function is designed by combining the dataset and the corresponding JND images to optimize the learning direction of anchor attributes, including:

[0018] Perform pixel-by-pixel loss calculation on the images in the dataset and compare the pixel values ​​of the paired images with those of the initial rendered images;

[0019] A difference-aware modulator is set up, and a loss function is designed by combining the JND image with the pixel value difference and using an activation function to optimize the learning direction of the anchor attribute.

[0020] Preferably, the anchor point is pruned based on a combination of opacity and cumulative JND score, specifically as follows:

[0021] A preset judgment threshold is set, and the judgment threshold is compared with the opacity and the cumulative JND score. If both are less than the threshold, the corresponding anchor point is pruned.

[0022] To address the aforementioned problems, the present invention also provides a three-dimensional Gaussian compression system based on just-perceived distortion, the system comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus, and the processor calls logical instructions in the memory to execute the three-dimensional Gaussian compression method based on just-perceived distortion described in any of the preceding claims.

[0023] The present invention has the following beneficial effects:

[0024] 1. Achieving maximum compression without affecting human visual perception based on Just Noticeable Difference (JND). JND represents the minimum difference detectable by the human visual system (HVS). By acquiring JND images, it reflects the differences in perceptual sensitivity of the human eye to various regions. Training a 3DGS compression method model based on anchor points can identify information redundancy during the compression process and safely remove it without being noticed, achieving efficient compression while maintaining visual quality. Finally, through... Outputting rendered images, that is, effectively outputting high-quality rendered images, significantly reducing the amount of data while maintaining the indistinguishability of human visual perception, provides an effective technical solution for the storage and transmission of 3D visual data.

[0025] 2. The three-dimensional Gaussian compression system based on just-perceived distortion provided by this invention has the same beneficial effects as the three-dimensional Gaussian compression method based on just-perceived distortion provided by this invention, and will not be described in detail here. Attached Figure Description

[0026] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A flowchart illustrating the steps of a three-dimensional Gaussian compression method based on just-perceived distortion, as provided in an embodiment of the present invention;

[0028] Figure 2 This is a flowchart illustrating an implementation of a three-dimensional Gaussian compression method based on just-perceived distortion, as provided in one embodiment of the present invention. Detailed Implementation

[0029] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a three-dimensional Gaussian compression method and system based on just-perceptible distortion proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0031] The following description, in conjunction with the accompanying drawings, details a specific scheme for a three-dimensional Gaussian compression method and system based on just-perceived distortion provided by the present invention.

[0032] To better illustrate, Just-perceptible difference (JND) refers to the minimum change in stimulus that can be detected by the human sensory system during perception. For example, in the visual domain, the human eye can only distinguish the difference between two colors when the brightness difference reaches a certain threshold; the corresponding minimum brightness difference is the JND. In practical applications, JND is widely used in image processing, audio coding, display technology, etc. By utilizing the insensitivity of the human eye or ear to differences below JND, data compression can be achieved, reducing the burden of information transmission or storage while maintaining perceptual quality without significant impact.

[0033] Three-dimensional Gaussian compression refers to a technique for data compression based on a three-dimensional Gaussian distribution model. It utilizes the Gaussian model to approximate the statistical characteristics of many natural three-dimensional data, allowing a large amount of data to be represented with fewer parameters, thus achieving efficient compression.

[0034] Please combine Figure 1 and Figure 2 It illustrates a flowchart and implementation flowchart of a three-dimensional Gaussian compression method and system based on just-perceptible distortion provided in the first embodiment of the present invention. The method includes:

[0035] Step S1: Collect and integrate images to create a dataset, and input the dataset into the JND model to obtain the corresponding JND image;

[0036] Step S2: Input the dataset and the corresponding JND image into the anchor-based 3DGS compression method model for training, obtain the opacity of the anchor and the cumulative value of the JND score in turn, and design a loss function to optimize the anchor attributes;

[0037] Step S3: Combine opacity and JND score accumulation to determine whether anchor points should be pruned, and complete the training of the anchor point-based 3DGS compression method model;

[0038] Step S4: The trained anchor-based 3DGS compression method model is then... Output rendered image.

[0039] As an optional implementation method, a dataset is created by collecting a large number of representative natural scene images, such as urban landscapes, indoor environments, portraits, and natural scenery, to ensure the diversity and universality of the dataset.

[0040] In this embodiment, the JND model refers to the KLT-JND model (Karhunen-Loève Transform Just Noticeable Difference Model), which is a KLT-based visual perception model used in image and video quality assessment, compressed sensing, and signal processing. It determines the amount of redundant information in a signal that can be compressed or removed without affecting subjective perception quality by simulating the sensitivity of the human eye to changes in image details.

[0041] To better illustrate, the anchor-based 3DGS compression method model is a technical framework for efficient compression of three-dimensional Gaussian sphere models. It introduces anchor points as reference points to perform structured representation and quantized compression of the Gaussian distribution parameters in the original 3DGS model.

[0042] Furthermore, anchor point attributes include features, scale, and offset.

[0043] It can be explained that in the initialization stage of the anchor-based 3DGS compression method model, the 3D scene in the dataset and the corresponding JND image is initialized as voxels. That is, the 3D scene is spatially divided and discretized into a series of irregular or irregular voxel units. Each voxel is placed into an anchor point, which serves as the basis for subsequent 3DGS parameter optimization. Among them, features determine the recognition and matching ability of anchor points in a specific scene; scale reflects the aerial range or physical size covered by the anchor point; offset is used to describe the displacement of the anchor point relative to the actual target position.

[0044] Furthermore, in the anchor-based 3DGS compression method model, minimum perceptible information is introduced to optimize the compression strategy and parameter training.

[0045] It can be explained that the minimum perceptible information, i.e., the analysis of JND images, utilizes the redundancy characteristics of the human perceptual system to remove or compress information exceeding the perceptual threshold as much as possible without affecting subjective visual quality, in order to achieve higher compression efficiency. In other words, JND theory is integrated into the training of the anchor-based 3DGS compression method model, and the visual importance of different anchor points is differentiated. This guides the fine design of the compression strategy from the perceptual level, and improves the balance between compression efficiency and visual quality by optimizing the parameter training process.

[0046] Further, step S2 includes:

[0047] Step S21: Convert the JND image into Gaussian elements for analysis based on the anchor-based 3DGS compression method model, and determine the anchor opacity and cumulative JND score by integrating the Gaussian opacity of the Gaussian elements and the JND importance score.

[0048] It can be explained that the anchor-based 3DGS compression method model uses a set of Gaussian primitives with various properties such as position, opacity, scaling factor, rotation quaternion, and spherical harmonic function to represent 3D data, that is, converting JND images into Gaussian primitives for analysis.

[0049] Further, in step S21, the Gaussian opacity of the Gaussian unit and the JND importance score are integrated to determine the opacity and cumulative JND score of the anchor point, including:

[0050] Step S211: Evaluate the Gaussian opacity of each Gaussian pixel corresponding to the JND image and back-transmit it to the anchor point to determine the cumulative opacity value of the anchor point.

[0051] Specifically, each Gaussian is defined by its covariance matrix and center point, i.e. , Indicates the first A high-level unit; Indicated by An exponential function with base 0; Indicates the center point. , Representing 3D dimension; based on the depth of Gaussian elements at different locations, project them onto a 2D image plane, and calculate the color of the pixel corresponding to the Gaussian element on the plane. The corresponding calculation formula is:

[0052]

[0053] in, Indicates color; Indicates the first The nth Gaussian point, i.e. the nth Each Gaussian element corresponds to a pixel on the plane; The total number of Gaussian points in a ray; Let represent another Gaussian point, and ; Indicates the first The color of a Gaussian point; , They represent the first The Gaussian point and the first Opacity of one Gaussian point.

[0054] It can be explained that color refers to the color attribute presented by visual markers or symbols used to identify Gaussian points in a specific scientific or technical field, used to distinguish or indicate the location, nature, or state of Gaussian points; opacity refers to the probabilistic description of the opacity of the foreground region in an image using an anchor-based 3DGS compression method model, with a value range of [value range missing]. 0 represents completely transparent, and 1 represents completely opaque.

[0055] Next, according to the first Similarly, the opacity of all Gaussian elements is determined, and the Gaussian elements are clustered using anchor points. A Multi-Layer Perceptron (MLP) is used to infer Gaussian properties from the features of the anchor points, with each anchor point determined by its position. and anchor point properties Composition, among which, Characteristics of anchor points Indicates the scale of the anchor point. This indicates the offset of the anchor point; it should be noted that, in this embodiment, Referring to 50 dimensions, Represents dimension. Indicates the anchor point; The dimension representing the anchor point scale; This represents the number of offsets for a single anchor point in three-dimensional space. In actual operation, i.e., during rendering, the feature... The data is input into the MLP to generate the properties of the Gaussian function, and the position of the Gaussian function property is determined by the sum of the anchor point position and the anchor point offset.

[0056] Finally, the Gaussian opacity is transmitted in reverse to the anchor point to obtain the opacity of the anchor point, and the opacity value is continuously accumulated during the training process to determine the cumulative opacity value of the anchor point.

[0057] Step S212: Input the JND image into the rasterization module, obtain the JND importance score of the Gaussian pixel based on the intersection point of a single Gaussian pixel and the pixel in the JND image and the corresponding pixel value, propagate backward to the anchor point through offset to generate the JND score of the anchor point, and determine the corresponding cumulative value of the JND score.

[0058] Understandably, in JND image compression, its two-dimensional information can guide the compression of two-dimensional images. However, when extending the JND image from a two-dimensional image to a three-dimensional Gaussian model, challenges related to dimension adaptation and content dependency arise. Therefore, considering that the importance of an anchor point is determined by the importance of the Gaussian elements it generates, and the importance of Gaussian elements is directly affected by the JND score of the pixels associated with it, compression can be achieved by pruning low-importance anchor points simply by obtaining the importance of the Gaussian elements.

[0059] Specifically, the JND image corresponding to the current iteration training image, i.e., the JND image trained in step S211 above, is input into the rasterization module. Based on all pixel values ​​affected by each Gaussian pixel, the JND importance score of the corresponding Gaussian pixel is evaluated, denoted as . Use indicator functions Determine the Gaussian element Is it related to pixels? Intersection, meaning that in a single viewpoint image, every pixel emits a ray of light into the scene. If a Gaussian point in the scene is hit by this ray, it is considered that the Gaussian pixel intersects with this ray, and the pixel's JND value is included in the calculation of the Gaussian pixel's JND score; thus, the JND importance score of a single pixel is... , Represents pixels The JND value is used to determine the JND score of the Gaussian element, and the corresponding calculation formula is as follows:

[0060]

[0061] in, Indicates the first JND score of one Gaussian unit; Indicates the first 1 pixel; Indicates the number of pixels; Indicates the first JND Importance Score of Individual Gaussian Yuan With pixels Indicator functions; Represents pixels JND importance score.

[0062] Next, the JND score of the Gaussian unit is propagated backward to the anchor point through the offset, generating the JND score of the anchor point for a single iteration; similarly, the JND scores of the corresponding anchor points in all iterations are determined in the same way.

[0063] Specifically, during each iteration, the cumulative JND score is not updated by simple accumulation like the cumulative opacity score. The cumulative value is usually much larger than the JND score, and simple addition would reduce the impact of the JND score on the cumulative value in each iteration. Therefore, different weights are assigned to each JND score during the accumulation process. In actual operation, adjusting the weight distribution between the accumulated JND values ​​in the previous iterations and the newly obtained JND score will have a significant impact on the shearing effect. Generally, the newly obtained JND score has a larger weight.

[0064] Step S22: Obtain the initial rendered image of the dataset using the anchor-based 3DGS compression method model, and design a loss function by combining the dataset and the corresponding JND image to optimize the learning direction of the anchor attributes.

[0065] Understandably, JND represents the minimum difference that HVS can detect; the human visual system can only perceive differences exceeding the JND threshold, and such differences degrade visual quality; while differences below the JND threshold are imperceptible to HVS and do not affect perceptual quality. Therefore, in image compression technology, it is used as a threshold to quantify the visibility of differences, so as to propose a perceptual quality loss metric based on JND, i.e., design a loss function.

[0066] Further, step S22 includes:

[0067] Step S221: Perform pixel-by-pixel loss calculation on the images in the dataset and compare the pixel values ​​of the paired images with those of the initial rendered image.

[0068] Specifically, during the training process described above, the anchor-based 3DGS compression method model used in the training process performs pixel-by-pixel loss calculation on the images in the dataset. Firstly, based on the image... Output the initial rendered image, denoted as Compare the pixel values ​​of the image with those of the initial rendered image, i.e. , This represents the difference in pixel value between two pixels at the same location in the image and the initial rendered image. This represents absolute value operations, preventing values ​​less than 0 and avoiding meaningless pixel calculations.

[0069] Step S222: Set up a difference-aware modulator, combine the JND image, and design a loss function using an activation function in conjunction with pixel value differences to optimize the learning direction of anchor point attributes.

[0070] Specifically, a difference perception modulator is set up, denoted as... Its corresponding definition is expressed as:

[0071]

[0072] in, This represents a JND image generated using an anchor-based 3DGS compression method model.

[0073] It can be explained that when pixel values ​​differ When the value is less than or equal to the value corresponding to the JND image, it means that all differences in the image are within the JND tolerance range. Only pixels with errors significantly exceeding JND are penalized to avoid over-optimization of minor errors; while when pixel values ​​differ... When the image exceeds JND, that is This indicates that the JND image is scaled proportionally.

[0074] Next, in this embodiment, the activation function used is ReLU, which is a widely used activation function in the field of deep learning. It has advantages such as simple computation and mitigating the gradient vanishing problem. In image processing tasks, it can effectively enhance the model's ability to extract features. The loss function is designed in conjunction with the pixel value difference activation function, and the corresponding calculation formula is as follows:

[0075]

[0076] in, Represents the loss function; Indicates the activation function; This represents the difference in pixel value between two pixels at the same location in the image and the initial rendered image. This indicates a difference-perceived modulator.

[0077] It can be explained that in the loss function, only pixels that significantly exceed the current average error level are penalized, which optimizes the learning direction of anchor point attributes, making the rendered image more in line with the quality of human visual perception. The loss function is replaced with the L1 loss, which penalizes all pixel deviations equally, regardless of whether they are perceived by the human eye. This contradicts the principle of JND image correspondence. Therefore, the anchor-based 3DGS compression method model used in this embodiment is replaced with a loss function designed according to JND image to better match the quality of human visual perception.

[0078] It should be further explained that, in practical applications, different weights are assigned to the JND loss (i.e., the loss function in step S22) and the SSIM (Structural Similarity Index Measure) loss (i.e., the loss function in the anchor-based 3DGS compression method model) at different training stages to measure the visual similarity between the compressed model and the original model. Once the number of anchors stabilizes in the later training stages, optimization towards human perceptual quality becomes more critical. Therefore, after a certain number of iterations, preferably 15,000 times in this embodiment, the weight of the JND loss is increased.

[0079] Furthermore, in step S3, the anchor point is pruned based on a combination of opacity and cumulative JND score, specifically as follows:

[0080] A preset judgment threshold is set, and the judgment threshold is compared with the opacity and the cumulative JND score. If both are less than the threshold, the corresponding anchor point is pruned.

[0081] Specifically, for anchor points with sufficient observation counts, if both the opacity and the cumulative JND score are less than the judgment threshold, it is considered that the area information reflected by the anchor point is more blurred and has less detail. At the same time, the quality changes in the area are not easily detected, i.e., it has low perceived quality importance, so it is pruned. Conversely, if either the opacity or the cumulative JND score is not less than the judgment threshold, or both are greater than and equal to the judgment threshold, it indicates that the area information reflected by the anchor point has high information density or human eye sensitivity, has a certain perceived quality, and can be retained to ensure the accuracy of compression.

[0082] As explained, in step S4, the trained anchor-based 3DGS compression method model is... Output rendered images, that is, for the input dataset, output the rendered image corresponding to each image in the dataset. It is a commonly used image compositing technique that can precisely control the superposition effect of pixels at different levels based on the opacity of Gaussian points, generating more realistic rendering effects.

[0083] Understandably, compression is achieved based on Just Noticeable Difference (JND) without affecting human visual perception. JND represents the minimum difference detectable by the human visual system (HVS). By acquiring JND images, the differences in perceptual sensitivity of the human eye to various regions can be reflected. Training a model based on anchor points in a 3DGS compression method model can identify information redundancy during the compression process and safely remove it without being noticed, achieving efficient compression while maintaining visual quality. Finally, through... Outputting rendered images, that is, effectively outputting high-quality rendered images, significantly reducing the amount of data while maintaining the indistinguishability of human visual perception, provides an effective technical solution for the storage and transmission of 3D visual data.

[0084] To better illustrate and verify the feasibility of the proposed 3D Gaussian compression method based on just-perceptible distortion, validation was performed using four datasets: MipNeRF360, BungeeNeRF, Tanks & Temples, and DeepBlending. MipNeRF360 contains high-quality multi-view image sequences of 10 indoor scenes, each providing 360-degree panoramic view data with high resolution and diverse lighting conditions. BungeeNeRF includes trampoline motion sequence images taken from different perspectives and lighting conditions, covering different stages of trampoline movement such as take-off, aerial posture, and landing, as well as shooting perspectives from different angles such as front, side, and top views. Tanks & Temples, i.e., Tanks and... The TemplesDataset contains 12 indoor scenes with complex geometries, such as tanks, temples, and rooms. These scenes feature numerous transparent objects, specular reflections, and fine textures. Temples refers to complex indoor scenes with historical architectural styles, such as temple-like structures with arches, columns, and intricate carvings. The DeepBlending dataset contains multiple image pairs with occlusion relationships, covering both indoor and outdoor environments, and includes precise annotations of occlusion boundaries and semantic segmentation information. The dataset is compared with the state-of-the-art HAC++ method, as shown in Table 1, which presents the compression results. JND-GS is the anchor-based 3DGS compression method model proposed in this invention.

[0085] Table 1 Comparison of Compression Results

[0086]

[0087] It can be noted that the JND-GS model achieved an average compression rate of 44.89% on the three datasets. Furthermore, incorporating JND Maps during compression helps maintain a high level of perceptual quality in the rendered images. Subjective quality assessments were conducted on the four datasets, and five images with IDs 00000, 00003, 00007, 00011, and 00015 were selected for use in each scene of the HAC++ and JND-GS models in the 2AFC experiment. Based on the judgment of 13 participants, the JND-GS model reduced storage space by an average of 47.4%, and 52.23% of participants preferred the JND-GS rendering results.

[0088] The second embodiment of the present invention provides a three-dimensional Gaussian compression system based on just-perceived distortion. The system includes a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory communicate with each other through the communication bus. The processor calls logical instructions in the memory to execute a three-dimensional Gaussian compression method based on just-perceived distortion as described in any embodiment of the present invention.

[0089] When it is in operation, it needs to use a three-dimensional Gaussian compression method based on just-perceptible distortion. Therefore, whether the system and program data are integrated or different hardware is configured to produce a function with similar effect to that achieved by the present invention, it is within the protection scope of the present invention. The system has the same beneficial effect as the aforementioned three-dimensional Gaussian compression method based on just-perceptible distortion, and will not be described in detail here.

[0090] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0091] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A three-dimensional Gaussian compression method based on just-perceived distortion, characterized in that, The method includes: Collect images and integrate them to create a dataset. Input the dataset into the JND model to obtain the corresponding JND image. The dataset and corresponding JND images are input into the anchor-based 3DGS compression method model for training. The opacity of the anchor points and the cumulative JND score are obtained in sequence. A loss function is designed to optimize the anchor point properties. The anchor point should be pruned based on a combination of opacity and cumulative JND score, thus completing the training of the anchor point-based 3DGS compression method model. The trained anchor-based 3DGS compression method model is passed through Output rendered image.

2. The three-dimensional Gaussian compression method based on just-perceived distortion according to claim 1, characterized in that, The anchor point attributes include features, scale, and offset.

3. The three-dimensional Gaussian compression method based on just-perceived distortion according to claim 2, characterized in that, In the anchor-based 3DGS compression method model, minimum perceptible information is introduced to optimize the compression strategy and parameter training.

4. The three-dimensional Gaussian compression method based on just-perceived distortion according to claim 3, characterized in that, The dataset and corresponding JND images are input into the anchor-based 3DGS compression model for training. The opacity of the anchor points and the cumulative JND score are obtained sequentially. A loss function is designed to optimize anchor point properties, including: The JND image is converted into Gaussian elements for analysis based on the anchor-based 3DGS compression method model. The Gaussian opacity of the Gaussian elements and the JND importance score are integrated to determine the opacity of the anchor points and the cumulative value of the JND score. The initial rendered images of the dataset are obtained by using an anchor-based 3DGS compression method model. A loss function is designed by combining the dataset and the corresponding JND images to optimize the learning direction of anchor attributes.

5. The three-dimensional Gaussian compression method based on just-perceived distortion according to claim 4, characterized in that, The Gaussian opacity and JND importance score of the Gaussian unit are integrated to determine the opacity and cumulative JND score of the anchor point, including: Evaluate the Gaussian opacity of each Gaussian pixel corresponding to the JND image and backpropagate it to the anchor point to determine the cumulative opacity value of the anchor point; The JND image is input into the rasterization module. The JND importance score of the Gaussian pixel is obtained based on the intersection point of the individual Gaussian pixel and the pixel in the JND image and the corresponding pixel value. The score is then propagated backward to the anchor point through offset to generate the JND score of the anchor point. The corresponding cumulative JND score value is then determined.

6. The three-dimensional Gaussian compression method based on just-perceived distortion according to claim 4, characterized in that, The initial rendered images of the dataset are obtained through an anchor-based 3DGS compression method. A loss function is designed based on the dataset and the corresponding JND images to optimize the learning direction of anchor attributes, including: Perform pixel-by-pixel loss calculation on the images in the dataset and compare the pixel values ​​of the paired images with those of the initial rendered images; A difference-aware modulator is set up, and a loss function is designed by combining the JND image with the pixel value difference and using an activation function to optimize the learning direction of the anchor attribute.

7. The three-dimensional Gaussian compression method based on just-perceived distortion according to claim 1, characterized in that, The decision to prune an anchor point is based on a combination of opacity and cumulative JND score. A preset judgment threshold is set, and the judgment threshold is compared with the opacity and the cumulative value of JND score. If both are less than the threshold, the corresponding anchor point is pruned.

8. A three-dimensional Gaussian compression system based on just-perceived distortion, characterized in that, The system includes a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory communicate with each other through the communication bus. The processor calls logical instructions in the memory to execute the three-dimensional Gaussian compression method based on just-perceptible distortion as described in any one of claims 1 to 7.

Images