Gaussian sputtering model editing method and system

Abstract

The application relates to the field of computer vision, in particular to a Gaussian sputtering model editing method and system, which comprises the following steps: obtaining three-dimensional Gaussian scene data and user-specified handle points and target points; generating a dynamic three-dimensional soft mask combined with geometric proximity weight and semantic perception weight to accurately define an editing area; generating a multi-modal guide field by fusing explicit physical heuristic deformation and local implicit three-plane completion to provide guide signals of geometric rationality and detail authenticity; adopting progressive layered optimization to decompose total editing amount into multiple steps, applying elastic continuity constraints in geometry and appearance optimization to realize long-distance stable editing; embedding near real-time preview in the optimization process to provide interactive feedback; and the application solves the problems of low editing precision, unstable long-distance editing, rough area control and lack of interactive feedback in the prior art through multi-modal guidance and progressive optimization.

Description

Technical Field

[0001] This application relates to the field of computer vision, specifically to a method and system for editing Gaussian sputtering models. Background Technology

[0002] In recent years, scene editing based on 3D Gaussian Splatting (3DGS) has become a research hotspot due to its advantages in explicit representation and real-time rendering. In direct interaction-driven editing, drag-and-drop control point-based editing is considered a key interactive paradigm for next-generation 3D content creation tools due to its high intuitiveness. Currently, the most representative technical solutions in this direction mainly include strong generative weak geometric control (DYG, DragYourGaussian) and strong geometric control weak detail generation (3DGS-Drag). The DYG method was the first to introduce the drag-and-drop editing paradigm from 2D images into 3DGS. Its core lies in introducing an implicit three-plane representation as a geometric "scaffolding" and using fractional distillation sampling (SDS) loss to drive Gaussian units to align with this geometric structure. However, this method relies entirely on the generative prior of the diffusion model, resulting in a slow optimization process. Furthermore, in long-distance dragging or complex structure editing, it is prone to content deviation or convergence difficulties due to the uncertainty of the generative prior. The 3DGS-Drag method proposes a two-stage framework of deformation guidance and diffusion guidance. First, a simple "copy-paste" approximate deformation is performed, and then a fine-tuned diffusion model is used for image correction.

[0003] However, the existing drag-and-drop based 3DGS editing technology has the following problems: (1) The guidance mechanism is unbalanced, the deformation guidance is too rough, and geometric tearing is easy to occur in complex areas such as joints and connections; its diffusion guidance relies heavily on the model that finely adjusts the original three-dimensional Gaussian scene, and it is easy to fail when the editing range exceeds the model's understanding range, making it difficult to balance geometric rationality and detail realism; (2) The stability and robustness of long-distance or complex structure editing are poor; (3) The editing area is not finely controlled, which can easily lead to editing overflow or hard boundaries. Summary of the Invention

[0004] To address the problems of existing technologies, this invention provides a Gaussian sputtering model editing method, comprising the following steps: S100: Acquire 3D Gaussian scene data and receive at least one set of handle points and target points specified by the user in the 3D Gaussian scene, and generate multi-view rendering images based on the 3D Gaussian scene data; S200: Based on the handle point, a dynamic three-dimensional soft mask is generated by combining geometric proximity weights and semantic awareness weights. The dynamic three-dimensional soft mask is used to define the region to be edited and indicate the degree to which each Gaussian element in the region is affected by editing. S300: Based on the handle point, the target point, and the dynamic 3D soft mask, a multimodal guide field is generated that integrates explicit physical-inspired deformation and local implicit three-plane completion. The multimodal guide field is used to provide geometric and appearance guidance signals required for editing. S400: Decompose the total editing volume into multiple sub-steps. In each sub-step, based on the dynamically updated multimodal guiding field, perform hierarchical geometric optimization and appearance optimization on the three-dimensional Gaussian scene data. Apply elastic continuity constraints in the geometric optimization sub-step of each sub-step and iteratively update the Gaussian meta-parameters within the region to be edited. S500: Performs denoising and densification post-processing on the optimized 3D Gaussian scene data to obtain the edited 3D Gaussian scene.

[0005] Further, step S200 specifically includes: for each Gaussian element, calculating its geometric proximity weight with the handle point, the geometric proximity weight being determined based on Euclidean distance and adaptive influence radius; the adaptive influence radius being calculated by statistically analyzing the average distance of the K nearest neighboring Gaussian elements around the handle point; A lightweight semantic segmentation network is used to extract the target editing semantic category corresponding to the handle point from the multi-view rendered image. 2D semantic labels are obtained through the lightweight segmentation network, and then fused with 3D back projection and multi-view confidence voting. Combined with semantic consistency discrimination and smoothing normalization, the semantic perception weights corresponding to each Gaussian unit are obtained. The geometric proximity weights are fused with the semantic perception weights and normalized to generate the dynamic three-dimensional soft mask.

[0006] Furthermore, generating the multimodal guiding field in step S300 specifically includes: Explicit Physics-Inspired Deformation: Based on Gaussian elements within the region to be edited indicated by a dynamic 3D soft mask, and combined with curvature adaptation weights, normalization factors, and distance decay weights of the local curvature, the smooth displacement vector of the Gaussian elements is calculated to obtain a set of deformed Gaussian elements without tearing or penetration, which is used to provide the geometric guidance signals required for editing. Local implicit three-plane complete field construction: Axially aligned bounding boxes are determined with Gaussian elements whose mask values ​​are greater than a predetermined threshold in the deformed Gaussian elements as the center, and a three-plane feature field is initialized within the bounding box; Joint optimization is performed using the geometric supervision provided by the deformed Gaussian elements and the appearance supervision provided by the original 3D Gaussian scene to obtain a local implicit three-plane complete field that combines geometric accuracy and appearance realism, which is used to provide guidance signals for appearance details required for editing; Guide image synthesis: The first guide image and the second guide image are obtained from the deformed Gaussian primitive and the local implicit three-plane full-field rendering, respectively. The dynamic fusion coefficient is calculated according to the drag distance and the region complexity. The first guide image and the second guide image are fused to generate the final guide image.

[0007] Furthermore, the hierarchical optimization in step S400 specifically includes: in each sub-step, first performing the geometry optimization sub-step, and then performing the appearance optimization sub-step; Geometric optimization substep: With the color parameters of the Gaussian elements fixed, the geometric parameters of the Gaussian elements are optimized using geometric loss, with the guiding image as the target. Appearance optimization sub-step: With fixed geometric parameters, the color parameters of Gaussian elements are optimized through appearance loss using a dynamic image buffer pool and a lightweight diffusion model to generate a high-quality appearance target image.

[0008] Furthermore, the elastic continuity constraint in step S400 is implemented through an elastic regularization term, which penalizes the abrupt shift vector of the Gaussian element position between the current optimization step and the previous optimization step, so as to ensure the smoothness of motion between multi-step optimizations.

[0009] Furthermore, the method also includes embedding a near real-time preview step during the iteration process of step S400: after each iteration, a preview image of the current 3D Gaussian scene is quickly rendered with a computational overhead lower than the final rendering quality, and sent to the user terminal for display in real time through a communication interface.

[0010] A Gaussian sputtering model editing system, the system being used to execute a Gaussian sputtering model editing method, comprising: An interactive client that provides a graphical user interface, captures user-specified handle points and target points, and displays preview images and editing results during the editing process; A computing server, communicating with the interactive client, includes: The scene management module is used to load, store, and update 3D Gaussian scene data; The mask generation module is used to execute step S200 to generate the dynamic three-dimensional soft mask; The guidance generation module is used to execute step S300 to generate the multimodal guidance field; The incremental optimization engine is used to perform step S400, which performs hierarchical incremental optimization of the Gaussian meta-parameters. The preview rendering module is used to perform the near real-time preview step.

[0011] An electronic device, comprising: one or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the Gaussian sputtering model editing method.

[0012] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the Gaussian sputtering model editing method.

[0013] The beneficial effects that this application can produce include: (1) This application improves the geometric rationality, detail realism and optimization efficiency of editing by integrating explicit physical-inspired deformation and local implicit three-plane completion multimodal guiding field; (2) This application enhances the stability and robustness of long-distance and complex structure editing through perceptually enhanced progressive hierarchical optimization and elastic continuity constraints; (3) This application achieves pixel-level precise local editing by combining geometric proximity and semantic awareness dynamic three-dimensional soft mask, effectively protecting non-target areas; (4) By embedding a near real-time preview function for the optimization loop, the optimization process is made transparent, which improves the user interaction experience and creation efficiency. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the user interface in an embodiment of the present invention; Figure 2 This is a flowchart of the core data processing flow of the backend system in an embodiment of the present invention; Figure 3 This is a client-server architecture diagram in an embodiment of the present invention; Figure 4 This is a user operation flowchart in an embodiment of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Please see Figure 2 This invention provides a method for editing a Gaussian sputtering model, comprising the following steps: S100 Initialization and Input Reception: Acquire 3D Gaussian scene data and receive at least one set of handle points and target points specified by the user in the 3D Gaussian scene; Generate multi-view rendering images based on the 3D Gaussian scene data, which are used for subsequent semantic perception and mask generation; Using the 3D Gaussian scene data as the rendering object, perform rendering operations from multiple preset perspectives to obtain the corresponding perspective rendering (RGB) images; Specifically, the background engine first loads and constructs complete 3D Gaussian scene data, which contains several Gaussian primitives, each Gaussian primitive having position parameters, rotation quaternions, etc. The scaling vector, spherical harmonic coefficients, and opacity are used. A 3D Gaussian scene data is used as the basis for scene representation and the data source for rendering. Using this 3D Gaussian scene data as input, multiple preset camera pose parameters are employed. Through differentiable rasterization rendering, projection calculations are performed on the 3D Gaussian scene from multiple different viewing angles to generate corresponding RGB rendered images. The generation of these multi-view RGB images depends on and originates from the loaded 3D Gaussian scene data; the two have a subordinate execution relationship between the data source and the rendering output. Furthermore, the multi-view RGB images serve as input for subsequent semantic segmentation, mask generation, and guided field construction. The specific logical process of generating multi-view RGB images using differentiable rasterization rendering: The background engine preloads and constructs a 3D Gaussian scene representation consisting of several 3D Gaussian primitives, and configures the corresponding camera intrinsics, camera extrinsics and camera pose matrix for each observation view. Based on the camera parameters of the current viewpoint, the three-dimensional Gaussian primitives in the three-dimensional space are sequentially projected onto the two-dimensional image plane of the current viewpoint through the differentiable rasterization rendering pipeline, so as to obtain the projection area, projection shape and projection weight of each Gaussian primitive on the two-dimensional image plane. Based on the position, covariance, rotation, scaling, opacity and spherical harmonic color attributes carried by each Gaussian element, according to the depth sorting from front to back and the α mixing and synthesis rule, the color accumulation and transparency mixing calculations are performed on all Gaussian elements projected to the same pixel position. Through the above differentiable projection and color synthesis operations, an RGB rendered image with realistic colors and geometric appearance is finally generated from the current observation perspective. Following the same logic, differentiable rasterization rendering and image synthesis are performed sequentially on multiple different observation perspectives to obtain multi-view RGB rendering images that correspond one-to-one with each perspective.

[0017] S200 Dynamic 3D Soft Mask Generation: Based on the handle point, combined with geometric proximity weights and semantic awareness weights, a dynamic 3D soft mask is generated. The dynamic 3D soft mask is used to define the area to be edited and indicate the degree to which each Gaussian element in the area is affected by editing. S300 Multimodal Guided Field Generation: Based on the handle point, the target point, and the dynamic 3D soft mask, a multimodal guided field is generated that integrates explicit physical-inspired deformation and local implicit three-plane completion. The multimodal guided field is used to provide geometric and appearance guidance signals required for editing. S400 Progressive Layered Optimization: The total amount of editing is decomposed into multiple sub-steps. The total number of steps K is specified by the user or is 10 by default. In each sub-step, based on the dynamically updated multimodal guiding field, the three-dimensional Gaussian scene data is subjected to layered geometric optimization and appearance optimization. Elastic continuity constraints are applied in the geometric optimization sub-step of each sub-step, and the Gaussian meta-parameters in the area to be edited are updated iteratively step by step. S500 Output and Post-processing: The optimized 3D Gaussian scene data is denoised and densified to obtain the edited 3D Gaussian scene.

[0018] Further, step S200 specifically includes: for each Gaussian element, calculating its geometric proximity weight with the handle point, the geometric proximity weight being determined based on Euclidean distance and adaptive influence radius; the adaptive influence radius being calculated by statistically analyzing the average distance of the K nearest neighboring Gaussian elements around the handle point; A lightweight semantic segmentation network is used to extract the target editing semantic category corresponding to the handle point from the multi-view rendered image. 2D semantic labels are obtained through the lightweight segmentation network, and then fused with 3D back projection and multi-view confidence voting. Combined with semantic consistency discrimination and smoothing normalization, the semantic perception weights corresponding to each Gaussian unit are obtained. The geometric proximity weights and the semantic perception weights are fused and normalized to generate the dynamic three-dimensional soft mask. Furthermore, generating the multimodal guiding field in step S300 specifically includes: Explicit Physics-Inspired Deformation: Based on Gaussian elements within the region to be edited indicated by a dynamic 3D soft mask, and combined with curvature adaptation weights, normalization factors, and distance decay weights of the local curvature, the smooth displacement vector of the Gaussian elements is calculated to obtain a set of deformed Gaussian elements without tearing or penetration, which is used to provide the geometric guidance signals required for editing. Local implicit three-plane full-field construction: An axially aligned bounding box is determined centered on Gaussian elements whose mask values ​​are greater than a predetermined threshold (e.g., 0.5) in the deformed Gaussian elements. A three-plane feature field is initialized within the bounding box. Joint optimization is performed using geometric supervision provided by the deformed Gaussian elements and appearance supervision provided by the original 3D Gaussian scene. The Adam optimizer is used for optimization, with a learning rate of 0.001, 100 iterations, and default loss weights. =1.0, =0.5, =0.2, =0.01, to obtain a local implicit three-plane full field that combines geometric accuracy and appearance realism, used to provide guidance signals for the appearance details required for editing; if there are local holes or slight defects in the deformed Gaussian elements, the nearest neighbor interpolation is used for simple filling to ensure the integrity of geometric supervision; Guide image synthesis: The first guide image and the second guide image are obtained from the deformed Gaussian primitive and the local implicit three-plane full-field rendering, respectively. The dynamic fusion coefficient is calculated according to the drag distance and the region complexity. The first guide image and the second guide image are fused to generate the final guide image.

[0019] Furthermore, the hierarchical optimization in step S400 specifically includes: in each sub-step, the geometry optimization sub-step is executed first, followed by the appearance optimization sub-step, with the two being executed alternately once; Geometric optimization substep: With the color parameters of the Gaussian elements fixed, the geometric parameters of the Gaussian elements are optimized using the guided image as the target. The optimization loss includes image reconstruction loss, perceptual loss, and depth smoothing constraint. Appearance optimization sub-step: With fixed geometric parameters, a high-quality appearance target image is generated using a dynamic image buffer pool and a lightweight diffusion model, and the color parameters of Gaussian elements are optimized under supervision using this target image.

[0020] Furthermore, the elastic continuity constraint in step S400 is implemented through an elastic regularization term, which penalizes the abrupt shift vector of the Gaussian element position between the current optimization step and the previous optimization step, so as to ensure the smoothness of motion between multi-step optimizations.

[0021] Furthermore, the method also includes embedding a near real-time preview step during the iteration process of step S400: every 100 iterations (user adjustable range 50-200 times), a preview image of the current 3D Gaussian scene is quickly rendered at 50% of the original resolution with simplified shaders (spherical harmonic lighting is disabled, and only primary colors are used), and sent to the user terminal for display in real time through the communication interface.

[0022] A Gaussian sputtering model editing system, the system being used to execute a Gaussian sputtering model editing method, comprising: An interactive client that provides a graphical user interface, captures user-specified handle points and target points, and displays preview images and editing results during the editing process; A computing server, communicating with the interactive client, includes: The scene management module is used to load, store, and update 3D Gaussian scene data; The mask generation module is used to execute step S200 to generate the dynamic three-dimensional soft mask; The guidance generation module is used to execute step S300 to generate the multimodal guidance field; The incremental optimization engine is used to perform step S400, which performs hierarchical incremental optimization of the Gaussian meta-parameters. The preview rendering module is used to perform the near real-time preview step.

[0023] An electronic device, comprising: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the Gaussian sputtering model editing method.

[0024] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the Gaussian sputtering model editing method.

[0025] The key steps are explained in detail below, combining the formula and specific parameters: The method and system provided by this invention can run on high-performance graphics workstations or servers, and users interact with them through a graphical user interface (GUI) on a terminal. The operating environment for this invention requires an NVIDIA A100 GPU (40GB or more of video memory) or equivalent computing power, an operating system of Ubuntu 20.04 or later, and at least 32GB of RAM. It is recommended that the client use a modern browser that supports WebGL. 1. Product Description like Figure 1 As shown, the user interface mainly includes a 3D scene view area, a control point editing toolbar, an editing parameter panel, a real-time preview and log area, and a file operation area. The 3D scene view area is the main display area, used to showcase the real-time rendering effect of the loaded 3D Gaussian scene from multiple angles. The handle point / target point editing toolbar provides buttons such as "Add Handle Point," "Add Target Point," "Clear Control Point," "Start Editing," and "Stop Editing." The editing parameter panel allows users to adjust advanced parameters, such as the asymptotic step size K and locality intensity (corresponding to mask parameters). , Preview frequency, etc. (usually provides default values); Real-time preview and log area: displays real-time feedback information such as preview image of the current optimization step, number of iterations, loss curve, etc.; File operation area: used to load .ply format 3D Gaussian splash (3DGS) models and save edited models; such as Figure 4As shown, here's an example of the user operation process: After loading a 3DGS model of a standing figure, the user clicks on the outside of the figure's right leg in the 3D view area to add a blue handle point; then clicks above the right leg to add a red target point. Clicking "Start Editing" initiates calculations in the system background. The user can see a low-resolution animation of the figure's right leg gradually closing in the "Live Preview Area." After editing, the final high-quality "legs together" figure model is displayed in the main view area.

[0026] 2. Technical Description like Figure 2 As shown, the core data processing flow of the backend system is as follows: Step S100: Initialization and input reception.

[0027] The background engine loads 3D Gaussian meta-scene data. , ,in For location, For rotation quaternions, For scaling vectors, The spherical harmonic coefficients, Opacity. Receives N sets of handle points from the front end. and target point data.

[0028] Step S200: Dynamic 3D soft mask generation.

[0029] To address the issue of imprecise control over the editing area, this step generates a soft mask. .

[0030] Geometric neighbor components For each handle point Calculate its distance to all Gaussian metacenters Euclidean distance The influence radius r is adaptively determined based on the local Gaussian element density (r is smaller for higher density), and the geometric proximity weight is calculated. : The adaptive influence radius is calculated by averaging the distances of the K nearest neighbor Gaussian elements around the handle point. The specific process is as follows: For each handle point specified by the user, the backend engine retrieves the K nearest Gaussian elements from the loaded 3D Gaussian scene data based on spatial proximity. The 3D Euclidean distances from the handle point to the centers of the K nearest neighbor Gaussian elements are calculated, and the arithmetic mean of the K distance values ​​is taken as the adaptive influence radius r corresponding to the current handle point. Here, K is a preset positive integer (e.g., 32). The adaptive influence radius r is adaptively adjusted according to the local Gaussian distribution density; the denser the Gaussian distribution, the smaller the influence radius, and the sparser the Gaussian distribution, the larger the influence radius, thus achieving adaptive and precise definition of the editing influence range. Semantic perceptual components The backend engine generates multi-view rendering (RGB) images based on 3D Gaussian scene rendering and simultaneously saves the camera intrinsic parameters, extrinsic parameters, and inverse projection matrix corresponding to each observation view. The RGB images from each view are input into a pre-trained lightweight semantic segmentation network (e.g., MobileNet as the backbone and DeepLabV3Lite as the segmentation head). The user interaction handle points are projected forward onto the 2D image plane, and the semantic category labels of the corresponding pixels (e.g., legs) are extracted as the target editing semantic category. Based on the camera inverse projection matrix, the 2D semantic labels are projected backward onto the 3D world space, establishing a mapping relationship between the 2D semantic labels and the 3D spatial position, and constructing a 3D semantic voxel constraint domain corresponding to the target category. A KD-Tree space acceleration index structure is pre-constructed for Gaussian primitives across the entire scene. A fast 3D range retrieval is performed through the acceleration structure (e.g., KD-Tree), and semantic-aware weights are assigned to Gaussian primitives that are retrieved and located within the semantic voxel constraint domain. The semantic perception weights of Gaussian elements outside the semantic voxels are set to 0, quickly completing the assignment of semantic perception component weights. The specific logic for obtaining the semantic perception weights is as follows: extract the 3D coordinates of the handle point input by the user's drag-and-drop interaction; project the handle point forward onto the 2D image plane of each viewpoint; query the semantic category corresponding to the handle point pixels to determine the target editing semantic category; for each observation viewpoint, perform 3D back projection using the camera inverse projection matrix to project the 2D semantic labels backward along the line of sight into 3D space, establishing a mapping relationship between the 2D semantic labels and their 3D spatial positions; and perform 3D back projection on a single 3D... The Gaussian element summarizes the semantic confidence scores of the Gaussian element from all visible perspectives, performs multi-perspective confidence score voting fusion, and obtains the original semantic confidence score of the Gaussian element. Semantic consistency discrimination is then performed: it is determined whether the semantic category of the Gaussian element is consistent with the target editing semantic category. If they are inconsistent, the original semantic confidence score is attenuated and suppressed, with the weight approaching 0; if they are consistent, the original fused confidence score is retained. Spatial smoothing normalization is then performed on the discriminated semantic confidence scores to suppress discrete noise and hard abrupt changes at edges, ultimately obtaining the semantic perception weight corresponding to the i-th Gaussian element, normalized to the [0, 1] interval and smoothly continuous. ; Fusion and Normalization: For each Gaussian element i, the geometric proximity weights and the semantic awareness weights are fused and normalized, and the final mask value is... and normalize to [0, 1]; and These are the geometric proximity weight coefficient and the semantic awareness weight coefficient, respectively, by default. =0.7, =0.3. The higher the value, the more the Gaussian element is affected by editing in subsequent optimizations; Step S300: Generation of multimodal guiding field.

[0031] To address the issue of inaccurate guidance mechanisms, this step generates a high-quality initial guidance signal for the first round of optimization.

[0032] Explicit Physics-Inspired Deformation: Dynamic 3D Soft Masking The Gausky element is deformed, and the specific process is as follows: (1) Obtain all Gaussian elements within the area to be edited indicated by the dynamic 3D soft mask, as well as at least one set of handle points and target points specified by the user; (2) For each Gaussian element, calculate the local curvature estimate at its location. (Calculated by the change of the neighboring Gaussian element normal vector), and through the curvature adaptation function. Generate curvature adaptation weights, with the curvature adaptation weights in high curvature regions being smaller than those in low curvature regions; (3) For each set of handle points Determine the local Gaussian neighborhood of the handle point. And calculate the weighted sum of the mask values ​​of all Gaussian elements in the neighborhood and the curvature adaptation weights. , as a normalization factor; (4) Calculate the Euclidean distance between the current Gaussian element and the handle point. And through Gaussian kernel function Generate distance decay weights; (5) Calculate the displacement vector of the current Gaussian element based on the normalization factor, the distance decay weight, and the curvature adaptation weight. The specific formula is as follows: (1); in Let be the displacement vector of the i-th Gaussian element. Let j be the control point of the j-th group. The 3D soft mask value of the i-th Gaussian element. Let j be the local Gaussian neighborhood of the j-th handle point. For the k-th Gaussian element in the neighborhood, The L2 norm (spatial distance); N is the number of handle point / target point pairs. The local curvature estimate of the location of Gaussian element i (calculated by the change of the normal vector of neighboring Gaussian elements); Let curvature be the adaptive function. The curvature suppression coefficient means that areas with high curvature (such as fingertips and clothing folds) deform less to preserve features; This represents the local neighborhood of the j-th handle point. The mask of all Gaussian elements is summed with the curvature adaptation value for normalization, so that the total displacement vector level is equal to the drag vector level. Matching; index term To weight the Gaussian elements based on distance, we make the Gaussian elements closer to the handle point move more, resulting in a pre-deformed set of Gaussian elements. .in The Gaussian kernel bandwidth (radius of influence); The j-th drag endpoint selected by the user. For the j-th handle point selected by the user, Let i be the center coordinates of the i-th 3D Gaussian element. The 3D spatial distance from the current Gaussian primitive point to the handle point; (6) Translate all Gaussian elements according to the calculated displacement vector to obtain the deformed Gaussian element set. ; Explicit physical-inspired deformation is employed, using dynamic 3D soft masks to determine the Gaussian subset to be edited. A physically constrained displacement field is constructed, with the handle point as the deformation anchor and the target point as the deformation endpoint. For each Gaussian element to be edited, the spatial distance attenuation weight between it and the handle point, the drag vector, and the normalization factor are calculated to determine the displacement vector of that Gaussian element. The displacement vector is positively correlated with the total drag vector and negatively correlated with the spatial distance. Curvature suppression constraints are applied to high-curvature regions, ensuring that the Gaussian elements deform collaboratively according to smooth physical laws, resulting in a pre-deformed Gaussian set that guarantees the geometric structure of the editing area is continuous, without tearing or penetration. 2. Construction of the Local Implicit Three-Plane Completion Full Field: Based on Gaussian Metasets middle A Gaussian element is used as the center to define an axially aligned bounding box (AABB), and a three-plane feature field is initialized within this AABB. (Three feature planes, resolution such as 64x64) A small multilayer perceptron (MLP) encoder E is used to extract appearance features from the multi-view rendering features of the original 3D Gaussian scene G. The geometric supervision provided by the deformed Gaussian primitives and the appearance supervision provided by the original 3D Gaussian scene are used to jointly optimize the three-plane feature field, resulting in a locally implicit three-plane complete field that combines geometric accuracy and appearance realism. This field is used to provide guidance signals for the appearance details required for editing. If there are local holes or slight defects in the deformed Gaussian primitives, nearest neighbor interpolation is used for simple filling to ensure the integrity of the geometric supervision. The three-plane feature field is quickly optimized by minimizing the color reconstruction loss, deformation depth alignment loss, original depth regularization loss, and feature plane smoothing regularization loss. (Approximately 100 iterations), Three-plane feature field reconstruction loss for: (2); in To obtain a rendering color map for a three-plane feature field. This is the true color map of the original 3D Gaussian scene. To optimize the three-plane feature field Rendering yields a second depth map. A first depth map is obtained from the deformed Gaussian set using differentiable rendering. The original depth map is obtained by rendering the original 3D Gaussian scene; The loss weights for color reconstruction force the appearance to conform to the original 3D Gaussian scene; Weights for depth geometry loss are applied, forcing the geometry to conform to the deformed shape. By drawing closer, it "injects" the geometric information of explicit deformation into the implicit field; The depth-regularized loss weights prevent the geometry from deviating excessively from the original shape, thus playing a stabilizing role. R(·) is the regularization weight, which is the total variation (TV) regularization used to smooth the feature plane; For the regularization term of the three-plane field, It is a three-plane characteristic field. L1 norm; It should be noted that geometric supervision uses the deformed Gaussian set as the geometric ground truth, and obtains the first depth map from the deformed Gaussian set through differentiable rendering. Simultaneously, from the three-plane feature field to be optimized Rendering to obtain a second depth map The difference between the first depth map and the second depth map is calculated to form the depth geometric loss term. It is used to inject the geometric deformation information generated by explicit physical-inspired deformation into a three-plane feature field, so that the geometric structure of the three-plane feature field is aligned with the deformed Gaussian element geometry. Appearance supervision uses the original 3D Gaussian scene G as the appearance ground truth, and renders a true color map from the original 3D Gaussian scene. Simultaneously, a rendering color map is obtained from the three-plane feature field rendering. The difference between the real color map and the rendered color map is calculated to constitute the color reconstruction loss term. This is used to force the three-plane feature field to learn the real color, texture and lighting information carried by the Gaussian elements of the original scene, ensuring that the appearance of the completed area is consistent with the original scene. Depth regularization supervision obtains the original depth map from the original 3D Gaussian scene rendering. Calculate its relationship with the second depth map The difference between them constitutes the depth regularization loss term. Regularization constraints are used to constrain the geometry of a three-plane characteristic field so that it does not deviate excessively from its original shape, thus maintaining geometric stability. Apply total variation regularization It is used to smooth feature planes and suppress noise and high-frequency artifacts; Through the joint optimization of geometric supervision and appearance supervision, the local implicit three-plane completion of the whole field can simultaneously achieve: the structural rationality of conforming to the geometric skeleton of the deformed Gaussian element, and the appearance realism consistent with the original scene, thereby generating a multimodal guiding signal that has both geometric fidelity and detail realism. 3. Guided image synthesis: From The rendering yields a guide color map and a guide depth map, which serve as the second guide image. Rendering yields a deformed color image And, as the first guiding image, the dynamic fusion coefficient γ is calculated: (3); in Average drag distance (after normalization); The variance of curvature within the editing region is used to measure structural complexity. , These are the drag distance weighting coefficient and the region complexity weighting coefficient, respectively; the larger the drag distance, the more complex the region. sigmoid(·) is the activation function. for , γ represents the local curvature of the editing region; γ is the explicit / implicit guidance fusion coefficient. The closer γ is to 1, the more dependent it is on implicit field guidance. Final synthesized guide image:

[0033] in This is the initial multimodal guide image. To implicitly complete the weights, This is a three-plane field implicit guidance image. For explicit deformation weights, Rendering images using explicit physics-inspired deformations; Step S400: Progressive hierarchical optimization.

[0034] To address the stability issue of long-distance editing, the total editing volume ΔP is divided into K progressive sub-optimization steps. For the k-th step (k=1 to K): 1. Dynamically guide and update the multimodal guidance field: calculate the target point of the current step. ;by With the goal of, The geometric parts (feature channels affecting depth) are rapidly fine-tuned (approximately 5-10 iterations), and the current step's guiding image is re-rendered. and .

[0035] 2. Layered optimization: a) Geometric optimization substep: Fix the color parameters of the Gaussian elements. Optimize geometric parameters , Using the guiding image as the target, the geometric parameters of the Gaussian elements are optimized through geometric loss, which includes two-dimensional mask weighting and L1 optimization. , The weighted average of perceptual loss and deep gradient smoothing constraint is expressed as: (5); in Let k be the geometric loss at step k, and k be the number of asymptotic optimization steps. yes A 2D projection mask projected onto the current viewpoint, used for region weighting; | D| is the absolute value of the gradient of the rendered depth map D, which serves as a smoothing constraint; For L1 loss weights, for Perceived loss weights For depth gradient smoothing constraints; Render the current Gaussian image. For the k-th step multimodal guidance image, The loss is a pixel traversal of the image; the loss uses the current step multimodal guided image as geometric supervision, fixes the appearance parameters and only optimizes the geometric properties of Gaussian elements in the editing region, constrains the smooth deformation of Gaussian spatial position and shape, eliminates geometric tearing, penetration and distortion, and ensures the geometric rationality and stability of the editing process.

[0036] b) Appearance optimization sub-step: Fix geometric parameters and optimize color parameters. Utilizing dynamic image buffer (Capacity 16, First-In-First-Out) This includes multi-view images of the original 3D Gaussian scene. As optimization progresses, the high-quality image rendered after the (k-1)th optimization step is stored. A lightweight low-rank adaptation (LoRA) module is used to fine-tune pre-trained diffusion models (such as SDXL, StableDiffusionXL). The fine-tuned model is based on the coarse image rendered by the current geometry. With text prompts (such as "Keep texture consistent"), generate high-quality target images. The appearance loss consists of region-weighted L2 loss and perception loss. The expression is: (6); in for , Weights for perceived loss; For the appearance target image at step k, The square of the L2 norm; Render the image using the current Gaussian rendering method; Color loss, To perceive loss; 3. Elastic Continuity Constraint: At the beginning of each geometric optimization step, an elastic regularization term is added to penalize abrupt shifts in the displacement vector. Loss function: (7); in For the weight of the elastic regularization term, Displacement penalty threshold (0.05); Let i be the center coordinates of the Gaussian element at step k. Let i be the center coordinates of the Gaussian element i at step k-1, exp( The term ) represents the exponential penalty term; this term imposes a stronger penalty on large, discontinuous displacements, ensuring a smooth editing trajectory. The elastic constraint loss is used to constrain the relative spatial position, displacement difference, and spacing changes between adjacent Gaussian elements within the dynamic 3D soft mask filtering range; by penalizing unnatural stretching, abnormal compression, abrupt displacement changes, and isolated drift behaviors between adjacent Gaussians, it simulates the elastic continuous deformation characteristics of solid objects, enabling the Gaussian elements in the editing area to undergo coordinated and smooth deformation as a whole, rather than independent and chaotic motion; Near real-time preview: Every 100 iterations in step S400, the current 3DGS state is quickly rendered at 50% of the original resolution with simplified shaders (spherical harmonic lighting disabled, only primary colors used) to generate a preview image. And push updates to the front-end interface in real time via WebSocket. Step S500: Output and Post-processing.

[0037] After all K steps are completed, the optimized 3D Gaussian scene data is post-processed with denoising and densification to obtain the edited 3D Gaussian scene, which is then output. Specifically: 1. Noise Reduction: Remove (like These are Gaussian elements, which are typically generated through optimization.

[0038] 2. Densification: Calculate the color gradient within the editing area. For gradients greater than a threshold and The Gaussian primitive is split into two sub-Gaussian primitives: the center of each sub-primitive is offset by ±0.01× the current scaling vector, the scaling vector is halved, the opacity is split evenly, and the color is inherited from the original primitive to enhance visual details.

[0039] 3. Output the final set of Gaussian elements. Save it as a .ply file.

[0040] like Figure 3As shown, this system adopts a client-server architecture. The interactive client is responsible for the GUI and user input capture. The computing server includes a scene management module, a mask generation module, a guided generation module, a progressive optimization engine, a preview rendering module, and a communication interface. The communication interface allows user commands to flow from the client to the server via a communication protocol (WebSocket) or the gRPC remote procedure call framework (gRPC, gRPCRemoteProcedureCall). The server returns the preview stream and final model data to the client. The data flow is the flow of user commands from the client to the server. The optimization status on the server side flows to the client through the preview rendering module. The final model data flows from the server to the client for storage.

[0041] It is worth noting that (1) the existing DYG method relies entirely on the prior of the diffusion model, which is prone to content deviation; the deformation guidance of the 3DGS-Drag method is too coarse, which is prone to geometric tearing in complex areas such as joints and connections. Its diffusion guidance also relies heavily on the model that fine-tunes the original scene, and it is prone to failure when the editing range exceeds the model's understanding range. The multimodal guidance field proposed in this invention, which integrates explicit physical-inspired deformation and local implicit three-plane completion, solves the problem of unbalanced guidance mechanism from the root. Specifically, explicit physical-inspired deformation introduces a displacement calculation formula with local curvature adaptation (Formula 1) to apply stronger deformation suppression to high curvature areas (such as fingertips and folds), so that geometric deformation follows the law of physical continuity and avoids tearing and penetration caused by "copy-paste" deformation in traditional methods. At the same time, the local implicit three-plane completion field uses the deformed Gaussian elements as geometric supervision and the original scene as appearance supervision for joint optimization, which can generate detail completion signals in the deformed area that are consistent with the lighting and texture of the original scene. By using a dynamic fusion coefficient γ (Formula 3) to adaptively balance the contributions of the two guiding sources based on drag distance and region complexity, this application achieves the following: in simple, short-distance editing, explicit deformation is the primary method to ensure efficiency, while in complex, long-distance editing, implicit completion is the primary method to ensure quality. The technical effect is that the edited 3D Gaussian scene exhibits no geometric tearing or penetration, and in visual appearance, continuous texture and natural lighting. (2) Existing methods are prone to content deviation or convergence difficulties due to excessive single-step optimization when dragging long distances or editing complex structures. This application decomposes the total editing amount into K sub-steps of progressive optimization and dynamically updates the multimodal guiding field in each step, so that the optimization target continuously and smoothly approaches the final state, avoiding the local minima trap commonly found in "one-step" optimization. This invention proposes a hierarchical optimization strategy that decouples geometry and appearance. In each sub-step, the color parameters are fixed first to optimize the geometric parameters (position, rotation, scaling), and the geometric loss is applied to the guiding image as the target (Equation 5) to ensure that the spatial layout of Gaussian units gradually approaches the target shape; then the geometric parameters are fixed to optimize the color parameters, and a high-quality appearance target image is generated using a dynamic image buffer pool and a lightweight diffusion model (LoRA fine-tuning) (Equation 6). This decoupling strategy prevents mutual interference between geometric deformation and color updates, making the optimization process more stable and convergent. Furthermore, the elastic continuity constraint (Formula 7) applies an exponential penalty to the abrupt displacement vector of the Gaussian element position between the current optimization step and the previous optimization step, effectively suppressing abrupt changes and discontinuities in the Gaussian element optimization process and ensuring the smoothness of the multi-step editing trajectory. The technical effects are as follows: This invention can stably complete large-scale deformation editing, and in long-distance editing (such as bending an outstretched arm to the top of the head) or editing of complex structures (such as jointed robotic arms or entwined vines), it will not produce distortion, jitter, or optimization failure, ensuring the visual smoothness of the editing trajectory; (3) Existing methods often result in coarse control of the editing area, leading to "overflow" of the editing effect into non-target areas or creating harsh seams at the boundaries. This invention achieves adaptive and fine definition of the editing area by combining geometric proximity and semantic awareness in a dynamic 3D soft mask. The geometric proximity component is based on an adaptive influence radius (calculated by statistically analyzing the average distance of the K nearest neighbor Gaussian units around the handle point), which dynamically adjusts the mask range according to the local Gaussian distribution density—the influence radius is small in dense areas and large in sparse areas, ensuring a natural transition of the mask boundary. The semantic awareness component obtains the 2D semantic labels of the handle point through a lightweight semantic segmentation network, and after back-projection into 3D space, assigns high weights to Gaussian units in the same semantic region, thereby restricting the editing to the same semantic object and effectively avoiding the overflow problem of arm deformation when editing the legs. At the same time, the soft transition weight ensures the natural fusion of the editing area boundary. Users only need to specify key control points to automatically and accurately lock and edit the target semantic part, greatly improving the convenience of operation and the local accuracy of the editing results. (4) Existing methods lack an effective interactive feedback mechanism, making it impossible for users to intuitively understand the optimization progress and intermediate effects during the editing process, resulting in a poor user experience and high trial-and-error costs. This invention embeds a low-overhead, fast rendering channel into the optimization loop. After each iteration, the current scene is rendered with a reduced resolution and simplified shaders, and pushed to the front-end interface in real time via an efficient communication protocol (such as WebSocket). At the same time, a front-end interruption listening mechanism is designed, allowing users to stop or adjust editing parameters at any time based on the preview results, thereby improving the user experience.

[0042] The above description is merely a few embodiments of this application and is not intended to limit this application in any way. Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any changes or modifications made by those skilled in the art without departing from the scope of the technical solution of this application using the disclosed technical content are equivalent to equivalent implementation cases and fall within the scope of the technical solution.

Claims

1. A method for editing a Gaussian sputtering model, characterized in that, Includes the following steps: S100: Acquire 3D Gaussian scene data and receive at least one set of handle points and target points specified by the user in the 3D Gaussian scene, and generate multi-view rendering images based on the 3D Gaussian scene data; S2 00: Based on the handle point, a dynamic three-dimensional soft mask is generated by combining geometric proximity weights and semantic awareness weights. The dynamic three-dimensional soft mask is used to define the region to be edited and indicate the degree to which each Gaussian element in the region is affected by editing. S300: Based on the handle point, the target point, and the dynamic 3D soft mask, a multimodal guide field is generated that integrates explicit physical-inspired deformation and local implicit three-plane completion. The multimodal guide field is used to provide geometric and appearance guidance signals required for editing. S400: Decompose the total editing volume into multiple sub-steps. In each sub-step, based on the dynamically updated multimodal guiding field, perform hierarchical geometric optimization and appearance optimization on the three-dimensional Gaussian scene data. Apply elastic continuity constraints in the geometric optimization sub-step of each sub-step and iteratively update the Gaussian meta-parameters within the region to be edited. S500: Performs denoising and densification post-processing on the optimized 3D Gaussian scene data to obtain the edited 3D Gaussian scene.

2. The method according to claim 1, characterized in that, Step S200 specifically includes: For each Gaussian element, its geometric proximity weight to the handle point is calculated. The geometric proximity weight is determined based on Euclidean distance and an adaptive influence radius. The adaptive influence radius is calculated by statistically analyzing the average distance of the K nearest neighboring Gaussian elements around the handle point. A lightweight semantic segmentation network is used to extract the target editing semantic category corresponding to the handle point from the multi-view rendered image. 2D semantic labels are obtained through the lightweight segmentation network, and then fused with 3D back projection and multi-view confidence voting. Combined with semantic consistency discrimination and smoothing normalization, the semantic perception weights corresponding to each Gaussian unit are obtained. The geometric proximity weights are fused with the semantic perception weights and normalized to generate the dynamic three-dimensional soft mask.

3. The method according to claim 1, characterized in that, The generation of the multimodal guiding field in step S300 specifically includes: Explicit Physics-Inspired Deformation: Based on Gaussian elements within the region to be edited indicated by a dynamic 3D soft mask, and combined with curvature adaptation weights, normalization factors, and distance decay weights of the local curvature, the smooth displacement vector of the Gaussian elements is calculated to obtain a set of deformed Gaussian elements without tearing or penetration, which is used to provide the geometric guidance signals required for editing. Local implicit three-plane complete field construction: Axially aligned bounding boxes are determined with Gaussian elements whose mask values ​​are greater than a predetermined threshold in the deformed Gaussian elements as the center, and a three-plane feature field is initialized within the bounding box; Joint optimization is performed using the geometric supervision provided by the deformed Gaussian elements and the appearance supervision provided by the original 3D Gaussian scene to obtain a local implicit three-plane complete field that combines geometric accuracy and appearance realism, which is used to provide guidance signals for appearance details required for editing; Guide image synthesis: The first guide image and the second guide image are obtained from the deformed Gaussian primitive and the local implicit three-plane full-field rendering, respectively. The dynamic fusion coefficient is calculated according to the drag distance and the region complexity. The first guide image and the second guide image are fused to generate the final guide image.

4. The method according to claim 3, characterized in that, The hierarchical optimization in step S400 specifically includes: In each sub-step, the geometry optimization sub-step is executed first, followed by the appearance optimization sub-step; Geometric optimization substep: With the color parameters of the Gaussian elements fixed, the geometric parameters of the Gaussian elements are optimized using geometric loss, with the guiding image as the target. Appearance optimization sub-step: With fixed geometric parameters, the color parameters of Gaussian elements are optimized through appearance loss using a dynamic image buffer pool and a lightweight diffusion model to generate a high-quality appearance target image.

5. The method according to claim 4, characterized in that, The elastic continuity constraint in step S400 is implemented through an elastic regularization term, which penalizes the abrupt shift vector of the Gaussian element position between the current optimization step and the previous optimization step, so as to ensure the smoothness of motion between multi-step optimization.

6. The method according to claim 5, characterized in that, The method further includes embedding a near real-time preview step during the iteration process of step S400: after each iteration, a preview image of the current 3D Gaussian scene is quickly rendered with a computational overhead lower than the final rendering quality, and sent to the user terminal for display in real time through a communication interface.

7. A Gaussian sputtering model editing system, characterized in that, The system is used to perform the method according to any one of claims 1 to 6, comprising: An interactive client that provides a graphical user interface, captures user-specified handle points and target points, and displays preview images and editing results during the editing process; A computing server, communicating with the interactive client, includes: The scene management module is used to load, store, and update 3D Gaussian scene data; The mask generation module is used to execute step S200 to generate the dynamic three-dimensional soft mask; The guidance generation module is used to execute step S300 to generate the multimodal guidance field; The incremental optimization engine is used to perform step S400, which performs hierarchical incremental optimization of the Gaussian meta-parameters. The preview rendering module is used to perform the near real-time preview step.

8. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 6.

Images