Point cloud shape editing method based on position coding
By using a position-based encoding method to map point cloud information into a high-dimensional space, and combining MLP networks and sparse point cloud geometric priors, the problems of low efficiency and poor controllability in traditional 3D modeling are solved, achieving high-precision point cloud shape editing and improving the editing effect of complex shapes.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-31
- Publication Date
- 2026-04-10
AI Technical Summary
Traditional 3D modeling methods are inefficient and have poor controllability in local point cloud editing, while texture editing results from 3D Gaussian splashing technology are prone to noise and instability.
A position-based encoding method is adopted to map the spatial information of point clouds to a high-dimensional feature space. A lightweight MLP network is combined for deformation prediction. Through rendering optimization and sparse point cloud geometric priors, SDF representation and Gaussian density control strategies are used to achieve high-precision point cloud shape editing.
It improves the accuracy and stability of point cloud editing, effectively handles local details and geometric structures of complex shapes, and enhances editing efficiency and controllability.
Smart Images
Figure CN121837567A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the field of computer vision, and particularly relates to a point cloud shape editing method based on position coding. BACKGROUND
[0002] Traditional three-dimensional modeling methods are difficult to meet the accuracy and efficiency required for local editing of point clouds. Therefore, more automated and accurate control of local editing of point clouds is particularly important for current three-dimensional editing technology. Traditional point cloud editing methods mainly focus on the basic processing stage, and still face problems such as low efficiency and poor controllability, which limit their application in complex editing tasks.
[0003] Although the three-dimensional Gaussian splatting technique can handle texture changes in point cloud editing, it only uses SDS to lose the optimization function of this single mode, resulting in noise and instability in the texture editing result. SUMMARY
[0004] The purpose of the present application is to provide a point cloud shape editing method based on position coding, which aims to solve the technical problems existing in the prior art identified in the background.
[0005] The present application is implemented as follows: a point cloud shape editing method based on position coding, comprising the following steps:
[0006] For the center position coordinates of each Gaussian point in the input point cloud , a Fourier position coding function is applied to map it from a three-dimensional space to a 256-dimensional high-dimensional feature space. The coding process is realized by a set of preset frequency triangular wave functions, which significantly enhances the perception and expression ability of the subsequent network for the local geometric structure and spatial relationship of the point cloud.
[0007] The high-dimensional position coding is input into a deformation network composed of a lightweight MLP. The network is responsible for learning and predicting the displacement vector required by each Gaussian point to achieve the target shape. By adding this displacement to the original coordinates, the spatial position of each Gaussian point is relocated, thereby completing the overall or local deformation of the point cloud. The network is initialized with a zero displacement prediction strategy to ensure stability at the beginning of training.
[0008] The Gaussian point set adjusted by displacement is rendered differentially to obtain a rendered image . At the same time, a target image is generated using a multi-modal pre-training model based on text prompts . By calculating the learning perceptual image block similarity (LPIPS) loss between the rendered image and the target image, the deformation process is driven to align with the target at the visual level, providing a robust supervision signal for multi-view geometric consistency.
[0009] A front view is selected from a set of multi-view reference images, and a depth map D is obtained using a monocular depth estimation model. By combining the camera intrinsic matrix K, the two-dimensional pixel points are reconstructed into three-dimensional sparse point cloud through back projection . After denoising and other post-processing of the point cloud, a voxel grid is constructed in its bounding box, the distance from each voxel center to the nearest point of the point cloud is calculated to form a discrete distance field, and a continuous signed distance field (SDF) representation is obtained through trilinear interpolation. The SDF is used as a geometric prior, and is introduced into the optimization process through a loss function to constrain the geometric structure of the deformed point cloud and the reference image to be consistent.
[0010] Using SDF representation, the distribution of Gaussian points is refined and controlled. A set of densification and pruning standards combining geometric distance and point attribute is developed: new Gaussian points are adaptively added in the area close to the surface of the object (SDF value is small) and the gradient is large, to improve the detail expression; for the Gaussian points far from the surface (SDF value is large) or with low contribution, they are pruned and removed. This strategy dynamically and accurately fits the target geometric surface with the distribution of Gaussian points, and finally completes the high-quality point cloud shape editing.
[0011] The beneficial effects of the present application are:
[0012] By position encoding of three-dimensional Gaussian, the spatial information in the point cloud is mapped to a new high-dimensional feature space, so that the high-frequency geometric information is better captured, and the complex deformation and local details in the point cloud editing process are handled. Further, a geometric prior based on sparse point cloud is introduced, so that the deformation network can create new structures in the optimization process according to the sparse point cloud information in the reference image. At the same time, this paper also proposes a SDF-based geometric-aware Gaussian density control strategy to enhance the control of local geometric structure in the shape editing process. This method can effectively deal with the editing task of complex shapes, and further improves the accuracy and stability of shape editing. BRIEF DESCRIPTION OF DRAWINGS
[0013] Figure 1 is a flowchart of a point cloud shape editing method based on position encoding of the present application;
[0014] Figure 2 is a deformation network structure diagram based on position encoding;
[0015] Figure 3 is a multi-layer perceptron structure diagram;
[0016] Figure 4 is a geometric-aware Gaussian density control strategy diagram. DETAILED DESCRIPTION
[0017] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.
[0018] As shown in Figure 1 and Figure 2 , a point cloud shape editing method based on position coding includes:
[0019] 1. High-dimensional position coding generation
[0020] For each Gaussian point in the input point cloud, we define the center position coordinates as ( ). In order to encode its spatial information, Fourier Positional Encoding is used.
[0021] The specific implementation process is as follows:
[0022] 1) Define a projection matrix , where is the number of frequencies of each coordinate (i.e. 2 dimensional encoding is generated for each coordinate). Each pair of values in the matrix represents the projection coefficient of a specific frequency of sine and cosine wave.
[0023] 2) Apply Fourier Positional Encoding to the coordinates to generate an encoding vector:
[0024] ;
[0025] This formula takes the dot product of the coordinates and each frequency vector , and then takes the sine and cosine values respectively, thereby generating a feature vector containing different frequency components.
[0026] 3) In this embodiment, we set =128. Since two components (sin and cos) are generated for each frequency, a high-dimensional position coding vector with a dimension of =256 is finally obtained. This 256-dimensional vector is rich in spatial position information of the point, providing rich and high-discrimination input features for the subsequent deformation network.
[0027] 2. Shape deformation displacement prediction based on MLP
[0028] As shown in Figure 3 , a lightweight multi-layer perceptron (MLP) network is used to learn the mapping relationship from position coding to displacement, realizing the deformation of the point cloud.
[0029] Deformation network structure: build a lightweight MLP network . The network can be stacked by multiple fully connected layers (Fully Connected, FC) and ReLU activation functions. The input generates a 256-dimensional position encoding , and the network finally outputs a three-dimensional displacement vector .
[0030] Displacement prediction and coordinate update: for each Gaussian point, input its position encoding into the deformation network to get the predicted displacement, and use the displacement to update the center position of the Gaussian point:
[0031] ;
[0032] This operation realizes the spatial relocation of each point in the point cloud. The displacement of multiple points collectively realizes the shape transformation of the entire point cloud.
[0033] Network initialization strategy: in order to alleviate the problem that the difference between the deformed point cloud and the initial point cloud is too large due to random initialization at the beginning of training, the last layer weight and bias of the deformation network are specially initialized in this embodiment, so that the predicted displacement is close to the zero vector at the beginning of training. This ensures that the optimization process starts from a point cloud close to the original state, improving the training stability.
[0034] In actual training, assuming that the batch size is B and each sample contains N Gaussian points. Then the tensor shape input into the deformation network is [B, 256, N], and the output displacement tensor shape is [B, N, 3].
[0035] 3. Image consistency-based rendering optimization
[0036] Through differentiable rendering and perceptual loss function, it is ensured that the deformed point cloud is visually consistent with the target.
[0037] Point cloud rendering: render the Gaussian points after updating the position to generate a two-dimensional image .
[0038] Target image generation: use a multi-modal pre-trained model fine-tuned on the point cloud editing task. Given a text prompt describing the target shape , the model can generate the corresponding target image .
[0039] Calculate LPIPS loss: in order to measure the rendering image and the target image For perceptual similarity, we adopt the LPIPS (Learned Perceptual Image Patch Similarity) loss function based on the pre-trained VGG network:
[0040] ;
[0041] This loss can better align the structure and content of the images by comparing the differences in the multi-layer features of the VGG network, providing a strong, human visual perception-compliant supervision signal for the deformation of point clouds.
[0042] 4. SDF construction based on sparse point cloud geometry prior
[0043] Extract the geometric structure information from the reference image as a strong constraint for the deformation process to maintain geometric reasonableness.
[0044] Sparse point cloud generation: prepare the reference image set where is the front view.
[0045] Process the image using an existing monocular depth estimation model to obtain its depth D, where represents the distance from the pixel point to the camera.
[0046] Given the camera's intrinsic matrix K, each pixel point is back-projected to the three-dimensional space by the following formula:
[0047] ;
[0048] Traverse the pixels with valid depth in the image to generate the initial sparse point cloud:
[0049] .
[0050] Point cloud post-processing: clean up the generated sparse point cloud, remove outliers (isolated points) and invalid far points caused by depth estimation errors.
[0051] Construct the signed distance field (SDF):
[0052] Generate a regular voxel grid within the enclosing area of the sparse point cloud .
[0053] For each voxel grid center point , calculate its Euclidean distance to the nearest point in the sparse point cloud:
[0054] ;
[0055] where, For a specific point in the sparse point cloud, is the center point of the voxel grid.
[0056] Generate continuous SDF:
[0057] Convert the discrete distance field to a continuous SDF representation using a trilinear interpolation method, which can be queried at the center position of any three-dimensional Gaussian point .
[0058] 5. Adaptive control of Gaussian density based on geometry perception
[0059] As Figure 4 shown, dynamically adjust the number and distribution of Gaussian points according to SDF information to finely represent the edited geometry shape.
[0060] SDF query: For each Gaussian point in the current optimization, query the continuous SDF value.
[0061] Densification criteria and execution:
[0062] ;
[0063] where, is the average gradient of the Gaussian point in the secondary training iteration, is a control factor that determines the degree of influence of the geometry information on the growth of the Gaussian point, is a Gaussian function based on the SDF value, which monotonically decreases as the distance s from a certain position to the nearest surface increases. When (densification threshold), a new Gaussian point will be generated at the corresponding position, and the densification of the Gaussian point will be performed. The densification threshold is set to .
[0064] In areas close to the surface (strong geometry information) and with large contributions to rendering (large gradient), densification is more likely to be needed.
[0065] Pruning criteria and execution:
[0066] ;
[0067] where, is the cumulative opacity of the Gaussian point in K iterations. is a control factor that represents the balance weight of transparency and geometry distance, and the larger the value, the more likely it is to prune the Gaussian point with lower transparency and farther from the surface. When (pruning threshold), it means that the Gaussian point is far from the surface and can be removed. The pruning threshold is set to
[0068] For the Gaussian point far from the surface (1− , and low self-opacity (low importance), it is more likely to be pruned.
[0069] The technical features of the above-described embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described, but as long as the combinations of the technical features do not contradict each other, they should be considered within the scope of the present disclosure.
[0070] The above-described embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be pointed out that, for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.
[0071] The above-described only the preferred embodiments of the present application, and not to limit the present application, any modification, equivalent replacement and improvement within the spirit and principles of the present application, etc., should be included within the protection scope of the present application.
Claims
1. A point cloud shape editing method based on position encoding, characterized in that, The method includes: Step S1: For the original 3D coordinates of the point cloud Take the center coordinates of the Gaussian point corresponding to the original 3D coordinates of the point cloud. The Fourier position coding is applied to convert it into a 256-dimensional position code; Step S2: Use a deformable network composed of MLPs to dynamically adjust the center position of the Gaussian points and predict the displacement vector of each Gaussian point. The displacement vector of each Gaussian point is predicted and applied to the spatial position of the original point to reposition each Gaussian point in the point cloud, thereby achieving spatial transformation. Step S3: Render the repositioned Gaussian points using the VGG-based learning-aware image patch similarity loss function. Step S4: Extract the reference image set Front view The input front view is estimated using a monocular depth estimation model. Given the depth D, based on the camera's intrinsic parameter matrix K and the depth D, the front view is... Two-dimensional pixels are projected into three-dimensional space to generate a sparse point cloud. And perform post-processing, based on the post-processed sparse point cloud Calculating sparse point clouds SDF representation; Step S5: Utilize sparse point clouds The SDF indicates that Gaussian point densification and pruning operations are performed to optimize the distribution of Gaussian points.
2. The method according to claim 1, characterized in that, Step S1 includes: For the center position coordinates of each Gaussian point ( Application of Fourier position coding The center coordinates of each Gaussian point are transformed into a 256-dimensional position code, enhancing the network's ability to express the geometric relationships of the point cloud. Specifically, this is implemented as follows: ； in, Encode the Fourier position of each Gaussian point. It is a projection matrix. It is the frequency of each coordinate. To represent the transpose of a matrix, Indicates the first Projection vectors.
3. The method according to claim 2, characterized in that, Step S2 includes: Encode the Fourier position of each Gaussian point. Input is a deformable network composed of lightweight MLPs Predict the displacement vector of each Gaussian point. The position update formula is: ； = ； ； in, Output the spatial displacement of the Gaussian point based on the updated position coordinates. Spatial transformation is achieved by applying the predicted displacement vector of each Gaussian point to the spatial location of the original point, thereby repositioning each Gaussian point in the point cloud.
4. The method according to claim 3, characterized in that, Step S3 includes: A differentiable rendering algorithm based on 3DGS is applied to each repositioned Gaussian point to obtain the rendered image. ; Using a multimodal pre-trained model in a point cloud editing framework based on 3D Gaussian splashing technology, specific text prompts are provided. Generate target image ; Will , The input is a VGG-based learning-aware image patch similarity loss function, which provides robust support for geometric alignment of multi-view rendering results. The loss function is defined as follows: ； It employs a VGG-based learning-based perceptual image patch similarity (LPIPS) loss function to calculate a scalar loss value, which measures the degree of dissimilarity between two images at the perceptual semantic level. This represents the image rendered after optimizing the Gaussian positions. This indicates that a finely tuned multimodal pre-trained model in a point cloud editing framework based on 3D Gaussian splashing technology uses specific text prompts. The generated target image.
5. The method according to claim 4, characterized in that, Step S4 includes: Extracting the reference image set Front view in The input image is estimated using a monocular depth estimation model. The depth D represents the distance from each pixel in the reference image to the camera; This is a side view image. This is a rear-view image; Based on the camera's intrinsic parameter matrix K and depth D, the front view is... Two-dimensional pixels Projected into 3D space, generating sparse point clouds. ,in: ； sparse point cloud Perform post-processing operations, including removing outliers and invalid points; Based on sparse point cloud Calculate the SDF representation to guide 3DGS optimization.
6. The method according to claim 4, characterized in that, The sparse point cloud-based The calculation of SDF representation specifically includes: In sparse point clouds A regular voxel mesh is generated within the enclosing region; For each voxel, calculate the distance from the voxel center to the nearest point in the sparse point cloud. : ； in, For specific points in a sparse point cloud, It is the center point of the voxel grid; The discrete distance field is transformed into a continuous SDF representation using trilinear interpolation; SDF is used as a geometric prior and introduced into the optimization process of 3D Gaussian points through a loss function: ； This represents the geometric prior loss, and the average shape error is calculated to maintain the geometric consistency between the point cloud and the reference image. For sparse dot clouds, For specific points in a sparse point cloud, This represents the distance from the Gaussian point to the target surface. This indicates the center position of a three-dimensional Gaussian point.
7. The method according to claim 1, characterized in that, Step S5 includes: For each Gaussian point location, query the corresponding SDF value; Adding new Gaussian points to the geometric surface improves the accuracy of densification and surface coverage. The Gaussian point densification standard is defined as follows: ； in, This represents the average gradient of the Gaussian point in the current training iteration. As a control factor, it determines the degree to which geometric information influences the growth of Gaussian points. For a Gaussian function based on SDF values, when Denseization threshold When this happens, a new Gaussian point is generated at the corresponding position to densify the Gaussian points; The pruning standard for Gaussian points is defined as follows: ； in, This represents the cumulative opacity of the Gaussian point over K iterations. , where is the control factor, representing the balanced weight between transparency and geometric distance, when Pruning threshold When this is the case, it indicates that pruning and removal can be performed.