3D Gaussian model non-rigid deviation correction method and system based on space deformation field and medium
By rapidly updating the center position and attributes of the 3DGS model using a non-rigid registration algorithm based on spatial deformation fields, the problem of 3DGS model positioning deviation was solved, enabling real-time model correction and accurate matching, thus meeting the real-time delivery requirements of engineering sites.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- EMIAN TECH (SHENZHEN) CO LTD
- Filing Date
- 2026-02-03
- Publication Date
- 2026-05-12
AI Technical Summary
Existing 3DGS models have positioning deviations when processing large-scale industrial scanning, and cannot accurately overlap with pre-made BIM or CAD drawings. Traditional adjustment methods are time-consuming and cannot meet the real-time delivery requirements of engineering sites.
A non-rigid registration algorithm based on spatial deformation field is adopted. By obtaining the spatial deformation map and normalized interpolation weights, the center position, rotation quaternion and covariance matrix of the 3DGS model are quickly updated to achieve real-time model correction.
It achieves precise matching between 3DGS models and deformable standard models, avoiding a time-consuming retraining process that can take several hours, improving work efficiency, and meeting the rapid adjustment needs of engineering sites.
Smart Images
Figure CN122023744A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer graphics technology, and in particular to a non-rigid correction method, system and medium for 3D Gaussian models based on spatial deformation fields. Background Technology
[0002] In the current technological context, 3DGS (3D Gaussian Splatting) technology can generate extremely realistic 3D scenes and is widely used in 3D modeling, virtual reality, and other fields. However, when processing large-scale industrial scanning, due to the cumulative drift errors caused by underlying data sources such as multi-view reconstruction and synchronous positioning and map building, the generated 3DGS model, while appearing very realistic, exhibits significant positioning deviations in practical applications and cannot accurately align with pre-fabricated BIM (Building Information Modeling) or CAD (Computer-Aided Design) drawings. This positioning deviation affects the model's application scope and accuracy, becoming particularly prominent when high-precision engineering site requirements necessitate real-time delivery.
[0003] In traditional processing methods, when a single 3D light field model needs to be moved, the center position of the Gaussian sphere is usually simply adjusted. While this operation is simple, it has obvious drawbacks. If the rotation and scaling properties of the Gaussian sphere are not adjusted simultaneously, visual problems such as the "Venetian blind effect," artifacts, or blurring will occur in the rendering results. Although some remedial methods mention solving these problems, these methods usually require retraining the model during the adjustment process. This process is extremely time-consuming, especially when processing large-scale data, and can take several hours, which is far from meeting the needs of real-time delivery in engineering projects.
[0004] In summary, existing technologies have many shortcomings, and there is an urgent need for a real-time, efficient, and accurate 3D model adjustment method to meet the needs of practical applications. This invention addresses these technical challenges by proposing a direct mapping method based on discrete deformation maps. The aim is to provide a solution that can quickly and accurately correct 3DGS models while ensuring the anisotropy and geometric consistency of the model, thus meeting the stringent requirements of large-scale scene scanning and real-time applications. Summary of the Invention
[0005] Based on this, it is necessary to propose a non-rigid correction method, system, and medium based on a 3D Gaussian model using a spatial deformation field to address the above problems.
[0006] A non-rigid bias correction method based on a 3D Gaussian model with a spatial deformation field, the method comprising: Obtain the original 3DGS model to be corrected, the original 3DGS model including the properties of each Gaussian sphere, the properties including the original center position.
[0007] The original 3DGS model is registered with a preset deformation standard model using a non-rigid registration algorithm to obtain a spatial deformation map, which includes control nodes and corresponding affine transformation matrices.
[0008] Obtain the K nearest neighbor control nodes of the original center position of each Gaussian sphere in the spatial deformation diagram, and determine the normalized interpolation weight of each Gaussian sphere based on the distance between the original center position and each nearest neighbor control node.
[0009] The original center position of each Gaussian sphere is updated using the normalized interpolation weights.
[0010] Based on the updated original center position of the Gaussian sphere, rasterization rendering is performed to obtain the corrected 3DGS model.
[0011] The attributes include the original rotation quaternion and the original covariance matrix. The process of rasterizing based on the updated original center position to obtain the corrected 3DGS model specifically includes: Using the normalized interpolation weights, update the original rotation quaternion and the original covariance matrix of each Gaussian sphere.
[0012] Based on the updated original center position of the Gaussian sphere, the original rotation quaternion, and the original covariance matrix, rasterization rendering is performed to obtain the corrected 3DGS model.
[0013] Specifically, updating the original rotation quaternion and original covariance matrix of each Gaussian sphere using the normalized interpolation weights includes: Rotation components are extracted from the affine transformation matrix of each control node and converted into quaternions. The quaternions are then weighted and interpolated based on the normalized interpolation weights to obtain the local spatial distortion rotation.
[0014] The final rotation quaternion of each Gaussian sphere is determined based on the local spatial twist rotation amount and the original rotation quaternion of the Gaussian sphere.
[0015] Determine the Jacobian matrix of the spatial deformation diagram at the center of the Gaussian sphere.
[0016] The original covariance matrix of the Gaussian sphere is transformed using the Jacobian matrix to determine the final covariance matrix of each Gaussian sphere.
[0017] Specifically, determining the Jacobian matrix at the original center position of the Gaussian sphere for the spatial deformation map includes: Using the original center position of the Gaussian sphere as input and the final center position of the Gaussian sphere as output, construct the deformation function of the spatial deformation diagram at the original center position of the Gaussian sphere.
[0018] The partial derivatives of the deformed function are determined using the finite difference method.
[0019] Arrange the partial derivatives in a preset order to determine the Jacobian matrix of the spatial deformation diagram at the original center position of the Gaussian sphere.
[0020] Specifically, updating the original center position of each Gaussian sphere using the normalized interpolation weights includes: Using the normalized interpolation weights, according to The original center position of each Gaussian sphere is updated to determine the final center position of each Gaussian sphere, where, Let be the final center position of the i-th Gaussian sphere. For normalized interpolation weights, Let be the radial transformation matrix corresponding to the j-th nearest neighbor control node. Let be the original center position of the i-th Gaussian sphere.
[0021] Specifically, determining the final rotation quaternion of each Gaussian sphere based on the local spatial distortion rotation amount and the original rotation quaternion of the Gaussian sphere includes: according to Determine the final rotation quaternion for each Gaussian sphere, where, Let be the final rotation quaternion of the i-th Gaussian sphere. This represents the amount of local spatial distortion and rotation. Let be the original rotation quaternion of the i-th Gaussian sphere.
[0022] Specifically, the step of transforming the original covariance matrix of the Gaussian spheres based on the Jacobian matrix to determine the final covariance matrix of each Gaussian sphere includes: according to Determine the final covariance matrix for each Gaussian sphere, where, Let be the final covariance matrix of the i-th Gaussian sphere. For Jacobian matrices, Let be the original covariance matrix of the i-th Gaussian sphere. It is the transpose of the Jacobian matrix.
[0023] Specifically, determining the normalized interpolation weight of each Gaussian sphere based on the distance between the original center position and the positions of each nearest neighbor control node includes: according to Determine the distance between the original center position and the control node position of each Gaussian sphere, where, Let be the distance between the original center position of the i-th Gaussian sphere and the position of the j-th control node. Let be the original center position of the i-th Gaussian sphere. Let j be the position of the j-th control node.
[0024] according to Determine the interpolation weights for each Gaussian sphere, where, For interpolation weights, It is a very small positive value.
[0025] according to Determine the normalized interpolation weights for each Gaussian sphere, where, For normalized interpolation weights, It is the sum of the difference weights of all K nearest neighbor control nodes of the i-th Gaussian sphere.
[0026] A non-rigid spin correction system based on a 3D Gaussian model of a spatial deformation field, the system comprising: The original 3DGS model acquisition module is used to acquire the original 3DGS model to be corrected. The original 3DGS model includes the properties of each Gaussian sphere, including the original center position.
[0027] The spatial deformation map acquisition module is used to register the original 3DGS model with a preset deformation standard model using a non-rigid registration algorithm to obtain a spatial deformation map, which includes control nodes and corresponding affine transformation matrices.
[0028] The normalized interpolation weight determination module is used to obtain the K nearest neighbor control nodes of the original center position of each Gaussian sphere in the spatial deformation diagram, and determine the normalized interpolation weight of each Gaussian sphere based on the distance between the original center position and each nearest neighbor control node.
[0029] The attribute update module is used to update the original center position of each Gaussian sphere using the normalized interpolation weights.
[0030] The 3DGS model acquisition module after correction is used to perform rasterization rendering based on the updated original center position of the Gaussian sphere to obtain the 3DGS model after correction.
[0031] A computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of the method described above.
[0032] The embodiments of the present invention have the following beneficial effects: This invention utilizes a non-rigid registration algorithm to analyze the geometric differences between a pre-defined deformable standard model (such as a BIM / CAD model) and a 3DGS model with deviations, obtaining a spatial deformation map containing a set of discrete control nodes and their corresponding transformation matrices. This algorithm considers not only global displacement but also local rotation and scaling changes, avoiding Venetian blind effects, artifacts, or blurring problems. This invention can obtain a spatial deformation map based on the accuracy of the pre-defined deformable standard model. Furthermore, it obtains the K nearest neighbor control nodes of the original center position of each Gaussian sphere in the spatial deformation map. Based on the distance between the original center position and each nearest neighbor control node, it determines the normalized interpolation weights of each Gaussian sphere. The original center position of each Gaussian sphere is updated based on these normalized interpolation weights. This allows for rapid and accurate adjustment of the 3DGS model without additional training, enabling the model to precisely match the deformable standard model. This avoids the time-consuming retraining process of traditional methods, which can take several hours, and achieves real-time model correction. This allows for rapid adjustment of the 3DGS model on-site to meet actual needs, significantly improving work efficiency. Attached Figure Description
[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0034] in: Figure 1 A flowchart illustrating an embodiment of a non-rigid spin correction method for a 3D Gaussian model based on a spatial deformation field provided by the present invention. Figure 2 A flowchart illustrating another embodiment of a non-rigid spin correction method based on a spatial deformation field using a 3D Gaussian model provided by the present invention. Figure 3 A schematic diagram of the structure of an embodiment of a non-rigid spin correction system based on a spatial deformation field using a 3D Gaussian model provided by the present invention; Figure 4 A schematic diagram of the structure of an embodiment of the medium provided by the present invention. Detailed Implementation
[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0036] like Figure 1 As shown, Figure 1 This is a flowchart illustrating an embodiment of a non-rigid spin correction method for a 3D Gaussian model based on a spatial deformation field, provided by the present invention. The method includes: S101: Obtain the original 3DGS model to be corrected. The original 3DGS model includes the properties of each Gaussian sphere, including the original center position.
[0037] S102: The original 3DGS model is registered with the preset deformation standard model using a non-rigid registration algorithm to obtain a spatial deformation map, which includes control nodes and the corresponding affine transformation matrix.
[0038] For example, the original 3DGS model to be corrected is obtained, which includes the properties of each Gaussian sphere, including the original center position.
[0039] Furthermore, the preset deformation standard models include BIM (Building Information Modeling) or CAD (Computer-Aided Design). A set of reference point clouds is obtained by 3D sampling of the BIM or CAD data. The coordinates of each reference point cloud in the set are a data representation of the actual deformation standard. A non-rigid registration algorithm is used to directly align the Gaussian sphere properties of the 3DGS model to be corrected with the precise coordinates of the reference point clouds, obtaining a spatial deformation map. This spatial deformation map includes control nodes and corresponding affine transformation matrices, achieving geometric consistency between the 3DGS model and BIM / CAD. The affine transformation matrices include deformation methods such as translation, rotation, scaling, and shearing.
[0040] S103: Obtain the K nearest neighbor control nodes of the original center position of each Gaussian sphere in the spatial deformation map, and determine the normalized interpolation weight of each Gaussian sphere based on the distance between the original center position and each nearest neighbor control node.
[0041] For example, for each Gaussian sphere in the 3DGS model to be corrected, its K nearest neighbor control nodes in the spatial deformation map are searched. Control nodes act as "anchor points" for local spatial deformation, and their transformations affect neighboring Gaussian spheres. The final deformation of a Gaussian sphere is a weighted combination of the deformations of all its K nearest neighbor control nodes.
[0042] Furthermore, the distance between the original center position and the control node position of each Gaussian sphere is determined according to the following formula: ; in, Let be the distance between the original center position of the i-th Gaussian sphere and the position of the j-th control node. Let be the original center position of the i-th Gaussian sphere. Let j be the position of the j-th control node.
[0043] The interpolation weights for each Gaussian sphere are determined using the formula shown below: ; in, For interpolation weights, It is a very small positive value; The normalized interpolation weights for each Gaussian sphere are determined using the formula shown below: ; in, For normalized interpolation weights, It is the sum of the difference weights of all K nearest neighbor control nodes of the i-th Gaussian sphere.
[0044] S104: Update the original center position of each Gaussian sphere using normalized interpolation weights.
[0045] For example, the original center position of each Gaussian sphere is updated using normalized interpolation weights according to the formula shown below to determine the final center position of each Gaussian sphere: ; in, Let be the final center position of the i-th Gaussian sphere. For normalized interpolation weights, Let be the radial transformation matrix corresponding to the j-th nearest neighbor control node. Let be the original center position of the i-th Gaussian sphere.
[0046] S105: Perform rasterization rendering based on the original center position of the updated Gaussian sphere to obtain the corrected 3DGS model.
[0047] For example, the attributes of each Gaussian sphere also include the original rotation quaternion and the original covariance matrix. The original rotation quaternion and the original covariance matrix of each Gaussian sphere are updated using normalized interpolation weights. Based on the updated original center position, original rotation quaternion and original covariance matrix of the Gaussian sphere, rasterization rendering is performed to obtain the corrected 3DGS model.
[0048] As described above, this invention utilizes a non-rigid registration algorithm to analyze the geometric differences between a pre-defined deformable standard model (such as a BIM / CAD model) and a 3DGS model with deviations, obtaining a spatial deformation map containing a set of discrete control nodes and their corresponding transformation matrices. This algorithm considers not only global displacement but also local rotation and scaling changes, avoiding Venetian blinds, artifacts, or blurring issues. This invention can obtain a spatial deformation map based on the accuracy of the pre-defined deformable standard model. Furthermore, it obtains the K nearest neighbor control nodes of the original center position of each Gaussian sphere in the spatial deformation map. Based on the distance between the original center position and each nearest neighbor control node, it determines the normalized interpolation weight of each Gaussian sphere. The original center position of each Gaussian sphere is updated based on the normalized interpolation weight. This allows for rapid and accurate adjustment of the 3DGS model without additional training, enabling the model to accurately match the deformable standard model. This avoids the time-consuming retraining process of traditional methods, which can take several hours, and achieves real-time model correction. This allows for rapid adjustment of the 3DGS model on-site to meet actual needs, significantly improving work efficiency.
[0049] like Figure 2 As shown, Figure 2 This is a flowchart illustrating another embodiment of a non-rigid spin correction method for a 3D Gaussian model based on a spatial deformation field provided by the present invention. The method includes: S201: Obtain the original 3DGS model to be corrected. The original 3DGS model includes the properties of each Gaussian sphere, including the original center position.
[0050] S202: The original 3DGS model is registered with the preset deformation standard model using a non-rigid registration algorithm to obtain a spatial deformation map, which includes control nodes and the corresponding affine transformation matrix.
[0051] S203: Obtain the K nearest neighbor control nodes of the original center position of each Gaussian sphere in the spatial deformation map, and determine the normalized interpolation weight of each Gaussian sphere based on the distance between the original center position and each nearest neighbor control node.
[0052] S204: Update the original center position of each Gaussian sphere using normalized interpolation weights.
[0053] It should be noted that steps S201-S204 are in Figure 1 The implementation scenarios shown have been discussed in detail and will not be repeated here.
[0054] S205: The attributes also include the original rotation quaternion and the original covariance matrix. The original rotation quaternion and the original covariance matrix of each Gaussian sphere are updated using normalized interpolation weights.
[0055] S206: Rasterization rendering is performed based on the updated original center position of the Gaussian sphere, the original rotation quaternion, and the original covariance matrix to obtain the corrected 3DGS model.
[0056] For example, the attributes also include the original rotation quaternion and the original covariance matrix. Rotation components are extracted from the affine transformation matrix of each control node and converted into quaternions. Weighted interpolation is performed on the quaternions based on normalized interpolation weights to obtain the local space distortion rotation. The final rotation quaternion for each Gaussian sphere is determined based on the local space distortion rotation and the original rotation quaternion of the Gaussian sphere, as shown in the following equation: ; in, Let be the final rotation quaternion of the i-th Gaussian sphere. This represents the amount of local spatial distortion and rotation. Let be the original rotation quaternion of the i-th Gaussian sphere.
[0057] Furthermore, the Jacobian matrix of the spatial deformation map at the original center position of the Gaussian sphere is determined. Specifically, using the original center position of the Gaussian sphere as input and the final center position of the Gaussian sphere as output, a deformation function of the spatial deformation map at the original center position of the Gaussian sphere is constructed. Minimal offsets are applied to the x, y, and z coordinates of the original center position using the finite difference method, and the deformation output at the offset center position is calculated, thus obtaining the partial derivatives of the deformation function in the three directions. The partial derivatives are arranged in a preset order to determine the Jacobian matrix of the spatial deformation map at the original center position of the Gaussian sphere.
[0058] Furthermore, the original covariance matrix of the Gaussian sphere is transformed according to the Jacobian matrix to determine the final covariance matrix of each Gaussian sphere, as shown in the following equation: ; in, Let be the final covariance matrix of the i-th Gaussian sphere. For Jacobian matrices, Let be the original covariance matrix of the i-th Gaussian sphere. It is the transpose of the Jacobian matrix.
[0059] Furthermore, rasterization rendering is performed based on the updated original center position of the Gaussian sphere, the original rotation quaternion, and the original covariance matrix to obtain the corrected 3DGS model.
[0060] As described above, this invention treats the 3DGS model as a discrete set of points with anisotropic properties, uses a spatial deformation map as a reference, and simultaneously updates the original position center, original rotation quaternion, and original covariance matrix of each Gaussian sphere in the 3DGS model to be corrected. This achieves millisecond-level model correction while maintaining rendering quality. Rasterization rendering is then performed based on the updated original position center, original rotation quaternion, and original covariance matrix of the Gaussian spheres to obtain a corrected 3DGS model that is precisely aligned with the deformed standard model. This method is executed entirely during the inference phase, requiring no retraining, achieving a significant efficiency leap from hours to milliseconds, and meeting the needs of rapid, accurate, and real-time model correction in engineering settings.
[0061] like Figure 3 As shown, Figure 3 This is a schematic diagram of an embodiment of a non-rigid 3D Gaussian model correction system based on a spatial deformation field provided by the present invention. A non-rigid 3D Gaussian model correction system 10 based on a spatial deformation field includes: The original 3DGS model acquisition module 11 is used to acquire the original 3DGS model to be corrected. The original 3DGS model includes the properties of each Gaussian sphere, including the original center position.
[0062] The spatial deformation map acquisition module 12 is used to register the original 3DGS model with the preset geometric deformation standard through a non-rigid registration algorithm to obtain a spatial deformation map, which includes control nodes and corresponding affine transformation matrices.
[0063] The normalized interpolation weight determination module 13 is used to obtain the K nearest neighbor control nodes of the original center position of each Gaussian sphere in the spatial deformation map, and determine the normalized interpolation weight of each Gaussian sphere based on the distance between the original center position and each nearest neighbor control node.
[0064] The attribute update module 14 is used to update the original center position of each Gaussian sphere using normalized interpolation weights. The 3DGS model acquisition module 15 after correction is used to perform rasterization rendering based on the original center position of the updated Gaussian sphere to obtain the 3DGS model after correction.
[0065] For example, in the original 3DGS model acquisition module 11, the original 3DGS model to be corrected is acquired. The original 3DGS model includes the attributes of each Gaussian sphere, including the original center position. In the spatial deformation map acquisition module 12, the original 3DGS model is registered with a preset geometric deformation standard using a non-rigid registration algorithm to obtain a spatial deformation map. The spatial deformation map includes control nodes and the corresponding affine transformation matrix.
[0066] In the normalized interpolation weight determination module 13, the distance between the original center position and the control node position of each Gaussian sphere is determined according to the following formula: ; in, Let be the distance between the original center position of the i-th Gaussian sphere and the position of the j-th control node. Let be the original center position of the i-th Gaussian sphere. Let j be the position of the j-th control node.
[0067] The interpolation weights for each Gaussian sphere are determined using the formula shown below: ; in, For interpolation weights, It is a very small positive value.
[0068] The normalized interpolation weights for each Gaussian sphere are determined using the formula shown below: ; in, For normalized interpolation weights, It is the sum of the difference weights of all K nearest neighbor control nodes of the i-th Gaussian sphere.
[0069] In attribute update module 14, the original center position of each Gaussian sphere is updated using normalized interpolation weights according to the formula shown below, thus determining the final center position of each Gaussian sphere: ; in, Let be the final center position of the i-th Gaussian sphere. For normalized interpolation weights, Let be the radial transformation matrix corresponding to the j-th nearest neighbor control node. Let be the original center position of the i-th Gaussian sphere.
[0070] In the 3DGS model acquisition module 15 after correction, rasterization rendering is performed based on the original center position of the updated Gaussian sphere to obtain the 3DGS model after correction.
[0071] like Figure 4 As shown, Figure 4 This is a schematic diagram of the structure of an embodiment of the medium provided by the present invention. The medium 20 stores at least one computer program 21, which is executed by a processor to implement... Figure 1 and Figure 2The method shown is detailed above and will not be repeated here. In one embodiment, the medium 20 can be a storage chip, hard disk, portable hard disk, USB flash drive, optical disk, or other read / write storage device, or even a server, etc.
[0072] Furthermore, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific or sequential order shown to achieve the desired result. In some implementations, multitasking and parallel processing are possible or may be advantageous.
[0073] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer-readable storage media are basically similar to the method embodiments, and therefore described more simply; relevant parts can be referred to the descriptions of the method embodiments.
[0074] The apparatus, device, non-volatile computer-readable storage medium and method provided in the embodiments of this specification are corresponding. Therefore, the apparatus, device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, device and non-volatile computer storage medium will not be repeated here.
[0075] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.
[0076] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components. Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0077] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0078] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0079] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0080] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0081] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0082] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0083] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0084] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.
[0085] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.
[0086] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.
Claims
1. A non-rigid bias correction method based on a 3D Gaussian model using a spatial deformation field, characterized in that, The method includes: Obtain the original 3DGS model to be corrected, the original 3DGS model including the properties of each Gaussian sphere, the properties including the original center position; The original 3DGS model is registered with a preset deformation standard model using a non-rigid registration algorithm to obtain a spatial deformation map, which includes control nodes and corresponding affine transformation matrices. Obtain the K nearest neighbor control nodes of the original center position of each Gaussian sphere in the spatial deformation diagram, and determine the normalized interpolation weight of each Gaussian sphere based on the distance between the original center position and each nearest neighbor control node. The original center position of each Gaussian sphere is updated using the normalized interpolation weights. Based on the updated original center position of the Gaussian sphere, rasterization rendering is performed to obtain the corrected 3DGS model.
2. The non-rigid correction method for a 3D Gaussian model based on a spatial deformation field according to claim 1, characterized in that, The attributes also include the original rotation quaternion and the original covariance matrix. The process of performing rasterization rendering based on the updated original center position to obtain the corrected 3DGS model specifically includes: Using the normalized interpolation weights, update the original rotation quaternion and the original covariance matrix of each Gaussian sphere; Based on the updated original center position of the Gaussian sphere, the original rotation quaternion, and the original covariance matrix, rasterization rendering is performed to obtain the corrected 3DGS model.
3. The non-rigid correction method for a 3D Gaussian model based on a spatial deformation field according to claim 2, characterized in that, The step of updating the original rotation quaternion and original covariance matrix of each Gaussian sphere using the normalized interpolation weights specifically includes: Rotation components are extracted from the affine transformation matrix of each control node and converted into quaternions. The quaternions are then weighted and interpolated based on the normalized interpolation weights to obtain the local space distortion rotation amount. The final rotation quaternion of each Gaussian sphere is determined based on the local spatial twist rotation amount and the original rotation quaternion of the Gaussian sphere. Determine the Jacobian matrix of the spatial deformation diagram at the center of the Gaussian sphere; The original covariance matrix of the Gaussian sphere is transformed using the Jacobian matrix to determine the final covariance matrix of each Gaussian sphere.
4. The non-rigid correction method for a 3D Gaussian model based on a spatial deformation field according to claim 3, characterized in that, The determination of the Jacobian matrix at the original center position of the spatial deformation map of the Gaussian sphere specifically includes: Using the original center position of the Gaussian sphere as input and the final center position of the Gaussian sphere as output, construct the deformation function of the spatial deformation diagram at the original center position of the Gaussian sphere; The partial derivatives of the deformable function are determined by the finite difference method. Arrange the partial derivatives in a preset order to determine the Jacobian matrix of the spatial deformation diagram at the original center position of the Gaussian sphere.
5. The non-rigid correction method for a 3D Gaussian model based on a spatial deformation field according to claim 1, characterized in that, The step of updating the original center position of each Gaussian sphere using the normalized interpolation weights specifically includes: Using the normalized interpolation weights, according to The original center position of each Gaussian sphere is updated to determine the final center position of each Gaussian sphere, where, Let be the final center position of the i-th Gaussian sphere. For normalized interpolation weights, Let be the radial transformation matrix corresponding to the j-th nearest neighbor control node. Let be the original center position of the i-th Gaussian sphere.
6. The non-rigid correction method for a 3D Gaussian model based on a spatial deformation field according to claim 3, characterized in that, The step of determining the final rotation quaternion of each Gaussian sphere based on the local spatial distortion rotation amount and the original rotation quaternion of the Gaussian sphere specifically includes: according to Determine the final rotation quaternion for each Gaussian sphere, where, Let be the final rotation quaternion of the i-th Gaussian sphere. This represents the amount of local spatial distortion and rotation. Let be the original rotation quaternion of the i-th Gaussian sphere.
7. The non-rigid spin correction method for a 3D Gaussian model based on a spatial deformation field according to claim 3, characterized in that, The step of transforming the original covariance matrix of the Gaussian spheres based on the Jacobian matrix to determine the final covariance matrix of each Gaussian sphere specifically includes: according to Determine the final covariance matrix for each Gaussian sphere, where, Let be the final covariance matrix of the i-th Gaussian sphere. For Jacobian matrices, Let be the original covariance matrix of the i-th Gaussian sphere. It is the transpose of the Jacobian matrix.
8. The non-rigid correction method for a 3D Gaussian model based on a spatial deformation field according to claim 1, characterized in that, The step of determining the normalized interpolation weight of each Gaussian sphere based on the distance between the original center position and the positions of each nearest neighbor control node specifically includes: according to Determine the distance between the original center position and the control node position of each Gaussian sphere, where, Let be the distance between the original center position of the i-th Gaussian sphere and the position of the j-th control node. Let be the original center position of the i-th Gaussian sphere. This represents the position of the j-th control node; according to Determine the interpolation weights for each Gaussian sphere, where, For interpolation weights, It is a very small positive value; according to Determine the normalized interpolation weights for each Gaussian sphere, where, For normalized interpolation weights, It is the sum of the difference weights of all K nearest neighbor control nodes of the i-th Gaussian sphere.
9. A non-rigid spin correction system based on a 3D Gaussian model of a spatial deformation field, characterized in that, The system includes: The original 3DGS model acquisition module is used to acquire the original 3DGS model to be corrected. The original 3DGS model includes the properties of each Gaussian sphere, including the original center position. The spatial deformation map acquisition module is used to register the original 3DGS model with a preset deformation standard model using a non-rigid registration algorithm to obtain a spatial deformation map, which includes control nodes and corresponding affine transformation matrices. The normalized interpolation weight determination module is used to obtain the K nearest neighbor control nodes of the original center position of each Gaussian sphere in the spatial deformation diagram, and determine the normalized interpolation weight of each Gaussian sphere based on the distance between the original center position and each nearest neighbor control node. The attribute update module is used to update the original center position of each Gaussian sphere using the normalized interpolation weights. The 3DGS model acquisition module after correction is used to perform rasterization rendering based on the updated original center position of the Gaussian sphere to obtain the 3DGS model after correction.
10. A computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of the method as claimed in any one of claims 1 to 8.