Gaussian parameter generation method and device, electronic equipment, and storage medium

Abstract

The disclosure provides a Gaussian parameter generation method and device, electronic equipment and storage medium, the method comprising: performing adjacent point query on a target Gaussian point in a first Gaussian splash model to determine a set of adjacent points of the target Gaussian point; the target Gaussian point is one of a plurality of Gaussian points, each Gaussian point having a basic parameter including a center position coordinate; determining local description information of the target Gaussian point according to adjacent Gaussian points in the set of adjacent points, the local description information including relative coordinates of the adjacent Gaussian points relative to the target Gaussian point and distance values between the adjacent Gaussian points and the target Gaussian point; generating geometric parameters of the plurality of Gaussian points through a parameter generation network according to the basic parameters and the local description information of the plurality of Gaussian points; and the geometric parameters of the plurality of Gaussian points are used to generate a rendered image of the first Gaussian splash model. According to the embodiments of the disclosure, the training and application effect of the 3D GS model can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method and apparatus for generating Gaussian parameters, an electronic device, and a computer-readable storage medium. Background Technology

[0002] 3D Gaussian Splatting (3DGS) is a scene reconstruction and rendering technique based on explicit 3D Gaussian point cloud representation. This technique models the scene using 3D Gaussian functions, where each Gaussian point (also called a Gaussian primitive) includes attributes such as center position, color, opacity, scale, and rotation. A 3DGS model is a 3D scene representation model built using 3DGS technology, which can be obtained through training.

[0003] In a 3DGS model, the geometric parameters of Gaussian points, including scale and rotation, determine the extent to which the Gaussian points cover in 3D space and the direction in which they unfold. However, in related technologies, the geometric parameters of Gaussian points are all independent learnable parameters during training, isolated point-by-point free variables, which fail to reflect the relationships between Gaussian points, leading to poor training and application performance of the 3DGS model. Summary of the Invention

[0004] This disclosure provides a method and apparatus for generating Gaussian parameters, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] In a first aspect, this disclosure provides a Gaussian parameter generation method, which includes: performing a neighbor point query on a target Gaussian point in a first Gaussian splash model to determine a set of neighbor points for the target Gaussian point; the target Gaussian point is one of multiple Gaussian points in the first Gaussian splash model, each Gaussian point having basic parameters, the basic parameters including center position coordinates; determining local description information of the target Gaussian point based on the neighboring Gaussian points in the set of neighboring points, the local description information including the relative coordinates of the neighboring Gaussian points relative to the target Gaussian point, and the distance value between the neighboring Gaussian points and the target Gaussian point; generating geometric parameters of the multiple Gaussian points through a parameter generation network based on the basic parameters and local description information of the multiple Gaussian points; wherein, the geometric parameters of the multiple Gaussian points are used to generate a rendered image of the first Gaussian splash model.

[0006] Secondly, this disclosure provides a Gaussian parameter generation apparatus, comprising: a neighbor point determination module, configured to perform a neighbor point query on a target Gaussian point in a first Gaussian splatter model to determine a set of neighbor points for the target Gaussian point; the target Gaussian point is one of multiple Gaussian points in the first Gaussian splatter model, each Gaussian point having basic parameters, the basic parameters including position coordinates, color information, opacity information, and low-dimensional features; a local description determination module, configured to determine local description information of the target Gaussian point based on the neighboring Gaussian points in the set of neighboring points, the local description information including the relative coordinates of the neighboring Gaussian points relative to the target Gaussian point, and the distance value between the neighboring Gaussian points and the target Gaussian point; and a parameter generation module, configured to generate geometric parameters of the multiple Gaussian points through a parameter generation network based on the basic parameters and local description information of the multiple Gaussian points; wherein the geometric parameters of the multiple Gaussian points are used to generate a rendered image of the first Gaussian splatter model.

[0007] Thirdly, this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the above-described Gaussian parameter generation method.

[0008] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described Gaussian parameter generation method.

[0009] Fifthly, this disclosure provides a computer program product comprising computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the above-described Gaussian parameter generation method.

[0010] The embodiments provided in this disclosure can perform a neighbor point query to determine a set of neighbor points for a target Gaussian point; determine the local description information of the target Gaussian point based on the neighboring Gaussian points in the set of neighbor points; and generate the geometric parameters of multiple Gaussian points through a parameter generation network based on the basic parameters and local description information of multiple Gaussian points, so that the generated geometric parameters come from the local geometric structure, thereby improving the accuracy of the generated geometric parameters and improving the training and application effect of the 3DGS model.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0013] Figure 1 A flowchart of a Gaussian parameter generation method provided in an embodiment of this disclosure.

[0014] Figure 2 A block diagram of a Gaussian parameter generation device provided in an embodiment of this disclosure.

[0015] Figure 3 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0016] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0017] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0018] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0019] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0020] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0021] In fields such as 3D scene digital reconstruction technology, neural network-based scene rendering technology, and machine learning-based 3D geometric data representation technology, scene reconstruction and rendering are usually achieved through 3D Gaussian splash models.

[0022] Among related technologies, there are methods that utilize trained neural radiance fields (NeRF) to sample along training rays and initialize parameters such as position, opacity, color, scale, and rotation of a 3DGS model. The scale can be set based on the distance to the nearest sampling point or the length of the ray segment, while rotation can be initialized based on the normal estimated by the density gradient. Other related technologies include methods that predict representative features of scene points using multi-resolution feature meshes and MLPs (Multilayer Perceptrons), which can be used in 3DGS models. The output can include parameters such as position changes, color, opacity, scale, and rotation. Finally, there are dynamic 3D scene reconstruction workflows that generate 4D Gaussian parameters, including opacity, color, scale, rotation, and center position, using 4D anchor point feature vectors and fully connected networks, and then obtain a 3D Gaussian point cloud based on target time slices.

[0023] However, related technologies that initialize 3DGS models using neural radiation fields focus more on generating initial Gaussian points from NeRF representations. After initialization, scale and rotation still primarily enter the subsequent 3DGS optimization process. Schemes that predict scene attributes from multi-resolution coordinate features or generate dynamic Gaussian parameters from 4D anchor points, while using features and MLPs to generate Gaussian attributes, emphasize multi-resolution coordinate grids or 4D anchor points as input representations, respectively. These schemes still treat the geometric parameters (scale and rotation) of Gaussian points as independent learnable parameters during training, as isolated point-by-point free variables. They fail to reflect the relationships between Gaussian points. Especially during point set additions, deletions, pruning, recombination, compression, and export in the engineering training process, inconsistent geometric parameter generation rules lead to deterioration in the training and application performance of the 3DGS model.

[0024] According to embodiments of this disclosure, a Gaussian parameter generation method is provided. This method first maintains the basic parameters of Gaussian points in an existing or training 3DGS point set, and then generates or corrects the geometric parameters of Gaussian points before rendering by utilizing the point-by-point learnable features of Gaussian points, neighborhood context information of Gaussian points, and a shared parameter generation network. This ensures that the geometric parameters of Gaussian points originate from local geometric structures and have reusable local generation rules, thereby improving the training and application effects of the 3DGS model.

[0025] According to embodiments of this disclosure, without altering camera input, image reconstruction loss, and the main 3DGS differentiable rendering process, the geometric parameters of Gaussian points can be transformed from simple point-by-point free parameters into a mechanism jointly generated or modified by point features, local neighborhood context, and a shared parameter generation network (or materializer). Simultaneously, this mechanism ensures consistency between the feature table, neighborhood index, basic parameters, and generated geometric parameters in scenarios such as point densification, deletion, cropping, recombination, compression, and export, as well as during short-range recovery training.

[0026] Among them, the feature table is used to represent the form for storing the point feature data of Gaussian points; the neighborhood index is used to represent the index identifier ID of the neighboring Gaussian points in the set of neighboring points of the Gaussian point; the basic parameters are used to represent the basic inherent attribute parameters of the Gaussian point, including the center position coordinates, color information, opacity information and low-dimensional features, etc.; the geometric parameters include scale information and rotation information.

[0027] The Gaussian parameter generation method according to embodiments of this disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. The method can be implemented by a processor calling computer-readable program instructions stored in memory. Alternatively, the method can be executed by a server.

[0028] Figure 1 A flowchart illustrating a Gaussian parameter generation method provided in this disclosure. (Refer to...) Figure 1 The method includes:

[0029] In step S11, a neighbor point query is performed on the target Gaussian point in the first Gaussian splash model to determine the set of neighbor points of the target Gaussian point; the target Gaussian point is one of a plurality of Gaussian points in the first Gaussian splash model, and each Gaussian point has basic parameters, including the center position coordinates.

[0030] In step S12, local description information of the target Gaussian point is determined based on the neighboring Gaussian points in the neighboring point set. The local description information includes the relative coordinates of the neighboring Gaussian points relative to the target Gaussian point, and the distance between the neighboring Gaussian points and the target Gaussian point.

[0031] In step S13, the geometric parameters of the multiple Gaussian points are generated by a parameter generation network based on the basic parameters and local description information of the multiple Gaussian points; wherein, the geometric parameters of the multiple Gaussian points are used to generate the rendered image of the first Gaussian splash model.

[0032] For example, an existing 3DGS model can be read, or a 3DGS model can be initialized based on point cloud data. The point set of this 3DGS model includes multiple Gaussian points, each of which includes center coordinates, color information, and opacity information. Furthermore, a set of low-dimensional features, such as an 8-dimensional or 16-dimensional feature vector, is established for each Gaussian point. This disclosure does not limit the specific dimension of the low-dimensional features.

[0033] In some possible implementations, during the pre-training or initial training phase of the 3DGS model, scale and rotation information are not optimized separately as primary point-by-point free parameters, but are recovered by the neighborhood and parameter generation network in subsequent training phases. Thus, after the pre-training or initial training phase, a first Gaussian splash model is obtained, which includes multiple Gaussian points, each with fundamental parameters, including at least the center position coordinates.

[0034] In some possible implementations, for any Gaussian point in the Gaussian point set of the first Gaussian splash model (hereinafter referred to as the target Gaussian point), a neighbor query can be performed in step S11 to determine the set of neighboring points of the target Gaussian point. The neighbor query method includes any one of fixed-number neighbor query, radius-range neighbor query, hierarchical neighbor query, and spatial block neighbor query. The fixed-number neighbor query can use the k-Nearest Neighbors (kNN) algorithm. This disclosure does not limit the specific method of neighbor query.

[0035] In some possible implementations, the basic parameters include at least the center coordinates, used to determine the geometric relationships between Gaussian points. Based on the center coordinates of all Gaussian points in the set, a neighbor lookup is performed on the target Gaussian point to find the spatially closest Gaussian points, for example, eight. The numbers of these neighboring Gaussian points are recorded, resulting in the set of neighboring points for the target Gaussian point. When the Gaussian point set is split, copied, added to, or deleted, a neighbor lookup is re-performed on the Gaussian points in the affected region, refreshing the neighborhood index of the affected region, i.e., the numbers of the neighboring Gaussian points.

[0036] In some possible implementations, in step S12, the local description information of the target Gaussian point is determined based on the basic parameters of the neighboring Gaussian points in the neighboring point set and the basic parameters of the target Gaussian point. This local description information is used to characterize the local geometric structure information around the target Gaussian point.

[0037] In some possible implementations, the local description information includes at least the relative coordinates of neighboring Gaussian points relative to the target Gaussian point, and the distance values ​​between the neighboring Gaussian points and the target Gaussian point. The relative coordinates reflect the local shape orientation, and the distance values ​​statistically reflect the local sparsity. Both the relative coordinates and the distance values ​​can be calculated using the center coordinates of the neighboring Gaussian points and the center coordinates of the target Gaussian point. For example, if there are 8 neighboring Gaussian points, 8 relative coordinates and 8 distance values ​​can be obtained.

[0038] In some possible implementations, the basic parameters also include color information, opacity information, and low-dimensional features. Among them, color information and opacity information are the conventional rendering attributes of Gaussian points, which can be trained using conventional 3D Gaussian methods; the low-dimensional features are a low-dimensional feature vector, a set of numbers updated during training, used to record the local state of the Gaussian point.

[0039] In some possible implementations, the local description information further includes at least one of the following: the average feature of the low-dimensional features of the neighboring Gaussian points, the variance information of the low-dimensional features of the neighboring Gaussian points, the difference information between the low-dimensional features of the neighboring Gaussian points and the low-dimensional features of the target Gaussian point, the maximum value of the distance between the neighboring Gaussian points and the target Gaussian point, and the average value of the distance between the neighboring Gaussian points and the target Gaussian point. This enriches the content of the local description information, allowing it to better reflect the local geometric structure.

[0040] The average feature of the low-dimensional features of neighboring Gaussian points can be obtained by averaging the corresponding elements of the low-dimensional feature vectors of multiple neighboring Gaussian points, resulting in a feature vector with the same dimension as the low-dimensional features. The variance information of the low-dimensional features of neighboring Gaussian points can be obtained by calculating the variance of the corresponding elements of the low-dimensional feature vectors of multiple neighboring Gaussian points, resulting in a feature vector. The difference information between the low-dimensional features of neighboring Gaussian points and the low-dimensional features of the target Gaussian point can be obtained by subtracting the low-dimensional feature vectors of multiple neighboring Gaussian points from the low-dimensional features of the target Gaussian point, resulting in a difference vector corresponding to the number of neighboring Gaussian points. The maximum value of the distance between a neighboring Gaussian point and the target Gaussian point can be obtained by directly comparing multiple distance values. The average value of the distance between a neighboring Gaussian point and the target Gaussian point can be obtained by averaging multiple distance values.

[0041] It should be understood that those skilled in the art can set the content and specific calculation method of the partial description information according to the actual situation, and this disclosure does not impose any restrictions on this.

[0042] The variance information and the difference information between features are used to reflect whether the current target Gaussian point and its neighboring Gaussian points belong to the same local state. In this way, the subsequent parameter generation network sees not an isolated Gaussian point, but a local geometric segment.

[0043] For example, if most of the neighboring Gaussian points of a Gaussian point are distributed near the same plane, then the relative coordinate statistics will show a dominant planar direction, and the parameter generation network can recover the Gaussian point that expands along the plane and has a narrow normal. If the neighboring Gaussian points are distributed along a thin line, the parameter generation network can recover the Gaussian point that is elongated along that line. If the neighboring Gaussian points are distributed in multiple directions and the distances vary greatly, it indicates that the Gaussian point may be in a boundary, occlusion, or sparse region. In this case, the parameter generation network should output a more conservative scale to prevent a single Gaussian point from covering too much space. Local descriptive information is the key difference between the method disclosed in this publication and methods that directly optimize geometric parameters R / S, that is, the geometric parameters R / S come from the local structure, rather than from independent drift of a single point.

[0044] By performing the above processing on each Gaussian point in the Gaussian point set of the first Gaussian splash model in this way, local descriptive information of each Gaussian point can be obtained.

[0045] In some possible implementations, in step S13, the geometric parameters of multiple Gaussian points are generated by a parameter generation network based on the basic parameters and local description information of the multiple Gaussian points. The parameter generation network can be a small, multi-layer neural network, such as a multilayer perceptron (MLP), a fully connected network, or a local attention network, or it can be replaced by a point cloud aggregation network. This disclosure does not limit the specific network structure of the parameter generation network.

[0046] In some possible implementations, the basic parameters and local description information of each Gaussian point can be used as the basis for recovering the geometric parameters R / S of the Gaussian point. These parameters are processed in the parameter generation network, and the corresponding geometric parameters of the Gaussian point are output. The geometric parameters output by the model may include scale information and rotation information; or they may include corrected scale information relative to the reference scale information and corrected rotation information relative to the reference rotation information, thereby further calculating the scale information and rotation information based on the corrected scale information and corrected rotation information. This disclosure does not limit the type of geometric parameters or the specific settings of the reference scale information and reference rotation information.

[0047] In some possible implementations, the geometric parameters of multiple Gaussian points are used to generate a rendered image of the first Gaussian splash model. That is, using the basic parameters of multiple Gaussian points and the generated geometric parameters, the first Gaussian splash model is rendered by a renderer to generate a rendered image of the target viewpoint for training or display.

[0048] According to embodiments of this disclosure, a neighboring point set can be determined by performing a neighboring point query on a target Gaussian point; local description information of the target Gaussian point can be determined based on the neighboring Gaussian points in the neighboring point set; and geometric parameters of multiple Gaussian points can be generated by a parameter generation network based on the basic parameters and local description information of multiple Gaussian points, so that the generated geometric parameters come from the local geometric structure, thereby improving the accuracy of the generated geometric parameters and improving the training and application effect of the 3DGS model.

[0049] The method for generating Gaussian parameters according to embodiments of this disclosure will now be described in detail.

[0050] As mentioned above, for the target Gaussian point in the Gaussian point set of the first Gaussian splash model, a neighbor point query can be performed on the target Gaussian point in step S11 to determine the neighbor point set of the target Gaussian point.

[0051] In some possible implementations, step S11 may include: performing a neighbor query on the target Gaussian point using a preset neighbor query method to determine the set of neighbor points of the target Gaussian point, wherein the neighbor query method includes any one of fixed number of neighbor queries, radius range neighbor queries, hierarchical neighbor queries, and spatial block neighbor queries.

[0052] In other words, based on the center coordinates of each Gaussian point in the Gaussian point set, a neighboring point query can be performed on the target Gaussian point to find the spatially closest Gaussian points, such as 8 neighboring Gaussian points. The numbers of the neighboring Gaussian points are recorded to obtain the set of neighboring points of the target Gaussian point.

[0053] The nearest neighbor query methods include any one of the following: fixed-number nearest neighbor query, radius-range nearest neighbor query, hierarchical nearest neighbor query, and spatial block nearest neighbor query. Fixed-number nearest neighbor query can be implemented using the k-Nearest Neighbors (kNN) algorithm, selecting the K Gaussian points closest to the target Gaussian point as neighboring Gaussian points. Radius-range nearest neighbor query can set a radius range, selecting Gaussian points within that radius as neighboring Gaussian points. Hierarchical nearest neighbor query can divide the 3D space containing the first Gaussian splash model into multiple spatial layers, selecting Gaussian points within the same spatial layer as the target Gaussian point as neighboring Gaussian points. Spatial block nearest neighbor query can first determine the spatial block containing the target Gaussian point, selecting Gaussian points within the same spatial block as neighboring Gaussian points.

[0054] It should be understood that those skilled in the art can set the nearest point query method according to the actual task requirements, and this disclosure does not impose any restrictions on this.

[0055] By employing various nearest-neighbor query methods, the adaptability of nearest-neighbor queries can be improved, thereby enhancing the accuracy of local descriptive information in representing local geometric structures.

[0056] In some possible implementations, step S11 may include: in the case of a change in the Gaussian point set of the first Gaussian splash model, performing a neighbor point query on the target Gaussian point in the changed Gaussian point set, and updating the neighbor point set of the target Gaussian point; wherein, the change in the Gaussian point set includes at least one of adding Gaussian points, deleting Gaussian points, and replacing Gaussian points.

[0057] For example, a change to a Gaussian point set refers to at least one of the following: addition, deletion, or replacement of Gaussian points. Adding Gaussian points includes methods such as splitting and cloning; deleting Gaussian points includes methods such as direct deletion, trimming, and compression; and replacing Gaussian points includes methods such as recombination and coordinate transformation. This disclosure does not limit the specific methods of changing a Gaussian point set.

[0058] When the Gaussian point set of the first Gaussian splash model changes, the basic parameters of the Gaussian points in the changed Gaussian point set can be determined first. The Gaussian parameter generation method according to embodiments of this disclosure further includes: when the target Gaussian point is an added Gaussian point, determining the basic parameters of the target Gaussian point through a preset parameter generation method, wherein the parameter generation method includes any one of the following: parent point inheritance method, neighborhood averaging method, random small perturbation method, and initialization network generation method.

[0059] In some possible implementations, the parent point inheritance method is used to indicate that the basic parameters of the target Gaussian point are determined based on the basic parameters of the Gaussian point copied from the target Gaussian point; the neighborhood averaging method is used to indicate that the basic parameters of the target Gaussian point are determined based on the average of the basic parameters of the neighboring Gaussian points in the set of neighboring points of the target Gaussian point.

[0060] In other words, for any Gaussian point in the modified Gaussian point set, i.e. the target Gaussian point, if the target Gaussian point is an added Gaussian point, the basic parameters of the target Gaussian point can be determined through a preset parameter generation method, including the center position coordinates, color information, opacity information, and low-dimensional features.

[0061] In some possible implementations, the parameter generation method is related to the way Gaussian points are added. If it is a copying method, the parameter generation method can adopt the parent point inheritance method. The parent point inheritance method is used to indicate the basic parameters of the target Gaussian point based on the basic parameters of the Gaussian point copied from the target Gaussian point. That is, the newly added point inherits the basic parameters of the parent point.

[0062] In this case, the parameter generation method can also adopt the neighborhood averaging method. The neighborhood averaging method is used to determine the basic parameters of the target Gaussian point based on the average of the basic parameters of the neighboring Gaussian points in the set of neighboring points of the target Gaussian point. For example, first, the set of neighboring points of the parent point of the target Gaussian point is determined as the set of neighboring points of the target Gaussian point; then, the average of the basic parameters of each neighboring Gaussian point in the set of neighboring points is determined as the basic parameters of the target Gaussian point. For example, the average of the center position coordinates of each neighboring Gaussian point is determined as the center position coordinates of the target Gaussian point, and the average of the low-dimensional features of each neighboring Gaussian point is determined as the low-dimensional features of the target Gaussian point.

[0063] In this case, the parameter generation method can also adopt the random small perturbation method, that is, add random small perturbation to the basic parameters of the parent point of the target Gaussian point to obtain the basic parameters of the target Gaussian point.

[0064] In some possible implementations, if the Gaussian points are added directly, the parameters can be generated using an initialization network. The basic parameters of the target Gaussian points are generated using a preset initialization network, and this disclosure does not restrict the specific network structure of the initialization network.

[0065] This method enables the initialization of basic parameters for newly added Gaussian points, improving the versatility and robustness of the 3DGS model.

[0066] In some possible implementations, if the Gaussian point set of the first Gaussian splash model changes, the neighbor query can be re-executed for all Gaussian points in the changed Gaussian point set, or the neighbor query can be re-executed for the Gaussian points in the affected area, and the neighbor set of the corresponding Gaussian point can be updated.

[0067] In this way, the accuracy of the neighboring Gaussian points included in the neighboring point set can be improved, thereby increasing the accuracy of the subsequent local description information in representing the local geometric structure of the Gaussian points.

[0068] The Gaussian parameter generation method according to embodiments of this disclosure is particularly suitable for handling situations where the Gaussian point set of a Gaussian splash model changes. In the training of a 3D Gaussian splash model, operations such as copying, splitting, adding, and pruning Gaussian points are frequently performed. If the geometric parameters of each Gaussian point are independent, the geometric parameters of newly added Gaussian points often require additional initialization, and then need to be gradually corrected through training. However, according to embodiments of this disclosure, as long as a newly added Gaussian point has basic parameters (center position coordinates, low-dimensional features, etc.) and can find its own set of neighboring points, the geometric parameters R / S can be recovered through the same parameter generation network. The basic parameters of the newly added Gaussian point can inherit from the parent point or be taken as the average value of neighborhood features, etc.; after the set of neighboring points is updated, the geometric parameters R / S will be recalculated according to the local structure, making the recombination, pruning, and recovery training of the Gaussian point set more natural, significantly improving training efficiency and training effect.

[0069] In some possible implementations, in step S12, the local description information of the target Gaussian point can be determined based on the basic parameters of the neighboring Gaussian points in the neighboring point set and the basic parameters of the target Gaussian point.

[0070] The basic parameters include the center position coordinates, which are used to determine the geometric relationship between Gaussian points; the regular rendering attributes of Gaussian points, such as color information and opacity information, which can be obtained by training in the conventional 3D Gaussian method; and low-dimensional features, which are feature vectors updated with training to record the local state of the Gaussian point.

[0071] The local description information includes at least the relative coordinates of the neighboring Gaussian points relative to the target Gaussian point, and the distance between the neighboring Gaussian points and the target Gaussian point. Both the relative coordinates and the distance can be calculated using the center coordinates of the neighboring Gaussian points and the center coordinates of the target Gaussian point.

[0072] The local description information further includes at least one of the following: the average feature of the low-dimensional features of the neighboring Gaussian points, the variance information of the low-dimensional features of the neighboring Gaussian points, the difference information between the low-dimensional features of the neighboring Gaussian points and the low-dimensional features of the target Gaussian point, the maximum value of the distance between the neighboring Gaussian points and the target Gaussian point, and the average value of the distance between the neighboring Gaussian points and the target Gaussian point.

[0073] The average feature of the low-dimensional features of neighboring Gaussian points can be obtained by averaging the corresponding elements of the low-dimensional feature vectors of multiple neighboring Gaussian points, resulting in a feature vector with the same dimension as the low-dimensional features. The variance information of the low-dimensional features of neighboring Gaussian points can be obtained by calculating the variance of the corresponding elements of the low-dimensional feature vectors of multiple neighboring Gaussian points, resulting in a feature vector. The difference information between the low-dimensional features of neighboring Gaussian points and the low-dimensional features of the target Gaussian point can be obtained by subtracting the low-dimensional feature vectors of multiple neighboring Gaussian points from the low-dimensional features of the target Gaussian point, resulting in a difference vector corresponding to the number of neighboring Gaussian points. The maximum value of the distance between a neighboring Gaussian point and the target Gaussian point can be obtained by directly comparing multiple distance values. The average value of the distance between a neighboring Gaussian point and the target Gaussian point can be obtained by averaging multiple distance values.

[0074] It should be understood that those skilled in the art can set the content and specific calculation method of the partial description information according to the actual situation, and this disclosure does not impose any restrictions on this.

[0075] By performing the above processing on each Gaussian point in the Gaussian point set of the first Gaussian splash model, local descriptive information of each Gaussian point can be obtained.

[0076] In some possible implementations, in step S13, the geometric parameters of multiple Gaussian points can be generated through a parameter generation network based on the fundamental parameters and local description information of these points. That is, the fundamental parameters and local description information of each Gaussian point can be used as the basis for recovering the geometric parameters R / S of the Gaussian point, input into the parameter generation network for processing, and the corresponding geometric parameters of the Gaussian point can be output. The geometric parameters output by the model may include scale information and rotation information; or may include corrected scale information relative to the reference scale information and corrected rotation information relative to the reference rotation information, thereby further calculating the scale information and rotation information based on the corrected scale information and corrected rotation information.

[0077] In some possible implementations, the Gaussian splash model can be trained in multiple rounds during the training scenario. The processing steps S11-S13 correspond to one round of training of the Gaussian splash model.

[0078] In this case, the first Gaussian splash model may include the Gaussian splash model trained in the previous round, and step S13 may include: for the target Gaussian point, inputting the basic parameters and local description information of the target Gaussian point into the parameter generation network to obtain the first geometric parameters of the target Gaussian point in the current round; determining the geometric parameters of the target Gaussian point in the current round based on the parameter weights of the first geometric parameters in the current round, the first geometric parameters, and the baseline geometric parameters, wherein the parameter weights are positively correlated with the number of training rounds.

[0079] For example, to ensure training stability, a "default first, then restore" approach can be adopted during training: In the early stages of training, conservative baseline geometric parameters are used as an aid, and the parameter weights of the geometric parameters generated by the parameter generation network are set to low; as the base parameters of the Gaussian points and the parameter generation network gradually stabilize, the parameter weights of the geometric parameters generated by the parameter generation network are increased, so that the final output geometric parameters are mainly recovered by the neighborhood of the Gaussian points and the parameter generation network. In this way, it is possible to avoid the parameter generation network directly outputting abnormal geometric parameters before it has learned the local structure, thereby improving the model training efficiency.

[0080] In some possible implementations, the parameter weights of the geometric parameters are positively correlated with the number of training epochs. For example, the parameter weight in the first training epoch is 0.1, and the parameter weight in the tenth training epoch is 0.2, etc. The baseline geometric parameters can be fixed default geometric parameters, or the geometric parameters generated in the current training epoch can be used as the baseline geometric parameters after a certain number of training epochs. This disclosure does not impose any restrictions on the specific mapping relationship between parameter weights and the number of training epochs, nor on the selection of default geometric parameters.

[0081] In some possible implementations, for a target Gaussian point, its basic parameters and local description information are input into a parameter generation network to obtain the first geometric parameters of the target Gaussian point in the current iteration. Then, based on the parameter weights of the first geometric parameters in the current iteration, the first geometric parameters themselves, and the reference geometric parameters, the geometric parameters of the target Gaussian point in the current iteration are determined. This involves a weighted sum of the first geometric parameters and the reference geometric parameters, and the result is used as the geometric parameters of the target Gaussian point in the current iteration.

[0082] This approach can improve the stability and efficiency of model training.

[0083] Specifically, when outputting scale information in the geometric parameters, the parameter generation network needs to provide scale values ​​in three dimensions, representing the range covered by the Gaussian ellipsoid corresponding to the Gaussian point in different directions. When outputting rotation information in the geometric parameters, the network needs to provide a rotation representation that conforms to the rules, enabling the Gaussian ellipsoid to rotate in the correct direction. Regardless of the numerical representation used, it must ultimately be converted into scale and rotation information that the renderer can use. The key point here is not which specific rotation representation or rotation encoding is used, but rather that the source of the geometric parameter R / S changes from a "point-by-point parameter table" to "neighborhood structure recovery".

[0084] To improve the stability of model training, range constraints can be added during training.

[0085] In some possible implementations, step S13 may include: for the target Gaussian point, inputting the basic parameters and local description information of the target Gaussian point into the parameter generation network to obtain the second geometric parameters of the target Gaussian point in the current round; adjusting the second geometric parameters according to the range constraint conditions to determine the geometric parameters of the target Gaussian point in the current round.

[0086] For example, for a target Gaussian point, the basic parameters and local description information of the target Gaussian point are input into the parameter generation network to obtain the second geometric parameters of the target Gaussian point in the current round; then, the second geometric parameters are adjusted according to the range constraints to determine the geometric parameters of the target Gaussian point in the current round.

[0087] In some possible implementations, the range constraints include at least one of the following: a single-wheel variation range constraint on geometric parameters, a maximum scale constraint on scale information, and a normalization constraint on rotation information.

[0088] The single-round variation limit is used to characterize the upper limit of the variation range of geometric parameters in each training round. If the variation range of the second geometric parameter generated by the parameter generation network in the current round exceeds the single-round variation limit relative to the geometric parameter determined in the previous round, the second geometric parameter is adjusted to be within the single-round variation limit to obtain the geometric parameter of the current round.

[0089] The maximum scale constraint of the scale information is used to characterize the maximum scale value of the Gaussian point in three dimensions. If the initial scale information generated by the parameter generation network in the current round exceeds the maximum scale constraint, the scale information is adjusted to within the maximum scale constraint to obtain the scale information of the current round.

[0090] The rotation information normalization constraint is used to characterize the normalization process performed on rotation information in each training round. That is, the initial rotation information generated by the parameter generation network in the current round is normalized to obtain the rotation information for the current round.

[0091] In some possible implementations, the range constraint can be combined with the parameter weights mentioned above, that is, the initial geometric parameters generated by the parameter generation network in the current round are weighted and summed with the baseline geometric parameters; the result is adjusted according to the range constraint to obtain the final output geometric parameters.

[0092] This approach can further improve the stability and efficiency of model training.

[0093] In the training scenario, after obtaining the geometric parameters of multiple Gaussian points of the first Gaussian splash model through steps S11-S13 in the current round of training, the current round of training can continue.

[0094] In some possible implementations, the Gaussian parameter generation method according to embodiments of this disclosure further includes: rendering the first Gaussian splash model based on the basic parameters and geometric parameters of the plurality of Gaussian points to obtain a rendered image of the target viewpoint; and training the first Gaussian splash model and the parameter generation network based on the rendered image and the sample image of the target viewpoint to obtain a second Gaussian splash model trained in the current round and a parameter generation network trained in the current round.

[0095] For example, a first Gaussian splash model can be rendered using the base parameters of multiple Gaussian points and the generated geometric parameters through a renderer, generating a rendered image from the target viewpoint. The network loss is calculated based on the difference between the rendered image and the sample image from the target viewpoint. Then, the network parameters of the parameter generation network are updated in reverse based on the network loss, while simultaneously updating the base parameters of the multiple Gaussian points of the first Gaussian splash model, resulting in the second Gaussian splash model trained in the current round and the parameter generation network trained in the current round, thus completing one round of training. The sample image is, for example, a real image. This disclosure does not limit the specific type of loss function used to calculate the network loss.

[0096] The training iterations for each round of the training process are described as follows: Each iteration first updates or reads the set of neighboring points based on the center coordinates of the current Gaussian point; then, based on the basic parameters of the current Gaussian point and the basic parameters of the neighboring Gaussian points in the set of neighboring points, the local description information of the current Gaussian point is statistically obtained; the basic parameters and local description information of the current Gaussian point are input into the parameter generation network to generate the geometric parameters of the current Gaussian point and restore the R / S information; the renderer uses the basic parameters of multiple Gaussian points and the restored geometric parameters to generate a rendered image; the network loss is calculated based on the difference between the rendered image and the sample image; then, the basic parameters of multiple Gaussian points of the Gaussian splash model and the network parameters of the parameter generation network are updated in reverse based on the network loss, completing one round of training iteration.

[0097] Since the network loss originates from the rendered image, the parameter generation network learns not the standard local fitting rules from geometry textbooks, but rather geometric parameter recovery rules that are useful for rendering quality. In other words, the parameter generation network learns: what kind of neighborhood structure should be recovered into what kind of geometric parameters in the current Gaussian splash model processing task.

[0098] In this way, the parameter generation network can gradually learn to recover suitable geometric parameters for rendering from the local neighborhood structure. Through multiple rounds of training iterations, a trained Gaussian splash model and a trained parameter generation network can be obtained.

[0099] This approach improves the training performance of 3DGS models and yields a parameter-sharing parameter generation network, thereby enhancing the accuracy of subsequently generated geometric parameters.

[0100] According to the Gaussian parameter generation method of the present disclosure, from the perspective of model storage, it can change the parameter storage method of 3DGS models. Related technologies require explicitly saving geometric parameters for each Gaussian point, which occupies a large amount of storage space. According to the embodiments of the present disclosure, only the basic parameters of the Gaussian points and the network parameters of the shared parameter generation network can be saved, while the geometric parameters can be obtained by neighbor point lookup and parameter generation network calculation during loading or rendering, thereby saving storage space. If compatibility with related 3D Gaussian renderers is required, geometric parameter restoration can also be performed once during the export stage, and the geometric parameter R / S output by the parameter generation network can be solidified and written into the related 3DGS model. In other words, according to the embodiments of the present disclosure, both the compact representation of "restoring geometric parameter R / S at runtime" and the compatible representation of "solidifying geometric parameter R / S during export" are supported.

[0101] According to the Gaussian parameter generation method of this disclosure, the geometric parameters R / S of a 3DGS model can be deferred from the point-by-point explicit optimization. Local descriptive information is constructed using the basic parameters of Gaussian points and neighbor point queries; then, the geometric parameters of each Gaussian point are recovered through a shared parameter generation network. The core of the solution according to this disclosure is not simply adding a parameter generation network, nor is it indiscriminately utilizing neighborhood information. Instead, it changes the generation location and dependencies of the geometric parameters R / S: and the geometric parameters R / S are no longer isolated point parameters, but rather geometric results recovered from the local point set structure.

[0102] According to the Gaussian parameter generation method of this disclosure, the basic parameters of Gaussian points in an existing or training 3DGS point set are maintained first, and the generation process of the geometric parameters of the Gaussian points is postponed. The generated geometric parameters are then recovered using a set of neighboring points and a shared parameter generation network. The generated geometric parameters better match the local geometric structure of the region where the Gaussian point is located, rather than each Gaussian point having its own isolated set of free geometric parameters. This allows the same set of recovery rules to be reused even when the Gaussian point set changes. After the point set changes, only the basic parameters of the Gaussian points and the set of neighboring points are needed to re-obtain the geometric parameters R / S that match the local structure. Project experiments show that the method according to the embodiments of this disclosure can significantly improve training quality and provide a more stable geometric parameter input for processes such as adding, deleting, pruning, reorganizing, compressing, and exporting point sets, as well as for short-range recovery training.

[0103] The Gaussian parameter generation method according to the embodiments of this disclosure can construct local description information for each Gaussian point, including relative coordinates, distance, average features of low-dimensional features of neighboring Gaussian points, and difference information of low-dimensional features between neighboring Gaussian points and the current Gaussian point, so that the subsequently generated geometric parameters R / S can reflect the structure of the surrounding point set; the local description information is mapped to geometric parameters using a parameter generation network shared across the entire scene, so that the same set of geometric parameter R / S recovery rules can be reused in cases such as adding points, copying points, splitting points, and recombining points; after the Gaussian point set is added, deleted, split, or clipped, the basic parameters of the Gaussian points and the set of neighboring points are updated synchronously, and then the geometric parameters R / S are restored again through the set of neighboring points and the parameter generation network, thereby improving the generation efficiency.

[0104] The Gaussian parameter generation method according to the embodiments of this disclosure can support two deployment modes: a deployment mode that retains the basic parameters and parameter generation network of Gaussian points and restores the geometric parameters R / S at runtime; and a deployment mode that solidifies the restored geometric parameters R / S into ordinary 3D Gaussian parameters during the export stage for compatibility with existing renderers.

[0105] Compared with the neural radiation field-assisted initialization scheme in related technologies, the embodiments of this disclosure do not only determine the initial center position coordinates and geometric parameters R / S of the Gaussian points, but also continuously use the local neighborhood context to generate or correct the geometric parameters R / S during 3DGS training and post-processing. Compared with the coordinate feature grid prediction and 4D anchor point MLP generation schemes in related technologies, the embodiments of this disclosure directly target the existing 3DGS point set, using the basic parameters of the Gaussian points, the local statistics of neighbor point queries, and the shared parameters to generate a network to handle the topology synchronization after the Gaussian point set changes. This is suitable for incremental training, reorganization, pruning, and compatible export processes in engineering, significantly improving the accuracy and consistency of the generated geometric parameters and enhancing the training and application effects of the 3DGS model.

[0106] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0107] In addition, this disclosure also provides a Gaussian parameter generation apparatus, an electronic device, and a computer-readable storage medium, all of which can be used to implement any of the Gaussian parameter generation methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding section of the method and will not be repeated here.

[0108] Figure 2 A block diagram of a Gaussian parameter generation device provided in an embodiment of this disclosure.

[0109] Reference Figure 2 This disclosure provides a Gaussian parameter generation apparatus, which includes:

[0110] The neighbor point determination module 21 is used to perform a neighbor point query on the target Gaussian point in the first Gaussian splash model and determine the neighbor point set of the target Gaussian point; the target Gaussian point is one of a plurality of Gaussian points in the first Gaussian splash model, and each Gaussian point has basic parameters, including position coordinates, color information, opacity information and low-dimensional features.

[0111] The local description determination module 22 is used to determine the local description information of the target Gaussian point based on the neighboring Gaussian points in the neighboring point set. The local description information includes the relative coordinates of the neighboring Gaussian points relative to the target Gaussian point, and the distance between the neighboring Gaussian points and the target Gaussian point.

[0112] The parameter generation module 23 is used to generate the geometric parameters of the multiple Gaussian points through a parameter generation network based on the basic parameters and local description information of the multiple Gaussian points; wherein, the geometric parameters of the multiple Gaussian points are used to generate the rendered image of the first Gaussian splash model.

[0113] In some possible implementations, the first Gaussian splash model includes the Gaussian splash model trained in the previous round; the geometric parameters include scale information and rotation information; or the geometric parameters include corrected scale information relative to the reference scale information and corrected rotation information relative to the reference rotation information; the device further includes: a rendering module, used to render the first Gaussian splash model according to the basic parameters and geometric parameters of the plurality of Gaussian points to obtain a rendered image of the target viewpoint.

[0114] The training module is used to train the first Gaussian splash model and the parameter generation network based on the rendered image and the sample image from the target viewpoint, so as to obtain the second Gaussian splash model and the parameter generation network trained in the current round.

[0115] In some possible implementations, the first Gaussian splash model includes the Gaussian splash model trained in the previous round; the parameter generation module is used to: for the target Gaussian point, input the basic parameters and local description information of the target Gaussian point into the parameter generation network to obtain the first geometric parameters of the target Gaussian point in the current round; determine the geometric parameters of the target Gaussian point in the current round based on the parameter weights of the first geometric parameters in the current round, the first geometric parameters, and the baseline geometric parameters, wherein the parameter weights are positively correlated with the number of training rounds.

[0116] In some possible implementations, the first Gaussian splash model includes a Gaussian splash model trained in the previous round, and a parameter generation module, used to: for the target Gaussian point, input the basic parameters and local description information of the target Gaussian point into the parameter generation network to obtain the second geometric parameters of the target Gaussian point in the current round; adjust the second geometric parameters according to range constraints to determine the geometric parameters of the target Gaussian point in the current round, wherein the range constraints include at least one of a single-round variation range constraint of the geometric parameters, a maximum scale constraint of the scale information, and a normalization constraint of the rotation information.

[0117] In some possible implementations, the neighbor point determination module is used to: perform a neighbor point query on the target Gaussian point using a preset neighbor point query method to determine the set of neighbor points of the target Gaussian point, wherein the neighbor point query method includes any one of fixed number of neighbor queries, radius range neighbor queries, hierarchical neighbor queries, and spatial block neighbor queries.

[0118] In some possible implementations, the neighbor point determination module is used to: perform a neighbor point query on the target Gaussian point in the changed Gaussian point set when the Gaussian point set of the first Gaussian splash model changes, and update the neighbor point set of the target Gaussian point; wherein the change in the Gaussian point set includes at least one of Gaussian point addition, Gaussian point deletion, and Gaussian point replacement.

[0119] In some possible implementations, the device further includes: a basic parameter generation module, configured to: determine the basic parameters of the target Gaussian point by means of a preset parameter generation method when the target Gaussian point is an added Gaussian point, wherein the parameter generation method includes any one of the following: parent point inheritance method, neighborhood averaging method, random small perturbation method, and initialization network generation method, wherein the parent point inheritance method is used to indicate that the basic parameters of the target Gaussian point are determined based on the basic parameters of the Gaussian point copied from the target Gaussian point; the neighborhood averaging method is used to indicate that the basic parameters of the target Gaussian point are determined based on the average of the basic parameters of the neighboring Gaussian points in the neighboring point set of the target Gaussian point.

[0120] In some possible implementations, the basic parameters also include color information, opacity information, and low-dimensional features; the local description information also includes at least one of the following: the average feature of the low-dimensional features of the neighboring Gaussian points, the variance information of the low-dimensional features of the neighboring Gaussian points, the difference information between the low-dimensional features of the neighboring Gaussian points and the low-dimensional features of the target Gaussian point, the maximum value of the distance between the neighboring Gaussian points and the target Gaussian point, and the average value of the distance between the neighboring Gaussian points and the target Gaussian point.

[0121] Figure 3This is a block diagram of an electronic device provided in an embodiment of the present disclosure.

[0122] Reference Figure 3 This disclosure provides an electronic device, which includes: at least one processor 701; at least one memory 702; and one or more I / O interfaces 703 connected between the processor 701 and the memory 702; wherein the memory 702 stores one or more computer programs that can be executed by the at least one processor 701, and the one or more computer programs are executed by the at least one processor 701 to enable the at least one processor 701 to perform the above-described Gaussian parameter generation method.

[0123] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the above-described Gaussian parameter generation method. The computer-readable storage medium may be volatile or non-volatile.

[0124] This disclosure also provides a computer program product including computer-readable code, which, when run in a processor of an electronic device, executes the above-described Gaussian parameter generation method.

[0125] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0126] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0127] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0128] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0129] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0130] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0131] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0132] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0133] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0134] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for general illustrative purposes only and should not be construed as limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A method for generating Gaussian parameters, characterized in that, include: A neighbor point query is performed on the target Gaussian point in the first Gaussian splash model to determine the set of neighbor points of the target Gaussian point; the target Gaussian point is one of multiple Gaussian points in the first Gaussian splash model, and each Gaussian point has basic parameters, including the center position coordinates; Based on the neighboring Gaussian points in the neighboring point set, the local description information of the target Gaussian point is determined. The local description information includes the relative coordinates of the neighboring Gaussian points relative to the target Gaussian point, and the distance between the neighboring Gaussian points and the target Gaussian point. Based on the basic parameters and local description information of the plurality of Gaussian points, the geometric parameters of the plurality of Gaussian points are generated by a parameter generation network; wherein, the geometric parameters of the plurality of Gaussian points are used to generate the rendered image of the first Gaussian splash model.

2. The method according to claim 1, characterized in that, The first Gaussian splash model includes the Gaussian splash model trained in the previous round; the geometric parameters include scale information and rotation information; Alternatively, the geometric parameters may include corrected scale information relative to the reference scale information and corrected rotation information relative to the reference rotation information; The method further includes: Based on the basic parameters and geometric parameters of the multiple Gaussian points, the first Gaussian splash model is rendered to obtain a rendered image from the target perspective; Based on the rendered image and the sample image from the target viewpoint, the first Gaussian splash model and the parameter generation network are trained to obtain the second Gaussian splash model and the parameter generation network trained in the current round.

3. The method according to claim 1, characterized in that, The first Gaussian splash model includes the Gaussian splash model trained in the previous round; The step of generating geometric parameters for the plurality of Gaussian points using a parameter generation network based on the basic parameters and local description information of the plurality of Gaussian points includes: For the target Gaussian point, the basic parameters and local description information of the target Gaussian point are input into the parameter generation network to obtain the first geometric parameters of the target Gaussian point in the current round; Based on the parameter weights of the first geometric parameter in the current round, the first geometric parameter, and the baseline geometric parameter, the geometric parameters of the target Gaussian point in the current round are determined, wherein the parameter weights are positively correlated with the number of training rounds.

4. The method according to claim 1, characterized in that, The first Gaussian splash model includes the Gaussian splash model trained in the previous round. The step of generating geometric parameters for the plurality of Gaussian points using a parameter generation network based on the basic parameters and local description information of the plurality of Gaussian points includes: For the target Gaussian point, the basic parameters and local description information of the target Gaussian point are input into the parameter generation network to obtain the second geometric parameters of the target Gaussian point in the current round; The second geometric parameter is adjusted according to the range constraint conditions to determine the geometric parameters of the target Gaussian point in the current wheel. The range constraint conditions include at least one of the following: a single wheel variation range constraint of the geometric parameter, a maximum scale constraint of the scale information, and a normalization constraint of the rotation information.

5. The method according to claim 1, characterized in that, The step of performing a neighbor point query on the target Gaussian point in the first Gaussian splash model to determine the set of neighbor points of the target Gaussian point includes: By using a preset neighbor query method, a neighbor query is performed on the target Gaussian point to determine the set of neighbor points of the target Gaussian point. The nearest neighbor query method includes any one of the following: fixed number of nearest neighbor queries, radius range nearest neighbor queries, hierarchical nearest neighbor queries, and spatial block nearest neighbor queries.

6. The method according to claim 1, characterized in that, The step of performing a neighbor point query on the target Gaussian point in the first Gaussian splash model to determine the set of neighbor points of the target Gaussian point includes: In the case of a change in the Gaussian point set of the first Gaussian splash model, a neighbor point query is performed on the target Gaussian point in the changed Gaussian point set, and the neighbor point set of the target Gaussian point is updated. The change in the Gaussian point set includes at least one of Gaussian point addition, Gaussian point deletion, and Gaussian point replacement.

7. The method according to claim 6, characterized in that, The method further includes: When the target Gaussian point is an added Gaussian point, the basic parameters of the target Gaussian point are determined by a preset parameter generation method. The parameter generation method includes any one of the following: parent point inheritance method, neighborhood averaging method, random small perturbation method, and initialization network generation method. The parent point inheritance method is used to indicate that the basic parameters of the target Gaussian point are determined based on the basic parameters of the Gaussian point copied from the target Gaussian point; the neighborhood averaging method is used to indicate that the basic parameters of the target Gaussian point are determined based on the average of the basic parameters of the neighboring Gaussian points in the neighboring point set of the target Gaussian point.

8. The method according to claim 1, characterized in that, The basic parameters also include color information, opacity information, and low-dimensional features; The local description information further includes at least one of the following: the average feature of the low-dimensional features of the neighboring Gaussian points, the variance information of the low-dimensional features of the neighboring Gaussian points, the difference information between the low-dimensional features of the neighboring Gaussian points and the low-dimensional features of the target Gaussian point, the maximum value of the distance between the neighboring Gaussian points and the target Gaussian point, and the average value of the distance between the neighboring Gaussian points and the target Gaussian point.

9. A Gaussian parameter generation device, characterized in that, include: The neighbor point determination module is used to perform a neighbor point query on the target Gaussian point in the first Gaussian splash model and determine the neighbor point set of the target Gaussian point; the target Gaussian point is one of multiple Gaussian points in the first Gaussian splash model, and each Gaussian point has basic parameters, including position coordinates, color information, opacity information and low-dimensional features; The local description determination module is used to determine the local description information of the target Gaussian point based on the neighboring Gaussian points in the neighboring point set. The local description information includes the relative coordinates of the neighboring Gaussian points relative to the target Gaussian point, and the distance value between the neighboring Gaussian points and the target Gaussian point. The parameter generation module is used to generate the geometric parameters of the multiple Gaussian points through a parameter generation network based on the basic parameters and local description information of the multiple Gaussian points; wherein, the geometric parameters of the multiple Gaussian points are used to generate the rendered image of the first Gaussian splash model.

10. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the Gaussian parameter generation method as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the Gaussian parameter generation method as described in any one of claims 1-8.

Images