Dynamic scene reconstruction method and device based on multi-scale Gaussian sphere

Abstract

The invention discloses a dynamic scene reconstruction method and device based on a multi-scale Gaussian sphere, and relates to the field of computer vision, and the method comprises the steps: employing a motion recovery structure algorithm to carry out the processing of a to-be-reconstructed video frame sequence, generating a sparse point cloud, carrying out the initialization of the sparse point cloud, and generating a 3D Gaussian sphere set; processing the 3D Gaussian ball set by adopting a double-domain deformation model and an adaptive timestamp to obtain a deformed 3D Gaussian ball set; performing multi-scale Gaussian processing on the deformed 3D Gaussian ball set to generate a multi-scale Gaussian ball set; gaussian screening based on the pixel coverage rate is carried out on the multi-scale Gaussian ball set, and an optimized multi-scale Gaussian ball set is obtained; and performing Alpha hybrid processing based on the optimized multi-scale Gaussian ball set, and reconstructing to obtain an anti-aliasing dynamic rendering scene image. According to the method, the problems of high calculation overhead, aliasing effect and the like of the current dynamic scene reconstruction are solved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a dynamic scene reconstruction method and device based on a multi-scale Gaussian sphere. Background Art

[0002] The 3D reconstruction and real-time rendering of dynamic scenes are core challenges in the field of computer vision and graphics, especially in application scenarios such as virtual reality, autonomous driving, and film and television special effects. Traditional dynamic scene reconstruction technology mainly relies on representation methods based on voxels, point clouds, or grids, combined with motion estimation methods such as optical flow and non-rigid registration to achieve dynamic modeling. However, such methods generally have problems such as high computational complexity, difficulty in handling rapid non-rigid deformations, and aliasing effects caused by loss of high-frequency details.

[0003] The recently proposed 3D Gaussian Splatting technology can achieve real-time rendering speed by parameterizing scenes with explicit Gaussian primitives. However, its native framework has a major problem in dynamic scenes. When directly expanded to a time series model, the exponential growth of the number of Gaussian primitives leads to a sharp increase in video memory usage and computational overhead. On the other hand, existing anti-aliasing solutions mostly use pre-filtering or post-processing super-resolution technology, which cannot achieve frequency domain adaptive detail preservation in the reconstruction stage and will introduce additional computational load.

[0004] Therefore, how to reduce the computational overhead of 3D Gaussian splashing in dynamic scene reconstruction and how to suppress the aliasing effect have become urgent issues to be solved in this field. Summary of the invention

[0005] The purpose of this application is to propose a dynamic scene reconstruction method and device based on a multi-scale Gaussian sphere to address the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a dynamic scene reconstruction method based on a multi-scale Gaussian sphere, comprising the following steps:

[0007] Obtain a video frame sequence to be reconstructed, use a structure-from-motion algorithm to process the video frame sequence to be reconstructed, generate a sparse point cloud, initialize the sparse point cloud, and generate a 3D Gaussian sphere set;

[0008] The 3D Gaussian sphere set is processed by using a dual-domain deformation model and an adaptive time stamp to obtain a deformed 3D Gaussian sphere set;

[0009] Perform multi-scale Gaussian processing on the deformed 3D Gaussian sphere set to generate a multi-scale Gaussian sphere set; perform Gaussian screening based on pixel coverage on the multi-scale Gaussian sphere set to obtain an optimized multi-scale Gaussian sphere set;

[0010] The optimized multi-scale Gaussian sphere set is projected onto a two-dimensional plane to generate the corresponding 2D Gaussian distribution on the two-dimensional plane, and the optimized multi-scale Gaussian sphere set contained in a single pixel point of the 2D Gaussian on the two-dimensional plane determined by the distribution of the 2D Gaussian on the two-dimensional plane is subjected to Alpha blending processing to reconstruct an anti-aliased dynamic rendering scene image.

[0011] Preferably, a structure-from-motion algorithm is used to process the video frame sequence to be reconstructed, generate a sparse point cloud, initialize the sparse point cloud, and generate a 3D Gaussian sphere set, specifically including:

[0012] The video frame sequence to be reconstructed and the known camera parameters are input into the motion recovery structure algorithm, and a set of sparse point clouds P with position information and color information is output. cloud and its corresponding camera pose;

[0013] For the sparse point cloud P cloud Each point in is initialized as a 3D Gaussian ball G, and each 3D Gaussian ball is defined as:

[0014] ;

[0015] Among them, G(x) represents the function corresponding to the 3D Gaussian sphere, x represents a three-dimensional space point in the 3D Gaussian sphere, T represents transposition, is the center position of the 3D Gaussian sphere, is the covariance matrix of the 3D Gaussian sphere, which can be obtained by parameterizing the scaling vector s and the rotation quaternion q, expressed as: ; is the scaling matrix converted by scaling vector s, R is the rotation matrix converted by rotating quaternion q;

[0016] Finally, a 3D Gaussian sphere set is constructed. , represents the i-th 3D Gaussian sphere.

[0017] Preferably, a dual-domain deformation model and an adaptive timestamp are used to process the 3D Gaussian sphere set to obtain a deformed 3D Gaussian sphere set, specifically including:

[0018] An adaptive timestamp is introduced to scale the normalized frame index time of each 3D Gaussian ball in the 3D Gaussian ball set to obtain the scaled time, as shown in the following formula:

[0019] ;

[0020] in, represents the time after scaling, represents the normalized frame index time, and They represent the time scaling factor and basic factor of the 3D Gaussian sphere respectively;

[0021] Based on the time-varying residual after the input scaled time and the dual-domain deformation model, the basic properties of each 3D Gaussian sphere at the reference time are dynamically modeled, and the dynamic properties after deformation with the scaled time are obtained, as shown in the following formula:

[0022] ;

[0023] Among them, S(t s ) is the time after scaling of the 3D Gaussian sphere The dynamic properties after deformation, S0 is the basic properties of the 3D Gaussian ball at the reference time t0, including the center position, rotation information and color information, and the time-varying residual D(t s ) is fitted by the time domain polynomial and the frequency domain Fourier series, as shown in the following formula:

[0024] ;

[0025] in, is an N-order polynomial, n represents the nth order in the polynomial, are the polynomial coefficients of the nth order, is the L-order Fourier series, l represents the l-th order in the Fourier series, , are the sine Fourier coefficients and cosine Fourier coefficients of the lth order;

[0026] Through the 3D Gaussian sphere with time after scaling The dynamic properties after deformation get the deformed 3D Gaussian sphere , and construct a deformed 3D Gaussian sphere set , represents the i-th deformed 3D Gaussian sphere.

[0027] Preferably, multi-scale Gaussian processing is performed on the deformed 3D Gaussian sphere set to generate a multi-scale Gaussian sphere set, specifically including:

[0028] Project each deformed 3D Gaussian sphere in the deformed 3D Gaussian sphere set into the 2D Gaussian corresponding to the deformed 3D Gaussian sphere , and calculate the pixel coverage of the 2D Gaussian corresponding to the deformed 3D Gaussian ball in the screen space, as shown in the following formula:

[0029] ;

[0030] ;

[0031] ;

[0032] Among them, μ k Indicates the center position of the 2D Gaussian corresponding to the deformed 3D Gaussian ball, V k represents the covariance matrix of the 2D Gaussian corresponding to the deformed 3D Gaussian sphere, u represents the horizontal axis length of the 2D Gaussian corresponding to the deformed 3D Gaussian sphere in the plane, and v represents the vertical axis length of the 2D Gaussian corresponding to the deformed 3D Gaussian sphere in the plane. represents the opacity of the deformed 3D Gaussian sphere, and They represent the horizontal and vertical coordinate components of the deformed 3D Gaussian sphere projected onto the two-dimensional plane, Represents the preset Gaussian opacity threshold, It means taking the minimum value of the two, S k Indicates the pixel coverage of the 2D Gaussian corresponding to the deformed 3D Gaussian sphere in screen space;

[0033] Screening out deformed 3D Gaussian spheres whose pixel coverage is less than a pixel coverage threshold from the deformed 3D Gaussian sphere set, obtaining screened Gaussian spheres and constructing the screened Gaussian sphere set;

[0034] All the filtered Gaussian balls in the filtered Gaussian ball set are divided into voxel grids according to spatial positions, and multiple filtered Gaussian balls in each voxel of the voxel grid are averaged and pooled to generate an aggregated Gaussian ball, as shown in the following formula:

[0035] ;

[0036] ;

[0037] ;

[0038] Among them, G m is the set of filtered Gaussian balls within the mth voxel, Represents G m The number of Gaussian balls after screening in, , and G m The center position, scaling information and color information of the filtered Gaussian sphere, , and They are the center position, scaling information, and color information of the aggregated Gaussian sphere. represents the pixel coverage threshold, is the average pixel coverage of the set of filtered Gaussian spheres in each voxel;

[0039] The aggregated Gaussian sphere or the deformed 3D Gaussian sphere whose pixel coverage is greater than or equal to the pixel coverage threshold is used as a multi-scale Gaussian sphere, and a multi-scale Gaussian sphere set is constructed. As a preferred method, Gaussian screening based on pixel coverage is performed on the multi-scale Gaussian sphere set to obtain an optimized multi-scale Gaussian sphere set, which specifically includes:

[0040] According to the current rendering resolution, filter out the multi-scale Gaussian spheres that meet the conditions from the multi-scale Gaussian sphere set. The screening conditions are:

[0041] ;

[0042] in, Represents a set of multi-scale Gaussian balls The pixel coverage of a single multi-scale Gaussian sphere in , represents the pixel coverage threshold, and A collection of multi-scale Gaussian spheres at different rendering resolutions The maximum pixel coverage and minimum pixel coverage of the multi-scale Gaussian sphere, is the relative maximum pixel coverage threshold, is the relative minimum pixel coverage threshold, and Represents "and" and "or" respectively;

[0043] The optimized multi-scale Gaussian spheres suitable for rendering at the current resolution are filtered out using the filtering conditions and a set of optimized multi-scale Gaussian spheres is constructed.

[0044] Preferably, the optimized multi-scale Gaussian ball set is projected onto a two-dimensional plane to generate a corresponding 2D Gaussian distribution on the two-dimensional plane, and an Alpha blending process is performed on the optimized multi-scale Gaussian ball set contained in a single pixel point of a 2D Gaussian on the two-dimensional plane determined by the distribution of the 2D Gaussian on the two-dimensional plane to reconstruct an anti-aliased dynamic rendering scene image, specifically including:

[0045] The projection formula is used to project each optimized multi-scale Gaussian ball in the optimized multi-scale Gaussian ball set from 3D to 2D, and the center position and covariance matrix of the 2D Gaussian corresponding to the optimized multi-scale Gaussian ball are obtained, and the distribution of the 2D Gaussian corresponding to the optimized multi-scale Gaussian ball on the two-dimensional plane is determined, as shown in the following formula:

[0046] ;

[0047] ;

[0048] in, is the center position of the 2D Gaussian corresponding to the optimized multi-scale Gaussian ball, is the covariance matrix of the 2D Gaussian corresponding to the optimized multi-scale Gaussian sphere, P is the projection matrix, W is the view transformation matrix, J is the Jacobian matrix of the affine approximation, represents the center position of a single optimized multi-scale Gaussian ball in the set of optimized multi-scale Gaussian balls, represents the covariance matrix of a single optimized multi-scale Gaussian ball in the optimized multi-scale Gaussian ball set, and T represents the transpose;

[0049] Determine the set of optimized multi-scale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane according to the distribution of the 2D Gaussian balls corresponding to the optimized multi-scale Gaussian balls on the two-dimensional plane;

[0050] The color and opacity of each optimized multi-scale Gaussian ball in the set of optimized multi-scale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane are input into the Alpha blending formula to obtain the final color information of a single pixel point of the 2D Gaussian on the two-dimensional plane, as shown in the following formula:

[0051] ;

[0052] Among them, C is the final color information of a single pixel, is the color of the pth optimized multi-scale Gaussian ball in the set of optimized multi-scale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane, is the opacity of the pth optimized multi-scale Gaussian ball in the set of optimized multi-scale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane, b is the number of optimized multi-scale Gaussian balls in the set of optimized multi-scale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane, represents the cumulative transmittance of all optimized multiscale Gaussian balls before the p-th optimized multiscale Gaussian ball in the set of optimized multiscale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane, j is the index of all optimized multiscale Gaussian balls before the p-th optimized multiscale Gaussian ball in the set of optimized multiscale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane, is the opacity of the jth optimized multi-scale Gaussian ball in the set of optimized multi-scale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane;

[0053] The anti-aliased dynamic rendering scene image is constructed through the final color information of all pixels.

[0054] In a second aspect, the present invention provides a dynamic scene reconstruction device based on a multi-scale Gaussian sphere, comprising:

[0055] An initialization module is configured to obtain a video frame sequence to be reconstructed, process the video frame sequence to be reconstructed using a structure-from-motion algorithm, generate a sparse point cloud, initialize the sparse point cloud, and generate a 3D Gaussian sphere set;

[0056] A deformation module is configured to process the 3D Gaussian sphere set using a dual-domain deformation model and an adaptive timestamp to obtain a deformed 3D Gaussian sphere set;

[0057] The multi-scale Gaussian module is configured to perform multi-scale Gaussian processing on the deformed 3D Gaussian sphere set to generate a multi-scale Gaussian sphere set; perform Gaussian screening based on pixel coverage on the multi-scale Gaussian sphere set to obtain an optimized multi-scale Gaussian sphere set;

[0058] The reconstruction module is configured to project the optimized multi-scale Gaussian sphere set onto a two-dimensional plane, generate a corresponding 2D Gaussian distribution on the two-dimensional plane, and perform Alpha blending processing on the optimized multi-scale Gaussian sphere set contained in a single pixel point of the 2D Gaussian on the two-dimensional plane determined by the distribution of the 2D Gaussian on the two-dimensional plane, so as to reconstruct an anti-aliased dynamic rendering scene image.

[0059] In a third aspect, the present invention provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0060] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0061] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] (1) The dynamic scene reconstruction method based on multi-scale Gaussian sphere proposed in this paper uses a dual-domain deformation module to model the attributes (such as position, rotation and color) of each Gaussian sphere in both the time domain and the frequency domain, thereby reducing the computational overhead. And through the combination of polynomials and Fourier series, the dual-domain deformation model can capture complex motion trajectories without increasing too much computational complexity.

[0064] (2) The dynamic scene reconstruction method based on multi-scale Gaussian spheres proposed in the present invention uses multi-scale Gaussian processing to screen out the optimized multi-scale Gaussian sphere set that is most suitable for the current rendering resolution for rendering, thereby avoiding too many Gaussian spheres in a pixel point, so that the color information of the pixel point is only determined by the color information of the Gaussian spheres sorted in front, thereby causing an aliasing effect, so that a relatively good reconstruction effect can be shown at different rendering resolutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0066] Figure 1 A schematic diagram of a flow chart of a dynamic scene reconstruction method based on a multi-scale Gaussian sphere according to an embodiment of the present application;

[0067] Figure 2 A schematic diagram of a dynamic scene reconstruction device based on a multi-scale Gaussian sphere according to an embodiment of the present application;

[0068] Figure 3 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0070] Figure 1 A dynamic scene reconstruction method based on a multi-scale Gaussian sphere provided in an embodiment of the present application is shown, comprising the following steps:

[0071] S1, obtaining a video frame sequence to be reconstructed, using a structure-from-motion algorithm to process the video frame sequence to be reconstructed, generating a sparse point cloud, initializing the sparse point cloud, and generating a 3D Gaussian sphere set.

[0072] In a specific embodiment, a structure-from-motion algorithm is used to process a video frame sequence to be reconstructed, a sparse point cloud is generated, the sparse point cloud is initialized, and a 3D Gaussian sphere set is generated, specifically including:

[0073] The video frame sequence to be reconstructed and the known camera parameters are input into the motion recovery structure algorithm, and a set of sparse point clouds P with position information and color information is output. cloud and its corresponding camera pose;

[0074] For the sparse point cloud P cloud Each point in is initialized as a 3D Gaussian ball G, and each 3D Gaussian ball is defined as:

[0075] ;

[0076] Among them, G(x) represents the function corresponding to the 3D Gaussian sphere, x represents a three-dimensional space point in the 3D Gaussian sphere, T represents transposition, is the center position of the 3D Gaussian sphere, is the covariance matrix of the 3D Gaussian sphere, which can be obtained by parameterizing the scaling vector s and the rotation quaternion q, expressed as: ; is the scaling matrix converted by scaling vector s, R is the rotation matrix converted by rotating quaternion q;

[0077] Finally, a 3D Gaussian sphere set is constructed. , represents the i-th 3D Gaussian sphere.

[0078] Specifically, the video frame sequence and known camera parameters are input into the motion recovery structure algorithm (SFM), and a set of sparse point clouds P with position information and color information is output. cloud And the corresponding camera pose, for the sparse point cloud P cloud Initialize and generate a 3D Gaussian ball set for the sparse point cloud P cloud Each point in is initialized as a 3D Gaussian sphere. The function corresponding to the 3D Gaussian sphere is used to describe the probability density of point x in the Gaussian distribution; the x in the function corresponding to the 3D Gaussian sphere is used to describe the above sparse point cloud P cloud The position of a single point; the rotation quaternion q is a mathematical tool used to represent three-dimensional rotation. The rotation quaternion can be expressed as , where w q represents half of the cosine of the rotation angle, and x q Represents the component of the rotation axis on the x-axis, y q represents the component of the rotation axis on the y-axis, z q represents the component of the rotation axis on the z-axis, w q is the real part, Is the imaginary part. The formula for converting the rotation quaternion q to the rotation matrix R is as follows:

[0079] .

[0080] S2, the 3D Gaussian sphere set is processed using a dual-domain deformation model and an adaptive timestamp to obtain a deformed 3D Gaussian sphere set.

[0081] In a specific embodiment, step S2 specifically includes:

[0082] An adaptive timestamp is introduced to scale the normalized frame index time of each 3D Gaussian ball in the 3D Gaussian ball set to obtain the scaled time, as shown in the following formula:

[0083] ;

[0084] in, represents the time after scaling, represents the normalized frame index time, and They represent the time scaling factor and basic factor of the 3D Gaussian sphere respectively;

[0085] Based on the time-varying residual after the input scaled time and the dual-domain deformation model, the basic properties of each 3D Gaussian sphere at the reference time are dynamically modeled, and the dynamic properties after deformation with the scaled time are obtained, as shown in the following formula:

[0086] ;

[0087] Among them, S(t s ) is the time after scaling of the 3D Gaussian sphere The dynamic properties after deformation, S0 is the basic properties of the 3D Gaussian ball at the reference time t0, including the center position, rotation information and color information, and the time-varying residual D(t s ) is fitted by the time domain polynomial and the frequency domain Fourier series, as shown in the following formula:

[0088] ;

[0089] in, is an N-order polynomial, n represents the nth order in the polynomial, are the polynomial coefficients of the nth order, is the L-order Fourier series, l represents the l-th order in the Fourier series, , are the sine Fourier coefficients and cosine Fourier coefficients of the lth order;

[0090] Through the 3D Gaussian sphere with time after scaling The dynamic properties after deformation get the deformed 3D Gaussian sphere , and construct a deformed 3D Gaussian sphere set , represents the i-th deformed 3D Gaussian sphere.

[0091] Specifically, in the embodiment of the present application, a dual-domain deformation model is used to transform each 3D Gaussian sphere The basic properties of S0 (center position , rotation information q0, color information c0) for dynamic change modeling, and the dual-domain deformation model decomposes the motion into low-frequency trend terms and high-frequency detail terms. As a low-frequency trend term, the time domain polynomial describes the overall trajectory of the object's motion through a low-order polynomial, while as a high-frequency detail term, the frequency domain Fourier series captures periodic or non-rigid deformations through the superposition of the fundamental frequency and harmonics. This frequency division modeling method significantly improves the expressiveness of complex motion trajectories and reduces computational consumption compared to the original sampling processing of each video frame. At the same time, in order to balance the overfitting problem of violent motion, an adaptive timestamp is introduced to scale the time input of each 3D Gaussian sphere.

[0092] S3, performing multi-scale Gaussian processing on the deformed 3D Gaussian sphere set to generate a multi-scale Gaussian sphere set; performing Gaussian screening based on pixel coverage on the multi-scale Gaussian sphere set to obtain an optimized multi-scale Gaussian sphere set.

[0093] In a specific embodiment, performing multi-scale Gaussian processing on the deformed 3D Gaussian sphere set to generate a multi-scale Gaussian sphere set specifically includes:

[0094] Project each deformed 3D Gaussian sphere in the deformed 3D Gaussian sphere set into the 2D Gaussian corresponding to the deformed 3D Gaussian sphere , and calculate the pixel coverage of the 2D Gaussian corresponding to the deformed 3D Gaussian ball in the screen space, as shown in the following formula:

[0095] ;

[0096] ;

[0097] ;

[0098] Among them, μ k Indicates the center position of the 2D Gaussian corresponding to the deformed 3D Gaussian ball, V k represents the covariance matrix of the 2D Gaussian corresponding to the deformed 3D Gaussian sphere, u represents the horizontal axis length of the 2D Gaussian corresponding to the deformed 3D Gaussian sphere in the plane, and v represents the vertical axis length of the 2D Gaussian corresponding to the deformed 3D Gaussian sphere in the plane. represents the opacity of the deformed 3D Gaussian sphere, and They represent the horizontal and vertical coordinate components of the deformed 3D Gaussian sphere projected onto the two-dimensional plane, Represents the preset Gaussian opacity threshold, It means taking the minimum value of the two, S k Indicates the pixel coverage of the 2D Gaussian corresponding to the deformed 3D Gaussian sphere in screen space;

[0099] Screening out deformed 3D Gaussian spheres whose pixel coverage is less than a pixel coverage threshold from the deformed 3D Gaussian sphere set, obtaining screened Gaussian spheres and constructing the screened Gaussian sphere set;

[0100] All the filtered Gaussian balls in the filtered Gaussian ball set are divided into voxel grids according to spatial positions, and multiple filtered Gaussian balls in each voxel of the voxel grid are averaged and pooled to generate an aggregated Gaussian ball, as shown in the following formula:

[0101] ;

[0102] ;

[0103] ;

[0104] Among them, G m is the set of filtered Gaussian balls within the mth voxel, Represents G m The number of Gaussian balls after screening in, , and G m The center position, scaling information and color information of the filtered Gaussian sphere, , and They are the center position, scaling information, and color information of the aggregated Gaussian sphere. represents the pixel coverage threshold, is the average pixel coverage of the set of filtered Gaussian spheres in each voxel;

[0105] The aggregated Gaussian sphere or the deformed 3D Gaussian sphere whose pixel coverage is greater than or equal to the pixel coverage threshold is used as a multi-scale Gaussian sphere, and a multi-scale Gaussian sphere set is constructed. Specifically, in one embodiment, the preset Gaussian opacity threshold , other values ​​may be selected in other embodiments. The horizontal axis length u and the vertical axis length v are calculated by the above formula, and the smaller of the two is the pixel coverage S k .

[0106] Set a pixel coverage threshold S T Filter out all the k <ST The deformed 3D Gaussian ball is denoted by G small In one embodiment, the pixel coverage threshold S T The Gaussian sphere set after screening is divided into voxel grids according to spatial positions, and multiple small Gaussian spheres after screening in each voxel are averaged and pooled to generate a large aggregated Gaussian sphere. The aggregated Gaussian spheres of this part are then combined with the original pixels whose pixel coverage is greater than or equal to the pixel coverage threshold S. T The deformed Gaussian balls are combined into a multi-scale Gaussian ball set. Through the above steps, the original Gaussian ball set is transformed into a multi-scale Gaussian ball set, so that in the subsequent steps, the multi-scale Gaussian balls suitable for rendering can be screened out according to the real-time rendering resolution.

[0107] In a specific embodiment, Gaussian screening based on pixel coverage is performed on the multi-scale Gaussian sphere set to obtain an optimized multi-scale Gaussian sphere set, which specifically includes:

[0108] According to the current rendering resolution, filter out the multi-scale Gaussian spheres that meet the conditions from the multi-scale Gaussian sphere set. The screening conditions are:

[0109] ;

[0110] in, Represents a set of multi-scale Gaussian balls The pixel coverage of a single multi-scale Gaussian sphere in , represents the pixel coverage threshold, and A collection of multi-scale Gaussian spheres at different rendering resolutions The maximum pixel coverage and minimum pixel coverage of the multi-scale Gaussian sphere, is the relative maximum pixel coverage threshold, is the relative minimum pixel coverage threshold, and Represents "and" and "or" respectively;

[0111] The optimized multi-scale Gaussian spheres suitable for rendering at the current resolution are filtered out using the filtering conditions and a set of optimized multi-scale Gaussian spheres is constructed.

[0112] Specifically, in one embodiment, the relative maximum pixel coverage threshold The value is 1.5, which is the relative minimum pixel coverage threshold The value of is 0.5, and other values ​​may be selected in other embodiments. The above screening conditions may be used to screen out an optimized multi-scale Gaussian sphere set suitable for rendering at the current rendering resolution.

[0113] S4, projects the optimized multi-scale Gaussian sphere set onto a two-dimensional plane to generate a corresponding 2D Gaussian distribution on the two-dimensional plane, and performs Alpha blending processing on the optimized multi-scale Gaussian sphere set contained in a single pixel point of the 2D Gaussian on the two-dimensional plane determined by the distribution of the 2D Gaussian on the two-dimensional plane to reconstruct an anti-aliased dynamic rendering scene image.

[0114] In a specific embodiment, step S4 specifically includes:

[0115] The projection formula is used to project each optimized multi-scale Gaussian ball in the optimized multi-scale Gaussian ball set from 3D to 2D, and the center position and covariance matrix of the 2D Gaussian corresponding to the optimized multi-scale Gaussian ball are obtained, and the distribution of the 2D Gaussian corresponding to the optimized multi-scale Gaussian ball on the two-dimensional plane is determined, as shown in the following formula:

[0116] ;

[0117] ;

[0118] in, is the center position of the 2D Gaussian corresponding to the optimized multi-scale Gaussian ball, is the covariance matrix of the 2D Gaussian corresponding to the optimized multi-scale Gaussian sphere, P is the projection matrix, W is the view transformation matrix, J is the Jacobian matrix of the affine approximation, represents the center position of a single optimized multi-scale Gaussian ball in the set of optimized multi-scale Gaussian balls, represents the covariance matrix of a single optimized multi-scale Gaussian ball in the optimized multi-scale Gaussian ball set, and T represents the transpose;

[0119] Determine the set of optimized multi-scale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane according to the distribution of the 2D Gaussian balls corresponding to the optimized multi-scale Gaussian balls on the two-dimensional plane;

[0120] The color and opacity of each optimized multi-scale Gaussian ball in the set of optimized multi-scale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane are input into the Alpha blending formula to obtain the final color information of a single pixel point of the 2D Gaussian on the two-dimensional plane, as shown in the following formula:

[0121] ;

[0122] Among them, C is the final color information of a single pixel, is the color of the pth optimized multi-scale Gaussian ball in the set of optimized multi-scale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane, is the opacity of the pth optimized multi-scale Gaussian ball in the set of optimized multi-scale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane, b is the number of optimized multi-scale Gaussian balls in the set of optimized multi-scale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane, represents the cumulative transmittance of all optimized multiscale Gaussian balls before the p-th optimized multiscale Gaussian ball in the set of optimized multiscale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane, j is the index of all optimized multiscale Gaussian balls before the p-th optimized multiscale Gaussian ball in the set of optimized multiscale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane, is the opacity of the jth optimized multi-scale Gaussian ball in the set of optimized multi-scale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane;

[0123] The anti-aliased dynamic rendering scene image is constructed through the final color information of all pixels.

[0124] Specifically, the projection matrix, view transformation matrix, and affine approximation Jacobian matrix used in the process of projecting the optimized multi-scale Gaussian ball set are determined by the camera internal and external parameters, and the camera internal and external parameters include the camera posture determined in step S1. Since the color of the final single pixel point needs to be performed on a two-dimensional plane, it is necessary to first project the optimized multi-scale Gaussian ball set to the two-dimensional plane, and determine the center position and covariance matrix of the optimized multi-scale Gaussian ball set projected on the two-dimensional plane. These two parameters determine the position and shape of the 2D Gaussian, so as to determine the distribution of the 2D Gaussian on the two-dimensional plane, and then determine the optimized multi-scale Gaussian ball set contained in a pixel point of the 2D Gaussian on the two-dimensional plane. The final color information of a single pixel point of the 2D Gaussian on the two-dimensional plane is calculated by the optimized multi-scale Gaussian ball set contained in a single pixel point of the 2D Gaussian on the two-dimensional plane.

[0125] The position of a single pixel in a two-dimensional plane is fixed, and each pixel has color information. The final reconstructed anti-aliased dynamic rendering scene image needs to be constructed from the colors of all pixels. Alpha blending can superimpose the colors of the optimized multi-scale Gaussian sphere within a pixel to obtain the final color information of the pixel.

[0126] Further references Figure 2 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a dynamic scene reconstruction device based on a multi-scale Gaussian ball. Figure 1Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0127] The embodiment of the present application provides a dynamic scene reconstruction device based on a multi-scale Gaussian sphere, comprising:

[0128] Initialization module 1 is configured to obtain a video frame sequence to be reconstructed, process the video frame sequence to be reconstructed using a motion recovery structure algorithm, generate a sparse point cloud, initialize the sparse point cloud, and generate a 3D Gaussian sphere set;

[0129] A deformation module 2 is configured to process the 3D Gaussian sphere set using a dual-domain deformation model and an adaptive timestamp to obtain a deformed 3D Gaussian sphere set;

[0130] The multi-scale Gaussian module 3 is configured to perform multi-scale Gaussian processing on the deformed 3D Gaussian ball set to generate a multi-scale Gaussian ball set; perform Gaussian screening based on pixel coverage on the multi-scale Gaussian ball set to obtain an optimized multi-scale Gaussian ball set;

[0131] The reconstruction module 4 is configured to project the optimized multi-scale Gaussian sphere set onto a two-dimensional plane, generate a corresponding 2D Gaussian distribution on the two-dimensional plane, and perform Alpha blending processing on the optimized multi-scale Gaussian sphere set contained in a single pixel point of the 2D Gaussian on the two-dimensional plane determined by the distribution of the 2D Gaussian on the two-dimensional plane, to reconstruct an anti-aliased dynamic rendering scene image.

[0132] Figure 3 Schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention. Figure 3 As shown, the electronic device of this embodiment includes: a processor 301 and a memory 302; wherein the memory 302 is used to store computer-executable instructions; the processor 301 is used to execute the computer-executable instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0133] Optionally, the memory 302 may be independent or integrated with the processor 301 .

[0134] When the memory 302 is independently provided, the electronic device further includes a bus 303 for connecting the memory 302 and the processor 301 .

[0135] The embodiment of the present invention further provides a computer storage medium, in which computer execution instructions are stored. When the processor 301 executes the computer execution instructions, the above method is implemented.

[0136] The embodiment of the present invention further provides a computer program product, including a computer program. When the computer program is executed by the processor 301, the above method is implemented.

[0137] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0138] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0139] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The unit formed by the above modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0140] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor 301 to perform some steps of the methods of various embodiments of the present application.

[0141] It should be understood that the processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or the processor 301 may be any conventional processor 301, etc. The steps of the method disclosed in the invention may be directly embodied in the execution of the hardware processor 301, or may be executed by a combination of hardware and software modules in the processor 301.

[0142] The memory 302 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disk.

[0143] The bus 303 may be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 303 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus 303 in the drawings of the present application is not limited to only one bus 303 or one type of bus 303.

[0144] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0145] An exemplary storage medium is coupled to the processor 301, so that the processor 301 can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor 301. The processor 301 and the storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as ASIC). Of course, the processor 301 and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0146] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dynamic scene reconstruction method based on multi-scale Gaussian sphere, characterized in that: The following steps are involved: Acquire a video frame sequence to be reconstructed, process the video frame sequence to be reconstructed using a structure-from-motion algorithm to generate a sparse point cloud, initialize the sparse point cloud, and generate a 3D Gaussian sphere set; The 3D Gaussian sphere set is processed by using a dual-domain deformation model and an adaptive timestamp to obtain a deformed 3D Gaussian sphere set; Performing multi-scale Gaussian processing on the deformed 3D Gaussian sphere set to generate a multi-scale Gaussian sphere set; performing Gaussian screening based on pixel coverage on the multi-scale Gaussian sphere set to obtain an optimized multi-scale Gaussian sphere set; The optimized multi-scale Gaussian sphere set is projected onto a two-dimensional plane to generate a corresponding 2D Gaussian distribution on the two-dimensional plane, and Alpha blending is performed on the optimized multi-scale Gaussian sphere set contained in a single pixel point of the 2D Gaussian on the two-dimensional plane determined by the distribution of the 2D Gaussian on the two-dimensional plane to reconstruct an anti-aliased dynamic rendering scene image.

2. The dynamic scene reconstruction method based on multi-scale Gaussian sphere according to claim 1, characterized in that: The video frame sequence to be reconstructed is processed by using a structure-from-motion algorithm to generate a sparse point cloud, and the sparse point cloud is initialized to generate a 3D Gaussian sphere set, specifically including: The video frame sequence to be reconstructed and the known camera parameters are input into the motion recovery structure algorithm, and a set of sparse point clouds P with position information and color information is output. cloud and its corresponding camera pose; For the sparse point cloud P cloud Each point in is initialized as a 3D Gaussian ball G, and each 3D Gaussian ball is defined as: ； Among them, G(x) represents the function corresponding to the 3D Gaussian sphere, x represents a three-dimensional space point in the 3D Gaussian sphere, T represents transposition, is the center position of the 3D Gaussian sphere, is the covariance matrix of the 3D Gaussian sphere, which can be obtained by parameterizing the scaling vector s and the rotation quaternion q, and is expressed as: ; is the scaling matrix converted by scaling vector s, R is the rotation matrix converted by rotating quaternion q; Finally, a 3D Gaussian sphere set is constructed. , represents the i-th 3D Gaussian sphere.

3. The dynamic scene reconstruction method based on multi-scale Gaussian sphere according to claim 1, characterized in that: The 3D Gaussian sphere set is processed using a dual-domain deformation model and an adaptive timestamp to obtain a deformed 3D Gaussian sphere set, specifically including: An adaptive timestamp is introduced to scale the normalized frame index time of each 3D Gaussian sphere in the 3D Gaussian sphere set to obtain the scaled time, as shown in the following formula: ； in, represents the time after scaling, represents the normalized frame index time, and They represent the time scaling factor and basic factor of the 3D Gaussian sphere respectively; Based on the time-varying residual after inputting the scaled time and the dual-domain deformation model, the basic properties of each 3D Gaussian sphere at the reference time are dynamically modeled to obtain the dynamic properties deformed with the scaled time, as shown in the following formula: ； Among them, S(t s ) is the time of the 3D Gaussian sphere after scaling The dynamic properties after deformation, S0 is the basic properties of the 3D Gaussian ball at the reference time t0, including the center position, rotation information and color information, and the time-varying residual D(t s ) is fitted by the time domain polynomial and the frequency domain Fourier series, as shown in the following formula: ； in, is an N-order polynomial, n represents the nth order in the polynomial, are the polynomial coefficients of the nth order, is the L-order Fourier series, l represents the l-th order in the Fourier series, , are the sine Fourier coefficients and cosine Fourier coefficients of the lth order; The 3D Gaussian sphere is scaled by the time The dynamic properties after deformation get the deformed 3D Gaussian sphere , and construct a deformed 3D Gaussian sphere set , represents the i-th deformed 3D Gaussian sphere.

4. The dynamic scene reconstruction method based on multi-scale Gaussian sphere according to claim 1, characterized in that: Performing multi-scale Gaussian processing on the deformed 3D Gaussian sphere set to generate a multi-scale Gaussian sphere set specifically includes: Each deformed 3D Gaussian sphere in the deformed 3D Gaussian sphere set is projected into a 2D Gaussian sphere corresponding to the deformed 3D Gaussian sphere. , and calculate the pixel coverage of the 2D Gaussian corresponding to the deformed 3D Gaussian ball in the screen space, as shown in the following formula: ； ； ； Among them, μ k Indicates the center position of the 2D Gaussian corresponding to the deformed 3D Gaussian ball, V k represents the covariance matrix of the 2D Gaussian corresponding to the deformed 3D Gaussian sphere, u represents the horizontal axis length of the 2D Gaussian corresponding to the deformed 3D Gaussian sphere in the plane, and v represents the vertical axis length of the 2D Gaussian corresponding to the deformed 3D Gaussian sphere in the plane. represents the opacity of the deformed 3D Gaussian sphere, and They represent the horizontal and vertical coordinate components of the deformed 3D Gaussian sphere projected onto the two-dimensional plane, Represents the preset Gaussian opacity threshold, It means taking the minimum value of the two, S k Indicates the pixel coverage of the 2D Gaussian corresponding to the deformed 3D Gaussian sphere in screen space; Screening out deformed 3D Gaussian spheres whose pixel coverage is less than a pixel coverage threshold from the deformed 3D Gaussian sphere set, obtaining screened Gaussian spheres and constructing the screened Gaussian sphere set; All the filtered Gaussian balls in the filtered Gaussian ball set are divided into voxel grids according to spatial positions, and multiple filtered Gaussian balls in each voxel of the voxel grid are averaged and pooled to generate an aggregated Gaussian ball, as shown in the following formula: ； ； ； Among them, G m is the set of filtered Gaussian balls within the mth voxel, Represents G m The number of Gaussian balls after screening in, , and G m The center position, scaling information and color information of the filtered Gaussian sphere, , and They are the center position, scaling information, and color information of the aggregated Gaussian sphere. represents the pixel coverage threshold, is the average pixel coverage of the set of filtered Gaussian spheres in each voxel; The aggregated Gaussian sphere or the deformed 3D Gaussian sphere whose pixel coverage is greater than or equal to the pixel coverage threshold is used as a multi-scale Gaussian sphere, and a multi-scale Gaussian sphere set is constructed. .

5. The dynamic scene reconstruction method based on multi-scale Gaussian sphere according to claim 1, characterized in that: Performing Gaussian screening based on pixel coverage on the multi-scale Gaussian sphere set to obtain an optimized multi-scale Gaussian sphere set specifically includes: According to the current rendering resolution, a multi-scale Gaussian sphere that meets the conditions is selected from the multi-scale Gaussian sphere set, and the selection conditions are: ； in, Represents a set of multi-scale Gaussian balls The pixel coverage of a single multi-scale Gaussian sphere in , represents the pixel coverage threshold, and A collection of multi-scale Gaussian spheres at different rendering resolutions The maximum pixel coverage and minimum pixel coverage of the multi-scale Gaussian sphere, is the relative maximum pixel coverage threshold, is the relative minimum pixel coverage threshold, and Represents "and" and "or" respectively; The screening condition is used to screen out optimized multi-scale Gaussian spheres suitable for rendering at the current resolution and to construct an optimized multi-scale Gaussian sphere set.

6. The dynamic scene reconstruction method based on multi-scale Gaussian sphere according to claim 1, characterized in that: The optimized multi-scale Gaussian ball set is projected onto a two-dimensional plane to generate a corresponding distribution of 2D Gaussians on the two-dimensional plane, and an alpha blending process is performed on the optimized multi-scale Gaussian ball set contained in a single pixel point of a 2D Gaussian on the two-dimensional plane determined by the distribution of the 2D Gaussians on the two-dimensional plane to reconstruct an anti-aliased dynamic rendering scene image, specifically including: Each optimized multi-scale Gaussian ball in the optimized multi-scale Gaussian ball set is projected from 3D to 2D by the projection formula to obtain the center position and covariance matrix of the 2D Gaussian corresponding to the optimized multi-scale Gaussian ball, and determine the distribution of the 2D Gaussian corresponding to the optimized multi-scale Gaussian ball on the two-dimensional plane, as shown in the following formula: ； ； in, is the center position of the 2D Gaussian corresponding to the optimized multi-scale Gaussian ball, is the covariance matrix of the 2D Gaussian corresponding to the optimized multi-scale Gaussian sphere, P is the projection matrix, W is the view transformation matrix, J is the Jacobian matrix of the affine approximation, represents the center position of a single optimized multi-scale Gaussian ball in the optimized multi-scale Gaussian ball set, represents the covariance matrix of a single optimized multi-scale Gaussian sphere in the optimized multi-scale Gaussian sphere set, and T represents transpose; Determine the optimized multi-scale Gaussian sphere set contained in a single pixel point of the 2D Gaussian on the two-dimensional plane according to the distribution of the 2D Gaussian corresponding to the optimized multi-scale Gaussian sphere on the two-dimensional plane; The color and opacity of each optimized multi-scale Gaussian ball in the set of optimized multi-scale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane are input into the Alpha blending formula to obtain the final color information of a single pixel point of the 2D Gaussian on the two-dimensional plane, as shown in the following formula: ； Among them, C is the final color information of a single pixel, is the color of the pth optimized multi-scale Gaussian ball in the set of optimized multi-scale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane, is the opacity of the pth optimized multi-scale Gaussian ball in the set of optimized multi-scale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane, b is the number of optimized multi-scale Gaussian balls in the set of optimized multi-scale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane, represents the cumulative transmittance of all optimized multiscale Gaussian balls before the p-th optimized multiscale Gaussian ball in the set of optimized multiscale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane, j is the index of all optimized multiscale Gaussian balls before the p-th optimized multiscale Gaussian ball in the set of optimized multiscale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane, is the opacity of the jth optimized multi-scale Gaussian ball in the set of optimized multi-scale Gaussian balls contained in a single pixel point of the 2D Gaussian on the two-dimensional plane; The anti-aliased dynamic rendering scene image is constructed through the final color information of all pixels.

7. A dynamic scene reconstruction device based on multi-scale Gaussian sphere, characterized in that: include: An initialization module is configured to obtain a video frame sequence to be reconstructed, process the video frame sequence to be reconstructed using a structure-from-motion algorithm to generate a sparse point cloud, initialize the sparse point cloud, and generate a 3D Gaussian sphere set; A deformation module is configured to process the 3D Gaussian sphere set using a dual-domain deformation model and an adaptive timestamp to obtain a deformed 3D Gaussian sphere set; The multi-scale Gaussian module is configured to perform multi-scale Gaussian processing on the deformed 3D Gaussian sphere set to generate a multi-scale Gaussian sphere set; perform Gaussian screening based on pixel coverage on the multi-scale Gaussian sphere set to obtain an optimized multi-scale Gaussian sphere set; The reconstruction module is configured to project the optimized multi-scale Gaussian sphere set onto a two-dimensional plane, generate a corresponding 2D Gaussian distribution on the two-dimensional plane, and perform Alpha blending processing on the optimized multi-scale Gaussian sphere set contained in a single pixel point of the 2D Gaussian on the two-dimensional plane determined by the distribution of the 2D Gaussian on the two-dimensional plane, so as to reconstruct an anti-aliased dynamic rendering scene image.

8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images