Three-Dimensional Reconstruction Method with Specular Reflection in Dynamic Scenes Based on Dual Environment Maps

Abstract

The present invention proposes a three-dimensional reconstruction method for a dynamic scene with specular reflection based on dual environment maps. The method includes: initializing a three-dimensional Gaussian as a basic unit for characterizing the spatial structure of the scene through three-dimensional point clouds, and transforming the three-dimensional Gaussian through a deformation field in combination with timestamps; using a rasterization rendering module to perform projection transformation operations, rasterization operations, and attribute weighting processing on the changed three-dimensional Gaussian at corresponding moments in sequence according to the camera pose; performing reflection decoupling using a dual environment map module according to the normal map and time parameters; and performing a mixing process on the base color map, reflection intensity map, and final reflection color map to generate a rendering result. The present invention introduces the design of dual environment maps. The static environment map is used to capture the basic light distribution of the scene and provide a stable light reference; while the dynamic environment map can be dynamically adjusted according to time changes to adapt to the real-time changes of light in the scene, enhancing the modeling ability for complex light conditions in dynamic scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of computer image processing, and particularly to a three-dimensional reconstruction method for a dynamic scene with specular reflection based on dual environment maps. Background Art

[0002] Three-dimensional reconstruction is one of the core tasks in the field of computer vision, aiming to recover the three-dimensional structure of a scene from images or videos. In recent years, with the rapid development of virtual reality, augmented reality, and robotics technologies, the demand for three-dimensional reconstruction of dynamic scenes has been increasing. However, when there are specular reflection objects (such as glass and metal surfaces) in a dynamic scene, the reconstruction task becomes extremely challenging. This is because specular reflection introduces complex non-diffuse reflection phenomena, leading to deviations in the model's estimation of the object's shape and position, thus interfering with the accuracy of traditional reconstruction methods. In this patent, the influence of the changing light conditions over time in a dynamic scene on the specular reflection objects in the scene is considered. Although traditional methods can accurately model the specular reflection characteristics under static light assumptions, under dynamic light conditions, they cannot adapt to the changes in light direction and intensity in real time, resulting in artifacts in the specular highlight areas.

[0003] In recent years, Neural Radiance Fields (NeRF) have for the first time achieved high-quality synthesis of new views of complex three-dimensional scenes from multi-view static images by representing the scene as a continuous implicit neural radiance field. However, its core framework is based on the static scene assumption and is difficult to handle dynamic objects or time-varying scene content. Therefore, researchers introduced the concept of a deformation field to extend NeRF to dynamic scenes. This mechanism requires joint optimization of the deformation field and radiance fields, resulting in a significant increase in the number of model parameters and computational complexity, and significantly extending the training time.

[0004] In contrast, 3D Gaussian Splatting can significantly reduce the computational overhead while ensuring high-quality rendering through efficient point cloud representation and rendering techniques, providing a new solution for scene reconstruction. However, because it uses spherical harmonic coefficients to model color distribution, this method has limitations in representing highly directional lighting effects (such as specular reflection), making it difficult to capture the sharp features of highlights. Although some studies have alleviated this problem by improving the model architecture or introducing additional lighting representation methods, most current work is still mainly limited to static scenes and is difficult to effectively handle the complex changes in lighting and object motion in dynamic scenes.

[0005] The related research faces the following problems: First, the complexity of object motion and light changes in dynamic scenes makes it difficult to directly apply traditional static scene reconstruction methods; Second, although existing NeRF-based improvement methods have enhanced the modeling ability of specular reflection and highlight features to a certain extent, their computational cost and training time have increased significantly, restricting the scalability of practical applications. Summary of the Invention

[0006] In view of the above situation, the main objective of the present invention is to propose a three-dimensional reconstruction method for dynamic scenes with specular reflection based on dual environment maps to solve the above technical problems.

[0007] The present invention proposes a three-dimensional reconstruction method for dynamic scenes with specular reflection based on dual environment maps. The method includes the following steps:

[0008] Step 1: Construct a dual environment map module based on the dual environment map mechanism and a rasterization rendering module based on the rasterization mechanism. The dual environment map module and the rasterization rendering module constitute specular reflection rendering, and the dynamic scene representation and specular reflection rendering constitute a three-dimensional reconstruction model;

[0009] Among them, the dynamic scene representation includes a deformation field composed of a multi-layer perceptron, and the dual environment map module includes a feed-forward neural network composed of a multi-layer perceptron, a dynamic environment map, and a static environment map;

[0010] Step 2: Collect an image sequence of the same dynamic scene and the time stamp corresponding to each frame of the image, and perform image processing through an initial point cloud generation tool to generate a three-dimensional point cloud and camera poses;

[0011] Step 3: Based on the dynamic scene representation, initialize a three-dimensional Gaussian with the three-dimensional point cloud as the basic unit representing the scene spatial structure, and use the deformation field to calculate the three-dimensional Gaussian position and time input to obtain the changed three-dimensional Gaussian at the corresponding moment;

[0012] Step 4: Based on specular reflection rendering, use the rasterization rendering module to perform projection transformation operations, rasterization operations, and attribute weighting processing on the changed three-dimensional Gaussian at the corresponding moment according to the camera poses to obtain a normal map, a base color map, and a reflection intensity map;

[0013] Step 5: According to the normal map and time parameters, use the dual environment map module for reflection decoupling to obtain the final reflection color map;

[0014] Step 6: Mix the base color map, the reflection intensity map, and the final reflection color map to generate a rendering result, construct a photometric loss based on the rendering result, and use the photometric loss to optimize the three-dimensional reconstruction model to obtain an optimized three-dimensional reconstruction model;

[0015] Based on the optimized 3D reconstruction model to confirm the final rendering result.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0017] 1. The deformation field adopted by the present invention dynamically adjusts the position and shape of the three-dimensional Gaussian distribution by learning the deformation parameters that change over time, thereby depicting the movement and change of an object in the time dimension and achieving an efficient representation of a dynamic scene.

[0018] 2. The present invention introduces a dual environment map design. The static environment map is used to capture the basic light distribution of the scene and provide a stable light reference; while the dynamic environment map can be dynamically adjusted according to time changes to adapt to the real-time changes of light in the scene, enhancing the modeling ability for complex light conditions in the dynamic scene.

[0019] The additional aspects and advantages of the present invention will be partially given in the following description, partially become apparent from the following description, or be understood through the embodiments of the present invention. Description of the Drawings

[0020] Figure 1 It is a flowchart of the steps of the three-dimensional reconstruction method for a dynamic scene with specular reflection based on dual environment maps proposed by the present invention.

[0021] Figure 2 It is an overall framework diagram of the three-dimensional reconstruction method for a dynamic scene with specular reflection based on dual environment maps proposed by the present invention. Detailed Embodiment

[0022] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals are the same or similar elements or elements with the same or similar functions from beginning to end. The embodiments described below with reference to the drawings are exemplary and are only used to explain the present invention and should not be construed as a limitation of the present invention.

[0023] Referring to the following description and drawings, these and other aspects of the embodiments of the present invention will be clear. In these descriptions and drawings, some specific embodiments of the embodiments of the present invention are specifically disclosed as some ways to implement the principles of the embodiments of the present invention, but it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0024] Please refer to Figure 1 , the embodiments of the present invention propose a three-dimensional reconstruction method for a dynamic scene with specular reflection based on dual environment maps, and the method includes the following steps:

[0025] Step 1: Construct a dual environment mapping module based on the dual environment mapping mechanism, and construct a rasterization rendering module based on the rasterization mechanism. The dual environment mapping module and the rasterization rendering module constitute specular reflection rendering, and the dynamic scene representation and specular reflection rendering constitute a 3D reconstruction model;

[0026] Among them, the dynamic scene representation includes a deformation field composed of a multi-layer perceptron. The dual environment mapping module includes a feed-forward neural network composed of a multi-layer perceptron, a dynamic environment map, and a static environment map.

[0027] Step 2: Collect an image sequence of the same dynamic scene and the time stamp corresponding to each frame of the image, and perform image processing through an initial point cloud generation tool to generate a 3D point cloud and a camera pose.

[0028] Step 3: Based on the dynamic scene representation, initialize a 3D Gaussian with the 3D point cloud as the basic unit representing the scene space structure, and use the deformation field to calculate the 3D Gaussian position and time input to obtain the changed 3D Gaussian at the corresponding moment.

[0029] Please refer to Figure 2 , in Step 3, initializing a 3D Gaussian with the 3D point cloud as the basic unit representing the scene space structure and using the deformation field to calculate the 3D Gaussian position and time input specifically includes the following steps:

[0030] Initialize a 3D Gaussian according to the 3D point cloud to obtain the original value of the 3D Gaussian; among them, the original value of the 3D Gaussian includes the mean of the 3D Gaussian, the rotation vector of the 3D Gaussian, the scaling vector of the 3D Gaussian, the density of the 3D Gaussian, and the reflection intensity of the 3D Gaussian;

[0031] Perform sine embedding addition processing on the mean of the 3D Gaussian and the time parameter of the time stamp respectively to obtain the mean embedding output of the 3D Gaussian and the embedding output of the time parameter;

[0032] Concatenate the mean embedding output of the 3D Gaussian and the embedding output of the time parameter to obtain a feature that fuses position and time information;

[0033] Input the feature that fuses position and time information into the deformation field for change amount calculation to obtain the change amount of the 3D Gaussian; among them, the change amount of the 3D Gaussian includes the displacement increment of the 3D Gaussian, the rotation increment of the 3D Gaussian, and the scaling increment of the 3D Gaussian;

[0034] Add the original value of the 3D Gaussian and the change amount of the 3D Gaussian to obtain the changed 3D Gaussian at the corresponding moment.

[0035] Perform sine embedding additional processing on the mean value of the three-dimensional Gaussian and the time parameter of the timestamp respectively to obtain the mean embedding output of the three-dimensional Gaussian and the embedding output of the time parameter. The relational expressions existing in the corresponding process are as follows:

[0036] ;

[0037] Among them, represents after sine embedding processing, represents the input, represents the sine function, represents the cosine function, represents the pi, represents the frequency set number, represents the coefficient.

[0038] Add the original value of the three-dimensional Gaussian and the change amount of the three-dimensional Gaussian to calculate the three-dimensional Gaussian after change at the corresponding moment. The relational expressions existing in the corresponding process are as follows:

[0039] ;

[0040] Among them, represents the three-dimensional Gaussian after change at the corresponding moment , represents the mean value of the three-dimensional Gaussian, represents the displacement increment of the three-dimensional Gaussian, represents the rotation vector of the three-dimensional Gaussian, represents the rotation increment of the three-dimensional Gaussian, represents the scaling vector of the three-dimensional Gaussian, represents the scaling increment of the three-dimensional Gaussian, represents the density of the three-dimensional Gaussian, represents the reflection intensity of the three-dimensional Gaussian.

[0041] It should be noted that the deformation field consists of 8 multi-layer perceptrons (MLPs).

[0042] Furthermore, initialize the three-dimensional Gaussian through the point cloud as the basic unit for characterizing the scene space structure, and use the deformation field to process the three-dimensional Gaussian position and time input to calculate the change amount of the three-dimensional Gaussian at the corresponding moment to achieve dynamic scene characterization.

[0043] Step 4, Based on specular reflection rendering, use the rasterization rendering module to perform projection transformation operation, rasterization operation and attribute weighting processing on the three-dimensional Gaussian after change at the corresponding moment according to the camera pose to obtain the normal map, base color map and reflection intensity map.

[0044] In step 4, according to the camera pose, the rasterization rendering module performs projection transformation operations, rasterization operations, and attribute weighting processing on the changed three-dimensional Gaussians at the corresponding time to obtain a normal map, a base color map, and a reflection intensity map. The relationships in the corresponding process are as follows:

[0045] ;

[0046] Among them, represents the normal map, represents the number of two-dimensional Gaussians covering the generated image pixels, represents the normal of the three-dimensional Gaussian, represents the three-dimensional Gaussian, represents the base color map, represents the color of the three-dimensional Gaussian, represents the reflection intensity map, represents the reflection intensity of the three-dimensional Gaussian, represents the viewing direction.

[0047] It should be noted that the generated image mentioned in the definition of is generated by three-dimensional Gaussians, camera parameters, and time parameters.

[0048] Step 5: According to the normal map and time parameters, use the dual environment map module to perform reflection decoupling to obtain the final reflection color map.

[0049] In step 5, according to the normal map and time parameters, use the dual environment map module to perform reflection decoupling to obtain the final reflection color map, which specifically includes the following steps:

[0050] Use the dual environment map module to construct environment maps, and obtain the constructed static environment map and the constructed dynamic environment map respectively;

[0051] Perform reflection calculations according to the normal map and the viewing direction to obtain a first calculation result, and use the first calculation result to query the constructed static environment map to obtain the base static reflection color of the scene;

[0052] Perform reflection calculations according to the normal map and the viewing direction to obtain a second calculation result, input the second calculation result and the embedded output of the time parameter into a feedforward neural network for mechanism processing to obtain an output feature, and use the output feature to query the constructed dynamic environment map to obtain the dynamic reflection color that changes with time;

[0053] Add the base static reflection color of the scene and the dynamic reflection color that changes with time to obtain the final reflection color map.

[0054] Perform reflection calculations based on the normal map and viewing direction to obtain a first calculation result, and use the first calculation result to query the constructed static environment map to obtain the basic static reflection color of the scene. The relational expressions for the corresponding process are as follows:

[0055] ;

[0056] Among them, represents the basic static reflection color of the scene, represents the static environment map, represents the L2 norm;

[0057] In the step of performing reflection calculations based on the normal map and viewing direction to obtain a second calculation result, inputting the second calculation result and the embedded output of the time parameter into a feed-forward neural network for mechanism processing to obtain an output feature, and using the output feature to query the constructed dynamic environment map to obtain the dynamic reflection color that changes with time, the relational expressions for the corresponding process are as follows:

[0058] ;

[0059] Among them, represents the dynamic reflection color that changes with time, represents the dynamic environment map, represents a feed-forward neural network composed of 4 MLP layers, represents the embedded output of the time parameter;

[0060] In the step of adding the basic static reflection color of the scene and the dynamic reflection color that changes with time to obtain the final reflection color map, the relational expressions for the corresponding process are as follows:

[0061] ;

[0062] Among them, represents the final reflection color map.

[0063] Furthermore, the dual environment map mechanism queries the static reflection color from the static environment map using the normal map, combines the normal map processed by the MLP and the time parameter to query the dynamic environment map to obtain the dynamic reflection color, and finally adds the static reflection color and the dynamic reflection color to obtain the final reflection color, realizing specular reflection rendering.

[0064] Furthermore, the transformed 3D Gaussian is projected onto the 2D image plane according to the camera parameters of the current view, and the differentiable rasterization technique is used to efficiently calculate the coverage of each pixel by the corresponding 2D Gaussian and its contribution weight. Based on these weights, the relevant attributes of multiple 3D Gaussians covering the same pixel are weighted and summed to generate the base color map, normal map, and reflection intensity map. To further perform specular reflection modeling, the dual environment map module uses the normal map, view direction, and time generated in the rasterization step as inputs to query and obtain the final reflection color. Finally, based on the base color map, reflection intensity map, and final reflection color generated by rasterization, a rendered image with high-fidelity dynamic specular reflection characteristics is synthesized.

[0065] Step 6: Blend the base color map, reflection intensity map, and final reflection color map to generate the rendering result;

[0066] Construct a photometric loss based on the rendering result, and use the photometric loss to optimize the 3D reconstruction model to obtain an optimized 3D reconstruction model;

[0067] Based on the optimized 3D reconstruction model, confirm to obtain the final rendering result.

[0068] In Step 6, when blending the base color map, reflection intensity map, and final reflection color map to generate the rendering result, the following relationship exists in the corresponding process:

[0069] ;

[0070] where represents the rendering result.

[0071] Construct a photometric loss based on the rendering result, and the following relationship exists in the corresponding process:

[0072] ;

[0073] where represents the photometric loss, represents the weight parameter, represents at the error between the rendering result and the real image under the regular loss calculation rule, represents at the error between the rendering result and the real image under the regular loss calculation rule.

[0074] It should be understood that each part of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following techniques known in the art or a combination thereof can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), and the like.

[0075] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0076] The above-described embodiments merely represent several implementation manners of the present invention, and the description thereof is relatively specific and detailed. However, it should not be construed as a limitation on the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the appended claims.

Claims

1. A three-dimensional reconstruction method for a dynamic scene with specular reflection based on dual environment mapping, characterized in that, The method includes the following steps: Step 1: Construct a dual environment mapping module based on the dual environment mapping mechanism, and construct a rasterization rendering module based on the rasterization mechanism. The dual environment mapping module and the rasterization rendering module constitute specular reflection rendering, and the dynamic scene representation and specular reflection rendering constitute a 3D reconstruction model; Among them, the dynamic scene representation includes a deformation field composed of a multi-layer perceptron. The dual environment mapping module includes a feed-forward neural network composed of a multi-layer perceptron, a dynamic environment map, and a static environment map; Step 2: Collect an image sequence of the same dynamic scene and the time stamp corresponding to each frame of the image, and perform image processing through an initial point cloud generation tool to generate a 3D point cloud and a camera pose; Step 3: Based on the dynamic scene representation, initialize a 3D Gaussian with the 3D point cloud as the basic unit for characterizing the scene spatial structure, and use the deformation field to calculate the 3D Gaussian position and time input to obtain the changed 3D Gaussian at the corresponding moment; Step 4: Based on specular reflection rendering, use the rasterization rendering module to perform projection transformation operations, rasterization operations, and attribute weighting processing on the changed 3D Gaussian at the corresponding moment according to the camera pose to obtain a normal map, a base color map, and a reflection intensity map; Step 5: According to the normal map and time parameters, use the dual environment mapping module to perform reflection decoupling to obtain the final reflection color map; Step 6: Perform blending processing on the base color map, the reflection intensity map, and the final reflection color map to generate a rendering result. Based on the rendering result, construct a photometric loss, and use the photometric loss to optimize the 3D reconstruction model to obtain an optimized 3D reconstruction model; Based on the optimized 3D reconstruction model to confirm the final rendering result.

2. The three-dimensional reconstruction method with specular reflection in a dynamic scene based on dual environment mapping according to claim 1, wherein In the said Step 3, initializing a 3D Gaussian with the 3D point cloud as the basic unit for characterizing the scene spatial structure, and using the deformation field to calculate the 3D Gaussian position and time input to obtain the changed 3D Gaussian at the corresponding moment specifically includes the following steps: Initialize a 3D Gaussian according to the 3D point cloud to obtain the original value of the 3D Gaussian; among them, the original value of the 3D Gaussian includes the mean value of the 3D Gaussian, the rotation vector of the 3D Gaussian, the scaling vector of the 3D Gaussian, the density of the 3D Gaussian, and the reflection intensity of the 3D Gaussian; Perform sine embedding additional processing on the mean value of the 3D Gaussian and the time parameter of the time stamp respectively to obtain the mean value embedding output of the 3D Gaussian and the embedding output of the time parameter; Concatenate the mean value embedding output of the 3D Gaussian and the embedding output of the time parameter to obtain a feature that fuses position and time information; Input the feature that fuses position and time information into the deformation field for change amount calculation to obtain the change amount of the 3D Gaussian; among them, the change amount of the 3D Gaussian includes the displacement increment of the 3D Gaussian, the rotation increment of the 3D Gaussian, and the scaling increment of the 3D Gaussian; Perform addition calculation on the original value of the 3D Gaussian and the change amount of the 3D Gaussian to obtain the changed 3D Gaussian at the corresponding moment.

3. The three-dimensional reconstruction method with specular reflection in a dynamic scene based on dual environment maps according to claim 2, wherein Perform sine embedding additional processing on the mean value of the 3D Gaussian and the time parameter of the time stamp respectively to obtain the mean value embedding output of the 3D Gaussian and the embedding output of the time parameter. The existing relational expressions for the corresponding process are as follows: ； Among them, indicates being processed by sine embedding, represents the input, represents the sine function, represents the cosine function, represents pi, represents the number of frequency sets, represents the coefficient.

4. The three-dimensional reconstruction method with specular reflection in a dynamic scene based on dual environment maps according to claim 3, wherein Add the original value of the three-dimensional Gaussian to the variation of the three-dimensional Gaussian to calculate the three-dimensional Gaussian after variation at the corresponding moment. The relational expressions existing in the corresponding process are as follows: ； Among them, represents the corresponding moment the three-dimensional Gaussian after change, represents the mean value of the three-dimensional Gaussian, represents the displacement increment of the three-dimensional Gaussian, represents the rotation vector of the three-dimensional Gaussian, represents the rotation increment of the three-dimensional Gaussian, represents the scaling vector of the three-dimensional Gaussian, represents the scaling increment of the three-dimensional Gaussian, represents the density of the three-dimensional Gaussian, represents the reflection intensity of the three-dimensional Gaussian.

5. The three-dimensional reconstruction method with specular reflection in a dynamic scene based on dual environment mapping according to claim 4, characterized in that, In the step 4, use the rasterization rendering module to perform projection transformation operation, rasterization operation and attribute weighting processing on the three-dimensional Gaussian after variation at the corresponding moment according to the camera pose to obtain a normal map, a base color map and a reflection intensity map. The relational expressions existing in the corresponding process are as follows: ； Among them, represents a normal map, represents the number of two-dimensional Gaussians covering the generated image pixels, represents the normal of the three-dimensional Gaussian, represents the three-dimensional Gaussian, represents the base color map, represents the color of the three-dimensional Gaussian, represents the reflection intensity map, represents the reflection intensity of the three-dimensional Gaussian, represents the viewing direction.

6. The three-dimensional reconstruction method with specular reflection in a dynamic scene based on dual environment mapping according to claim 5, wherein In the step 5, perform reflection decoupling using the dual environment map module according to the normal map and the time parameter to obtain the final reflection color map, which specifically includes the following steps: Use the dual environment map module to construct an environment map, and respectively obtain the constructed static environment map and the constructed dynamic environment map; Perform reflection calculation according to the normal map and the viewing direction to obtain a first calculation result, and use the first calculation result to query the constructed static environment map to obtain the base static reflection color of the scene; Perform reflection calculation according to the normal map and the viewing direction to obtain a second calculation result, input the second calculation result and the embedded output of the time parameter into a feedforward neural network for mechanism processing to obtain an output feature, and use the output feature to query the constructed dynamic environment map to obtain the dynamic reflection color that changes with time; Add the base static reflection color of the scene to the dynamic reflection color that changes with time to obtain the final reflection color map.

7. The three-dimensional reconstruction method with specular reflection in a dynamic scene based on dual environment mapping according to claim 6, characterized in that, Perform reflection calculation according to the normal map and the viewing direction to obtain a first calculation result, and use the first calculation result to query the constructed static environment map to obtain the base static reflection color of the scene. The relational expressions existing in the corresponding process are as follows: ； Among them, represents the base static reflection color of the scene, represents the static environment map, represents the L2 norm; In the step of performing reflection calculation according to the normal map and the viewing direction to obtain a second calculation result, input the second calculation result and the embedded output of the time parameter into a feedforward neural network for mechanism processing to obtain an output feature, and use the output feature to query the constructed dynamic environment map to obtain the dynamic reflection color that changes with time, the relational expressions existing in the corresponding process are as follows: ； Among them, represents the dynamic reflection color that changes over time, represents the dynamic environment map, represents a feedforward neural network composed of 4 MLP layers, represents the embedded output of the time parameter; In the step of adding the base static reflection color of the scene to the dynamic reflection color that changes with time to obtain the final reflection color map, the relational expressions existing in the corresponding process are as follows: ； Among them, represents the final reflection color map.

8. The three-dimensional reconstruction method with specular reflection in a dynamic scene based on dual environment mapping according to claim 7, wherein, In the step 6, perform blending processing on the base color map, the reflection intensity map and the final reflection color map to generate a rendering result. The relational expressions existing in the corresponding process are as follows: ； Among them, represents the rendering result.

9. The three-dimensional reconstruction method with specular reflection in a dynamic scene based on dual environment mapping according to claim 8, wherein Construct a photometric loss based on the rendering result. The relational expressions existing in the corresponding process are as follows: ； Among them, represents the photometric loss, represents the weight parameter, represents at the error between the rendering result and the real image under the regular loss calculation rule, represents at the error between the rendering result and the real image under the regular loss calculation rule.

Images