A method and system for reconstructing three-dimensional object material based on surface light source

Abstract

The present invention discloses a method and system for reconstructing the material of three-dimensional objects based on a surface light source. First, in a darkroom environment, a pre-calibrated camera-surface light source system is used to capture images of a given object, and these images are used to register the camera. Second, a neural network-based free viewpoint interpolation method is used to preliminarily reconstruct the object's geometry. Ray tracing is then used to precompute the light source-guided visibility. Finally, within the framework of inverse rendering, a linear transformation cosine method is used to achieve efficient rendering of physically based materials under a surface light source. This method further optimizes the geometric model while reconstructing the parametric material of the model surface with high quality. The present invention requires only simple capture and acquisition with readily available camera equipment to achieve rapid, high-quality material reconstruction of three-dimensional objects. The system can be used in fields such as virtual reality object modeling.

Description

Technical Field

[0001] The present invention relates to the fields of computer graphics and three-dimensional vision, and in particular to a method and system for reconstructing three-dimensional object material based on surface light source. Background Art

[0002] With advances in computer vision and graphics, digital twins of real-world objects, as a new type of three-dimensional asset, have shown broad promise in a variety of application areas, including virtual reality and game and film production. The core of digital twins lies in the ability to quickly and comprehensively reconstruct the geometry, materials, and other physical properties of various real-world objects, enabling further editing and reuse. Therefore, in the process of acquiring real-world objects and constructing their digital twins, low-cost and efficient acquisition and efficient, high-quality reconstruction of the objects' geometry and materials have become crucial research areas.

[0003] Among them, geometry and material reconstruction based on reverse rendering has become an important research direction in the graphics and vision community. In the field of computer vision, reverse rendering reconstructs the geometry and material of objects in the real world based on multiple photographs by modeling the light propagation process. Such reconstructed objects are highly editable: not only can we render objects from different perspectives and under different lighting conditions, but we can also edit the material of the objects to give them different appearances. At the same time, reverse rendering is based on image reconstruction and can be completed through a camera or mobile phone. Compared with traditional dedicated geometry acquisition devices such as 3D scanners, and dedicated material acquisition devices such as Light Stage, it has lower requirements for hardware equipment, reduces the cost of reconstruction, and expands the scope of application.

[0004] In order to solve the problem of indeterminate solution between the appearance and physical properties of objects in reverse rendering, existing methods adopt controllable active lighting methods to reduce the degrees of freedom of the lighting model. For example, a single point light source, surface light source or three-dimensional LED matrix is ​​used to illuminate the object in a dark room. Although existing methods can reconstruct the material of three-dimensional objects with good quality, they still face the following challenges: (1) The sampling efficiency of point light source-based methods is low: point light sources can only illuminate a small area on the object at a time, which is only helpful for material reconstruction in this area. This problem is more serious on high-light objects. Secondly, the brightness distribution of point light sources is uneven, and the high-light area is easily overexposed, which affects the accuracy of image-based material reconstruction methods. (2) The reconstruction method based on surface light sources is slow and noisy: In reverse rendering, the classic method for processing surface light sources is ray tracing. This method is based on sampling. At low sampling numbers, the reconstruction result is noisy, and at high sampling rates, it will bring huge computational overhead. (3) The design, construction and maintenance costs of complex lighting-collection systems based on devices such as three-dimensional LED matrices are high, making them difficult to popularize. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by providing a method and system for reconstructing three-dimensional object materials based on surface light sources. This system utilizes data collected in a darkroom environment and, through reverse rendering of the surface light source, achieves efficient and high-quality reconstruction of materials. This system more thoroughly decouples materials from illumination, simplifies the sampling process, achieves high reconstruction efficiency, and achieves high-precision reconstruction results. Furthermore, it can be performed using consumer-grade equipment, with low setup and maintenance costs.

[0006] In order to achieve the above object, the present invention adopts the following technical solution, a method for reconstructing the material of a three-dimensional object based on a surface light source, comprising the following steps:

[0007] (1) In a darkroom environment, use a pre-calibrated camera-area light source system to capture images of a given object and use these images to register the camera;

[0008] (2) Preliminary reconstruction of the object geometry through a neural network-based free viewpoint interpolation method;

[0009] (3) Pre-calculating light source-guided visibility through ray tracing;

[0010] (4) In the framework of inverse rendering, the linear transformation cosine method is used to achieve efficient rendering of physically based materials under surface light sources. While further iteratively optimizing the geometric model, the parametric material of the model surface is reconstructed with high quality.

[0011] Furthermore, step (1) is specifically as follows: the camera-surface light source system consists of a SLR camera equipped with a fixed-focus lens and a surface light source of an LED matrix, and the two are fixed by a camera bracket; registering the camera is specifically as follows: using OpenCV to calibrate the camera intrinsic parameters, and calibrate the positions of the surface light source and the camera installed with AprilTag; in a darkroom scene, a given object is placed on a surface without highlights and rich in texture to reduce indirect light interference from the supporting plane, and at the same time, the same spatial point in the three-dimensional world is marked as a feature point in pictures of different perspectives to improve the accuracy of camera registration; during the shooting process, the ISO, exposure time and aperture size of the camera are kept unchanged, and the shooting color temperature of the camera is aligned with the illumination color temperature of the surface light source; through the software CapturingReality, the captured picture is used to calculate the camera intrinsic parameters and extrinsic parameters, the image is dedistorted, and the center of the picture is aligned with the optical center of the camera-surface light source system.

[0012] Furthermore, step (2) is specifically as follows: based on TensoSDF, input the picture taken under the active lighting environment of the darkroom, preliminarily reconstruct the rough geometry, convert it into a triangular mesh model, and reduce the number of model faces in MeshLab, organize the mesh topology, smooth the mesh and generate UV mapping.

[0013] Furthermore, step (3) is specifically as follows: pre-calculating the visibility guided by the light source by ray tracing is specifically as follows: decomposing the rendering result with shadows into the shading result calculated by linear cosine transform without considering the shadows and the pre-calculated visibility, and correcting the rendering result with the pre-calculated visibility so as to correctly handle the shadows; using the ray tracing implemented in OptiX, pre-calculating the visibility guided by the light source for each image according to the initialized geometry.

[0014] Furthermore, the iterative process in step (4) is specifically as follows: the geometric model and material are optimized through multiple rounds of iterations, and the steps of each round of iteration are as follows:

[0015] 4.1) Using differentiable grating, rasterize the scene to generate G-Buffer;

[0016] 4.2) Coloring using the differentiable linear transformation cosine algorithm;

[0017] 4.3) Use pre-calculated shadow information to correct the shading result and generate the rendering result;

[0018] 4.4) The loss function of the rendering result is sent back through the reverse rendering framework to optimize the scene.

[0019] Furthermore, in step (4), a simplified version of the Disney standard material model is introduced. By giving the roughness, metallicity, base color and transmittance parameters of the given material, the bidirectional reflectance distribution function is defined to determine the optical properties of the material.

[0020] Furthermore, step 4.1) is specifically as follows: in each round of iteration, the model is rasterized using the hardware-accelerated deferred shading-based differentiable rasterizer NVDiffRast and the result is saved in the G-Buffer.

[0021] Furthermore, step 4.2) is specifically as follows: according to the material properties at the shading point and the direction of the outgoing light, a lookup table of the cosine term of the linear transformation is queried to obtain a linear transformation, a linear transformation is performed on the integrand of the rendering equation, the original integral problem is converted into an integral problem of the spherical distribution function within the polygon, and its analytical solution is calculated; then, the result is corrected according to the calculation of the Fresnel correction term, and efficient forward rendering is supported by implementing the forward rendering function in CUDA.

[0022] Furthermore, step 4.4) is specifically as follows: comparing the rendered result with the collected data, using the color-based L2 loss function to evaluate the similarity between the two images, returning the gradient from the loss function to the scene parameters, and continuously adjusting these parameters so that the rendered image of the scene after adjustment is as similar as possible to the collected data at each perspective; and supporting efficient training by implementing the inverse return function in CUDA.

[0023] On the other hand, the present invention also provides a three-dimensional object material reconstruction system based on a surface light source, the system comprising:

[0024] The camera shooting module is used to capture images of a given object in a darkroom environment using a pre-calibrated camera-area light source system and use these images to register the camera;

[0025] The preliminary reconstruction module is used to preliminarily reconstruct the geometry of the object through a free viewpoint interpolation method based on a neural network;

[0026] Ray tracing module, for pre-calculating light-guided visibility via ray tracing;

[0027] The rendering and reconstruction module is used to achieve efficient rendering of physically based materials under surface light sources using the linear transformation cosine method within the framework of inverse rendering. While further iteratively optimizing the geometric model, it reconstructs the parametric material of the model surface with high quality.

[0028] The beneficial effects of the present invention are:

[0029] 1. Darkroom environment: All light in the darkroom comes from artificially controlled light sources. The light source position, brightness, color and other information are known, which reduces the degree of freedom of the reverse rendering problem and is conducive to decoupling materials and lighting;

[0030] 2. Surface light source: Compared to point light sources, it can illuminate a larger area while effectively avoiding overexposure and improving the efficiency of material sampling. Compared to other light source arrays, surface light sources are easier to obtain - a mass-produced LCD screen can be used;

[0031] 3. Linear transformation cosine: It avoids the amount of computation caused by the sampling integration of the light source, greatly improving the rendering efficiency of the surface light source, thereby increasing the reconstruction speed and reducing the noise of the reconstruction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Flowchart of the method of the present invention.

[0033] Figure 2 Schematic diagram of the surface light source during the calibration stage.

[0034] Figure 3 Schematic diagram of the darkroom shooting scene.

[0035] Figure 4 This is a new perspective rendering of the final reconstruction result of the present invention.

[0036] Figure 5 This is a re-lighting rendering of the final reconstruction result of the present invention. DETAILED DESCRIPTION

[0037] The specific embodiments of the present invention are further described in detail below with reference to the accompanying drawings.

[0038] Figure 1 The present invention shows the overall process of a three-dimensional object material reconstruction method based on surface light source, which includes the following four steps: 1. In a darkroom environment, a pre-calibrated camera-surface light source system is used to capture pictures of a given object, and these pictures are used to register the camera; 2. The geometry of the object is preliminarily reconstructed through a free viewpoint interpolation method based on a neural network; 3. Visibility information guided by the light source is precalculated through ray tracing; 4. Within the framework of inverse rendering, the linear transformation cosine method is used to achieve efficient rendering of physically based materials under surface light sources, and the parametric material of the model surface is reconstructed with high quality while further optimizing the geometric model.

[0039] Each key step in the present invention is further described in detail below:

[0040] (1) In a darkroom environment, use a pre-calibrated camera-surface light source system to capture images of a given object and use these images to register the camera. Specifically, the camera-surface light source system consists of a single-lens reflex camera and an LED matrix with a uniform and constant brightness. The two are fixed with a professional camera bracket. During the entire calibration and shooting process, the focal length of the camera is kept unchanged and used as a fixed-focus camera. First, calibrate the camera system, including calibrating the camera's internal parameters and calibrating the relative position of the camera's optical center and the center of the light source:

[0041] a. Calibrate the camera's intrinsic parameters. Use the camera to be calibrated to capture a calibration plate of known physical dimensions. Use OpenCV (see OpenCV team. 2024. OpenCV, https: / / opencv.org) to calibrate the camera's focal length, optical center, and distortion coefficients.

[0042] b. Calibrate the relative position of the camera optical center and the light source center. Install the following on the surface of the camera-surface light source system: Figure 2The AprilTag shown in the figure (see The APRIL Robotics Laboratory at the University of Michigan investigates Autonomy, Perception, Robotics, Interfaces, and Learning. 2010. AprilTag, https: / / april.eecs.umich.edu / software / apriltag) is aligned with the center of the light source. A camera system equipped with the AprilTag is used to photograph the mirror surface, and the relative position of the light source and camera is determined using the calibration method proposed by Whelan et al. (see Whelan T, Goesele M, Lovegrove SJ, et al. Reconstructing scenes with mirror and glass surfaces[J]. ACM Trans. Graph., 2018, 37(4): 102.).

[0043] Take pictures of a given object, specifically: Figure 3 In the darkroom scene shown, the given object is placed on a textured surface without highlights to improve the accuracy of the feature point-based camera registration algorithm. The camera is handheld and captured. During the capture process, the camera's ISO, exposure time, and aperture are kept constant to obtain a dataset with consistent brightness across multiple viewpoints. Furthermore, the camera's color temperature is aligned with that of the area light source to reduce color shift in the captured image.

[0044] To register the camera using these images, we use the CapturingReality software (see CapturingReality. 2016. Reality capture, http: / / capturingreality.com.) after data acquisition. We also use the software to mark common points in the 3D world in the multi-view images as feature points to further improve registration accuracy. We use the captured images to calculate camera intrinsic and extrinsic parameters, dedistort the images, and align the image center with the optical center of the camera system.

[0045] Segment the foreground and background of an image using the Segment Anything Model (see Meta. 2023. Segment Anything, https: / / segment-anything.com / ). For each image, mark the foreground area with a box and use the algorithm to segment the image. The result is saved as a black-and-white image, with white pixels representing the foreground and black pixels representing the background.

[0046] (2) The geometry of the object is preliminarily reconstructed using a neural network-based free viewpoint interpolation method. Specifically, TensoSDF (see Li J, Wang L, Zhang L, et al. Tensosdf: Roughness-aware tensorial representation for robust geometry and material reconstruction[J]. ACM Transactions on Graphics (TOG), 2024, 43(4): 1-13.) is used to input a darkroom image, preliminarily reconstruct the rough geometry, and convert it into a triangular mesh model. Then, the Quadric Edge Collapse Decimation algorithm is used in MeshLab to simplify the model and optimize the mesh topology. The mesh is then smoothed using the HC Laplacian Smooth algorithm, and finally, a UV mapping is generated for the mesh using Blender or Xatlas. Although this algorithm can also reconstruct the initial material in this step, the quality of the material reconstruction is not high enough, so it is not used in this invention.

[0047] (3) Pre-calculating light source-guided visibility information through ray tracing. Specifically, although using differentiable linear cosine transforms to process surface light sources in the inverse rendering framework can significantly improve the efficiency of the algorithm, it ignores shadows caused by occlusion. Even when the camera is very close to the light source, there are still non-negligible shadows in the captured data. If these are not taken into account, the algorithm will mistakenly use the base color to fit the shadow effect.

[0048] Since the light source has a very strong directionality and its position is known in the shooting setting of the present invention, the present invention is inspired by Heitz et al. (see Heitz E, Hill S, McGuire M. Combining analytic directillumination and stochastic shadows[C] / / Proceedings of the ACM SIGGRAPHsymposium on interactive 3D graphics and games. 2018: 1-11.) and proposes light source guided visibility precomputation. Decomposed into the shading result calculated by linear cosine transform without considering shadows and precomputed visibility :

[0049]

[0050] in, , is visibility (the value is 0 when the light source is not visible in the direction of the shading point; conversely, the value is 1 when the light source is visible), is the bidirectional reflectance distribution function, which describes the material properties of the object. is the projection of the surface light source on the hemisphere at the shading point, is the brightness of the light source, represents the direction of the incident light, represents the direction of the outgoing light, represents the location of the shading point, It is the angle between the incident light direction and the normal direction of the shading point.

[0051] Precompute visibility for each image in the training dataset Based on the initial geometry, the optimization was performed using Monte Carlo integration using ray tracing with a sampling rate of 1024 as implemented in OptiX. Since the object material was unknown before optimization began, a perfect Lambert diffuse material was assumed. The computation took approximately 15 seconds on 200 training images.

[0052] (4) In the framework of inverse rendering, the linear transformation cosine method is used to achieve efficient rendering of physically based materials under surface light sources. While further optimizing the geometric model, the parametric material of the model surface is reconstructed with high quality. Specifically, in each round of iteration, the model is rasterized using the hardware-accelerated deferred shading-based differentiable rasterizer NVDiffRast (see LaineS, Hellsten J, Karras T, et al. Modular primitives for high-performance differentiable rendering[J]. ACM Transactions on Graphics (ToG), 2020, 39(6):1-14.) and the result is saved in the G-Buffer. Then, according to the geometric and material information of each pixel, the differentiable linear transformation cosine algorithm is used to achieve fast physically based shading, and the shading result is corrected using pre-calculated visibility information to render an image with shadows. The rendered results are compared with the collected data, and the similarity between the two images is evaluated using the color-based L2 loss function. The gradient from the loss function is propagated back to the scene parameters along the differentiable rendering process. By continuously adjusting these parameters, the rendered images of the adjusted scene at each perspective are made as similar as possible to the collected data.

[0053] High-quality parametric materials are used to recreate the model surface. Specifically, a simplified Disney standard material model (Brent Burley and Walt Disney Animation Studios. 2012. Physically-based shading at Disney. In Acm Siggraph, Vol. 2012. vol. 2012, 1–7) is introduced. Physically meaningful parameters such as roughness, metallicity, base color, and transmittance are given to define a bidirectional reflectance distribution function (BRDF), which in turn determines the material's optical properties. In the rendering equation, given the outgoing direction, the four-dimensional BRDF is further simplified to a two-dimensional spherical distribution function with respect to the incoming direction.

[0054] Shading is performed using a differentiable linearly transformed cosine algorithm. Specifically, a lookup table of linearly transformed cosine terms (Eric Heitz, Jonathan Dupuy, Stephen Hill, and David Neubelt. 2016. Real-time polygonal light shading with linearly transformed cosines. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1–8.) is queried based on the material properties at the shading point and the direction of the outgoing light. This yields a linear transformation that transforms the integrand of the rendering equation, i.e., the two-dimensional spherical distribution function, into the spherical cosine distribution function. This transformation reduces the spherical integral problem of solving the rendering equation to the integral of the spherical cosine distribution function within a polygon, which has an analytical solution (see Lambert J H. Photometria sive de mensura et gradibusluminus, colorum et umbrae (1760)[J]. Published in German by E. Anding under the title Lambert's Photometrie, Verlag von Wilhelm Engelmann, Leipzig, 1892.). Next, a table of pre-calculated Fresnel correction values ​​from the linear cosine transform is searched to correct the rendering result, introducing the effect of the Fresnel term. The table lookup, linear transformation, and line integral operations involved in the forward rendering process are all differentiable. By implementing the forward rendering and inverse return functions for these operations in CUDA, efficient forward rendering and gradient backpropagation are supported, thereby achieving efficient inverse rendering based on area light sources.

[0055] The original integral problem is transformed into an integral of the spherical cosine distribution function within a polygon, for which an analytical solution exists. Specifically, to project the polygonal area light source onto the integrating hemisphere, each vertex of the area light source polygon is first normalized and projected onto the unit sphere. Then, based on the positional relationship between each vertex and the tangent plane at the shading point, the portion below the tangent plane is clipped.

[0056] During this process, there is a certain probability that two vertices in the clipped polygon will be too close, which will cause numerical instability in forward rendering and backpropagation. Therefore, in this case, we actively merge close vertices to make the optimization process more stable without significantly affecting the rendering results and gradients.

[0057] like Figure 4As shown, the rendered image of the reconstruction result of the real shot data is compared with the captured image, and as Figure 5 The re-illuminated rendering of a real-life 3D object qualitatively demonstrates the present invention's ability to accurately restore material texture, color, and specular reflections. Comparative experiments also quantitatively demonstrate the present invention's advantages over existing solutions. Table 1 compares the present invention with other existing solutions in terms of new viewpoint synthesis, re-illumination, and reconstruction time.

[0058] Table 1. Quantitative comparison of the present invention and the prior art solutions on synthetic data sets

[0059]

[0060] In summary, the present invention provides a three-dimensional object material reconstruction method based on surface light source, which has the characteristics of low rendering overhead, high shooting sampling rate and low equipment requirements, and supports fast, high-quality and efficient reconstruction of three-dimensional object materials.

[0061] On the other hand, corresponding to the aforementioned embodiment of the method for reconstructing 3D object material based on a surface light source, the present invention also provides an embodiment of a system for reconstructing 3D object material based on a surface light source. This system includes a camera capture module, a preliminary reconstruction module, a ray tracing module, and a rendering and reconstruction module. The implementation of each module is described in detail in the aforementioned embodiment of the method for reconstructing 3D object material based on a surface light source.

[0062] The camera shooting module is used to capture images of a given object in a darkroom environment using a pre-calibrated camera-area light source system and use these images to register the camera;

[0063] The preliminary reconstruction module is used to preliminarily reconstruct the geometry of the object through a free viewpoint interpolation method based on a neural network;

[0064] Ray tracing module, for pre-calculating light-guided visibility via ray tracing;

[0065] The rendering and reconstruction module is used to achieve efficient rendering of physically based materials under surface light sources using the linear transformation cosine method within the framework of inverse rendering. While further iteratively optimizing the geometric model, it reconstructs the parametric material of the model surface with high quality.

[0066] The above embodiments are used to illustrate the present invention rather than to limit the present invention. Any modifications and changes made to the present invention within the spirit of the present invention and the protection scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A method for reconstructing three-dimensional object material based on surface light source, characterized in that: The following steps are involved: (1) In a darkroom environment, use a pre-calibrated camera-area light source system to capture images of a given object and use these images to register the camera; (2) Preliminary reconstruction of the object geometry through a free viewpoint interpolation method based on a neural network; (3) Pre-calculate the visibility of light source guidance through ray tracing; specifically, decompose the rendering result with shadows into the shading result without considering shadows calculated by linear transformation cosine and pre-calculated visibility, and use the pre-calculated visibility to correct the rendering result so as to correctly handle shadows; use ray tracing implemented in OptiX to pre-calculate the visibility of light source guidance for each image according to the initialization geometry; (4) A simplified version of the Disney standard material model is introduced. By giving the material's roughness, metallicity, base color, and transmittance parameters, a bidirectional reflectance distribution function is defined to determine the material's optical properties. Within the framework of inverse rendering, the linear transformation cosine method is used to achieve efficient rendering of physically based materials under surface light sources. The geometric model and material are optimized through multiple rounds of iteration. The steps of each round of iteration are as follows: 4.1) Using differentiable grating, rasterize the scene to generate G-Buffer; 4.2) Shading using a differentiable linear transformation cosine algorithm. Specifically, the method involves querying a lookup table for the linear transformation cosine term based on the material properties at the shading point and the direction of the outgoing light ray. This method then performs a linear transformation on the integrand of the rendering equation, transforming the original integral problem into an integral of the spherical distribution function within the polygon and computing its analytical solution. Then, the result is corrected based on the calculated Fresnel correction term, and efficient forward rendering is supported by implementing the forward rendering function in CUDA; 4.3) Use pre-computed visibility information to correct the shading result and generate the rendering result; 4.4) The loss function of the rendering result is returned through the inverse rendering framework to reconstruct the parametric material of the model surface with high quality.

2. The method for reconstructing three-dimensional object material based on surface light source according to claim 1, characterized in that: Step (1) is specifically as follows: the camera-surface light source system consists of a SLR camera equipped with a fixed-focus lens and a surface light source of an LED matrix, and the two are fixed with a camera bracket; registering the camera is specifically as follows: using OpenCV to calibrate the camera internal parameters, and calibrate the position of the surface light source and the camera installed with AprilTag; in a darkroom scene, the given object is placed on a surface without highlights and with rich textures to reduce indirect lighting interference from the supporting plane, and at the same time, the same spatial point in the three-dimensional world is marked as a feature point in pictures with different perspectives to improve the accuracy of camera registration; During the shooting process, the camera's ISO, exposure time, and aperture size are kept constant, and the camera's shooting color temperature is aligned with the color temperature of the area light source. Using the software CapturingReality, the captured image is used to calculate the camera's intrinsic and extrinsic parameters, dedistort the image, and align the image center with the optical center of the camera-area light source system.

3. The method for reconstructing three-dimensional object material based on surface light source according to claim 1, characterized in that: Step (2) is as follows: Based on TensoSDF, input the picture taken in the darkroom active lighting environment, preliminarily reconstruct the rough geometry, convert it into a triangular mesh model, and reduce the number of model faces in MeshLab, organize the mesh topology, smooth the mesh and generate UV mapping.

4. The method for reconstructing three-dimensional object material based on surface light source according to claim 1, characterized in that: Step 4.1) is as follows: In each round of iteration, the model is rasterized using the hardware-accelerated deferred shading-based differentiable rasterizer NVDiffRast and the result is saved in the G-Buffer.

5. The method for reconstructing three-dimensional object material based on surface light source according to claim 1, characterized in that: Step 4.4) specifically involves comparing the rendered image with the captured data, evaluating the similarity between the two images using a color-based L2 loss function, propagating the gradient from the loss function back to the scene parameters, and continuously adjusting these parameters so that the rendered image of the scene at each viewpoint is as similar as possible to the captured data. Efficient training is supported by implementing the inverse propagation function in CUDA.

6. A three-dimensional object material reconstruction system based on a surface light source for implementing the method according to any one of claims 1 to 5, characterized in that: The system includes: The camera shooting module is used to capture images of a given object in a darkroom environment using a pre-calibrated camera-area light source system and use these images to register the camera; The preliminary reconstruction module is used to preliminarily reconstruct the geometry of the object through a free viewpoint interpolation method based on a neural network; Ray tracing module, for pre-calculating light-guided visibility via ray tracing; The rendering and reconstruction module is used to achieve efficient rendering of physically based materials under surface light sources using the linear transformation cosine method within the framework of inverse rendering. While further iteratively optimizing the geometric model, it reconstructs the parametric material of the model surface with high quality.

Images