Three-dimensional object material reconstruction method and system based on area light source

Abstract

The invention discloses a three-dimensional object material reconstruction method and system based on an area light source, and the method comprises the steps: firstly, in a darkroom environment, employing a pre-calibrated camera-area light source system to shoot and collect pictures of a given object, and employing the pictures to register a camera; secondly, preliminarily reconstructing the geometry of the object through a free viewpoint interpolation method based on a neural network; secondly, pre-calculating visibility guided by a light source through ray tracing; and finally, under a reverse rendering framework, realizing high-efficiency rendering of a physical-based material under an area light source by using a linear cosine transformation method, and reconstructing a parameterized material on the surface of the model with high quality while further optimizing the geometric model. According to the invention, the rapid and high-quality material reconstruction of the three-dimensional object can be realized only by simply shooting and collecting by using the shooting equipment which is easy to obtain. The method can be used in the fields of virtual reality object modeling and the like.

Description

Technical Field

[0001] The present invention relates to the fields of computer graphics and 3D vision, and particularly to a method and system for reconstructing the material of a 3D object based on an area light source. Background Art

[0002] With the progress of computer vision and computer graphics, the digital twin of real-world objects, as a new type of 3D asset, has shown broad prospects in multiple application fields such as virtual reality, game and movie production. The core of the digital twin lies in being able to quickly and comprehensively reconstruct the physical properties such as the geometry and material of various real-world objects, so as to be further edited and used. Therefore, in the process of collecting real-world objects to build their digital twins, low-cost and high-efficiency acquisition and high-efficiency and high-quality reconstruction of the geometry and material of the objects have become crucial research directions.

[0003] Among them, the geometry and material reconstruction based on inverse rendering has become an important research direction in the graphics and vision community. In the field of computer vision, inverse rendering reconstructs the geometry and material of objects in the real world based on multiple captured photos by modeling the light propagation process. The reconstructed objects have strong editability: we can not only render the objects from different perspectives under different lighting conditions, but also edit the object materials to make them show different appearances. At the same time, inverse rendering is based on image reconstruction and can be completed by a camera or a mobile phone. Compared with traditional special geometric acquisition devices such as 3D scanners and special material acquisition devices such as Light Stage, it has lower requirements for hardware devices, reduces the cost of reconstruction, and expands the application scope.

[0004] To solve the problem of the under-determined solution between the appearance and physical properties of objects in inverse rendering, existing methods adopt the method of controllable active lighting to reduce the degree of freedom of the lighting model. For example, a single point light source, area light source or 3D LED matrix is used to illuminate the object in a dark room. Although existing methods can reconstruct the material of 3D objects with good quality, they still face the following challenges: (1) The sampling efficiency of the method based on a point light source is low: A point light source can only illuminate a small part of the object at a time and only helps in reconstructing the material within that area. This problem is more serious for specular objects. Secondly, the brightness distribution of the point light source is uneven, and the specular area is prone to overexposure, affecting the accuracy of the image-based material reconstruction method. (2) The method based on an area light source is slow in reconstruction and has large noise: In inverse rendering, the classic method for processing an area light source is ray tracing. This method is based on sampling. At low sampling numbers, the reconstruction result has large noise, while at high sampling rates, it will bring huge computational overhead. (3) The design, construction, and maintenance costs of complex lighting-acquisition systems based on devices such as 3D LED matrices are high and not convenient for popularization. Summary of the Invention

[0005] In view of the deficiencies of the prior art, the present invention provides a method and system for reconstructing the material of a three-dimensional object based on a surface light source, which can utilize the data collected in a darkroom environment and achieve efficient and high-quality reconstruction of the material through reverse rendering of the surface light source. This system can more thoroughly decouple the material from the lighting, has a simple sampling process, high reconstruction efficiency, high-precision reconstruction results, and can be completed using consumer-grade devices, with low setup and maintenance costs.

[0006] To achieve the above objectives, the present invention adopts the following technical solutions. A method for reconstructing the material of a three-dimensional object based on a surface light source includes the following steps:

[0007] (1) In a darkroom environment, use a pre-calibrated camera-surface light source system to take pictures of a given object, and use these pictures to register the camera;

[0008] (2) Through a free-viewpoint interpolation method based on a neural network, preliminarily reconstruct the geometry of the object;

[0009] (3) Pre-compute the visibility guided by the light source through ray tracing;

[0010] (4) In the framework of reverse rendering, use the method of linear transformation cosine to achieve efficient rendering of physically based materials under a surface light source, and while further iteratively optimizing the geometric model, high-quality parametric materials on the surface of the model are reconstructed.

[0011] Further, step (1) is specifically as follows: The camera-surface light source system consists of a single-lens reflex camera equipped with a fixed-focus lens and a surface light source of an LED matrix, and the two are fixed using a camera bracket; Registering the camera is specifically as follows: Use OpenCV to calibrate the internal parameters of the camera, and calibrate the position of the surface light source installed with AprilTag and the camera; In a darkroom scene, place the given object on a surface without highlights and with rich textures to reduce the interference of indirect light from the support plane. At the same time, mark the same spatial point in the three-dimensional world in pictures from different perspectives as feature points to improve the accuracy of camera registration; During the shooting process, keep the ISO, exposure time, and aperture size of the camera unchanged, and align the shooting color temperature of the camera with the illumination color temperature of the surface light source; Through the software CapturingReality, use the taken pictures to calculate the internal and external parameters of the camera, de-distort the images, and align the center of the picture with the optical center of the camera-surface light source system.

[0012] Further, step (2) is specifically as follows: Based on TensoSDF, input the pictures taken in the darkroom active illumination environment, preliminarily reconstruct the approximate geometry, convert it into a triangular mesh model, and reduce the number of model faces in MeshLab, organize the mesh topology, smooth the mesh, and generate a UV mapping.

[0013] Further, step (3) is specifically as follows: The visibility guided by the light source through ray tracing pre - calculation is specifically: Decompose the rendering result with shadows into the shading result without considering shadows calculated by linear cosine transform and the pre - calculated visibility, and use the pre - calculated visibility to correct the rendering result to correctly handle shadows; Adopt ray tracing implemented in OptiX to pre - calculate the visibility guided by the light source for each image according to the initialized geometry.

[0014] Further, the iteration process in step (4) is specifically as follows: Optimize the geometry model and materials through multiple rounds of iteration. The steps of each round of iteration are specifically as follows:

[0015] 4.1) Use a differentiable rasterizer to rasterize the scene to generate a G - Buffer;

[0016] 4.2) Shade using a differentiable linear transformation cosine algorithm;

[0017] 4.3) Correct the shading result with pre - calculated shadow information to generate a rendering result;

[0018] 4.4) Back - propagate the loss function of the rendering result through a reverse rendering framework to optimize the scene.

[0019] Further, in step (4), a simplified version of the Disney standard material model is introduced. By specifying the roughness, metallicity, base color, and transmittance parameters of the material, define the bidirectional reflectance distribution function, and then determine the optical properties of the material.

[0020] Further, step 4.1) is specifically as follows: In each round of iteration, use a hardware - accelerated differentiable rasterizer based on deferred shading, NVDiffRast, to rasterize the model and save the result in the G - Buffer.

[0021] Further, step 4.2) is specifically as follows: Query the lookup table of the linear transformation cosine term according to the material properties and the outgoing ray direction at the shading point to obtain a linear transformation, perform a linear transformation on the integrand of the rendering equation, transform the original integral problem into an integral problem of a spherical distribution function within a polygon, and calculate its analytical solution; Then correct the result according to the calculated Fresnel correction term, and support efficient forward rendering by implementing a forward rendering function in CUDA.

[0022] Further, step 4.4) is specifically as follows: Compare the rendering result with the acquired data, use a color - based L2 loss function to evaluate the similarity between the two images, back - propagate the gradient from the loss function to the scene parameters, and by continuously adjusting these parameters, make the rendered images of the adjusted scene at each perspective as similar as possible to the acquired data; Support efficient training by implementing a reverse back - propagation function in CUDA.

[0023] On the other hand, the present invention also provides a three-dimensional object material reconstruction system based on a surface light source, the system comprising:

[0024] The camera shooting module is used to capture images of a given object in a darkroom environment using a pre-calibrated camera-area light source system and use these images to register the camera;

[0025] The preliminary reconstruction module is used to preliminarily reconstruct the geometry of the object through a free viewpoint interpolation method based on a neural network;

[0026] Ray tracing module, for pre-calculating light-guided visibility via ray tracing;

[0027] The rendering and reconstruction module is used to achieve efficient rendering of physically based materials under surface light sources using the linear transformation cosine method within the framework of inverse rendering. While further iteratively optimizing the geometric model, it reconstructs the parametric material of the model surface with high quality.

[0028] The beneficial effects of the present invention are:

[0029] 1. Darkroom environment: All light in the darkroom comes from artificially controlled light sources. The light source position, brightness, color and other information are known, which reduces the degree of freedom of the reverse rendering problem and is conducive to decoupling materials and lighting;

[0030] 2. Surface light source: Compared to point light sources, it can illuminate a larger area while effectively avoiding overexposure and improving the efficiency of material sampling. Compared to other light source arrays, surface light sources are easier to obtain - a mass-produced LCD screen can be used;

[0031] 3. Linear transformation cosine: It avoids the amount of computation caused by the sampling integration of the light source, greatly improving the rendering efficiency of the surface light source, thereby increasing the reconstruction speed and reducing the noise of the reconstruction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Flowchart of the method of the present invention.

[0033] Figure 2 Schematic diagram of the surface light source during the calibration stage.

[0034] Figure 3 Schematic diagram of the darkroom shooting scene.

[0035] Figure 4 This is a new perspective rendering of the final reconstruction result of the present invention.

[0036] Figure 5 This is a re-lighting rendering of the final reconstruction result of the present invention. DETAILED DESCRIPTION

[0037] The following further elaborates on the specific implementation manners of the present invention in conjunction with the accompanying drawings.

[0038] Figure 1 The overall process of a three-dimensional object material reconstruction method based on a surface light source provided by the present invention is shown, including the following four steps: 1. In a darkroom environment, use a pre-calibrated camera-surface light source system to capture pictures of a given object, and use these pictures to register the camera; 2. Through a free-viewpoint interpolation method based on a neural network, preliminarily reconstruct the geometry of the object; 3. Pre-compute the visibility information guided by the light source through ray tracing; 4. In the framework of inverse rendering, use the method of linear transformation cosine to achieve efficient rendering of physically based materials under the surface light source, while further optimizing the geometric model and reconstructing the parametric materials on the surface of the model with high quality.

[0039] The following further elaborates on each key step in the present invention:

[0040] (1) In a darkroom environment, use a pre-calibrated camera-surface light source system to capture pictures of a given object, and use these pictures to register the camera. Specifically: The camera-surface light source system consists of a surface light source with a uniform and constant brightness composed of a single-lens reflex camera and an LED matrix, and the two are fixed using a professional camera bracket. During the entire process of calibration and shooting, keep the focal length of the camera unchanged and use it as a fixed-focus camera. First, calibrate this camera system, including calibrating the internal parameters of the camera and the relative positions of the camera optical center and the light source center:

[0041] a. Calibrate the internal parameters of the camera. Use the camera to be calibrated to capture a calibration board with a known physical size, and calibrate the focal length, optical center, and distortion coefficients of the camera through OpenCV (see OpenCV team. 2024. OpenCV, https: / / opencv.org).

[0042] b. Calibrate the relative positions of the camera optical center and the light source center. Install on the surface of the surface light source of the camera-surface light source system as Figure 2The AprilTag shown (see The APRIL Robotics Laboratory at the University of Michigan investigates Autonomy, Perception, Robotics, Interfaces, and Learning. 2010. AprilTag, https: / / april.eecs.umich.edu / software / apriltag) aligns the center of the AprilTag with the center of the light source plane. Use the camera system equipped with the AprilTag to take pictures of the mirror surface, and determine the relative position between the light source and the camera through the calibration method proposed by Whelan et al. (see Whelan T, Goesele M, Lovegrove S J, et al. Reconstructing scenes with mirror and glass surfaces[J]. ACM Trans. Graph., 2018, 37(4): 102.).

[0043] Take pictures of a given object for acquisition, specifically: In the darkroom scene as Figure 3 shown, place the given object on a surface without highlights and with rich texture to improve the accuracy of the feature point-based camera registration algorithm. Hold the camera and take pictures of the given object. During the shooting process, keep the ISO, exposure time, and aperture size of the camera unchanged to obtain a dataset with consistent brightness from multiple perspectives. At the same time, align the shooting color temperature of the camera with the illumination color temperature of the surface light source to reduce the color shift in the shooting.

[0044] Use these pictures to register the camera, specifically: After completing the data acquisition, through the software CapturingReality (see CapturingReality. 2016. Reality capture, http: / / capturingreality.com.), and at the same time, by marking the same spatial point in the three-dimensional world as a feature point in the multi-perspective pictures, further improve the registration accuracy. Use the taken pictures to calculate the internal and external parameters of the camera, undistort the images, and align the center of the picture with the optical center of the camera system.

[0045] Segment the foreground and background of the image through the Segment Anything Model (see Meta. 2023. Segment Anything, https: / / segment-anything.com / ). For each captured image, mark the area where the foreground subject is located with a square box, use this algorithm to achieve segmentation, and save the result as a black-and-white image, where the foreground is identified by white pixels and the background is identified by black pixels.

[0046] (2) Initially reconstruct the geometry of the object through a neural network-based free-viewpoint interpolation method, specifically: through TensoSDF (see Li J, Wang L, Zhang L, et al. Tensosdf: Roughness-aware tensorial representation for robust geometry and material reconstruction[J]. ACM Transactions on Graphics (TOG), 2024, 43(4): 1-13.), input the images captured in a darkroom environment, initially reconstruct the approximate geometry, and convert it into a triangular mesh model. Then, in MeshLab, simplify the model and optimize the mesh topology wiring through the Quadric Edge Collapse Decimation algorithm, then smooth the mesh through the HC Laplacian Smooth algorithm, and finally generate UV mapping for the mesh through Blender or Xatlas. In this step, although this algorithm can also reconstruct the initial material, the quality of the material reconstruction is not high enough, so it is not adopted in the present invention.

[0047] (3) Pre-compute the visibility information guided by the light source through ray tracing, specifically: Although using a differentiable linear cosine transform to handle area lights in the framework of inverse rendering can significantly improve the efficiency of the algorithm, it ignores the shadows caused by occlusion. Although when the distance between the camera and the light source is very close, there are still non-negligible shadows in the captured data. If not considered, the algorithm will wrongly fit the shadow effect with the base color.

[0048] Since, under the shooting settings of the present invention, the light source has a very strong directivity and its position is known, the present invention is inspired by Heitz et al. (see Heitz E, Hill S, McGuire M. Combining analytic direct illumination and stochastic shadows[C] / / Proceedings of the ACM SIGGRAPH symposium on interactive 3D graphics and games. 2018: 1-11.), and proposes pre-computation of visibility guided by the light source. The rendering result with shadows is decomposed into the shading result without considering shadows calculated by linear cosine transform and pre-computed visibility :

[0049]

[0050] wherein, , is visibility (when the light source is invisible in this direction at the shading point, the value is 0; conversely, when the light source is visible, the value is 1), is the bidirectional reflectance distribution function, which describes the material properties of the object, is the projection of the area light source on the integral hemisphere at the shading point, is the brightness of the light source, represents the direction of the incident light, represents the direction of the outgoing light, represents the position of the shading point, is the angle between the incident light direction and the normal direction of the shading point.

[0051] When pre-computing the visibility for each image in the training dataset, according to the initialized geometry, ray tracing with a sampling rate of 1024 implemented in OptiX is used to calculate by Monte Carlo integration. Since the material of the object is unknown before the optimization starts, it is assumed that the object is an ideal Lambertian diffuse material. It takes about 15 seconds to calculate on 200 training images.

[0052] (4) Under the framework of reverse rendering, the method of linear transformation cosine is used to achieve efficient rendering of physically-based materials under area lights. While further optimizing the geometric model, the parametric materials on the model surface are reconstructed with high quality. Specifically, in each iteration, the hardware-accelerated differentiable rasterizer NVDiffRast based on deferred shading (see Laine S, Hellsten J, Karras T, et al. Modular primitives for high-performance differentiable rendering[J]. ACM Transactions on Graphics (ToG), 2020, 39(6): 1-14.) is used to rasterize the model and save the results in the G-Buffer. Then, according to the geometric and material information of each pixel, a fast physically-based shading is achieved using the differentiable linear transformation cosine algorithm, and the shading result is corrected with pre-computed visibility information to render an image with shadows. The rendered result is compared with the acquired data, and the similarity between the two images is evaluated using the color-based L2 loss function. The gradient from the loss function is backpropagated along the differentiable rendering process to the scene parameters. By continuously adjusting these parameters, the rendered images of the adjusted scene from various viewpoints are made as similar as possible to the acquired data.

[0053] The parametric materials on the model surface are reconstructed with high quality. Specifically, a simplified Disney standard material model is introduced (Brent Burley and Walt Disney Animation Studios. 2012. Physically-based shading at disney. In Acm Siggraph, Vol. 2012. vol. 2012, 1–7.). By specifying physically meaningful parameters such as the roughness, metallicity, base color, and transmittance of the material, the bidirectional reflectance distribution function is defined, and then the optical properties of the material are determined. In the rendering equation, given the outgoing direction, the four-dimensional bidirectional reflectance distribution function is further simplified to a two-dimensional spherical distribution function with respect to the incoming direction.

[0054] Coloring is performed using the differentiable linearly transformed cosine algorithm. Specifically, a lookup table of linearly transformed cosine terms is queried based on the material properties at the coloring point and the outgoing light direction (Eric Heitz, Jonathan Dupuy, Stephen Hill, and David Neubelt. 2016. Real-time polygonal light shading with linearly transformed cosines. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1–8.). A linear transformation is obtained that can transform the integrand of the rendering equation, i.e., the two-dimensional spherical distribution function, into a spherical cosine distribution function. Based on this transformation, the spherical integral problem of solving the rendering equation is reduced to the integral problem of the spherical cosine distribution function with an analytical solution within the polygon (see Lambert J H. Photometria sive de mensura et gradibus luminus, colorum et umbrae (1760)[J]. Published in german by E. Anding under the title Lambert’s Photometrie, Verlag von Wilhelm Engelmann, Leipzig, 1892.). Then, a table of pre-computed Fresnel correction values for the linear cosine transform is looked up to correct the rendering result and introduce the effect of the Fresnel term. The lookup table queries, linear transformations, and line integral operations involved in the above forward rendering process are all differentiable. By implementing the forward rendering function and the reverse pass function for the above operations in CUDA, efficient forward rendering and gradient backpropagation are supported, thus enabling efficient reverse rendering based on area lights.

[0055] The original integral problem is transformed into the integral problem of the spherical cosine distribution function with an analytical solution within the polygon. Specifically, to project the polygonal area light onto the integration hemisphere, first, each vertex of the area light polygon is regularized and projected onto the unit sphere. Then, according to the positional relationship between each vertex and the tangent plane of the coloring point, the part below the tangent plane is clipped off.

[0056] During this process, there is a certain probability that in the polygon after clipping, two vertices are too close, which will lead to numerical instability in forward rendering and backpropagation. Therefore, in this case, the close vertices are actively merged to make the optimization process more stable without significantly affecting the rendering result and the gradient.

[0057] Such as Figure 4As shown, the rendered image of the reconstruction result of the real-shot data is compared with the captured image, and the relighting rendering image of the real-shot 3D object as shown in Figure 5 qualitatively proves that the present invention can correctly restore characteristics such as material texture color and specular reflection. At the same time, the comparative experiment quantitatively proves the advantages of the present invention compared with the existing technical solutions. Table 1 shows the index comparison of the present invention and other existing technical solutions in terms of novel view synthesis, relighting, and reconstruction time.

[0058] Table 1. Quantitative comparison between the present invention and the existing technical solutions on the synthetic dataset

[0059]

[0060] In summary, the present invention provides a method for reconstructing the material of a 3D object based on a surface light source, which has the characteristics of low rendering overhead, high shooting sampling rate, and low equipment requirements, and supports fast, high-quality, and efficient reconstruction of the material of a 3D object.

[0061] On the other hand, corresponding to the foregoing embodiments of a method for reconstructing the material of a 3D object based on a surface light source, the present invention also provides an embodiment of a system for reconstructing the material of a 3D object based on a surface light source. The system includes a camera shooting module, a preliminary reconstruction module, a ray tracing module, and a rendering and reconstruction module; the implementation processes of each module please refer to the specific steps of the foregoing embodiments of the method for reconstructing the material of a 3D object based on a surface light source.

[0062] The camera shooting module is used to capture pictures of a given object using a pre-calibrated camera-surface light source system in a darkroom environment, and register the camera using these pictures;

[0063] The preliminary reconstruction module is used to preliminarily reconstruct the geometry of the object by a free-viewpoint interpolation method based on a neural network;

[0064] The ray tracing module is used to pre-compute the visibility guided by the light source through ray tracing;

[0065] The rendering and reconstruction module is used to implement efficient rendering of physically based materials under a surface light source by a method of linear transformation cosine in the framework of inverse rendering, and while further iteratively optimizing the geometric model, high-quality parametric materials on the surface of the model are reconstructed.

[0066] The above embodiments are used to explain the present invention, rather than limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims of the present invention fall within the protection scope of the present invention.

Claims

1. A method for reconstructing the material of a three-dimensional object based on a surface light source, characterized in that, It includes the following steps: (1) In a darkroom environment, use a pre-calibrated camera-light panel system to capture pictures of a given object, and use these pictures to register the camera; (2) Through a free-viewpoint interpolation method based on a neural network, preliminarily reconstruct the geometry of the object; (3) Pre-compute the visibility guided by the light source through ray tracing; (4) Under the framework of inverse rendering, use the method of linear transformation cosine to achieve efficient rendering of physically based materials under a light panel. While further iteratively optimizing the geometry model, parametric materials on the surface of the model are reconstructed with high quality.

2. The three-dimensional object material reconstruction method based on a surface light source according to claim 1, wherein, Step (1) is specifically as follows: The camera-light panel system consists of a single-lens reflex camera equipped with a fixed-focus lens and a light panel of an LED matrix, and the two are fixed using a camera support; Camera registration is specifically as follows: Use OpenCV to calibrate the internal parameters of the camera, and calibrate the position of the light panel with an AprilTag installed and the camera; In a darkroom scene, place the given object on a surface without highlights and with rich textures to reduce the interference of indirect light from the support plane. At the same time, mark the same spatial point in the three-dimensional world in pictures from different perspectives as feature points to improve the accuracy of camera registration; During the shooting process, keep the ISO, exposure time, and aperture size of the camera unchanged, and align the shooting color temperature of the camera with the illumination color temperature of the light panel; Through the software CapturingReality, use the captured pictures to calculate the internal and external parameters of the camera, de-distort the images, and align the center of the picture with the optical center of the camera-light panel system.

3. The three-dimensional object material reconstruction method based on a surface light source according to claim 1, characterized in that Step (2) is specifically as follows: Based on TensoSDF, input the pictures taken in the darkroom active illumination environment, preliminarily reconstruct the general geometry, convert it into a triangular mesh model, and reduce the number of model faces, organize the mesh topology, smooth the mesh, and generate a UV mapping in MeshLab.

4. The three-dimensional object material reconstruction method based on a surface light source according to claim 1, characterized in that Step (3) is specifically as follows: Pre-computing the visibility guided by the light source through ray tracing is specifically as follows: Decompose the rendered result with shadows into the shading result without considering shadows calculated by linear cosine transformation and the pre-computed visibility, and use the pre-computed visibility to correct the rendered result to correctly handle shadows; Adopt ray tracing implemented in OptiX to pre-compute the visibility guided by the light source for each image according to the initialized geometry.

5. The three-dimensional object material reconstruction method based on a surface light source according to claim 1, wherein The iterative process in step (4) is specifically as follows: Optimize the geometry model and materials through multiple rounds of iteration. The steps of each round of iteration are specifically as follows: 4.1) Use a differentiable rasterizer to rasterize the scene to generate a G-Buffer; 4.2) Shade using a differentiable linear transformation cosine algorithm; 4.3) Correct the shading result with pre-computed shadow information to generate a rendered result; 4.4) Backpropagate the loss function of the rendered result through the inverse rendering framework to optimize the scene.

6. The method for reconstructing the material of a three-dimensional object based on a surface light source according to claim 1, wherein In step (4), a simplified version of the Disney standard material model is introduced. By specifying the roughness, metallicity, base color, and transmittance parameters of the given material, define the bidirectional reflectance distribution function, and then determine the optical properties of the material.

7. The method for reconstructing the material of a three-dimensional object based on a surface light source according to claim 5, wherein Step 4.1) Specifically: In each iteration, the hardware-accelerated differentiable rasterizer NVDiffRast based on deferred shading is used to rasterize the model and save the result in the G-Buffer.

8. The method for reconstructing the material of a three-dimensional object based on a surface light source according to claim 5, wherein Step 4.2) Specifically: According to the material properties and outgoing ray directions at the shading points, look up the lookup table of the linear transformation cosine term to obtain the linear transformation, perform a linear transformation on the integrand of the rendering equation, transform the original integral problem into an integral problem of the spherical distribution function within the polygon, and calculate its analytical solution; Then, correct the result according to the calculated Fresnel correction term, and support efficient forward rendering by implementing the forward rendering function in CUDA.

9. The three-dimensional object material reconstruction method based on a surface light source according to claim 5, characterized in that Step 4.4) Specifically: Compare the rendering result with the acquired data, use the color-based L2 loss function to evaluate the similarity between the two images, backpropagate the gradient from the loss function to the scene parameters, and by continuously adjusting these parameters, make the rendered images of the adjusted scene at each view as similar as possible to the acquired data; support efficient training by implementing the backward backpropagation function in CUDA.

10. A three-dimensional object material reconstruction system based on a surface light source for implementing the method according to any one of claims 1-9, characterized in that, The system includes: A camera shooting module, which is used to shoot and collect pictures of a given object using a pre-calibrated camera-light source system in a darkroom environment, and register the camera using these pictures; A preliminary reconstruction module, which is used to preliminarily reconstruct the geometry of the object by means of neural network-based free-viewpoint interpolation method; A ray tracing module, which is used to pre-compute the visibility guided by the light source through ray tracing; A rendering and reconstruction module, which is used to implement efficient rendering of physically based materials under a surface light source by means of the linear transformation cosine method in the framework of inverse rendering, and while further iteratively optimizing the geometric model, reconstruct the parameterized materials on the surface of the model with high quality.

Images