Structured light methods for high-precision acquisition of geometry and materials from a single viewpoint

Abstract

This invention discloses a structured light method for high-precision acquisition of geometry and material information from a single viewpoint. The method simulates the light field by introducing the rendering equation of an LCD mask, encodes geometric information using structured light, and decodes it using a nearest neighbor search algorithm to obtain the object's geometric information. Material information is encoded using optical path multiplexing, and differentiable optimization is performed based on the image captured in material mode and the acquired geometric information to obtain the object's material information. This invention combines two active light-based acquisition methods and implements an efficient encoding and decoding scheme, thus solving the problem of simultaneously acquiring the geometric and material information of static objects from a single viewpoint. This invention offers high acquisition accuracy and excellent generalization ability.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically relating to a structured light method for high-precision acquisition of geometry and material properties from a single viewpoint. Background Technology

[0002] In recent years, groundbreaking achievements in computer vision and graphics have made significant contributions to the development of applications in a range of scenarios, including cultural relic preservation, e-commerce, and visual effects. However, along with this rapid development, many critical problems have emerged that urgently need to be solved. Among them, the problem of simultaneously acquiring geometric and material information of static objects is a particularly challenging issue.

[0003] Active lighting methods comprise a large class of approaches for acquiring high signal-to-noise ratio (SNR) data, including structured illumination and illumination multiplexing. Structured illumination first projects a pre-calculated pattern onto the object using a projector, then uses triangulation to acquire geometric information. Illumination multiplexing, based on a physical model, obtains material information about the object by programming the brightness of different lights.

[0004] However, existing methods still have many problems in the joint acquisition of geometric and material information, mainly in the following aspects:

[0005] 1) Equipment design issues

[0006] Directly combining structured light illumination and optical path multiplexing methods would lead to overly complex hardware design. The use of multiple projectors and cameras would also cause information contention issues. In other words, a single light source cannot simultaneously acquire geometric and material information.

[0007] 2) Data acquisition accuracy issues

[0008] Existing methods typically employ a combination of active and passive light methods to acquire both material and geometric information of static objects. However, due to the limitations in accuracy of passive light methods, this combination often results in suboptimal outcomes. Summary of the Invention

[0009] The purpose of this invention is to address the shortcomings of existing technologies by providing a structured light method for high-precision acquisition of geometry and materials from a single viewpoint.

[0010] The objective of this invention is achieved through the following technical solution: a structured light method for high-precision acquisition of geometry and material properties from a single viewpoint, the method comprising the following steps:

[0011] A lightweight hardware prototype consisting of LED modules, LCD masks, and cameras;

[0012] The parameters of the hardware prototype are extracted, and light field simulation is performed by introducing the rendering equation of the LCD mask;

[0013] An optimizer for LCD mask pixel values ​​was built based on simulated light fields.

[0014] The optimal LCD mask pixel values ​​are obtained based on the optimizer, and the pixel values ​​of the LCD mask are modified on the hardware prototype. At the same time, a single LED in the LED module is selected and set to the brightest state.

[0015] The geometric pattern was captured using a modified hardware prototype;

[0016] Decode the image captured in geometric mode to obtain the depth information of the object;

[0017] A neural network targeting LED illumination parameters was built based on simulated light fields;

[0018] Based on the LED illumination parameters obtained from the neural network training, the brightness of the LED module is modified on the hardware prototype, and the LCD mask in the hardware prototype is set to a transparent state.

[0019] The material mode was captured using the modified hardware prototype;

[0020] Based on the image and depth information captured in the material mode, perform differentiable optimization to obtain the optimized BRDF parameters;

[0021] The final rendering result is achieved by using optimized BRDF parameters and depth information.

[0022] Furthermore, the hardware prototype is designed as follows: an object is placed inside a bounding box, the LED module and the LCD mask are placed vertically and parallel to each other; a camera is placed above the LCD mask with the lens facing the center of the bounding box; the distance from the LED module to the LCD mask is equal to the distance from the LCD mask to the center of the bounding box.

[0023] Furthermore, the parameters of the hardware prototype include the position parameters of the LED module, the light intensity distribution parameters, the position parameters of the LCD mask, the position parameters of the camera, the internal parameters of the camera, the camera's response curve, and the color calibration parameters.

[0024] Furthermore, the simulated light field uses the ray originating from the center of the LED module and passing through the center of the LCD mask as the horizontal axis x; the ray originating from the center of the LED module and passing through the left LED in the same row as the vertical axis y; and a three-dimensional point 2d away from the center of the LED module along the horizontal axis as the origin of the coordinate system, where d is the distance between the center of the LED module and the center of the LCD mask; and determines the z-axis based on the x-axis and y-axis.

[0025] Furthermore, the calibration of the position parameters and light intensity distribution parameters is performed using an image-based differentiable optimization method, specifically as follows:

[0026] Known rendering equation Where, x k Let x be a 3D point within the bounding box. l Let A be a point on the current LED light. Each LED light is modeled as a surface light source, and its area is denoted as A. For x k After the j-th LCD mask pattern M j The brightness of the projected image on the final image, ω is the value from x. k To x l The unit direction vector, L(x) l ,-ω) is from x l The brightness of the emitted light in the direction of -ω For x l x k The pixel value at the intersection of the line and the LCD mask, where p is the material reflectivity. n k and n l x k and x l The normal vector at that location;

[0027] By setting initial values ​​for position parameters and light intensity distribution parameters, a simulated light field based on the initial values ​​and rendering equations can be constructed. The Euclidean distance between the real-shot image and the simulated image output through the simulated light field is calculated pixel by pixel and input into the Adam optimizer for optimization. The final converged value is used as the result of the position parameters and light intensity distribution parameters.

[0028] Furthermore, the optimizer for LCD mask pixel values ​​built based on simulated light fields specifically includes:

[0029] By performing the process in batches, each batch takes 3D points that exist in the bounding box and belong to the same camera ray as input, obtains the results of the batch of 3D points in the simulated light field, selects one of the 3D points in the batch as a label, and uses the cross-entropy of the batch of 3D points as a loss function to optimize the pixel values ​​of the LCD mask.

[0030] Furthermore, in the shooting process of the geometric mode, the LCD mask is modified each time by the corresponding mask pixel value, and the time is continuously set.

[0031] Furthermore, the step of decoding the image captured in the geometric mode to obtain the depth information of the object specifically involves:

[0032] For a single pixel in a real-world image, its coordinates in the camera coordinate system are obtained through the camera intrinsic parameter matrix, thus obtaining the equation of the camera ray to which it belongs. The optimizer calculates the encoding of candidate 3D points belonging to the same camera ray in the simulated light field. The encoding obtained from the post-processing of the real-world image is compared with the encoding of the candidate 3D points, and the depth information of the most matching 3D point is selected as the decoding result.

[0033] Furthermore, the neural network built based on the simulated light field for LED illumination parameters specifically includes:

[0034] Light up all LEDs, set the LCD mask to transparent, and photograph the object after the light field is projected. Construct a lumitexel training set by randomly selecting parameter information from sampling points. The LED illumination parameters are set as parameters for the linear encoder part of the neural network. Input the constructed lumitexel training set into the neural network, calculate the Euclidean distance between the output lumitexel and the input lumitexel, and use the result as the loss function to complete the optimization of the LED illumination parameters.

[0035] Furthermore, the image and depth information captured based on the material mode are subjected to differentiable optimization to obtain the optimized BRDF parameters, specifically as follows:

[0036] Using depth information and BRDF material information obtained through regression neural network decoding as input, the Euclidean distance is calculated pixel by pixel from the rendering results obtained by simulating light field and the actual captured image. The Euclidean distance is used as the loss function, and the Adam optimizer is used to obtain the optimized BRDF parameters.

[0037] The beneficial effects of this invention are as follows:

[0038] (1) A method for jointly acquiring geometric and material information of static objects is proposed.

[0039] This invention simulates the light field by introducing the rendering equation of an LCD mask, encodes geometric information using structured light, and decodes it using a nearest neighbor search algorithm to obtain the object's geometric information. It also encodes the object's material information using optical path multiplexing and performs differentiable optimization based on the image captured in material mode and the acquired geometric information to obtain the object's material information. This invention combines two active light-based acquisition methods and achieves an efficient encoding and decoding scheme, thus solving the problem of simultaneously acquiring the geometric and material information of static objects from a single viewpoint.

[0040] (2) Its acquisition accuracy surpasses the most advanced mainstream methods currently available.

[0041] The method of this invention achieves an average geometric accuracy of 0.27 mm and an average material accuracy of 0.94 (measured by SSIM) on the test object, surpassing the most advanced mainstream methods available today.

[0042] (3) It has excellent generalization ability

[0043] This invention's method is independent of the object itself and is applicable to static objects of various materials, sizes, and shapes. In testing experiments, the accuracy and completeness of this invention's method on all test objects surpassed that of current state-of-the-art mainstream methods.

[0044] (4) Designed and implemented a simple and lightweight hardware prototype.

[0045] The hardware prototype provided in this embodiment of the invention consists of 64x48 LEDs, a 1920x1080 resolution LCD mask, and a Canon EOS R5 camera. Compared to mainstream multi-projector and multi-camera combination devices, it is more lightweight, simple, and easy to operate. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the process for high-precision acquisition of geometry and material provided in an exemplary embodiment of the present invention;

[0047] Figure 2 This is a hardware prototype design diagram provided in an exemplary embodiment of the present invention;

[0048] Figure 3 This is a neural network structure diagram for LED illumination parameters provided by an exemplary embodiment of the present invention. Detailed Implementation

[0049] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0050] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0051] This invention describes a structured light method for high-precision acquisition of geometry and material properties from a single viewpoint, such as... Figure 1 As shown, it includes the following steps:

[0052] Step 1: Design and implement a lightweight hardware prototype consisting of an LED module, an LCD mask, and a camera.

[0053] Hardware prototype design drawings as follows Figure 2 As shown, specifically, the object is placed inside the bounding box, the LED module and the LCD mask are placed vertically and parallel to each other; the camera is placed above the LCD mask, with the lens facing the center of the bounding box; the distance from the LED module to the LCD mask is equal to the distance from the LCD mask to the center of the bounding box.

[0054] In one embodiment, the LED module comprises 64x48 LEDs, with a spacing of 1 cm between two adjacent LEDs. The LCD mask is 59.8 cm long and 33.6 cm wide, with a resolution of 1920x1080. The camera is a Canon EOS R5 with a focal length of 24mm and an aperture of f / 22. The bezel is a 15 cm x 15 cm x 15 cm cube, with its center 15 cm from the center of the LCD mask and the center of the LCD mask 15 cm from the center of the LED module.

[0055] Step 2: Extract the parameters of the hardware prototype and simulate the light field by introducing the rendering equation of the LCD mask.

[0056] The parameters of the hardware prototype include the position parameters of the LED module, the light intensity distribution parameters, the position parameters of the LCD mask, the position parameters of the camera, the internal parameters of the camera, the camera's response curve, and the color calibration parameters.

[0057] A three-dimensional coordinate system for simulating the light field is constructed as follows: the x-axis is the ray that starts from the center of the LED module and passes through the center of the LCD mask; the y-axis is the ray that starts from the center of the LED module and passes through the left LED in the same row; the origin of the coordinate system is a three-dimensional point 2d away from the center of the LED module along the horizontal axis, where d is the distance between the center of the LED module and the center of the LCD mask; the z-axis is determined based on the x-axis and y-axis.

[0058] The location parameters and light intensity distribution parameters were calibrated using an image-based differentiable optimization method, specifically as follows:

[0059] The known rendering equation is: Where, x k Let x be a 3D point within the bounding box. l Let A be a point on the current LED light. Each LED light is modeled as a 2mm x 2mm surface light source, and its area is denoted as A. For x k After the j-th LCD mask pattern M j The brightness of the projected image on the final image, ω is the value from x. k To x l The unit direction vector, L(x) l ,-ω) is from x l The brightness of the emitted light in the direction of -ω For x l x k The pixel value at the intersection of the line and the LCD mask, where ρ is the material reflectivity. n k and n l x k and x l The normal vector at that location.

[0060] Therefore, by setting initial values ​​for the position parameters and light intensity distribution parameters, a simulated light field can be constructed based on the initial values ​​and the rendering equation. The Euclidean distance between the captured image and the simulated image output through this simulated light field is calculated pixel-by-pixel and input into the Adam optimizer for optimization. The final converged value is used as the result for the position parameters and light intensity distribution parameters.

[0061] The camera's internal parameters and response curves are calibrated using the camera calibration module in OpenCV. Color calibration parameters are obtained by taking images of the X-Rite colorchecker test chart under different lighting conditions.

[0062] Step 3: Based on the simulated light field described above, build an optimizer for the pixel values ​​of the LCD mask.

[0063] To acquire geometric information, a method similar to structured light is used, employing a single LED as the light source and an LCD mask as the projection pattern, projected onto the object surface. A pixel in the LCD mask can only be set to 0 or 255, where 0 indicates no light transmission and 255 indicates complete light transmission. By setting different pixel values, different LCD mask patterns can be constructed. The optimizer in this step is built to design the optimal projection pattern. A batch of 3D points existing within the bounding box and belonging to the same camera ray is input. Each 3D point is placed in the light field after single-lamp projection, and its brightness in the final image can be obtained using the rendering equation mentioned in step 2. In this embodiment, a single LED is set to project 18 projection patterns; therefore, each 3D point is encoded into an 18-bit brightness vector. This regularized brightness vector is called the encoding. By selecting one 3D point from this batch as a label, the cross-entropy of this batch of 3D points can be calculated. Using this cross-entropy as the loss function, an optimizer for the LCD mask pixel values ​​can be built.

[0064] Step 4: Based on the optimizer described above, obtain the optimal LCD mask pixel values ​​and modify the LCD mask pixel values ​​on the hardware prototype. At the same time, select a single LED in the LED module and set it to the brightest state.

[0065] Step 5: Using the modified hardware prototype described above, complete the capture of the geometric pattern.

[0066] In this embodiment, four LEDs are selected to build an optimizer for the LCD mask pixel values. One set of LCD mask pixel values ​​is obtained for each LED, and each set corresponds to 18 optimal projection patterns, which are used for 18 shots. In the geometric mode shooting process, the LCD mask is modified by the corresponding mask pixel values ​​each time, and the duration is 20 seconds. The 18-bit brightness vector obtained from the 18 shots is regularized and used as the real-shot code at each pixel.

[0067] Step 6: Decode the image captured in the above geometric mode to obtain the depth information of the object.

[0068] For a single pixel in a real-world image, its coordinates in the camera coordinate system can be obtained through the camera intrinsic parameter matrix, thus yielding the equation of the camera ray to which it belongs. The optimizer in step 3 calculates the encoding of candidate 3D points belonging to the same camera ray in the simulated light field. The encoding obtained from post-processing the real-world image is compared with the encoding of the candidate 3D points, and the depth information of the best-matching 3D point (i.e., the one with the closest Euclidean distance) is selected as the decoded result.

[0069] Step 7: Based on the simulated light field described in Step 2, build a neural network for LED illumination parameters.

[0070] To acquire material information, a method similar to optical path multiplexing is needed, where all LEDs are simultaneously illuminated, the LCD mask is set to transparent, and the object after the light field is projected is photographed. The neural network in this step is built to design the optimal LED lighting parameters.

[0071] Given the observed value V of a sampling point p on the surface of an object on the image, and the reflection function f r The relationship between the light intensity of each light source and the light intensity of each light source can be described as follows:

[0072]

[0073] Where I represents the emission information of each light source l, including: the spatial position x of the light source l. l The normal vector n of the light source l l The luminous intensity I(l) of light source l, P includes parameter information of sampling point p, including: the spatial position x of the sampling point. p BRDF (Bidirectional Reflectance Distribution Function) material parameters {n p ,t,α x α y , ρ d , ρ s}, n p Let t represent the normal vector of the sampling point in the world coordinate system, and t represent the x-axis direction of the sampling point in the local coordinate system. p The parameter 't' is used to transform the incident and exit directions from the world coordinate system to the local coordinate system, and 'α' is used to transform the incident and exit directions from the world coordinate system to the local coordinate system. x α y ρ represents the roughness coefficient. d ρ represents diffuse reflectance. s Ψ(x) represents the specular reflectance. l The expression ,·) describes the intensity distribution of light source l under different incident directions, and V represents x. l For x p A binary function for visibility, (·) + For the dot product of two vectors, negative values ​​are truncated to 0. r (ω′ i ;ω′ o , P) is ω′ o When fixed, regarding ω′ i The two-dimensional reflection function, Where, ω′ i ω′ represents the direction of incident light in the world coordinate system. o ω represents the direction of the emitted light in the world coordinate system. i Let ω be the incident direction in the local coordinate system. oLet ω be the exit direction in the local coordinate system. h D is the half-path vector in the local coordinate system. GGX For microsurface distribution, F is the Fresnel term, and G is the microsurface distribution term. GGx This represents the shading coefficient function.

[0074] Lumitexel is defined as: Furthermore, it can be represented as the sum of diffuse lumitexel and specular lumitexel. A lumitexel training set is constructed by randomly selecting the parameter information of sampling point p. In this embodiment, the constructed neural network is as follows: Figure 3 As shown, each rectangle represents a layer of neurons, and the number in the rectangle indicates the number of neurons in that layer. The LED illumination parameters are set as parameters for the linear encoder part of the neural network. The constructed lumitexel training set is input into the neural network, the Euclidean distance between the output lumitexel and the input lumitexel is calculated, and the result is used as the loss function to complete the optimization of the LED illumination parameters.

[0075] Step 8: Based on the LED illumination parameters obtained from the neural network training, modify the brightness of the LED module on the hardware prototype. At the same time, set the LCD mask in the hardware prototype to a transparent state.

[0076] Step 9: Using the modified hardware prototype described above, complete the shooting in material mode.

[0077] There are 32 sets of LED illumination parameters obtained through neural network training, with each set corresponding to one shot. Each shot modifies the LED module using the corresponding illumination parameters for a duration of 0.2 seconds.

[0078] Step 10: Based on the image captured in material mode and the depth information decoded in step 6, perform differentiable optimization to obtain the optimized BRDF parameters.

[0079] A regression neural network with 16-dimensional neural parameters as input and BRDF parameters as output is constructed, using an MLP (Multilayer Perceptron) structure. Specifically, the 16-dimensional neural parameters are both inputs and parameters that can be optimized, and are updated during the optimization process after initialization. Using depth information and BRDF material information decoded by the regression neural network as input, the rendering results obtained through simulated light fields and the actual captured images are used to calculate the Euclidean distance pixel-by-pixel. The Euclidean distance is used as the loss function, and the Adam optimizer is employed to obtain the optimized BRDF parameters.

[0080] Step 11: Using the optimized BRDF parameters and depth information described above, the final rendering result is achieved.

[0081] The above description is merely a preferred embodiment of the present invention. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make many possible variations and modifications to the technical solutions of the present invention using the methods and techniques disclosed above, or modify them into equivalent embodiments with equivalent changes, without departing from the scope of the technical solutions of the present invention. Therefore, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall still fall within the protection scope of the technical solutions of the present invention.

Claims

1. A structured light method for high-precision acquisition of geometry and material properties from a single viewpoint, characterized in that, Includes the following steps: A lightweight hardware prototype consisting of LED modules, LCD masks, and cameras; The parameters of the hardware prototype are extracted, and light field simulation is performed by introducing the rendering equation of the LCD mask; An optimizer for LCD mask pixel values ​​was built based on simulated light fields. The optimal LCD mask pixel values ​​are obtained based on the optimizer, and the pixel values ​​of the LCD mask are modified on the hardware prototype. At the same time, a single LED in the LED module is selected and set to the brightest state. The geometric pattern was captured using a modified hardware prototype; Decode the image captured in geometric mode to obtain the depth information of the object; A neural network targeting LED illumination parameters was built based on simulated light fields; Based on the LED illumination parameters obtained from the neural network training, the brightness of the LED module is modified on the hardware prototype, and the LCD mask in the hardware prototype is set to a transparent state. The material mode was captured using the modified hardware prototype; Based on the image and depth information captured in the material mode, perform differentiable optimization to obtain the optimized BRDF parameters; The final rendering result is achieved by using optimized BRDF parameters and depth information.

2. The structured light method for high-precision acquisition of geometry and material properties under a single viewpoint, as described in claim 1, is characterized in that... The hardware prototype is designed as follows: an object is placed inside a bounding box, the LED module and the LCD mask are placed vertically and parallel to each other; a camera is placed above the LCD mask with the lens facing the center of the bounding box; the distance from the LED module to the LCD mask is equal to the distance from the LCD mask to the center of the bounding box.

3. The structured light method for high-precision acquisition of geometry and material properties under a single viewpoint, as described in claim 1, is characterized in that... The parameters of the hardware prototype include the position parameters of the LED module, the light intensity distribution parameters, the position parameters of the LCD mask, the position parameters of the camera, the internal parameters of the camera, the camera's response curve, and the color calibration parameters.

4. The structured light method for high-precision acquisition of geometry and material properties under a single viewpoint, as described in claim 1, is characterized in that... The simulated light field has a horizontal axis x as the ray that originates from the center of the LED module and passes through the center of the LCD mask. The y-axis is defined by the ray that originates from the center of the LED module and passes through the left LED in the same row; the origin of the coordinate system is defined by a three-dimensional point 2d away from the center of the LED module along the horizontal axis, where d is the distance between the center of the LED module and the center of the LCD mask; the z-axis is determined based on the x-axis and y-axis.

5. The structured light method for high-precision acquisition of geometry and material properties under a single viewpoint, as described in claim 3, is characterized in that... The calibration of the position parameters and light intensity distribution parameters is performed using an image-based differentiable optimization method, specifically as follows: Known rendering equation Where, x k Let x be a 3D point within the bounding box. l Let A be a point on the current LED light. Each LED light is modeled as a surface light source, and its area is denoted as A. For x k After the j-th LCD mask pattern M j The brightness of the projected image on the final image, ω is the value from x. k To x l The unit direction vector, L(x) l ,ω) is from x l The brightness of the emitted light in the direction of -ω For x l , k The pixel value at the intersection of the line and the LCD mask, where ρ is the material reflectivity. n k and n l x k and x l The normal vector at that location; By setting initial values ​​for position parameters and light intensity distribution parameters, a simulated light field based on the initial values ​​and rendering equations can be constructed. The Euclidean distance between the real-shot image and the simulated image output through the simulated light field is calculated pixel by pixel and input into the Adam optimizer for optimization. The final converged value is used as the result of the position parameters and light intensity distribution parameters.

6. The structured light method for high-precision acquisition of geometry and material properties from a single viewpoint, as described in claim 1, is characterized in that... The optimizer for LCD mask pixel values ​​built based on simulated light fields is as follows: By performing the process in batches, each batch takes 3D points that exist in the bounding box and belong to the same camera ray as input, obtains the results of the batch of 3D points in the simulated light field, selects one of the 3D points in the batch as a label, and uses the cross-entropy of the batch of 3D points as a loss function to optimize the pixel values ​​of the LCD mask.

7. The structured light method for high-precision acquisition of geometry and material properties under a single viewpoint, as described in claim 1, is characterized in that... In the shooting process of the geometric mode, the LCD mask is modified each time by the corresponding mask pixel value, and the time is continuously set.

8. The structured light method for high-precision acquisition of geometry and material properties under a single viewpoint, as described in claim 1, is characterized in that... Decoding the image captured in geometric mode to obtain the object's depth information specifically involves: For a single pixel in a real-world image, its coordinates in the camera coordinate system are obtained through the camera intrinsic parameter matrix, thus obtaining the equation of the camera ray to which it belongs. The optimizer calculates the encoding of candidate 3D points belonging to the same camera ray in the simulated light field. The encoding obtained from the post-processing of the real-world image is compared with the encoding of the candidate 3D points, and the depth information of the most matching 3D point is selected as the decoding result.

9. The structured light method for high-precision acquisition of geometry and material properties under a single viewpoint, as described in claim 1, is characterized in that... The neural network for LED illumination parameters built based on simulated light fields is specifically as follows: Light up all LEDs, set the LCD mask to transparent, and photograph the object after the light field is projected. Construct a lumitexel training set by randomly selecting parameter information from sampling points. The LED illumination parameters are set as parameters for the linear encoder part of the neural network. Input the constructed lumitexel training set into the neural network, calculate the Euclidean distance between the output lumitexel and the input lumitexel, and use the result as the loss function to complete the optimization of the LED illumination parameters.

10. The structured light method for high-precision acquisition of geometry and material properties under a single viewpoint, as described in claim 1, is characterized in that... The images and depth information captured based on the material mode are subjected to differentiable optimization to obtain the optimized BRDF parameters, specifically: Using depth information and BRDF material information obtained through regression neural network decoding as input, the Euclidean distance is calculated pixel by pixel from the rendering results obtained by simulating light field and the actual captured image. The Euclidean distance is used as the loss function, and the Adam optimizer is used to obtain the optimized BRDF parameters.

Images