A method for obtaining material properties of an object surface based on a three-dimensional scanner

Abstract

The application discloses a kind of based on three-dimensional scanner's object surface material attribute acquisition method, three-dimensional scanner includes the structure light projector of two color industrial cameras and a white light source;Method includes the following steps: object is placed on turntable, mechanical arm movement drives three-dimensional scanner to carry out structure light scanning, and the deformed stripe image modulated by object is photographed by color industrial camera, and the stripe image photographed is transmitted to processor;Processor processes stripe image, and obtains the point cloud data of the three-dimensional coordinates of object surface;Data fusion is carried out again, and the high dynamic range image of each view angle is obtained;The point cloud data obtained from different view angles is processed, and high-quality three-dimensional model is generated;Texture coordinates of three-dimensional model are obtained;After obtaining the texture map of segmentation;Object surface material attribute is obtained.The application is faster to collect object surface material attribute, and the material texture details of reconstruction are sufficient, real, and the accuracy of material attribute acquisition is high.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a method for obtaining the surface material properties of an object based on a 3D scanner. Background Technology

[0002] In computer graphics and computer vision, the material properties of object surfaces are typically represented using Spatially-Varying Bidirectional Reflectance Distribution Functions (SVBRDFs). Direct acquisition methods for SVBRDFs utilize controlled multi-angle lighting and imaging devices to sample the BRDF in the angular domain; however, such acquisition devices are very expensive and time-consuming. Currently, approximating high-dimensional BRDFs using parametric BRDF models and fitting them with limited measurement data is a common approach. The Disney prototype model, defined by base color, metallicity, and roughness, is the most commonly used parametric BRDF model. Deep learning can estimate geometry and material from single-view images. However, these supervised learning methods require large amounts of data for training and have poor generalization ability, failing to obtain high-quality material reconstruction results.

[0003] In recent years, the emergence of Differentiable Rendering has made self-supervised appearance acquisition and modeling possible. It makes each step of the rendering process differentiable, allowing the derivative of the rendered output with respect to input parameters (such as material properties, lighting conditions, geometry, etc.) to be calculated. This characteristic makes Differentiable Rendering important in tasks such as reverse engineering, 3D reconstruction, and material reconstruction. Existing Differentiable Rendering methods include rasterization-based methods and physically based methods. Rasterization is a traditional graphics rendering technique whose basic principle is to convert 3D geometry into a 2D image through rasterization operations. This process includes the following steps: Vertex processing: Transforming and calculating the lighting of the vertices of the 3D model, projecting them into a 2D screen coordinate system. Primitive assembly: Assembling the processed vertices into primitives (such as triangles). Rasterization: Projecting the primitives onto a pixel grid on the screen, determining which pixels are covered by the primitives. Fragment processing: Interpolating the covered pixels to determine their color, depth, and other attributes, and then shading them, applying texture mapping, lighting, and other effects. However, rasterization-based rendering may fall short when handling complex geometries, material models, and light source distributions because it ignores light transmission phenomena between surfaces and struggles to render complex material models and extended light sources. Ray tracing is a rendering technique in computer graphics used to generate high-quality images. Path tracing is a ray tracing-based rendering algorithm that simulates various light transmission phenomena by tracing the propagation path of light rays and calculating the interaction between light rays and object surfaces. Because path tracing can simulate complex optical phenomena such as reflection, refraction, shadows, scattering, and mutual reflection, it supports full global illumination and samples complex material models and extended light sources. Therefore, path tracing-based differentiable rendering is called physically based differentiable rendering.

[0004] However, existing technologies require a large amount of real data collection, and there are also problems such as slow collection of object surface material properties and insufficient or unrealistic material texture details in the reconstructed material texture. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method for obtaining the material properties of an object surface based on a 3D scanner, which is faster in acquiring the material properties of the object surface, and the reconstructed material texture details are sufficient and realistic, and the accuracy of material property acquisition is high.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a method for obtaining the surface material properties of an object based on a 3D scanner, wherein the 3D scanner includes two color industrial cameras mounted on a robotic arm and a structured light projector with a white light source; the method for obtaining the surface material properties of an object includes the following steps:

[0007] Step S1: Place the object on the turntable. The robotic arm moves to drive the 3D scanner to perform structured light scanning. The 3D scanner projects a preset stripe pattern and captures the deformed stripe image modulated by the object through a color industrial camera. The captured stripe image is then transmitted to the processor.

[0008] Step S2: The processor processes the stripe image to obtain point cloud data of the three-dimensional coordinates of the object surface;

[0009] Step S3: Change the light intensity at each viewpoint and take multiple exposures. Repeat the processing steps of step S2, and then perform data fusion to obtain high dynamic range images for each viewpoint.

[0010] Step S4: Register, denoise, simplify, and smooth the point cloud data obtained from different perspectives, and generate triangular meshes. Then, fill holes and smooth the triangular meshes to generate a high-quality 3D model.

[0011] Step S5: Unfold and optimize the texture coordinates of the 3D model to obtain the texture coordinates of the 3D model;

[0012] Step S6: Remove uneven lighting, highlights and shadows from the images obtained from different viewpoints, and segment the images to obtain segmented texture maps;

[0013] Step S7: Optimize the object's material properties using physically based differentiable rendering until the loss function converges or the predetermined number of iterations is reached, and finally obtain the object's surface material properties.

[0014] The above-mentioned method for obtaining the surface material properties of an object based on a 3D scanner includes a step of calibrating the 3D scanner before step S1. The specific process is as follows:

[0015] Step S01: Ensure that the industrial camera and structured light projector remain relatively stationary, and prepare the calibration board. Allow the structured light projector to project uniform light, and use the industrial camera to take images of the calibration board at different angles to calibrate the internal and external parameters of the industrial camera.

[0016] Step S02: The structured light projector projects a phase-shifted fringe pattern. An industrial camera is used to capture images of the calibration board at different angles. By processing the captured images of the calibration board, the coordinates of the image point of each center point on the calibration board on the structured light projector are calculated. The intrinsic parameters of the structured light projector and the relative pose between the industrial camera and the structured light projector are calibrated using the same method as the camera calibration.

[0017] Step S03: The structured light projector undergoes current-brightness curve calibration to facilitate subsequent high dynamic range image synthesis. The current-brightness curve calibration process includes projecting uniform light from the structured light projector under different currents and capturing images using an industrial camera. The relationship between current and brightness is obtained by statistically analyzing the average grayscale of small areas within the images. Color calibration is performed on the color industrial camera by capturing images of a standard color chart under uniform illumination from the structured light projector, calculating the color mapping relationship, and establishing a conversion matrix from standard color to camera-acquired color to standard color to ensure accurate color reproduction. The specific formula is as follows:

[0018]

[0019] Among them, a ij represents the linear transformation coefficient, and represents the contribution weight of the camera's j-th channel to the standard color's i-th channel; b i This represents the offset (Bias), used to compensate for overall brightness deviations in the color channels.

[0020] The above-mentioned method for obtaining the surface material properties of an object based on a 3D scanner, specifically the process in step S2 where the processor processes the stripe image to obtain the point cloud data of the object's 3D coordinates, is as follows:

[0021] Step 201: Perform N-step phase shifts, and the light intensity I of the i-th fringe image at coordinates (x, y) is... i (x,y) can be represented as:

[0022]

[0023] Where A(x,y) is the average light intensity, B(x,y) is the fringe modulation degree, and φ(x,y) is the principal phase value to be solved; i takes the value of a natural number from 0 to N;

[0024] Step 202: The general formula for obtaining the principal phase value for N-step phase shift is expressed as:

[0025]

[0026] The principal phase value obtained by the phase shift method is the phase enclosed in the interval [-π, π].

[0027] Step 203: Let the principal phase values ​​of the two-order fringes with fringe period λ1 and spatial frequency f1, and fringe period λ2 and spatial frequency f2 be φ1(x,y) and φ2(x,y) respectively. According to the heterodyne principle, a new set of phases is obtained. The fringe period of the equivalent wavelength after heterodyne is:

[0028]

[0029] The corresponding spatial frequencies are f1-f2, and the principal phase value at point (x,y) is:

[0030] φ b (x,y)=φ1(x,y)-φ2(x,y)

[0031] Step 204: Perform phase expansion on φ1(x,y) and φ2(x,y) respectively to obtain the absolute phase value Φ1(x,y) of the first color industrial camera with fringe period λ1 and spatial frequency f1, and the absolute phase value Φ2(x,y) of the second color industrial camera with fringe period λ2 and spatial frequency f2.

[0032] Step 205: Based on Φ1(x,y) and Φ2(x,y), obtain the coordinates (u,v) and (u',v') of the matching point in the principal point coordinate system on the images of the first and second color industrial cameras.

[0033] The search range for stereo matching is along the horizontal epipolar line. The unfolded absolute phase is used as the feature of stereo matching. The process of searching for matching points in the right camera phase image is as follows: find the point with the closest phase and perform interpolation. First, based on the phase of the point to be matched in the left camera phase image, search for the point with the closest phase value to the point in the left image on the straight line with the same vertical coordinate as the point to be matched in the left camera phase image. This point is used as the initial matching point. Then, based on the phases of the left and right neighboring points, perform linear interpolation to obtain the right camera matching point with the same phase as the point to be matched in the left camera.

[0034] Step 206: The equation for the 3D coordinates of the object surface reconstructed from the known matching point coordinates (u,v) and (u',v') is expressed as:

[0035]

[0036] Eliminating x yields:

[0037] z(u-u')=2hf

[0038] The three-dimensional coordinates x, y, z of the object's surface are represented as:

[0039]

[0040] Where f is the effective focal length after epipolar correction, d is the parallax, b is the baseline length, and h is half of the baseline length.

[0041] The above-mentioned method for obtaining the surface material properties of an object based on a 3D scanner, specifically the step 204 of performing phase expansion on φ1(x,y) to obtain the absolute phase value Φ1(x,y) of the first color industrial camera with a fringe period of λ1 and a spatial frequency of f1, is as follows:

[0042] Step 204A1: When the zero-phase points are the same, the phase value is inversely proportional to the period of the sinusoidal fringe, which can be expressed by the formula:

[0043]

[0044] Where Φ1(x,y) represents the absolute phase value of the first color industrial camera with fringe period λ1 and spatial frequency f1, Φ b (x,y) represents a fringe period of λ. b The absolute phase value with spatial frequency f1-f2;

[0045] Step 204A2, take Φ b (x,y)=φ b (x,y);

[0046] Step 204A3: Substitute the formula from step 204A2 into the formula from step 204A1 to obtain:

[0047]

[0048] The above-mentioned method for obtaining the surface material properties of an object based on a 3D scanner, specifically the step 204 of performing phase expansion on φ2(x,y) to obtain the absolute phase value Φ2(x,y) of the first color industrial camera with a fringe period of λ2 and a spatial frequency of f2, is as follows:

[0049] Step 204B1: When the zero-phase points are the same, the phase value is inversely proportional to the period of the sine stripes, which can be expressed by the formula:

[0050]

[0051] Where Φ2(x,y) represents the absolute phase value of the second color industrial camera with a fringe period of λ2 and a spatial frequency of f2, Φ b (x,y) represents a fringe period of λ. b The absolute phase value with spatial frequency f1-f2;

[0052] Step 204B2, take Φ b (x,y)=φ b (x,y);

[0053] Step 204B3: Substitute the formula from step 204B2 into the formula from step 204B1 to obtain:

[0054]

[0055] The above-mentioned method for obtaining the surface material properties of an object based on a 3D scanner, specifically the process of data fusion in step S3 to obtain high dynamic range images for each viewpoint, is as follows:

[0056] Step S301: Establish the mapping relationship between pixel value Z∈[0,255] and linear radiance:

[0057]

[0058] Step S302: Real-time conversion is achieved by pre-computing a 256th order lookup table to convert discrete pixel observations into linear data that is proportional to physical radiance.

[0059] Step S303: Using the Robertson fusion algorithm, solve for the scene radiosity E using the weighted least squares method:

[0060]

[0061] Among them, Z ij Let Δt be the observation value of pixel i in the j-th exposure. j For the j-th exposure time, f -1 (Z ij w(Z) is the inverse response function, which converts pixel values ​​into linear radiosity; ij E is a weighting function used to suppress the influence of noise regions. i denoted as radiance of pixel i; M is the total number of pixels; P is the total number of exposures.

[0062] Step S304: Map E to the standard dynamic range using linear normalization:

[0063]

[0064] Among them, E min E is the minimum radiometric estimate for the scene. max E represents the estimated maximum radioactivity of the scene. norm This represents the scene radiance value after linear normalization.

[0065] The above-mentioned method for obtaining the surface material properties of an object based on a 3D scanner, specifically step S4, involves registering, denoising, simplifying, and smoothing point cloud data obtained from different viewpoints, generating triangular meshes, filling holes in the triangular meshes, smoothing them, and generating a high-quality 3D model.

[0066] Step S401: For each point p i Find its nearest point q in the point cloud from another perspective. j :

[0067]

[0068] Where q represents a point in the 3D model, and Q represents all points in the 3D model point cloud data;

[0069] Step S402: Given N pairs of nearest point pairs (p i ,q j ), calculate the rotation matrix R and translation vector t; the specific process is as follows:

[0070] Step S4021: Calculate the centroid of the point cloud. and

[0071]

[0072] Step S4022: Calculate the centroid-free point:

[0073]

[0074] Step S4023: Construct the covariance matrix H:

[0075]

[0076] Step S4024: Solve for the rotation matrix R by performing singular value decomposition on the covariance matrix H:

[0077] H=UΣV T

[0078] R = VU T

[0079] Step S4025: Calculate the translation vector t:

[0080]

[0081] Step S403: Perform noise reduction, simplification, and smoothing operations on the point cloud data;

[0082] Step S404: Generate a triangular mesh model using Delaunay triangulation;

[0083] Step S405: Fill holes and correct self-intersection problems on the mesh, and smooth the mesh to generate a high-quality 3D model.

[0084] The above-mentioned method for obtaining the surface material properties of an object based on a 3D scanner, specifically the process of unfolding and optimizing the texture coordinates of the 3D model in step S5 to obtain the texture coordinates of the 3D model, is as follows:

[0085] Step S501: Automatically select the edges on the model surface that need to be cut and mark them as cutting seams;

[0086] Step S502: Unfold the model spherically from the cut seam. Calculate and optimize the texture coordinates of each vertex based on the unfolding result. Use the Brown-Conrady distortion model to correct the coordinates (x,y) on the normalized image to (x′,y′). The calculation formula is as follows:

[0087]

[0088] Where, r 2 =x 2 +y 2 k1 and k2 are radial distortion coefficients, and p1 and p2 are tangential distortion coefficients.

[0089] The above-mentioned method for obtaining the surface material properties of an object based on a 3D scanner, specifically step S6, involves processing images obtained from different viewpoints to remove uneven lighting, highlights, and shadows, and then segmenting the images to obtain the segmented texture map.

[0090] Step S601: Divide the image of each viewpoint by the irradiance of the light source to remove the influence of uneven lighting;

[0091] Step S602: Backproject each pixel on the texture map onto the original image, and take the color corresponding to the median brightness of all images as the color of that pixel. Combine this with the texture coordinates to obtain a diffuse texture map with highlights and shadows removed. The specific process is as follows:

[0092] Step S6021: Calculate the projection matrix based on the camera pose and intrinsic and extrinsic parameters of each viewpoint. The calculation formula is as follows:

[0093]

[0094] Among them, P GL Denotes the projection matrix, (f x f y (c) is the focal length, and the principal point offset is (c) x c y ), where w is the width, h is the height, and f is the height. n f is the depth value of the near-plane. f This represents the depth value of the far plane.

[0095] Step S6022: Render the depth map D corresponding to each viewpoint using the projection matrix. k Used for subsequent visibility checks;

[0096] Step S6023: For each pixel p in the texture map uv To verify the consistency between its projected depth and mesh depth at viewpoint k:

[0097] |D mesh (p uv )-D k (π k (p uv ))|<∈ depth (∈=1.0)

[0098] in, This indicates the grid depth corresponding to a pixel. This represents the projection depth corresponding to a pixel. This represents the projection matrix above; The threshold representing the depth difference; Step S6024: Find all images containing the pixel, sample the color of the pixel in these images and perform gamma correction. Then, median filtering is applied to all viewpoint color values ​​to remove highlight and low-light pixels, resulting in the final de-highlighted median texture map. ; in, Represents the sRGB value of a pixel; This represents the final color value of a pixel; Step S603: Use the Simple Linear Iterative Clustering (SLIC) algorithm on the texture map to segment it into different regions based on color, assuming that the metallicity and roughness are the same in each region; the specific process is as follows: Step S60301: Divide the image into different superpixel regions, with each region as a node, and adjacent nodes are connected by edges; Step S60302: Evaluate whether two adjacent superpixels can be merged by calculating two key indicators: edge weight and boundary strength score; specifically: When constructing the graph model, edge weights are calculated based on the color difference between adjacent superpixels, given two superpixels. and Their color differences are measured by L2 distance:

[0099] Among them, c i For SuperPixel S i The average chromaticity value, c j For SuperPixel S jThe average chromaticity value, w ij Indicates the boundary weight between two regions;

[0100] Boundary strength score B ij Defined as:

[0101]

[0102] Where I(x) represents the intensity value of the image at pixel x, B ij Used to measure the range of intensity variation between two regions; the larger the value, the more distinct the boundary.

[0103] Sort all edge weights from smallest to largest, prioritizing merging regions with small color differences and weak boundary variations; for each edge, the merging condition is:

[0104] w ij <δ c and B ij <δ b

[0105] Where, δ c δ is the color difference threshold. b The boundary strength threshold;

[0106] When the edge weight is less than the preset color difference threshold δ c And the boundary strength score is less than the preset threshold δ b At that time, the two superpixels are merged into a new material cluster;

[0107] After merging, the graph model is updated, and the edge weights and boundary scores between the new nodes and adjacent regions are recalculated; the clustering process is repeated until no more adjacent regions meet the merging conditions, or the preset minimum number of clusters is reached; finally, the segmented texture map is obtained.

[0108] In the above-described method for obtaining the surface material properties of an object based on a 3D scanner, the loss function in step S7 is:

[0109]

[0110] Where, λ L1 for The weight, λ r for The weight, λ m for The weights, S is the number of pixels in the segmented region, I i I is the rendered pixel value of pixel i. target Let r be the target pixel value of pixel i. i and r j Let m be the roughness parameter of adjacent pixels i and j.i and m j These are the metallicity parameters of adjacent pixels i and j. It is the L1 distance. For roughness regularization, This is the metallicity regularization term.

[0111] Compared with existing technologies, this invention has the following advantages: It utilizes a structured light 3D scanner to reconstruct the geometric model of an object, and uses a structured light projector as a known light source. It optimizes the surface reflection properties of the object using physically based differentiable rendering, synthesizes and segments a median texture from multiple viewpoints, assuming that the metallicity and roughness are the same in each region. By introducing a known geometric model and light source, and segmenting the texture, it can significantly reduce ambiguity in material optimization and improve the accuracy of material property acquisition. This invention solves the problems of existing technologies, such as the need for extensive real-world data collection, slow acquisition of object surface material properties, and insufficient detail and realism in the reconstructed material textures.

[0112] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0113] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0114] like Figure 1 As shown, the present invention discloses a method for obtaining the surface material properties of an object based on a 3D scanner. The 3D scanner includes two color industrial cameras mounted on a robotic arm and a structured light projector with a white light source. The method for obtaining the surface material properties of an object includes the following steps:

[0115] Step S0, the calibration of the 3D scanner, is as follows:

[0116] Step S01: Ensure that the industrial camera and structured light projector remain relatively stationary, and prepare the calibration board. Allow the structured light projector to project uniform light, and use the industrial camera to take images of the calibration board at different angles to calibrate the internal and external parameters of the industrial camera.

[0117] Step S02: The structured light projector projects a phase-shifting fringe pattern. An industrial camera is used to capture images of the calibration board at different angles. By processing the captured images of the calibration board, the image point coordinates of each center point on the calibration board on the structured light projector are calculated. The intrinsic parameters of the structured light projector and the relative pose between the industrial camera and the structured light projector are calibrated using the same method as the camera calibration. The coordinates of the structured light projector in the camera coordinate system can be used as the position of the light source in the subsequent differentiable rendering process.

[0118] Step S03: The structured light projector undergoes current-brightness curve calibration to facilitate subsequent high dynamic range image synthesis. The current-brightness curve calibration process includes projecting uniform light from the structured light projector under different currents and capturing images using an industrial camera. The relationship between current and brightness is obtained by statistically analyzing the average grayscale of small areas within the images. Color calibration is performed on the color industrial camera by capturing images of a standard color chart under uniform illumination from the structured light projector, calculating the color mapping relationship, and establishing a conversion matrix from standard color to camera-acquired color to standard color to ensure accurate color reproduction. The specific formula is as follows:

[0119]

[0120] Among them, a ij represents the linear transformation coefficient, and represents the contribution weight of the camera's j-th channel to the standard color's i-th channel; b i This represents the offset (Bias), used to compensate for overall brightness deviations in the color channels.

[0121] Step S1: Place the object on the turntable. The robotic arm moves to drive the 3D scanner to perform structured light scanning. The 3D scanner projects a preset stripe pattern and captures the deformed stripe image modulated by the object through a color industrial camera. The captured stripe image is then transmitted to the processor.

[0122] In practice, the movement path of the robotic arm is pre-set according to the shape of the object to ensure that the 3D scanner can scan the object comprehensively from different angles and positions;

[0123] Step S2: The processor performs phase deconvolution, phase unrolling and phase matching on the stripe image to obtain point cloud data of the three-dimensional coordinates of the object surface.

[0124] In this embodiment, the specific process by which the processor processes the stripe image in step S2 to obtain the point cloud data of the three-dimensional coordinates of the object surface is as follows:

[0125] Step 201: Perform N-step phase shifts, and the light intensity I of the i-th fringe image at coordinates (x, y) is... i (x,y) can be represented as:

[0126]

[0127] Where A(x,y) is the average light intensity, B(x,y) is the fringe modulation degree, and φ(x,y) is the principal phase value to be solved; i takes the value of a natural number from 0 to N;

[0128] Step 202: The general formula for obtaining the principal phase value for N-step phase shift is expressed as:

[0129]

[0130] The principal phase value obtained by the phase shift method is the phase enclosed in the interval [-π, π].

[0131] To remove phase ambiguity, the wrapped phase needs to be unfolded, and the correct absolute phase is calculated by projecting multi-level fringe with similar periods.

[0132] Step 203: Let the principal phase values ​​of the two-order fringes with fringe period λ1 and spatial frequency f1, and fringe period λ2 and spatial frequency f2 be φ1(x,y) and φ2(x,y) respectively. According to the heterodyne principle, a new set of phases is obtained. The fringe period of the equivalent wavelength after heterodyne is:

[0133]

[0134] The corresponding spatial frequencies are f1-f2, and the principal phase value at point (x,y) is:

[0135] φ b (x,y)=φ1(x,y)-φ2(x,y)

[0136] If the fringe period λ of the equivalent wavelength b If it can cover the entire field of view, that is, ensure that there is only one fringe period within the field of view, then a fringe covering the entire field can be obtained.

[0137] Step 204: Perform phase expansion on φ1(x,y) and φ2(x,y) respectively to obtain the absolute phase value Φ1(x,y) of the first color industrial camera with fringe period λ1 and spatial frequency f1, and the absolute phase value Φ2(x,y) of the second color industrial camera with fringe period λ2 and spatial frequency f2.

[0138] Step 205: Based on Φ1(x,y) and Φ2(x,y), obtain the coordinates (u,v) and (u',v') of the matching point in the principal point coordinate system on the images of the first and second color industrial cameras.

[0139] The search range for stereo matching is along the horizontal epipolar line. The unfolded absolute phase is used as the feature of stereo matching. The process of searching for matching points in the right camera phase image is as follows: find the point with the closest phase and perform interpolation. First, based on the phase of the point to be matched in the left camera phase image, search for the point with the closest phase value to the point in the left image on the straight line with the same vertical coordinate as the point to be matched in the left camera phase image. This point is used as the initial matching point. Then, based on the phases of the left and right neighboring points, perform linear interpolation to obtain the right camera matching point with the same phase as the point to be matched in the left camera.

[0140] Step 206: The equation for the 3D coordinates of the object surface reconstructed from the known matching point coordinates (u,v) and (u',v') is expressed as:

[0141]

[0142] Eliminating x yields:

[0143] z(u-u')=2hf

[0144] The three-dimensional coordinates x, y, z of the object's surface are represented as:

[0145]

[0146] Where f is the effective focal length after epipolar correction, d is the parallax, b is the baseline length, and h is half of the baseline length.

[0147] In this embodiment, the specific process of performing phase expansion on φ1(x,y) in step 204 to obtain the absolute phase value Φ1(x,y) of the first color industrial camera with fringe period λ1 and spatial frequency f1 is as follows:

[0148] Step 204A1: When the zero-phase points are the same, the phase value is inversely proportional to the period of the sinusoidal fringe, which can be expressed by the formula:

[0149]

[0150] Where Φ1(x,y) represents the absolute phase value of the first color industrial camera with fringe period λ1 and spatial frequency f1, Φ b (x,y) represents a fringe period of λ. b The absolute phase value with spatial frequency f1-f2;

[0151] Step 204A2, due to λ b It can cover the entire field of view, and its absolute phase value is equal to the principal phase value, therefore we take Φ. b (x,y)=φ b (x,y);

[0152] Step 204A3: Substitute the formula from step 204A2 into the formula from step 204A1 to obtain:

[0153]

[0154] In this embodiment, the specific process of performing phase expansion on φ2(x,y) in step 204 to obtain the absolute phase value Φ2(x,y) of the first color industrial camera with a fringe period of λ2 and a spatial frequency of f2 is as follows:

[0155] Step 204B1: When the zero-phase points are the same, the phase value is inversely proportional to the period of the sine stripes, which can be expressed by the formula:

[0156]

[0157] Where Φ2(x,y) represents the absolute phase value of the second color industrial camera with a fringe period of λ2 and a spatial frequency of f2, Φ b (x,y) represents a fringe period of λ. b The absolute phase value with spatial frequency f1-f2;

[0158] Step 204B2, due to λ b It can cover the entire field of view, and its absolute phase value is equal to the principal phase value, therefore we take Φ. b (x,y)=φ b (x,y);

[0159] Step 204B3: Substitute the formula from step 204B2 into the formula from step 204B1 to obtain:

[0160]

[0161] Step S3: Change the light intensity at each viewpoint and take multiple exposures. Repeat the processing steps of step S2, and then perform data fusion to obtain high dynamic range images for each viewpoint.

[0162] In practice, during the 3D scanning of the object, the robotic arm rotates to each position while simultaneously using an industrial camera and a structured light projector for illumination and imaging. For each viewpoint, multiple exposures are performed, and images are also captured using the structured light projector at different brightness settings to obtain more detailed information under varying lighting conditions. Specifically, N images with different exposures are acquired, covering 10... -4 cd / m 2 ~10 4 cd / m 2 The brightness range;

[0163] In this embodiment, the specific process of performing data fusion in step S3 to obtain high dynamic range images for each viewpoint is as follows:

[0164] Step S301: To eliminate the influence of the nonlinear response of the color industrial camera, establish a mapping relationship between pixel value Z∈[0,255] and linear radiosity:

[0165]

[0166] Step S302: Real-time conversion is achieved by pre-computing a 256th order lookup table (LUT) to convert discrete pixel observations into linear data that is proportional to physical radiance;

[0167] Step S303: Using the Robertson fusion algorithm, solve for the scene radiosity E using the weighted least squares method:

[0168]

[0169] Among them, Z ij Let Δt be the observation value of pixel i in the j-th exposure. j For the j-th exposure time, f -1 (Z ij w(Z) is the inverse response function, which converts pixel values ​​into linear radiosity; ij E is a weighting function used to suppress the influence of noise regions. i denoted as radiance of pixel i; M is the total number of pixels; P is the total number of exposures.

[0170] Step S304: Map E to the standard dynamic range using linear normalization:

[0171]

[0172] Among them, E min E is the minimum radiometric estimate for the scene. max E represents the estimated maximum radioactivity of the scene. norm This is the linearly normalized scene radiance value. The scene radiance value represents the exposure of the image.

[0173] The final result is a high dynamic range image for each viewpoint.

[0174] Step S4: Register, denoise, simplify, and smooth the point cloud data obtained from different perspectives, and generate triangular meshes. Then, fill holes and smooth the triangular meshes to generate a high-quality 3D model.

[0175] In this embodiment, the specific process of registering, denoising, simplifying, and smoothing the point cloud data obtained from different perspectives in step S4, generating a triangular mesh, filling holes and smoothing the triangular mesh to generate a high-quality 3D model is as follows:

[0176] Step S401: For each point p i Find its nearest point q in the point cloud from another perspective. j :

[0177]

[0178] Where q represents a point in the 3D model, and Q represents all points in the 3D model point cloud data;

[0179] Step S402: Given N pairs of nearest point pairs (p i ,q j ), calculate the rotation matrix R and translation vector t; the specific process is as follows:

[0180] Step S4021: Calculate the centroids p and q of the point cloud:

[0181]

[0182] Step S4022: Calculate the centroid-free point:

[0183]

[0184] Step S4023: Construct the covariance matrix H:

[0185]

[0186] Step S4024: Solve for the rotation matrix R by performing singular value decomposition (SVD) on the covariance matrix H:

[0187] H=UΣV T

[0188] R = VU T

[0189] Step S4025: Calculate the translation vector t:

[0190]

[0191] In practice, the calculation of R must ensure its orthogonality;

[0192] Step S403: Perform noise reduction, simplification, and smoothing operations on the point cloud data;

[0193] Step S404: Generate a triangular mesh model using Delaunay triangulation;

[0194] Step S405: Fill holes and correct self-intersection problems on the mesh, and smooth the mesh to generate a high-quality 3D model.

[0195] In practice, when registering point cloud data obtained from different perspectives, the point cloud data from different perspectives is first coarsely registered using the robot arm pose and turntable angle, and then finely registered using the ICP (Iterative Closest Point) algorithm, as described in step S401.

[0196] Step S5: Unfold and optimize the texture coordinates of the 3D model to obtain the texture coordinates of the 3D model; in order to reduce distortion and improve the texture mapping quality.

[0197] In this embodiment, the specific process of performing texture coordinate unpacking and optimization on the 3D model in step S5 to obtain the texture coordinates of the 3D model is as follows:

[0198] Step S501: Automatically select the edges on the model surface that need to be cut and mark them as cutting seams;

[0199] Step S502: Unfold the model spherically from the cut seam. Calculate and optimize the texture coordinates of each vertex based on the unfolding result. Use the Brown-Conrady distortion model to correct the coordinates (x,y) on the normalized image to (x′,y′). The calculation formula is as follows:

[0200]

[0201] Where, r 2 =x 2 +y 2 k1 and k2 are radial distortion coefficients, and p1 and p2 are tangential distortion coefficients, thus minimizing distortion as much as possible.

[0202] Step S6: Remove uneven lighting, highlights and shadows from the images obtained from different viewpoints, and segment the images to obtain segmented texture maps;

[0203] In this embodiment, the specific process of removing uneven lighting, removing highlights and shadows from images obtained from different viewpoints, and segmenting the images to obtain segmented texture maps in step S6 is as follows:

[0204] Step S601: In order to avoid the uneven lighting of images from different viewpoints affecting subsequent material segmentation, the image from each viewpoint is divided by the irradiance of the light source using the pre-calibrated light source position to remove the influence of uneven lighting.

[0205] Step S602: To avoid the influence of highlights and shadows, each pixel on the texture map is back-projected onto the original image, and the color corresponding to the median brightness in all images is taken as the color of that pixel. Combined with the texture coordinates, a diffuse texture map with highlights and shadows removed is obtained; the specific process is as follows:

[0206] Step S6021: Calculate the projection matrix based on the camera pose and intrinsic and extrinsic parameters of each viewpoint. The calculation formula is as follows:

[0207]

[0208] Among them, P GLDenotes the projection matrix, (f x f y (c) is the focal length, and the principal point offset is (c) x c y ), where w is the width, h is the height, and f is the height. n f is the depth value of the near plane. f This represents the depth value of the far plane.

[0209] Step S6022: Render the depth map D corresponding to each viewpoint using the projection matrix. k Used for subsequent visibility checks;

[0210] Step S6023: For each pixel p in the texture map uv To verify the consistency between its projected depth and mesh depth at viewpoint k:

[0211] |D mesh (p uv )-D k (π k (p uv ))|<∈ depth (=1.0)

[0212] Among them, D mesh (p uv ) represents the grid depth corresponding to a pixel, D k (π k (p uv )) represents the projection depth corresponding to a pixel, π k (p uv ) represents the projection matrix above; ∈ depth The threshold representing the depth difference;

[0213] In this embodiment, ∈ depth The value of is 1;

[0214] Step S6024: Find all images containing the pixel, sample the color of the pixel in these images and perform gamma correction. γ∈{1.0,2.2}, and then median filtering is performed on all viewpoint color values ​​to remove highlight pixels and low-light pixels, resulting in the final de-highlighted median texture map.

[0215] in, Represents the sRGB value of a pixel; This represents the final color value of a pixel;

[0216] Step S603: The Simple Linear Iterative Clustering (SLIC) algorithm is used on the texture map to segment it into different regions based on color, assuming that the metallicity and roughness are the same in each region, which facilitates subsequent material optimization; the specific process is as follows:

[0217] Step S60301: Divide the image into different superpixel regions, with each region as a node, and adjacent nodes are connected by edges;

[0218] Step S60302: Evaluate whether two adjacent superpixels can be merged by calculating two key indicators: edge weight and boundary strength score; specifically:

[0219] When constructing the graph model, edge weights are calculated based on the color difference between adjacent superpixels, given two superpixels S. i and S j Their color differences are measured by L2 distance:

[0220] w ij =‖c i -c j ||2

[0221] Among them, c i For SuperPixel S i The average chromaticity value, c j For SuperPixel S j The average chromaticity value, w ij Indicates the boundary weight between two regions;

[0222] Boundary strength is evaluated by calculating the range of intensity changes between adjacent superpixels, and a boundary strength score of B is assigned. ij Defined as:

[0223]

[0224] Where I(x) represents the intensity value of the image at pixel x, B ij Used to measure the range of intensity variation between two regions; the larger the value, the more distinct the boundary.

[0225] Sort all edge weights from smallest to largest, prioritizing merging regions with small color differences and weak boundary variations; for each edge, the merging condition is:

[0226] w ij <δ c and B ij <δ b

[0227] Where, δ c δ is the color difference threshold. b The boundary strength threshold;

[0228] When the edge weight is less than the preset color difference threshold δ c And the boundary strength score is less than the preset threshold δ b At that time, the two superpixels are merged into a new material cluster;

[0229] After merging, the graph model is updated, and the edge weights and boundary scores between the new nodes and adjacent regions are recalculated; the clustering process is repeated until no more adjacent regions meet the merging conditions, or the preset minimum number of clusters is reached; finally, the segmented texture map is obtained.

[0230] Step S7: Optimize the object's material properties using physically based differentiable rendering until the loss function converges or the predetermined number of iterations is reached, and finally obtain the object's surface material properties.

[0231] In practice, the SVDRDF model is used to describe the surface reflection properties. The SVDRDF model combines specular reflection and gloss variation, and its formula is based on the micro-element theory, specifically:

[0232]

[0233] Where, ω i It is the incident direction, ω o Here, n is the exit direction, n is the surface normal vector, and h is the half-range vector, defined as... ρ is diffuse reflectance, s is specular reflectance, D(h) is the micro-element distribution function, and F(ω) is the surface element distribution function. i G(ω,h) is the Fresnel reflection coefficient, and G(ω) is the Fresnel reflection coefficient. i ,ω o h) is the geometric occlusion function, and n is the surface normal vector;

[0234] Then, load the 3D model of the object, set the light source position and intensity, and set the initial values ​​for the base color, metallicity, and roughness texture maps. Next, define a camera viewpoint identical to the real viewpoint to perform physically based rendering on the 3D model, obtain the result, compare it with the actual captured image, obtain the loss function, and calculate the partial derivative of the loss function with respect to each parameter using automatic differentiation to construct the gradient. Update the parameters based on the calculated gradient, and iterate to optimize to minimize the loss function. When updating the roughness parameters, use the previous segmentation results to smooth each region to ensure that the roughness is the same within the same region. Repeat the above forward rendering, loss calculation, and parameter optimization until the loss function converges or the predetermined number of iterations is reached to obtain the material properties of the object.

[0235] In this embodiment, the loss function in step S7 is:

[0236]

[0237] Where, λ L1 for The weight, λ r for The weight, λ m for The weights, S is the number of pixels in the segmented region, I i I is the rendered pixel value of pixel i. target Let r be the target pixel value of pixel i. i and r j Let m be the roughness parameter of adjacent pixels i and j. i and m j These are the metallicity parameters of adjacent pixels i and j. It is the L1 distance. For roughness regularization, This is the metallicity regularization term.

[0238] In summary, this invention utilizes a structured light 3D scanner to reconstruct the geometric model of an object, and uses a structured light projector as a known light source. It optimizes the surface reflection properties of the object using physically based differentiable rendering, synthesizes and segments a median texture from multiple viewpoint images without illumination, and assumes that the metallicity and roughness are the same in each region. By introducing a known geometric model and light source, and by segmenting the texture, the ambiguity of material optimization can be significantly reduced, and the accuracy of material property acquisition can be improved.

[0239] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the present invention. Any simple modifications, alterations, or equivalent structural changes made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for obtaining the surface material properties of an object based on a 3D scanner, characterized in that: The 3D scanner includes two color industrial cameras mounted on a robotic arm and a structured light projector with a white light source; the method for obtaining the material properties of the object surface includes the following steps: Step S1: Place the object on the turntable. The robotic arm moves to drive the 3D scanner to perform structured light scanning. The 3D scanner projects a preset stripe pattern and captures the deformed stripe image modulated by the object through a color industrial camera. The captured stripe image is then transmitted to the processor. Step S2: The processor processes the stripe image to obtain point cloud data of the three-dimensional coordinates of the object surface; Step S3: Change the light intensity at each viewpoint and take multiple exposures. Repeat the processing steps of step S2, and then perform data fusion to obtain high dynamic range images for each viewpoint. Step S4: Register, denoise, simplify, and smooth the point cloud data obtained from different perspectives, and generate triangular meshes. Then, fill holes and smooth the triangular meshes to generate a high-quality 3D model. Step S5: Unfold and optimize the texture coordinates of the 3D model to obtain the texture coordinates of the 3D model; Step S6: Remove uneven lighting, highlights and shadows from the images obtained from different viewpoints, and segment the images to obtain segmented texture maps; Step S7: Optimize the object's material properties using physically based differentiable rendering until the loss function converges or the predetermined number of iterations is reached, and finally obtain the object's surface material properties.

2. The method for obtaining the surface material properties of an object based on a 3D scanner according to claim 1, characterized in that: Before step S1, there is also a step of calibrating the 3D scanner, the specific process of which is as follows: Step S01: Ensure that the industrial camera and structured light projector remain relatively stationary, and prepare the calibration board. Allow the structured light projector to project uniform light, and use the industrial camera to take images of the calibration board at different angles to calibrate the internal and external parameters of the industrial camera. Step S02: The structured light projector projects a phase-shifted fringe pattern. An industrial camera is used to capture images of the calibration board at different angles. By processing the captured images of the calibration board, the coordinates of the image point of each center point on the calibration board on the structured light projector are calculated. The intrinsic parameters of the structured light projector and the relative pose between the industrial camera and the structured light projector are calibrated using the same method as the camera calibration. Step S03: Perform current-brightness curve calibration on the structured light projector to facilitate subsequent high dynamic range image synthesis; The current-brightness curve calibration process includes projecting uniform light from the structured light projector under different currents and capturing images using an industrial camera. The relationship between current and brightness is obtained by statistically analyzing the average grayscale of small areas within the images. Color calibration is performed on the color industrial camera by photographing a standard color chart under uniform illumination from the structured light projector, calculating the color mapping relationship, and establishing a conversion matrix from standard color to camera-acquired color to standard color to ensure accurate color reproduction. The specific formula is as follows: Among them, a ij represents the linear transformation coefficient, and represents the contribution weight of the camera's j-th channel to the standard color's i-th channel; b i This represents the offset, used to compensate for overall brightness deviations in the color channels.

3. A method for obtaining the surface material properties of an object based on a 3D scanner according to claim 1 or 2, characterized in that: The specific process by which the processor in step S2 processes the stripe image to obtain the point cloud data of the three-dimensional coordinates of the object surface is as follows: Step 201: Perform N-step phase shifts, and the light intensity I of the i-th fringe image at coordinates (x, y) is... i (x,y) can be represented as: Where A(x,y) is the average light intensity, B(x,y) is the fringe modulation degree, and φ(x,y) is the principal phase value to be solved; i takes the value of a natural number from 0 to N; Step 202: The general formula for obtaining the principal phase value for N-step phase shift is expressed as: The principal phase value obtained by the phase shift method is the phase enclosed in the interval [-π, π]. Step 203: Let the principal phase values ​​of the two-order fringes with fringe period λ1 and spatial frequency f1, and fringe period λ2 and spatial frequency f2 be φ1(x,y) and φ2(x,y) respectively. According to the heterodyne principle, a new set of phases is obtained. The fringe period of the equivalent wavelength after heterodyne is: The corresponding spatial frequencies are f1-f2, and the principal phase value at point (x,y) is: φ b (x,y)=φ1(x,y)-φ2(x,y) Step 204: Perform phase expansion on φ1(x,y) and φ2(x,y) respectively to obtain the absolute phase value Φ1(x,y) of the first color industrial camera with fringe period λ1 and spatial frequency f1, and the absolute phase value Φ2(x,y) of the second color industrial camera with fringe period λ2 and spatial frequency f2. Step 205: Based on Φ1(x,y) and Φ2(x,y), obtain the coordinates (u,v) and (u',v') of the matching point in the principal point coordinate system on the images of the first and second color industrial cameras. The search range for stereo matching is on the horizontal epipolar line. The unfolded absolute phase is used as the feature of stereo matching. The process of searching for matching points in the right camera phase image is as follows: find the point with the closest phase and perform interpolation. First, based on the phase of the point to be matched in the left camera phase image, search for the point with the closest phase value to the point in the left image on the straight line with the same vertical coordinate as the point to be matched in the left camera phase image. This point is used as the initial matching point. Then, based on the phases of the left and right neighboring points, perform linear interpolation to obtain the right camera matching point with the same phase as the point to be matched in the left camera. Step 206: The equation for the 3D coordinates of the object surface reconstructed from the known matching point coordinates (u,v) and (u',v') is expressed as: Eliminating x yields: z(u-u')=2hf The three-dimensional coordinates x, y, z of the object's surface are represented as: Where f is the effective focal length after epipolar correction, d is the parallax, b is the baseline length, and h is half of the baseline length.

4. The method for obtaining the surface material properties of an object based on a 3D scanner according to claim 3, characterized in that: The specific process described in step 204 for performing phase expansion on φ1(x,y) to obtain the absolute phase value Φ1(x,y) of the first color industrial camera with fringe period λ1 and spatial frequency f1 is as follows: Step 204A1: When the zero-phase points are the same, the phase value is inversely proportional to the period of the sinusoidal fringe, which can be expressed by the formula: Where Φ1(x,y) represents the absolute phase value of the first color industrial camera with fringe period λ1 and spatial frequency f1, Φ b (x,y) represents a fringe period of λ. b The absolute phase value with spatial frequency f1-f2; Step 204A2, take Φ b (x,y)=φ b (x,y); Step 204A3: Substitute the formula from step 204A2 into the formula from step 204A1 to obtain:

5. The method for obtaining the surface material properties of an object based on a 3D scanner according to claim 3, characterized in that: The specific process described in step 204 for performing phase expansion on φ2(x,y) to obtain the absolute phase value Φ2(x,y) of the first color industrial camera with a fringe period of λ2 and a spatial frequency of f2 is as follows: Step 204B1: When the zero-phase points are the same, the phase value is inversely proportional to the period of the sine stripes, which can be expressed by the formula: Where Φ2(x,y) represents the absolute phase value of the second color industrial camera with a fringe period of λ2 and a spatial frequency of f2, Φ b (x,y) represents a fringe period of λ. b The absolute phase value with spatial frequency f1-f2; Step 204B2, take Φ b (x,y)=φ b (x,y); Step 204B3: Substitute the formula from step 204B2 into the formula from step 204B1 to obtain:

6. A method for obtaining the surface material properties of an object based on a 3D scanner according to claim 1 or 2, characterized in that: The specific process of performing data fusion in step S3 to obtain high dynamic range images for each viewpoint is as follows: Step S301: Establish the mapping relationship between pixel value Z∈[0,255] and linear radiance: Step S302: Real-time conversion is achieved by pre-computing a 256th order lookup table to convert discrete pixel observations into linear data that is proportional to physical radiance. Step S303: Using the Robertson fusion algorithm, solve for the scene radiosity E using the weighted least squares method: Among them, Z ij Let Δt be the observation value of pixel i in the j-th exposure. j For the j-th exposure time, f -1 (Z ij w(Z) is the inverse response function, which converts pixel values ​​into linear radiosity; ij E is a weighting function used to suppress the influence of noise regions. i denoted as radiance of pixel i; M is the total number of pixels; P is the total number of exposures. Step S304: Map E to the standard dynamic range using linear normalization: Among them, E min E is the minimum radiometric estimate for the scene. max E represents the estimated maximum radioactivity of the scene. norm This represents the scene radiance value after linear normalization.

7. A method for obtaining the surface material properties of an object based on a 3D scanner according to claim 1 or 2, characterized in that: The specific process described in step S4, which involves registering, denoising, simplifying, and smoothing point cloud data obtained from different perspectives, generating triangular meshes, filling holes in the triangular meshes, and smoothing them to generate a high-quality 3D model, is as follows: Step S401: For each point p i Find its nearest point q in the point cloud from another perspective. j : Where q represents a point in the 3D model, and Q represents all points in the 3D model point cloud data; Step S402: Given N pairs of nearest point pairs (p i ,q j ), calculate the rotation matrix R and translation vector t; the specific process is as follows: Step S4021: Calculate the centroid of the point cloud. and Step S4022: Calculate the centroid-free point: Step S4023: Construct the covariance matrix H: Step S4024: Solve for the rotation matrix R by performing singular value decomposition on the covariance matrix H: H=UΣV T R=VU T Step S4025: Calculate the translation vector t: Step S403: Perform noise reduction, simplification, and smoothing operations on the point cloud data; Step S404: Generate a triangular mesh model using Delaunay triangulation; Step S405: Fill holes and correct self-intersection problems on the mesh, and smooth the mesh to generate a high-quality 3D model.

8. A method for obtaining the surface material properties of an object based on a 3D scanner according to claim 1 or 2, characterized in that: The specific process of unpacking and optimizing the texture coordinates of the 3D model in step S5 to obtain the texture coordinates of the 3D model is as follows: Step S501: Automatically select the edges on the model surface that need to be cut and mark them as cutting seams; Step S502: Unfold the model spherically from the cut seam. Calculate and optimize the texture coordinates of each vertex based on the unfolding result. Use the Brown-Conrady distortion model to correct the coordinates (x,y) on the normalized image to (x′,y′). The calculation formula is as follows: Where, r 2 =x 2 +y 2 k1 and k2 are radial distortion coefficients, and p1 and p2 are tangential distortion coefficients.

9. A method for obtaining the surface material properties of an object based on a 3D scanner according to claim 1 or 2, characterized in that: The specific process described in step S6, which involves removing uneven lighting, highlights, and shadows from images obtained from different viewpoints, and then segmenting the images to obtain the segmented texture map, is as follows: Step S601: Divide the image of each viewpoint by the irradiance of the light source to remove the influence of uneven lighting; Step S602: Backproject each pixel on the texture map onto the original image, and take the color corresponding to the median brightness of all images as the color of that pixel. Combine this with the texture coordinates to obtain a diffuse texture map with highlights and shadows removed. The specific process is as follows: Step S6021: Calculate the projection matrix based on the camera pose and intrinsic and extrinsic parameters of each viewpoint. The calculation formula is as follows: in, Represents the projection matrix, ( , () is the focal length, and the principal point offset is () , ), For width, For height, This represents the depth value of the near-plane. This represents the depth value of the far plane. Step S6022: Render the depth map corresponding to each viewpoint using the projection matrix. Used for subsequent visibility checks; Step S6023: For each pixel in the texture map Verify its perspective Consistency between the projected depth and the mesh depth: in, This indicates the grid depth corresponding to a pixel. This represents the projection depth corresponding to a pixel. This represents the projection matrix above; The threshold representing the depth difference; Step S6024: Find all images containing the pixel, sample the color of the pixel in these images and perform gamma correction. Then, median filtering is applied to all viewpoint color values ​​to remove highlight and low-light pixels, resulting in the final de-highlighted median texture map. ; in, Represents the sRGB value of a pixel; This represents the final color value of a pixel; Step S603: Use the Simple Linear Iterative Clustering (SLIC) algorithm on the texture map to segment it into different regions based on color, assuming that the metallicity and roughness are the same in each region; the specific process is as follows: Step S60301: Divide the image into different superpixel regions, with each region as a node, and adjacent nodes are connected by edges; Step S60302: Evaluate whether two adjacent superpixels can be merged by calculating two key indicators: edge weight and boundary strength score; specifically: When constructing the graph model, edge weights are calculated based on the color difference between adjacent superpixels, given two superpixels. and Their color differences are measured by L2 distance: in, For superpixels Average chromaticity value, For superpixels Average chromaticity value, Indicates the boundary weight between two regions; Boundary strength rating Defined as: in, Indicates the image in pixels The intensity value at that location, Used to measure the range of intensity variation between two regions; the larger the value, the more distinct the boundary. Sort all edge weights from smallest to largest, prioritizing merging regions with small color differences and weak boundary variations; for each edge, the merging condition is: in, The color difference threshold. The boundary strength threshold; When the edge weight is less than the preset color difference threshold And the boundary strength score is less than the preset threshold. At that time, the two superpixels are merged into a new material cluster; After merging, the graph model is updated, and the edge weights and boundary scores between the new nodes and adjacent regions are recalculated; the clustering process is repeated until no more adjacent regions meet the merging conditions, or the preset minimum number of clusters is reached; finally, the segmented texture map is obtained.

10. A method for obtaining the surface material properties of an object based on a 3D scanner according to claim 1 or 2, characterized in that: The loss function mentioned in step S7 is: Where, λ L1 for The weight, λ r for The weight, λ m for The weights are S, where S is the number of pixels in the segmented region, and I is the weight. i I is the rendered pixel value of pixel i. target Let r be the target pixel value of pixel i. i and r j Let m be the roughness parameter of adjacent pixels i and j. i and m j These are the metallicity parameters of adjacent pixels i and j. It is the L1 distance. For roughness regularization, This is the metallicity regularization term.

Images