A three-dimensional reconstruction method and system based on polarization prior and physical constraint 3DGS

Abstract

The application discloses a three-dimensional reconstruction method and system based on polarization prior and physical constraint 3DGS, relates to three-dimensional computer vision and reverse engineering technology, and comprises the following steps: collecting multi-polarization state images of industrial parts from multiple perspectives, and separating diffuse reflection and highlight components from the multi-polarization state images; analyzing the polarization normal prior feature field of the industrial parts by using polarization information, constructing an initial three-dimensional geometric point cloud model according to the polarization normal prior feature field and a deep network; performing three-dimensional Gaussian processing on the initial three-dimensional geometric point cloud model, and binding physical parameters to each Gaussian body; classifying and analyzing the optical properties of high-reflective surfaces, and combining a bidirectional reflectance distribution function to establish a highlight reflection model; introducing physical calculation of light direction, observation direction and micro-surface normal in volume rendering, and simulating complex light interaction; and driving gradient descent by using a joint loss function, dynamically evolving Gaussian point cloud density, and realizing efficient generation of a high-precision three-dimensional model of the industrial parts.

Description

Technical Field

[0001] This invention relates to the fields of 3D computer vision and reverse engineering technology, and in particular to a 3D reconstruction method and system based on polarization prior and physical constraint 3DGS. Background Technology

[0002] High-precision 3D reconstruction technology for industrial parts has become a crucial foundation for precision defect detection, reverse engineering, and digital twin model construction. The geometric accuracy and material physical property reproduction of 3D reconstruction directly affect the reliability of subsequent industrial inspections and the realism of virtual simulations. In actual industrial production, many key components (such as precision machined metal parts, aero-engine blades, and semi-transparent composite materials) have surfaces with strong high reflectivity or weak texture. These "high reflectivity, weak texture" properties pose significant challenges to traditional 3D reconstruction.

[0003] Currently, the acquisition of 3D reconstruction data for industrial parts is mainly achieved through the following methods: First, traditional multi-view stereo vision (MVS) or structured light scanning. These methods rely on surface diffuse reflection and texture features for feature matching. In highly reflective areas, feature points are easily lost and depth maps abruptly change, resulting in a large number of holes and severe noise in the generated point cloud. Second, developing powder is sprayed onto the surface of the part to eliminate reflection. However, this physical contact not only increases the complexity of the process and changes the true surface morphology of the part, but is also usually strictly prohibited in non-destructive testing scenarios. Summary of the Invention

[0004] To address the problems in existing technologies where feature point loss and abrupt depth map changes are common in highly reflective areas, leading to numerous holes and severe noise in the generated point cloud, this invention provides a 3D reconstruction method and system based on polarization prior and physically constrained 3DGS. This method achieves sub-millimeter level high-precision 3D geometric reconstruction in highly reflective and complex curved surface scenes, while also physically and realistically restoring the optical properties of materials. The specific technical solution is as follows: A 3D reconstruction method based on polarization prior and physical constraints 3DGS includes the following steps: acquiring multi-view, multi-polarization state images of industrial parts, and separating diffuse reflection and specular components from the multi-polarization state images; analyzing the polarization normal prior feature field of the industrial parts using polarization information, and constructing an initial 3D geometric point cloud model based on the polarization normal prior feature field and depth network; performing 3D Gaussian processing on the initial 3D geometric point cloud model, and binding physical parameters to each Gaussian volume; classifying and analyzing the optical properties of highly reflective surfaces, and establishing a specular reflection model based on the bidirectional reflection distribution function; and reconstructing the model according to the observation direction. Incident light direction With micro-surface normal Construct rendering equations, calculate the final physical representation color of each Gaussian point based on the rendering equations, and then... -blending cumulative integral equation Calculate pixel rays The final output is luminosity, energy, and color; gradient descent is driven by a joint loss function to dynamically evolve the Gaussian point cloud density.

[0005] Preferably, acquiring multi-view, multi-polarization images of industrial parts and separating diffuse reflection and specular components from these images includes: obtaining light intensity information in four polarization directions, and reconstructing four linear polarization intensity images based on an edge-aware polarization demosaic interpolation algorithm using spatial gradients, denoted as follows: , , and ,in, The intensity of 0-degree polarized light. The intensity of the light is polarized at 45 degrees. The intensity of light polarized at 90 degrees. The intensity of the light is 135 degrees polarized; a four-dimensional Stokes vector is constructed based on four linearly polarized intensity images and polarization optics theory. Based on Stokes vectors Calculate the linear polarization degree and polarization phase angle ; An adaptive linear polarization threshold truncation function is set to physically separate the original mixed image into diffuse dominant image components. and specular reflection dominance of image components .

[0006] Preferably, the polarization normal prior feature field of the industrial part is analyzed using polarization information. Based on the polarization normal prior feature field and the depth network, an initial three-dimensional geometric point cloud model is constructed, including: establishing the linear polarization degree and the observed zenith angle based on Fresnel's law of refraction. The mapping relationship is used to solve for the zenith angle of the pixels on the part surface; combined with the zenith angle... With polarization phase angle Calculate the a priori characteristic field of the absolute polarization normal. The absolute polarization normal prior feature field As a physical hard constraint input to a multi-scale depth estimation neural network, an orthogonal energy functional between the depth gradient and the normal is constructed to correct the depth distortion error in highly reflective regions. The optimized depth map is then back-projected into 3D space, and an initial 3D geometric point cloud is constructed using a Poisson surface reconstruction algorithm.

[0007] Preferably, performing 3D Gaussian processing on the initial 3D geometric point cloud model and binding physical parameters to each Gaussian volume includes: transforming the initial geometric point cloud into a continuously differentiable 3D Gaussian probability density function. ; the three-dimensional covariance matrix Decompose into diagonal scaling matrices The product form of the rotation matrix; binding the initial physical vector characterizing the microsurface normal to the principal axis direction of the rotation matrix; assigning a base opacity to each Gaussian volume. And physical dimension parameters.

[0008] Preferably, the optical properties of highly reflective surfaces are classified and analyzed, and a high-reflectance model is established by combining the bidirectional reflection distribution function. This includes: constructing a standard illumination geometry in the three-dimensional world coordinate system and the local tangent space of the Gaussian volume, and defining the forward viewing direction. Incident light direction With micro-surface normal Calculate the half-range vectors of the incident direction and the observation direction according to the optical reflection law. Constructing the GGX normal distribution function model Establish metallicity learning parameters and calculate Fresnel reflection terms using the Schlick approximation formula. A geometric occlusion function is constructed based on the Schlick-GGX geometric approximation model. ,in, Direction of incident light The light attenuation function, For observation direction The light attenuation function is used; the Cook-Torrance microsurface model is employed to describe the hyperbola reflectance characteristics of Gaussian points, and a hyperbola reflectance model is established. .

[0009] Preferably, based on the direction of observation Incident light direction With micro-surface normal Construct rendering equations, calculate the final physical representation color of each Gaussian point based on the rendering equations, and then... -blending cumulative integral equation Calculate the pixel ray r The final output luminous energy color includes: real-time reading of the current observation direction along the forward propagation integration path of the projected light. Incident light direction With micro-surface normal And based on the energy of the external incident light Real-time integration to solve the complete rendering equation: ,in , The diffuse primary color of the Gaussian volume; real-time calculation of the diffuse primary color of each Gaussian point in the current virtual camera view. The final physical representation of color: ; Adopting based on -blending cumulative integral equation Calculate pixel rays The final output is luminous energy color, where, This represents the total number of Gaussian bodies in three-dimensional space that effectively intersect with the ray after being sorted by depth from near to far. For the first The actual opacity attenuation value of a Gaussian body at the current ray projection point.

[0010] Preferably, using a joint loss function to drive gradient descent and dynamically evolve the Gaussian point cloud density includes: constructing a joint loss function with multiple dimensions: , For the loss of luminance and color, For structure-aware loss, For normal cosine similarity loss, For polarization physical consistency loss; every certain number of iterations, accumulate the average position gradient shift of all 3D Gaussian bodies on the view plane, and perform Gaussian point cloning and splitting operations based on variance threshold; at preset intervals, adjust the opacity parameters of the Gaussian bodies. Reset to a minimum value close to zero.

[0011] Preferably, the degree of linear polarization The calculation formula is: The polarization phase angle The calculation formula is: ;in, This represents the total light intensity component of the image, which is equivalent to an unpolarized grayscale image acquired by a conventional RGB camera. This represents the difference in preferred intensity between linearly polarized light in the horizontal and vertical directions. This represents the difference in preferred intensity of linearly polarized light along a diagonal direction of ±45 degrees. .

[0012] Preferably, a three-dimensional Gaussian probability density function The expression is: ,in, It is the mean vector of the Gaussian body, whose initial value is directly inherited and anchored to the three-dimensional spatial coordinates of the initial point cloud, forming the spatial position anchor point of the Gaussian body; It is The three-dimensional covariance matrix.

[0013] This invention also provides a 3D reconstruction system based on polarization prior and physical constraint 3DGS, comprising: a multi-view polarization image acquisition and separation module, used to acquire multi-view multi-polarization state images of industrial parts and separate diffuse reflection and specular components from the multi-polarization state images; and a depth reconstruction module based on polarization normal, used to analyze the polarization normal prior feature field of industrial parts using polarization information, and construct an initial 3D geometric point cloud model based on the polarization normal prior feature field and depth network. The Physical 3DGS memory construction module is used to perform 3D Gaussian processing on the initial 3D geometric point cloud model and bind physical parameters to each Gaussian volume; the micro-surface material modeling module is used to classify and analyze the optical properties of highly reflective surfaces and establish a specular reflection model in conjunction with the bidirectional reflection distribution function; the BRDF volumetric rendering acceleration engine module is used to accelerate rendering based on the viewing direction. Incident light direction With micro-surface normal Construct rendering equations, calculate the final physical representation color of each Gaussian point based on the rendering equations, and then... -blending cumulative integral equation Calculate the pixel ray r The final output is luminosity, energy, and color; the joint optimization and density control module is used to drive gradient descent using a joint loss function to dynamically evolve the Gaussian point cloud density.

[0014] Compared with existing technologies, the beneficial effects of this invention are as follows: By introducing 3D Gaussian sputtering (3DGS) technology with polarization prior information and physical constraints, this invention effectively solves the limitations of traditional 3D reconstruction methods on highly reflective and weakly textured industrial parts. First, the acquisition of multi-view, multi-polarization state images and the separation of diffuse reflection and specular highlights provide clean basic data for subsequent processing, avoiding interference from specular highlights on feature extraction. Second, the analysis of the polarization normal prior feature field combined with a depth network can accurately correct the depth distortion in highly reflective areas, constructing a more accurate initial 3D geometric point cloud. Furthermore, by transforming the initial point cloud into a 3D Gaussian volume and binding physical parameters, combined with a specular reflection model based on the bidirectional reflectance distribution function (BRDF), a physically accurate restoration of the material's optical properties is achieved, making the reconstruction results not only geometrically accurate but also visually closer to the real object. The BRDF volumetric rendering acceleration engine efficiently calculates the physical representation color of each Gaussian point and obtains the final pixel color through α-blending cumulative integration, ensuring rendering efficiency and quality. Finally, the combined loss function-driven gradient descent and dynamic density evolution strategy further optimizes the distribution and representation of Gaussian point clouds, ensuring reconstruction accuracy while improving model robustness. This invention achieves sub-millimeter-level high-precision 3D geometric reconstruction and realistic restoration of material optical properties without requiring physical preprocessing of the part surface. Attached Figure Description

[0015] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0016] Figure 1 The flowchart shows the three-dimensional reconstruction method based on polarization prior and physical constraint 3DGS in Example 1. Figure 2 This is a block diagram of the 3D reconstruction system based on polarization prior and physical constraints 3DGS in Example 2. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0019] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0020] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0021] Example 1 Please refer to Figure 1 , Figure 1 This is a flowchart of a 3D reconstruction method based on polarization prior and physically constrained 3DGS. This embodiment provides a 3D reconstruction method based on polarization prior and physically constrained 3DGS, which includes the following steps: S1. Collect multi-polarization images of industrial parts from multiple perspectives, and separate diffuse reflection and specular components from the multi-polarization images.

[0022] Specifically, step S1 includes the following steps: S11. Obtain the light intensity information in four polarization directions, and reconstruct four linear polarization intensity images based on the edge-aware polarization demosaic interpolation algorithm of spatial gradient, and denote them as follows: , , and ,in, The intensity of 0-degree polarized light. The intensity of the light is polarized at 45 degrees. The intensity of light polarized at 90 degrees. The intensity of the polarized light is 135 degrees.

[0023] High-precision acquisition and physical separation of multi-view polarized images are the primary prerequisites for obtaining the intrinsic geometry and material properties of highly reflective parts. Traditional RGB cameras cause pixel potential well overflow (overexposure) when facing specular reflection, completely losing the underlying structural information. This step solves this problem by introducing Stokes polarization optics theory.

[0024] In this example, a high-precision multi-DOF robotic arm equipped with a multi-view polarization array camera is used to acquire multi-polarization image data of industrial parts from different perspectives by changing the shooting position. Specifically, a six-DOF precision industrial robotic arm is used as the core spatial positioning and motion execution mechanism, and its end flange is equipped with a micro-polarization array camera based on focal plane beam splitting (DoFP) technology. The complementary metal-oxide-semiconductor (CMOS) sensor surface of this camera is photolithographically etched with nanoscale metal wire grid micro-polarizers in a 2×2 macropixel array, which can simultaneously capture light intensity information in four different polarization directions (0°, 45°, 90°, and 135°) in a single hardware exposure cycle.

[0025] This hardware architecture effectively eliminates the minute time differences, mechanical vibration displacement, and motion blur errors caused by multi-frame sequential acquisition in traditional rotating polarizer systems. The camera resolution is set to 2448×2048 pixels, and combined with a low-distortion dual telecentric industrial lens, the imaging magnification remains unchanged regardless of object distance, strictly guaranteeing the orthogonal projection characteristics of edge pixels and thus eliminating perspective distortion.

[0026] Preferably, in terms of lighting space configuration, a composite lighting architecture is adopted, consisting of a uniformly diffuse integrating sphere light source and a programmable directional LED light source array. The radiation spectrum of the light source is strictly calibrated, and the illuminance of the central area is stabilized within the range of 10,000-15,000 lux by a programmable controller in a closed loop. This aims to fully excite and maintain a stable polarization state with an extremely high signal-to-noise ratio on the surface of highly reflective industrial parts. Regarding the trajectory planning for data acquisition, the robotic arm continuously samples along a set Fibonacci spiral trajectory, acquiring images from approximately 300 to 500 viewpoints within the hemispherical space above the part under test. This ensures an absolutely uniform distribution of observation viewpoints and avoids redundancy caused by dense sampling at extreme points.

[0027] Construct a four-dimensional Stokes vector. Calculate the linear polarization degree (DoLP) and polarization phase angle (AOP) for each pixel based on the Stokes vector. Set an adaptive linear polarization degree threshold truncation function to precisely separate the original image, which contains complex ambient light reflections, into a pure diffuse-dominated image component and a specular-dominated image component at the physical level.

[0028] At each fixed spatial pose point, the camera sensor outputs a raw Bayer mosaic image containing four linear polarization states. Since adjacent pixels correspond to different polarization transmission directions, direct extraction leads to jagged edges and polarization crosstalk artifacts. By introducing an edge-aware polarization de-mosaic interpolation algorithm based on spatial gradients, four full-resolution, spatially aligned linear polarization intensity images are reconstructed, denoted as follows: (0-degree polarized light intensity) (45-degree polarized light intensity) (Intensity of 90-degree polarized light) and (Intensity of 135-degree polarized light).

[0029] S12. Construct a four-dimensional Stokes vector based on four linear polarization intensity images and polarization optics theory. Based on Stokes vectors Calculate the linear polarization degree and polarization phase angle .

[0030] Stokes vectors The polarization characteristics of a light beam are fully characterized by a macroscopic linear combination of light intensities. The mathematical definition and rigorous calculation formula are as follows: In the formula, This represents the total light intensity component of the image, which is equivalent to an unpolarized grayscale image acquired by a conventional RGB camera. This represents the difference in preferred intensity between linearly polarized light in the horizontal and vertical directions. This represents the difference in preferred intensity of linearly polarized light along a diagonal direction of ±45 degrees. Representing right-handed and left-handed circularly polarized light ( and The intensity difference. Since the circularly polarized component of light reflection is extremely low and negligible on the surfaces of most amorphous birefringent industrial metals and polymer plastics, it is assumed in the computational architecture that… .

[0031] Based on the solved and error-compensated Stokes vector Calculate the linear polarization degree DoLP and polarization phase angle of the current light wave at a single pixel size. .

[0032] Among them, linear polarization degree The degree of linear polarization characterizes the proportion of energy in a light beam that is completely linearly polarized. Its calculation formula is as follows: .

[0033] polarization phase angle This reflects the two-dimensional projection direction of the maximum amplitude of the polarization photoelectric vector vibration, and its calculation formula is: It should be noted that, due to the limitations of the mathematical domain of the arctangent function, the calculated polarization phase angle... The range is strictly enclosed in between.

[0034] S13. Set the adaptive linear polarization degree threshold cutoff function. The original blended image is physically separated into diffuse dominant image components. and specular reflection dominance of image components .

[0035] The reflected light from the surface of industrial parts is, at the microscopic physical level, a linear superposition of diffuse reflection and specular reflection components. Based on the optical Umow effect and the principle of polarization dichroism, diffuse light undergoes complex and deep multiple surface scatterings within the material, resulting in its original polarization state being strongly disrupted by random lattice collisions. Consequently, when emitted from the macroscopic surface, it appears as approximately unpolarized light (corresponding to an extremely low degree of linear polarization). (value); while specular highlighting occurs directly and terminates at the outermost micro-surface of the object, strictly following Fresnel's law of reflection, exhibiting highly partially linearly polarized characteristics (corresponding to extremely high degree of linear polarization). (Value). Based on this physical prior, this method extracts a sliding window of a local image. By analyzing the distribution pattern of the linear polarization histogram within the background, a Gaussian mixture model (GMM) is constructed, or the Otsu method is used to express the probability density function as a weighted sum of the distributions of diffuse and specular reflectance. Through iterative optimization, the mathematical mean of background depolarization is accurately fitted, and the optimal adaptive polarization cutoff threshold is dynamically calculated based on this. Then, a Boolean-type specular mask matrix is ​​constructed. : .

[0036] After obtaining the mask matrix, the inverse mathematical transformation of the Stokes vector is used to perfectly and continuously separate the original mixed image affected by strong light interference into two independent data assets at the physical level.

[0037] Since specular reflection is assumed to be fully polarized, the intensity component of pure diffuse reflection is calculated. Simultaneously, the intensity component of the specular reflection from the stripped-off specular surface was calculated. .

[0038] Thus, a portion of the data is extracted as pure diffuse reflection dominant images that have removed specular blinding interference and completely preserved the processing texture and material background color. This data will be used in subsequent deep neural networks for extremely stable multi-view feature matching and initial geometric basis reconstruction. The other portion is specular reflection dominant images that simply contain optical reflection characteristics. This data will serve as an absolutely real physical supervision signal for inverse rendering supervision and physical parameterization learning of micro-surface roughness and metallicity in subsequent steps.

[0039] S2. Analyze the polarization normal prior feature field of industrial parts using polarization information, and construct an initial three-dimensional geometric point cloud model based on the polarization normal prior feature field and depth network.

[0040] Step S2 specifically includes: S21. Establishing the relationship between linear polarization degree and observed zenith angle based on Fresnel's law of refraction. The mapping relationship is used to solve for the zenith angle of the pixels on the surface of the part.

[0041] The degree of linear polarization (DoLP) of highly reflective surfaces is not random image noise, but rather correlated with the observed zenith angle (Zenith Angle) of the camera's line of sight. ) and the inherent complex refractive index of industrial parts materials. There exists an extremely strict microscopic physical mapping relationship. The real part... The attenuation and refraction ratio of the speed of light in the medium determines the imaginary part. It represents the absorption and attenuation coefficient of light wave energy by the medium.

[0042] According to the Fresnel equations for microsurface optics, the physical reflectivity parallel to the incident plane (p-polarized, i.e., TM wave) is... for: Physical reflectivity perpendicular to the incident plane (s-polarized, i.e., TE wave) for: .

[0043] For the high-light separation mask The linear polarization degree of the emitted beam from the selected specular reflection dominant pixels can be directly derived from the energy difference of the physical reflectivity: .

[0044] For industrial parts with known basic material properties (e.g., the complex refractive index of aluminum alloy surfaces is typically calibrated as follows), or standard polycarbonate plastic Since the above equations are highly nonlinear transcendental equations, analytical solutions cannot be directly obtained. Therefore, an energy objective function based on the residual between the observed and theoretical linear polarization degrees is constructed, and a Levenberg-Marquardt nonlinear least squares optimization algorithm with a trust-region strategy is used. By iteratively calculating the Jacobian matrix, the true zenith angle corresponding to each specular pixel can be solved with high accuracy in a very short computation cycle. .

[0045] S22, combined with zenith angle With polarization phase angle Calculate the a priori characteristic field of the absolute polarization normal. .

[0046] In accurately obtaining the zenith angle Then, combined with the polarization phase angle (i.e., azimuth angle), the system constructs a three-dimensional surface normal vector field in the camera's local coordinate system. The mapping relationship from spherical coordinates to three-dimensional Cartesian coordinates is as follows: .

[0047] Due to polarization phase angle The calculation is based on the arctangent function, which has inherent limitations in physical measurement. Ambiguity, i.e., the direction of electric vector vibration. and In Stokes vector and The macroscopic representations are completely equivalent. This mathematical ambiguity leads to severe "normal flipping" artifacts in the initially calculated three-dimensional normal field at the actual boundaries between protrusions and depressions on the part surface. To completely eliminate this physical ambiguity, this invention introduces a multi-view epipolar geometric prior.

[0048] Specifically, low-frequency coarse depth maps are generated using matching pixels from adjacent cameras. The spatial partial derivative of the depth map in the pixel plane is calculated to extract the gradient direction of the rough surface. Then, a minimum spanning tree based on local photometric confidence is constructed for global phase unrolling, and the projection direction of the polarization normal vector is forced to maintain physical consistency with the rough gradient direction. .

[0049] After unwinding correction and Gaussian edge-preserving filtering smoothing, the absolute polarization normal prior feature field, which is unaffected by specular reflection interference and has an extremely high signal-to-noise ratio and sub-millimeter-level topographic resolution, was successfully extracted. .

[0050] S23, the absolute polarization normal prior feature field As a physical hard constraint input to a multi-scale depth estimation neural network, an orthogonal energy functional between the depth gradient and the normal is constructed to correct the depth distortion error in highly reflective regions.

[0051] After obtaining the a priori absolute polarization normal ,Will As a high-weight physical hard constraint, it is injected into the feature extraction and regularization layers of a pre-trained deep estimation neural network (such as the improved MVSNet architecture based on 3D cost volume) through tensor concatenation.

[0052] Furthermore, to force the neural network output to conform to the true physical geometry, an orthogonal geometrically consistent energy functional is constructed that combines the depth range and the normal gradient. Assume the high-resolution continuous depth map predicted by the neural network in the two-dimensional image coordinate domain is... Then the three-dimensional surface points defined by this depth map The two local orthogonal tangent vectors can be approximated by their first-order partial derivatives. If the local normal vector formed by the partial derivative vectors of the depth map in space is required, it must be related to the prior polarization normal. Maintaining extremely strict orthogonal alignment. To this end, the system defines and incorporates a geometric gradient regularization loss term into the network's total loss function. : .

[0053] By backpropagating the energy functional through gradient descent, the network is forced to "flatten" the random spike noise in the depth map caused by overexposure of highlights or weak textures during the optimization training process, and the spatial continuity of the polarization normal is used to perfectly infer and fill the depth holes.

[0054] S24. Backproject the optimized depth map into three-dimensional space and use the Poisson surface reconstruction algorithm to construct the initial three-dimensional geometric point cloud.

[0055] Using the optimized and converged high-precision depth map and camera calibration intrinsic parameter matrix, it is back-projected pixel-by-pixel to the 3D world coordinate system. To ensure the data quality of subsequent 3D Gaussian initialization, a statistical outlier removal filter algorithm is used to remove stray aerial noise points, and a Poisson surface reconstruction algorithm is used to solve for the isosurface of the 3D indicator function. Multi-view point clouds are then deeply fused to generate a dense, smooth, and perfectly closed initial 3D geometric point cloud. This high-quality, hole-free geometric point cloud will serve as the core skeleton base for subsequent 3DGS physical property binding.

[0056] S3. Perform 3D Gaussian processing on the initial 3D geometric point cloud model and bind physical parameters to each Gaussian volume.

[0057] Traditional 3DGS (3D Gaussian Splatting) technology is essentially a pure radiation field visual synthesis method, which merely uses a three-dimensional Gaussian volume as a differentiable rendering primitive carrying color and opacity. This traditional architecture deeply couples the geometry of the object's surface, material reflectivity, and external ambient lighting together, completely lacking the underlying optical and physical meaning. As a result, it cannot perform physically correct relighting in complex and ever-changing lighting environments, nor can it separate the high-precision material properties required for industrial inspection.

[0058] Therefore, the core of step S3 lies in reconstructing the underlying memory data structure, binding the physical material properties of the micro-surface with the prior depth of polarization geometry to the three-dimensional Gaussian architecture, and constructing a "physically constrained 3DGS" with strict physical meaning.

[0059] Specifically, step S3 includes: S31. Convert the initialized geometric point cloud into a continuously differentiable three-dimensional Gaussian probability density function. .

[0060] Using the hole-free, high-precision initial geometric point cloud generated in step S2 as the prior skeleton basis, each discrete 3D coordinate point in the point cloud is instantiated into a continuous, smooth, and fully differentiable 3D multivariate Gaussian probability density function in 3D space. For any coordinate point in the continuous 3D space... , No. The probability distribution density of a Gaussian body is defined as: .

[0061] In the physical equations, It is the mean vector of the Gaussian body (i.e., the coordinates of the absolute center position of the Gaussian body in three-dimensional space). Its initial value is directly inherited and anchored from the three-dimensional space coordinates of the initial point cloud, forming the spatial position anchor point of the Gaussian body. It is The three-dimensional covariance matrix, from a mathematical perspective, determines the stretching scale, flattening degree, and overall volume influence range of the Gaussian body in three orthogonal directions in three-dimensional space.

[0062] This instantiation operation transforms the originally discrete, non-differentiable initial geometric point cloud into a continuous voxelized representation that supports full-link automatic differential computation by deep learning frameworks (such as PyTorch).

[0063] S32, The three-dimensional covariance matrix Decompose into diagonal scaling matrices The product form of the three-dimensional rotation matrix.

[0064] In the backpropagation optimization process based on gradient descent, if we directly apply... covariance matrix Unconstrained optimization of the nine elements of the covariance matrix is ​​highly susceptible to extreme gradients, causing the matrix to lose its positive semidefiniteness. If the eigenvalues ​​of the covariance matrix become negative, the Gaussian distribution loses its mathematical meaning and causes the underlying operators of the entire CUDA rendering system to crash (e.g., failure to invert or calculate the determinant). To ensure absolute mathematical and physical stability over extremely large iterations, the covariance matrix is... It can be rigorously analytically decomposed into a rotation matrix. With a diagonal scaling matrix The product form: .

[0065] Wherein, the diagonal scaling matrix It consists of three independent scaling factors, constrained by an exponential activation function, which independently control the stretching scale of the Gaussian volume along the three principal axes of the local coordinate system, ensuring that the volume is always positive. The expression for the diagonal scaling matrix is: Rotation matrix This completely abandons the Euler angle representation, which is prone to causing gimbal lock-up, and adopts a unit quaternion. High-dimensional parameterization not only ensures the smoothness of the rotation operation in three-dimensional space, but also guarantees the continuous differentiability of the gradient. The expression for the rotation matrix is: .

[0066] S33. Bind the initial physical vector characterizing the micro-surface normal to the principal axis direction of the rotation matrix.

[0067] Traditional 3DGS tends to generate extremely large ellipsoids that penetrate the object's surface to fit the viewpoint color during free optimization, resulting in a large number of redundant floating artifacts. In this step, the absolute polarization normal obtained by unwinding the epipolar constraint in step S2 is... It is directly mapped to the initial reference value of the normal of the Gaussian micro-surface and physically bound to the rotation matrix. On a certain principal axis (e.g., the Z-axis unit vector of the local coordinate system). Further, the scaling factor along this principal normal axis is reduced by a regularization penalty. The compression is brought close to zero. This extremely stringent physical binding requires that the 3D Gaussian volume must exhibit a "flat disk-like" or "panel-like" microstructure in 3D space, and must be tightly and orthogonally attached to the surface of the real industrial part. This constraint greatly suppresses the disordered generation of redundant Gaussian volumes within the model, reducing the dimensionality of pure volume rendering primitives back to a geometric category with well-defined surfaces.

[0068] S34. Assign a base opacity to each Gaussian body. And physical dimension parameters.

[0069] For each Gaussian body, initialize a sigmoid nonlinear activation function that is strictly limited to... The basic opacity scalar within the range Furthermore, in this step, a complete topological reconstruction of the underlying GPU CUDA operator's memory data structure was performed, decomposing macroscopic color into micro-surface physical material parameters, which are then directly attached to each Gaussian volume node as learnable high-dimensional tensor properties. Specifically, each Gaussian volume is newly bound with the following physical dimension parameters: Diffuse base color (Base Color / Albedo) The intrinsic albedo of ambient light is stripped away. Only the 0th-order spherical harmonic function (i.e., three constant scalars corresponding to the RGB channels) is used to represent the intrinsic diffuse color, which is unaffected by any lighting or viewing angle. This ensures the absolute purity of the model when extracting the color texture of industrial parts (such as surface rust and paint peeling), and is not affected by overexposure of highlights.

[0070] Metallic factor A continuously differentiable scalar parameter is introduced to characterize the physical material properties of the micro-surface containing the Gaussian body: whether it is an insulator (dielectric, such as the plastic casing or glass panel of an electric meter) or a conductor (machined metal part). This parameter will directly determine the fundamental reflectivity in Fresnel reflection in subsequent illumination calculations. The optical calculation logic is that the highlights of metals have their own primary color, while the highlights of non-metals are pure white.

[0071] Micro-surface roughness ): Controls the unevenness and roughness of the surface at the microscopic level. This is a parameter of great physical significance, as it directly determines the shape of the normal distribution function (NDF). When When the surface area approaches 0, the Gaussian body behaves as an extremely smooth mirror, producing extremely small but extremely sharp highlight patches; when... When the value approaches 1, the micro-surface is extremely rough, and the high light energy is diffused widely in all directions, forming a large area of ​​soft floodlight. This parameter enables the network to infer the micro-nano processing precision of the part surface from the image.

[0072] S4. Classify and analyze the optical properties of highly reflective surfaces, and establish a high-reflection model in conjunction with the bidirectional reflection distribution function.

[0073] To accurately reproduce the realistic visual appearance of industrial parts under various extremely complex and dynamically changing lighting conditions, and to completely eliminate the drawbacks of geometry-lighting depth coupling in traditional 3D compositing techniques, this step embeds a Cook-Torrance micro-surface bidirectional reflectance distribution function model that strictly adheres to the law of conservation of physical energy into the feedforward pipeline of differentiable rendering. For the surface rendering of each 3D Gaussian volume, instead of using the implicit output of a black-box neural network, its reflectance function is explicitly defined as an energy-conserving linear interpolation combination of the physical diffuse reflection term and the specular reflection term. Specifically, step S4 includes the following steps: S41. Construct a standard lighting geometry in the three-dimensional world coordinate system and the local tangent space of the Gaussian volume, and define the forward viewing direction respectively. Incident light direction With micro-surface normal .

[0074] Before performing any photometric integration, a standard lighting geometry is first constructed in the 3D world coordinate system and the local tangent space of the Gaussian volume. Assuming the projection ray of the virtual camera intersects the current 3D Gaussian volume at a valid point in space, the system extracts the incident light direction from the controlled point light source (or image-based global environment map HDRI sampling points) and rigorously normalizes it to a unit vector. Simultaneously, the observation direction pointing from the intersection point to the location of the virtual camera's optical center is normalized to a unit vector. The current Gaussian body, due to its underlying rotation matrix... The physical constraints, whose defined fundamental microsurface normals are extracted and represented as unit vectors. .

[0075] S42. Calculate the half-range vectors of the incident direction and the observation direction according to the optical reflection law. .

[0076] To accurately assess at the microscopic level how many extremely tiny geometric patches, whose normals perfectly reflect incident light rays into the camera's sensor, the system calculates the half-range vectors of the incident and viewing directions based on the laws of optical reflection. Its mathematical definition is: .

[0077] In addition, the system calculates and caches key geometric projection dot products, including incident cosines. With observation cosine By introducing The truncation function strictly eliminates invalid transmitted light rays from below the physical surface, ensuring the physical correctness of the single-sided material.

[0078] S43. Construct the GGX normal distribution function model ,in, The surface roughness physical parameters are autonomously learned by the Gaussian body through gradient descent during backpropagation. Approaching 0 indicates extreme smoothness; at this point, Degenerates into Dirac at mathematical limits. The function behaves as a mirror-like, perfectly directional reflection; when Approaching 1 indicates extreme roughness. At this point, the normals of the tiny facets exhibit a random chaotic distribution, and the high-light energy is uniformly scattered, forming a large area of ​​soft floodlight.

[0079] The normal distribution function determines the geometry, sharpness, and brightness attenuation gradient of the highlight patch on the object's surface. Considering the unique optical properties of machined industrial parts, this method abandons the traditional Blinn-Phong model, which leads to abrupt truncation of highlight edges, and instead adopts the industry-leading isotropic Trowbridge-Reitz (GGX) distribution model. This distribution possesses a longer "tail" effect than traditional models, perfectly matching the complex gradient reflective characteristics of actual industrial CNC machined metal surfaces.

[0080] S44. Establish metallicity learning parameters and calculate Fresnel reflection terms using the Schlick approximation formula. .

[0081] The Fresnel term describes the proportion of energy reflected by light at different incident angles (especially the physical phenomenon where reflectivity increases sharply and approaches 100% at grazing angles). To balance physical accuracy with the computational efficiency of backpropagation in extremely dense rendering pipelines with millions of Gaussian points, this method employs a highly optimized Schlick approximation to replace the original complex Fresnel complex integral equations: .

[0082] in, This is the material's basic optical reflectivity when the line of sight is perpendicular to the object's angle (0 degrees). In our physical material parameterization model, to completely eliminate the optical barrier between insulators and conductors, It is not a fixed value, but rather a metallicity factor carried by the Gaussian body. and diffuse primary color Dynamic interpolation is used to obtain the following: For insulators (dielectrics, such as the plastic casing of an electric meter or resin parts), their refractive index is typically around 1.5. The system uses physical laws to obtain this refractive index. The anchoring value is 0.04 and the highlight is pure white; for conductive materials (metal parts), the free electron oscillations cause the reflected light to have intrinsic color characteristics, therefore... The base color of the object is mixed in: .

[0083] Through this continuous remapping, the true optical response from pure dielectrics to pure metals in the continuous physical spectral domain is perfectly reproduced.

[0084] S45. Constructing a geometric occlusion function based on the Schlick-GGX geometric approximation model. ,in, Direction of incident light The light attenuation function, For observation direction The light attenuation function.

[0085] Geometric functions simulate the energy loss caused by the mountain-like undulations at the microscopic level in the path of light propagation. Physically, this is strictly distinguished into two phenomena: the self-shadowing effect, where incident light is blocked by protruding peaks before reaching the micro-surface, and the masking effect, where reflected light is blocked on its way to the camera. Combined geometric terms are calculated using separate Smith formulas: .

[0086] To maintain absolute mathematical consistency with the aforementioned GGX normal distribution function, the system employs a Schlick-GGX geometric approximation model specifically derived for GGX, whose one-sided ray attenuation function... Rigorous definition: .

[0087] To match direct illumination analysis based on known controlled light source distributions and prevent energy overexposure and artifacts caused by high-frequency illumination sampling when processing rough surfaces, the system performs remapping processing on roughness parameters within the physical safety domain. The roughness remapping coefficient is defined as follows: This ingenious mathematical parameter remapping not only ensures the numerical stability of the model at extreme grazing angles, but also fundamentally enforces that the photometric integral of the entire micro-surface model is always less than or equal to one, achieving an extremely stringent physical energy conservation closed loop in the rendering pipeline.

[0088] S46. The Cook-Torrance microsurface model is used to describe the hyperbola reflectance characteristics of Gaussian points, and a hyperbola reflectance model is established by combining the bidirectional reflectance distribution function: .

[0089] Total reflection function of industrial parts surface Defined as a linear combination of diffuse reflection and specular reflection components. To address the extremely strong specular reflection effect of highly reflective industrial materials, the system rigorously constructs a specular reflection term based on microsurface optics theory. This core physics term consists of three coupled high-dimensional functions, including a normal distribution function that describes the statistical distribution of normals on a microscopic surface. Fresnel equations describing the changes in light wave energy reflectivity from different observation perspectives. And the geometric occlusion function, which describes the attenuation of light energy caused by mutual occlusion between microsurface geometries. .

[0090] In the denominator of this formula, The term, as a calculus conversion factor for the projected solid angle, is used to accurately map and convert the local reflection energy at the microscopic level to the radiance distribution on the macroscopic surface, ensuring the absolute energy conservation of light during its propagation in space from a mathematical perspective.

[0091] Step S5: Based on the direction of observation Incident light direction With micro-surface normal Construct rendering equations, calculate the final physical representation color of each Gaussian point based on the rendering equations, and then... -blending cumulative integral equation Calculate the pixel ray r The final output is luminous energy and color.

[0092] After introducing the complex specular reflection model, the next step is to reconstruct the 3DGS forward volumetric rendering engine to support physically accurate lighting integral calculations while maintaining extremely high rendering frame rates, thereby completely decoupling geometry, material properties, and ambient lighting. Specifically, step S5 includes the following steps: S51. Read the current observation direction in real time along the forward propagation integration path of the projected light. Incident light direction With micro-surface normal And based on the energy of the external incident light Real-time integration to solve the complete rendering equation: ,in , The diffuse primary color of a Gaussian body: Traditional neural radiation fields and early 3DGS technologies typically output color directly as a black-box implicit function of spatial coordinates and viewing angle (such as multilayer perceptrons or spherical harmonic functions). This approach not only fails to respond to dynamic lighting but also easily produces self-illumination artifacts that violate optical laws. In contrast, this system's rendering engine strictly adheres to the law of conservation of energy at the microscopic optical level. The emitted color of each 3D Gaussian volume is no longer directly read from video memory but must be determined based on the energy of external incident light. Solve the complete rendering equations in real time using integrals.

[0093] Furthermore, in the formula This is the diffuse reflectance coefficient. According to the physical law of conservation of energy, if photons incident on an object's surface are reflected at the interface (i.e., converted into high-energy light), these photons cannot penetrate the surface to participate in subsurface scattering. Simultaneously, for metallic conductors, photons entering the material are rapidly absorbed by free electrons and converted into heat energy. Therefore, metallic materials do not exhibit intrinsic diffuse reflectance in the traditional sense. Based on this microscopic principle, the system uses the diffuse reflectance coefficient... With the high-light Fresnel reflection term and the metallicity parameters of Gaussian bodies Perform rigorous dynamic coupling calculations: .

[0094] S52. Real-time calculation of each Gaussian point in the current virtual camera view. The final physical representation of color: .

[0095] During the actual high-precision scanning and training phase, we acquire images in extremely controlled lighting environments (such as multi-directional uniform LED dot matrix or pre-calibrated high dynamic range (HDRI) environment maps), with continuous hemispherical incident light fields. It can be discretized under the Monte Carlo integral framework as The sum of the energy of a directional light source with known direction and intensity. During forward propagation rendering, the system calculates in real-time the energy of each Gaussian point within the current virtual camera's viewpoint. The final physical representation of color (Covering the energy superposition of physical diffuse reflection and micro-surface specular reflection).

[0096] S53, adopting a system based on -blending cumulative integral equation Calculate pixel rays The final output is luminous energy color, where, It is the total number of Gaussian bodies that intersect the ray in effective space in three-dimensional space after being sorted by depth from near to far. It is the first The actual opacity attenuation value of a Gaussian body at the current ray projection point.

[0097] The absolute physical representation of color for each Gaussian body in three-dimensional space was determined. Next, the rendering engine needs to accurately project the Gaussian distribution in three-dimensional space onto the view plane of the two-dimensional virtual camera. For each pixel ray on the two-dimensional image screen... The system first performs efficient 3D Gaussian frustum culling in global video memory, and then uses a hardware-level radix sorting algorithm to perform extremely fast depth sorting on millions of Gaussian volumes within the field of view. (Along pixel rays) It adopts the classic transparency-based blending ( The discretized volume integral formula (-blending) is used to accumulate and calculate the final output luminous energy and color of the pixel. .

[0098] In the formula, It is the first The actual opacity attenuation value of a Gaussian body at the current ray projection point. This attenuation value is determined by the base opacity of the Gaussian body. Multiplying by the probability density decay factor of its two-dimensional projected multivariate Gaussian distribution yields the following: ,in, These are the two-dimensional pixel coordinates of the current view plane. and These are the mean vectors of a three-dimensional Gaussian body. and the three-dimensional covariance matrix The analytical results of mathematical projection using affine camera transformation. Specifically, the projection covariance matrix satisfies the formula... ,in For the camera's world-to-view transformation matrix, It is an affine approximation of the Jacobian matrix of the perspective projection function.

[0099] Preferably, to overcome the computational performance bottleneck of traditional ray tracing, the entire complex matrix projection, intersection, and optical accumulation calculation process described above has been completely reconstructed and ported to a customized CUDA underlying operator. A block-based rendering architecture is adopted, dividing the 2D screen into multiple independent processing units of 16×16 pixels, directly processing the reading of Gaussian volume data and the analysis of physical formulas in parallel within the ultra-high-speed shared memory of the GPU streaming multiprocessor (SM). Simultaneously, the system introduces an early ray termination mechanism at the operator level, whereby the accumulated opacity along a ray... When the value approaches an absolute threshold (e.g., 0.999), subsequent Gaussian volume calculations will be forcibly terminated. This underlying optimization, achieved through hardware and software collaboration, still enables millisecond-level real-time rendering and synthesis of viewpoint images from millions of Gaussian points with micro-surface material parameters, even under an extremely large physical parameter calculation load.

[0100] S6. Use the joint loss function to drive gradient descent and dynamically evolve the Gaussian point cloud density.

[0101] This step is the core computational engine driving the spatial topology evolution of Gaussian point clouds, the convergence of micro-surface physical parameters (roughness, metallicity), and sub-millimeter-level high-precision reconstruction. Unlike traditional neural radiation fields that rely solely on color fitting, this invention utilizes a joint composite loss function with multimodal physical constraints, combined with an adaptive density control algorithm tailored to the physical parameters, and employs an adaptive moment estimation optimizer for end-to-end large-scale parallel optimization, achieving efficient generation of high-precision 3D models of industrial parts. Specifically, step S6 includes the following steps: S61. Construct a joint loss function with multiple dimensions. , For the loss of luminance and color, For structure-aware loss, For normal cosine similarity loss, This is the loss due to polarization physical consistency.

[0102] In each training iteration of the gradient descent-based backpropagation phase, error gradients between the rendered synthetic image and the original multi-view acquired image are collected from multiple high-dimensional feature spaces. To balance the convergence speed across different physical dimensions, the total loss function... The dynamic weighted scheduling mechanism is designed as follows: .in, For the loss of luminance and color, For structure-aware loss, For normal cosine similarity loss, This represents the polarization physical consistency loss. In a preferred embodiment, to ensure that the model balances geometric convergence of the specular surface and accurate decoupling of the physical material during the initial stage of gradient descent, the initial values ​​of the aforementioned dynamic weight coefficients are preferably configured as follows: During the model optimization iteration process, the system can adaptively and dynamically decay or fine-tune the above weights based on the current convergence state of the loss.

[0103] Photometric and color loss As a fundamental constraint on visual fidelity, the system incorporates pixel-level... Absolute error and local structural similarity (SSIM) are used to simultaneously measure the overall luminosity distribution at low frequencies and the texture detail at high frequencies. Their mathematical expression is: .

[0104] in, This is a dynamic weighting coefficient (usually initialized to 0.2). The predicted image is synthesized by integrating the microsurface rendering equation. This is a high dynamic range realistic image obtained after physical component separation. Using... Norm rather Norms can effectively enhance the robustness of models to small specular highlight noise.

[0105] The structure-aware loss is expressed mathematically as follows: .

[0106] To ensure that the rotation matrix of the 3D Gaussian volume accurately represents the micro-geometry of the object's surface, the system extracts a rendering depth map on the 2D view plane based on the volume rendering integral, and calculates the rendering normal in screen space using the spatial partial derivatives of the depth map. This is combined with the absolute physical polarization normal prior extracted in step two and de-wound using multiple views. Calculate the cosine distance penalty: .

[0107] Cosine distance penalty The constraint forces the minimum variance axis of the Gaussian body to be strictly perpendicular to the physical surface, which greatly reduces the generation of floating artifacts in the air and redundant internal geometry, so that the model can maintain an extremely smooth and continuous geometric surface even in highly reflective areas with drastic changes in lighting.

[0108] To overcome the dilemma of multiple solutions in the decoupling of material properties, the system utilizes the physical parameters (roughness) rendered from the forward rendering. , metallicity Diffuse primary color and geometric observation angle Substituting the Fresnel equation and the micro-surface distribution function, the rendered linear polarization map of the current viewpoint is predicted analytically. Then it was compared with the measured Stokes degree of polarization. conduct Mean square penalty for norm: .

[0109] This loss constitutes a closed-loop optical verification, forcing the neural network to learn absolutely physically correct metallicity and roughness values, completely avoiding overfitting caused by forcibly fitting highlights using incorrect primary colors.

[0110] To limit the unlimited expansion of GPU memory and ensure the lightweight nature of model data assets, the system adjusts the three-axis scaling factor of the covariance matrix. Anisotropy constraints are applied. This specifically penalizes abnormally large Gaussian bodies and extremely elongated "needle-shaped" Gaussian bodies that cause jagged edges in rendering; volume regularization penalizes the loss. The formula for calculation is: ,in The threshold for the proportion of extreme values ​​that can be tolerated (e.g., set to 10). This is the volume penalty coefficient.

[0111] S62. Every certain number of iterations, accumulate the average position gradient movement of all 3D Gaussian volumes on the view plane, and perform Gaussian point cloning and splitting operations based on the variance threshold.

[0112] The number of points and spatial topology in a 3D Gaussian point cloud are not fixed. During gradient descent, a high-frequency adaptive density control strategy is implemented to achieve local refinement and global redundancy pruning based on local geometric complexity.

[0113] Accumulate the average position gradient shift of all 3D Gaussian volumes on the 2D view plane every certain number of iterations (e.g., every 100 iterations). To assess its current optimization status.

[0114] Cloning operation: When the position gradient of a Gaussian body exceeds the system's preset sensitivity threshold. (For example, set to 0.0002), and its three-dimensional volume scaling factor is If the norm is smaller than the average scale of the current local scene, it indicates that the Gaussian volume is located in a region with dense geometric features (such as high-frequency edges of industrial parts, precision chamfers, or the interior of deep holes), resulting in geometric underfitting. The system will extract all the physical properties of the Gaussian volume and clone it along the direction of maximum descent of the view gradient to create a new Gaussian volume with a slight physical offset for subsequent optimization.

[0115] Splitting operation: When the position gradient of a Gaussian body also exceeds the threshold However, its volume is enormous (its variance excessively covers large areas of geometrically variable surfaces with drastically changing curvature), so the system will forcibly terminate its current state and split it into two entirely new Gaussian bodies. After the split, the volume scaling factor of the new Gaussian bodies will be proportionally reduced to a fraction of their original size. times (e.g.) And then, using the original covariance distribution as the probability density function, a new random sampling distribution of spatial locations is performed.

[0116] S63, The opacity parameter of the Gaussian body is set at a preset interval. Reset to a minimum value close to zero.

[0117] To escape local optima and eliminate accumulated rendering occlusion, the system adjusts the opacity parameters of all Gaussian volumes every relatively long period (e.g., 3000 iterations). The Gaussian bodies are forcibly reset to near-zero minimum values ​​(e.g., 0.01). In subsequent dozens of iterations, only those Gaussian bodies that truly contribute significantly to the photometric and gradient responses of the 2D rendered view regain high opacity. Afterward, the system traverses the entire memory pool, thoroughly eliminating redundant or occluded Gaussian bodies with opacities consistently below the truncation threshold (e.g., 0.005), thus freeing up GPU memory. By periodically forcibly decaying Gaussian points with extremely low opacity, lightweight data compression of the digital twin model is achieved. Through tens of thousands of joint physical gradient calculations and topological evolution iterations, the above optimization mechanism not only ensures the sub-millimeter-level three-dimensional structure reconstruction of the surface of highly reflective industrial parts, but also permanently solidifies their intrinsic physical material properties (such as the conductive texture of metal, the refractive index characteristics of transparent media, and the micro-roughness of CNC machining marks) in the form of explicit mathematical parameters in extremely lightweight 3DGS digital assets.

[0118] Example 2 Please refer to Figure 2 , Figure 2 This is a block diagram of a 3D reconstruction system based on polarization prior and physically constrained 3DGS. This embodiment provides a 3D reconstruction system based on polarization prior and physically constrained 3DGS, applied to the 3D reconstruction method based on polarization prior and physically constrained 3DGS in Embodiment 1, which includes: The multi-view polarization image acquisition and separation module is used to acquire multi-view, multi-polarization images of industrial parts and separate diffuse reflection and specular components from the multi-polarization images. The depth reconstruction module based on polarization normal is used to analyze the polarization normal prior feature field of industrial parts using polarization information, and to construct an initial three-dimensional geometric point cloud model based on the polarization normal prior feature field and the depth network. The Physics 3DGS memory building module is used to perform 3D Gaussian processing on the initial 3D geometric point cloud model and bind physical parameters to each Gaussian volume. The micro-surface material modeling module is used to classify and analyze the optical properties of highly reflective surfaces and establish a high-reflectance model by combining the bidirectional reflection distribution function. BRDF volumetric rendering acceleration engine module, used to adjust rendering based on viewing direction. Incident light direction With micro-surface normal Construct rendering equations, calculate the final physical representation color of each Gaussian point based on the rendering equations, and then... -blending cumulative integral equation Calculate the pixel ray r The final output is luminous energy and color; The joint optimization and density control module is used to drive gradient descent using a joint loss function to dynamically evolve the Gaussian point cloud density.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A three-dimensional reconstruction method based on polarization prior and physical constraint 3DGS, characterized in that, Includes the following steps: Acquire multi-view, multi-polarization images of industrial parts, and separate diffuse reflection and specular components from the multi-polarization images; The polarization normal prior feature field of industrial parts is analyzed using polarization information. Based on the polarization normal prior feature field and depth network, an initial three-dimensional geometric point cloud model is constructed. The initial 3D geometric point cloud model is processed by 3D Gaussian, and physical parameters are bound to each Gaussian volume; The optical properties of highly reflective surfaces are classified and analyzed, and a high-reflectance model is established by combining the bidirectional reflection distribution function. According to the direction of observation Incident light direction With micro-surface normal Construct rendering equations, calculate the final physical representation color of each Gaussian point based on the rendering equations, and then... -blending cumulative integral equation Calculate pixel rays The final output is luminous energy and color; Gradient descent is driven by a joint loss function to dynamically evolve the Gaussian point cloud density.

2. The three-dimensional reconstruction method based on polarization prior and physical constraint 3DGS according to claim 1, characterized in that, The acquisition of multi-view, multi-polarization images of industrial parts, and the separation of diffuse reflection and specular components from the multi-polarization images, includes: The light intensity information in four polarization directions is acquired, and four linear polarization intensity images are reconstructed based on an edge-aware polarization demosaic interpolation algorithm using spatial gradient, and denoted as follows: , , and ,in, The intensity of 0-degree polarized light. The intensity of the light is polarized at 45 degrees. The intensity of light polarized at 90 degrees. The intensity of polarized light at 135 degrees; A four-dimensional Stokes vector is constructed based on four linear polarization intensity images and polarization optics theory. Based on Stokes vectors Calculate the linear polarization degree and polarization phase angle ; By setting an adaptive linear polarization degree threshold truncation function, the original mixed image is physically separated into diffuse reflection dominant image components. and specular reflection dominance of image components .

3. The three-dimensional reconstruction method based on polarization prior and physical constraint 3DGS according to claim 1, characterized in that, The process of analyzing the polarization normal prior feature field of industrial parts using polarization information, and constructing an initial 3D geometric point cloud model based on the polarization normal prior feature field and the depth network includes: Establishing the relationship between linear polarization degree and observed zenith angle based on Fresnel's law of refraction The mapping relationship is used to solve for the zenith angle of the pixels on the surface of the part; Combined with zenith angle With polarization phase angle Calculate the a priori characteristic field of the absolute polarization normal. ; The absolute polarization normal prior feature field As a physical hard constraint input to a multi-scale depth estimation neural network, an orthogonal energy functional between the depth gradient and the normal is constructed to correct the depth distortion error in highly reflective regions. The optimized depth map is back-projected into 3D space, and the initial 3D geometric point cloud is constructed using the Poisson surface reconstruction algorithm.

4. The three-dimensional reconstruction method based on polarization prior and physical constraint 3DGS according to claim 1, characterized in that, The process of performing 3D Gaussian processing on the initial 3D geometric point cloud model and binding physical parameters to each Gaussian volume includes: The initial geometric point cloud is transformed into a continuously differentiable three-dimensional Gaussian probability density function. ; The three-dimensional covariance matrix Decompose into diagonal scaling matrices The product form of the three-dimensional rotation matrix; The initial physical vector characterizing the micro-surface normal is bound to the principal axis direction of the rotation matrix; Assign basic opacity to each Gaussian body And physical dimension parameters.

5. The three-dimensional reconstruction method based on polarization prior and physical constraint 3DGS according to claim 1, characterized in that, The classification and analysis of the optical properties of highly reflective surfaces, and the establishment of a high-reflectance model based on the bidirectional reflection distribution function, include: Construct a standard lighting geometry in the three-dimensional world coordinate system and the local tangent space of the Gaussian volume, and define the forward viewing direction accordingly. Incident light direction With micro-surface normal ; Calculate the half-range vectors of the incident and observation directions based on the optical reflection law. ; Constructing the GGX normal distribution function model ; Establish metallicity learning parameters and calculate Fresnel reflection terms using the Schlick approximation formula. ; Constructing a geometric occlusion function based on the Schlick-GGX geometric approximation model ,in, Direction of incident light The light attenuation function, For observation direction The light attenuation function; The Cook-Torrance microsurface model is used to describe the hyperbola reflectance characteristics of Gaussian points, and a hyperbola reflectance model is established. .

6. The three-dimensional reconstruction method based on polarization prior and physical constraint 3DGS according to claim 5, characterized in that, According to the direction of observation Incident light direction With micro-surface normal Construct rendering equations, calculate the final physical representation color of each Gaussian point based on the rendering equations, and then... -blending cumulative integral equation Calculate the pixel ray r The final output luminous energy color includes: The current observation direction is read in real time along the forward propagation integration path of the projected light. Incident light direction With micro-surface normal And based on the energy of the external incident light Real-time integration to solve the complete rendering equation: ,in , The diffuse primary color of a Gaussian body; Real-time calculation of each Gaussian point in the current virtual camera view. The final physical representation of color: ; Adopting based on -blending cumulative integral equation Calculate pixel rays The final output is luminous energy color, where, This represents the total number of Gaussian bodies in three-dimensional space that effectively intersect with the ray after being sorted by depth from near to far. For the first The actual opacity attenuation value of a Gaussian body at the current ray projection point.

7. The three-dimensional reconstruction method based on polarization prior and physical constraint 3DGS according to claim 1, characterized in that, The method of using a joint loss function to drive gradient descent and dynamically evolve the Gaussian point cloud density includes: Construct a joint loss function that includes multiple dimensions: , For the loss of luminance and color, For structure-aware loss, For normal cosine similarity loss, This is due to the loss of polarization physical consistency. Every certain number of iterations, the average position gradient shift of all 3D Gaussian volumes on the view plane is accumulated, and Gaussian point cloning and splitting operations based on the variance threshold are performed. The opacity parameter of the Gaussian body will be adjusted at preset intervals. Reset to a minimum value close to zero.

8. The three-dimensional reconstruction method based on polarization prior and physical constraint 3DGS according to claim 2, characterized in that, The degree of linear polarization The calculation formula is: The polarization phase angle The calculation formula is: ;in, This represents the total light intensity component of the image, which is equivalent to an unpolarized grayscale image acquired by a conventional RGB camera. This represents the difference in preferred intensity between linearly polarized light in the horizontal and vertical directions. This represents the difference in preferred intensity of linearly polarized light along a diagonal direction of ±45 degrees. .

9. The three-dimensional reconstruction method based on polarization prior and physical constraint 3DGS according to claim 4, characterized in that, The three-dimensional Gaussian probability density function The expression is: ,in, It is the mean vector of the Gaussian body, whose initial value is directly inherited and anchored to the three-dimensional spatial coordinates of the initial point cloud, forming the spatial position anchor point of the Gaussian body; It is The three-dimensional covariance matrix.

10. A three-dimensional reconstruction system based on polarization prior and physical constraint 3DGS, characterized in that, The three-dimensional reconstruction method based on polarization prior and physical constraint 3DGS, applied to any one of claims 1 to 9, includes: The multi-view polarization image acquisition and separation module is used to acquire multi-view, multi-polarization images of industrial parts and separate diffuse reflection and specular components from the multi-polarization images. The depth reconstruction module based on polarization normal is used to analyze the polarization normal prior feature field of industrial parts using polarization information, and to construct an initial three-dimensional geometric point cloud model based on the polarization normal prior feature field and the depth network. The Physics 3DGS memory building module is used to perform 3D Gaussian processing on the initial 3D geometric point cloud model and bind physical parameters to each Gaussian volume. The micro-surface material modeling module is used to classify and analyze the optical properties of highly reflective surfaces and establish a high-reflectance model by combining the bidirectional reflection distribution function. BRDF volumetric rendering acceleration engine module, used to adjust rendering based on viewing direction. Incident light direction With micro-surface normal Construct rendering equations, calculate the final physical representation color of each Gaussian point based on the rendering equations, and then... -blending cumulative integral equation Calculate the pixel ray r The final output is luminous energy and color; The joint optimization and density control module is used to drive gradient descent using a joint loss function to dynamically evolve the Gaussian point cloud density.

Images