Multi-view point cloud fusion and optical optimization high reflection curved surface reconstruction method and system

Abstract

This invention belongs to the field of visual measurement technology. It discloses a method and system for reconstructing high-reflectivity surfaces using multi-view point cloud fusion and optical optimization. The method includes: acquiring multi-source heterogeneous data such as fringe images, polarization images, and anchor point depths, and mapping them to a unified workpiece coordinate system to form a standardized dataset; calculating the normal of sampling points based on the data, establishing a slope field, and recovering the height through integration to obtain an initial geometric description of the surface; constructing candidate optical paths through reverse ray tracing, generating confidence weights by combining polarization degree, reflection count, uncertainty, and incident angle indices, and normalizing them within a local neighborhood; fusing multi-source point clouds under weight and anchor point constraints to establish a parameterized surface model, and constructing a joint optimization objective function including geometric error and optical consistency error; further, adaptively adjusting the weight rules based on residual distribution, performing closed-loop iterative optimization, and outputting a high-precision, stable initial surface shape of the high-reflectivity surface.

Description

Technical Field

[0001] This invention relates to the field of visual measurement technology, and more specifically, to a method and system for reconstructing high-reflectivity curved surfaces through multi-view point cloud fusion and optical optimization. Background Technology

[0002] Highly reflective surfaces are widely used in aerospace mirrors, optical imaging systems, and precision molds, and their surface accuracy directly determines the focusing and imaging performance of optical systems. However, common methods have limitations when performing inspections under high reflectivity conditions. Interferometry is sensitive to vibration and environmental disturbances, making it difficult to apply to large-aperture or complex clamping methods; structured light and laser scanning are prone to overexposure on highly reflective surfaces, generating a large number of false point clouds; while confocal or white light interferometry offers high longitudinal accuracy, its coverage is limited and cannot meet the requirements for large-area curved surfaces.

[0003] To improve applicability, existing studies have attempted to employ multi-source fusion, such as merging structured light and displacement sensor data. However, these often involve simple averaging or fitting. Due to the lack of differentiated discrimination against multiple reflections, overexposure, and noise points, low-quality data is still included with equal weight, resulting in artifacts in areas of strong reflection and affecting the overall reconstruction accuracy and stability.

[0004] Therefore, the main problem with existing technologies is that under high reflectivity conditions, the same pixel often exhibits both single specular reflection and multiple reflection signals simultaneously, accompanied by local overexposure and a decrease in polarization. Existing methods typically process these data equally during fusion, failing to distinguish which observations originate from the true master reflection path and which are affected by scattering or overexposure. This leads to the introduction of false point clouds into the integration, resulting in local artifacts and overall height drift in surface reconstruction.

[0005] In view of this, the present invention proposes a method and system for high-reflectivity surface reconstruction with multi-view point cloud fusion and optical optimization to solve the above problems. Summary of the Invention

[0006] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, the present invention provides the following technical solution: a high-reflectivity surface reconstruction method based on multi-view point cloud fusion and optical optimization, comprising:

[0007] S1: Simultaneously acquire stripe images, polarization images, and confocal cloud data of the highly reflective surface using multiple sensors, and calculate the degree of polarization and polarization angle of each pixel by combining the stripe images and polarization images, and calculate the normal and slope of each pixel.

[0008] S2: Based on the calibrated intrinsic and extrinsic parameters of the camera and projector, reverse ray tracing is performed on each pixel to generate candidate reflection paths, and their validity is determined by the consistency score of multi-frame observations to determine the reflection count indicator of the pixel.

[0009] S3: For each candidate reflection path, obtain the path uncertainty indicator based on the difference in travel estimation from different frames, and combine it with polarization degree, reflection number indicator, incident reflection angle deviation index and saturation / overexposure marker to generate multiple factor scores for each pixel.

[0010] S4: Combine multiple factor scores into an initial confidence score, and normalize the initial confidence score within the local neighborhood of the pixel to form the final confidence weight.

[0011] S5: After completing the weight calculation, the point cloud generated by the fringe phase is fused with the confocal cloud to obtain a globally consistent point cloud result;

[0012] S6: Based on the fused point cloud, a joint optimization function is introduced to obtain the initial surface shape result, and dynamic simulation and visualization output are performed to verify the stability and optical consistency of the surface under multiple poses.

[0013] S7: Repeat S2 to S6 using the surface obtained in S6 as the new geometric prior, and update the parameter set according to the residual after each iteration until the convergence condition is met.

[0014] Furthermore, the calculation of credibility weights includes:

[0015] Calculate signal quality factor based on polarization degree and saturation mask;

[0016] Set the reflection attenuation factor based on the number of reflections at the measurement point;

[0017] An angle penalty factor is set based on the angular deviation between the incident direction and the reflection direction relative to the surface normal;

[0018] The path instability penalty factor is set based on the statistical variance of the path length of multiple reflections;

[0019] The confidence weights of the point cloud points are formed by weighting each factor and then normalized within the point neighborhood.

[0020] Furthermore, the construction of candidate reflection paths includes: generating a primary mirror path, a first-order scattering path, and a secondary scattering path for each observed pixel through reverse tracing, and limiting the number of candidate paths to no more than a predetermined upper limit.

[0021] Furthermore, the determination of the number of reflections includes: performing a reprojection consistency score on candidate paths under multi-frame observation; when the consistency score of a candidate path is higher than a predetermined threshold, the path is counted as a valid path, and the number of valid paths is the reflection count indicator.

[0022] Furthermore, each candidate reflection path is determined by three parts: the projector light source point, the curved surface reflection point, and the camera imaging point. Under single-frame conditions, the travel estimate is calculated based on the three points. Under multi-frame conditions, a set of travel estimates is formed, the mean and standard deviation are calculated, and mapped to a normalized uncertainty score. The weighted average of all valid path uncertainty scores for a pixel is taken as the path uncertainty indicator.

[0023] Furthermore, an absolute height constraint is introduced during the point cloud fusion process. Points with a signal-to-noise ratio (SNR) not lower than a preset SNR threshold are directly added to the optimization as strong constraint points, while points with an SNR lower than the preset SNR threshold are introduced into the optimization through a penalty term.

[0024] Furthermore, the joint optimization function includes a geometric error term, an optical consistency error term, and a regularization term. The geometric error term represents the geometric difference between the surface and the point cloud, the optical consistency error term represents the difference between the predicted optical response and the measured optical response, and the regularization term restricts the second-order curvature of the surface.

[0025] Furthermore, in dynamic simulation, the predicted image is calculated using a geometric factor and a visibility function. The geometric factor represents the relationship between the surface normal, incident direction, reflection direction, and squared distance attenuation. The visibility function takes a value based on whether the point is occluded under the pose, taking 1 when visible and 0 when invisible.

[0026] Furthermore, during the dynamic simulation and closed-loop optimization process, a geometric layer, an optical layer, and a confidence layer are generated. The geometric layer represents the height, slope, and curvature distribution. The optical layer includes the simulation image, the measured image, and the residual map. The confidence layer includes the weight distribution, the visibility distribution, and the saturation mask.

[0027] A high-reflectivity surface reconstruction system based on multi-view point cloud fusion and optical optimization, implementing a high-reflectivity surface reconstruction method based on multi-view point cloud fusion and optical optimization, including:

[0028] Data acquisition module: Utilizes multiple sensors to simultaneously acquire stripe images, polarization images, and confocal cloud data of highly reflective surfaces from multiple perspectives, multiple exposures, and multiple polarization states;

[0029] Data preprocessing module: Performs phase calculation on the stripe image and calculates the degree of polarization and polarization angle of each pixel in combination with the polarization image, thereby calculating the normal and slope of each pixel;

[0030] Reflection path determination module: Based on the calibrated internal and external parameters of the camera and projector, reverse ray tracing is performed on each pixel to generate candidate reflection paths, and the validity of the paths is determined by the consistency score of multi-frame observations to determine the number of reflections of the pixel.

[0031] Path uncertainty calculation module: For each candidate reflection path, the path uncertainty indicator is obtained based on the difference in travel estimation from different frames, and combined with polarization degree, reflection number indicator, incident reflection angle deviation index and saturation / overexposure marker, multiple factor scores are generated for each pixel.

[0032] Credibility weight generation module: Combines multiple factor scores into an initial credibility score, and normalizes the initial credibility score in the local neighborhood of the pixel to form the final credibility weight;

[0033] Point cloud fusion module: After completing the weight calculation, the point cloud generated by the fringe phase is fused with the confocal cloud to obtain a globally consistent point cloud result;

[0034] Surface optimization module: The surface shape is represented by a parametric surface. The parametric surface is optimized under a joint optimization function, and dynamic simulation and visualization output are performed to verify the stability and optical consistency of the surface under multiple postures.

[0035] Iterative update module: The surface obtained by the surface optimization module is used as the new geometric prior. All processes from the reflection path determination module to the surface optimization module are repeatedly executed. After each iteration, the parameter set is updated according to the residual until the convergence condition is met.

[0036] The technical effects and advantages of this invention are as follows:

[0037] First, this invention proposes a pixel-level reliability weighting mechanism based on path integral: several candidate reflected light paths are generated for each measurement point, and the results are comprehensively evaluated based on factors such as multi-view / multi-frame consistency score, path length discreteness (uncertainty), polarization degree, number of reflections and incident / reflection angle deviation, and the final weight is normalized and synthesized in the local neighborhood, thereby highlighting reliable single mirror observations and suppressing artifacts caused by multiple reflections or unstable measurements, thus improving the quality of the fused input.

[0038] Secondly, an absolute height anchor point constraint is introduced into point cloud fusion and slope field integration. High-precision anchor points obtained by dispersive confocal displacement sensors are preferred. Strong constraints and penalty constraints are distinguished according to the signal-to-noise ratio and weighed in the integration process. This measure not only preserves local details but also provides absolute calibration for global height, effectively suppressing low-frequency drift of integration and improving the repeatability of results.

[0039] Then, the present invention establishes a joint optimization function: represented by a parameterized surface, which simultaneously incorporates a geometric error term, an optical consistency error term based on the selected reflection model, and a regularization term, and adopts a selective participation strategy to calculate optical consistency only for high-confidence pixels, thereby using optical information to correct geometric deviations caused by reflection while taking into account geometric accuracy, and improving the fitting robustness of high-reflectivity regions.

[0040] Finally, a closed-loop adaptive iterative mechanism based on multi-pose dynamic simulation is proposed: the image surface response is simulated using geometric factors and visibility functions based on the current surface, and the simulation and measured residuals are fed back to adaptively adjust the weight mapping, threshold and optimization parameters, and iteratively correct the surface to enhance robustness to complex multi-reflection scenarios, reduce manual parameter tuning and accelerate convergence.

[0041] In summary, the synergistic effect of four techniques—path integral-driven weight evaluation, absolute anchor point constraint, joint optimization function, and closed-loop adaptive method—enables this invention to exhibit higher accuracy, stability, and engineering applicability in the initial surface shape calculation of highly reflective surfaces, making it suitable for industrial testing and optical metrology applications. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the high-reflectivity surface reconstruction method with multi-view point cloud fusion and optical optimization in Embodiment 1 of the present invention;

[0043] Figure 2 This is a schematic diagram of path integral weight calculation in Embodiment 1 of the present invention;

[0044] Figure 3 This is a schematic diagram of the high-reflectivity surface reconstruction system with multi-view point cloud fusion and optical optimization in Embodiment 2 of the present invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Example 1

[0047] This embodiment provides a method for reconstructing high-reflectivity surfaces using multi-view point cloud fusion and optical optimization. See [link to relevant documentation]. Figure 1 ,include:

[0048] Deploy stripe projectors, polarized HDR cameras, and dispersive confocal displacement sensors to acquire multi-source heterogeneous data from highly reflective curved surfaces.

[0049] Specific methods include:

[0050] S111: Deploy a fringe projector on the high-reflectivity surface to project a sequence of sinusoidal fringes. Project 4 to 6 phase-shifted fringe images in each direction, with the fringe frequencies set in a pyramidal manner at 3 or more different frequency levels to balance large-scale coverage and local detail resolution.

[0051] The projection direction includes at least the horizontal and vertical directions to achieve full field coverage. Simultaneously, a polarized high dynamic range (HDR) camera synchronously acquires stripe images, using short, medium, and long exposures for each frame to obtain an HDR stripe image sequence covering both bright and dark areas.

[0052] S112: The polarization HDR camera is equipped with a built-in polarization filter array, or uses mechanically rotated polarizers to acquire four intensity images with polarization directions of 0°, 45°, 90°, and 135°, denoted as I0, I1, I2, I3, I4, I5, I6, I7, I8, I9, I10, I11, I2 ... 45 I 90 I 135 .

[0053] S113: The dispersive confocal displacement sensor performs sparse point or line scanning in key areas of the measured surface (including edge areas, high curvature areas, and functional parts such as assembly reference surfaces or boundary curves). The confocal probe detects height based on the difference in focal position corresponding to different wavelengths and outputs the absolute height value z. Each sampling point records its three-dimensional spatial coordinates and peak signal-to-noise ratio (SNR).

[0054] S114: Stripe projection, camera acquisition, and confocal scanning are all synchronized by a hardware trigger controller to ensure that all sensor samples have a consistent timestamp.

[0055] To achieve data alignment and fusion from multiple sensors, it is necessary to unify the data from various sensors into the same workpiece coordinate system W.

[0056] Let the point in the sensor coordinate system be... Then, coordinate transformation is performed using the external parameter calibration parameters (R,t):

[0057] ;

[0058] Where R is the rotation matrix (3×3), which is used to describe the rotation relationship between the sensor coordinate system and the working coordinate system;

[0059] t is a translation vector (3×1) used to describe the position of the origin of the sensor coordinate system relative to the working coordinate system;

[0060] The coordinates of the point in the workpiece coordinate system W are given.

[0061] It should be noted that the intrinsic parameters of the camera are calibrated by a checkerboard or dot array to obtain the mapping relationship between the pixel plane and the three-dimensional coordinates of the sensor; then, the extrinsic parameter calibration parameters (R,t) are obtained by calibration board or camera-point cloud joint calibration to achieve the unification of the camera coordinate system and the workpiece coordinate system.

[0062] Preprocessing of multi-source heterogeneous data such as stripe images, polarization images, and confocal clouds yields a unified, clean, and standardized dataset.

[0063] Specific methods include:

[0064] S121: On the pixel plane Representing pixel coordinates, the stripe images acquired at different exposure times are merged to obtain an HDR stripe image:

[0065] ;

[0066] in, For the k-th exposure, the pixel count is... grayscale value, For the weight function, The grayscale values ​​are those of the fused high dynamic range striped image.

[0067] The preferred weighting function is a triangular weighting based on pixel grayscale.

[0068] ;

[0069] If a pixel reaches the saturation threshold under all exposures, then a saturation mask is defined. ,otherwise .

[0070] S122: Apply a phase-shifting algorithm to the HDR stripe image sequence to obtain the wrapping phase. :

[0071] ;

[0072] in, For pixels The wrapping phase, Let be the phase shift of the k-th fringe.

[0073] The mass-guided phase unfolding method is used to wrap the phase. Unfolding into absolute phase , These correspond to the phase values ​​in the horizontal and vertical directions, respectively. In the region covered by the saturation mask, phase continuity is restored using neighborhood interpolation or a phase gradient-based compensation method.

[0074] S123: Calculate the Stokes parameters (I,Q,U) based on the four polarization images captured by the polarization HDR camera:

[0075] , , ;

[0076] in, This represents the total intensity, that is, the total brightness information of the light at the pixel. This represents the difference in light intensity along the horizontal and vertical polarization directions. This indicates the difference in light intensity along the ±45° polarization direction. , , , The image intensities are those acquired when the polarizer orientations are 0°, 45°, 90°, and 135°.

[0077] Further calculations of the degree of polarization and polarization angle:

[0078] , ;

[0079] in, Used to characterize the polarization properties of reflected light. It reflects the relationship between the normal and the polarization direction.

[0080] S124: For height data acquired by dispersive confocal imaging, points with an SNR below 10dB are filtered based on reflection peak width and peak signal-to-noise ratio (SNR). Isolated outliers are repaired using third-order spline interpolation. When the point cloud density is too high in a local area, sparsification is performed to reduce redundant points, and the cleaned sparse point cloud is output. That is, the three-dimensional spatial coordinates of point cloud i and signal-to-noise ratio .

[0081] After the coordinate unification operation described above, all types of data are mapped to the same workpiece coordinate system W and stored in a structured format as a standard dataset according to a preset format:

[0082] ;

[0083] in, For pixels Phase information under horizontal fringe projection For pixels Phase information under vertical fringe projection For pixels degree of polarization For pixels polarization angle, For pixels saturation mask.

[0084] The final output is a standardized dataset. .

[0085] The absolute phase result is converted into the normal information of the surface, and the three-dimensional point cloud and height distribution of the surface are further recovered.

[0086] Specific methods include:

[0087] S131: Absolute phase , The system's geometric calibration parameters (including the projector's optical center position, camera's imaging position, and attitude) are mapped to a reflection direction vector r(x,y). The calibration method adopts the "standard plane mirror calibration method," which involves projecting fringes onto a standard plane mirror with a known geometric position and acquiring the phase. The functional relationship between the phase and the reflection direction is solved by fitting the data. This functional relationship remains unchanged in subsequent measurement stages and is used to calculate the reflection direction of all measurement points.

[0088] S132: According to the law of specular reflection, the normal n(x,y) is the incident direction. The vector that bisects the reflection direction o(x,y):

[0089] ;

[0090] in, The incident direction from the projector to the surface point is calculated from the line connecting the optical center of the projector to the surface point. Let be the vector magnitude.

[0091] S133: Suppose the surface is represented by the height function z(x,y), then the normal can be expressed as:

[0092] ;

[0093] normal direction The local slope components are obtained by decomposition:

[0094] ;

[0095] in, Let be the slope of the surface in the x-direction. Let be the slope of the surface in the y-direction. These are the three components of the normal direction. It is a function of surface height.

[0096] For low-confidence regions (such as...) Values ​​below 0.3 or saturated mask The slope component of the region is marked with low weight to reduce its impact during subsequent fusion.

[0097] S134: Substitute the slope field (p,q) into the Poisson equation:

[0098] ;

[0099] In the workpiece coordinate system W, the equation is numerically solved using Fast Fourier Transform (FFT) or a multigrid algorithm to obtain the surface height distribution. .

[0100] CommonFocus Cloud Data Used as a boundary or interior point constraint, i.e., requiring:

[0101] ;

[0102] in, Let be the coordinates of the i-th common focus. The height value is obtained by integration.

[0103] This constraint ensures that the overall height does not drift and eliminates low-frequency errors.

[0104] S135: Final generation of dense point cloud :

[0105] ;

[0106] in, This is the signal-to-noise ratio of the height signal measured by the dispersive confocal displacement sensor at pixel (x,y).

[0107] Registration and fusion are performed using the ICP (Iterative Closest Point) method with normals to ensure that the point clouds are aligned in a unified workpiece coordinate system.

[0108] Weights are calculated based on path integrals to quantify the reliability of each measurement point, ensuring that reliable data receives high weights and unreliable data is automatically downweighted in subsequent data fusion and optimization.

[0109] See Figure 2 As shown, the specific methods include:

[0110] S141: Based on the calibrated intrinsic and extrinsic parameters of the camera and projector, for each observed pixel... First, reverse ray tracing is performed to generate a set of candidate reflection paths for that pixel. ,in This refers to the Mth candidate reflection path. Back-ray tracing starts from the camera center along the pixel ray direction and traces its potential reflection points in 3D space. ( This represents a three-dimensional Euclidean space (used to represent the coordinates of a point X=(x,y,z) in space) to accommodate situations where a single pixel on a highly reflective surface may correspond to multiple optical paths. For the same pixel, a primary specular candidate path is generated first, corresponding to the most direct and strongest specular reflection. If necessary, first-order scattering candidate paths and secondary scattering candidate paths can be additionally generated to account for light reaching the camera after one or more scattering events. To control computational complexity, an upper limit M is set on the number of candidate reflection paths per pixel; it is recommended to use M=3.

[0111] For each generated candidate reflection path Reprojection is performed on cameras at at least two different viewpoints or adjacent frames to obtain the reprojection position. and And calculate its reprojection position difference. :

[0112] ;

[0113] in, Represents Euclidean distance. and Candidate reflection paths The reprojection position coordinates in the a-th and b-th viewpoints (or adjacent frames).

[0114] And assign a consistency score :

[0115] ;

[0116] When a candidate reflection path is viewed from multiple perspectives A path is considered valid at a given time; the number of valid paths is the pixel's reflection count indicator. .

[0117] in, For scale parameters, This is the consistency threshold.

[0118] S142: Each candidate reflection path is determined by the projector light source point. Surface reflection point and camera imaging point The three parts are determined.

[0119] For a candidate reflection path, its travel estimate can be calculated. for:

[0120] ;

[0121] Candidate reflection paths under multi-view or multi-frame conditions By observing under different cameras or different exposure frames, a set of travel estimates can be formed. , Indicates candidate reflection path The travel estimate under the d-th camera viewpoint or the d-th frame exposure condition.

[0122] The estimated travel distances for this group were statistically analyzed, and the mean was calculated. with standard deviation :

[0123] , ;

[0124] in, Candidate reflection paths The mean travel distance under all observation conditions, where D is the total number of observations of the candidate reflection path at different viewpoints or frames.

[0125] The standard deviation Reflects candidate reflection paths stability, when A larger value indicates that the path is inconsistent across multiple views / frames, resulting in low reliability.

[0126] Finally, Mapped to normalized uncertainty score :

[0127] ;

[0128] in, This is a smoothing parameter used to avoid instability caused by the denominator approaching zero. Its value can be determined based on the signal-to-noise ratio and measurement accuracy within

[10] . −3 10 −1 Set within the range, preferably 10. −2 Magnitude.

[0129] Corresponding path uncertainty score The magnitude can be measured using the path uncertainty scale. Please provide an explanation. The values ​​are set based on experience and are used to provide a typical range of path uncertainty in actual measurements, as well as to assist in parameter adjustment and result evaluation.

[0130] For pixel q, take the uncertainty score of all valid paths. The weighted average as an indicator of path uncertainty .

[0131] It should be noted that for each valid path of pixel q, its consistency score is taken as the sovereign weight, and then normalized so that its sum is 1. The path uncertainty indicator of pixel q is defined as the weighted average of the uncertainty scores of all its valid paths to that weight.

[0132] Based on the degree of polarization, polarization is mapped to a polarization factor. When the degree of polarization is greater than or equal to the threshold of the degree of polarization hour, Set to 1; when the degree of polarization is less than or equal to the lower threshold of the degree of polarization. hour, Take 0; when it is in between: .

[0133] For reflection count indicator Define the attenuation factor :

[0134] ;

[0135] in, This is the attenuation coefficient.

[0136] When the incident and reflected directions deviate from the normal, an angle penalty factor is defined. :

[0137] ;

[0138] in, For the included angle deviation, This is the maximum allowed angle.

[0139] set up This is a pixel saturation indicator. It is denoted as q when the imaging intensity corresponding to pixel q reaches the sensor saturation threshold or when overexposure occurs. ,otherwise .

[0140] The set of factors of pixel q is represented as:

[0141] ;

[0142] It should be noted that, in the preferred embodiment of the present invention, the threshold, scale parameter, factor, and maximum angle used to generate pixel factor scores can be determined through offline calibration or online adaptive method. The offline calibration method involves collecting data from several standard samples (mirror surfaces, semi-mirror surfaces, coated surfaces, etc.) under multiple viewing angles, multiple exposures, and multiple polarization states, statistically obtaining the distribution of polarization degree, reprojection difference, and travel estimation, and setting initial thresholds and weights based on the statistical results. The online adaptive method involves dynamically adjusting the mapping rules or thresholds according to the spatial distribution of geometric and optical residuals during the closed-loop iteration process to improve convergence and robustness. An empirical range is given as an example: consistency threshold. Threshold on polarization degree Threshold under polarization degree Path uncertainty scale , (For applications requiring stronger artifact suppression, a value of 1.0 to 2.0 can be used.) .

[0143] S143: Combine the scores of each factor into an initial confidence score. :

[0144] ;

[0145] S144: To prevent numerical instability, the weights within the neighborhood of each point are normalized.

[0146] ;

[0147] in, As the final credibility weight, Let be the set of neighborhoods of point q. Let q represent a point in the neighborhood set of point q.

[0148] After completing the weight calculation, the point cloud generated by the fringe phase is robustly fused with the confocal cloud to eliminate scale drift and low-frequency errors, resulting in a globally consistent dense point cloud.

[0149] Specific methods include:

[0150] S151: Match the point cloud data before fusion, including:

[0151] KD-tree search is used to establish the nearest neighbor relationship between the striped point cloud and the confocal point cloud;

[0152] Statistical methods are used to eliminate point cloud errors: if the distance error of a point exceeds three times the standard deviation of the mean, it is judged as an outlier and eliminated.

[0153] Outlier points can be further removed through consistency checks (such as neighborhood normal angle constraints).

[0154] S152: After completing point cloud matching and cleaning, utilize the final confidence weight. A fusion optimization model is constructed to minimize the weighted residuals, and a robust loss function is used to reduce the impact of outliers while ensuring the overall smoothness of the point cloud. The objective function is:

[0155] ;

[0156] in, The objective function for fusion optimization is q, where q is the point index in the point cloud. Each point index q in the point cloud is obtained by back-projecting the corresponding pixel q=(u,v) into 3D space, thereby establishing the correspondence between pixels and the point cloud. Let q be the final credibility weight. To optimize the 3D position of point q, For the original measured location or the locally fitted predicted location, Robust loss functions (such as the Huber function) are used to reduce the impact of outliers. This is a balancing factor used to control the smoothing regularization term. To smooth out regularization terms (such as second-order differences), constrain the overall smoothness of the point cloud.

[0157] S153: Based on the weighted optimization model, the absolute height constraint provided by the confocal cloud is further introduced to eliminate the overall drift caused by the integral.

[0158] When the signal-to-noise ratio At that time, direct enforcement:

[0159] ;

[0160] When the signal-to-noise ratio When this happens, a penalty term is introduced:

[0161] ;

[0162] in, The index representing the confocal anchor point. This indicates the optimized position of the point. This represents the coordinates measured by confocal measurement. This represents the maximum weight value. This represents the confocal cloud constraint energy, used to incorporate the absolute height information measured by the confocal cloud during the point cloud fusion optimization process.

[0163] S154: After introducing the weights and anchor point constraints, the optimization problem of the fusion optimization model is solved numerically.

[0164] Iterative optimization is performed using the iterative reweighted least squares (IRLS) method. In each iteration, the weights are updated according to the residual size. The weights of high residual differences are gradually reduced, while the weights of low residual differences are kept at a relatively high level.

[0165] The iterative process continues until the objective function is reached. Convergence or reaching the preset number of iterations.

[0166] The actual solution is completed based on the constraints in step S153, ensuring that the weight adjustment and absolute height constraint take effect simultaneously under a unified framework.

[0167] S155: Obtain a globally unified fused point cloud. :

[0168] ;

[0169] in, Represents the three-dimensional coordinates of a point. Indicates the normal direction. and Indicates polarization characteristics, This represents the final credibility weight of that point.

[0170] By introducing a joint optimization function based on the fused point cloud, and taking into account both the geometric distribution and optical response consistency of the point cloud, a surface representation that balances geometric accuracy and optical performance is obtained.

[0171] Specific methods include:

[0172] S161: In the fused point cloud Based on this, a parametric surface model is used to represent the workpiece surface. A non-uniform rational B-spline (NURBS) surface is used, denoted as:

[0173] ;

[0174] in, For the surface in parameters The corresponding three-dimensional coordinate points below, To control the vertices, To control vertex weights, g and h control the row and column indices of the network. , Along the parameter direction , The nonrational B-spline basis functions are used to define the weight distribution and influence range of the control vertices in the corresponding directions. , This indicates that the number of vertices in the control network in the ξ and η directions is reduced by one. This is the set of surface parameters.

[0175] In practical applications, the control grid density is adaptively set based on the curvature a priori, with denser control points placed in areas of greater curvature variation. Based on the discrete point cloud, the surface is transformed into a continuous curved surface model.

[0176] S162: To ensure that the surface's geometric shape is consistent with the point cloud data, a geometric error term is defined:

[0177] ;

[0178] in, For geometric error terms, For parameter set The surface represented This represents the Euclidean distance from a point to a surface. This is the final credibility weight.

[0179] Through weight control, high-confidence points have a greater impact on surface fitting.

[0180] S163: High-reflectivity curved surfaces are sensitive to optical response, introducing an optical uniformity error term. Defined as:

[0181] ;

[0182] The optical consistency error term is calculated only for pixels whose confidence weight is higher than a predetermined threshold. The threshold is set according to actual needs.

[0183] It should be noted that, Calculated using the reflection model:

[0184] ;

[0185] in, For optical consistency error, For point The measured optical response, For the optical response predicted based on the surface model, For curved surfaces The direction of the place, The incident direction, The direction of reflection, For the incident light intensity, The surface reflection function is preferably the Cook-Torrance model or the simplified Blinn-Phong model.

[0186] The measured optical response refers to the response observed at a spatial point through an optical sensor (such as a camera, CCD, photodetector, etc.). These are the actual measured values ​​obtained from the experiment or sensors. These are quantities such as light intensity, brightness, and radiance directly collected by the experiment or sensors; they are not theoretical calculations, but actual observation results with noise and errors.

[0187] S164: To prevent the surface from overfitting noise points, a regularization term is introduced:

[0188] ;

[0189] in, This is a regularization term that restricts excessive curvature of the surface. Define the domain of the surface. and It is the second-order partial derivative, used to characterize curvature.

[0190] S165: Combining the three terms from S162 to S164, define the joint optimization function:

[0191] ;

[0192] in, For joint optimization functions, and The parameters used to adjust the weights of geometry, optics, and regularization are calibrated through experimental experience.

[0193] The preferred method is the quasi-Newton method (L-BFGS), which iteratively updates the parameter set. until The threshold value is less than the objective function or the objective function converges.

[0194] in, This represents the update magnitude of all control vertex coordinates and weights in one iteration, i.e.:

[0195] ;

[0196] in, To control the vertices The amount of coordinate change in three-dimensional space. To control vertex weights The change in quantity.

[0197] The sum of the squares of the changes in all control vertices and their corresponding control vertex weights, and the square root of the result, is obtained. The formula used to determine whether the iteration has converged is:

[0198] ;

[0199] Set the maximum number of iterations to 100 and the convergence threshold to 1e-6.

[0200] S166: Through the above optimization, the optimal parameter set is obtained. The corresponding surface This is the initial surface shape result.

[0201] Obtaining the initial surface shape Then, dynamic simulation and visualization output are performed to verify the stability and optical consistency of the surface under multiple postures.

[0202] Specific methods include:

[0203] S171: In different postures Next, the curved surface points Projected onto the camera's image plane, the pixel coordinates are obtained. Combining surface normal Incident direction Reflection direction and reflection function Calculate and predict optical response :

[0204] ;

[0205] in, For the imaging kernel function, The incident radiation intensity, Geometric factors are used to characterize points. The combined geometric effect of the normal, incident direction, reflection direction, and squared distance attenuation. This is a visibility function used to indicate points. Whether the camera is visible at attitude t, set to 1 if visible, and 0 if invisible.

[0206] The imaging kernel function Used to characterize points on a surface Image plane pixels under pose t The contribution of the imaging kernel function is that it includes the point spread function and pixel response function of the optical system. It can also be approximated by the δ function under simplified conditions, considering only the geometric projection relationship.

[0207] S172: Predict the image Compared with the measured image Comparisons include:

[0208] Image plane mean square error ;

[0209] Root mean square of surface fitting residuals ;

[0210] in, Represents a set of pixels. This represents a set of merged point clouds.

[0211] The measured image This refers to the image data acquired by an actual optical imaging device at attitude t.

[0212] S173: Based on the error evaluation results, generate multi-layered visualization images to intuitively demonstrate the conformity between the curved surface and optical properties, specifically including:

[0213] Geometric layers: These include height maps, slope maps, and curvature distribution maps. The height map is derived from the surface model. The z(x,y) values ​​in a Cartesian coordinate system are mapped to the slope plot; the slope plot is obtained by the partial derivatives of the surface at each sampling point. , The calculation is then mapped to a two-dimensional coordinate system to generate the curvature distribution map, which is obtained by calculating the principal curvature from the surface normal field and represented in grayscale or pseudocolor.

[0214] Optical layers include simulated images, measured images, and residual maps. The simulated images are generated from the predicted optical response calculated by a formula, the measured images are directly acquired by the camera, and the residual maps are obtained by calculating the difference between the predicted image and the measured image pixel by pixel and mapping it to a grayscale or pseudo-color image.

[0215] Confidence layer: includes weight distribution, visibility distribution, and saturation mask. The weight distribution map is derived from the final confidence weights. Spatial mapping generation; visibility distribution is generated by a function The values ​​are mapped to obtain the saturation mask; the saturation mask is obtained by marking the pixels in the measured image that have reached the saturation threshold.

[0216] By comparing the predicted results with the measured results, the residuals are calculated and fed back to correct the surface parameters, thus achieving iterative optimization of the geometry-optics joint approach.

[0217] Specific methods include...

[0218] S181: Define the image plane residual at attitude t. :

[0219] ;

[0220] in, These are the pixel values ​​of the measured image. The pixel values ​​of the calculated predicted image.

[0221] In the surface domain, the geometric residual is defined. :

[0222] ;

[0223] in, To merge point cloud coordinates, This is the current surface model.

[0224] S182: Incorporate the residuals into the objective function to construct an updated joint optimization model. :

[0225] ;

[0226] in, , It is a regulating factor.

[0227] S183: Use iterative optimization methods to optimize the parameter set. Update the algorithm until the objective function converges. Quasi-Newton methods, gradient descent methods, or trust region methods can be used.

[0228] Set the maximum number of iterations to 100 to 200, when the improvement of the objective function in two consecutive iterations is less than 10. -6 Or the parameter update magnitude is less than 10 -5 When the convergence occurs, it can be considered complete.

[0229] During the iteration process, when the mean square error of the image plane... Root mean square of surface residual At the same time less than the preset threshold (e.g. , When this happens, the iteration can be terminated early.

[0230] S184: The final parameter set is obtained through the above closed-loop optimization. The corresponding surface This is the optimized surface shape.

[0231] Example 2

[0232] See Figure 3 As shown, this embodiment provides a high-reflectivity surface reconstruction system based on multi-view point cloud fusion and optical optimization. The implementation of the high-reflectivity surface reconstruction method based on multi-view point cloud fusion and optical optimization includes:

[0233] Data acquisition module: Utilizes multiple sensors to simultaneously acquire stripe images, polarization images, and confocal cloud data of highly reflective surfaces from multiple perspectives, multiple exposures, and multiple polarization states;

[0234] Data preprocessing module: Performs phase calculation on the stripe image and calculates the degree of polarization and polarization angle of each pixel in combination with the polarization image, thereby calculating the normal and slope of each pixel;

[0235] Reflection path determination module: Based on the calibrated internal and external parameters of the camera and projector, reverse ray tracing is performed on each pixel to generate candidate reflection paths, and the validity of the paths is determined by the consistency score of multi-frame observations to determine the number of reflections of the pixel.

[0236] Path uncertainty calculation module: For each candidate reflection path, the path uncertainty indicator is obtained based on the difference in travel estimation from different frames, and combined with polarization degree, reflection number indicator, incident reflection angle deviation index and saturation / overexposure marker, multiple factor scores are generated for each pixel.

[0237] Credibility weight generation module: Combines multiple factor scores into an initial credibility score, and normalizes the initial credibility score in the local neighborhood of the pixel to form the final credibility weight;

[0238] Point cloud fusion module: After completing the weight calculation, the point cloud generated by the fringe phase is fused with the confocal cloud to obtain a globally consistent point cloud result;

[0239] Surface optimization module: The surface shape is represented by a parametric surface. The parametric surface is optimized under a joint optimization function, and dynamic simulation and visualization output are performed to verify the stability and optical consistency of the surface under multiple postures.

[0240] Iterative update module: The surface obtained by the surface optimization module is used as the new geometric prior. All processes from the reflection path determination module to the surface optimization module are repeatedly executed. After each iteration, the parameter set is updated according to the residual until the convergence condition is met.

[0241] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0242] In conclusion, the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for high-reflective curved surface reconstruction with multi-view point cloud fusion and optical optimization, characterized in that, include: S1: Simultaneously acquire stripe images, polarization images, and confocal clouds of the highly reflective surface using multiple sensors, and calculate the degree of polarization and polarization angle of each pixel by combining the stripe images and polarization images, and calculate the normal and slope of each pixel. S2: Based on the calibrated intrinsic and extrinsic parameters of the camera and projector, reverse ray tracing is performed on each pixel to generate candidate reflection paths, and their validity is determined by the consistency score of multi-frame observations to determine the reflection count indicator of the pixel. S3: For each candidate reflection path, obtain the path uncertainty indicator based on the difference in travel estimation from different frames, and combine it with polarization degree, reflection number indicator, incident reflection angle deviation index and saturation / overexposure marker to generate multiple factor scores for each pixel. S4: Combine multiple factor scores into an initial confidence score, and normalize the initial confidence score within the local neighborhood of the pixel to form the final confidence weight. The calculation of the credibility weight includes: Calculate the signal quality factor based on polarization degree and saturation mask; Set the reflection attenuation factor based on the number of reflections at the measurement point; An angle penalty factor is set based on the angular deviation between the incident direction and the reflection direction relative to the surface normal; The path instability penalty factor is set based on the statistical variance of the path length of multiple reflections; The confidence weights of the point cloud points are formed by weighting each factor and then normalized within the point neighborhood. S5: After completing the weight calculation, the point cloud generated by the fringe phase is fused with the confocal cloud to obtain a globally consistent point cloud result; In the point cloud fusion process, an absolute height constraint is introduced. Points with a signal-to-noise ratio (SNR) not lower than a preset SNR threshold are directly added to the optimization as strong constraint points, while points with an SNR lower than the preset SNR threshold are introduced into the optimization through a penalty term. S6: Based on the fused point cloud, a joint optimization function is introduced to obtain the initial surface shape result, and dynamic simulation and visualization output are performed to verify the stability and optical consistency of the surface under multiple poses. The joint optimization function includes a geometric error term, an optical consistency error term, and a regularization term. The geometric error term is the geometric difference between the surface and the point cloud, the optical consistency error term is the difference between the predicted optical response and the measured optical response, and the regularization term restricts the second curvature of the surface. S7: Repeat S2 to S6 using the surface obtained in S6 as the new geometric prior, and update the parameter set according to the residual after each iteration until the convergence condition is met.

2. The high-reflectivity surface reconstruction method with multi-view point cloud fusion and optical optimization according to claim 1, characterized in that, The construction of the candidate reflection path includes: generating a primary mirror path, a first-order scattering path, and a secondary scattering path for each observed pixel through reverse tracing, and limiting the number of candidate paths to no more than a predetermined upper limit.

3. The method for reconstructing high-reflectivity surfaces through multi-view point cloud fusion and optical optimization according to claim 1, characterized in that, The determination of the number of reflections includes: performing a reprojection consistency score on candidate paths under multi-frame observation; when the consistency score of a candidate reflection path is higher than a predetermined threshold, the path is counted as a valid path, and the number of valid paths is the reflection count indicator.

4. The method for reconstructing high-reflectivity surfaces through multi-view point cloud fusion and optical optimization according to claim 1, characterized in that, Each candidate reflection path is determined by three parts: the projector light source point, the curved surface reflection point, and the camera imaging point. Under single-frame conditions, the travel estimate is calculated based on the three points. Under multi-frame conditions, a set of travel estimates is formed, the mean and standard deviation are calculated, and mapped to a normalized uncertainty score. The weighted average of all valid path uncertainty scores for a pixel is taken as the path uncertainty indicator.

5. The method for reconstructing high-reflectivity surfaces through multi-view point cloud fusion and optical optimization according to claim 1, characterized in that, In dynamic simulation, the predicted image is calculated using a geometric factor and a visibility function. The geometric factor represents the relationship between the surface normal, incident direction, reflection direction, and squared distance attenuation. The visibility function takes a value based on whether the point is occluded under the pose, taking 1 when visible and 0 when invisible.

6. The method for reconstructing high-reflectivity surfaces through multi-view point cloud fusion and optical optimization according to claim 1, characterized in that, During dynamic simulation and closed-loop optimization, geometric layers, optical layers, and confidence layers are generated. The geometric layer represents the distribution of height, slope, and curvature. The optical layer includes the simulation image, the measured image, and the residual map. The confidence layer includes the weight distribution, the visibility distribution, and the saturation mask.

7. A high-reflectivity surface reconstruction system based on multi-view point cloud fusion and optical optimization, implementing the high-reflectivity surface reconstruction method based on multi-view point cloud fusion and optical optimization as described in any one of claims 1-6, characterized in that, include: Data acquisition module: Utilizes multiple sensors to simultaneously acquire stripe images, polarization images, and confocal cloud data of highly reflective surfaces from multiple perspectives, multiple exposures, and multiple polarization states; Data preprocessing module: Performs phase calculation on the stripe image and calculates the degree of polarization and polarization angle of each pixel in combination with the polarization image, thereby calculating the normal and slope of each pixel; Reflection path determination module: Based on the calibrated internal and external parameters of the camera and projector, reverse ray tracing is performed on each pixel to generate candidate reflection paths, and the validity of the paths is determined by the consistency score of multi-frame observations to determine the number of reflections of the pixel. Path uncertainty calculation module: For each candidate reflection path, the path uncertainty indicator is obtained based on the difference in travel estimation from different frames, and combined with polarization degree, reflection number indicator, incident reflection angle deviation index and saturation / overexposure mark, multiple factor scores are generated for each pixel. Credibility weight generation module: Combines multiple factor scores into an initial credibility score, and normalizes the initial credibility score in the local neighborhood of the pixel to form the final credibility weight; Point cloud fusion module: After completing the weight calculation, the point cloud generated by the fringe phase is fused with the confocal cloud to obtain a globally consistent point cloud result; Surface optimization module: The surface shape is represented by a parametric surface. The parametric surface is optimized under a joint optimization function, and dynamic simulation and visualization output are performed to verify the stability and optical consistency of the surface under multiple postures. Iterative update module: The surface obtained by the surface optimization module is used as the new geometric prior. All processes from the reflection path determination module to the surface optimization module are repeatedly executed. After each iteration, the parameter set is updated according to the residual until the convergence condition is met.

Images