A high-density connector three-dimensional online measurement system

Abstract

This invention belongs to the field of precision measurement technology, specifically relating to an online three-dimensional dimension measurement system for high-density connectors. The system includes a model building unit, an image acquisition unit, a depth calculation unit, a normal map generation unit, a feature fusion unit, a virtual rendering unit, an iterative optimization unit, and a measurement output unit. By fusing sparse depth point clouds generated by structured light with high-resolution normal maps generated by photometric stereo, high-precision restoration of pin edges is achieved. A parametric three-dimensional geometric model is used for simulation rendering, and the differences are compared with actual online images. Gradient descent processing logic is used to iteratively update the model's geometric variables, achieving accurate mapping between model parameters and physical entities. This invention, through a model-driven inverse mapping mechanism, filters non-physical noise, overcomes interference from highly reflective environments, and achieves millisecond-level online response while ensuring micron-level measurement accuracy, meeting the requirements of full-volume online inspection.

Description

Technical Field

[0001] This invention belongs to the field of precision measurement technology, specifically relating to an online three-dimensional dimension measurement system for high-density connectors. Background Technology

[0002] With the continuous evolution of the precision electronics manufacturing field, high-density connectors, as core components for signal transmission and electrical connection, have a decisive impact on the operational reliability of the entire system due to the accuracy of their microscopic geometric parameters. On modern automated production lines, real-time online monitoring of critical dimensions such as connector pin coplanarity, pin pitch, and height has become a key technological guarantee for improving yield rates. Such systems typically integrate high-resolution vision sensors and spatial geometry reconstruction algorithms, aiming to achieve high-frequency, non-contact precision measurement of complex microstructures through digital means.

[0003] Online 3D dimension measurement systems primarily rely on optical principles such as structured light scanning or laser triangulation to reconstruct spatial point cloud data by acquiring modulated images of the target object's surface. The core objective of this technology is to transform the physical characteristics of pins into high-precision digital geometric models through high-speed optical acquisition and complex numerical calculations. To meet the requirements of production line cycle times, the measurement system must maintain micron-level spatial resolution while possessing extremely high algorithm robustness and real-time computational performance to ensure the capture of microstructural deformations during high-speed operation.

[0004] Existing technologies face challenges when processing targets with both high reflectivity and dense, fine structures. Due to the significant difference in reflectivity between the metal pins and the insulating base, traditional solutions are prone to localized highlight overflow or dark shadows under single illumination, leading to the loss of crucial geometric feature data. Existing linear processing logic often weakens gradient information at pin edges while eliminating optical noise, causing systematic biases in the extraction of key feature points. The massive raw point clouds generated by high-density connectors pose a problem for the efficiency of voxelization and semantic association in the 3D reconstruction process, making it difficult for the system to achieve a balance between sub-pixel accuracy and millisecond-level online response performance. Summary of the Invention

[0005] The purpose of this invention is to provide an online three-dimensional dimension measurement system for high-density connectors, which can solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A high-density connector 3D dimension online measurement system includes a model building unit, an image acquisition unit, a depth calculation unit, a normal map generation unit, a feature fusion unit, a virtual rendering unit, an iterative optimization unit, and a measurement output unit, as follows: The model building unit is used to build a parametric three-dimensional geometric model based on the preset structural parameters of the high-density connector. The parametric three-dimensional geometric model includes the pin elastic deformation vector and the base positioning reference. The parametric three-dimensional geometric model is configured as a dynamic digital twin with physical properties of material reflectivity and normal vector constraint. The parametric three-dimensional geometric model establishes an initial framework by importing computer-aided design information and presets a range of variable parameters based on the geometric characteristics of the pin and the base.

[0007] The image acquisition unit is used to acquire multimodal image information of the high-density connector. The image acquisition unit is equipped with multiple light sources in different directions and image sensing components. By switching the on state of the light sources within a preset time, it can acquire reflection characteristic images generated under different lighting conditions and acquire modulation stripe images generated by structured light projection.

[0008] The depth calculation unit is used to demodulate the modulated stripe image acquired by the image acquisition unit and obtain sparse depth point cloud data of the high-density connector surface based on the triangulation principle. The sparse depth point cloud data serves as a skeleton constraint describing the macroscopic morphology of the connector.

[0009] The normal map generation unit is used to calculate the reflection characteristic images acquired by the image acquisition unit under different illumination angles based on photometric stereo vision processing logic, and solve the high-resolution normal map of the high-density connector pin surface and base surface. The high-resolution normal map reconstructs the fine geometric features of the pin surface by analyzing the mapping relationship between illumination intensity and surface normal vector.

[0010] The feature fusion unit is used to perform cross-modal fusion of the sparse depth point cloud data provided by the depth calculation unit and the high-resolution normal map provided by the normal map generation unit. The high-resolution normal map is used as texture details to perform edge interpolation on the sparse depth point cloud, restore the geometric features of the pin edge that are obscured by the high reflectivity area, and generate fused point cloud data with high-precision edge definition.

[0011] The virtual rendering unit is used to perform a simulated rendering process based on the parametric three-dimensional geometric model in the model building unit and the preset lighting parameters of the current environment to generate a simulated image corresponding to the actual acquisition scene. The simulated rendering process simulates the physical reflection path of light on the highly reflective material surface of the pin and the diffuse reflective material surface of the base.

[0012] The iterative optimization unit is used to compare the simulated image generated by the virtual rendering unit with the online actual image acquired by the image acquisition unit, calculate the rendering loss between the two, and use gradient descent processing logic to iteratively update the geometric variables in the parameterized three-dimensional geometric model until the rendering loss is reduced to below a preset threshold, thereby achieving accurate mapping between model parameters and physical entities.

[0013] The measurement output unit is used to extract measurement indicators from the parameterized three-dimensional geometric model after optimization iteration. The measurement indicators include pin coplanarity, pin spacing, pin height, and pin bending angle. The measurement indicators are compared with the preset tolerance range to output the dimensional measurement results of the high-density connector.

[0014] Preferably, when constructing the parametric three-dimensional geometric model, the model building unit defines the pins as flexible geometries with specific elastic moduli and the base as a rigid reference plane. By introducing prior physical constraints, the model building unit ensures that the parametric three-dimensional geometric model always conforms to the physical assembly logic of the high-density connector during the iteration process.

[0015] Furthermore, the image acquisition unit controls the brightness level of the light source to keep the image sensing component within a preset linear response range when acquiring high reflectivity pin areas, thus avoiding charge saturation. When acquiring low reflectivity base areas, the exposure gain is increased to ensure that the image data covers the complete dynamic range.

[0016] Furthermore, the depth calculation unit uses a multi-frequency phase-shifting method to perform phase unwrapping on the structured light stripes. The depth calculation unit filters out phase noise caused by abrupt changes in the material of the object surface by calculating the mapping relationship between the phase distribution and spatial coordinates.

[0017] Furthermore, during the solution process, the normal vector generation unit performs least-squares fitting on the grayscale values ​​under multiple light source directions to eliminate some abnormal light intensity data caused by mirror reflection on the metal surface, thereby improving the accuracy of normal vector calculation.

[0018] Furthermore, when performing edge interpolation, the feature fusion unit performs surface completion on the hole region in the sparse depth point cloud according to the surface gradient change trend reflected in the normal map. The feature fusion unit is configured to automatically increase the sampling density of the pin edge region when a sharp deflection of the normal vector is detected.

[0019] Furthermore, when generating simulated images, the virtual rendering unit introduces a bidirectional reflection distribution function model to accurately describe the specular reflection component of the connector pin metal material and the diffuse reflection component of the base insulation material, ensuring that the simulated image has a high degree of consistency with the actual acquired image in terms of light and shadow distribution.

[0020] Furthermore, when calculating the rendering loss, the iterative optimization unit not only compares the pixel-level grayscale differences, but also compares the feature point location distribution and structural similarity index in the simulated image and the actual image. The iterative optimization unit guides the variable search direction of the parameterized model through a multi-scale loss function, avoiding the optimization process from getting stuck in a local optimum.

[0021] Furthermore, the measurement output unit can process massive image feature information in real time. By parsing the optimized parameterized model, it replaces the gridded reconstruction step of the original massive point cloud, controlling the calculation time of a single measurement within a preset time period, thus meeting the real-time requirements of online inspection on the production line.

[0022] Furthermore, the parameterized three-dimensional geometric model also includes thermal expansion compensation coefficients for different ambient temperatures. The model building unit dynamically corrects the linear dimension parameters in the model based on preset ambient temperature monitoring data to eliminate thermal stress errors caused by ambient temperature differences in precision dimension measurements.

[0023] Furthermore, the image sensor in the image acquisition unit is configured to have sub-pixel resolution extraction capability, and the image acquisition unit improves the positioning accuracy of the pin edge position by interpolating the acquired light intensity signal.

[0024] Furthermore, the feature fusion unit employs a multi-scale feature alignment algorithm to uniformly calibrate the spatial coordinate systems from different sensing modalities, ensuring that the depth constraint information and the normal detail information are spatially unbiased and coincident.

[0025] Furthermore, when performing gradient descent processing, the iterative optimization unit presets different step size factors based on the sensitivity of each variable in the parameterized model. For key geometric parameters that have a significant impact on dimensional accuracy, a smaller iteration step size is used to ensure the stability and repeatability of the measurement results.

[0026] Furthermore, the measurement output unit has an automatic alarm function. When the detected pin coplanarity deviation exceeds a preset allowable threshold, the measurement output unit immediately triggers an abnormal signal and records the spatial position index and specific defect feature data of the defective connector.

[0027] Furthermore, the system has a pre-set standardized connector database. The model building unit can automatically call the corresponding parameterized template from the standardized connector database according to the model of the target to be tested, so as to realize rapid adaptation and flexible switching of high-density connectors of various specifications.

[0028] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention introduces a cross-modal feature-driven mechanism to fuse a high-resolution normal map generated by photometric stereo with a sparse depth point cloud generated by structured light. It fully utilizes the sensitivity of the normal map to subtle surface gradients and the macroscopic spatial constraint capability of the depth point cloud, overcoming the measurement blind zone problem that is easily generated by high-density connectors in the high reflectivity environment of metal pins. It achieves super-resolution level edge reconstruction and improves the sharpness and accuracy of pin edge extraction.

[0029] 2. This invention abandons the traditional linear processing logic of denoising first and then reconstructing. By constructing a parameterized three-dimensional geometric model with physical properties and adopting a model-to-image inverse mapping mechanism, the system can self-correct when faced with dust, vibration or slight oil pollution interference in the production site, relying on the geometric constraints and physical prior knowledge of the parameterized model itself. This filters out point cloud noise with non-physical characteristics, improves the measurement repeatability accuracy to the micrometer level, and solves the feature blurring problem caused by excessive denoising in traditional solutions.

[0030] 3. This invention reduces the data load of the algorithm by transforming the 3D reconstruction problem into a model parameter optimization problem. It eliminates the need for voxelization of massive point clouds and complex mesh reconstruction processes. By mapping image features to model parameters, it achieves millisecond-level online detection response speed while ensuring sub-pixel-level measurement accuracy. This meets the cycle time requirements of modern precision manufacturing lines for full-volume online detection of high-density connectors and has high industrial practical value.

[0031] 4. The dynamic digital twin architecture adopted in this invention has good scalability and environmental adaptability. It can dynamically compensate according to the reflectivity distribution of different materials and environmental parameters, ensuring the measurement consistency of the system in different batches of products, different lighting environments, and different operating temperatures, thereby reducing the false detection rate and false negative rate of the system. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the overall technical solution architecture of the present invention; Figure 2 This is a schematic diagram of the core principle framework of the present invention based on parametric 3D geometric model and iterative optimization of rendering loss; Figure 3This is a logical flowchart of the fusion of high-resolution normal map and sparse depth point cloud cross-modal features in this invention. Figure 4 This is a schematic diagram of the multi-level interaction relationship and data flow between the physical acquisition scene, the simulated rendering space, and the parameterized model variables in this invention; Figure 5 This is a flowchart illustrating the logical flow of parameter iterative search and measurement index parsing output based on rendering difference gradients in this invention. Figure 6 This is a schematic diagram comparing the core principle of this invention with existing traditional point cloud reconstruction techniques in terms of noise interference resistance and measurement repeatability accuracy. Detailed Implementation

[0033] Example 1: Please refer to the appendix Figure 1 To be continued Figure 6 A high-density connector three-dimensional online measurement system includes a model building unit, an image acquisition unit, a depth calculation unit, a normal map generation unit, a feature fusion unit, a virtual rendering unit, an iterative optimization unit, and a measurement output unit. The model building unit is used to build a parametric three-dimensional geometric model based on the preset structural parameters of the high-density connector. The parametric three-dimensional geometric model includes the pin elastic deformation vector and the base positioning reference. The parametric three-dimensional geometric model is configured as a dynamic digital twin with physical properties of material reflectivity and normal vector constraint. The parametric three-dimensional geometric model establishes an initial framework by importing computer-aided design information and presets a range of variable parameters based on the geometric characteristics of the pin and the base.

[0034] In its implementation, the model building unit pre-stores computer-aided design (CAD) raw data files for high-density connectors of different specifications and models. These data files define the ideal design state of the connector, including the theoretical coordinate positions of the pins, the cross-sectional geometry, the theoretical pitch between the pins, and the flatness reference of the base. By parsing these design files, the model building unit extracts the topological logic relationship of the high-density connector and transforms it into a parameterized expression that can be driven by the algorithm engine in real time.

[0035] Furthermore, the parametric 3D geometric model in the model building unit is not a simple static mesh set, but a dynamic digital twin entity integrating physical evolution logic. The pin elastic deformation vector is defined as a set of dynamic parameters containing multiple degrees of freedom, used to simulate various physical deformations that may occur in the pin during production, assembly, and transportation. These deformations include, but are not limited to, translational deviations along the coordinate axes, rotational deviations around the pin root, changes in the bending curvature of the pin shaft, and local torsion at the pin end. By setting reasonable physical constraint boundaries for these deformation components, the model building unit ensures that the model remains within the realistically physically achievable range during subsequent iterative optimization, avoiding non-physical distortion results. The base positioning reference is configured as the origin reference of the spatial coordinate system of the entire measurement system. By extracting the feature planes on the base surface as anchor points, translational and rotational errors caused by positional fluctuations during the assembly line transport of the connector under test are eliminated.

[0036] The parametric 3D geometric model also incorporates deeply integrated material reflectivity properties. The model building unit assigns a high-spectral-reflectivity metallic material model to the pin surface and a high-diffuse-reflectivity insulating plastic material model to the base surface. This physical property-based modeling approach enables the model to predict the theoretical brightness distribution under different illumination angles. The normal vector constraint property specifies the strong correlation between the normal direction of each microfacet on the pin surface and its geometric contour. For example, the side normal of a cylindrical pin must be perpendicular to its central axis. This a priori geometric constraint provides a solid theoretical foundation for the subsequent fusion of photometric stereo data.

[0037] The image acquisition unit is used to acquire multimodal image information of the high-density connector. The unit is equipped with multiple light sources in different orientations and image sensing components. By switching the on / off states of the light sources within a preset time, it acquires images of reflection characteristics generated under different lighting conditions and images of modulated stripes generated by structured light projection. As the physical sensing front end of the entire system, the image acquisition unit is deployed above the key inspection station on the production line. To address the tiny and complex geometry of the high-density connector, the image sensing components utilize an industrial area array camera with ultra-high resolution and high dynamic range. Its pixel size is precisely calibrated to ensure that grayscale transitions at the pin edges can be captured at the sub-pixel level.

[0038] In terms of light source configuration, the image acquisition unit includes a multifunctional illumination system that integrates a symmetrically distributed ring array of light sources and a high-frequency structured light projector. The ring array of light sources comprises at least four independently controlled illumination sectors, located around the target under test, providing incident light at different azimuth angles for photometric stereoscopic calculation. During the measurement period, the image acquisition unit, through a hardware synchronization controller, sequentially illuminates the light sources at different azimuth angles within millisecond-level time slices, and drives the industrial camera to synchronously capture images, obtaining a set of reflection characteristic images with different light and shadow distributions under a single viewing angle. These images record the brightness transitions on the pin surface as the incident light direction changes, serving as the raw material for calculating the high-precision normal map.

[0039] The structured light projector in the image acquisition unit is configured to project coded modulated stripes onto the connector surface. These stripes typically employ Gray code combined with multi-frequency phase-shifted sinusoidal stripes. Due to the high reflectivity of the connector pins, the image acquisition unit also integrates adaptive brightness control logic. When acquiring modulated stripe images, the system pre-scans and detects the reflection intensity distribution on the connector surface, dynamically adjusting the projector's luminous intensity and the camera's exposure time. This ensures that the image sensing component remains within a preset linear response range when acquiring images of highly reflective pin areas, preventing overexposure and blurring of the stripes due to charge saturation. When acquiring images of the low-reflectivity black base area, the system uses multi-frame overlay or increases exposure gain to ensure that the image data covers the full dynamic range and achieves clear stripe contrast.

[0040] The depth calculation unit is used to demodulate the modulated stripe image acquired by the image acquisition unit and obtain sparse depth point cloud data of the high-density connector surface based on the triangulation principle. The sparse depth point cloud data serves as a skeleton constraint describing the macroscopic morphology of the connector.

[0041] The depth calculation unit employs a multi-frequency phase-shifting method to perform phase unwrapping of the structured light fringes. Specifically, for the image acquired by the image acquisition unit... The system first calculates the modulated stripe images at different frequencies for each pixel. At various frequencies The phase quality evaluation operator specifically adopts a comprehensive evaluation index based on modulation amplitude and phase derivative variance to automatically filter out phase noise points caused by abrupt changes in the pin surface material.

[0042] The specific calculation logic is as follows: For the... A stripe image with frequencies, which at pixels The light intensity distribution model at that location is: ; in, Indicates the first A frequency stripe image in coordinates grayscale value at that location Indicates background light intensity. Indicates the modulation amplitude. This represents the absolute phase to be solved.

[0043] The encapsulated phase was extracted using a four-step phase-shifting method. Then, combined with the absolute order provided by Gray code Phase expansion is performed using the mapping relationship between phase distributions of different frequencies to calculate the unique absolute phase across the entire field. The formula is: ; in, This represents the absolute phase value after unwrapping. This represents the second-to-last integer obtained by decoding Gray code. Indicates being in The package phase within the interval.

[0044] Furthermore, the phase quality evaluation operator The expression is: ; in, This represents the phase quality evaluation value. and These represent the phase derivatives in the horizontal and vertical directions, respectively. and This represents the size of the local window used for derivative calculation. To prevent extremely small constants with a denominator of zero (such as taking...) ).when When the quality falls below a preset quality threshold, the pixel is marked as a noise point and removed from the depth point cloud, thus generating sparse depth point cloud data. It is passed to the feature fusion unit.

[0045] After receiving multiple phase-shifted fringe images, the depth resolution unit extracts the phase information of each pixel using digital signal processing algorithms. To address the multi-valuedness problem during phase unwrapping, the depth resolution unit employs a multi-frequency phase-shifting method to unfold the structured light fringes. The system uses sinusoidal fringes of different periods for phase reconstruction, calculates the mapping relationship between phase distributions of different frequencies, and combines this with the absolute order information provided by Gray code to achieve a unique solution for the phase across the entire field.

[0046] During the phase-to-depth coordinate conversion process, the depth calculation unit utilizes pre-calibrated geometric parameters between the camera and projector, based on the principle of precision triangulation, to calculate the three-dimensional coordinates of each sampling point on the connector surface. Due to the strong specular reflection characteristics of the metal pins of the high-density connector, stripes often produce significant phase noise or breaks at the pin edges. The depth calculation unit automatically filters out phase noise points caused by abrupt changes in the material of the object's surface by setting a phase quality evaluation operator, generating a set of relatively sparse but highly accurate depth point cloud data. Although this point cloud data may lack subtle features at the pin edges, it accurately describes the connector's macroscopic position in space, the overall height range of the pins, and the undulation trend of the base, providing crucial skeletal constraints for subsequent refined modeling.

[0047] The normal map generation unit is used to calculate the reflection characteristic images acquired by the image acquisition unit under different illumination angles based on photometric stereo vision processing logic, and to solve for the high-resolution normal map of the high-density connector pin surface and base surface. The high-resolution normal map reconstructs the subtle geometric features of the pin surface by analyzing the mapping relationship between illumination intensity and surface normal vector. The normal map generation unit establishes an illumination transmission model to map the pixel brightness value under incident light at different angles to the normal vector direction of the surface at that point.

[0048] During the solution process, the normal map generation unit establishes the following non-Lambert-corrected light transmission model for the connector surface: ; in, Indicates the first Pixels under illumination by a light source The measured grayscale value is obtained directly from the reflectance characteristic image acquired by the image acquisition unit; Indicates surface albedo; Let represent the unit normal vector to be solved; Indicates the first The unit vectors of each light source direction are obtained by pre-calibration of the system; The specular reflection index describes the non-Lambert properties of metals; This indicates the error term introduced by abnormal highlights.

[0049] When collecting Images of light sources at different azimuth angles ( An overdetermined system of equations is constructed. To eliminate anomalous light intensity data points, the normal map generation unit introduces a robust kernel function. Construct the weighted least squares objective function: ; in, The loss function represents the normal estimation. Indicates that for the first The weighting coefficient of each light source.

[0050] When a strong specular highlight is detected on the pin surface due to light from a certain direction, the system dynamically calculates the weight by comparing the difference between the current pixel grayscale and the median grayscale of the local neighborhood. ; ; in, This indicates the preset threshold for detecting highlight anomalies. This indicates that the pixel is in The median grayscale value under each light source. By weighting... Outlier components are removed, and a system of linear equations consisting of the remaining valid equations is solved using Singular Value Decomposition (SVD), outputting a high-resolution normal map. It is passed to the feature fusion unit.

[0051] During the calculation process, the normal map generation unit employs a processing logic combining the Lambert reflection model and non-Lambert correction. Addressing the non-Lambert characteristics of the pin's metal material, the normal map generation unit performs least-squares fitting on the grayscale values ​​from multiple light source directions (e.g., eight azimuth angles) during calculation. By increasing the number of light sources, the system can construct an overdetermined set of equations, eliminating some anomalous light intensity data points caused by mirror reflection from the metal surface. When light from a certain direction illuminates the pin surface, producing strong specular highlights that cause the pixel's brightness value to deviate significantly from the theoretical reflection curve, the normal map generation unit automatically identifies and removes this anomalous component, using brightness data from other directions for robust normal vector estimation. The resulting high-resolution normal map records the microscopic gradient changes on the pin surface with extremely high spatial resolution (equivalent to camera resolution), capturing minute indentations or scratches as small as a few micrometers, as well as precise pointing information of the pin edges and their rounded corners.

[0052] The feature fusion unit is used to perform cross-modal fusion of the sparse depth point cloud data provided by the depth calculation unit and the high-resolution normal map provided by the normal map generation unit. The high-resolution normal map is used as texture details to perform edge interpolation on the sparse depth point cloud, restore the geometric features of the pin edge that are obscured by the high reflectivity area, and generate fused point cloud data with high-precision edge definition.

[0053] The feature fusion unit employs a partial differential equation spatial interpolation algorithm based on Poisson equation constraints. This algorithm integrates sparse depth point clouds. The known depth values ​​in the image are used as Dirichlet boundary conditions to generate a high-resolution normal map. The gradient field in the graph is used as the Neumann boundary condition to construct the Poisson equation and solve for the fused compact depth map. The specific partial differential equation is expressed as follows: ; in, Represents the Laplace operator. This represents the fused depth map to be solved. Represents the high-resolution normal map in pixel coordinates The divergence at a given point can be calculated using the following formula: ; in, and These are the normal vectors in the image coordinate system. shaft and Components in the axial direction.

[0054] When a sharp deflection of the normal vector in the pin edge region is detected (i.e.) Greater than the preset edge threshold The feature fusion unit introduces an adaptive anisotropic diffusion coefficient. Increase the constraint strength in this area: ; in, The diffusion coefficient, representing spatial variation, This represents the hyperparameters for edge enhancement (values ​​range from 5 to 10). A modified Laplace equation is constructed by introducing the Poisson equation, and a fast numerical solution is obtained using the multigrid method. Finally, the geometric features of the pin edge obscured by the highly reflective area are restored, generating fused point cloud data with high-precision edge definition. It is then output to the virtual rendering unit.

[0055] This step is the core of achieving super-resolution edge reconstruction. The feature fusion unit employs a spatial interpolation algorithm based on partial differential equation constraints, using depth information as the absolute reference value for point cloud coordinates and normal information as the constraint condition for surface gradients.

[0056] During edge interpolation, the feature fusion unit performs surface completion on void regions in the sparse depth point cloud based on the surface gradient change trend reflected in the normal map. Since the normal map is extremely sensitive to edges, when a sharp deflection (i.e., abrupt curvature change) of the normal vector is detected in the edge region of the pin, the feature fusion unit is configured to automatically increase the sampling density in that region, using the normal integral to guide the depth value for high-frequency detail recovery. This fusion strategy cleverly compensates for the data loss problem at edges during structured light scanning, reconstructing geometric details obscured by high reflectivity using the sharp gradient information obtained from photometric stereo. The generated fused point cloud possesses both the macroscopic accuracy of structured light and the microscopic detail of photometric stereo, providing extremely high-quality input observations for subsequent model optimization.

[0057] The virtual rendering unit is used to perform a simulated rendering process based on the parametric three-dimensional geometric model in the model building unit and the preset lighting parameters of the current environment, to generate a simulated image corresponding to the actual acquisition scene. The simulated rendering process simulates the physical reflection path of light on the highly reflective material surface of the pin and the diffuse reflective material surface of the base.

[0058] The virtual rendering unit introduces the Cook-Torrance bidirectional reflection distribution function model to accurately describe the reflection characteristics of the connector material. This model decomposes reflection into diffuse and specular components, and its energy transfer process is mathematically expressed as follows: ; in, This represents the value of the bidirectional reflection distribution function. This represents the direction vector of the incident light. This represents the direction vector of the outgoing light (pointing towards the image sensing component). This represents the diffuse reflection coefficient of the base insulation material. Represents the surface normal vector. Represents a half-angle vector With normal vector The angle between them, half-angle vector .

[0059] The specific definitions of each physical sub-model in the formula are as follows: The Fresnel term is used to describe the change in specular reflectivity of the pin metal material under different viewing angles, and is calculated using the Schlick approximation formula. This represents the geometric attenuation term, used to calculate the self-shading and shadowing effects caused by the pin's microsurface structure; The normal distribution function is represented by the GGX distribution, which describes the micro-roughness of the pin surface.

[0060] Based on the aforementioned physical model and the normal distribution of each facet in the parametric 3D geometric model, the virtual rendering unit performs Monte Carlo integration on the light path emitted from the light source, reflected from the connector surface, and finally entering the image sensing component, generating a simulated image. Output to the iterative optimization unit.

[0061] The virtual rendering unit is actually a physical rendering engine running in computer memory, which not only takes into account geometry, but also deeply integrates optical physical properties.

[0062] To achieve high-fidelity alignment between the simulated image and the actual acquired image, the virtual rendering unit incorporates a bidirectional reflectance distribution function (BRDF) model when generating the simulated image. This model accurately describes the specular reflection component of the connector pin's metal material and the diffuse reflection component of the base's insulating material. Based on the normal distribution of each facet in the parametric 3D geometric model, preset light source position information, the spectral distribution characteristics of the light source, and the camera's intrinsic parameter model, the virtual rendering unit calculates the energy transfer process of each ray emitted from the light source, reflected from the connector surface, and finally entering the camera's imaging plane. This physical-level rendering ensures a high degree of consistency between the generated simulated image and the actually acquired online image in terms of light and shadow distribution, highlight area location, and shadow occlusion relationships, cleverly transforming the 3D measurement problem into an image-domain comparison and optimization problem.

[0063] The iterative optimization unit compares the simulated image generated by the virtual rendering unit with the online actual image acquired by the image acquisition unit, calculates the rendering loss between the two, and uses gradient descent processing logic to iteratively update the geometric variables in the parameterized 3D geometric model until the rendering loss is reduced to below a preset threshold, thus achieving accurate mapping between model parameters and physical entities. This iterative optimization unit breaks away from the traditional one-way data processing flow and establishes a closed-loop feedback mechanism.

[0064] The multi-scale loss function evaluation system calculated by the iterative optimization unit consists of a weighted average of grayscale loss, structural similarity loss, and reprojection feature point loss, and its specific formula is as follows: ; in, Indicates the total rendering loss. Represents actual online images, Represents a simulated image. and These represent the sets of edge feature points extracted from the actual image and the simulated image, respectively. , , These are the balancing weight coefficients for the corresponding loss terms.

[0065] Using automatic differentiation techniques, the total loss described above is calculated relative to the geometric variable vector in the parametric 3D geometric model. (including parameters such as pin translation, rotation, and bending curvature) gradient When performing gradient descent, an adaptive step size factor is used to iteratively update the model: ; in, Indicates the first The updated geometric variable vector after the next iteration. Indicates the first The geometric variable vector of the next iteration This represents the global base learning rate. This represents a diagonal matrix composed of the sensitivities of each variable.

[0066] The diagonal matrix elements in The contribution of variables to dimensional accuracy is preset: ; ; in, Indicates the preset minimum iteration step size (e.g.) (A set of key geometric parameters that significantly affect dimensional accuracy) (e.g., pin height parameters); Indicates the preset larger iteration step size (e.g.) (), a set of auxiliary parameters that have a small impact on the final result. (e.g., base background parameters). When the convergence condition is met. When the iteration stops, the model parameters after precise mapping are output to the measurement output unit.

[0067] When calculating the rendering loss, the iterative optimization unit not only compares pixel-level grayscale differences (i.e., mean squared error), but also introduces a perceptual structural similarity index (SSIM) and reprojection error of feature point location distribution. This multi-scale loss function evaluation system can comprehensively capture subtle deviations between simulation and real-world scenes. The iterative optimization unit guides the variable search direction of the parameterized model through the multi-scale loss function and uses automatic differentiation technology to calculate the gradient of the loss function with respect to the model's geometric parameters (such as the 3D offset and bending angle of the pins). To avoid getting trapped in local optima during the optimization process, the iterative optimization unit presets different step size factors based on the sensitivity of each variable in the parameterized model when performing gradient descent. For key geometric parameters that significantly affect dimensional accuracy, such as the coplanarity height of the pins, a smaller iteration step size is used to ensure measurement stability and repeatability; while for auxiliary parameters that have a smaller impact on the final result, a larger initial step size is used to accelerate convergence. This mechanism enables the system to read model parameters from images, enhancing its robustness to environmental noise (such as dust and vibration in the production site), as these random noises are naturally suppressed as random outliers that do not meet geometric constraints during the rendering comparison process.

[0068] The measurement output unit is used to extract measurement indicators from the parameterized 3D geometric model after optimization iteration. These indicators include pin coplanarity, pin spacing, pin height, and pin bending angle. The unit compares these indicators with a preset tolerance range and outputs the dimensional measurement results of the high-density connector. Since the parameterized model has achieved sub-pixel-level precise alignment with the physical entity through iterative optimization, the extraction process of measurement indicators becomes simple and accurate.

[0069] The measurement output unit can process massive image feature information in real time. By parsing the optimized parametric model, it extracts the key attribute values ​​of each pin's geometric entity in the model. This replaces the time-consuming denoising, smoothing, voxelization, and complex mesh reconstruction steps of the original massive point cloud in traditional technologies. This path from image features to model parameters reduces the computational load and controls the total computation time of a single complete measurement within a preset time period (e.g., one hundred milliseconds), meeting the stringent real-time requirements of modern precision manufacturing lines for online full-volume inspection of high-density connectors.

[0070] The measurement output unit has an automatic alarm function. When the detected pin coplanarity deviation, pin pitch deviation, or any critical geometric index exceeds the preset engineering allowable threshold, the measurement output unit immediately triggers an abnormal signal to the production line control system (PLC) and records in detail the unique serial number, spatial location index, and specific defect characteristic data of the non-conforming connector (such as which pin has been bent to what extent).

[0071] To further improve measurement accuracy in complex industrial environments, the parametric three-dimensional geometric model also includes thermal expansion compensation coefficients for different ambient temperatures. The model building unit acquires dynamic temperature difference information in the workshop in real time based on preset ambient temperature monitoring data and dynamically corrects the linear dimensional parameters in the model accordingly. This thermal stress compensation mechanism eliminates the systematic errors in precision dimensional measurements caused by minute material expansion due to ambient temperature differences, ensuring consistent micron-level measurement results across different seasons and operating durations.

[0072] The system has a pre-set standardized connector database. The model building unit can automatically call the corresponding parameterized template from the standardized connector database according to the model of the target to be tested. When the production line undergoes flexible switching or replaces connector products of different specifications, the system only needs to call the new template by scanning a code or other identification methods to complete the initial loading of measurement parameters within seconds, realizing rapid adaptation and flexible online testing of various specifications of high-density connectors.

[0073] Example 2: Based on the high-density connector three-dimensional online measurement system described in Example 1, this example provides a system implementation method based on a distributed edge computing architecture, which aims to further improve the data processing throughput and system response real-time performance on a large-scale parallel production line.

[0074] In this embodiment, the image acquisition unit is divided into multiple parallel front-end acquisition branches, each equipped with an independent industrial camera and multimodal light source array, deployed on different production lines. Each front-end acquisition branch is connected to a nearby edge computing unit via Gigabit Ethernet or a Camera Link bus. The depth calculation unit, normal map generation unit, and feature fusion unit are all deployed within these edge computing units.

[0075] The edge computing unit employs a heterogeneous parallel computing architecture, integrating a high-performance multi-core central processing unit (CPU) and a graphics processing unit (GPU) with thousands of computing cores. Under this architecture, the depth calculation unit utilizes the parallel computing capabilities of the GPU to perform real-time pixel-level phase calculation on the modulated stripe image, compressing the originally time-consuming phase-shifting demodulation process. The normal map generation unit utilizes the high-speed video memory at the edge to store multiple frames of reflectance characteristic images in real time and uses a matrix operation accelerator to complete the least-squares fitting of the photometric stereo.

[0076] More importantly, the model building unit and iterative optimization unit are deployed on a central server cluster, interacting with each edge computing unit via a high-speed local area network. After the edge computing unit completes feature fusion and generates high-precision fused point cloud data, it does not directly upload the full amount of data. Instead, it first performs a lightweight processing based on key feature point extraction, sending only key feature descriptors such as pin edges and key positioning references to the central server.

[0077] The iterative optimization unit in the central server employs an asynchronous optimization mechanism. It maintains multiple parametric 3D geometric model instances of the targets to be measured and, based on feature descriptors input from different edge devices, utilizes a high-computing cluster for high-concurrency virtual rendering and rendering loss calculation. Since virtual rendering is a computationally intensive task, the central server uses a load balancing algorithm to distribute rendering tasks from different connectors to different computing nodes. This cloud-edge collaborative architecture not only alleviates the computational pressure on individual devices but also enables the system to simultaneously monitor measurement data from dozens of production lines.

[0078] In Embodiment 2, the image sensor in the image acquisition unit is further configured to have sub-pixel-level resolution extraction capability. The image acquisition unit improves the positioning accuracy of the pin edges in the digital domain by performing bicubic interpolation on the acquired raw light intensity signal. This processing method, combined with model comparison in the subsequent iterative optimization unit, enables the system to achieve measurement resolution exceeding the physical pixel size even when physical resolution is limited.

[0079] To ensure spatial consistency of multimodal data, the feature fusion unit employs a multi-scale feature alignment algorithm. This algorithm finds a coarse correspondence between the depth point cloud and the normal map at a low-resolution scale. At the original resolution scale, it uses a registration criterion based on mutual information to uniformly calibrate the spatial coordinate systems from different sensing modalities. This ensures that depth constraint information (describing macroscopic undulations) and normal detail information (describing microscopic edges) are spatially unbiasedly coincident, eliminating fusion artifacts caused by installation deviations between the camera, projector, and light source.

[0080] In Embodiment 2, the measurement output unit also integrates a big data analysis module. This module continuously collects all measurement index data uploaded from the production line and uses trend analysis algorithms to predict the wear status of the production equipment. For example, when it is detected that the pin coplanarity of multiple consecutive batches of products is slowly drifting in a certain direction, the measurement output unit will issue an early warning, prompting maintenance personnel to inspect or calibrate the upstream stamping mold. This transforms the invention from a simple measuring tool into an intelligent inspection platform with process quality prediction capabilities.

[0081] Example 3: In some extremely high-density precision connector testing scenarios, the pin spacing may be less than 0.3 mm, and the surface may be coated with an extremely thin layer of anti-oxidation grease, which poses significant challenges to optical measurement. To address these extreme conditions, this example further enhances the hardware and algorithmic aspects of Example 1.

[0082] A polarization control module is introduced into the image acquisition unit. This module incorporates a precisely adjusted polarizer at the front end of each light source and a rotating analyzer driven by a stepper motor at the front end of the industrial camera lens. By controlling the relative polarization angle between the polarizer and the analyzer, the image acquisition unit can selectively filter out specular reflection components from the metal pin surface (this portion of light often carries severe saturation information and does not carry depth information), while retaining diffuse or multiple reflection components that carry the surface roughness and geometric features of the object.

[0083] Accordingly, when performing simulated rendering, the BRDF model of the virtual rendering unit is extended to a physical model that includes polarization components. The virtual rendering unit not only simulates the energy intensity distribution of light but also the evolution of the polarization state of light after reflection from the pin surface. This allows the iterative optimization unit to perform comparisons in a multi-dimensional space including intensity, phase, and polarization when calculating rendering losses. In this way, even if there are interference stripes caused by trace amounts of oil on the pin surface, the system can accurately identify the true geometric contour of the metal surface through differences in polarization information.

[0084] To address the mutual occlusion issue caused by the extremely high pin density, the image acquisition unit in Embodiment 3 employs a multi-camera array architecture. Multiple cameras observe the connector from different oblique overhead angles. The depth calculation unit and normal map generation unit generate local depth maps and normal maps from multiple perspectives, respectively. The feature fusion unit undertakes a more complex spatial stitching task, utilizing the parameterized 3D geometric model as a global logical base map to map local geometric features from different perspectives to the same unified model coordinate system, eliminating blind spots in single-view observation.

[0085] In this embodiment, the iterative optimization unit employs a parameter constraint logic based on Markov random fields (MRF).

[0086] The iterative optimization unit employs parameter-constrained logic based on Markov random fields (MRFs). By adding a structural consistency penalty term to the multi-scale loss function, the optimization problem is transformed into an energy minimization problem. Specifically, the formula for the added MRF energy penalty term is: ; in, This represents the energy of the structural consistency penalty in a Markov random field. This represents the set of all pin geometry variable nodes in a parametric 3D geometric model. Represents a node The set of adjacent pin nodes in the physical topology of a connector. and Representing nodes respectively and nodes Geometric parameters (such as height values). This represents the potential function between adjacent nodes.

[0087] As a preferred approach, the two-node potential function employs a truncated linear penalty model: ; in, This represents the cutoff threshold, used to tolerate normal height tolerance fluctuations between adjacent pins as allowed in the connector design drawings. It is used when abnormal noise causes a drop in the height parameter of a pin. When a non-physical, isolated jump occurs, It will increase sharply and exceed the threshold. At this point, the potential function outputs a constant penalty value. This suppresses the non-physical update direction of the isolated parameter during global gradient backpropagation, ensuring that the spacing error and height consistency between adjacent pins meet the physical assembly logic.

[0088] It treats the geometric variables of each pin as nodes in a random field, leveraging repetitive patterns in connector design (e.g., the theoretical height of pins in the same row should be consistent, and the spacing error between adjacent pins should fluctuate within a very small range) to add a structural consistency penalty term to the loss function. This means that if the iterative optimization algorithm attempts to adjust the height of a pin to an isolated value that is extremely inconsistent with its neighboring pins (usually caused by anomalous noise or occlusion), the MRF constraint mechanism will suppress it, forcing it to satisfy both image features and the overall structural logic. This global optimization strategy further improves the reliability of measurements in extremely complex environments.

[0089] The measurement output unit in Embodiment 3 also supports virtual assembly evaluation. It not only outputs the geometry of the pins, but also uses an optimized parametric 3D model to simulate the insertion and removal process of the connector and the target socket in digital space. By calculating the physical interference of the pins on the insertion and removal path, it predicts the electrical connection reliability of the product.

[0090] Example 4: This example describes an online measurement system architecture with self-learning and self-evolution capabilities, which is a further intelligent upgrade based on the previous examples.

[0091] The model building unit integrates a training module for a Generative Adversarial Network (GAN). During the initial deployment phase of the system, the model building unit not only imports CAD drawings but also performs deep learning on a large number of images of known qualified products and typical defective products (such as broken needles, needle misalignment, and surface damage). In this way, the parametric 3D geometric model learns not only the ideal geometric parameters of the pins but also common nonlinear deformation patterns encountered in actual production. This gives the generated dynamic digital twin greater expressiveness, enabling it to describe extremely complex actual deformations with a very small number of parameters.

[0092] The iterative optimization unit introduces a reinforcement learning-driven search strategy. Traditional gradient descent methods sometimes converge slowly or get trapped in undesirable local optima when dealing with complex connector models with thousands of variables. In Example 4, the iterative optimization unit trains a deep reinforcement learning agent to learn how to dynamically adjust the step size factor and search direction based on the current rendering loss distribution. The agent can identify key deviation regions in the image and prioritize allocating computational resources to optimize the key pin parameters that contribute most to the final accuracy.

[0093] The measurement output unit, while outputting results, stores the successful experiences of each measurement (i.e., the correspondence between images, fused point clouds, and final model parameters) in a local knowledge base. When the system encounters a novel connector structure or unique surface defects, it can automatically update the parametric templates and physical property presets in the model building unit through an online incremental learning mechanism. This self-evolutionary capability allows the measurement system's adaptability to complex environments and measurement accuracy to continuously improve with the increase in the number of test samples during long-term operation, achieving truly intelligent operation without human intervention.

[0094] In Embodiment 4, the image acquisition unit employs an event-based sensing mechanism. By integrating an event camera with a traditional area array camera, the system can capture the relative edge displacement of the pins during high-speed conveyor belt movement with microsecond-level temporal resolution. The depth calculation unit utilizes event stream data for motion deblurring compensation, ensuring that even with continuous high-speed operation of the conveyor belt (rather than traditional stop-and-go detection), clear images of modulation stripes and reflectivity characteristics can still be acquired, thus improving the overall production cycle time (UPH) of the production line.

[0095] The measurement output unit is also equipped with an augmented reality (AR) based interactive interface. When a non-conforming product is detected, the system not only outputs an alarm but also projects the defect location in the 3D model onto the handheld terminal or head-mounted display of the production line maintenance personnel in real time. The maintenance personnel can see the comparison and overlay effect between the parametric model and the extracted defect pin, which facilitates manual review and equipment debugging.

[0096] The technical features in the above embodiments can be combined to adapt to different industrial application needs. For example, the edge computing architecture of Embodiment 2 can be combined with the polarization control module of Embodiment 3 to build a top-level measurement system that has both high throughput and can handle highly reflective grease environments.

[0097] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims. Although the measurement of high-density connectors is described in detail in the above embodiments, the core idea of ​​parametric model reverse rendering comparison used in this system can also be extended to the field of online three-dimensional dimension detection of other electronic components with high reflectivity and complex and fine structures (such as chip packaging leads, precision metal stamping parts, etc.), without departing from the technical spirit of the present invention.

[0098] Those skilled in the art will understand that the units and modules described in this invention can be hardware entities implemented by hardware circuits (such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc.), or software program code stored in a computer-readable storage medium and executed by a processor, or a combination of both. Data flow and instruction interaction between units in the system are accomplished through standard bus protocols or network communication protocols. Their internal logic can be flexibly optimized and adjusted according to the evolution of actual computing hardware, provided that it does not violate the limitations of the claims of this invention.

[0099] All physical parameters mentioned in this invention, such as wavelength, frequency, step size, and threshold, should be experimentally calibrated based on the specific material and size range of the object under test, as well as factors such as ambient light background and vibration frequency of the production line, during actual engineering deployment. These parameters should be set in a preset parameter configuration table within the system and dynamically invoked by the iterative optimization unit and the measurement output unit. During system operation, through continuous closed-loop feedback, these engineering parameters will continuously tend towards optimality to achieve the micrometer-level repeatability measurement accuracy and millisecond-level online response speed pursued by this invention.

Claims

1. A high-density connector three-dimensional dimension on-line measurement system, characterized by, include: The model building unit is used to construct a parametric three-dimensional geometric model with physical properties such as material reflectivity and normal vector constraint based on the preset structural parameters of the high-density connector. The image acquisition unit is used to acquire multimodal image information of the high-density connector. The multimodal image information includes reflection characteristic images generated under different azimuth angle light source switching conditions, and modulation stripe images generated by structured light projection. The depth calculation unit is used to demodulate the modulated stripe image acquired by the image acquisition unit and acquire sparse depth point cloud data describing the macroscopic shape of the connector based on the triangulation principle. The normal map generation unit is used to calculate the reflection characteristic image based on the photometric stereo vision processing logic, and solve the high-resolution normal map of the micro gradient changes of the recording pin surface and the base surface. The feature fusion unit is used to perform cross-modal fusion of the sparse depth point cloud data and the high-resolution normal map to generate fused point cloud data with high-precision edge definition. A virtual rendering unit is used to perform physical simulation rendering based on the parameterized three-dimensional geometric model to generate a simulated image; The iterative optimization unit is used to calculate the rendering loss between the simulated image and the online actual image, and to iteratively update the parameterized three-dimensional geometric model to realize the mapping between model parameters and physical entities; The measurement output unit is used to parse measurement parameters from the parameterized three-dimensional geometric model after the iteration is completed, and output the dimensional measurement results of the high-density connector.

2. The online three-dimensional dimension measurement system for high-density connectors according to claim 1, characterized in that: The parameterized three-dimensional geometric model constructed by the model building unit is configured as a dynamic digital twin integrating physical evolution logic. The dynamic digital twin includes pin elastic deformation vectors and base positioning references. The elastic deformation vector of the pin is defined as a set of dynamic parameters containing multiple degrees of freedom. The set of dynamic parameters includes translational deviation parameters along the coordinate axis, rotational deviation parameters around the root of the pin, bending curvature change parameters of the pin rod, and local torsional parameters at the end of the pin. The model building unit limits the range of geometric variable variation of the model during the iteration process by setting physical constraint boundaries for the dynamic parameter set; The base positioning reference is configured as the origin reference of the spatial coordinate system of the entire system. By extracting the feature plane of the base surface as the anchor point, the position fluctuation error of the connector under test during the pipeline conveying process is eliminated. The parametric three-dimensional geometric model also includes a metal material model with high specular reflection characteristics preset for the pin surface, and an insulating material model with high diffuse reflection characteristics preset for the base surface. In addition, the parameterized three-dimensional geometric model also includes thermal expansion compensation coefficients for different ambient temperatures, and the model building unit dynamically corrects the linear dimension parameters in the model based on preset ambient temperature monitoring data.

3. The online three-dimensional dimension measurement system for high-density connectors according to claim 2, characterized in that: The image acquisition unit includes a multi-functional lighting system, an image sensing component, and a hardware synchronization controller. The multifunctional lighting system integrates a set of symmetrically distributed ring array light sources and a high-frequency structured light projector. The ring array light sources contain at least four independently controlled lighting sectors, which are located around the target to be measured. The hardware synchronization controller is used to sequentially light up illumination sectors in different directions within a millisecond time slice, and drive the image sensing component to synchronously capture images to obtain a reflection characteristic image of the brightness transition law of the recording pin surface as the incident light direction changes. The image acquisition unit also integrates brightness adaptive control logic. When acquiring modulated stripe images, the system detects the reflection intensity distribution on the connector surface through pre-scanning and dynamically adjusts the luminous intensity of the structured light projector and the exposure time of the image sensing component. This ensures that the image sensing component remains within a preset linear response range when acquiring high reflectivity pin areas, and simultaneously increases the exposure gain when acquiring low reflectivity base areas to ensure that the image data covers the complete dynamic range. The image acquisition unit also includes a polarization control module, which includes a polarizer mounted on the front end of the light source and a rotating analyzer mounted on the front end of the image sensing component. By controlling the relative polarization angle between the polarizer and the rotating analyzer, direct specular reflection components from the surface of the metal pins are filtered out.

4. The online three-dimensional dimension measurement system for high-density connectors according to claim 3, characterized in that: The depth calculation unit uses a multi-frequency phase-shifting method to perform phase unfolding on the structured light stripes. The depth calculation unit uses sinusoidal stripes of different periods to reconstruct the phase. By calculating the mapping relationship between phase distributions of different frequencies and combining the absolute order information provided by Gray code, the unique solution of the phase of the entire field is achieved. The depth calculation unit analyzes the continuity and reliability of the phase distribution by setting a phase quality evaluation operator, automatically filters out phase noise points caused by abrupt changes in the pin surface material, and generates a set of sparsely distributed but high absolute position accuracy depth point cloud data. The depth calculation unit calculates the three-dimensional coordinates of each sampling point on the connector surface based on the pre-calibrated geometric parameters between the image sensing component and the structured light projector, using the principle of precision triangulation. The sparse depth point cloud data is used to describe the macroscopic position of the connector in space, the overall height range of the pins, and the undulation trend of the base, serving as the skeleton constraints in the subsequent refined modeling process.

5. The online three-dimensional dimension measurement system for high-density connectors according to claim 4, characterized in that: During the solution process, the normal map generation unit maps the pixel brightness value under incident light at different angles to the normal vector direction of the pixel surface by establishing a light transmission model. The normal map generation unit adopts a processing logic based on a combination of Lambert reflection model and non-Lambert correction. To address the non-Lambert characteristics of the pin metal material, the normal map generation unit constructs an overdetermined set of equations by performing least-squares fitting on the grayscale values ​​under multiple light source directions, thereby eliminating abnormal light intensity data points caused by mirror reflection from the metal surface. When a strong specular highlight is detected on the pin surface caused by light from a certain direction, causing the brightness value of the pixel to deviate from the theoretical reflection curve, the normal map generation unit automatically identifies and removes the abnormal component, and uses the brightness data from other directions to estimate the normal vector. The high-resolution normal map generated by the normal map generation unit records the microscopic gradient changes on the pin surface with a spatial resolution equivalent to the physical resolution of the image sensing component, in order to capture the precise pointing information of the pin edge and its rounded corners.

6. The online three-dimensional dimension measurement system for high-density connectors according to claim 5, characterized in that: The feature fusion unit adopts a spatial interpolation algorithm based on partial differential equation constraints, using depth information as the absolute reference value of point cloud coordinates and normal information as the constraint condition of surface gradient. When performing edge interpolation, the feature fusion unit performs surface completion on the hollow regions in the sparse depth point cloud according to the surface gradient change trend reflected in the normal map. The feature fusion unit is configured to automatically increase the sampling density of the pin edge region when a sharp deflection of the normal vector is detected in the pin edge region, use the normal integral to guide the depth value to restore high-frequency details, restore the geometric features of the pin edge that are obscured by the highly reflective region, and generate fused point cloud data with high-precision edge definition. The feature fusion unit also employs a multi-scale feature alignment algorithm. First, it finds a rough correspondence between the depth point cloud and the normal map at a low-resolution scale. Then, at the original resolution scale, it uses a registration criterion based on mutual information to uniformly calibrate the spatial coordinate systems from different sensing modes, ensuring that the depth constraint information and the normal detail information coincide in spatial position.

7. The online three-dimensional dimension measurement system for high-density connectors according to claim 6, characterized in that: When generating simulated images, the virtual rendering unit introduces a bidirectional reflection distribution function model to accurately describe the specular reflection component of the connector pin metal material and the diffuse reflection component of the base insulation material. The virtual rendering unit calculates the energy transfer process of each ray of light from the light source, through reflection from the connector surface, and finally into the imaging plane of the image sensing component, based on the normal distribution of each facet in the parametric 3D geometric model, the preset light source position information, the spectral distribution characteristics of the light source, and the intrinsic parameter model of the image sensing component. The simulated image generated by the virtual rendering unit is consistent with the actual online image in terms of light and shadow distribution, highlight area location, and shadow occlusion relationship; The bidirectional reflection distribution function model in the virtual rendering unit is further extended to a physical model that includes polarization components. It not only simulates the energy intensity distribution of light, but also the polarization state evolution process of light after reflection on the pin surface. In a multi-dimensional space that includes intensity, phase, and polarization, it provides a basis for comparison for subsequent iterative optimization.

8. The online three-dimensional dimension measurement system for high-density connectors according to claim 7, characterized in that: When calculating the rendering loss, the iterative optimization unit adopts a multi-scale loss function evaluation system. The multi-scale loss function evaluation system not only compares the grayscale difference between the simulated image and the online actual image at the pixel level, but also compares the perceptual layer structural similarity index and the reprojection error of the feature point position distribution between the two. The iterative optimization unit uses automatic differentiation technology to calculate the gradient of the multi-scale loss function with respect to the geometric variables in the parameterized three-dimensional geometric model, and uses gradient descent processing logic to iteratively update the geometric variables; When performing gradient descent, the iterative optimization unit presets different step size factors according to the sensitivity of each variable in the parameterized three-dimensional geometric model. For key geometric parameters that have a large impact on dimensional accuracy, a small iteration step size is used, and for auxiliary parameters that have a small impact on the final result, a large iteration step size is used. The iterative optimization unit also employs parameter constraint logic based on Markov random fields, treating the geometric variables of each pin as nodes in the random field, adding a structural consistency penalty term to the multi-scale loss function, and constraining the spacing error and height consistency between adjacent pins through the repeatability rules in the pin design, thereby suppressing non-physical deformation caused by abnormal noise or occlusion.

9. The online three-dimensional dimension measurement system for high-density connectors according to claim 8, characterized in that: The system adopts a heterogeneous parallel architecture based on distributed edge computing. The image acquisition unit is divided into multiple parallel front-end acquisition branches, and each front-end acquisition branch is connected to the edge computing unit deployed nearby. The depth calculation unit, normal map generation unit, and feature fusion unit are deployed in the edge computing unit, and the image data is processed in real time at the pixel level using the graphics processing unit integrated in the edge computing unit. The model building unit and iterative optimization unit are deployed on the central server cluster and receive lightweight feature descriptors from the edge computing unit through a high-speed local area network. The central server cluster uses an asynchronous optimization mechanism to maintain multiple parametric 3D geometric model instances of the targets to be tested, and uses a load balancing algorithm to distribute the rendering tasks of different connectors to different computing nodes. The measurement output unit can process massive image feature information in real time and extract measurement indicators by directly analyzing the optimized parametric three-dimensional geometric model. The measurement indicators include pin coplanarity, pin spacing, pin height, and pin bending angle. The measurement output unit is also equipped with an augmented reality-based interactive interface. When the detected pin index exceeds the preset tolerance range, an abnormal alarm is triggered and the defect location in the 3D model is projected to the mobile terminal.

10. The online three-dimensional dimension measurement system for high-density connectors according to claim 9, characterized in that: The model building unit integrates a generative adversarial network training module, which learns from images of known qualified and defective products, enabling the parametric 3D geometric model to describe the nonlinear deformation patterns in actual production. The iterative optimization unit introduces a reinforcement learning-driven search strategy, which dynamically adjusts the search direction based on the current rendering loss distribution by training a reinforcement learning agent, and prioritizes scheduling computing resources to optimize the key pin parameters that contribute the most to the final accuracy. The image acquisition unit is also equipped with an event-driven perception mechanism, which captures the relative displacement of the edges of the pins during high-speed movement by integrating an event camera. The depth calculation unit uses event stream data to perform motion deblurring compensation in order to obtain a clear modulation stripe image when the conveyor belt is running at high speed. The system also has a pre-set standardized connector database, and the model building unit automatically calls the corresponding parameterized template from the standardized connector database according to the model of the target to be tested; The measurement output unit also integrates a big data analysis module, which collects historical measurement index data and uses trend analysis algorithms to predict the wear status of upstream production equipment, thereby enabling early warning and feedback on process quality.

Images