A system for detecting the profile of an automobile part

Abstract

The application relates to the technical field of automobile detection, and discloses a system for detecting the outer shape surface profile and line profile of automobile parts, which comprises the following steps: a multi-view camera is used to build a micron-level global absolute coordinate system, so that the cumulative error of robot movement is eliminated; through an elastic dynamics model, the gravity and inertia deformation of an execution terminal are compensated in real time; a composite sensing head is used to collect isomorphic surface profile data of pixel-level light intensity modulation and line profile data based on a phase deflection gradient. A master control center uses a probability field registration and a variation method surface line fusion algorithm to generate a high-fidelity three-dimensional model and perform normal and tangential error decoupling evaluation.

Description

Technical Field

[0001] This invention relates to the field of automotive inspection technology, and more specifically, to a system for inspecting the external surface contours and line contours of automotive parts. Background Technology

[0002] In the field of high-precision manufacturing, especially in the production of automotive body panels and aerospace skins, the accuracy requirements for inspecting the three-dimensional shape and geometric properties of parts are extremely high. Traditional contact coordinate measuring machines (CMMs) are no longer sufficient to meet the needs of online full inspection due to their low efficiency. Currently, the industry generally adopts non-contact optical scanning systems mounted on the end effector of industrial robots as the mainstream solution. These systems utilize the robot's flexible motion capabilities to drive optical probes to perform multi-angle imaging and reconstruction of the workpiece surface, aiming to achieve digital quality control and process feedback in the manufacturing process.

[0003] However, in practical applications, such systems face severe accuracy bottlenecks and data integrity issues. First, from a mechanical execution perspective, industrial robots are essentially multi-jointed cantilever structures with limited rigidity. Under the combined effects of gravity, external loads, and acceleration, the arm and joints inevitably undergo elastic deformation. This deformation leads to a discrepancy between the actual pose of the robot's end effector and the theoretical commands issued by the controller. This discrepancy accumulates non-linearly with the arm span, making it difficult to unify the collected measurement data into an absolute and accurate global coordinate reference, severely impacting the evaluation of the position of components within the vehicle's coordinate system. Second, from an optical imaging perspective, automotive components typically possess complex curvature variations and high dynamic range surface reflectivity (such as high-gloss paint and metallic luster). Single-mode fringe projection technology is prone to overexposure in highly reflective areas, and when dealing with minute edges or rounded corners, the modulation transfer function of the grating fringes often fails to resolve sharp edge information, resulting in blurred key geometric boundaries and an inability to accurately reproduce the microscopic details of the workpiece. Summary of the Invention

[0004] This invention provides a system for detecting the external surface contour and line contour of automotive parts, which solves the technical problems mentioned in the background art.

[0005] This invention provides a system for detecting the surface and line contours of automotive parts, including a global field generator, an execution terminal, a composite sensor head, and a main control center.

[0006] The global field generator is used to construct a global rigid constraint field, which provides an absolute coordinate reference for the execution terminal.

[0007] The execution terminal calculates the end pose error matrix in real time and performs dynamic compensation based on the global rigid constraint field and the preset elastic dynamics model, so as to drive the composite sensor head to reach the predetermined detection pose.

[0008] The composite sensor head collects heterogeneous surface contour data and line contour data based on phase deflection gradient of automotive parts under a predetermined detection pose.

[0009] The main control center receives heterogeneous surface contour data and line contour data, performs spatial alignment using a probability field registration algorithm, and uses a variational surface-line fusion reconstruction algorithm to generate a high-fidelity 3D model from the aligned heterogeneous surface contour data and line contour data. Based on the high-fidelity 3D model, the detection results are output.

[0010] The beneficial effects of this invention include: by introducing a global rigid constraint field and real-time compensation of robot elastic dynamics, the measurement accuracy is decoupled from the robot body accuracy, achieving micron-level absolute positioning without high hardware costs; at the same time, by utilizing multimodal variational fusion of structured light and deflection techniques, high-fidelity reconstruction of micro-edges and high-frequency features is achieved while ensuring the accuracy of macroscopic surface contour dimensions, and the optimization of die stamping and trimming processes can be directly guided by decoupling normal and tangential errors, significantly improving the quality control level of high-end precision manufacturing. Attached Figure Description

[0011] Figure 1 This is a flowchart of the system execution of the present invention. Detailed Implementation

[0012] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0013] This embodiment provides an adaptive detection system for heterogeneous surface contours and high-frequency line contours of automotive parts based on global photogrammetry field constraints.

[0014] The testing system provided in this application can be applied to welding workshops, stamping workshops, or precision metrology laboratories in automobile manufacturing. The specific object being tested can be an automotive body panel with a complex free-form surface, such as a side panel, fender, door panel, or hood. These components typically have surfaces with large curvature (surface profile) and weak stamping edges or radius (r-angle) (line profile), and the surface materials are often a mixture of black electrophoretic paint, galvanized steel sheet, or high-reflectivity aluminum alloy, placing extremely high demands on the dynamic range and feature extraction capabilities of the testing system.

[0015] In one possible implementation, the detection system mainly includes four core subsystems: a global field generator, an execution terminal, a composite sensor head, and a main control center.

[0016] The global field generator is used to construct a global rigid constraint field covering the entire inspection station, providing the execution terminal with an absolute coordinate reference independent of the mechanical body.

[0017] Specifically, the global field generator can include a multi-view photogrammetry camera array (e.g., 4-8 high-resolution industrial cameras) mounted on the top frame of the inspection station. These cameras are arranged in a ring or array, with a field of view covering the entire robot's workspace.

[0018] In addition, several coded markers are affixed around the non-measurement areas of the tooling fixtures and components. These markers serve as spatial anchors for the global rigid constraint field. By identifying these markers, the global field generator uses bundle adjustment to calculate and maintain a micrometer-level global coordinate system in real time, thereby eliminating zero-point drift or accumulated errors caused by long-term robot operation.

[0019] The actuator is the carrier that drives the movement of the sensor. In this embodiment, the actuator can be a highly rigid six-axis or seven-axis industrial robot.

[0020] It should be noted that although industrial robots have high repeatability, their absolute positioning accuracy is often affected by the weight of the arm, joint flexibility, and acceleration / deceleration inertia, resulting in an error of 0.1mm to 0.5mm. To overcome this deficiency, this embodiment pre-defines an elastic dynamics model in the controller of the execution terminal. This model can calculate the elastic deformation displacement of the end effector in real time based on the robot's current joint angles, link stiffness, and end effector load (i.e., the weight of the composite sensor head), and perform feedforward compensation, thereby ensuring that the sensor can reach the predetermined detection pose with extremely high attitude accuracy.

[0021] The composite sensor head is mounted on the flange end of the actuator. To simultaneously satisfy the overall measurement of the surface profile and the feature extraction of the line profile, the composite sensor head adopts a multimodal heterogeneous design.

[0022] The integrated composite sensor head includes:

[0023] Multi-frequency phase-shift structured light projector: Used to project coded sinusoidal stripe light onto the object under test. By changing the intensity distribution of the projected light, this projector can adapt to surfaces with different reflectivities and acquire high dynamic range (HDR) surface profile depth data.

[0024] Telecentric LCD deflection screen: Used to display high-contrast orthogonal sinusoidal stripes (such as alternating black and white stripes). Unlike structured light, it utilizes the principle of specular reflection to allow the camera to capture a mirror image of the stripes reflected from the object's surface. This module is extremely sensitive to changes in surface gradients and is specifically designed to capture subtle feature lines (such as slip lines and impact lines).

[0025] Binocular high-resolution camera: As a shared visual acquisition unit, it acquires image data in both structured light and deflection modes. The binocular design helps to provide more robust stereoscopic vision constraints.

[0026] The main control center connects to the aforementioned components via a high-speed communication bus (such as EtherCAT or GigEVision). The main control center can be a workstation containing a high-performance GPU or an edge computing cluster. Its main functional modules include:

[0027] Motion control module: Receives feedback from the global field, solves the dynamic model, and sends the corrected motion commands to the execution terminal.

[0028] Acquisition synchronization module: precisely controls the microsecond-level synchronous triggering of the projector and camera in the composite sensor head.

[0029] Heterogeneous Algorithm Engine: Runs probability field registration algorithm and variational surface-line fusion reconstruction algorithm to fuse the collected heterogeneous data (surface data + line gradient) to generate a high-fidelity 3D model, and outputs the final detection report based on RPS benchmark.

[0030] Through the coordinated operation of the aforementioned system hardware architecture, the embodiments of this application can achieve full-size, high-precision, adaptive inspection of complex automotive parts without sacrificing cycle time.

[0031] This embodiment provides a method for detecting the surface and line contours of automotive parts. This method operates based on the aforementioned system hardware architecture.

[0032] The detection method mainly includes the following steps S201 to S204:

[0033] S201: Construct a global rigid constraint field and establish an absolute coordinate reference.

[0034] At the start of the inspection process (e.g., during the power-on or reset phase of the production line station), the system first performs an initialization operation. The global field generator (multi-view camera group) performs a full-field scan of the station space to identify the coded marker points around the tooling and parts.

[0035] In step S201, the system does not rely on the robot's joint encoders to determine its spatial position. Instead, it uses bundle adjustment to calculate the three-dimensional coordinates of all marker points in the same world coordinate system, thereby constructing a global rigid constraint field. This constraint field serves as a unified interface for all subsequent measurement data, ensuring that even if the robot base experiences minor settlement or thermal deformation, the measurement reference remains accurate and reliable.

[0036] S202: Based on the elastic dynamics model, the execution terminal is driven to perform dynamic pose compensation.

[0037] After establishing the baseline, the main control center plans the motion path of the execution terminal (robot) based on the CAD model of the component to be tested. During the robot's movement, or during the deceleration phase before reaching the predetermined detection pose, the system executes the following sub-steps in real time:

[0038] Pose tracking: The global field generator captures the target installed at the end of the robot in real time to obtain the actual physical pose of the end.

[0039] Error calculation: The nominal pose fed back by the robot controller is compared with the actual physical pose, and the elastic deformation component under the current posture is calculated by combining the preset elastic dynamics model (which takes into account gravity, load and joint stiffness).

[0040] Closed-loop compensation: The elastic deformation component is converted into a compensation command and superimposed on the robot's motion control loop to drive the composite sensor head to accurately reach the predetermined shooting position and wait for the shaking to stabilize.

[0041] S203: Under a predetermined detection pose, heterogeneous multimodal data are acquired in parallel or serially.

[0042] When the composite sensor head is stably hovering above the surface being measured, the system enters the sensing phase. To accommodate both macroscopic surface features and microscopic characteristics, step S203 employs a heterogeneous data acquisition strategy:

[0043] On one hand, heterogeneous surface contour data is acquired: the structured light projector in the composite sensor head projects multi-frequency phase-shifted fringes onto the surface of the component. For common black paint or highly reflective areas on the surface of automotive components, the system automatically adjusts the projected light intensity (i.e., adopts an HDR strategy) to ensure that the fringe images captured by the camera are neither overexposed nor underexposed. By demodulating these fringe images, surface contour data containing absolute depth information (DepthMap) of the object's surface is obtained.

[0044] On the other hand, line profile data based on phase deflection gradient is acquired: at this time, the projector is turned off, and the LCD deflection screen in the composite sensor head is turned on, displaying high-contrast sinusoidal stripes. The camera captures the specular reflection image of the screen stripes on the surface of the component. Since specular reflection is extremely sensitive to changes in angle, this data can capture the tiny deflection of the surface normal vector (i.e., gradient information), thus presenting high-frequency features such as waistlines and edges with high clarity.

[0045] S204: High-dimensional fusion and intelligent evaluation of heterogeneous data.

[0046] The control center receives two types of data with drastically different attributes (one is depth-based and low-frequency; the other is normal gradient-based and high-frequency), and performs the following processing in sequence:

[0047] Probabilistic field registration (spatial alignment): Due to slight time differences or vibrations between the two acquisitions, and the different data types, direct stitching will produce ghosting. Step S204 uses a probabilistic field registration algorithm (instead of the traditional point-to-point ICP) to calculate the maximum overlap in statistical distribution between the surface data and the gradient data, achieving precise spatial alignment between the two.

[0048] Variational fusion (data reconstruction): By solving partial differential equations, the low-frequency accuracy of surface data is fused with the high-frequency sharpness of line data to generate a high-fidelity 3D model that is noise-free and has extremely clear edges.

[0049] Intelligent Evaluation: Finally, the high-fidelity model is compared with the theoretical CAD model. Unlike traditional single color difference maps, the system uses RPS (Reference Point System) to align model constraints and decouples the calculated deviation vector into normal deviation (corresponding to surface profile) and tangential deviation (corresponding to line profile), ultimately outputting an inspection report.

[0050] This embodiment provides a data processing and evaluation method based on heterogeneous optics. The method relies on the system hardware architecture described in Embodiment 1 and is executed after the execution terminal drives the composite sensor head to the predetermined detection pose. It mainly includes four core steps: adaptive acquisition, probability field registration, variational fusion, and decoupled evaluation.

[0051] First, the system performs adaptive heterogeneous data acquisition based on surface reflectivity. Addressing the complex optical characteristics of automotive parts, such as the coexistence of black and white or high reflectivity and diffuse reflection, a step-by-step strategy is employed: First, heterogeneous surface contour data (HDR structured light mode) is acquired. The system controls a multi-frequency phase-shift projector to project a pre-coded pattern. A binocular camera calculates the surface reflectivity distribution within the current field of view and automatically calculates the optimal projection intensity for each pixel based on the sensor response range and incident angle. Subsequently, the projector generates an adjusted sinusoidal fringe sequence based on this optimal intensity. The camera acquires multi-step phase-shift light intensity images and demodulates the phase map. This pixel-level light energy modulation ensures sufficient signal-to-noise ratio in the black electrophoretic paint area and prevents overexposure in the aluminum alloy area. Next, line contour data (phase deflection mode) is acquired. At this time, the projector is turned off, and a telecentric LCD deflection screen displays orthogonal sinusoidal fringes. The camera acquires the fringe mirror image reflected from the part surface. Since minute changes in surface curvature can cause significant deflection of reflected light paths, the system calculates the phase difference in the horizontal and vertical directions and solves for a high-precision surface gradient field based on the physical stripe pitch and screen distance. It then calculates the second derivative of the gradient to generate a characteristic line response function, which serves as the basis for the line profile data.

[0052] Secondly, probabilistic field registration based on a Gaussian mixture model is performed. Given that step S301 acquired two sets of heterogeneous data—surface data with absolute coordinate accuracy and line data with extremely high relative shape accuracy—and that the gradient data, as discrete normal vectors, is difficult to correlate one-to-one with the point cloud, this system abandons the traditional ICP algorithm and instead adopts a probabilistic field registration algorithm. The system treats the surface contour point cloud as the reference target and the local shape after integrating the gradient data as the source data, constructing a Gaussian mixture model (GMM). The log-likelihood function is used to characterize the overlap of the two sets of data in statistical distribution. By maximizing this likelihood function to solve for the rigid body transformation matrix, and by using an annealing strategy to gradually reduce the correlation bandwidth parameter during the iteration process, the gradient data can be attached to the skeleton of the surface contour point cloud, achieving sub-pixel-level spatial alignment.

[0053] Next, a variational method-based surface-line fusion reconstruction is performed. This step aims to resolve the mathematical contradiction between the low-frequency accuracy but high-frequency noise of structured light data and the high-frequency sharpness but low-frequency deformation of deflection data. The system constructs a minimization functional that includes a data fidelity term (constraining the reconstructed surface to not deviate from the structured light base depth), a gradient fidelity term (constraining the normal vector field to be consistent with the deflection gradient field), and a smoothing term (suppressing noise). Subsequently, the corresponding Euler-Lagrange partial differential equation of this functional is derived and solved using numerical methods. The resulting high-fidelity 3D model not only retains the correct macroscopic dimensions of the body panels but also perfectly reproduces the sharp stamped edges and R-angle details.

[0054] Finally, decoupling evaluation is performed based on feature topology and RPS constraints. After obtaining the high-fidelity model, the system calculates the principal curvature distribution of the model, extracts ridge points where the derivative of the principal curvature is zero, and fits them as NURBS curves. Next, the physical fixture clamping logic is simulated, identifying the RPS positioning reference points on the parts, and aligning the high-fidelity model with the CAD theoretical model using constraints. Instead of outputting a single color difference map, the system outputs decoupling errors: calculating the normal deviation to reflect the unevenness, springback, or torsion of the part surface, mainly guiding the surface compensation of stamping dies; and calculating the tangential deviation to reflect the positional offset of the feature lines on the surface, mainly guiding the adjustment of the trimming die cutting edge.

[0055] It should be noted that regarding alternative configurations for the end effector and the global field system, the end effector is not limited to a seven-axis industrial robot; it can also be replaced by a high-precision gantry robot or the detection mechanism of a numerical measuring machine (CMM). When using a high-rigidity CMM, its end effector elastic deformation is extremely small, so the real-time elastic dynamics compensation module can be simplified or turned off, performing only static temperature error compensation, but the global field generator is still needed to provide absolute coordinate calibration. Similarly, the global field generator can be replaced by a laser tracker system or a multi-point laser scanner array, as long as the alternative system can track the end effector target pose in real time and provide micron-level absolute coordinates independent of the robot's encoder, thus meeting the requirements for constructing a global rigid constraint field.

[0056] It should be noted that regarding the simplification of the composite sensor head and data acquisition mode, the composite sensor head can be decomposed into two independent sub-sensor heads: a dedicated multi-frequency phase-shift structured light measurement head and an independent phase-deflection grating measurement head. These two can be sequentially switched by the robot or driven in parallel by two independent execution terminals. The core is that the main control center can receive and fuse these two heterogeneous data sets. In specific applications (such as focusing only on the body-in-white skeleton without highly reflective edges), the system can simplify the acquisition mode by only enabling the structured light projector to acquire heterogeneous surface contour data while closing the deflection grating. In this case, the subsequent fusion algorithm is downgraded to a standard point cloud reconstruction algorithm, but high-precision stitching is still achieved using global field constraints.

[0057] It should be noted that the probability field registration algorithm based on Gaussian mixture models can be replaced by probability density estimation (PDE) registration algorithms or the standard point-to-surface iterative nearest point (ICP) algorithm, especially when the data noise is low or the heterogeneity is small, to achieve higher computational efficiency. Variational surface-line fusion reconstruction algorithms can also be implemented using multi-scale filtering algorithms or Kalman filtering, for example, using Kalman filtering to use high-frequency gradient data as real-time correction for low-frequency depth data predictions. Furthermore, the decoupled evaluation method for normal and tangential bias can be replaced by traditional full-field chromatic aberration visualization, or simply calculating the geometric distance between the feature lines extracted based on the principal curvature and the CAD model.

[0058] like Figure 1 As shown, a system for detecting the surface and line contours of automotive parts includes a global field generator, an execution terminal, a composite sensor head, and a main control center.

[0059] The global field generator is used to construct a global rigid constraint field, which provides an absolute coordinate reference for the execution terminal.

[0060] The execution terminal calculates the end pose error matrix in real time and performs dynamic compensation based on the global rigid constraint field and the preset elastic dynamics model, so as to drive the composite sensor head to reach the predetermined detection pose.

[0061] The composite sensor head collects heterogeneous surface contour data and line contour data based on phase deflection gradient of automotive parts under a predetermined detection pose.

[0062] The main control center receives heterogeneous surface contour data and line contour data, performs spatial alignment using a probability field registration algorithm, and uses a variational surface-line fusion reconstruction algorithm to generate a high-fidelity 3D model from the aligned heterogeneous surface contour data and line contour data. Based on the high-fidelity 3D model, the detection results are output.

[0063] In one embodiment of the present invention, constructing a global rigid constraint field specifically includes:

[0064] Minimize the reprojection error function using bundle adjustment. The reprojection error function is defined as follows:

[0065] ;

[0066] Where n is the number of cameras and m is the number of coded markers. This represents the observed pixel coordinates of the j-th coded marker point in the i-th image. This represents the camera projection function, which includes distortion parameters; K represents the camera intrinsic parameter matrix. Let i represent the rotation and translation matrices of the i-th camera. This represents the three-dimensional spatial coordinates of the j-th encoding marker point to be optimized. This represents the observation weights based on the confidence level of feature point recognition;

[0067] By minimizing the Joint solution and This establishes the global rigid constraint field.

[0068] In this embodiment, the design principle for constructing a global rigid constraint field lies in external benchmarking, aiming to solve the problem of insufficient absolute positioning accuracy (typically exceeding 0.5 mm) in traditional industrial robots due to wear of joint reducers and cumulative errors in the arm. The system does not rely on feedback from the robot's own joint encoders, but instead constructs an absolute coordinate system independent of the robot arm's thermal deformation and mechanical wear through a multi-view photogrammetry camera array positioned at the top of the inspection station. In specific operation, the global field generator first performs multi-view imaging of the coded marker points distributed around the tooling and components. The image processing unit performs Gaussian smoothing and binarization on the acquired images, and uses a sub-pixel edge detection algorithm to extract the center coordinates of the coded marker points; these coordinates are the observed pixel coordinates. Its extraction accuracy can typically reach the 0.02 pixel level. To establish a high-precision mathematical model, the system uses bundle adjustment to construct a reprojection error minimization function. The key parameter in the minimization function, the camera intrinsic matrix K, is a preset parameter obtained offline using the Zhang Zhengyou calibration method by photographing a high-precision ceramic checkerboard calibration board before the system leaves the factory. It includes focal length, principal point coordinates, and radial and tangential distortion coefficients. The parameter observation weights in the minimization function... This is not a fixed value, but rather dynamically calculated based on the confidence level of feature point recognition. Specifically, The value of the error function is determined by three factors: the contrast between the marker point and the background, the cosine of the angle between the camera's optical axis and the marker point's normal, and the pixel area occupied by the marker point in the image. When the marker point has strong reflection, the shooting angle is too large, or the imaging area is too small, the system will automatically reduce the weight of that point to suppress the interference of low-quality data on global optimization. Finally, the error function is solved iteratively using the nonlinear least squares method to jointly optimize the pose (extrinsic parameter) of all cameras and the three-dimensional coordinates of spatial points. When the root mean square error converges to a preset threshold (e.g., 0.3 pixels), the micrometer-scale global rigid constraint field is considered to be completed.

[0069] In one embodiment of the present invention, based on a global rigid constraint field and a preset elastic dynamics model, the end-effector pose error matrix is ​​calculated in real time and dynamic compensation is performed. Specifically, this includes: calculating the elastic deformation displacement of the end-effector caused by the force based on the stiffness model. To simultaneously characterize the contributions of joint flexibility and link flexibility to end-effector displacement, an equivalent stiffness matrix in joint space is constructed. :

[0070]

[0071] in, For the Jacobian matrix of the robot, Here is the joint stiffness matrix. Here is the stiffness matrix of the link; This is a link flexibility mapping matrix determined by the robot's structure and kinematics. It is used to correlate the link's elastic deformation degrees of freedom (or modal coordinates) with the small joint displacements, thereby equivaling the link's bending / torsion and other flexibility to the end-effector compensation calculation.

[0072] Under this equivalent stiffness, the generalized force formed by the external load and gravity at the end of the joint space is:

[0073]

[0074] in, External load vector, The mass distribution of the robot's links and joint angles The generalized force vector of gravity is derived. From this, the minute elastic deformation of the joint is obtained. and end-Cartesian space elastic deformation displacement :

[0075]

[0076] The elastic deformation displacement Error spinor transformed into Lie algebra space And use exponential mapping to calculate the corrected end pose. :

[0077]

[0078] in, The nominal pose matrix fed back by the robot controller; Let be the exponential mapping function from Lie algebras to Lie groups; the system is based on the stated... Drive the composite sensor head to the predetermined detection position.

[0079] In this embodiment, the design principle of real-time elastic dynamics compensation for robot end-effector pose is based on physical model feedforward control. This aims to eliminate static gravity sagging and dynamic inertial jitter caused by the flexibility of the arm during high-speed movement of the robot when carrying a heavy composite sensor head (typically weighing over 15 kg). The system considers the industrial robot not to be an absolutely rigid body, but rather a series of flexible links connected by a spring-damped system. Therefore, by pre-calculating the physical deformation and applying reverse compensation commands, it can ensure that the actual position reached by the end-effector coincides with the theoretical position. The establishment of this compensation model relies on precise stiffness parameters, including the joint stiffness matrix. The identification is derived from load-bearing experiments, specifically by applying a known load to the end effector and measuring displacement under locked joint conditions, followed by inversion using Hooke's Law matrix form. During real-time compensation calculations, the system needs to solve for the gravity vector in real time. With external load The gravity vector is derived in real time based on the robot's current joint angles, combined with the known mass distribution and center-of-mass Jacobian matrix of each link. The external load vector includes not only the static gravity of the sensor head but also the inertial force generated by the robot's acceleration and deceleration, calculated as the product of the robot's inertia matrix and the joint angular acceleration. Based on these parameters, the system uses the stiffness equation to calculate the elastic deformation displacement of the end effector in Cartesian space. To ensure the orthogonality of the rotation matrix and avoid gimbal lock, this system introduces Lie algebra theory to account for small deformation displacements. Error spinor transformed into Lie algebra space Finally, the error spinor is converted into a correction matrix using an exponential mapping function and superimposed on the nominal pose command issued by the robot controller, thereby achieving high-frequency, high-precision dynamic compensation for the robot's end-effector position and attitude.

[0080] Specifically, The equivalent linear elastic stiffness of each link along the translational and rotational directions in its local coordinate system is used to characterize the stiffness of the link. Its value is obtained by FEA under given material parameters, boundary constraints, and load conditions. During real-time compensation calculations, the stiffness is determined by the aforementioned... Mapping the linkage flexibility equivalently to the joint space (or end-effector space), and... Together they constitute the equivalent stiffness, from which the elastic deformation displacement at the end can be calculated. This is to compensate for the link bending / torsional load response that cannot be covered when only joint flexibility is considered.

[0081] In one embodiment of the present invention, under a predetermined detection pose, heterogeneous surface contour data and line contour data based on phase deflection gradient of automotive parts are collected respectively, specifically including:

[0082] For heterogeneous surface contour data acquisition, pixel calculation Optimal projection intensity at the location :

[0083] ;

[0084] in For ideal grayscale response values, For surface reflectivity, The projection angle;

[0085] based on Project N-step phase-shifted fringes and collect light intensity I. k And demodulate the phase :

[0086] ;

[0087] For line profile data acquisition based on phase deflection gradient, the surface gradient is calculated. :

[0088] ;

[0089] in Let P be the phase difference, P be the screen stripe pitch, and d be the measurement distance; then calculate the characteristic line response function. :

[0090] ;

[0091] in The second derivative of the surface. It is a noise suppression factor.

[0092] In this embodiment, a multimodal perception strategy is employed to address the contradiction that a single sensor cannot simultaneously handle high dynamic range (HDR) surface imaging and micron-level feature line capture of vehicle body panels. When acquiring heterogeneous surface contour data, the system first employs adaptive fringe projection technology. Specifically, the composite sensor head first projects a uniform low-frequency pre-coded pattern onto the object being measured. After acquisition by the camera, the base brightness of each pixel is calculated, thereby obtaining the surface reflectivity distribution R(u,v). The ideal grayscale response value of the camera sensor in the formula... This is a preset parameter, typically set to 80% of the sensor's saturated grayscale value (e.g., 200 for an 8-bit depth camera), designed to ensure the highest signal-to-noise ratio without overexposure; while the angle between the projected light and the surface normal... This is calculated based on the coarse point cloud normals obtained from a pre-scan. The system uses these parameters to calculate the optimal projection intensity at the pixel level. The optimal projection intensity characterization artificially flattens the dynamic range of the scene's illumination by reducing the projection brightness in highly reflective areas and increasing the projection brightness in darkly absorbing areas, ensuring that the subsequently acquired sinusoidal fringe sequence is in the optimal light-sensitive fringe zone across the entire CCD target surface. When acquiring line profile data based on phase deflection gradients, the system utilizes the principle of optical differentiation, leveraging the extreme sensitivity of LCD light screen reflection imaging to changes in surface angle. In this step, the physical fringe pitch P involved in the formula is an inherent hardware parameter of the LCD display (e.g., 0.2 mm), and the distance d from the screen to the object being measured is a known quantity obtained through hand-eye calibration during the system calibration phase. The system calculates the phase difference of the reflected fringes... The surface gradient is derived, which is essentially the projection component of the surface normal vector onto the imaging plane. To accurately extract feature lines that are imperceptible to the human eye (such as stamping edges), the system introduces a feature line response function. The characteristic line response function is designed based on differential geometry theory, that is, the characteristic line is mathematically represented as the local maximum point of curvature. Therefore, by calculating the derivative of the gradient (i.e. the second derivative of the surface) and subtracting the Gaussian noise term, the smooth surface background can be filtered out, and the weak edge signal can be highlighted, thereby obtaining line contour data containing only high-frequency geometric information.

[0093] In one embodiment of the present invention, receiving heterogeneous surface contour data and line contour data, and performing spatial alignment using a probability field registration algorithm, specifically includes:

[0094] The heterogeneous surface contour data is defined as a point set P. i The local shape of the line contour data is defined as q. j ;

[0095] Construct and maximize the log-likelihood function to solve for the optimal rigid body transformation matrix R,t:

[0096] ;

[0097] in, These represent the number of points in the two sets of data, respectively. The associated bandwidth parameter gradually decreases during the iteration process, R is the rotation matrix, and t is the translation vector.

[0098] In this embodiment, since structured light acquires dense 3D point clouds (surface data) while deflection acquires sparse and high-precision normal gradient fields (line data), their properties are different, making the traditional nearest-point iteration (ICP) algorithm prone to getting trapped in local optima. Therefore, this system employs a probability field registration algorithm, transforming the data alignment problem into a maximum likelihood estimation problem of a statistical distribution. Specifically, the system does not force a one-to-one correspondence between points, but treats the heterogeneous surface contour data as a set of observation samples with a Gaussian probability distribution, constructing the target probability field from the gradient data. When constructing the log-likelihood function, the correlation bandwidth parameter in the formula... It plays a regulating role. (Related bandwidth parameters) It is not fixed, but rather dynamically adjusted according to the annealing strategy. In the early stages of iteration, Choosing a larger preset value (e.g., 2.0 mm) allows for greater spatial tolerance, aiming to quickly establish a macroscopic overlap between two geographically distant datasets and prevent the algorithm from being biased by local noise; as the number of iterations increases, Gradually shrinking to the micrometer level (e.g., 0.05 millimeters) means tightening the peaks of the probability distribution, forcing the data to mesh finely at the micrometer level. This design simulates the annealing process, ensuring high-precision and tight alignment of heterogeneous data in the global coordinate system even in the absence of explicit feature correspondences.

[0099] In one embodiment of the present invention, a variational surface-line fusion reconstruction algorithm is used to uniformly generate a high-fidelity 3D model from aligned heterogeneous surface contour data and line contour data, specifically including:

[0100] Construct and minimize the energy functional J(Z), which is defined as:

[0101] ;

[0102] in, For the height field of the high-fidelity 3D model to be reconstructed, The base depth provided for the heterogeneous surface contour data The gradient vector field provided for the line profile data based on phase deflection gradient, where α, β, and γ are the weights of the data fidelity term, gradient fidelity term, and smoothing regularization term, respectively, is used to derive the Euler-Lagrange equation corresponding to the functional through variational principles. This equation takes the form of a Poisson equation.

[0103] ;

[0104] The equations are solved using numerical methods to obtain the fused high-fidelity 3D model Z.

[0105] In this embodiment, based on the energy minimization theory of the Variational Method, the aim is to solve the technical problem of seamlessly integrating low-precision surface depth information with high-precision line gradient information. The system constructs an energy functional J(Z) with three constraints. It seeks an optimal surface Z such that, while maintaining its overall shape without deviating from the structured light measurement results, its local surface normal variation rate approximates the gradient field measured by deflection as closely as possible. The three weight parameters α, β, and γ in the functional are empirical hyperparameters preset based on the material properties and processing technology of the components. Specifically, the data fidelity weight α is used to anchor the absolute dimensional accuracy of the model, preventing low-frequency drift during integration; it is typically set to a large value. The gradient fidelity weight β is used to adjust the sharpness of feature lines; the higher its value, the closer the reconstructed edges are to the micrometer-level details obtained by deflection. The smoothing weight γ is used for regularization constraints, suppressing high-frequency noise by penalizing the second derivative of the surface. During the solution process, the system utilizes the variational principle to transform the functional extremum problem in integral form into solving Euler-Lagrange partial differential equations in the form of the Poisson equation. Specifically, the extremum points of the functional must satisfy the condition that its variation is zero, thus transforming the complex optimization search problem into a mature problem of solving sparse linear equations, ultimately generating a high-fidelity 3D model that combines macroscopic dimensional accuracy with microscopic feature sharpness.

[0106] In one embodiment of the present invention, the output of detection results based on a high-fidelity 3D model specifically includes:

[0107] Calculate the principal curvatures k1, k2 and principal directions t1, t2 of the model, and extract feature line points that satisfy the following conditions:

[0108] ;

[0109] The extracted point set is parametrically fitted using the NURBS formula:

[0110] ;

[0111] In the RPS positioning reference point set At this point, the alignment transformation R,t is solved by minimizing the sum of squared normal distances:

[0112] ;

[0113] Where, n k The theoretical normal vector at the reference point restricts specific degrees of freedom.

[0114] Calculate measurement points CAD Theoretical Points The deviation between them is decoupled as follows:

[0115] Normal deviation (stamping depth error):

[0116] ;

[0117] Tangential deviation (trimming position error):

[0118] .

[0119] In this embodiment, the detection logic of physical inspection fixtures in the automotive manufacturing industry is simulated to achieve process traceability of measurement errors. During the feature line extraction stage, the system does not simply detect image edges, but calculates the principal curvature k1 and principal direction t1 of the reconstructed model surface based on differential geometry principles. The system sets the feature line point extraction condition as follows: the derivative of the principal curvature along the principal direction is zero, thereby finding the ridge line with the most severe surface curvature, which corresponds to the peak position of the edge of the stamped part. The system introduces the RPS (Reference Point System) benchmark alignment algorithm. This algorithm locates the reference point set and its corresponding theoretical normal vector. The data originates from original CAD design drawings or digital prototypes (DMUs) of automotive parts. The alignment process essentially solves a constrained optimization problem: ensuring the spatial position of the measured model coincides with the theoretical digital model while maintaining tolerance requirements for the reference points. This completely replaces traditional physical positioning fixtures. Finally, the system decomposes the positional deviation vector between the measured point and the theoretical point into two orthogonal components: normal deviation... It is the projection of the deviation vector onto the theoretical normal vector, the normal deviation. This reflects the surface accuracy of the stamping die and the amount of metal springback; tangential deviation It is the magnitude of the cross product of the deviation vector and the theoretical normal vector, and the tangential deviation. This reflects the trimming or laser cutting position error of the parts' edges. This decoupled calculation allows the test results to directly guide mold compensation in the stamping workshop and trimming adjustments in the welding workshop, realizing a cycle from quality control to process optimization.

[0120] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A system for detecting the surface and line contours of automotive parts, comprising a global field generator, an execution terminal, a composite sensor head, and a main control center, characterized in that: The global field generator is used to construct a global rigid constraint field, which provides an absolute coordinate reference for the execution terminal. The execution terminal calculates the end pose error matrix in real time and performs dynamic compensation based on the global rigid constraint field and the preset elastic dynamics model, so as to drive the composite sensor head to reach the predetermined detection pose. The composite sensor head collects heterogeneous surface contour data and line contour data based on phase deflection gradient of automotive parts under a predetermined detection pose. The main control center receives heterogeneous surface contour data and line contour data, performs spatial alignment using a probability field registration algorithm, and obtains aligned heterogeneous surface contour data and line contour data, including: The heterogeneous surface contour data is used as a surface contour point cloud data set, and the local shape obtained by integrating the line contour data based on the phase deflection gradient is used as a gradient data set. A log-likelihood function is constructed based on a Gaussian mixture model. The log-likelihood function characterizes the statistical distribution overlap between the surface contour point cloud data set and the gradient data set after rotation and translation transformation. The rigid body transformation matrix is ​​solved by maximizing the log-likelihood function, and the associated bandwidth parameter is reduced by using an annealing strategy during the iterative solution process. Furthermore, a variational surface-line fusion reconstruction algorithm is used to unify the aligned heterogeneous surface contour data and line contour data into a high-fidelity 3D model, including: Construct a minimization functional, which consists of the integrals of the data fidelity term, the gradient fidelity term, and the smoothing term; The data fidelity term characterizes the difference between the surface to be reconstructed and the heterogeneous surface contour data, and the gradient fidelity term characterizes the difference between the gradient field of the surface to be reconstructed and the line contour data based on the phase deflection gradient. Derive the Euler-Lagrange equation corresponding to the minimized functional. The Euler-Lagrange equation is a partial differential equation that includes the weights of the data fidelity term, the gradient fidelity term, and the smoothing term. Solving the Euler-Lagrange equations yields the high-fidelity 3D model. The detection results are output based on a high-fidelity 3D model.

2. The automotive component outline and line contour detection system according to claim 1, characterized in that, Constructing a global rigid constraint field includes: The global field generator is used to identify coded markers affixed around tooling and components; A reprojection error function is constructed, which is composed of the weighted sum of squared differences between the observed pixel coordinates of the encoded marker points on the image and the theoretical projection coordinates calculated by the camera projection function; The reprojection error function is minimized using the bundle adjustment method, and the camera intrinsic parameter matrix, camera rotation and translation matrix, and the three-dimensional coordinates of all coded marker points are calculated.

3. The automotive component outline and line contour detection system according to claim 1, characterized in that, Based on the global rigid constraint field and a preset elastic dynamics model, the end-effector pose error matrix is ​​calculated in real time and dynamically compensated to drive the composite sensor head to the predetermined detection pose, including: The global field generator is used to track the target set at the end of the execution terminal in real time; The elastic dynamics model is constructed based on the joint stiffness matrix, link stiffness matrix, Jacobian matrix, external load, and gravity vector; The elastic deformation displacement at the end of the execution terminal is calculated using the elastic dynamics model, and the elastic deformation displacement is converted into an error spinor. The nominal pose of the execution terminal is exponentially mapped and corrected using the error screw to obtain the corrected end pose.

4. The automotive component outline and line contour detection system according to claim 1, characterized in that, Under a predetermined detection pose, heterogeneous surface contour data and line contour data based on phase deflection gradient of automotive parts are collected, including: The composite sensor head is used to project a precoded pattern to obtain the surface reflectivity distribution. The optimal projection intensity is calculated based on the surface reflectivity distribution, the ideal grayscale response value of the camera sensor, and the angle between the projected ray and the surface normal. A sinusoidal fringe sequence is generated based on the optimal projection intensity, multi-step phase-shifted light intensity images are acquired and the phase is demodulated to obtain the heterogeneous surface contour data; The composite sensor head is controlled to display orthogonal sinusoidal stripes and to acquire the stripe image reflected from the surface of the automotive parts; The phase difference between the horizontal and vertical directions is calculated. Based on the phase difference, the physical fringe pitch, and the distance from the screen to the object being measured, the surface gradient is calculated. Based on the second derivative of the surface gradient, the characteristic line response function is calculated to obtain the line profile data based on the phase deflection gradient.

5. The automotive component outline and line contour detection system according to claim 1, characterized in that, The detection results are output based on the high-fidelity 3D model, including: Calculate the principal curvature and principal direction of the high-fidelity 3D model, extract feature line points that satisfy the condition that the derivative of the principal curvature along the principal direction is zero, and fit the feature line points into a non-uniform rational B-spline curve; Obtain a set of positioning reference points for automotive parts, and constrain and align the high-fidelity 3D model with the theoretical model by minimizing the sum of squared normal distances between the high-fidelity 3D model and the theoretical model at the positioning reference points; Calculate the positional deviation vector between the measured points on the high-fidelity 3D model and the theoretical points on the theoretical model; The projection component of the position deviation vector onto the theoretical normal vector is taken as the normal deviation, and the magnitude of the vector product of the position deviation vector and the theoretical normal vector is taken as the tangential deviation. The normal deviation and the tangential deviation are then output.

Images