A laser projection conversion method for painting defects coordinates of a follow-up vehicle body
By establishing a cloud database of vehicle feature points and dynamically filtering high-confidence feature points, combined with Kalman filter prediction and non-rigid deformation compensation, high-precision tracking projection of defect coordinates in dynamic production environments was achieved, solving the problem of inaccurate positioning caused by vehicle movement and improving projection accuracy and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHENCHANG (SUZHOU) TECHNOLOGY CO LTD
- Filing Date
- 2026-04-15
- Publication Date
- 2026-06-19
AI Technical Summary
In the automotive painting process, factors such as speed fluctuations, posture deflections, and local deformations caused by vehicle movement make it difficult to accurately locate defect coordinates using laser projection. Existing technologies struggle to achieve high-precision defect coordinate transformation in dynamic production environments.
By establishing a cloud database of vehicle feature points, dynamically selecting high-confidence feature points, constructing a local coordinate system, and using Kalman filtering to predict vehicle body pose, combined with non-rigid deformation compensation and advance projection correction, high-precision following projection of defect coordinates is achieved.
Under conditions of vehicle speed fluctuations and attitude changes, the projection positioning error is controlled within ±3mm, which is more than 60% more accurate than existing methods, significantly improving robustness and adaptability.
Smart Images
Figure CN122243995A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of automotive coating quality inspection and laser-assisted repair technology, and particularly relates to a laser projection conversion method for the coordinates of defects in automotive body coating topcoat. Background Technology
[0002] In automotive painting production, the detection and repair of topcoat defects (such as pinholes, particles, scratches, and orange peel) is a crucial step in ensuring the quality of the vehicle's appearance. Traditional manual visual inspection combined with manual marking is inefficient and prone to missing defects. Therefore, in recent years, automated defect detection and laser projection-assisted repair systems based on machine vision have emerged. These systems typically first use a camera at the inspection station to capture images of the vehicle's surface. Image processing algorithms then identify defects and calculate their coordinates in three-dimensional space. Subsequently, as the vehicle moves to the projection station, a laser projector projects a laser spot or outline onto the corresponding location on the vehicle's surface based on these coordinates, providing operators with intuitive repair guidance. This approach avoids the time-consuming process of manually searching for defects and significantly improves repair efficiency.
[0003] However, in actual assembly line production, the vehicle body is not stationary but moves continuously on the painting conveyor line. Most existing technologies assume that the vehicle body remains stationary between the inspection and projection stations or moves at a strictly ideal straight line at a uniform speed, thus using a one-time static calibration result to directly convert the defect coordinates at the inspection station to the projection station. But in real-world conditions, the vehicle body often takes several seconds or even longer to move from the inspection station to the projection station. During this time, the conveyor line speed may fluctuate, and the vehicle body may experience slight deflection, pitch, or tilt due to uneven tracks, roller slippage, or acceleration / deceleration. These dynamic changes cause inconsistencies between the actual spatial positions of various points on the vehicle body surface at the inspection and projection times, resulting in a shift in the originally accurate defect coordinates during projection. This can range from the laser spot deviating from the defect center to hitting the wrong area completely, rendering it useless for guidance.
[0004] To address deviations caused by vehicle movement, some improvements have involved attaching manual markers to the vehicle body or adding additional visual tracking equipment between the inspection and projection stations to dynamically correct coordinates by tracking the markers' positions in real time. However, manual markers require pre-application and regular maintenance, increasing labor costs and downtime on the production line. Furthermore, markers are easily covered or contaminated by paint during the painting process, leading to tracking failure. In addition, when the vehicle surface has large areas of high gloss, reflection, or sparse texture, relying solely on image feature point matching is often unstable, easily resulting in feature point loss or mismatches, leading to coordinate system calculation failures or significant errors.
[0005] Another long-overlooked issue is that the vehicle body is not an ideal rigid body. For movable components such as doors, hoods, and trunk lids, minute displacements (on the order of millimeters) may occur relative to the main body frame due to assembly gaps, their own weight, or temporary opening and closing during the production line. Existing coordinate transformation methods typically treat the entire vehicle body as a rigid body, assuming that the relative distances between all feature points remain constant, thus failing to handle such localized non-rigid deformations. When defects happen to be located in areas such as doors, ignoring deformation introduces additional positioning errors, affecting projection accuracy.
[0006] Furthermore, laser projectors themselves have a response delay. From the time the control system issues a deflection command to the time the galvanometer actually deflects the laser beam and stabilizes its output, a certain physical time is required. For a continuously moving vehicle, this time allows the vehicle to have already moved forward a certain distance. Traditional methods do not consider this delay, causing the laser projection position to always lag behind the actual position of the vehicle, resulting in a trailing phenomenon; the higher the speed, the more pronounced the deviation.
[0007] The aforementioned problems are particularly prominent on high-speed mixed-flow painting lines, severely restricting the practical application of laser projection technology in dynamic production environments. Therefore, there is an urgent need for a laser projection conversion method that can truly enable defect coordinates to follow the moving vehicle body and dynamically compensate for various interference factors. Summary of the Invention
[0008] This invention provides a laser projection conversion method for the coordinates of defects in the paint coating of a moving vehicle body, in order to solve the problem of inaccurate defect projection positioning under conditions such as fluctuations in vehicle speed, attitude deflection, local deformation, and occlusion of feature points.
[0009] Specifically, the technical solution provided by this invention is as follows: A laser projection conversion method for determining the coordinates of defects in the topcoat of a moving vehicle body includes: S1. Establish a vehicle feature point cloud database: acquire a three-dimensional point cloud of the vehicle surface, extract several stable feature points, record the three-dimensional coordinates of each feature point in the vehicle coordinate system, and assign a quality label to each feature point. The quality label includes geometric stability, anti-occlusion coefficient and local stiffness index. S2. Defect identification and dynamic feature point cloud acquisition at the inspection station: Collect vehicle images at the inspection station, identify defects and obtain defect pixel coordinates, and simultaneously extract natural feature points from the image and match them with feature points in the database. Calculate dynamic weights based on the geometric stability, anti-occlusion coefficient, and current image clarity of the feature points, and select feature points with weights higher than a preset threshold to form a high-confidence feature point set. S3. Defect coordinate anchoring at the inspection station: Using the weighted center of the high confidence feature point set as the origin and the principal direction obtained from the weighted principal component analysis as the coordinate axis, a local coordinate system of the vehicle body at the inspection time is constructed, and the defect coordinates are converted into anchored local coordinates under this local coordinate system. S4. Vehicle motion pose tracking and prediction: During the movement of the vehicle from the detection station to the projection station, the temporal changes of the local coordinate system of the vehicle are obtained through at least one intermediate image acquisition station, and the local coordinate system parameters when the vehicle reaches the projection station are predicted by a filtering model. S5. Reconstruction and Deformation Compensation of Local Coordinate System at Projection Station: Acquire vehicle body image at projection station, match feature points and use the predicted local coordinate system parameters as initial values, perform weighted iterative nearest point registration, and reconstruct the current vehicle body local coordinate system; For defects located in non-rigid areas, perform deformation compensation on the anchored local coordinates based on the change in distance between feature point pairs between the detection station and the projection station. S6. Coordinate Restoration and Advance Projection Correction: The compensated anchored local coordinates are restored to absolute spatial coordinates through the current vehicle body local coordinate system, and the advance position is corrected according to the response delay of the laser projector and the instantaneous speed of the vehicle body to obtain the final projection coordinates.
[0010] Furthermore, the geometric stability is defined as the normalized product of the rate of curvature change of the local region where the feature point is located and the edge response intensity, and its calculation formula is as follows:
[0011] in, and These are the maximum and minimum principal curvatures of the local surface at the feature point, obtained by fitting a quadratic surface or principal component analysis. G The gradient magnitude of the feature point. G max For maximum gradient magnitude, geometric stability Sg The range is 0~1; The anti-occlusion coefficient is obtained by statistical analysis of historical production images, representing the probability value that the feature point is occluded at the detection station or projection station; the local rigidity index is determined based on the historical measurement values of whether the feature point is located on a movable part of the vehicle body and its relative displacement. Feature points with a rigidity index greater than a preset threshold are defined as rigid reference points, otherwise they are defined as flexible auxiliary points. The current image sharpness is obtained by calculating the variance of the Laplacian response in the neighborhood of the feature point, and the dynamic weight is the product of geometric stability, anti-occlusion coefficient and normalized Laplacian variance.
[0012] Furthermore, in step S3, the weighting center... The calculation formula is:
[0013] in, For the first i Dynamic weights of each feature point These are the three-dimensional coordinates of the feature point in the camera coordinate system. n This represents the total number of feature points; The weighted principal component analysis includes the following steps: Calculate the weighted covariance matrix:
[0014] Eigenvalue decomposition is performed on the weighted covariance matrix C to obtain three eigenvalues λ1 ≥ λ2 ≥ λ3 and their corresponding eigenvectors e1, e2, e3. Let e1 be the X-axis of the local coordinate system. d axis, e2 is Y d Axis, Z d The axis is determined by e1×e2.
[0015] Furthermore, in step S3, firstly, based on the intrinsic parameter matrix and depth information of the camera at the inspection station, the pixel coordinates of the defect are back-projected onto the camera coordinate system to obtain the three-dimensional coordinates of the defect in the camera coordinate system. Q cam Then, the defect coordinates Q cam Converted to anchored local coordinates in the vehicle's local coordinate system at the detection time. ,in, R d It is a rotation matrix with eigenvectors e1, e2, and e3 as column vectors.
[0016] Further, step S4 includes: When the vehicle moves to each intermediate image acquisition station, the camera at that station is triggered to acquire an image of the vehicle. Natural feature points in the image are extracted and matched with the vehicle model feature point cloud database. The matched feature points are used to construct the local coordinate system of the vehicle at that moment, and the coordinates of the origin of the local coordinate system at that moment are recorded. O ( t k ) and rotation matrix R ( t k ) and their corresponding timestamps t k ; A Kalman filter model is established using the three-dimensional coordinates of the origin of the local coordinate system of the vehicle body and the Euler angles as state variables. The process model assumes that the vehicle body moves in uniform linear motion with constant angular velocity between adjacent sampling times, and the observed values are the local coordinate system parameters calculated at each intermediate station. Based on the total movement time of the vehicle body from the inspection station to the projection station and the time-series observations obtained from intermediate stations, the predicted local coordinate system parameters at the moment the vehicle body arrives at the projection station are extrapolated through the Kalman filter prediction step, including the predicted origin coordinates. O pred and predict rotation matrix R pred .
[0017] Further, in step S5, the objective function of the weighted iterative nearest point registration is:
[0018] in, The coordinates of the vehicle body for the feature points in the database. The current observation coordinates, For dynamic weights of feature points, and Let be the rotation matrix and translation vector to be determined, with initial values taken from the predicted local coordinate system parameters, i.e. , .
[0019] Furthermore, in step S5, the deformation compensation adopts an inverse distance weighted method, and the compensation amount... and the compensated anchored local coordinates The calculation formulas are as follows: ,
[0020] in, For the first j The change in distance between each feature point and the detection station and the projection station. To anchor the local coordinates of the defects before compensation, For the first j The local coordinates of each feature point at the detection time. k The number of feature point pairs participating in the compensation.
[0021] Further, in step S6, the absolute spatial coordinates and the final projected coordinates are calculated as follows:
[0022]
[0023]
[0024] in, and These represent absolute spatial coordinates and final projected coordinates, respectively. The projection coordinates before the forward position correction; and These are the rotation matrix and translation vector of the laser projector relative to the camera, respectively, obtained from calibration. This represents the vehicle's displacement from the time the laser projector receives the command to the time it actually emits light. d This is the unit vector of the vehicle's motion direction in the laser projector coordinate system.
[0025] Preferably, the method further includes converting the final projection coordinates into laser galvanometer deflection commands to control the laser projector to project the defect points or defect contours onto the vehicle surface: When the projected object is a defect point, the final projected coordinates are transformed to the laser projector coordinate system to obtain three-dimensional coordinates. According to the formula Calculate the horizontal deflection angle of the laser galvanometer i x and vertical deflection angle i y The angle value is converted into a driving voltage signal and sent to the galvanometer controller to control the laser beam to deflect to the target position. When the projection object is a defect contour, the boundary key points of the defect contour are transformed according to S3~S6 to obtain the advanced correction coordinates of each key point in the laser projector coordinate system. The laser projector deflects the laser beam to the position of each key point in sequence, and linear interpolation scanning is performed between adjacent key points through the galvanometer to form a closed defect contour light spot.
[0026] Based on the above method, the present invention also provides a laser projection conversion system for the coordinates of defects in the paint finish of a moving vehicle body, for realizing the laser projection conversion method described above.
[0027] Compared with the prior art, the present invention has at least the following beneficial effects: This invention introduces a dynamic feature point cloud anchoring mechanism, converting defect coordinates from absolute spatial coordinates at the time of detection to anchored local coordinates relative to the local coordinate system of the vehicle's feature point cloud, thus truly attaching the defect coordinates to the vehicle surface. When the vehicle moves from the detection station to the projection station, regardless of fluctuations in the conveyor speed or slight deflection or vibration of the vehicle, the defect coordinates move synchronously with the vehicle. At the projection moment, the current local coordinate system is reconstructed to accurately restore the absolute coordinates for projection. This mechanism fundamentally solves the projection deviation problem caused by changes in vehicle pose in traditional static calibration methods. It ensures that the projection positioning error remains within ±3mm even with vehicle speed fluctuations of ±20% and yaw angle changes of ±5°, representing an accuracy improvement of over 60% compared to existing methods.
[0028] This invention establishes a multi-dimensional feature point quality assessment and dynamic weighting system. By assigning three quality labels—geometric stability, anti-occlusion coefficient, and local stiffness index—to each feature point, and combining these with the current image sharpness (Laplacian variance) to calculate dynamic weights, intelligent feature point selection and weighted application are achieved. Even under adverse conditions such as partial feature point occlusion, illumination changes, or image blurring, a stable local coordinate system can still be constructed based on high-confidence feature points, significantly improving the robustness of coordinate transformation. The weighted iterative nearest-point registration method uses temporal prediction values as initial values, further reducing the registration search space, and can accurately calculate the vehicle pose even when the number of matched feature points is insufficient.
[0029] This invention introduces a non-rigid local deformation compensation mechanism for the first time in the field of paint defect projection. For the minute displacements that may occur in movable parts such as doors and hoods relative to the main body frame, the mechanism compensates for the anchored local coordinates by detecting the change in distance between feature point pairs between the detection station and the projection station, using an inverse distance weighting method. This innovation improves the accuracy of defect projection in non-rigid areas by 40%, overcoming the systematic errors introduced by traditional methods that treat the vehicle body as an ideal rigid body.
[0030] This invention proposes a dynamic compensation strategy combining temporal prediction and advance projection. By setting up an intermediate image acquisition station to obtain the temporal changes of the vehicle's local coordinate system, a Kalman filter is used to predict the vehicle's pose at the projection moment, providing accurate initial values for subsequent registration. Simultaneously, to address the inherent response delay of the laser projector, the projection coordinates are advanced based on the vehicle's instantaneous speed, effectively eliminating the trailing phenomenon under high-speed motion conditions. This strategy enables the system to adapt to painting line speeds up to 2 m / s, ensuring real-time projection performance and accuracy under dynamic conditions.
[0031] This invention enables the transmission and projection of defect information from points to contours. By anchoring key points of the defect contour to the local coordinate system and reconstructing them at the projection end, it provides operators with complete visual guidance of the defect boundaries, significantly improving repair efficiency and accuracy. Simultaneously, the closed-loop error verification mechanism after projection gives the system self-learning capabilities, enabling online updates to the registration model and further enhancing long-term operational stability.
[0032] In summary, this invention achieves high-precision tracking projection of defect coordinates on a moving vehicle body by organically integrating a series of innovative mechanisms such as dynamic anchoring of feature point clouds, multi-dimensional weight screening, temporal prediction registration, non-rigid compensation, advance correction, and contour projection. It is significantly superior to existing technologies in terms of positioning accuracy, robustness, adaptability, and intelligence level, and has outstanding substantive features and significant progress. Attached Figure Description
[0033] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.
[0034] Figure 1 This is a schematic diagram of the laser projection conversion method provided in an embodiment of the present invention. Detailed Implementation
[0035] The following detailed description of a laser projection conversion method for determining the coordinates of defects in a vehicle body paint finish, based on a specific embodiment, illustrates this invention. This embodiment uses an SUV model on a production line in an automotive painting workshop as an example. The production line conveyor chain speed is 1.2 m / s, the straight-line distance between the inspection station and the projection station is 6 m, and the time required for the vehicle body to move from the inspection station to the projection station is approximately 5 seconds. The entire system includes: a color industrial camera (5-megapixel resolution, equipped with a 25mm lens) and a line structured light 3D sensor installed at the inspection station; a high-brightness green laser projector (galvanometer response time 8ms) and an auxiliary monochrome camera (for real-time acquisition of vehicle body images) installed at the projection station; and an intermediate image acquisition station (using existing security cameras on the production line with 2-megapixel resolution) set every 2 meters between the inspection station and the projection station. All stations are connected to a central industrial control computer via Ethernet, which is equipped with an NVIDIA Jetson AGX Orin embedded AI computing platform.
[0036] I. Offline creation of vehicle feature point cloud database and feature point quality labels First, for this SUV model, a standard vehicle body was placed stationary at the inspection station. A high-precision 3D structured light scanner (accuracy ±0.1mm) was used to scan the entire exterior surface of the vehicle, obtaining a complete 3D point cloud model of the body. The point cloud coordinate system was defined as the vehicle body coordinate system: with the projection of the vehicle's front axle center onto the ground as the origin, the X-axis pointing towards the rear of the vehicle, the Y-axis pointing towards the passenger side, and the Z-axis pointing vertically upwards. Then, stable feature points were automatically extracted from the point cloud. Extraction rules included: finding intersections of edges with curvature changes greater than 0.5, local curvature minima (such as the center of a stamping dent), and geometric discontinuities (such as the endpoints of door gaps and the corner points of the fuel tank cap). After manual verification, 80 feature points were finally determined and stored in the database. Each feature point, in addition to recording its 3D coordinates (x, y, z) in the vehicle body coordinate system, also has three quality labels: 1. Geometric stability: defined as the product of the rate of change of curvature of the local region where the feature point is located and the edge response intensity, i.e. , and These are the maximum and minimum principal curvatures of the local surface at the feature point, respectively, which can be obtained by fitting a quadratic surface or principal component analysis.G The gradient magnitude of the feature point. G max For maximum gradient magnitude, geometric stability Sg The range is 0 to 1. For example, the geometric stability of a feature point located at the intersection of a sharp edge is 0.95, while that of a point located on a flat surface is only 0.4.
[0037] 2. Occlusion Resistance Coefficient: This coefficient is calculated by statistically analyzing images of 200 actual production vehicles to determine the probability that a feature point will be occluded by other components (such as robotic arms or operators) at the inspection or projection station. The lower the probability, the higher the coefficient, ranging from 0 to 1. For example, the occlusion resistance coefficient for a feature point near a door handle is 0.3, while the occlusion resistance coefficient for a fixed hole feature point along the lower edge of the windshield is 0.9.
[0038] 3. Local stiffness index: Determined based on historical measurements of whether the feature point is located on a movable part and its relative displacement. For feature points located on movable parts such as doors and hoods, the stiffness index is 0.2~0.5; for feature points located on fixed structures such as body side panels and A-pillars, the stiffness index is 0.9~1.0. In this embodiment, stiffness indexes ≥0.8 are referred to as rigid reference points, and the rest are flexible auxiliary points.
[0039] The database is indexed by vehicle model code, which facilitates rapid loading during mixed-line production.
[0040] II. Inspection Station: Defect Identification and Dynamic Feature Point Cloud Acquisition When the vehicle body enters the inspection station, a color industrial camera is triggered to capture an image of the vehicle body's side profile (this embodiment uses the right rear door area as an example). Simultaneously, a line structured light sensor scans the vehicle body surface to obtain depth information, which is used to convert the image pixel coordinates into three-dimensional coordinates in the camera coordinate system. The industrial control computer loads a pre-trained deep learning semantic segmentation model (based on the U-Net architecture, trained on 100,000 defect images) to perform pixel-by-pixel classification of the image and identify defect areas. Assuming a circular shrinkage defect with a diameter of 12mm is detected, the segmentation model outputs its contour pixel set and calculates the center point pixel coordinates as (1024, 768) (image resolution 2048×1536).
[0041] Simultaneously, the system extracts natural feature points from the same image. The ORB (Oriented FAST and Rotated BRIEF) algorithm is used, with a maximum of 500 feature points and a quality level threshold of 0.8. The position and orientation of each extracted feature point in the image are calculated. These feature points are then matched against feature points in an offline database. The matching employs the FLANN (Fast Nearest Neighbor) method, and a ratio test is performed (the ratio of the nearest neighbor distance to the second nearest neighbor distance is less than 0.7). Assuming a successful match yields 12 database feature points, each with known 3D coordinates in the vehicle body coordinate system (from the database) and pixel coordinates in the current image.
[0042] For each matched feature point, a dynamic weight is calculated. The dynamic weight is obtained by multiplying three parts: the geometric stability value in the database, the anti-occlusion coefficient, and the Laplacian variance of the feature point's neighborhood in the current image (reflecting image sharpness, normalized to 0-1). For example, a feature point located at the lower end of pillar A has a geometric stability of 0.9, an anti-occlusion coefficient of 0.8, and a Laplacian variance of 0.85, so its weight is 0.9 × 0.8 × 0.85 = 0.612. A weight threshold of 0.3 is set; feature points below this threshold are discarded. Simultaneously, the retained feature points must contain at least 3 rigid reference points (rigidity index ≥ 0.8). In this embodiment, 8 feature points are retained, including 5 rigid reference points and 3 flexible auxiliary points.
[0043] Laplacian variance is a classic method for evaluating image sharpness. Its calculation process is as follows: First, apply the Laplacian operator (usually using a 3×3 discrete convolution kernel, such as [[0,1,0], [1,-4,1], [0,1,0]]) to a selected region in the image to extract the second-order gradient response value of each pixel. Then, calculate the variance of the response values of all pixels in the region. The larger the variance, the richer the high-frequency components and the sharper the edges of the image. Conversely, the smaller the variance, the more blurred the image.
[0044] III. Inspection Station: Defect coordinates are anchored to the local coordinate system of the feature point cloud. Using the three-dimensional coordinates of the above 8 feature points (known in the vehicle body coordinate system, denoted as...) ) and their current 3D coordinates in the camera coordinate system (obtained through line structured light depth information, denoted as ) While it is possible to establish a transformation relationship from the vehicle coordinate system to the camera coordinate system, this invention does not require a global transformation, but instead directly constructs a local coordinate system based on the feature point cloud.
[0045] The construction method is as follows: First, calculate the centroid of the weighted feature point set. :
[0046] in n = 8, For the first i The dynamic weights of each feature point are then determined. Principal component analysis (PCA) is then performed on the weighted point set. Specifically, the weighted covariance matrix is calculated:
[0047] Perform eigenvalue decomposition on matrix C to obtain three eigenvalues λ1 ≥ λ2 ≥ λ3 and their corresponding eigenvectors e1, e2, e3. Let e1 be the X-axis of the local coordinate system. d axis, e2 is Y d Axis, Z d The axis is determined by e1 × e2. The origin is O. d Therefore, the vehicle's local coordinate system CS at the detection time is defined. d .
[0048] Next, the three-dimensional coordinates of the defect center point will be transformed to CS. d First, using line structured light data and camera intrinsic parameters, the pixel coordinates (1024, 768) of the defect center are back-projected onto a 3D point Q in the camera coordinate system. cam The back projection formula is: given the camera intrinsic parameter matrix K and the depth value d (obtained from structured light), we have Q cam = d K 1 [u,v,1] T Assuming Q in this embodiment cam =(150.2, (80.5, 1200.0) mm (camera coordinate system origin at camera optical center, Z-axis along optical axis). Then calculate the defect center at CS. d Local coordinates below:
[0049] in R d It is a rotation matrix with column vectors e1, e2, and e3. The calculation result is: Millimeters represent the position of the defect center in the local coordinate system. Simultaneously, for subsequent contour projection, the four corner points of the circumscribed rectangle of the defect contour (with known pixel coordinates) are also converted to local coordinates using the same method, resulting in... .
[0050] IV. Intermediate Workstation: Feature Point Cloud Temporal Tracking and Pose Prediction After leaving the inspection station, the vehicle moves towards the projection station at a speed of approximately 1.2 m / s. Intermediate image acquisition stations, set up every 2 meters (corresponding to approximately 1.67 seconds), are triggered to take pictures. At each intermediate station, the system repeats the above feature point extraction and matching process to obtain the local coordinate system CS(…) at the current moment. t k ), that is, the origin O ( t k ) and rotation matrix R( t k Record these parameters along with the timestamp.
[0051] In this embodiment, the vehicle body passes through two intermediate stations (2m and 4m away from the inspection station, respectively), plus the inspection station itself ( t = 0), obtaining local coordinate system parameters at three time points. Kalman filtering was used to predict the local coordinate system at the projection station (6m from the detection station, corresponding to approximately 5.0 seconds). The state variables were selected as the three-dimensional coordinates (x, y, z) of the origin and the Euler angles (...). θ, ψ) (i.e., rotation angles about the X, Y, and Z axes). The process model assumes that the vehicle body moves at a constant linear velocity for a short period of time. The observed values are the poses calculated from the intermediate workstations. The Kalman filter prediction step yields the predicted value CS at the projection time. pred : Origin O pred = (5000, 0, 300) mm, rotation matrix R pred The corresponding Euler angles are (0.5°, 0.2°, 1.0°) (cumulative change relative to the detection time). If there is no intermediate station, the speed signal from the conveyor line encoder can be directly extrapolated, but in this embodiment, due to visual feedback, the accuracy is higher.
[0052] V. Projection Workstation: Real-time Registration and Local Coordinate System Reconstruction When the vehicle body enters the field of view of the camera at the projection station, the auxiliary camera (2 megapixels) is triggered to take a picture. Similarly, ORB feature points are extracted and matched against the database. In this embodiment, due to changes in lighting and partial occlusion, only 6 feature points are matched, of which only one rigid reference point is occluded (this point is located at the door hinge and was not detected due to occlusion by the robotic arm). At this point, if conventional ICP registration is performed directly, it is prone to failure due to insufficient feature points. This invention employs weighted ICP registration and uses the predicted value CS... pred As an initial transformation, it significantly improves robustness.
[0053] The specific steps are as follows: Let the coordinates of the currently matched feature point in the vehicle coordinate system be... (Given) The observed coordinates in the current camera coordinate system are: (Depth is obtained through PnP calculation using a monocular camera. Since the projection station in this embodiment is not equipped with structured light, at least four feature points are used to solve PnP to obtain the depth of each point.) The goal of weighted ICP is to find the rotation matrix R. cur Translation vector T cur Minimize the weighted sum of squared errors:
[0054] Where m = 6, The dynamic weights for each feature point are defined (calculated using the same method as before, but the Laplace variance is ignored here because the lighting at the projection station is stable). The initial values are set to... , Then, the singular value decomposition (SVD) method is used to solve it iteratively, and it usually converges in 3 to 5 iterations.
[0055] For the occluded rigid body reference point (denoted as...) Its current coordinates can be inferred from the registration results: Then, it is virtually added to the feature point set, but given a low weight (e.g., 0.2), and used only as a reference for subsequent non-rigid compensation. Finally, the current local coordinate system CS is obtained. cur : Origin O cur and rotation matrix R cur In this embodiment, O is calculated. cur = (5005,2,298) mm, which is within 5 mm of the predicted value, indicating that the prediction is effective.
[0056] VI. Non-rigid local deformation compensation Because the defect in this embodiment is located on the right rear door, and there is an assembly gap between the door and the side panel of the vehicle body, and slight relative displacement may occur due to vibration during vehicle movement, several pairs of feature points in the door area and side panel area are collected at the inspection station and projection station respectively to compensate for this non-rigid deformation. Specifically, two feature points Adet and Bdet on the door and one feature point Cdet on the door frame (this point is located in the rigid body area) are selected. At the projection station, the coordinates Acur, Bcur, and Ccur of these feature points are also observed (or calculated through registration). The deformation vector is defined as the change in distance between the door feature point and the door frame feature point.
[0057] More generally, for defects (Local coordinates at the detection time), there are several feature points in its vicinity. Take the closest pair of feature points and calculate the change in distance from detection to projection:
[0058] Where C is a reference point on the rigid body region (such as a feature point on a door frame). Then, the compensation amount for the defect point is determined by these Δd values. j The inverse distance weighted average yields:
[0059] In this embodiment, the distance from the defect center to the nearest feature point A on the car door is approximately 50mm, and Δd A = 1.2mm (the car door is tilted outwards); the distance from another feature point B is approximately 120mm, Δd B = 0.8mm. Δ is calculated. comp = 1.0mm. Note that the deformation direction is along the door normal, and it needs to be decomposed into the three axes of the local coordinate system according to geometric relationships. Simplified treatment: Assume that the deformation is mainly along the Z axis of the local coordinate system. d The local coordinates of the defect after compensation are as follows: (Axis, vehicle body side)
[0060] Substitute the values Millimeters. If the deformation is less than a preset threshold (e.g., 0.5 mm), no compensation will be made to avoid overcorrection.
[0061] VII. Defect Coordinate Restoration and Advanced Projection Correction Transform the compensated local coordinates back to the current absolute spatial coordinates (in the camera coordinate system):
[0062] Where R cur and T cur From step five. The calculation result is assumed to be... The coordinates are in millimeters (relative to the camera coordinate system at the projection station). Then, it needs to be transformed to the laser projector coordinate system. The projector and camera have already been jointly calibrated, resulting in the rotation matrix. Translation vector Therefore, the coordinates in the projector coordinate system are:
[0063] The calibration result in this embodiment is: Millimeters (the origin of the projector is located at the center of its galvanometer, and the Z-axis is along the optical axis).
[0064] At this point, if we directly rely on P projThe galvanometer deflection angle is calculated. Since there is approximately an 8ms physical delay between the projector receiving the command and the actual light output (including communication, calculation, and galvanometer response), and the vehicle moves at 1.2 m / s, the vehicle's movement within 8ms is δ = 1.2 m / s × 0.008 s = 9.6 mm. Therefore, a lead correction is required. The instantaneous velocity of the vehicle at the current projection position is then obtained. v cur The speed can be read in real time via a conveyor line encoder (1.22 m / s in this embodiment), or estimated from two consecutive frames of images from an auxiliary camera using optical flow. P proj The offset δ is in the opposite direction to the vehicle's direction of motion. The vehicle's direction of motion is known in the projector coordinate system (pre-calibrated, e.g., along the negative X-axis). Let the unit vector of the direction of motion be... d Then the coordinates after the advance correction are:
[0065] In this embodiment, d=( 1,0,0), therefore Millimeters. Finally, based on the galvanometer model of the laser projector, Convert to horizontal deflection angle i x and vertical deflection angle i y :
[0066] in for The coordinates. Substituting them, we get... i x ≈ 4.0° i y ≈ 2.86°. These angle values are converted into analog voltage signals and sent to the galvanometer driver.
[0067] 8. Laser profiling and closed-loop verification For defects that only project the center point, the above steps are sufficient. However, this embodiment aims to project the complete circular outline of the shrinkage hole to provide more intuitive boundary guidance. Therefore, the local coordinates of the four circumscribed rectangle corners of the defect outline have been saved in step three. Repeat steps six and seven for compensation, restoration, and advance correction at each corner point to obtain the advance-corrected coordinates of the four corner points in the projector coordinate system. Then, the projector sequentially deflects the laser beam to these points and rapidly interpolates and scans between the points to form a rectangular frame (in practice, Lissajous scanning can be used to fill this frame). To improve efficiency, only the four sides of the circumscribed rectangle can be projected, which can be achieved through galvanometer polygon scanning.
[0068] Simultaneously with projection, an auxiliary camera at the projection station captures the laser spot position and compares it with the theoretical projection position. For example, if the captured image shows the laser spot center coordinates as (1020, 755), while the theoretical position should be (1024, 768), the error is (4, 13) pixels, equivalent to approximately 2.1mm in actual distance. This error is recorded and used to correct the initial registration values for the next vehicle. Specifically, the error is used as an adjustment term for the observation noise covariance of the Kalman filter, or the weights of the weighted ICP in step five are updated online. For feature points that cause significant errors, their weights are reduced during the next registration. This creates a closed-loop self-learning capability.
[0069] Through the above steps, the present invention achieves accurate projection of defect coordinates under vehicle motion conditions. The overall process is as follows: Figure 1 As shown in the diagram. In actual testing, the above process was repeated for 100 vehicles, with an average projection error of ±2.3mm and a maximum error of no more than ±4mm, representing an improvement of approximately 70% in accuracy compared to the traditional static calibration method (average error ±8mm). Deformation compensation in the door area reduced the potential 3mm deviation to less than 1mm. Advanced projection correction completely eliminated the trailing phenomenon, ensuring that the projection position coincided with the defect even when the vehicle speed was briefly increased to 1.5m / s. The total delay from detection to projection was controlled within 0.5 seconds (excluding vehicle movement time), meeting the production line's 30JPH cycle time requirement.
[0070] Those skilled in the art should understand that the specific numerical values, vehicle models, and algorithm parameters (such as the number of feature points, weight thresholds, Kalman filter models, etc.) used in the above embodiments can be adjusted according to actual production line conditions without deviating from the scope of protection of this invention. For example, SIFT can be used instead of ORB in feature point matching to obtain better rotational invariance; the number of intermediate workstations can be flexibly set according to the workshop layout, or even no intermediate workstations can be set and the encoder extrapolation can be relied upon entirely; the non-rigid deformation compensation model can also adopt a more complex finite element simplified model. However, no matter how it is changed, as long as the core ideas of dynamic feature point cloud anchoring, weighted local coordinate system, temporal prediction, non-rigid compensation, and advance projection are adopted, they all fall within the scope of protection of this invention.
[0071] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; under the concept of the present invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the present invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.
Claims
1. A laser projection conversion method for determining the coordinates of defects in the topcoat of a moving vehicle body, characterized in that, include: S1. Establish a vehicle feature point cloud database: acquire a three-dimensional point cloud of the vehicle surface, extract several stable feature points, record the three-dimensional coordinates of each feature point in the vehicle coordinate system, and assign a quality label to each feature point. The quality label includes geometric stability, anti-occlusion coefficient and local stiffness index. S2. Defect identification and dynamic feature point cloud acquisition at the inspection station: Collect vehicle images at the inspection station, identify defects and obtain defect pixel coordinates, and simultaneously extract natural feature points from the image and match them with feature points in the database. Calculate dynamic weights based on the geometric stability, anti-occlusion coefficient, and current image clarity of the feature points, and select feature points with weights higher than a preset threshold to form a high-confidence feature point set. S3. Defect coordinate anchoring at the inspection station: Using the weighted center of the high confidence feature point set as the origin and the principal direction obtained from the weighted principal component analysis as the coordinate axis, a local coordinate system of the vehicle body at the inspection time is constructed, and the defect coordinates are converted into anchored local coordinates under this local coordinate system. S4. Vehicle motion pose tracking and prediction: During the movement of the vehicle from the detection station to the projection station, the temporal changes of the local coordinate system of the vehicle are obtained through at least one intermediate image acquisition station, and the local coordinate system parameters when the vehicle reaches the projection station are predicted by a filtering model. S5. Reconstruction and Deformation Compensation of Local Coordinate System at Projection Station: Acquire vehicle body image at projection station, match feature points and use the predicted local coordinate system parameters as initial values, perform weighted iterative nearest point registration, and reconstruct the current vehicle body local coordinate system; For defects located in non-rigid areas, perform deformation compensation on the anchored local coordinates based on the change in distance between feature point pairs between the detection station and the projection station. S6. Coordinate Restoration and Advance Projection Correction: The compensated anchored local coordinates are restored to absolute spatial coordinates through the current vehicle body local coordinate system, and the advance position is corrected according to the response delay of the laser projector and the instantaneous speed of the vehicle body to obtain the final projection coordinates.
2. The laser projection conversion method as described in claim 1, characterized in that, The geometric stability is defined as the normalized product of the rate of curvature change of the local region where the feature point is located and the edge response intensity, and its calculation formula is as follows: in, and These are the maximum and minimum principal curvatures of the local surface at the feature point, obtained by fitting a quadratic surface or principal component analysis. G The gradient magnitude of the feature point. G max For maximum gradient magnitude, geometric stability Sg The range is 0~1; The anti-occlusion coefficient is obtained by statistical analysis of historical production images, representing the probability value that the feature point is occluded at the detection station or projection station; the local rigidity index is determined based on the historical measurement values of whether the feature point is located on a movable part of the vehicle body and its relative displacement. Feature points with a rigidity index greater than a preset threshold are defined as rigid reference points, otherwise they are defined as flexible auxiliary points. The current image sharpness is obtained by calculating the variance of the Laplacian response in the neighborhood of the feature point, and the dynamic weight is the product of geometric stability, anti-occlusion coefficient and normalized Laplacian variance.
3. The laser projection conversion method as described in claim 1, characterized in that, In step S3, the weighting center The calculation formula is: in, For the first i Dynamic weights of each feature point These are the three-dimensional coordinates of the feature point in the camera coordinate system. n This represents the total number of feature points; The weighted principal component analysis includes the following steps: Calculate the weighted covariance matrix: Eigenvalue decomposition is performed on the weighted covariance matrix C to obtain three eigenvalues λ1 ≥ λ2 ≥ λ3 and their corresponding eigenvectors e1, e2, e3. e1 is taken as the X-axis of the local coordinate system. d axis, e2 is Y d Axis, Z d The axis is determined by e1×e2.
4. The laser projection conversion method as described in claim 3, characterized in that, In step S3, firstly, based on the intrinsic parameter matrix and depth information of the camera at the inspection station, the pixel coordinates of the defect are back-projected onto the camera coordinate system to obtain the three-dimensional coordinates of the defect in the camera coordinate system. Q cam Then, the defect coordinates Q cam Converted to anchored local coordinates in the vehicle's local coordinate system at the detection time ,in, R d It is a rotation matrix with eigenvectors e1, e2, and e3 as column vectors.
5. The laser projection conversion method as described in claim 1, characterized in that, Step S4 includes: When the vehicle moves to each intermediate image acquisition station, the camera at that station is triggered to acquire an image of the vehicle. Natural feature points in the image are extracted and matched with the vehicle model feature point cloud database. The matched feature points are used to construct the local coordinate system of the vehicle at that moment, and the coordinates of the origin of the local coordinate system at that moment are recorded. O ( t k ) and rotation matrix R ( t k ) and their corresponding timestamps t k ; A Kalman filter model is established using the three-dimensional coordinates of the origin of the local coordinate system of the vehicle body and the Euler angles as state variables. The process model assumes that the vehicle body moves in uniform linear motion with constant angular velocity between adjacent sampling times, and the observed values are the local coordinate system parameters calculated at each intermediate station. Based on the total movement time of the vehicle body from the inspection station to the projection station and the time-series observations obtained from intermediate stations, the predicted local coordinate system parameters at the moment the vehicle body arrives at the projection station are extrapolated through the Kalman filter prediction step, including the predicted origin coordinates. O pred and predict rotation matrix R pred .
6. The laser projection conversion method as described in claim 5, characterized in that, In step S5, the objective function of the weighted iterative nearest point registration is: in, The coordinates of the vehicle body for the feature points in the database. The current observation coordinates, For dynamic weights of feature points, and Let be the rotation matrix and translation vector to be determined, with initial values taken from the predicted local coordinate system parameters, i.e. , .
7. The laser projection conversion method as described in claim 6, characterized in that, In step S5, the deformation compensation adopts an inverse distance weighted method, and the compensation amount... and the compensated anchored local coordinates The calculation formulas are as follows: ， in, For the first j The change in distance between each feature point and the detection station and the projection station. To anchor the local coordinates of the defects before compensation, For the first j The local coordinates of each feature point at the detection time. k The number of feature point pairs participating in the compensation.
8. The laser projection conversion method as described in claim 7, characterized in that, In step S6, the absolute spatial coordinates and the final projected coordinates are calculated as follows: in, and These represent absolute spatial coordinates and final projected coordinates, respectively. The projection coordinates before the forward position correction; and These are the rotation matrix and translation vector of the laser projector relative to the camera, respectively, obtained from calibration. This represents the vehicle's displacement from the time the laser projector receives the command to the time it actually emits light. d This is the unit vector of the vehicle's motion direction in the laser projector coordinate system.
9. The laser projection conversion method as described in claim 1, characterized in that, This also includes converting the final projection coordinates into laser galvanometer deflection commands, controlling the laser projector to project the defect points or defect contours onto the vehicle surface: When the projected object is a defect point, the final projected coordinates are transformed to the laser projector coordinate system to obtain three-dimensional coordinates. According to the formula Calculate the horizontal deflection angle of the laser galvanometer θ x and vertical deflection angle θ y The angle value is converted into a driving voltage signal and sent to the galvanometer controller to control the laser beam to deflect to the target position. When the projection object is a defect contour, the boundary key points of the defect contour are transformed according to S3~S6 to obtain the advanced correction coordinates of each key point in the laser projector coordinate system. The laser projector deflects the laser beam to the position of each key point in sequence, and linear interpolation scanning is performed between adjacent key points through the galvanometer to form a closed defect contour light spot.
10. A laser projection conversion system for determining the coordinates of defects in the topcoat of a moving vehicle body, characterized in that, Used to implement the laser projection conversion method as described in any one of claims 1 to 9.