A 3D glue coating track synthesis method and system based on multi-coordinate system contour

Abstract

This invention provides a method and system for synthesizing 3D adhesive coating trajectories based on contours in multiple coordinate systems. The method includes: acquiring multiple sets of image data containing checkerboard corner points using a measuring instrument array; extracting the corresponding corner point coordinates from the multiple sets of image data based on a growing checkerboard corner point detection algorithm; obtaining the detected corner point coordinates in the corresponding three-dimensional coordinate system using multiple 3D line laser contour measuring instruments; using the checkerboard corner points corresponding to the detected corner point coordinates as iteration points for point cloud registration; transforming the point cloud registration problem into an optimization problem; solving for the transformation matrix between multiple three-dimensional coordinate systems; performing coordinate system unification based on the transformation matrix to obtain the target coordinate system; and synthesizing the 3D adhesive coating trajectory using the multi-array laser contour measuring instruments based on the target coordinate system. This invention achieves efficient and accurate point cloud registration, is applicable to trajectory synthesis in scenarios with overlapping point clouds, and improves the accuracy of the synthesized adhesive coating trajectory.

Description

Technical Field

[0001] This invention relates to the field of intelligent adhesive coating technology, and in particular to a method and system for synthesizing 3D adhesive coating trajectories based on contours in multiple coordinate systems. Background Technology

[0002] Adhesive coating technology has been widely used by major automobile manufacturers and auto parts manufacturers. It is commonly used in the body-in-white and hood coating in automobile welding workshops, as well as windshields in final assembly workshops. It directly affects the vehicle's shock absorption, noise reduction, wind protection and other performance characteristics.

[0003] Currently, there are two main non-contact glue coating inspection methods based on machine vision: 1) 2D multi-view camera. This method integrates multiple 2D area scan cameras around the glue gun, resulting in a compact design suitable for confined production environments. It offers fast inspection speed and can achieve 360° detection of glue width, position, and continuity. An integrated supplementary lighting system supports different exposure and parameter settings, adapting to various ambient light scenarios and demonstrating strong adaptability. 2) 3D line laser profile measuring instrument. Based on laser triangulation, the 3D line laser profile measuring instrument can reconstruct the 3D profile data of the glue coating in real-time, continuously, at high speed and with high precision, and then use this 3D profile data to complete subsequent inspection requirements. However, due to the limited space around the glue gun and the arbitrary spraying direction, some areas have blind spots that cannot be detected. Therefore, major industrial vision manufacturers currently use arrays of three or four 3D line laser profile measuring instruments for high-precision scanning to reconstruct the glue strip profile.

[0004] The solution of multiple 3D line laser profilometer arrays is currently the optimal solution for adhesive coating inspection, offering higher accuracy and stronger inspection capabilities in adhesive coating scenarios compared to 2D multi-view cameras. However, it also introduces new challenges. Taking an array composed of four laser profilometers as an example, the four sets of cameras and lasers have different viewing angles, and the reconstructed contours are all based on the coordinates of the 3D coordinate system under their respective viewing angles. Therefore, to make full use of this array, it is necessary to complete the transformation and unification of the coordinate systems under the four viewing angles before actual inspection. Essentially, this is a 3D point cloud registration problem. The mainstream solution is an improved algorithm based on 4PCS (4-Points Congruent Sets) and ICP (Iterative Closest Point).

[0005] As described above, the contour trajectory synthesis of a multi-array 3D line laser profilometer in multiple coordinate systems is essentially a 3D point cloud registration problem. Point cloud registration refers to the process of inputting two point cloud P... s (Source Cloud) and P t(Target point cloud) Solve for a transformation matrix T such that P s and P t The degree of overlap is as high as possible, which essentially solves for the special Euclidean group SE(3) between the source point cloud and the target point cloud:

[0006]

[0007]

[0008] For the registration problem of 3D point clouds, the most widely used point cloud registration algorithm is the Iterative Closest Point (ICP) algorithm and its improved algorithms. There are also 4PCS algorithms and their improved algorithms based on the RANSAC framework, but the ICP algorithm is obviously more favored.

[0009] Given the approximate relationship between the source and target point clouds, the Intermediate Point Collation (ICP) algorithm finds the nearest neighbor points between the two point clouds as corresponding points. It then uses a least-squares algorithm to solve for the optimal transformation matrix between these corresponding points. The ICP algorithm offers higher accuracy compared to other algorithms and does not require extracting feature points from the point clouds. However, it necessitates coarse registration before fine registration. This involves roughly searching the source and target point clouds to obtain a set of initial values ​​for corresponding points, and then iteratively solving to find the optimal solution to the least-squares problem.

[0010] Obviously, if the initial value is not near the optimal value, the ICP algorithm will need more iterations to obtain the global optimum. Therefore, the coarse registration of the ICP algorithm is very important and directly affects whether the global optimum can be obtained correctly in the future. In addition, due to the characteristic of the algorithm taking the nearest point as the corresponding point, the ICP algorithm cannot be used to solve the scenario where there is an overlap between the source point cloud and the target point cloud. Its application scenarios are very limited. It is important to note that the rotation matrix belongs to the special orthogonal group SO(3), where each column is pairwise orthogonal and the determinant is 1. This introduces additional constraints when the rotation matrix is ​​used as an optimization variable, making the optimization solution difficult.

[0011] Therefore, the current ICP registration algorithm and its improved algorithms have the following shortcomings: First, the correctness of the solution depends on the initial value obtained by coarse registration; second, there cannot be any overlap between the source point cloud and the target point cloud, but since the camera viewpoints of multi-array 3D line laser profilometers have a certain degree of overlap, there must also be overlap between the point clouds; third, the iterative optimization of ICP only considers the simple information of the point cloud in Euclidean space, ignoring the information of other parts of the multi-array profilometer system, such as camera distortion, positional information between the camera and the laser plane, etc., which inevitably limits the theoretical upper limit of its optimization. Summary of the Invention

[0012] Therefore, it is necessary to provide a method and system for synthesizing 3D adhesive application trajectories based on contours in multiple coordinate systems to address the aforementioned technical problems.

[0013] A method for synthesizing 3D adhesive application trajectories based on contours in multiple coordinate systems includes the following steps: acquiring multiple sets of image data containing checkerboard corner points based on a measuring instrument array, wherein the measuring instrument array includes multiple 3D line laser contour measuring instruments arranged in an array; extracting the corresponding corner coordinates from the multiple sets of image data based on a growing checkerboard corner detection algorithm, and obtaining the detected corner coordinates in the corresponding three-dimensional coordinate system through the multiple 3D line laser contour measuring instruments; using the checkerboard corner points corresponding to the detected corner coordinates as iteration points for point cloud registration, and transforming the point cloud registration problem into an optimization problem, solving the optimization problem by minimizing the error between the target point cloud and the source point cloud to obtain a transformation matrix between multiple three-dimensional coordinate systems; performing coordinate system unification based on the transformation matrix to obtain the target coordinate system, and synthesizing the 3D adhesive application trajectory of the multiple array contour measuring instruments based on the target coordinate system.

[0014] In one embodiment, acquiring multiple sets of image data containing checkerboard corner points based on a measuring instrument array includes: calibrating multiple 3D line laser profilometers using a sample checkerboard calibration plate; the calibrated multiple 3D line laser profilometers can scan multiple sets of point cloud contour data of the scanned object, and the multiple sets of point cloud contour data have overlapping point cloud regions; arranging the calibrated array of multiple 3D line laser profilometers at the object to be measured to obtain a measuring instrument array, and determining the corresponding checkerboard calibration plate according to the field of view of the measuring instrument array; placing the checkerboard calibration plate below the measuring instrument array, and ensuring that the 3D line laser profilometers in the measuring instrument array can capture the corner point regions of the checkerboard calibration plate, and that the measuring instrument array can cover the entire area of ​​the checkerboard calibration plate; moving and rotating the checkerboard calibration plate to obtain multiple sets of image data containing checkerboard corner points.

[0015] In one embodiment, the growth-based checkerboard corner detection algorithm extracts corresponding corner coordinates from the multiple sets of image data and obtains the detected corner coordinates in the corresponding three-dimensional coordinate system using the multiple 3D line laser profilometers. This includes: constructing templates of various scales based on the characteristics of checkerboard corners; convolving the multiple sets of image data with the templates and calculating the corner likelihood domain for each pixel to obtain a coarse corner range; processing the coarse corner range using a non-maximum suppression algorithm to obtain candidate corners; performing sub-pixel refinement processing on the positions and edge directions of the candidate corners to obtain sub-pixel refined corner positions and edge directions in the multiple sets of image data; growing the checkerboard by optimizing the energy function and solving the energy function using a discretized optimization strategy to obtain the corresponding target checkerboard, and obtaining the corner coordinates of all sub-pixels based on the target checkerboard; and obtaining the detected corner coordinates in multiple three-dimensional coordinate systems using the calibrated multiple 3D line laser profilometers.

[0016] In one embodiment, the sub-pixel refinement processing of the candidate corner point positions and edge directions to obtain the sub-pixel refined corner point positions and edge directions in the multiple sets of image data includes: assuming c is the ideal corner point, p is a pixel in the neighborhood of c, and g... p Let p be the gradient vector; minimize the following equation within the neighborhood N(c′) of the candidate corner point:

[0017]

[0018] Solving the above equation yields an analytical solution, which provides the refined corner point positions. The analytical solution to the above equation is:

[0019]

[0020] The refined edge direction is obtained by minimizing the following formula;

[0021]

[0022] In the formula, e1 and e2 represent the edge directions.

[0023] In one embodiment, growing the chessboard by optimizing the energy function includes: defining the energy function of the chessboard as:

[0024]

[0025] In the formula, E corners E is the negative of the total number of corner points in the current chessboard. structIt describes using two adjacent corner points to predict the matching degree of a third corner point; calculating the structural energy of each row and column corner point in the chessboard, comparing the structural energies, obtaining the maximum structural energy, and using the maximum structural energy as the structural energy of the chessboard.

[0026] In one embodiment, the step of using a discretized optimization strategy to solve the optimization energy function, obtain the corresponding target chessboard grid, and obtain the corner coordinates of all sub-pixels based on the target chessboard grid includes: given a seed corner point, searching along the edge direction of the seed corner point to obtain an initialized seed chessboard grid; based on the seed chessboard grid, growing chessboard grids from the four outermost edges to generate new pre-selected chessboard grids; detecting the minimum energy value among all pre-selected chessboard grids, and if the minimum energy value is less than the energy value of the original chessboard grid, the growth is successful, and the original chessboard grid is replaced by the pre-selected chessboard grid corresponding to the minimum energy value; continuing to grow until the energy value of the pre-selected chessboard grid no longer decreases, then obtaining the final target chessboard grid, and obtaining the corner coordinates of all sub-pixels based on the target chessboard grid.

[0027] In one embodiment, obtaining the coordinates of the detected corner points in multiple three-dimensional coordinate systems using calibrated 3D line laser profilometers includes: acquiring the sub-pixel coordinates of corner points in multiple sets of image data, and obtaining the coordinates of the corresponding corner points in multiple three-dimensional coordinate systems using calibrated 3D line laser profilometers, denoted as:

[0028]

[0029] In one embodiment, the step of transforming the point cloud registration problem into an optimization problem, and solving the optimization problem by minimizing the error between the target point cloud and the source point cloud to obtain a transformation matrix between multiple three-dimensional coordinate systems, includes: transforming the point cloud registration problem into an optimization problem, wherein the optimization objective of the optimization problem is:

[0030]

[0031] Transforming the above equation into vector form, we have:

[0032]

[0033] In the formula, p s ,p t p represents the corresponding points between the source point cloud and the target point cloud. i ,p′ i Let represent the i-th point in the target point cloud and the source point cloud, respectively; define the centroids of the two sets of points in the target point cloud and the source point cloud as:

[0034]

[0035] The error between the source point cloud and the target point cloud is obtained based on the centroid and corresponding point, and then processed through an error function to obtain the transformation matrix between multiple three-dimensional coordinate systems. The error function is:

[0036]

[0037] A 3D adhesive application trajectory synthesis system based on contours in multiple coordinate systems is characterized by comprising the following components for implementing the 3D adhesive application trajectory synthesis method based on contours in multiple coordinate systems as described above: an image data acquisition module for acquiring multiple sets of image data containing checkerboard corner points based on a measuring instrument array, wherein the measuring instrument array includes multiple 3D line laser contour measuring instruments arranged in an array; a corner point coordinate extraction module for extracting corresponding corner point coordinates from the multiple sets of image data based on a growing checkerboard corner point detection algorithm, and obtaining the detected corner point coordinates in the corresponding three-dimensional coordinate system through the multiple 3D line laser contour measuring instruments; a transformation matrix acquisition module for using the checkerboard corner points corresponding to the detected corner point coordinates as iteration points for point cloud registration, and converting the point cloud registration problem into an optimization problem, solving the optimization problem by minimizing the error between the target point cloud and the source point cloud to obtain a transformation matrix between multiple three-dimensional coordinate systems; and an adhesive application trajectory synthesis module for performing coordinate system one based on the transformation matrix to obtain the target coordinate system, and synthesizing the 3D adhesive application trajectory of the multiple array contour measuring instruments based on the target coordinate system.

[0038] Compared to existing technologies, the advantages and beneficial effects of this invention are as follows: Based on a measuring instrument array, multiple sets of image data containing checkerboard corner points are obtained. A growing checkerboard corner detection algorithm is used to extract the corresponding corner coordinates from the multiple sets of image data. The coordinates of the detected corner points in the corresponding three-dimensional coordinate system are obtained through multiple 3D line laser profile measuring instruments. The checkerboard corner points corresponding to the detected corner coordinates are used as iteration points for point cloud registration. The point cloud registration problem is transformed into an optimization problem, solved by minimizing the error between the target point cloud and the source point cloud. This yields a transformation matrix between multiple three-dimensional coordinate systems, avoiding the impact of coarse registration on the accuracy of the solution. The matrix serves as the initial value for subsequent optimization, improving computational accuracy and reducing the number of iterations. Coordinate system one is established based on the transformation matrix to obtain the target coordinate system. Then, 3D adhesive coating trajectory synthesis of the multi-array contour measuring instrument is performed based on this target coordinate system. Unifying the coordinate system using the transformation matrix fully considers camera distortion and positional information of the measuring instrument array, reducing optimization errors and further improving the accuracy of corner coordinates. This approach is applicable to trajectory synthesis in scenarios with overlapping point clouds, avoiding nonlinear optimization iterations and instead directly solving analytical solutions, thereby improving computational efficiency and achieving efficient and accurate point cloud registration, resulting in a more precise synthesized trajectory. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating a method for synthesizing 3D adhesive application trajectories based on contours in multiple coordinate systems, as described in one embodiment.

[0040] Figure 2 This is a schematic diagram of a 3D adhesive trajectory synthesis system based on contours in multiple coordinate systems, as shown in one embodiment. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0042] In one embodiment, such as Figure 1 As shown, a method for synthesizing 3D adhesive application trajectories based on contours in multiple coordinate systems is provided, including the following steps:

[0043] Step S101: Acquire multiple sets of image data containing checkerboard corner points based on the measuring instrument array, which includes multiple 3D line laser profile measuring instruments arranged in an array.

[0044] Specifically, a high-precision checkerboard is used to calibrate the 3D line laser profile measuring instrument. Multiple calibrated 3D line laser profile measuring instrument arrays are arranged at the object to be measured to obtain a measuring instrument array. A checkerboard calibration plate is set below the measuring instrument array, and the measuring instrument array can cover the entire area of ​​the checkerboard calibration plate. By moving the checkerboard calibration plate and using the measuring instrument array to perform measurements, multiple sets of image data containing checkerboard corner points are obtained.

[0045] Step S101 includes: calibrating multiple 3D line laser profile measuring instruments using a sample checkerboard calibration plate; the calibrated 3D line laser profile measuring instruments can scan multiple sets of point cloud profile data of the scanned object, and the multiple point cloud profile data have overlapping point cloud areas; arranging the calibrated multiple 3D line laser profile measuring instruments in an array at the object to be measured to obtain a measuring instrument array, and determining the corresponding checkerboard calibration plate according to the field of view of the measuring instrument array; placing the checkerboard calibration plate below the measuring instrument array, and ensuring that the 3D line laser profile measuring instruments in the measuring instrument array can capture the corner point areas of the checkerboard calibration plate, and that the measuring instrument array can cover the entire area of ​​the checkerboard calibration plate; moving and rotating the checkerboard calibration plate to obtain multiple sets of image data containing checkerboard corner points.

[0046] Specifically, a high-precision sample checkerboard calibration plate is used to first calibrate multiple 3D line laser profilometers, including two calibration processes: camera calibration and optical plane calibration. After calibration, point cloud data of the scanned object's contour can be obtained through laser scanning. For example, when using four 3D line laser profilometers, four sets of point cloud contour data in four different three-dimensional coordinate systems can be obtained. Since the four laser profilometers have overlapping viewpoints, there are also overlapping point cloud areas in the four sets of point cloud contour data.

[0047] Multiple calibrated 3D line laser profile measuring instruments are arranged at the object to be measured to obtain the corresponding measuring instrument array. For example, when using four 3D line laser profile measuring instruments, the four 3D line laser profile measuring instruments are set at the four positions of the object to be measured to form a measuring instrument array. At the same time, a high-precision checkerboard calibration plate of the corresponding size is selected according to the maximum field of view that the measuring instrument array can measure.

[0048] The selected checkerboard calibration plate is placed below the measuring array so that all 3D line laser profilometers in the array can capture images of some corner areas of the checkerboard calibration plate. It is not required that the checkerboard calibration plate be covered by a single measuring instrument, but all measuring instruments in the array must cover the entire area of ​​the checkerboard calibration plate to ensure that the array can obtain a complete image of the checkerboard calibration plate.

[0049] After setting up the measuring instrument array and the checkerboard calibration plate, multiple sets of images are captured by moving and rotating the checkerboard calibration plate. For example, four 3D line laser profile measuring instruments capture nine sets of images by moving and rotating the checkerboard calibration plate, resulting in 36 sets of image data containing checkerboard corner points.

[0050] Step S102: Based on the growth checkerboard corner detection algorithm, extract the corresponding corner coordinates from multiple sets of image data, and obtain the detection corner coordinates in the corresponding three-dimensional coordinate system through multiple 3D line laser profilometers.

[0051] Specifically, since mainstream camera calibration toolkits can only handle corner detection scenarios where the checkerboard edge color is fixed, the checkerboard size is specified in advance, and the image contains a complete checkerboard image, this invention is based on a growing checkerboard corner detection algorithm. It extracts the corresponding corner coordinates from multiple sets of image data and obtains the detection corner coordinates in the corresponding three-dimensional coordinate system through multiple 3D line laser profilometers. Thus, it can realize corner detection of images containing part of the checkerboard by growing checkerboard corners, so as to facilitate trajectory synthesis in point cloud overlapping scenarios.

[0052] Step S102 includes: constructing templates of different scales based on the characteristics of the checkerboard corner points; convolving multiple sets of image data with the templates and calculating the corner likelihood domain of each pixel to obtain the coarse corner point range; processing the coarse corner point range using a non-maximum suppression algorithm to obtain candidate corner points; performing sub-pixel refinement processing on the position and edge direction of the candidate corner points to obtain the sub-pixel refined corner point position and edge direction; growing the checkerboard by optimizing the energy function and solving the optimized energy function using a discretized optimization strategy to obtain the corresponding target checkerboard, and obtaining the corner point coordinates of all sub-pixels based on the target checkerboard; and obtaining the detected corner point coordinates in multiple three-dimensional coordinate systems using multiple calibrated 3D line laser profile measuring instruments.

[0053] Specifically, based on the characteristics of checkerboard corner points, templates of different scales are constructed, and these templates are convolved with multiple sets of image data. The corner likelihood domain of each pixel is calculated, and a large number of non-checkerboard corner points are eliminated to obtain the approximate corner range, which is denoted as the coarse corner range.

[0054] After obtaining the range of coarsely measured corner points, a non-maximum suppression algorithm is used to remove interference points. The maximum value is determined by searching for the maximum value within the range of coarsely measured corner points. The maximum value is then checked in the N*N region centered on the maximum value position to see if the point is the maximum value. If it is the maximum value, it is retained; otherwise, the loop continues until a preset number of maximum values ​​are obtained as candidate corner points.

[0055] To obtain more precise corner points, the corner point positions and edge directions of the candidate corner points are further refined to obtain the corresponding refined corner point positions and edge directions.

[0056] Through the above steps, corner points with sub-pixel precision can be obtained from multiple sets of image data. Since the image data only contains a portion of the checkerboard area and contains an irregular number of corner points, the checkerboard can be grown by optimizing the energy function and solved using a discretization optimization strategy to obtain the corresponding target checkerboard. Thus, all sub-pixel corner points can be obtained based on the target checkerboard.

[0057] By using a growth-based checkerboard corner detection algorithm, the sub-pixel coordinates of a corresponding number of corner points in a partially checkerboard image can be obtained. Then, using a 3D line laser profile measuring instrument that has been calibrated, the coordinates of these corner points in multiple different three-dimensional coordinate systems can be obtained.

[0058] In one embodiment, the sub-pixel refinement process specifically involves: Let c be an ideal corner point, p be a pixel within the neighborhood of c, and g... pLet p be the gradient vector; minimize the following expression within the neighborhood N(c′) of the candidate corner point:

[0059]

[0060] Solving the above equation yields an analytical solution, which provides the refined corner point positions. The analytical solution to the above equation is:

[0061]

[0062] The refined edge direction is obtained by minimizing the following formula;

[0063]

[0064] In the formula, e1 and e2 represent the edge directions.

[0065] Specifically, since we obtain candidate corner points, we can further refine them through sub-pixel refinement to obtain more precise corner points. Let c be the ideal corner point, p be a pixel in the neighborhood of c, and g... p Let p be the gradient vector at point p, then we should have However, in reality, the edges are not so sharp, and the gradient direction is not so ideal. Therefore, we consider minimizing equation (2) in the neighborhood N(c′) of the candidate corner point and calculate the corresponding analytical solution to obtain the refined corner point position; further refine the edge direction of the candidate corner point to obtain the sub-pixel refined corner point position and edge direction.

[0066] In one embodiment, the step of growing a chessboard by optimizing the energy function specifically involves defining the energy function of the chessboard as follows:

[0067]

[0068] In the formula, E corners E is the negative of the total number of corner points in the current chessboard. struct It describes how to predict the matching degree of a third corner point using two adjacent corner points; calculate the structural energy of each row and column corner point in the chessboard, compare the structural energies, obtain the maximum structural energy, and use the maximum structural energy as the structural energy of the chessboard.

[0069] Specifically, after subpixel refinement of the candidate corner points, since the image data only contains a portion of the checkerboard area, there will be an irregular number of corner points. Therefore, it is necessary to grow the checkerboard by optimizing the energy function so that the irregular corner points can be fully represented in the grown checkerboard, making it easier to obtain the corresponding three-dimensional coordinates.

[0070] By using the energy function of the chessboard, the structural energy of all corner points in each row and column of the chessboard is calculated. The obtained structural energies are compared, and the maximum structural energy is obtained and taken as the structural energy of the chessboard.

[0071] In one embodiment, the steps of solving the optimization energy function, obtaining the corresponding target chessboard, and obtaining the corner coordinates of all sub-pixels based on the target chessboard are as follows: Given a seed corner point, search along the edge direction of the seed corner point to obtain an initialized seed chessboard; based on the seed chessboard, grow chessboards from the four outermost edges to generate new pre-selected chessboards; detect the minimum energy value among all pre-selected chessboards; if the minimum energy value is less than the energy value of the original chessboard, the growth is successful, and the pre-selected chessboard corresponding to the minimum energy value replaces the original chessboard; continue growing until the energy value of the pre-selected chessboard no longer decreases, then obtain the final target chessboard, and obtain the corners of all sub-pixels based on the target chessboard.

[0072] Specifically, since the computational load is exponentially related to the size of the chessboard, the computational load is relatively large when the size of the chessboard is large. Therefore, a discretization optimization strategy can be used to solve the optimization energy function to improve computational efficiency.

[0073] Given a seed corner point, an initial seed chessboard is obtained by searching along its edge direction. For example, the seed chessboard has 3×3 corner points and a 2×2 chessboard. Based on the generated seed chessboard, chessboards are grown from the four outermost edges to generate four new pre-selected chessboards. If the minimum energy value in the new generator chessboard is less than the energy of the chessboard before expansion, the growth is successful, and the pre-selected chessboard corresponding to the minimum energy value replaces the original chessboard. Growth continues until the energy of the pre-selected chessboards in the four directions no longer decreases, and the corresponding target chessboard is obtained. Thus, all sub-pixel corner points are obtained through the target chessboard.

[0074] In one embodiment, the step of obtaining the corner coordinates specifically involves: obtaining the sub-pixel coordinates of the corner points in multiple sets of image data, and obtaining the coordinates of the corresponding corner points in multiple three-dimensional coordinate systems using multiple calibrated 3D line laser profilometers, denoted as:

[0075]

[0076] Specifically, by using a growth-based checkerboard corner detection algorithm, the sub-pixel coordinates of corner points in multiple sets of image data are detected. Then, the coordinates of the corresponding corner points in multiple different three-dimensional coordinate systems can be obtained through multiple 3D line laser profile measuring instruments that have been calibrated, i.e., the corner point coordinates are detected.

[0077] Step S103: The checkerboard corner points corresponding to the detected corner coordinates are used as iteration points for point cloud registration. The point cloud registration problem is transformed into an optimization problem. The optimization problem is solved by minimizing the error between the target point cloud and the source point cloud, and the transformation matrix between multiple three-dimensional coordinate systems is obtained.

[0078] Specifically, based on the coordinates of detected corner points in multiple different 3D coordinate systems, the corresponding checkerboard corner points are obtained, and these checkerboard corner points are used as iteration points for point cloud registration. The transformation matrix between multiple 3D coordinate systems is solved to unify the coordinate systems and unify multiple 3D coordinate systems into one coordinate system, which facilitates subsequent trajectory synthesis.

[0079] When performing point cloud registration, the point cloud registration problem is transformed into an optimization problem based on the least squares algorithm. This optimization problem has an analytical solution, and if a unique solution is found, the minimum solution is the global optimum. This avoids the impact of coarse registration on the accuracy of the solution. Instead, a more accurate transformation matrix is ​​used as the initial value for subsequent optimization, improving computational accuracy and reducing the number of iterations. Furthermore, the corner coordinates are standardized based on the transformation matrix, fully considering the positional information of the measuring instrument array, reducing optimization errors, and further improving the accuracy of the corner coordinates.

[0080] In one embodiment, the point cloud registration step in step S104 specifically involves: transforming the point cloud registration problem into an optimization problem, the optimization objective of which is:

[0081]

[0082] Transforming the above equation into vector form, we have:

[0083]

[0084] In the formula, p s ,p t p represents the corresponding points between the source point cloud and the target point cloud. i ,p′ i Let represent the i-th point in the target point cloud and the source point cloud, respectively; define the centroids of the two sets of points in the target point cloud and the source point cloud as:

[0085]

[0086] The error between the source and target point clouds is obtained based on the centroid and corresponding points. This error is then processed using an error function to solve for the transformation matrix between multiple 3D coordinate systems. The error function is:

[0087]

[0088] Specifically, after obtaining the coordinates of the detected corner points, the corresponding checkerboard corner points are used as iteration points for point cloud registration. The point cloud registration problem is then transformed into an optimization problem. By minimizing the error between the target point cloud and the source point cloud, the optimization objective is solved, and a set of optimal values ​​for r is found based on the error function. * ,t * , so that ζ (r,t) The minimum value is obtained, thus enabling the transformation matrix between multiple three-dimensional coordinate systems. This facilitates the unification of corner coordinates based on the transformation matrix, thereby enabling trajectory synthesis.

[0089] Step S104: Based on the transformation matrix, coordinate system one is performed to obtain the target coordinate system, and the 3D adhesive coating trajectory of the multi-array profile measuring instrument is synthesized based on the target coordinate system.

[0090] Specifically, by unifying the coordinate systems of multiple 3D coordinate systems based on the obtained transformation matrix, the adverse effects of camera distortion and position information of the measuring instrument array are reduced, optimization errors are reduced, and the accuracy of corner coordinates is further improved. It can also synthesize the 3D adhesive coating trajectory of the multi-array contour measuring instrument based on the target coordinate system, thereby avoiding the defects of the existing IPC and its improved algorithms. It is applicable to trajectory synthesis in scenarios with overlapping point clouds, and the trajectory synthesis efficiency and accuracy are high.

[0091] In this embodiment, multiple sets of image data containing checkerboard corner points are obtained based on a measuring instrument array. A growing checkerboard corner detection algorithm is used to extract the corresponding corner coordinates from the multiple sets of image data. The coordinates of the detected corner points in the corresponding three-dimensional coordinate system are obtained through multiple 3D line laser profilometers. The checkerboard corner points corresponding to the detected corner coordinates are used as iteration points for point cloud registration. The point cloud registration problem is transformed into an optimization problem, which is solved by minimizing the error between the target point cloud and the source point cloud. The transformation matrix between multiple three-dimensional coordinate systems is obtained, avoiding the impact of coarse registration on the accuracy of the solution. The transformation matrix is ​​used for subsequent optimization. The initial values ​​are optimized to improve computational accuracy and reduce the number of iterations. Coordinate system one is performed based on the transformation matrix to obtain the target coordinate system. Then, 3D adhesive coating trajectory synthesis of the multi-array contour measuring instrument is performed based on the target coordinate system. The corner coordinates are unified based on the transformation matrix, fully considering camera distortion and position information of the measuring instrument array, reducing optimization errors, and further improving the accuracy of corner coordinates. It is also applicable to trajectory synthesis in scenarios with overlapping point clouds. It avoids nonlinear optimization iteration and directly solves analytical solutions, thereby improving computational efficiency and achieving efficient and accurate point cloud registration, resulting in a more precise synthesized trajectory.

[0092] like Figure 2As shown, a 3D adhesive trajectory synthesis system 20 based on contours in multiple coordinate systems is provided, which can be used to implement the aforementioned 3D adhesive trajectory synthesis method based on contours in multiple coordinate systems. It includes: an image data acquisition module 21, a corner coordinate extraction module 22, a transformation matrix acquisition module 23, and an adhesive trajectory synthesis module 24, wherein:

[0093] Image data acquisition module 21 is used to acquire multiple sets of image data containing checkerboard corner points based on the measuring instrument array, which includes multiple 3D line laser profile measuring instruments arranged in an array.

[0094] The corner coordinate extraction module 22 is used for the growth-based checkerboard corner detection algorithm to extract the corresponding corner coordinates from multiple sets of image data, and obtain the detected corner coordinates in the corresponding three-dimensional coordinate system through multiple 3D line laser profile measuring instruments.

[0095] The transformation matrix acquisition module 23 is used to take the checkerboard corner points corresponding to the detected corner point coordinates as iteration points, perform point cloud registration, and transform the point cloud registration problem into an optimization problem. The optimization problem is solved by minimizing the error between the target point cloud and the source point cloud to obtain the transformation matrix between multiple three-dimensional coordinate systems.

[0096] The adhesive application trajectory synthesis module 24 is used to perform coordinate system one based on the transformation matrix to obtain the target coordinate system, and to perform 3D adhesive application trajectory synthesis of the multi-array profile measuring instrument based on the target coordinate system.

[0097] In one embodiment, the corner coordinate extraction module 22 is specifically used for: constructing templates of various scales based on the characteristics of the checkerboard corners; convolving multiple sets of image data with the templates and calculating the corner likelihood domain of each pixel to obtain the coarse measurement corner range; processing the coarse measurement corner range using a non-maximum suppression algorithm to obtain candidate corners; performing sub-pixel refinement processing on the position and edge direction of the candidate corners to obtain the sub-pixel refined corner position and edge direction in multiple sets of image data; growing the checkerboard by optimizing the energy function and solving the optimized energy function using a discretized optimization strategy to obtain the corresponding target checkerboard, and obtaining the corner coordinates of all sub-pixels based on the target checkerboard; and obtaining the detected corner coordinates in multiple three-dimensional coordinate systems using multiple calibrated 3D line laser profile measuring instruments.

[0098] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0099] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a computer storage medium (ROM / RAM, magnetic disk, optical disk) for execution by the computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Therefore, the present invention is not limited to any particular hardware and software combination.

[0100] The above description, in conjunction with specific embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such deductions or substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method for synthesizing 3D adhesive application trajectories based on contours in multiple coordinate systems, characterized in that, Includes the following steps: The measurement instrument array acquires multiple sets of image data containing checkerboard corner points, and the measurement instrument array includes multiple 3D line laser profile measuring instruments arranged in an array. The corner detection algorithm based on growth chessboard extracts the corresponding corner coordinates from the multiple sets of image data, and obtains the detected corner coordinates in the corresponding three-dimensional coordinate system through the multiple 3D line laser profilometers. Using the checkerboard corner points corresponding to the detected corner coordinates as iteration points, point cloud registration is performed. The point cloud registration problem is then transformed into an optimization problem. This optimization problem is solved by minimizing the error between the target point cloud and the source point cloud, resulting in transformation matrices between multiple 3D coordinate systems. The optimization objective of this optimization problem is: Transforming the above equation into vector form, we have: In the formula, p s ,p t p represents the corresponding points between the source point cloud and the target point cloud. i ,p′ i Let i and j represent the i-th point in the target point cloud and the source point cloud, respectively. Define the centroids of the two sets of points in the target point cloud and the source point cloud as: The error between the source point cloud and the target point cloud is obtained based on the centroid and corresponding point, and then processed through an error function to obtain the transformation matrix between multiple three-dimensional coordinate systems. The error function is: Based on the transformation matrix, coordinate system one is performed to obtain the target coordinate system, and the 3D adhesive coating trajectory of the multi-array contour measuring instrument is synthesized based on the target coordinate system.

2. The method for synthesizing 3D adhesive application trajectories based on contours in multiple coordinate systems according to claim 1, characterized in that, The acquisition of multiple sets of image data containing checkerboard corner points based on the measuring instrument array includes: Multiple 3D line laser profile measuring instruments are calibrated using a sample checkerboard calibration plate. After calibration, multiple 3D line laser profile measuring instruments can scan and obtain multiple sets of point cloud profile data of the scanned object, and the multiple sets of point cloud profile data have overlapping point cloud regions. The calibrated array of multiple 3D line laser profile measuring instruments is arranged on the object to be measured to obtain the measuring instrument array, and the corresponding checkerboard calibration plate is determined according to the field of view of the measuring instrument array. The checkerboard calibration plate is placed below the measuring instrument array, and the 3D line laser profile measuring instrument in the measuring instrument array can capture the corner area of ​​the checkerboard calibration plate. The measuring instrument array can cover the entire area of ​​the checkerboard calibration plate. Move and rotate the chessboard calibration plate to obtain multiple sets of image data containing the corner points of the chessboard.

3. The method for synthesizing 3D adhesive application trajectories based on contours in multiple coordinate systems according to claim 1, characterized in that, The growth-based checkerboard corner detection algorithm extracts the corresponding corner coordinates from the multiple sets of image data and obtains the detected corner coordinates in the corresponding three-dimensional coordinate system using the multiple 3D line laser profilometers, including: Based on the characteristics of the corner points of the chessboard, templates of different scales are constructed. The multiple sets of image data are convolved with the templates, and the corner likelihood domain of each pixel is calculated to obtain the coarse measurement range of the corner points. The range of the coarsely measured corner points is processed using a non-maximum suppression algorithm to obtain candidate corner points; The positions and edge directions of the candidate corner points are refined to obtain the refined corner point positions and edge directions in the multiple sets of image data; The chessboard grid is grown by optimizing the energy function, and the optimization energy function is solved by a discretization optimization strategy to obtain the corresponding target chessboard grid. The corner coordinates of all sub-pixels are obtained based on the target chessboard grid. By using multiple calibrated 3D line laser profile measuring instruments, the coordinates of the detected corner points in multiple three-dimensional coordinate systems are obtained.

4. The method for synthesizing 3D adhesive application trajectories based on contours in multiple coordinate systems according to claim 3, characterized in that, The sub-pixel refinement processing of the candidate corner points' positions and edge directions to obtain the sub-pixel refined corner point positions and edge directions in the multiple sets of image data includes: Let c be an ideal corner point, p be a pixel in the neighborhood of c, and g p Let p be the gradient vector; Minimize the following expression within the neighborhood N(c′) of the candidate corner point: Solving the above equation yields an analytical solution, which provides the refined corner point positions. The analytical solution to the above equation is: The refined edge direction is obtained by minimizing the following formula; In the formula, e1 and e2 represent the edge directions.

5. The method for synthesizing 3D adhesive application trajectories based on contours in multiple coordinate systems according to claim 3, characterized in that, The method of growing a chessboard grid by optimizing the energy function includes: The energy function of the chessboard is defined as follows: In the formula, E corners E is the negative of the total number of corner points in the current chessboard. struct It describes how to predict the matching degree of a third corner point using two adjacent corner points; Calculate the structural energy of each row and each column corner point in the chessboard, compare the structural energies, obtain the maximum structural energy, and take the maximum structural energy as the structural energy of the chessboard.

6. The method for synthesizing 3D adhesive application trajectories based on contours in multiple coordinate systems according to claim 3, characterized in that, The process employs a discretized optimization strategy to solve the optimization energy function, obtains the corresponding target chessboard grid, and then obtains the corner coordinates of all sub-pixels based on the target chessboard grid, including: Given a seed corner point, search along the edge direction of the seed corner point to obtain an initialized seed chessboard grid; Based on the seed chessboard, new pre-selected chessboards are generated by growing chessboards from the four outermost edges. The minimum energy value among all pre-selected chessboard squares is detected. If the minimum energy value is less than the energy value of the original chessboard square, the growth is successful, and the original chessboard square is replaced by the pre-selected chessboard square corresponding to the minimum energy value. Continue growing until the energy value of the pre-selected chessboard no longer decreases, then obtain the final target chessboard, and obtain the corner points of all sub-pixels based on the target chessboard.

7. The method for synthesizing 3D adhesive application trajectories based on contours in multiple coordinate systems according to claim 3, characterized in that, The process of obtaining the coordinates of the detected corner points in multiple three-dimensional coordinate systems using calibrated 3D line laser profile measuring instruments includes: Obtain the sub-pixel coordinates of corner points from multiple sets of image data, and use multiple calibrated 3D line laser profilometers to obtain the coordinates of the corresponding corner points in multiple three-dimensional coordinate systems, denoted as: 。 8. A 3D adhesive application trajectory synthesis system based on contours in multiple coordinate systems, characterized in that, A method for synthesizing 3D adhesive application trajectories based on contours in multiple coordinate systems, as described in any one of claims 1-7, includes: The image data acquisition module is used to acquire multiple sets of image data containing checkerboard corner points based on the measuring instrument array, wherein the measuring instrument array includes multiple 3D line laser profile measuring instruments arranged in an array. The corner coordinate extraction module is used to extract the corresponding corner coordinates from the multiple sets of image data based on the growth checkerboard corner detection algorithm, and obtain the detected corner coordinates in the corresponding three-dimensional coordinate system through the multiple 3D line laser profile measuring instruments. The transformation matrix acquisition module is used to take the checkerboard corner points corresponding to the detected corner point coordinates as iteration points, perform point cloud registration, and transform the point cloud registration problem into an optimization problem. The optimization problem is solved by minimizing the error between the target point cloud and the source point cloud to obtain the transformation matrix between multiple three-dimensional coordinate systems. The adhesive application trajectory synthesis module is used to perform coordinate system one based on the transformation matrix to obtain the target coordinate system, and to perform 3D adhesive application trajectory synthesis of the multi-array profile measuring instrument based on the target coordinate system.

Images