Workpiece positioning method based on Apriltag and point cloud matching

Abstract

The invention relates to the technical field of target positioning, in particular to a workpiece positioning method based on Apriltag and point cloud matching. The method comprises the following steps: calibrating internal parameters and external parameters of a plurality of cameras; carrying out distortion removal processing on the image; carrying out Apriltag identification on the image and extracting key points; resolving the key points by using a multi-view solid geometry principle, and determining coordinates of the key points in a world coordinate system; optimizing the positioning precision of the key points and generating a final source point cloud; acquiring a geometric center of the standard target point cloud as an original point of a workpiece coordinate system; performing first point cloud registration on the source point cloud and the target point cloud by using a point-plane ICP algorithm; certain disturbance is added to the first registration result, and second point cloud registration is carried out; and calculating and obtaining a final workpiece positioning result. According to the method, high-precision and stable workpiece positioning is realized, and higher robustness and applicability are shown when a large-range workpiece with weaker textures is processed.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-view stereo geometry and target positioning, and more specifically to a workpiece positioning method based on Apriltag label and point cloud matching. Background Art

[0002] High-precision workpiece positioning technology plays a vital role in industrial inspection, robot navigation, 3D reconstruction and other fields. Especially in intelligent manufacturing and automated production processes, accurate workpiece positioning is the core element to achieve high efficiency and precise operation.

[0003] Traditional positioning methods usually rely on a single visual sensor or laser scanning technology, however, these methods often have limitations when dealing with large-scale environments or target surfaces with weak textures. For example, when the visual sensor's viewing angle is limited or the lighting conditions are poor, its positioning accuracy and reliability are greatly reduced.

[0004] Apriltag is a widely used visual tagging system that includes a series of QR codes with complex black and white patterns that are carefully designed to be quickly and accurately recognized and decoded by computer vision algorithms. By deploying multiple cameras in different environments to capture images containing Apriltag, the three-dimensional coordinates of the tag can be calculated from different angles using multi-view geometry technology. This approach not only improves the comprehensiveness and accuracy of positioning, but also overcomes the viewing angle and distance limitations of a single visual system. Despite this, multi-view geometry methods still need to face technical challenges such as image distortion and environmental noise. Summary of the invention

[0005] The purpose of the present invention is to provide a workpiece positioning method and system based on Apriltag label and point cloud matching, so as to solve the problem of poor positioning accuracy of workpieces in complex environments in the prior art.

[0006] In order to achieve the above object, the present invention provides a workpiece positioning method based on Apriltag label and point cloud matching, comprising the following steps:

[0007] Step S1, calibrating the intrinsic and extrinsic parameters of multiple cameras to obtain camera intrinsic and extrinsic parameter data;

[0008] Step S2, performing distortion removal processing on the image, wherein all pixel information is retained;

[0009] Step S3, performing April tag recognition on the image and extracting key points, wherein the key points include the center point and four corner points of each tag label;

[0010] Step S4, using the principle of multi-view stereo geometry, solving the extracted key points to determine the coordinates of the key points in the world coordinate system;

[0011] Step S5, optimizing and improving the positioning accuracy of key points by using a nonlinear optimization method, and generating a final source point cloud based on the optimization result;

[0012] Step S6, reading the standard target point cloud, and performing rotation and translation operations to obtain the geometric center of the standard target point cloud as the origin of the workpiece coordinate system;

[0013] Step S7, using the point-to-surface ICP algorithm to perform a first point cloud registration on the source point cloud and the target point cloud to obtain a first registration result;

[0014] Step S8, adding a certain disturbance to the first registration result, and performing a second point cloud registration with the target point cloud to obtain a second registration result;

[0015] Step S9, calculating the conversion relationship between the origin of the workpiece coordinate system and the world coordinate system, thereby obtaining the final workpiece positioning result.

[0016] In one embodiment, the step S1 further comprises:

[0017] Step S11: for each camera, a checkerboard calibration plate is placed at different viewing angles of the camera to collect multiple image data, calculate the feature points of the checkerboard in each image, and use the homography of the checkerboard to calculate the homography matrix H of each image. i , solve the corresponding symmetric matrix B, and then solve the camera's intrinsic parameter matrix A;

[0018] Step S12, using a checkerboard calibration plate to collect multiple image data, taking the origin of the calibration plate as the origin of the world coordinate system, and obtaining the relative position transformation relationship from the world coordinate system to each camera, that is, the camera's external parameter T i ;

[0019] In step S13, the 3D coordinates of the feature points obtained in step S11 are back-projected onto the pixel plane, compared with the positions of the feature points actually identified in each image and the difference is calculated, so as to obtain the reprojection error of the feature points in the corresponding camera, and the error function is constructed using the Levenberg-Marquardt algorithm in nonlinear optimization, and the final camera internal and external parameters are obtained by minimizing the reprojection error.

[0020] In one embodiment, the step S2 further comprises:

[0021] Step S21, calculating the theoretical boundary of the image after distortion correction according to the intrinsic parameter matrix obtained in step S1 and the distortion coefficient, estimating a new field of view based on the theoretical boundary, adjusting the focal length and the principal point position according to the new field of view, thereby generating a new camera intrinsic parameter matrix;

[0022] Step S22: construct a distortion map for each pixel based on the new intrinsic parameter matrix and in combination with the distortion coefficient, and use the distortion map to resample the original image to ultimately generate a corrected image.

[0023] In one embodiment, the step S3 further comprises:

[0024] Step S31, performing binarization processing on the image;

[0025] Step S32, identifying and numbering connected domains in the binarized image;

[0026] Step S33, detecting quadrilaterals in the entire image, and obtaining the specific coordinates of the four corner points and the center point of each corresponding tag in the image;

[0027] Step S34, decoding the two-dimensional code in the quadrilateral area to obtain the coding information contained in the tag label.

[0028] In one embodiment, the step S31 further comprises:

[0029] The image is binarized using an adaptive threshold segmentation algorithm.

[0030] In one embodiment, the step S32 further comprises:

[0031] The connected domains in the binarized image are identified and numbered using the union-find algorithm.

[0032] In one embodiment, the step S4 further comprises:

[0033] Step S41, for each key point, collect all cameras that observe the key point, and store the collected camera information in a container;

[0034] Step S42, removing camera information whose reprojection error is greater than a certain range from the container;

[0035] Step S43, using a multi-camera triangulation method, all key points are measured and solved to obtain the coordinates of the key points in the world coordinate system.

[0036] In one embodiment, the step S5 further comprises:

[0037] Different weights are assigned to each camera according to the distance between the camera and the key point. A nonlinear optimization problem is constructed by using the reprojection error of the key point on the pixel screen. The key point coordinates are optimized by solving the nonlinear optimization problem.

[0038] In one embodiment, the step S7 further comprises:

[0039] Step S71, using a nearest neighbor algorithm to match reference points in the source point cloud with data points in the target point cloud;

[0040] Step S72, analyzing the data points in the target point cloud by a plane fitting algorithm to construct a plane model;

[0041] Step S73: Use an optimization algorithm to solve the rigid body transformation parameters between the source point cloud and the target point cloud to obtain the first registration result.

[0042] In one embodiment, the step S73 further comprises:

[0043] The Levenberg-Marquardt nonlinear optimization algorithm is used to solve the rigid body transformation parameters between the source point cloud and the target point cloud.

[0044] In one embodiment, the step S8 further comprises:

[0045] Step S81, adding disturbance to the source point cloud after the first step of registration;

[0046] Step S82, using the point-to-surface ICP algorithm to match the source point cloud with the target point cloud;

[0047] Step S83: Select the match with the smallest registration error from all the matching results as the final second registration result.

[0048] In one embodiment, in step S9, the final workpiece positioning result T is achieved by the following formula:

[0049] T=(T3T2T1) -1 ;

[0050] Among them, T1 is the first registration result of step S7, T2 is the rigid body transformation disturbance added during the second matching in step S8, and T3 is the second registration result of step S8.

[0051] In order to achieve the above object, the present invention provides a workpiece positioning system based on Apriltag label and point cloud matching, comprising:

[0052] a memory for storing instructions executable by a processor;

[0053] A processor is used to execute the instructions to implement any of the methods described above.

[0054] In order to achieve the above object, the present invention provides a computer-readable medium on which computer instructions are stored, wherein when the computer instructions are executed by a processor, any of the above methods is executed.

[0055] The present invention proposes a workpiece positioning method and system based on Apriltag label and point cloud matching. By combining multi-view stereo geometry technology, nonlinear optimization algorithm and point cloud registration technology, high-precision and stable workpiece positioning is achieved. It shows higher robustness and applicability when processing large-scale workpieces with weak textures. It is particularly suitable for complex industrial environments and plays an important role in improving the level of industrial automation and the autonomous operation capability of robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The above and other features, properties and advantages of the present invention will become more apparent through the following description in conjunction with the accompanying drawings and embodiments, in which the same reference numerals always represent the same features, wherein:

[0057] Figure 1 A flow chart of a workpiece positioning method based on Apriltag label and point cloud matching according to an embodiment of the present invention is disclosed;

[0058] Figure 2 A schematic diagram of camera internal and external parameter calibration according to an embodiment of the present invention is disclosed;

[0059] Figure 3 A key point positioning result diagram according to an embodiment of the present invention is disclosed;

[0060] Figure 4 A comparison diagram of point clouds generated by a positioning method according to an embodiment of the present invention and a standard model is disclosed;

[0061] Figure 5 A principle block diagram of a workpiece positioning system based on Apriltag label and point cloud matching according to an embodiment of the present invention is disclosed. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the invention and are not used to limit the invention.

[0063] The present invention proposes a workpiece positioning method and system based on Apriltag label and point cloud matching, which is mainly used in the fields of industrial inspection, workpiece positioning and three-dimensional reconstruction. In particular, it can achieve high-precision positioning of weak-textured workpieces in complex large scenes, which has a significant driving significance for improving the level of industrial automation and enhancing the autonomous operation capability of robots.

[0064] Figure 1 A flow chart of a workpiece positioning method based on Apriltag label and point cloud matching according to an embodiment of the present invention is disclosed, such as Figure 1 As shown, the workpiece positioning method based on Apriltag label and point cloud matching proposed in the present invention includes the following steps:

[0065] Step S1, calibrating the intrinsic and extrinsic parameters of multiple cameras to obtain camera intrinsic and extrinsic parameter data;

[0066] Step S2, performing distortion removal processing on the image, wherein all pixel information is retained;

[0067] Step S3, performing April tag recognition on the image and extracting key points, wherein the key points include the center point and four corner points of each tag label;

[0068] Step S4, using the principle of multi-view stereo geometry, solving the extracted key points to determine the coordinates of the key points in the world coordinate system;

[0069] Step S5, optimizing and improving the positioning accuracy of key points by using a nonlinear optimization method, and generating a final source point cloud based on the optimization result;

[0070] Step S6, reading the standard target point cloud, and performing rotation and translation operations to obtain the geometric center of the standard target point cloud as the origin of the workpiece coordinate system;

[0071] Step S7, using the point-to-surface ICP algorithm to perform a first point cloud registration on the source point cloud and the target point cloud to obtain a first registration result;

[0072] Step S8, adding a certain disturbance to the first registration result, and performing a second point cloud registration with the target point cloud to obtain a second registration result;

[0073] Step S9, calculating the conversion relationship between the origin of the workpiece coordinate system and the world coordinate system, thereby obtaining the final workpiece positioning result.

[0074] The present invention proposes a workpiece positioning method based on matching Apriltag labels with point clouds, performs necessary distortion correction on the image, and uses nonlinear optimization technology to further improve the positioning accuracy of key points, thereby generating an accurate source point cloud.

[0075] In addition, this method also integrates point cloud registration technology. First, the point-to-surface ICP (Iterative Closest Point) algorithm is used for preliminary registration, and then disturbance is introduced for secondary registration to ensure the best workpiece positioning effect.

[0076] Compared with traditional methods, this method combines multi-view stereo geometry and point cloud registration technology to show higher robustness and applicability when processing large-scale and weakly textured workpieces. This feature makes it particularly suitable for complex industrial environments and can effectively improve the accuracy and efficiency of workpiece positioning.

[0077] These steps will be described in detail below. It should be understood that within the scope of the present invention, the above-mentioned technical features of the present invention and the technical features specifically described below (such as embodiments) can be combined and related to each other to form a preferred technical solution.

[0078] Step S1, calibrating the intrinsic and extrinsic parameters of multiple cameras to obtain camera intrinsic and extrinsic parameter data.

[0079] In this embodiment, the calibration toolbox tool of MATLAB software can be used to calibrate the camera intrinsic parameters, and the C++ software can be used to calibrate the camera extrinsic parameters, and the nonlinear optimization library ceres can be used to optimize these camera intrinsic and extrinsic parameters.

[0080] More specifically, the step S1 further comprises:

[0081] Step S11: for each camera, a checkerboard calibration plate is placed at different viewing angles of the camera to collect multiple image data, calculate the feature points of the checkerboard in each image, and use the homography of the checkerboard to calculate the homography matrix H of each image. i , solve the corresponding symmetric matrix B, and then solve the camera's intrinsic parameter matrix A;

[0082] In this embodiment, a standard high-precision checkerboard calibration plate is used.

[0083] Among them, H i The relationship with B is realized by the following formula:

[0084]

[0085] Among them, H i =[h1,h2,h3] i ,h i =[h i1 ,h i2 ,h i3 ]T , b=[B 11 ,B 12 ,B 22 ,B 13 ,B 23 ,B 33 ] T , v ij =[h i1 h j1 ,h i1 h j2 +h i2 h j1 ,h i2 h j2 ,h i3 h j1 +h i1 h j3 ,h i3 h j2 +h i2 h j3 ,h i3 h j3 ] T ;

[0086] B ij is the element in row i and column j in B.

[0087] The relationship between B and A is achieved through the following formula:

[0088]

[0089] γ=-B 12 α 2 β / λ

[0090] u0=γv0 / β-B 13 α 2 / λ.

[0091] in,

[0092] Step S12, using a checkerboard calibration plate to collect multiple image data, taking the origin of the calibration plate as the origin of the world coordinate system, and obtaining the relative position transformation relationship from the world coordinate system to each camera, that is, the camera's external parameter T i ;

[0093] In this embodiment, a specially made large chessboard calibration plate (10×10 m) is used to collect images.

[0094] The camera's external parameter T i , achieved through the following formula:

[0095]

[0096] Among them, [r1 r2 r3] are the first three columns of the extrinsic rotation matrix, and t is the translation component of the extrinsic parameter, that is, the extrinsic parameter T of camera i. i =A[r1 r2 r3 t]; [h1 h2 h3]=λA[r1 r2 t].

[0097] In step S13, the 3D coordinates of the feature points obtained in step S11 are back-projected onto the pixel plane, compared with the positions of the feature points actually identified in each image and the difference is calculated, so as to obtain the reprojection error of the feature points in the corresponding camera, and the error function is constructed using the Levenberg-Marquardt algorithm in nonlinear optimization, and the final camera internal and external parameters are obtained by minimizing the reprojection error.

[0098] The constructed error function is implemented by the following formula:

[0099]

[0100] Among them, M j is the 3D coordinate of the feature point, and the relationship between k1, k2 and the intrinsic parameter is:

[0101]

[0102] Step S2, performing distortion removal processing on the image, wherein all pixel information is retained.

[0103] Image distortion removal that retains all pixel information to ensure image information is complete.

[0104] Furthermore, the step S2 further comprises:

[0105] Step S21, calculating the theoretical boundary of the image after distortion correction according to the intrinsic parameter matrix obtained in step S1 and the distortion coefficient, estimating a new field of view based on the theoretical boundary, adjusting the focal length and the principal point position according to the new field of view, thereby generating a new camera intrinsic parameter matrix;

[0106] Step S22, based on the new intrinsic parameter matrix, a distortion map is constructed for each pixel in combination with the distortion coefficient, the distortion map specifies the corrected position of each pixel in the image, and the original image is resampled using the distortion map to finally generate a corrected image.

[0107] Furthermore, in step S22, the distortion map is used to resample the original image by an interpolation method to generate a corrected image.

[0108] Step S3, perform April tag recognition on the image and extract key points, wherein the key points include the center point and four corner points of each tag label.

[0109] Furthermore, the step S3 further comprises:

[0110] Step S31, performing binarization processing on the image;

[0111] Step S32, identifying and numbering connected domains in the binarized image;

[0112] Step S33, detecting quadrilaterals in the entire image, and obtaining the specific coordinates of the four corner points and the center point of each corresponding tag in the image;

[0113] Step S34, decoding the two-dimensional code in the quadrilateral area to obtain the coding information contained in the tag label for subsequent point cloud registration.

[0114] Furthermore, the step S31 further comprises:

[0115] The image is binarized using an adaptive threshold segmentation algorithm.

[0116] The adaptive threshold segmentation algorithm is implemented by the following steps:

[0117] First, according to the brightness variation characteristics of each small area in the image, the local optimal threshold of each area is calculated;

[0118] Then, these local optimal thresholds are applied to the corresponding regions respectively to perform image binarization processing.

[0119] After such processing, the image can clearly highlight the edges and details of the Apriltag label under different lighting conditions, providing optimized input for subsequent label detection and feature point extraction.

[0120] Furthermore, the step S32 further comprises:

[0121] The connected domains in the binarized image are quickly identified and numbered using the union-find algorithm.

[0122] The union-find algorithm, also known as the disjoint-set algorithm, is a data structure and its corresponding algorithm used to manage and merge sets. Through merging and finding operations, it can quickly check and manage sets and dynamically adjust them when needed.

[0123] The union-find algorithm is implemented by the following steps:

[0124] First, assign a unique identifier to each pixel in the image;

[0125] Next, scan the image and merge adjacent pixels with the same color into the same connected domain;

[0126] Finally, all pixels are traversed to ensure that all pixels in each connected domain have the same root identifier, thereby completing the identification and numbering of the connected domain.

[0127] Step S4, using the multi-view stereo geometry principle, solve the extracted key points to determine the coordinates of the key points in the world coordinate system.

[0128] Furthermore, the step S4 further comprises:

[0129] Step S41, for each key point, collect all cameras that observe the key point, and store the collected camera information in a container;

[0130] Step S42, removing camera information whose reprojection error is greater than a certain range from the container;

[0131] Step S43, using a multi-camera triangulation method, all key points are measured and solved to obtain the coordinates of the key points in the world coordinate system.

[0132] Specifically, step S43 is implemented by the following formula:

[0133]

[0134] Among them, s i is the depth of the key point in camera i, x i , x j are the coordinates of the feature points on the normalized plane of cameras i and j respectively, and R,t is the rigid body transformation from camera i to camera j.

[0135] Step S5: Use a nonlinear optimization method to optimize and improve the positioning accuracy of the key points, and generate a final source point cloud based on the optimization results.

[0136] Furthermore, the step S5 further comprises:

[0137] Different weights are assigned to each camera according to the distance between the camera and the key point. A nonlinear optimization problem is constructed by using the reprojection error of the key point on the pixel screen. The key point coordinates are optimized by solving the nonlinear optimization problem.

[0138] Here, it is assumed that the information provided by the closer camera is more accurate, so the closer camera will have a greater weight in the optimization process. Using the reprojection error of the key point on the pixel plane, the following nonlinear optimization problem is constructed and implemented by the following formula:

[0139]

[0140] Among them, X is the coordinate of the feature point in 3D space;

[0141] w i is the optimization weight of the i-th camera in the calculation of this feature point;

[0142]

[0143] c i is the coordinate of the i-th camera in the world coordinate system.

[0144] Step S6, reading the standard target point cloud, and performing rotation and translation operations to obtain the geometric center of the standard target point cloud as the origin of the workpiece coordinate system.

[0145] The standard target point cloud is the basis for subsequent operations, and its quality directly affects the accuracy of the subsequent coordinate system establishment. Generally speaking, these point cloud data are obtained through high-precision scanners (such as laser scanners, structured light scanners, etc.), which can accurately capture the three-dimensional coordinate information of the object surface.

[0146] The rotation operation adjusts the direction of the point cloud by rotating it around a certain axis (such as the X-axis, Y-axis, or Z-axis) to align it with a preset coordinate system or a specific perspective.

[0147] The translation operation is to move the point cloud along a certain direction (X, Y, Z axis) in three-dimensional space to adjust its position.

[0148] Through the combined use of the above rotation and translation operations, the geometric center (or other specified point) of the standard target point cloud can be accurately positioned to the origin of the workpiece coordinate system.

[0149] Step S7, using the point-to-surface ICP algorithm to perform a first point cloud registration on the source point cloud and the target point cloud to obtain a first registration result.

[0150] Furthermore, the step S7 further comprises:

[0151] Step S71, using a nearest neighbor algorithm to match reference points in the source point cloud with data points in the target point cloud;

[0152] Step S72, analyzing the data points in the target point cloud by a plane fitting algorithm to construct a plane model;

[0153] Step S73: Use an optimization algorithm to solve the rigid body transformation parameters between the source point cloud and the target point cloud to obtain the first registration result.

[0154] The nearest neighbor algorithm is a commonly used search technique to quickly find the point closest to a given point in a data set.

[0155] The nearest neighbor algorithm in step S71 is implemented by the following steps:

[0156] First, define the position of each reference point in the source point cloud;

[0157] Then, the nearest neighbor algorithm is used to search for the N closest points for each reference point in the target point cloud.

[0158] This process usually uses spatial data structures, such as kd trees or ball trees, to significantly speed up the search efficiency.

[0159] In the present invention, the nearest neighbor algorithm is used to ensure that each reference point in the source point cloud can find its nearest corresponding point in space, thereby providing a solid foundation for subsequent point cloud registration and in-depth analysis.

[0160] A plane fitting algorithm is a statistical method used to find the plane that best represents the distribution of a set of points.

[0161] The plane fitting algorithm in step S72 is implemented by the following steps:

[0162] First, select the N nearest neighbor data points from the target point cloud;

[0163] Next, a plane fitting algorithm is used to calculate a best fitting plane by the least square method. This best fitting plane minimizes the sum of the vertical distances from these data points to the fitting plane.

[0164] The plane equation generated by the algorithm can be used to calculate the vertical distance from each reference point in the source point cloud to this plane, which is used to construct the error function, which is implemented by the following formula:

[0165] n · p+d=0

[0166] e i =n · (R i +t)+d

[0167] Where n is the unit normal vector of the plane, p is a point on the plane, d is the intercept, and e is the i is the distance between the current reference point and the plane, and R,t is the rigid body transformation between the source point cloud and the target point cloud.

[0168] Furthermore, the step S73 further comprises:

[0169] The Levenberg-Marquardt nonlinear optimization algorithm is used to accurately solve the rigid body transformation parameters between the source point cloud and the target point cloud. This algorithm combines the advantages of the gradient descent method and the Gauss-Newton method and is particularly suitable for solving nonlinear minimization problems involving error functions.

[0170] In step S73, the Levenberg-Marquardt nonlinear optimization algorithm is implemented by the following steps:

[0171] First, an error function is defined, which represents the distance between a point in the source point cloud and the corresponding point in the target point cloud after rigid body transformation;

[0172] Then, the Levenberg-Marquardt algorithm is used to iteratively adjust the rigid body transformation parameters (rotation and translation) with the goal of minimizing the error function defined above.

[0173] This process continues, gradually reducing the total error by continuously adjusting the transformation parameters until the error reaches a preset threshold or the optimal solution is found.

[0174] Step S8, adding a certain disturbance to the first registration result, and performing a second point cloud registration with the target point cloud to obtain a second registration result.

[0175] Furthermore, the step S8 further comprises:

[0176] Step S81, adding a disturbance ΔR, Δt to the source point cloud after the first step of registration;

[0177] Step S82, using the point-to-surface ICP algorithm to match the source point cloud with the target point cloud;

[0178] Step S83: Select the match with the smallest registration error from all the matching results as the final second registration result.

[0179] The point-to-surface ICP algorithm is an iterative closest point matching technology, which is mainly used to optimize the spatial alignment between point cloud data.

[0180] The point-to-surface ICP algorithm in step S82 is implemented by the following steps:

[0181] First, select each point in the source point cloud and find the nearest plane in the target point cloud;

[0182] Then, the distance from the point to the plane is calculated as the error indicator;

[0183] Next, the position and orientation of the source point cloud are iteratively adjusted to minimize the total distance from all points to the nearest plane, thereby achieving more accurate point cloud registration.

[0184] Step S9, calculating the conversion relationship between the origin of the workpiece coordinate system and the world coordinate system, thereby obtaining the final workpiece positioning result.

[0185] The final workpiece positioning result T is achieved by the following formula:

[0186] T=(T3T2T1) -1 ;

[0187] Among them, T1 is the first registration result of step S7, T2 is the rigid body transformation disturbance added during the second matching in step S8, and T3 is the second registration result of step S8.

[0188] Although the above methods are illustrated and described as a series of actions for simplicity of explanation, it should be understood and appreciated that these methods are not limited by the order of the actions, because according to one or more embodiments, some actions may occur in a different order and / or concurrently with other actions from those illustrated and described herein or not illustrated and described herein but understandable to those skilled in the art.

[0189] Figure 2 A schematic diagram of camera internal and external parameter calibration according to an embodiment of the present invention is disclosed. Figure 2 As shown in the figure, the XYZ coordinates represent the three axes of the three-dimensional coordinate system in the actual environment. C Y C Z C Represents the coordinate axes of the camera's three-dimensional coordinate system. The multiple numbered rectangles in front of the camera represent the spatial positions of the calibration plate obtained through solution during multiple calibration processes.

[0190] The multiple rectangles with serial numbers in front of the camera represent the calculated spatial positions of the calibration plate in multiple calibrations.

[0191] The camera calibration method involved in the present invention can effectively calculate the internal and external parameters of the camera, thereby laying a more solid foundation for the subsequent positioning algorithm.

[0192] Figure 3 The key point positioning result diagram according to an embodiment of the present invention is disclosed, such as Figure 3 As shown, the XYZ coordinate system on the ground is set as the origin of the world coordinate system, while the other coordinate systems represent the positions of multiple cameras in the air. The point cloud composed of circular points is composed of key points attached to the workpiece.

[0193] Figure 4 A comparison diagram of point clouds generated by a positioning method according to an embodiment of the present invention and a standard model is disclosed, such as Figure 4As shown, the white dense point cloud is the standard point cloud generated by the standard model of the workpiece, while the red points are the point cloud calculated by the method of the present invention. It can be clearly seen that the generated point cloud contour is highly consistent with the standard contour of the workpiece, which intuitively proves that the positioning accuracy of the method of the present invention is high.

[0194] Figure 5 The principle block diagram of a workpiece positioning system based on matching Apriltag tags with point clouds according to an embodiment of the present invention is disclosed. The workpiece positioning system based on matching Apriltag tags with point clouds may include an internal communication bus 401, a processor (processor) 402, a read-only memory (ROM) 403, a random access memory (RAM) 404, a communication port 405, and a hard disk 407. The internal communication bus 401 can realize data communication between components of the workpiece positioning system based on matching Apriltag tags with point clouds. The processor 402 can make judgments and issue prompts. In some embodiments, the processor 402 can be composed of one or more processors.

[0195] The communication port 405 can realize data transmission and communication between the workpiece positioning system based on the matching of the Apriltag label and the point cloud and the external input / output device. In some embodiments, the workpiece positioning system based on the matching of the Apriltag label and the point cloud can send and receive information and data from the network through the communication port 405. In some embodiments, the workpiece positioning system based on the matching of the Apriltag label and the point cloud can perform data transmission and communication with the external input / output device in a wired form through the input / output terminal 406.

[0196] The workpiece positioning system based on Apriltag label and point cloud matching can also include different forms of program storage units and data storage units, such as hard disk 407, read-only memory (ROM) 403 and random access memory (RAM) 404, which can store various data files used for computer processing and / or communication, and possible program instructions executed by processor 402. Processor 402 executes these instructions to implement the main part of the method. The results processed by processor 402 are transmitted to an external output device through communication port 405 and displayed on the user interface of the output device.

[0197] For example, the implementation process file of the above-mentioned workpiece positioning method based on Apriltag label and point cloud matching can be a computer program, which is stored in the hard disk 407 and can be recorded in the processor 402 for execution to implement the method of the present application.

[0198] When the implementation process file of the workpiece positioning method based on Apriltag label and point cloud matching is a computer program, it can also be stored in a computer-readable storage medium as a product. For example, a computer-readable storage medium may include, but is not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips), optical disks (e.g., compact disks (CDs), digital versatile disks (DVDs)), smart cards, and flash memory devices (e.g., electrically erasable programmable read-only memories (EPROMs), cards, sticks, key drives). In addition, the various storage media described herein can represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media (and / or storage media) that can store, contain and / or carry code and / or instructions and / or data.

[0199] The present invention proposes a workpiece positioning method and system based on Apriltag label and point cloud matching. By combining multi-view stereo geometry technology, nonlinear optimization algorithm and point cloud registration technology, high-precision positioning of key points in the world coordinate system is achieved, thereby achieving high-precision and stable workpiece positioning. It shows higher robustness and applicability when processing large-scale and weakly textured workpieces, and is particularly suitable for complex industrial environments. In addition, the present invention can be applied to fields such as industrial detection, robot navigation and three-dimensional reconstruction, and plays an important role in improving the level of industrial automation and the autonomous operation capability of robots.

[0200] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0201] Those skilled in the art will appreciate that information, signals, and data may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips cited throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or optical particles, or any combination thereof.

[0202] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. The technician may implement the described functionality in different ways for each specific application, but such implementation decisions should not be interpreted as resulting in a departure from the scope of the present invention.

[0203] The various illustrative logic modules and circuits described in conjunction with the embodiments disclosed herein may be implemented or executed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in cooperation with a DSP core, or any other such configuration.

[0204] The above embodiments are provided for persons familiar with the art to implement or use the present invention. Personnel familiar with the art can make various modifications or changes to the above embodiments without departing from the inventive concept of the present invention. Therefore, the protection scope of the present invention is not limited to the above embodiments, but should be the maximum scope of the innovative features mentioned in the claims.

Claims

1. A workpiece positioning method based on Apriltag label and point cloud matching, characterized in that: The following steps are involved: Step S1, calibrating the intrinsic and extrinsic parameters of multiple cameras to obtain camera intrinsic and extrinsic parameter data; Step S2, performing distortion removal processing on the image, wherein all pixel information is retained; Step S3, performing April tag recognition on the image and extracting key points, wherein the key points include the center point and four corner points of each tag label; Step S4, using the principle of multi-view stereo geometry, solving the extracted key points to determine the coordinates of the key points in the world coordinate system; Step S5, optimizing and improving the positioning accuracy of key points by using a nonlinear optimization method, and generating a final source point cloud based on the optimization result; Step S6, reading the standard target point cloud, and performing rotation and translation operations to obtain the geometric center of the standard target point cloud as the origin of the workpiece coordinate system; Step S7, using the point-to-surface ICP algorithm to perform a first point cloud registration on the source point cloud and the target point cloud to obtain a first registration result; Step S8, adding a certain disturbance to the first registration result, and performing a second point cloud registration with the target point cloud to obtain a second registration result; Step S9, calculating the conversion relationship between the origin of the workpiece coordinate system and the world coordinate system, thereby obtaining the final workpiece positioning result.

2. The workpiece positioning method based on Apriltag label and point cloud matching according to claim 1 is characterized in that: The step S1 further comprises: Step S11: for each camera, a checkerboard calibration plate is placed at different viewing angles of the camera to collect multiple image data, calculate the feature points of the checkerboard in each image, and use the homography of the checkerboard to calculate the homography matrix H of each image. i , solve the corresponding symmetric matrix B, and then solve the camera's intrinsic parameter matrix A; Step S12, using a checkerboard calibration plate to collect multiple image data, taking the origin of the calibration plate as the origin of the world coordinate system, and obtaining the relative position transformation relationship from the world coordinate system to each camera, that is, the camera's external parameter T i ; In step S13, the 3D coordinates of the feature points obtained in step S11 are back-projected onto the pixel plane, compared with the positions of the feature points actually identified in each image and the difference is calculated, so as to obtain the reprojection error of the feature points in the corresponding camera, and the error function is constructed using the Levenberg-Marquardt algorithm in nonlinear optimization, and the final camera internal and external parameters are obtained by minimizing the reprojection error.

3. The workpiece positioning method based on Apriltag label and point cloud matching according to claim 1 is characterized in that: The step S2 further comprises: Step S21, calculating the theoretical boundary of the image after distortion correction according to the intrinsic parameter matrix obtained in step S1 and the distortion coefficient, estimating the new field of view based on the theoretical boundary, adjusting the focal length and the principal point position according to the new field of view, thereby generating a new camera intrinsic parameter matrix; Step S22: construct a distortion map for each pixel based on the new intrinsic parameter matrix and in combination with the distortion coefficient, and use the distortion map to resample the original image to ultimately generate a corrected image.

4. The workpiece positioning method based on Apriltag label and point cloud matching according to claim 1 is characterized in that: The step S3 further comprises: Step S31, performing binarization processing on the image; Step S32, identifying and numbering connected domains in the binarized image; Step S33, detecting quadrilaterals in the entire image, and obtaining the specific coordinates of the four corner points and the center point of each corresponding tag in the image; Step S34, decoding the two-dimensional code in the quadrilateral area to obtain the coding information contained in the tag label.

5. The workpiece positioning method based on Apriltag label and point cloud matching according to claim 4 is characterized in that: The step S31 further comprises: The image is binarized using an adaptive threshold segmentation algorithm.

6. The workpiece positioning method based on Apriltag label and point cloud matching according to claim 4 is characterized in that: The step S32 further comprises: The connected domains in the binarized image are identified and numbered using the union-find algorithm.

7. The workpiece positioning method based on Apriltag label and point cloud matching according to claim 1 is characterized in that: The step S4 further comprises: Step S41, for each key point, collect all cameras that observe the key point, and store the collected camera information in a container; Step S42, removing camera information whose reprojection error is greater than a certain range from the container; Step S43, using a multi-camera triangulation method, all key points are measured and solved to obtain the coordinates of the key points in the world coordinate system.

8. The workpiece positioning method based on Apriltag label and point cloud matching according to claim 1 is characterized in that: The step S5 further comprises: Different weights are assigned to each camera according to the distance between the camera and the key point. A nonlinear optimization problem is constructed by using the reprojection error of the key point on the pixel screen. The key point coordinates are optimized by solving the nonlinear optimization problem.

9. The workpiece positioning method based on Apriltag label and point cloud matching according to claim 1 is characterized in that: The step S7 further comprises: Step S71, using a nearest neighbor algorithm to match reference points in the source point cloud with data points in the target point cloud; Step S72, analyzing the data points in the target point cloud by a plane fitting algorithm to construct a plane model; Step S73: Use an optimization algorithm to solve the rigid body transformation parameters between the source point cloud and the target point cloud to obtain the first registration result.

10. The workpiece positioning method based on Apriltag label and point cloud matching according to claim 9 is characterized in that: The step S73 further comprises: The Levenberg-Marquardt nonlinear optimization algorithm is used to solve the rigid body transformation parameters between the source point cloud and the target point cloud.

11. The workpiece positioning method based on Apriltag label and point cloud matching according to claim 1 is characterized in that: The step S8 further comprises: Step S81, adding disturbance to the source point cloud after the first step of registration; Step S82, using the point-to-surface ICP algorithm to match the source point cloud with the target point cloud; Step S83: Select the match with the smallest registration error from all the matching results as the final second registration result.

12. The workpiece positioning method based on Apriltag label and point cloud matching according to claim 1 is characterized in that: In step S9, the final workpiece positioning result T is achieved by the following formula: T=(T3T2T1) -1 ; Among them, T1 is the first registration result of step S7, T2 is the rigid body transformation disturbance added during the second matching in step S8, and T3 is the second registration result of step S8.

13. A workpiece positioning system based on Apriltag label and point cloud matching, characterized in that: include: a memory for storing instructions executable by a processor; A processor, configured to execute the instructions to implement the method according to any one of claims 1 to 12.

14. A computer-readable medium having computer instructions stored thereon, wherein when the computer instructions are executed by a processor, the method according to any one of claims 1 to 12 is performed.

Images