AprilTag-based mechanical arm quality inspection imaging point migration method and device

Abstract

This invention provides a method and device for migrating quality inspection imaging points of a robotic arm based on AprilTag, comprising: extracting the golden overlay map of quality inspection points of the original reference machine; the quality inspection points of the original reference machine include: marked quality inspection points and ordinary quality inspection points; based on the golden overlay map of the corresponding quality inspection points of the original reference machine, iteratively correcting each marked quality inspection point of the migrated replica machine; based on the results of the iterative correction, using SVD to solve the global rigid homogeneous transformation matrix Δ between the original reference machine and the migrated replica machine; using the global rigid homogeneous transformation matrix Δ, solving the pose of the ordinary quality inspection points of the migrated replica machine. This invention can quickly establish the coordinate system association between the workpiece and the robotic arm and realize the automatic migration of quality inspection imaging points. It can solve the problem that when replicating quality inspection points of existing robotic arms using isomorphic machines, the points cannot be reused due to differences in robotic arm precision, camera parameters, and assembly tolerances of the workpiece on the platform.

Description

Technical Field

[0001] This invention relates to a method and device for migrating quality inspection imaging points using a robotic arm based on AprilTag. Background Technology

[0002] With the rapid development of automation technology, robotic arms have been widely used in workpiece transportation, assembly, and quality inspection. In vision-based quality inspection scenarios using robotic arms, the precise setting of the imaging point (i.e., the pose required by the end-effector camera for a specific inspection area) is crucial to ensuring inspection quality. Existing technical solutions typically suffer from the following problems:

[0003] Manual teaching is inefficient: Traditional imaging point setting relies on manual point-by-point teaching. When a large number of points need to be detected, the workload is huge and it is difficult to achieve automated setting of continuous points.

[0004] Lack of flexibility and adaptability: When the workpiece is placed in a different position on the tooling, or when the production line switches to a different product model and a new production line is made, the previously taught points become invalid, requiring tedious recalibration and teaching.

[0005] Limited positioning accuracy: In practical applications, without a clear reference point, it is difficult to perform high-precision point migration and reuse in the global coordinate system by relying solely on image recognition. Summary of the Invention

[0006] The purpose of this invention is to provide a method and device for migrating quality inspection imaging points of a robotic arm based on AprilTag.

[0007] To address the above problems, this invention provides a method for migrating robotic arm quality inspection imaging points based on AprilTag, comprising:

[0008] Extract the gold-set map of the quality inspection points of the original benchmark machine tool; the quality inspection points of the original benchmark machine tool include: marked quality inspection points and ordinary quality inspection points;

[0009] Based on the golden set of corresponding quality inspection points of the original benchmark machine, each marked quality inspection point of the migrated replica machine is iteratively corrected.

[0010] Based on the results of iterative correction, the global rigid homogeneous transformation matrix Δ between the original reference machine and the migration replication machine is solved using SVD.

[0011] The pose of the ordinary quality inspection points of the migration replication machine is solved by using the global rigid homogeneous transformation matrix Δ.

[0012] Furthermore, in the above method, the golden ratio map of the quality inspection points of the original benchmark machine is extracted, including:

[0013] After completing the hand-eye calibration on the original reference machine, the homogeneous transformation matrix between the flange and the camera of the original reference machine is obtained. ;

[0014] The original benchmark machine's quality inspection points are divided into a set of marked quality inspection points with AprilTag and a set of ordinary quality inspection points without AprilTag.

[0015] The robotic arm of the manual teaching original benchmark machine moves sequentially to m marked quality inspection points. The optimal imaging pose is determined by the robotic arm controller reading the transformation matrix from the base coordinate system of each marked quality inspection point on the original reference machine to the flange. ;

[0016] At each marked quality inspection point An industrial camera captures images of the workpiece, a vision image processing unit identifies AprilTag markers, and calculates the transformation matrix from the camera coordinate system to the AprilTag. The AprilTag marker is fixedly affixed to a predetermined fixed position on the outer surface of the workpiece.

[0017] Based on the homogeneous transformation matrix of flange and camera The transformation matrix from the base coordinate system of each marked quality inspection point on the original reference machine to the flange. and the transformation matrix from the camera coordinate system to AprilTag Calculate the transformation matrix from the base coordinate system of the original reference machine to the Tag. ;

[0018] Continue teaching the original benchmark machine to n common quality inspection points. The transformation matrix from the base coordinate system of each ordinary quality inspection point of the original benchmark machine to the flange is collected and stored. All parameters of the transformation matrix are stored as the original gold benchmark dataset, i.e., the gold set.

[0019] Furthermore, in the above method, the transformation matrix from the base coordinate system of the original reference machine to the Tag is calculated. The formula is:

[0020] (1).

[0021] Furthermore, in the above method, based on the golden overlay map of the corresponding quality inspection points of the original benchmark machine, each marked quality inspection point of the migrated replica machine is iteratively corrected, including:

[0022] Step S20: Place the same workpiece on an isomorphic transfer and replication machine for correction. Independently complete the hand-eye calibration of the transfer and replication machine to obtain the transformation matrix from the flange to the camera of the transfer and replication machine. For each marked quality inspection point The transformation matrix of the flange in the base coordinate system of the original reference machine's marked quality inspection points. As the initial pose input, the initial... Set as ;

[0023] Step S21, for each quality control point containing the AprilTag marker... Perform iterations from step S22 to step S23:

[0024] Step S22, drive the robotic arm of the migration replication machine to the current position. For the corresponding pose, an industrial camera captures images of the same workpiece and calculates the result. Then, calculate by matrix multiplication. ;

[0025] Step S23: Calculate the transformation matrix from the camera coordinate system of the original reference camera to AprilTag. The transformation matrix from the camera coordinate system of the replica machine to AprilTag. Is the difference less than the threshold?

[0026] Step S241, if not, update Then, restart from step S22;

[0027] Step S242, if yes, record the transformation matrix from the base coordinate system of the replica machine to the flange. Transformation matrix from base coordinate system to AprilTag Transformation matrix from flange to camera Record the marked quality inspection points of the original reference machine at this time. To relocate the marked quality inspection points of the replica machine .

[0028] Furthermore, in the above method, the calculation ,include:

[0029] Calculate using formula (2) ,

[0030] (2).

[0031] Furthermore, in the above method, based on the results of iterative correction, the global rigid homogeneous transformation matrix Δ between the original reference machine and the migrated replication machine is solved using SVD, including:

[0032] Extract the transformation matrix from the base coordinate system of the original reference machine obtained in each step S1 to the April Tag. The homogeneous transformation matrix from the base coordinate system of the migration replication machine obtained in step S2 to AprilTag. This forms a pair of points; the SVD singular value decomposition rigid transformation algorithm is used to solve for the global rigid homogeneous transformation matrix Δ between the original reference machine and the migrated replication machine.

[0033] .

[0034] Furthermore, in the above method, the pose of the ordinary quality inspection points of the migration replication machine is solved using the global rigid homogeneous transformation matrix Δ, including:

[0035] For each ordinary quality inspection point of the original benchmark machine The common quality inspection points of the migration replication machine are calculated according to the fixed closed-loop mapping formula (3). The transformation matrix from the target base coordinate system to the flange:

[0036] (3);

[0037] The calculated Input the data to the robotic arm controller, execute the inverse kinematics calculation of the robotic arm, and output the locations of each ordinary quality inspection point on the migration replication machine. The corresponding joint angle value.

[0038] Furthermore, in the above method, after solving for the pose of the ordinary quality inspection points of the transfer replication machine using the global rigid homogeneous transformation matrix Δ, the method also includes:

[0039] Based on the converged transformation matrix from the target base coordinate system of the marked quality inspection points of the migration replication machine to the flange. The transformation matrix from the target base coordinate system of the ordinary quality inspection point of the migration replication machine to the flange is calculated. The corresponding joint angle values ​​drive the migration replication machine to move sequentially to each marked quality inspection point. and ordinary quality inspection points Acquire real-time images;

[0040] Single-point verification shows that the pixel position deviation of the AprilTag marker in the real-time image, the workpiece imaging field of view, the image clarity, and the shooting pitch angle are completely consistent with the original reference machine tool.

[0041] After single-point verification is passed, the transformation matrix of the target base coordinate system of the marked quality inspection points of all qualified replica machines to the flange is applied. Transformation matrix from the target coordinate system of a typical quality inspection point to the flange The joint angle values ​​of ordinary quality inspection points are used as migration points for solidification, and after solidification, they are put into mass production operation for visual quality inspection.

[0042] According to another aspect of the present invention, a computer-readable storage medium is also provided, having stored thereon computer-executable instructions, wherein when executed by a processor, the computer-executable instructions cause the processor to perform any of the methods described above.

[0043] According to another aspect of the present invention, a calculator device is also provided, comprising:

[0044] Processor; and

[0045] A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform any of the methods described above.

[0046] The key technical points of this invention are as follows:

[0047] Key point 1: Establish the workpiece coordinate system using AprilTag:

[0048] The original machine tool and the migrated machine tool use the same workpiece with the AprilTag attached, ensuring that the workpiece coordinate system represented by the AprilTag remains unchanged in both systems;

[0049] Key point 2: Indirect solution based on the invariance of spatial relationships (see the above formula derivation).

[0050] Key point 3: Keep the relative pose of the camera and AprilTag (workpiece) unchanged;

[0051] The ultimate goal of this invention is not to move the robotic arm to a fixed world coordinate point, but to keep the distance and angle of the camera lens relative to the workpiece surface (i.e., the image-taking posture) constant. This ensures the versatility and stability of the visual quality inspection algorithm.

[0052] Key point 4: Tight coupling between visual information and robot kinematics:

[0053] This method is not just visual recognition, nor is it simply robot motion. Instead, it is a complete closed-loop control process that maps the recognition results (pixel coordinates) of AprilTag to the robot's joint angles (or Cartesian pose) through the hand-eye matrix and robot forward kinematics.

[0054] The present invention has the following beneficial effects:

[0055] 1. High precision: AprilTag has sub-pixel level recognition accuracy. Combined with the absolute positioning accuracy of the robotic arm, it can ensure that the imaging repeatability accuracy is within ±0.5mm.

[0056] 2. Eliminating manual teaching and achieving "one-click changeover": Traditional methods require engineers to manually guide or teach the robotic arm to reposition after workpiece model changes or tooling relocations, typically taking hours or even half a day, severely impacting production line uptime. This invention, by affixing AprilTag markers and utilizing spatial geometric transformations to automatically calculate new positions, reduces changeover time from hours to minutes, and even achieves fully automated changeover, significantly reducing manual intervention.

[0057] 3. Reduced Costs and Cycles of Automation Transformation: For production lines with multiple product varieties and small batches, traditional 3D vision-guided solutions typically require expensive laser profilometers or large 3D vision systems for global positioning. This invention only requires low-cost paper AprilTag codes and ordinary 2D industrial cameras to achieve high-precision self-adaptive positioning. It eliminates the need for complex low-level secondary development of existing robotic arm control cabinets, resulting in low system integration difficulty and making it ideal for the intelligent transformation of older production lines.

[0058] In summary, this invention can quickly establish the coordinate system association between the workpiece and the robotic arm, and realize the automatic migration of quality inspection imaging points. It can solve the problem of unusable points due to differences in robotic arm precision, camera parameters, and workpiece assembly tolerances when replicating quality inspection points on existing robotic arm quality inspection machines. Attached Figure Description

[0059] Figure 1 This is a flowchart of a robotic arm quality inspection imaging point migration method based on AprilTag according to an embodiment of the present invention;

[0060] Figure 2 This is a flowchart of the single-point migration iteration of AprilTag according to an embodiment of the present invention. Detailed Implementation

[0061] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0062] In the field of traditional robotic arm quality inspection, a high-precision industrial camera is typically mounted at the end effector of the robotic arm. Moving the 6-axis joints of the robotic arm moves the end effector camera to photograph the workpiece surface for defect detection. When workpiece production increases, a single robotic arm inspection machine is often insufficient. Therefore, multiple identical machines are often replicated for simultaneous inspection. Although the same model of robotic arm, camera, and platform fixture are selected, the replicated machines cannot reuse the original machine's 6-axis joint positions due to differences in robotic arm precision, camera calibration parameters, and platform fixture tolerances. In other words, the images of the workpiece surface taken by the replicated machines and the identical machines at the same 6-axis joint positions are different.

[0063] Robotic arm quality inspection imaging points: An industrial camera is mounted on the end effector of the robotic arm to capture images of specific defect detection points on the surface of the workpiece.

[0064] AprilTag: April Tag is an open-source visual benchmarking system developed by Michigan State University. It's used for target localization, recognition, and 3D spatial computation using camera images, and has wide applications in fields such as robot navigation, augmented reality, and camera calibration.

[0065] The correct solution for solving a robotic arm problem is to use the joint coordinates of the robotic arm to determine its Cartesian coordinates.

[0066] Inverse kinematics of robotic arms: Solving for the joint coordinates of the robotic arm using its Cartesian coordinates may result in multiple solutions.

[0067] like Figure 1 As shown, the hardware system upon which this invention relies is pre-composed of an original reference machine, a migration and replication machine, several AprilTag visual reference markers, a workpiece to be inspected, a six-axis robotic arm, a 2D industrial camera, a fixed-focus industrial lens, a robotic arm controller, and a visual image processing unit; wherein,

[0068] The original reference machine and the migration and replication machine are identical machines with completely consistent hardware configuration and model.

[0069] A 2D industrial camera and a fixed-focus industrial lens are fixed to the end flange of a six-axis robotic arm to form an eye-in-hand structure.

[0070] Several AprilTag visual reference markers are fixedly affixed to the surface of the workpiece to be inspected and their position relative to the workpiece is fixed.

[0071] Here, The pose represented by the flange centerpoint of the robotic arm in the base coordinate system. It is obtained by directly reading the Cartesian coordinates from the teach pendant and then converting them.

[0072] Hand-eye matrix. The pose represented by the camera in the coordinate system of the robotic arm flange end effector. Obtained directly through the camera hand-eye calibration method (calibrateHandEye).

[0073] The AprilTag represents the pose in the camera coordinate system. It can be calculated using the open-source AprilTag algorithm, based on the image captured by the camera, camera intrinsics, and the size of the AprilTag.

[0074] The pose represented by AprilTag in the robot arm's base coordinate system. It is obtained through a series of matrix multiplications and cannot be read directly.

[0075] The relationship between these variables is as follows:

[0076]

[0077] We select a standard workpiece and affix AprilTags to as many quality inspection points as possible on the original reference machine, ensuring that the quality inspection images from the original reference machine are clear and centered. The quality inspection points on the original reference machine can be divided into two sets.

[0078] A quality control point has m marked locations containing the AprilTag marker, denoted as... i = 1, 2, 3, 4.....m;

[0079] Another set of n ordinary quality inspection points that do not contain the AprilTag marker is denoted as j=1,2,3,4.....n;

[0080] Formula derivation:

[0081] For each AprilTag on the original quality inspection machine (Ground Truth, GT), we have:

[0082] (1);

[0083] For each AprilTag of the transfer / replication machine (Transfer, TR), the following applies:

[0084] (2);

[0085] To ensure consistency between the images captured by the original reference camera and the migrated replica camera, it is necessary to ensure that the AprilTag is consistent in the camera coordinate system, i.e.

[0086] ;

[0087] Therefore, we obtain the following system of equations:

[0088] ;

[0089] ;

[0090] ;

[0091] ;

[0092] ;

[0093] ;

[0094] .

[0095] This invention provides a method for migrating imaging points in a robotic arm based on AprilTag, the method comprising the following steps:

[0096] Step S1: Extract the gold-set image of the quality inspection points of the original benchmark machine; the quality inspection points include: marked quality inspection points. and ordinary quality inspection points ;

[0097] Step S1 includes:

[0098] Step S10: Perform hand-eye calibration on the original reference machine to obtain the homogeneous transformation matrix between the flange and the camera of the original reference machine. ;

[0099] Step S11: Divide the quality inspection points of the original benchmark machine into a set of marked quality inspection points with AprilTag and a set of ordinary quality inspection points without AprilTag.

[0100] Step S12: The robotic arm of the original benchmark machine is manually taught to move sequentially to the m marked quality inspection points. The optimal imaging pose is determined by the robotic arm controller reading the transformation matrix from the base coordinate system of each marked quality inspection point on the original reference machine to the flange. ;

[0101] Step S13, at each marked quality inspection point An industrial camera captures images of the workpiece, a vision image processing unit identifies AprilTag markers, and calculates the transformation matrix from the camera coordinate system to the AprilTag. The AprilTag marker is fixedly affixed to a predetermined fixed position on the outer surface of the workpiece.

[0102] Step S14, based on the homogeneous transformation matrix of the flange and camera The transformation matrix from the base coordinate system of each marked quality inspection point on the original reference machine to the flange. and the transformation matrix from the camera coordinate system to AprilTag Calculate the transformation matrix from the base coordinate system of the original reference machine to the Tag. ;

[0103] Step S15: Continue teaching the original benchmark machine to n common quality inspection points. The transformation matrix from the base coordinate system of each ordinary quality inspection point of the original benchmark machine to the flange is collected and stored. All parameters of the transformation matrix are stored as the original gold benchmark dataset, i.e., the gold set.

[0104] Here, the gold set is extracted from the original benchmark (Ground Truth, GT):

[0105] Traverse all quality control points containing the AprilTag marker. Record Cartesian coordinates Take pictures of the workpiece and calculate... Then calculate using formula (1) ;

[0106] (1);

[0107] Traverse all ordinary quality inspection points that do not contain the AprilTag marker. Record Cartesian coordinates ;

[0108] Specifically, in one embodiment, the original reference machine and the migration replication machine are the same six-axis robotic arm eye-on-hand architecture; m=3 AprilTag 36h11 markers are fixedly pasted on the surface of the workpiece to be inspected, and the position and pose of the workpiece are permanently fixed relative to the workpiece; the preset position convergence threshold is 0.1mm, and the angle convergence threshold is 0.1°; the imaging points include m=3 quality inspection points with AprilTag markers and n=20 ordinary quality inspection points without AprilTag.

[0109] Step S2: Based on the golden set of the corresponding quality inspection points of the original benchmark machine, iteratively correct each marked quality inspection point of the migrated replica machine.

[0110] Step S2 includes:

[0111] Step S20: Place the same workpiece onto an isomorphic transfer replication machine for correction (Transfer, TR). Independently complete the hand-eye calibration of the machine on the transfer replication machine to obtain the transformation matrix from the flange to the camera of the transfer replication machine. For each marked quality inspection point The transformation matrix of the flange in the base coordinate system of the original reference machine's marked quality inspection points. As the initial pose input, the initial... Set as ;

[0112] Step S21, for each quality control point containing the AprilTag marker... Perform iterations from step S22 to step S23:

[0113] Step S22, drive the robotic arm of the migration replication machine to the current position. For the corresponding pose, an industrial camera captures images of the same workpiece and calculates the result. Then, by matrix multiplication, calculate using formula (2). ;

[0114] (2);

[0115] Step S23: Calculate the transformation matrix from the camera coordinate system of the original reference camera to AprilTag. The transformation matrix from the camera coordinate system of the replica machine to AprilTag. Is the difference less than the threshold?

[0116] Step S241, if not, update Then, restart from step S22;

[0117] Step S242, if yes, record the transformation matrix from the base coordinate system of the replica machine to the flange. Transformation matrix from base coordinate system to AprilTag Transformation matrix from flange to camera Record the marked quality inspection points of the original reference machine at this time. To relocate the marked quality inspection points of the replica machine ;

[0118] Here, the same workpiece is placed on an isomorphic transfer (TR) machine for correction, and for each quality inspection point containing the AprilTag marker... , proceed as Figure 2 The next iteration is shown; where, can be calculated With target value If the error is less than the threshold, the iteration is terminated directly and recorded. Mark the quality inspection point at this time. The corresponding location is recorded as the migration point. Otherwise proceed to step S241.

[0119] In step S2, visual iterative closed-loop correction is performed for each marked quality inspection point. The convergence goal is to make the six-DOF pose of the camera relative to AprilTag consistent with the original reference machine. The transformation matrix from the base coordinate system of the migration replication machine to the flange is iteratively updated. Continue until the pose error is less than a preset pose error threshold, then save the transformation matrix from the base coordinate system of the converged migration replication machine to AprilTag. And the transformation matrix from the base coordinate system corresponding to the marked points of the re-copying machine station after convergence to the flange. ;

[0120] Step S3: Based on the results of iterative correction, use SVD to solve the global rigid homogeneous transformation matrix Δ between the original reference machine and the migration replication machine.

[0121] Here, the transformation matrix from the base coordinate system of the original reference machine obtained in each step S1 to the April Tag is extracted. The homogeneous transformation matrix from the base coordinate system of the migration replication machine obtained in step S2 to AprilTag. This forms a pair of points; the SVD singular value decomposition rigid transformation algorithm is used to solve for the global rigid homogeneous transformation matrix Δ between the original reference machine and the migrated replication machine.

[0122] ;

[0123] For each AprilTag location, there are corresponding quality inspection points marked on the original benchmark machine. , and the corresponding step S1 recorded ; and the relocation points of the relocation machines. and the corresponding step S242 recorded ,

[0124] Use the SVD algorithm to find a set To another group homogeneous transformation matrix ,Right now:

[0125] ;

[0126] Step S4: Use the global rigid homogeneous transformation matrix Δ to solve the pose of the ordinary quality inspection points of the migration replication machine.

[0127] Here, each ordinary quality inspection point of the original benchmark machine is described. The common quality inspection points of the migration replication machine are calculated according to the fixed closed-loop mapping formula (3). The transformation matrix from the target base coordinate system to the flange:

[0128] (3);

[0129] The calculated Input the data to the robotic arm controller, execute the inverse kinematics calculation of the robotic arm, and output the locations of each ordinary quality inspection point on the migration replication machine. The corresponding joint angle value.

[0130] Here, the transformation matrix Δ is used to represent the transformation from the base coordinate system of the ordinary quality inspection points of the original reference machine to the flange. Transformation matrix from flange to camera of the original reference machine. Transformation matrix from flange to camera of the replication machine. Matrix multiplication is performed to calculate the locations of each ordinary quality inspection point on the migration replication machine. The transformation matrix from the target base coordinate system to the flange. For each routine quality inspection point of the relocated replica machine. The matrix from the target base coordinate system to the flange Perform inverse kinematics calculations on the robotic arm to obtain the joint angle values ​​of each ordinary point on the migration and replication machine.

[0131] Here, for ordinary quality inspection points without AprilTag... In the previous step S242, it was recorded that Calculate using the formula:

[0132] (3);

[0133] Thus obtain The transformation matrix from the target base coordinate system to the flange. ;

[0134] This led to the identification of the marked quality inspection points for the relocated replication equipment. Ordinary quality inspection points The quality inspection points correspond to the original machine tool's location markings. Ordinary quality inspection points .

[0135] Step S5: Based on the transformation matrix from the target base coordinate system of the marked quality inspection point of the migration replication machine to the flange obtained after convergence in step S2. Step S4: Calculate the transformation matrix from the target coordinate system of the ordinary quality inspection point of the migration replication machine to the flange. The corresponding joint angle values ​​drive the migration replication machine to move sequentially to each marked quality inspection point. and ordinary quality inspection points Real-time images are acquired; single-point verification is performed to ensure that the pixel position deviation of the AprilTag marker in the real-time image, the workpiece imaging field of view, image clarity, and shooting pitch angle are completely consistent with the original reference machine; after the single-point verification is qualified, the transformation matrix of the target base coordinate system of the marked quality inspection points of the migration and replication machine to the flange is transformed. Transformation matrix from the target coordinate system of a typical quality inspection point to the flange The joint angle values ​​of ordinary quality inspection points are used as migration points for solidification, and after solidification, they are put into mass production operation for visual quality inspection.

[0136] According to another aspect of the present invention, a computer-readable storage medium is also provided, having stored thereon computer-executable instructions, wherein when executed by a processor, the computer-executable instructions cause the processor to perform any of the methods described above.

[0137] According to another aspect of the present invention, a calculator device is also provided, comprising:

[0138] Processor; and

[0139] A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform any of the methods described above.

[0140] The key technical points of this invention are as follows:

[0141] Key point 1: Establish the workpiece coordinate system using AprilTag:

[0142] The original machine tool and the migrated machine tool use the same workpiece with the AprilTag attached, ensuring that the workpiece coordinate system represented by the AprilTag remains unchanged in both systems;

[0143] Key point 2: Indirect solution based on the invariance of spatial relationships (see the above formula derivation).

[0144] Key point 3: Keep the relative pose of the camera and AprilTag (workpiece) unchanged;

[0145] The ultimate goal of this invention is not to move the robotic arm to a fixed world coordinate point, but to keep the distance and angle of the camera lens relative to the workpiece surface (i.e., the image-taking posture) constant. This ensures the versatility and stability of the visual quality inspection algorithm.

[0146] Key point 4: Tight coupling between visual information and robot kinematics:

[0147] This method is not just visual recognition, nor is it simply robot motion. Instead, it is a complete closed-loop control process that maps the recognition results (pixel coordinates) of AprilTag to the robot's joint angles (or Cartesian pose) through the hand-eye matrix and robot forward kinematics.

[0148] The present invention has the following beneficial effects:

[0149] 2. High precision: AprilTag has sub-pixel level recognition accuracy. Combined with the absolute positioning accuracy of the robotic arm, it can ensure that the imaging repeatability accuracy is within ±0.5mm.

[0150] 2. Eliminating manual teaching and achieving "one-click changeover": Traditional methods require engineers to manually guide or teach the robotic arm to reposition after workpiece model changes or tooling relocations, typically taking hours or even half a day, severely impacting production line uptime. This invention, by affixing AprilTag markers and utilizing spatial geometric transformations to automatically calculate new positions, reduces changeover time from hours to minutes, and even achieves fully automated changeover, significantly reducing manual intervention.

[0151] 3. Reduced Costs and Cycles of Automation Transformation: For production lines with multiple product varieties and small batches, traditional 3D vision-guided solutions typically require expensive laser profilometers or large 3D vision systems for global positioning. This invention only requires low-cost paper AprilTag codes and ordinary 2D industrial cameras to achieve high-precision self-adaptive positioning. It eliminates the need for complex low-level secondary development of existing robotic arm control cabinets, resulting in low system integration difficulty and making it ideal for the intelligent transformation of older production lines.

[0152] In summary, this invention can quickly establish the coordinate system association between the workpiece and the robotic arm, and realize the automatic migration of quality inspection imaging points. It can solve the problem of unusable points due to differences in robotic arm precision, camera parameters, and workpiece assembly tolerances when replicating quality inspection points on existing robotic arm quality inspection machines.

[0153] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0154] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0155] Obviously, those skilled in the art can make various modifications and variations to the invention without departing from the spirit and scope of the invention. Therefore, if these modifications and variations fall within the scope of the claims of the invention and their equivalents, the invention is also intended to include these modifications and variations.

Claims

1. A method for migrating imaging points in a robotic arm based on AprilTag, characterized in that, include: Extract the gold-set diagram of the quality inspection points of the original benchmark machine; The quality inspection points of the original benchmark machine include: marked quality inspection points and ordinary quality inspection points; Based on the golden set of corresponding quality inspection points of the original benchmark machine, each marked quality inspection point of the migrated replica machine is iteratively corrected. Based on the results of iterative correction, the global rigid homogeneous transformation matrix Δ between the original reference machine and the migration replication machine is solved using SVD. The pose of the ordinary quality inspection points of the migration replication machine is solved by using the global rigid homogeneous transformation matrix Δ.

2. The robotic arm quality inspection imaging point migration method based on AprilTag as described in claim 1, characterized in that, Extract the golden ratio map of the quality inspection points of the original benchmark machine, including: After completing the hand-eye calibration on the original reference machine, the homogeneous transformation matrix between the flange and the camera of the original reference machine is obtained. ; The original benchmark machine's quality inspection points are divided into a set of marked quality inspection points with AprilTag and a set of ordinary quality inspection points without AprilTag. The robotic arm of the manual teaching original benchmark machine moves sequentially to m marked quality inspection points. The optimal imaging pose is determined by the robotic arm controller reading the transformation matrix from the base coordinate system of each marked quality inspection point on the original reference machine to the flange. ; At each marked quality inspection point An industrial camera captures images of the workpiece, a vision image processing unit identifies AprilTag markers, and calculates the transformation matrix from the camera coordinate system to the AprilTag. The AprilTag marker is fixedly affixed to a predetermined fixed position on the outer surface of the workpiece. Based on the homogeneous transformation matrix of flange and camera The transformation matrix from the base coordinate system of each marked quality inspection point on the original reference machine to the flange. and the transformation matrix from the camera coordinate system to AprilTag Calculate the transformation matrix from the base coordinate system of the original reference machine to the Tag. ; Continue teaching the original benchmark machine to n common quality inspection points. The transformation matrix from the base coordinate system of each ordinary quality inspection point of the original benchmark machine to the flange is collected and stored. All parameters of the transformation matrix are stored as the original gold benchmark dataset, i.e., the gold set.

3. The robotic arm quality inspection imaging point migration method based on AprilTag as described in claim 2, characterized in that, Calculate the transformation matrix from the base coordinate system of the original reference machine to the Tag. The formula is: （1）。 4. The robotic arm quality inspection imaging point migration method based on AprilTag as described in claim 2, characterized in that, Based on the golden ratio map of the corresponding quality inspection points of the original benchmark machine, each marked quality inspection point of the migrated replica machine is iteratively corrected, including: Step S20: Place the same workpiece on an isomorphic transfer and replication machine for correction. Independently complete the hand-eye calibration of the transfer and replication machine to obtain the transformation matrix from the flange to the camera of the transfer and replication machine. For each marked quality inspection point The transformation matrix of the flange in the base coordinate system of the original reference machine's marked quality inspection points. As the initial pose input, the initial... Set as ; Step S21, for each quality control point containing the AprilTag marker... Perform iterations from step S22 to step S23: Step S22, drive the robotic arm of the migration replication machine to the current position. For the corresponding pose, an industrial camera captures images of the same workpiece and calculates the result. Then, calculate by matrix multiplication. ; Step S23: Calculate the transformation matrix from the camera coordinate system of the original reference camera to AprilTag. The transformation matrix from the camera coordinate system of the replica machine to AprilTag. Is the difference less than the threshold? Step S241, if not, update Then, restart from step S22; Step S242, if yes, record the transformation matrix from the base coordinate system of the replica machine to the flange. Transformation matrix from base coordinate system to AprilTag Transformation matrix from flange to camera Record the marked quality inspection points of the original reference machine at this time. To relocate the marked quality inspection points of the replica machine .

5. The robotic arm quality inspection imaging point migration method based on AprilTag as described in claim 4, characterized in that, calculate ,include: Calculate using formula (2) , （2）。 6. The robotic arm quality inspection imaging point migration method based on AprilTag as described in claim 4, characterized in that, Based on the results of iterative correction, the global rigid homogeneous transformation matrix Δ between the original reference machine and the migrated replication machine is solved using SVD, including: Extract the transformation matrix from the base coordinate system of the original reference machine obtained in each step S1 to the April Tag. The homogeneous transformation matrix from the base coordinate system of the migration replication machine obtained in step S2 to AprilTag. This forms a pair of points; the SVD singular value decomposition rigid transformation algorithm is used to solve for the global rigid homogeneous transformation matrix Δ between the original reference machine and the migrated replication machine. 。 7. The robotic arm quality inspection imaging point migration method based on AprilTag as described in claim 6, characterized in that, Using the global rigid homogeneous transformation matrix Δ, the pose of ordinary quality inspection points on the migration replication machine is solved, including: For each ordinary quality inspection point of the original benchmark machine The common quality inspection points of the migration replication machine are calculated according to the fixed closed-loop mapping formula (3). The transformation matrix from the target base coordinate system to the flange: （3）； The calculated Input the data to the robotic arm controller, execute the inverse kinematics calculation of the robotic arm, and output the locations of each ordinary quality inspection point on the migration replication machine. The corresponding joint angle value.

8. The robotic arm quality inspection imaging point migration method based on AprilTag as described in claim 1, characterized in that, After solving for the pose of the ordinary quality inspection points of the migration replication machine using the global rigid homogeneous transformation matrix Δ, the process also includes: Based on the converged transformation matrix from the target base coordinate system of the marked quality inspection points of the migration replication machine to the flange. The transformation matrix from the target base coordinate system of the ordinary quality inspection point of the migration replication machine to the flange is calculated. The corresponding joint angle values ​​drive the migration replication machine to move sequentially to each marked quality inspection point. and ordinary quality inspection points Acquire real-time images; Single-point verification shows that the pixel position deviation of the AprilTag marker in the real-time image, the workpiece imaging field of view, the image clarity, and the shooting pitch angle are completely consistent with the original reference machine tool. After single-point verification is passed, the transformation matrix of the target base coordinate system of the marked quality inspection points of all qualified replica machines to the flange is applied. Transformation matrix from the target coordinate system of a typical quality inspection point to the flange The joint angle values ​​of ordinary quality inspection points are used as migration points for solidification, and after solidification, they are put into mass production operation for visual quality inspection.

9. A computer-readable storage medium having stored thereon computer-executable instructions, wherein, When the computer-executable instructions are executed by the processor, the processor causes the processor to perform the method as claimed in any one of claims 1 to 8.

10. A calculator device, wherein, include: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method as claimed in any one of claims 1 to 8.

Images