Pose correction method, system and computer readable storage medium based on 2D vision
Patent Information
- Application Number
- CN202610897018.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-18
- Publication Date
- 2026-09-15
Smart Images
Figure CN122747014A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of robotic automated production technology, and more specifically to a pose correction method, system, and computer-readable storage medium based on 2D vision. Background Technology
[0002] In modern industrial automated production lines, the use of Automated Guided Vehicles (AGVs) or Autonomous Mobile Robots (AMRs) equipped with robotic arms for multi-station loading and unloading has become a trend. However, existing AMR solutions typically face the following technical challenges: First, when the AGV transports the robotic arm to the vicinity of the target equipment (such as a CNC machine tool), its own stopping accuracy is usually at the centimeter or millimeter level, resulting in insufficient secondary positioning accuracy, which makes it difficult to meet the loading and unloading requirements of high-precision machining (error must be within 0.1mm).
[0003] Secondly, ground flatness, load changes, and cumulative sensor errors often cause positional deviations between the robot's end effector and the target workstation, which are greatly affected by environmental changes.
[0004] Furthermore, traditional teaching methods cannot adaptively adjust when the AGV stops at different deviations, leading to grasping failures or damage to the mold.
[0005] Finally, to address spatial pose deviations, existing technologies typically employ 3D camera solutions. However, 3D cameras are extremely expensive and highly sensitive to metallic reflections at the processing site. Ordinary 2D vision solutions, on the other hand, often only detect planar deviations and cannot perceive changes in height and tilt.
[0006] Solving these difficulties usually requires expensive ground positioning devices, high-precision AGV navigation algorithms, or complex 3D vision systems, but this increases system costs and reduces flexibility.
[0007] Therefore, there is an urgent need for a method and system that can use low-cost 2D vision to achieve spatial pose correction and overcome positioning errors of mobile platforms. Summary of the Invention
[0008] The purpose of this invention is to provide a 2D vision-based pose correction method, system, and computer-readable storage medium to solve the problem of low loading and unloading accuracy of composite robots due to AGV positioning deviation in a low-cost manner, and to replace expensive 3D camera solutions to achieve high-precision and highly flexible automated loading and unloading.
[0009] To achieve this objective, the present invention adopts the following technical solution: In a first aspect, the present invention provides a pose correction method based on 2D vision, which includes the following steps: S1. An image of a calibration reference object is acquired by a 2D vision acquisition unit as an image to be processed. The calibration reference object is set at the target position and has a geometric feature array with a predetermined pattern. S2. Detect feature points of the geometric feature array in the image to be processed; S3. Based on the feature points, calculate the distance deviation from the 2D vision acquisition unit to the calibration reference, the tilt angle deviation of the 2D vision acquisition unit relative to the calibration reference, the planar position deviation, and the planar rotation angle deviation; S4. Combining the distance deviation, tilt angle deviation, planar position deviation, and planar rotation angle deviation, calculate the spatial pose deviation of the actuator relative to the reference pose. S5. Based on the spatial pose deviation, control the actuator to adjust its pose.
[0010] Preferably, step S3 specifically includes: S31. Based on the change ratio of the size of the feature point relative to the known actual size, calculate the distance deviation from the 2D vision acquisition unit to the calibration reference object; S32. Based on the degree of distortion of the shape of the feature point arrangement relative to the shape viewed from the front, calculate the tilt angle deviation of the 2D vision acquisition unit relative to the calibration reference. S33. Based on the overall displacement and rotation of the feature points, calculate the planar position deviation and the planar rotation angle deviation.
[0011] In some preferred embodiments, before step S1, the method further includes: Establish the transformation relationship between the coordinate system of the actuator and the coordinate system of the 2D vision acquisition unit; A baseline pose reference is established based on the aforementioned transformation relationship.
[0012] Specifically, establishing the transformation relationship between the coordinate system of the actuator and the coordinate system of the 2D vision acquisition unit includes: The actuator is controlled to take pictures of the calibration reference object in multiple different spatial poses; Based on image data from multiple spatial poses, a transformation relationship between the coordinate systems is established using a hand-eye calibration algorithm.
[0013] In some preferred embodiments, before step S2, the method further includes: The image to be processed is preprocessed, and the preprocessing includes at least one of image denoising, contrast enhancement, and grayscale normalization.
[0014] Preferably, step S2 specifically includes: S21. Detect candidate feature points; S22. Based on the roundness, size consistency and spatial distribution pattern of the candidate feature points, select effective feature points from the candidate feature points; S23. Remove abnormal feature points whose deviation exceeds the preset threshold.
[0015] In some preferred embodiments, the pose adjustment includes performing at least two pose corrections, specifically including: The first pose adjustment is performed based on the calculated spatial pose deviation. After the first pose adjustment, the steps of acquiring images, detecting feature points, and calculating spatial pose deviation are repeated to obtain the residual pose deviation. A second pose adjustment is performed based on the residual pose deviation.
[0016] Specifically, the residual translation deviation after the first pose adjustment is less than 10mm, and the final translation positioning accuracy after the second pose adjustment is less than 2mm.
[0017] Furthermore, after the second pose adjustment, the following is also included: Determine whether the spatial pose deviation meets the convergence condition, the convergence condition including: The residual pose deviation is less than a preset threshold; and / or The similarity between the current image and the reference image is greater than the preset similarity threshold.
[0018] In some preferred embodiments, step S4 further includes: Based on the transformation relationship between the coordinate system of the actuator and the coordinate system of the 2D vision acquisition unit, the pose deviation in the coordinate system of the 2D vision acquisition unit is converted into the pose deviation in the coordinate system of the actuator.
[0019] In some preferred embodiments, the geometric feature array is selected from at least one of a dot array, a checkerboard array, an ArUco marker array, an AprilTag marker array, and a concentric circle array.
[0020] Specifically, the geometric feature array is a dot array, comprising M×N circular feature points, where M and N are both positive integers not less than 4, and the center-to-center distance between the circular feature points is 5mm to 100mm.
[0021] In some preferred embodiments, the 2D vision-based pose correction method is applied to an automated system, the actuator is mounted on a mobile platform, and the 2D vision-based pose correction method further includes: The actuator is moved to the working position via the mobile platform.
[0022] Specifically, the mobile platform is an automated guided vehicle or an autonomous composite robot.
[0023] In some preferred embodiments, the actuator is a multi-degree-of-freedom robotic arm having at least three translational degrees of freedom and at least two rotational degrees of freedom.
[0024] Secondly, the present invention provides a pose correction system based on 2D vision, comprising: Executive agency; 2D vision acquisition unit; Calibration reference; The control unit is configured to perform the 2D vision-based pose correction method as described above.
[0025] Preferably, the 2D vision-based pose correction system further includes a mobile platform, the actuator is disposed on the mobile platform, the mobile platform is an automated guided vehicle or an autonomous composite robot, the 2D vision acquisition unit is fixedly installed on the actuator, and the geometric feature array is selected from at least one of dot array, checkerboard array, ArUco marker array, AprilTag marker array, and concentric circle array.
[0026] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the pose correction method based on 2D vision as described above.
[0027] Compared with the prior art, the present invention has the following beneficial effects: This invention addresses several issues. First, it identifies feature points on a geometric feature array on a calibration reference object using a 2D vision acquisition unit. Based on the size changes, shape distortions, and displacement rotations of these feature points, it simultaneously calculates distance deviation, tilt angle deviation, and planar deviation, thus achieving a complete solution for the six-degree-of-freedom spatial pose. This replaces expensive 3D cameras with a low-cost 2D vision solution. Second, by comprehensively calculating spatial pose deviations from multiple dimensions and controlling the actuators accordingly, it effectively compensates for the positioning errors of the mobile platform, improving the positioning accuracy of automated operations. Third, this method avoids the sensitivity of traditional 3D vision to metal reflections, enhancing its adaptability and stability in complex industrial environments.
[0028] The present invention has other features and advantages, which will be apparent from or will be set forth in detail in the accompanying drawings and the following detailed description, which together serve to explain the particular principles of the invention. Attached Figure Description
[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0030] Figure 1 This is a flowchart illustrating the pose correction method based on 2D vision according to Embodiment 1 of the present invention.
[0031] Figure 2 This is a schematic diagram of the pose correction system based on 2D vision according to Embodiment 2 of the present invention.
[0032] Explanation of reference numerals in the attached figures: 100. Mobile platform; 200. Actuator; 300. 2D vision acquisition unit; 400. Calibration reference object; 410. Geometric feature array; 500. Control unit; 600. Target device. Detailed Implementation
[0033] To illustrate the possible application scenarios, technical principles, implementable specific solutions, and achievable objectives and effects of this application in detail, the following description, in conjunction with the listed specific embodiments and accompanying drawings, provides a detailed explanation. The embodiments described herein are merely illustrative of the technical solutions of this application and are therefore intended to limit the scope of protection of this application.
[0034] Example 1 Please see Figure 1 and Figure 2 The pose correction method based on 2D vision in this embodiment can be applied to, for example... Figure 2 The automated production system shown can be a composite robot, comprising a mobile platform 100, an actuator 200, a 2D vision acquisition unit 300, a calibration reference object 400, and a control unit 500. The mobile platform 100 is an automated guided vehicle or an autonomous composite robot, and the actuator 200 is a multi-degree-of-freedom robotic arm with at least three translational degrees of freedom and at least two rotational degrees of freedom. The 2D vision acquisition unit 300 is fixedly mounted on the actuator 200, and the calibration reference object 400 is fixedly positioned at a designated location on the target equipment 600 (such as a CNC machine tool).
[0035] Before executing step S1, the initial calibration of the system needs to be completed. Specifically, the transformation relationship between the coordinate system of the actuator 200 and the coordinate system of the 2D vision acquisition unit 300 needs to be established. The prerequisite steps for step S1 include: The control actuator 200 takes pictures of the calibration reference object 400 in multiple different spatial poses; Based on image data from multiple spatial poses, a transformation relationship between coordinate systems is established using a hand-eye calibration algorithm.
[0036] Specifically, the camera intrinsic parameters are first calibrated to obtain the camera intrinsic parameters and lens distortion coefficients of the 2D vision acquisition unit 300. Understandably, to ensure calibration accuracy, at least 10 sets of data in different poses need to be acquired. Each set of data includes the pose of the actuator 200 end relative to the base coordinate system and the image of the calibration reference object 400 acquired in that pose.
[0037] The control actuator 200 moves the 2D vision acquisition unit 300, causing the calibration reference object 400 to appear at different positions and angles in the field of view. The vision system identifies features on the calibration reference object 400 to obtain the pose relationship between the camera optical center and the calibration reference object 400. By solving the equation AX=XB, the transformation matrix between the coordinate system of the actuator 200's end effector and the coordinate system of the 2D vision acquisition unit 300 is obtained.
[0038] A reference pose is established based on the transformation relationship. Specifically, the actuator 200 is manually operated to reach the ideal material handling position, and the 2D vision acquisition unit 300 captures a picture of the calibration reference object 400 and saves the image as a template image. The pose of the actuator 200 when the template is captured, the image of the calibration reference object 400 corresponding to the template is captured, and the pose of the actuator 200 when teaching material handling are performed are recorded and saved.
[0039] S1. The image of the calibration reference object 400 is acquired by the 2D vision acquisition unit 300 as the image to be processed. The calibration reference object 400 is set at the target position and has a geometric feature array 410 with a predetermined pattern.
[0040] In practical applications, the mobile platform 100 moves the actuator 200 to the working position, that is, moves the actuator 200 to the working range of the target device 600 and completes a rough docking. At this time, even if the mobile platform 100 has a positioning deviation (e.g., ±40mm in the front-back and left-right directions, ±20mm in the vertical direction, and ±40° in the rotation angle), the system can still compensate for it through subsequent visual correction.
[0041] The actuator 200 moves to the preset image capture position, and the 2D vision acquisition unit 300 acquires the image of the calibration reference object 400 as the image to be processed.
[0042] Before performing step S2, the image to be processed needs to be preprocessed, which includes at least one of image denoising, contrast enhancement, and grayscale normalization. The specific processing methods for image denoising, contrast enhancement, and grayscale normalization are not the focus of this invention and will not be described in detail here.
[0043] It is worth noting that in this embodiment, one or more 2D vision acquisition units 300 can acquire images of the calibration reference object 400 as images to be processed. Each 2D vision acquisition unit 300 can acquire one or more images of the calibration reference object 400, and the automated production system selects the optimal one or more images as images to be processed. When there are multiple images to be processed, they can be integrated into a single image through fitting or other methods, or the method can be applied to each image separately, and the final spatial pose deviation can be obtained by weighting the results of each spatial pose deviation. Further details are omitted here.
[0044] S2. Detect feature points of geometric feature array 410 in the image to be processed.
[0045] Step S2 specifically includes: S21. Detect candidate feature points.
[0046] The Hough circle detection algorithm or the Harris corner detection algorithm is used to identify possible feature point locations in the image to be processed. For the dot array, the Hough circle detection algorithm is used to detect all circular regions, while for the checkerboard array, the Harris corner detection algorithm is used to detect all corners. For example, after Hough circle detection, 150 candidate circular feature points were detected.
[0047] S22. Based on the roundness, size consistency and spatial distribution pattern of candidate feature points, select effective feature points from the candidate feature points.
[0048] If a roundness threshold (e.g., greater than 0.85) is set to remove irregularly shaped points, approximately 120 feature points will be retained after roundness filtering. Checking the standard deviation of feature point sizes (e.g., less than 10%) ensures size consistency, and approximately 105 feature points will be retained after size filtering.
[0049] Based on the known arrangement rules of the geometric feature array 410 (such as a regular array of 10×10), points that conform to the spatial distribution are selected, and 100 valid feature points are retained after spatial distribution selection.
[0050] S23. Remove abnormal feature points whose deviation exceeds the preset threshold.
[0051] The RANSAC algorithm or other outlier detection methods are used to remove outliers with significantly deviated positions. For example, the RANSAC algorithm is used to further remove two outliers, ultimately retaining 98 reliable feature points for subsequent pose calculation.
[0052] The three-step screening process described above ensures the reliability and accuracy of the final detected feature point set. Multiple experiments have verified that rigorously screened feature points can effectively improve pose calculation accuracy.
[0053] S3. Based on feature points, calculate the distance deviation from the 2D vision acquisition unit 300 to the calibration reference 400, the tilt angle deviation of the 2D vision acquisition unit 300 relative to the calibration reference 400, the planar position deviation, and the planar rotation angle deviation.
[0054] Step S3 specifically includes: S31. Based on the ratio of the change in the size of the feature points relative to the known actual size, calculate the distance deviation between the 2D vision acquisition unit 300 and the calibration reference object 400.
[0055] Understandably, according to the camera imaging model, the size of feature points in an image is inversely proportional to the distance from the camera to the calibration reference 400. For example, when the 2D vision acquisition unit 300 is far from the calibration reference 400, the size of the feature points in the image becomes smaller; when the distance is closer, the size of the feature points becomes larger.
[0056] The actual pixel size d of the feature points in the image is measured and compared with the known actual physical size D. The current distance is calculated using the formula Z=f×D / d (where Z is the distance and f is the focal length), and the distance deviation ΔZ is obtained by comparing it with the reference distance.
[0057] For example, given that the actual diameter D of a circular feature point on calibration reference 400 is 20mm and the camera focal length f is 12mm, in the reference pose, the diameter of the feature point in the image is 80 pixels, and the calculated reference distance Z0 is 3000mm. In the current pose, if the diameter of the feature point in the image is 76 pixels, then the current distance Z1 is 3158mm, and the distance deviation ΔZ = Z1 - Z0 = 158mm, indicating that the camera is 158mm further away than the reference pose.
[0058] S32. Based on the degree of distortion of the shape of the feature point arrangement relative to the shape viewed from the front, calculate the tilt angle deviation of the 2D vision acquisition unit 300 relative to the calibration reference object 400.
[0059] Understandably, when the 2D vision acquisition unit 300 is directly facing the calibration reference 400, circular feature points appear as perfect circles in the image, and the array appears as a regular rectangular arrangement. When the 2D vision acquisition unit 300 is tilted relative to the calibration reference 400, the circles appear as ellipses, and the rectangular array appears as a trapezoidal distortion. By analyzing the degree and direction of this shape distortion, the tilt angle can be determined.
[0060] The homography matrix decomposition method or the PnP algorithm is employed. Based on the detected feature point positions in the image and their known actual spatial positions, a correspondence between 2D image coordinates and 3D spatial coordinates is established. By solving the homography matrix H and decomposing H, the rotation matrix R and translation vector t of the camera relative to the calibration reference 400 are obtained. The pitch angle (rotation around the X-axis, denoted as ΔRx) and yaw angle (rotation around the Y-axis, denoted as ΔRy) are extracted from the rotation matrix R, representing the tilt angle deviation.
[0061] If the rotation matrix R is obtained by solving the PnP algorithm, after decomposition, we get ΔRx=3.2° and ΔRy=-1.8°, which means that the camera has a tilt angle of 3.2° and a yaw angle of -1.8° relative to the reference pose.
[0062] S33. Based on the overall displacement and rotation of the feature points, calculate the planar position deviation and the planar rotation angle deviation.
[0063] The detected feature point array is matched and compared with the feature point array in the template image. A feature point matching algorithm is used to establish the correspondence between the current feature points and the template feature points. The translation of the entire feature point array, i.e., the displacement of the array center point on the image plane, is calculated to obtain the planar position deviation (ΔX, ΔY). The rotation angle of the entire feature point array, i.e., the rotation angle of the array relative to the template array around the image center, is calculated to obtain the planar rotation angle deviation ΔRz.
[0064] For example, comparing the current image with the template image, it was found that the center of the feature point array was shifted 120 pixels to the right and 80 pixels downward in the image. Combining the camera intrinsic parameters and the current distance, the actual planar position deviations were calculated to be ΔX=35mm and ΔY=25mm. At the same time, the array as a whole was rotated clockwise by 5.6°, that is, the planar rotation angle deviation ΔRz=5.6°.
[0065] Through the above steps S31, S32 and S33, the distance deviation, tilt angle deviation and plane deviation were calculated, and the complete deviation information of the camera relative to the calibration reference 400 in six degrees of freedom in space was obtained.
[0066] S4. Calculate the spatial pose deviation of the actuator 200 relative to the reference pose by combining the distance deviation, tilt angle deviation, planar position deviation, and planar rotation angle deviation.
[0067] The six deviation components (ΔX, ΔY, ΔZ, ΔRx, ΔRy, ΔRz) calculated in step S3 are combined to form a complete pose deviation vector in the 300 coordinate system of the 2D vision acquisition unit.
[0068] Furthermore, step S4 also includes: Based on the transformation relationship between the coordinate system of the actuator 200 and the coordinate system of the 2D vision acquisition unit 300, the pose deviation in the coordinate system of the 2D vision acquisition unit 300 is converted into the pose deviation in the coordinate system of the actuator 200.
[0069] It is understandable that the pose deviation calculated in the camera coordinate system is transformed to the end coordinate system of the actuator 200 by using the hand-eye calibration transformation matrix obtained in the initial calibration stage. Then, by combining the transformation relationship between the end coordinate system of the actuator 200 and the base coordinate system, the pose deviation in the base coordinate system of the actuator 200 is finally obtained.
[0070] By using the hand-eye calibration matrix, the current pose of actuator 200, and the pose of actuator 200 at the template imaging position, the poses of the current calibration reference 400 and the template calibration reference 400 in the base coordinate system of actuator 200 are calculated. Based on the pose transformation matrices of the two calibration references 400 in the base coordinate system of actuator 200, the spatial pose deviation of the actuator 200 end effector relative to the reference pose is calculated in real time.
[0071] If the pose deviation in the camera coordinate system is ΔX cam =35mm,ΔY cam =25mm,ΔZ cam =158mm,ΔRx cam =3.2°,ΔRy cam =-1.8°,ΔRz cam =5.6°, and the pose deviation of the actuator in the 200 base coordinate system is obtained by coordinate transformation through the hand-eye calibration matrix. base =40mm,ΔY base =-30mm,ΔZ base =20mm,ΔRx base =2.5°, ΔRy base =1.2°, ΔRz base =-38°.
[0072] S5. Adjust the pose of the actuator 200 based on spatial pose deviation control.
[0073] Preferably, adjusting the pose includes performing at least two pose corrections, specifically including: The first pose adjustment is performed based on the calculated spatial pose deviation. The control unit 500 calculates the spatial pose deviation (ΔX) based on step S4. base ΔY base ΔZ base ΔRx base ΔRy base , ΔRz base The motion command of the actuator 200 is generated, and the actuator 200 is controlled to make the first pose adjustment, so that the 2D vision acquisition unit 300 is initially adjusted back to the vicinity of the template position.
[0074] For example, the actuator 200 can be translated (-40mm, +30mm, -20mm) and rotated (-2.5°, -1.2°, +38°) in the base coordinate system to make the end pose of the actuator 200 close to the reference pose.
[0075] After the first pose adjustment, the steps of image acquisition, feature point detection, and spatial pose deviation calculation are repeated to obtain the residual pose deviation. After the actuator 200 is adjusted to the correct position, the process of steps S1 to S4 is repeated, that is, the 2D vision acquisition unit 300 performs a second image capture (repeating step S1), performs image preprocessing (repeating the preprocessing step), detects feature points (repeating step S2), calculates the deviation (repeating step S3), and calculates the residual pose deviation after the first adjustment (repeating step S4).
[0076] The residual translational deviation after the first pose adjustment is less than 10 mm. In one embodiment, the residual pose deviations measured after the first adjustment are ΔX'=4.2 mm, ΔY'=-3.8 mm, ΔZ'=2.1 mm, ΔRx'=0.3°, ΔRy'=-0.2°, ΔRz'=0.4°, and the magnitude of the translational deviation is √(4.2). 2 +3.8 2 +2.1 2 The accuracy is approximately 5.9 mm, less than 10 mm, which is significantly better than the initial positioning accuracy of the mobile platform 100.
[0077] A second pose adjustment is performed based on the residual pose deviation. The control unit 500 controls the actuator 200 to perform a second pose adjustment based on the residual pose deviation calculated in the second calculation, further improving positioning accuracy.
[0078] Based on the residual pose deviations (ΔX'=4.2mm, ΔY'=-3.8mm, ΔZ'=2.1mm, ΔRx'=0.3°, ΔRy'=-0.2°, ΔRz'=0.4°), the actuator 200 is controlled to make fine adjustments (-4.2mm, +3.8mm, -2.1mm, -0.3°, +0.2°, -0.4°).
[0079] The final translational positioning accuracy after the second pose adjustment is less than 2 mm. In most cases, the final positioning accuracy can reach within 0.3 mm. In an experiment containing 20 sets of tests, the final pose deviations measured after the second adjustment were: a range of 0.254 mm in the X-axis direction, a range of 0.225 mm in the Y-axis direction, a range of 0.18 mm in the Z-axis direction, and a range of less than 0.05° in the rotation angle.
[0080] After the second pose adjustment, this method also includes: To determine whether the spatial pose deviation meets the convergence criteria, the convergence criteria in this embodiment include: The residual pose deviation is less than the preset threshold; And / or the similarity between the current image and the reference image is greater than a preset similarity threshold.
[0081] The preset thresholds can be set to translational deviation less than 0.5 mm and rotational deviation less than 0.1°. In other words, when the measured residual pose deviation meets the above threshold requirements, it is determined that the convergence condition has been met.
[0082] And / or, the SSIM metric is used to calculate the similarity between the current image and the reference image (template image). That is, when the similarity is greater than a preset similarity threshold (e.g., 0.98), it is determined that the convergence condition is met.
[0083] If the convergence condition is met, the pose correction is complete, and the actuator 200 has been adjusted to the precise position. If the convergence condition is not met, a third pose correction can be performed, and the above process can be repeated until the convergence condition is met or the preset maximum number of corrections is reached.
[0084] After pose correction is completed, the vision system, based on the feedback from the second (or last) capture and combined with the teaching pick-and-place pose recorded during the reference pose, corrects the pose of the actuator 200 during the teaching pick-and-place process to the final pick-and-place coordinates. Using the transformation matrix between the current actuator 200 base coordinate system and the actuator 200 base coordinate system at the template position, the same transformation is performed on the teaching pick-and-place pose to obtain the corrected pick-and-place coordinates.
[0085] The actuator 200 performs gripping or placing actions based on these precise coordinates to complete the workpiece unloading or loading operation. This method effectively eliminates the cumulative errors caused by inaccurate docking of the moving platform 100, ensuring the loading and unloading accuracy of high-precision machined workpieces.
[0086] To more clearly illustrate the practical application of the present invention, the following is combined with... Figure 2 Describe a complete job workflow example.
[0087] In practical applications, after receiving a work instruction, the mobile platform 100 travels along a predetermined path to the vicinity of the target device 600 and docks. Due to factors such as uneven ground, load variations, and cumulative sensor errors, the actual docking position of the mobile platform 100 usually deviates from the ideal position.
[0088] The control unit 500 initiates the pose correction process. The actuator 200 moves to the preset image capture position, and the 2D vision acquisition unit 300 acquires the first image. After image preprocessing, the feature detection module detects candidate feature points in the image to be processed. After roundness filtering, size consistency checking, and spatial distribution filtering, valid feature points are retained. The RANSAC algorithm is used to further eliminate abnormal feature points, and finally a reliable set of feature points is obtained for pose calculation.
[0089] The pose calculation module calculates distance deviation, tilt angle deviation, planar position deviation, and planar rotation angle deviation based on the detected feature points. After coordinate system transformation, the control unit 500 generates the first pose adjustment command, which controls the actuator 200 to perform preliminary pose correction.
[0090] After the first adjustment, the 2D vision acquisition unit 300 acquires a second image, repeats the feature detection and pose calculation process, and obtains the residual pose deviation. According to the technical solution of the present invention, the residual translation deviation after the first pose adjustment is less than 10mm, which meets the conditions for performing a second fine adjustment.
[0091] A second fine adjustment is performed based on the residual deviation. After the adjustment is completed, a third image is acquired and the final residual deviation is calculated. When the residual pose deviation is less than a preset threshold and the similarity between the current image and the template image meets the requirements, the pose correction is considered complete. According to the technical solution of this invention, the final translational positioning accuracy after the second pose adjustment is less than 2mm, and in most cases it can reach within 0.3mm.
[0092] After the pose correction is completed, the control unit 500 corrects the taught pick-and-place pose to the actual coordinate system according to the transformation relationship between the current pose and the reference pose. The actuator 200 moves to the corrected pick-and-place position to complete the workpiece gripping or placement action. As can be seen from this example, the two-step vision correction method of the present invention can effectively compensate for the docking deviation of the moving platform 100 and ensure high-precision automated operation.
[0093] In this embodiment, the geometric feature array 410 is selected from at least one of the following: dot array, checkerboard array, ArUco marker array, AprilTag marker array, and concentric circle array.
[0094] Specifically, in this embodiment, the geometric feature array 410 is a dot array, which includes M×N circular feature points, where M and N are both positive integers not less than 4, and the center-to-center spacing between the circular feature points is 5mm to 100mm. In one specific embodiment, a 10×10 dot array is used, with a total of 100 circular feature points. The diameter of the circular feature points is 10mm, the center-to-center spacing is 20mm, the overall size of the calibration plate is 200mm×200mm, and the manufacturing tolerance is ±0.05mm.
[0095] In fact, dot arrays have advantages such as distinctive features, ease of detection, mature detection algorithms, fast computation speed, insensitivity to changes in lighting, and good suppression of metal reflections. In industrial applications, the success rate and accuracy of dot arrays are superior to those of checkerboard arrays.
[0096] Alternatively, a checkerboard array can be used as the geometric feature array 410, suitable for high-contrast environments. ArUco marker arrays or AprilTag marker arrays can also be used; these markers have unique codes, which can avoid mismatches of feature points. A concentric circle array can also be used, providing richer feature information through the center and rings. In practical use, the array type of the geometric feature array 410 needs to be selected according to actual requirements; no restrictions are imposed here.
[0097] It is worth noting that, in different application scenarios, the system parameters in this embodiment can be reasonably adjusted according to actual needs.
[0098] For applications requiring higher precision, larger-scale geometric feature arrays, higher-resolution industrial cameras, and more refined processing methods can be used at the algorithm level.
[0099] For example, in applications with a large working distance, the size of the calibration reference object 400 needs to be enlarged accordingly, and the 2D vision acquisition unit 300 can be equipped with a telephoto lens to ensure sufficient image resolution.
[0100] For special environments such as dust, oil, vibration, and strong light, the environmental adaptability of the system can be improved by adopting measures such as sealing protection, adjusting exposure parameters, and introducing filtering algorithms.
[0101] Example 2 Please see Figure 1 and Figure 2 The 2D vision-based pose correction system of this embodiment is used to execute the 2D vision-based pose correction method described in Embodiment 1. The 2D vision-based pose correction system of this embodiment belongs to an automated production system and can be, but is not limited to, a composite robot.
[0102] The pose correction system based on 2D vision in this embodiment includes an actuator 200, a 2D vision acquisition unit 300, a calibration reference 400, and a control unit 500. The actuator 200 is a multi-degree-of-freedom robotic arm with at least three translational degrees of freedom and at least two rotational degrees of freedom, preferably a six-degree-of-freedom articulated robot.
[0103] The end of the actuator 200 may be fitted with a gripper, suction cup, or other material handling tool. In one specific embodiment, the actuator 200 is a six-axis industrial robot.
[0104] The 2D vision acquisition unit 300 is fixedly mounted on the actuator 200, preferably on the end effector of the actuator 200. The 2D vision acquisition unit 300 includes an industrial camera and a lens.
[0105] The calibration reference 400 is fixedly installed at a designated position on the target equipment 600 (such as a CNC machine tool). The calibration reference 400 has a geometric feature array 410 with a predetermined pattern. The geometric feature array 410 is selected from at least one of a dot array, a checkerboard array, an ArUco marker array, an AprilTag marker array, and a concentric circle array. Preferably, in this embodiment, the geometric feature array 410 is a 10×10 dot array, with the diameter of the circular feature points being 10mm and the center-to-center spacing being 20mm. The calibration plate adopts a white background with black dots, is made of aluminum alloy, and has an anodized surface to reduce reflection. The calibration reference 400 is fixed to the worktable or side wall of the target equipment 600 by magnetic attraction or bolts to ensure that its position remains stable throughout the entire operation.
[0106] The control unit 500 is configured to execute the 2D vision-based pose correction method described in Embodiment 1. The control unit 500 includes a processor and a memory, which stores software modules such as image processing algorithms, hand-eye calibration algorithms, and pose calculation algorithms. The control unit 500 is connected to the actuator 200 and the 2D vision acquisition unit 300 via an industrial communication bus (such as EtherCAT, Profinet, etc.) to achieve real-time data transmission and motion control.
[0107] Preferably, the 2D vision-based pose correction system of this embodiment further includes a mobile platform 100, on which the actuator 200 is mounted. The mobile platform 100 is an automated guided vehicle or an autonomous composite robot. The mobile platform 100 uses laser navigation or magnetic tape navigation. The mobile platform 100 is wirelessly connected to the control unit 500 to receive work instructions and feedback position information.
[0108] It is understandable that the workflow of this system is as described in Example 1. Through the strategy of coarse positioning + two-step visual correction, it achieves high-precision and high-flexibility automated loading and unloading, while having better adaptability to environmental factors such as metal reflection and dust in industrial sites.
[0109] Example 3 Please see Figure 1 and Figure 2 The computer-readable storage medium of this embodiment stores a computer program thereon, which, when executed by a processor, implements the pose correction method based on 2D vision as described in Embodiment 1.
[0110] The software architecture of the control unit 500 adopts a modular design, including an image acquisition and preprocessing module, a feature detection and matching module, a pose calculation and coordinate transformation module, and a motion control and communication module.
[0111] The image acquisition and preprocessing module is responsible for communicating with the 2D vision acquisition unit 300 and acquiring images, and performing preprocessing operations such as denoising, contrast enhancement, and grayscale normalization on the images.
[0112] The feature detection and matching module is responsible for detecting and matching feature points in the geometric feature array 410. The module includes multiple algorithms such as Hough circle detection, Harris corner detection, and ArUco marker detection. Upon system startup, the appropriate algorithm is automatically selected based on the type of the calibration reference object 400. Feature selection employs a multi-level strategy: first, selection is based on geometric features such as roundness and size; then, topological selection is performed based on the array arrangement pattern; and finally, the RANSAC algorithm is used to remove outliers.
[0113] The pose calculation and coordinate transformation module calculates spatial pose deviations based on feature points. Internally, the module includes a distance deviation calculation unit, an angle deviation calculation unit, and a planar deviation calculation unit. The angle deviation calculation unit employs the PnP algorithm or homography matrix decomposition method. The coordinate transformation unit is responsible for transforming the pose deviations from the camera coordinate system to the base coordinate system of the actuator 200.
[0114] The motion control and communication module is responsible for communicating with the actuator 200 and the mobile platform 100, sending motion commands and receiving pose feedback. Communication with the actuator 200 can use industrial buses such as EtherCAT or Profinet to achieve real-time motion synchronization.
[0115] Combination Figure 1 and Figure 2 This invention, on the one hand, identifies feature points of the geometric feature array on the calibration reference object through a 2D vision acquisition unit. Based on the size changes, shape distortions, and displacement rotations of these feature points, it can simultaneously calculate distance deviation, tilt angle deviation, and planar deviation, thereby achieving a complete solution for the six-degree-of-freedom spatial pose. This replaces expensive 3D cameras with a low-cost 2D vision solution. On the other hand, by comprehensively calculating spatial pose deviations from multiple dimensions and controlling the actuator to adjust the pose accordingly, it can effectively compensate for the positioning error of the mobile platform and improve the positioning accuracy of automated operations. Furthermore, this method avoids the problem of traditional 3D vision being sensitive to metal reflections, enhancing its adaptability and stability in complex industrial environments.
[0116] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A 2D vision-based pose correction method, characterized in that, Includes the following steps: The image of the calibration reference object is acquired by a 2D vision acquisition unit as the image to be processed. The calibration reference object is set at the target position and has a geometric feature array with a predetermined pattern. Detect feature points of the geometric feature array in the image to be processed; Based on the feature points, calculate the distance deviation from the 2D vision acquisition unit to the calibration reference, the tilt angle deviation of the 2D vision acquisition unit relative to the calibration reference, the planar position deviation, and the planar rotation angle deviation; By combining the distance deviation, tilt angle deviation, planar position deviation, and planar rotation angle deviation, the spatial pose deviation of the actuator relative to the reference pose is calculated; The actuator is controlled to adjust its posture based on the spatial pose deviation.
2. The 2D vision-based pose correction method of claim 1, wherein, The calculation of the distance deviation from the 2D vision acquisition unit to the calibration reference object, the tilt angle deviation of the 2D vision acquisition unit relative to the calibration reference object, the planar position deviation, and the planar rotation angle deviation based on the feature points specifically includes: Based on the ratio of the change in the size of the feature point relative to the known actual size, the distance deviation from the 2D vision acquisition unit to the calibration reference is calculated. Based on the degree of distortion of the shape of the feature point arrangement relative to the shape viewed from the front, the tilt angle deviation of the 2D vision acquisition unit relative to the calibration reference is calculated. Based on the overall displacement and rotation of the feature points, the planar position deviation and planar rotation angle deviation are calculated.
3. The 2D vision-based pose correction method of claim 1, wherein, Before acquiring the image of the calibration reference object, the process also includes: Establish the transformation relationship between the coordinate system of the actuator and the coordinate system of the 2D vision acquisition unit; A baseline pose reference is established based on the aforementioned transformation relationship.
4. The 2D vision-based pose correction method of claim 3, wherein, The establishment of the transformation relationship between the coordinate system of the actuator and the coordinate system of the 2D vision acquisition unit specifically includes: The actuator is controlled to take pictures of the calibration reference object in multiple different spatial poses; Based on image data from multiple spatial poses, a transformation relationship between the coordinate systems is established using a hand-eye calibration algorithm.
5. The pose correction method based on 2D vision as described in claim 1, characterized in that, Before detecting feature points of the geometric feature array in the image to be processed, the method further includes: The image to be processed is preprocessed, and the preprocessing includes at least one of image denoising, contrast enhancement, and grayscale normalization.
6. The pose correction method based on 2D vision as described in claim 1, characterized in that, The detection of feature points of the geometric feature array in the image to be processed specifically includes: Detect candidate feature points; Based on the roundness, size consistency, and spatial distribution pattern of the candidate feature points, valid feature points are selected from the candidate feature points; Remove abnormal feature points whose deviation exceeds a preset threshold.
7. The pose correction method based on 2D vision as described in claim 1, characterized in that, The pose adjustment includes performing at least two pose corrections, specifically including: The first pose adjustment is performed based on the calculated spatial pose deviation. After the first pose adjustment, the steps of acquiring images, detecting feature points, and calculating spatial pose deviation are repeated to obtain the residual pose deviation. A second pose adjustment is performed based on the residual pose deviation.
8. The pose correction method based on 2D vision as described in claim 7, characterized in that, The residual translational deviation after the first pose adjustment is less than 10mm, and the final translational positioning accuracy after the second pose adjustment is less than 2mm.
9. The pose correction method based on 2D vision as described in claim 7, characterized in that, Following the second pose adjustment, the following is also included: Determine whether the spatial pose deviation meets the convergence condition, the convergence condition including: The residual pose deviation is less than a preset threshold; and / or The similarity between the current image and the reference image is greater than the preset similarity threshold.
10. The pose correction method based on 2D vision as described in claim 1, characterized in that, The calculation of the spatial pose deviation of the execution mechanism relative to the reference pose also includes: Based on the transformation relationship between the coordinate system of the actuator and the coordinate system of the 2D vision acquisition unit, the pose deviation in the coordinate system of the 2D vision acquisition unit is converted into the pose deviation in the coordinate system of the actuator.
11. The pose correction method based on 2D vision as described in claim 1, characterized in that, The geometric feature array is selected from at least one of the following: dot array, checkerboard array, ArUco marker array, AprilTag marker array, and concentric circle array.
12. The pose correction method based on 2D vision as described in claim 11, characterized in that, The geometric feature array is a dot array, comprising M×N circular feature points, where M and N are both positive integers not less than 4, and the center-to-center distance between the circular feature points is 5mm to 100mm.
13. The pose correction method based on 2D vision as described in claim 1, characterized in that, The 2D vision-based pose correction method is applied to an automated system, wherein the actuator is mounted on a mobile platform, and the 2D vision-based pose correction method further includes: The actuator is moved to the working position via the mobile platform.
14. The pose correction method based on 2D vision as described in claim 13, characterized in that, The mobile platform is an automated guided vehicle or an autonomous composite robot.
15. The pose correction method based on 2D vision as described in claim 1, characterized in that, The actuator is a multi-degree-of-freedom robotic arm, which has at least three translational degrees of freedom and at least two rotational degrees of freedom.
16. A pose correction system based on 2D vision, characterized in that, include: Executive agency; 2D vision acquisition unit; Calibration reference; The control unit is configured to perform the 2D vision-based pose correction method as described in any one of claims 1 to 15.
17. The pose correction system based on 2D vision as described in claim 16, characterized in that, It also includes a mobile platform, on which the actuator is mounted. The mobile platform is an automated guided vehicle or an autonomous composite robot. The 2D vision acquisition unit is fixedly installed on the actuator. The geometric feature array is selected from at least one of the following: dot array, checkerboard array, ArUco marker array, AprilTag marker array, and concentric circle array.
18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the pose correction method based on 2D vision as described in any one of claims 1 to 15.