A method for hand-eye calibration error compensation of structured light cameras based on L-shaped calibration blocks

Abstract

This invention discloses a method for compensating hand-eye calibration errors of a structured light camera based on an L-shaped calibration block, belonging to the field of machine vision and robot control technology. First, a hand-eye calibration matrix is ​​calculated using a standard calibration board to establish the initial transformation relationship between the camera and the robot's base coordinate system. Then, the L-shaped calibration block is used to collect the visual observation coordinates of feature corner points and the robot's actual coordinates under various poses, constructing a spatially corresponding point pair dataset. Based on this, a two-level cascaded comprehensive error compensation model is constructed: the first level uses a global rigid compensation matrix based on singular value decomposition to eliminate overall deviations; the second level uses a radial basis function neural network to establish a mapping model for local nonlinear residual errors, compensating for position-related residual errors. Finally, during robot operation, this model is used to perform real-time high-precision correction of visual guidance commands. This invention can effectively compensate for the composite errors after hand-eye calibration, significantly improving the absolute positioning accuracy of vision-guided robots.

Description

Technical Field

[0001] This invention relates to the field of machine vision and robot control technology, specifically to a method for compensating hand-eye calibration errors of a structured light camera based on an L-shaped calibration block. Background Technology

[0002] The theoretical upper limit of the operational accuracy of vision-guided robot systems is determined by the accuracy of hand-eye calibration. Traditional hand-eye calibration methods rely on a calibration board to solve the hand-eye transformation matrix. This process inevitably introduces errors, leading to a systematic deviation between the visual observation coordinate system and the robot's actual execution coordinate system. Furthermore, inherent mechanical manufacturing and motion control errors in the robot itself create a pose-related nonlinear residual error field in its workspace. Therefore, after hand-eye calibration, the total system error is a superposition of overall rigidity deviation and local nonlinear residuals. Traditional single hand-eye calibration matrices cannot effectively compensate for this composite error of overall rigidity deviation and local nonlinear residuals, making it difficult for vision-guided robots to meet the requirements of high-precision operations in terms of absolute positioning accuracy. Summary of the Invention

[0003] The purpose of this invention is to provide a method for compensating hand-eye calibration errors of a structured light camera based on an L-shaped calibration block. By decomposing the total system error after hand-eye calibration into an overall rigid deviation and a local nonlinear residual error, a two-level compensation model combining a global rigid compensation matrix and a radial basis function neural network is adopted to perform real-time high-precision correction of visual guidance commands, thereby effectively eliminating the influence of composite errors on robot positioning accuracy and meeting the requirements of high-precision operation.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for compensating hand-eye calibration errors in structured light cameras based on L-shaped calibration blocks, applicable to eye-external configurations, includes the following steps: S1: Calculate the hand-eye calibration matrix and establish the initial transformation relationship between the structured light camera coordinate system and the robot base coordinate system; S2: Collect corresponding point pairs in space, place the L-shaped calibration block on the worktable in various poses, obtain the visual observation coordinates of the feature corner points on the L-shaped calibration block and the actual coordinates of the robot, and construct a dataset for error compensation; S3: Construct a comprehensive error compensation model. Utilize the spatial corresponding point pairs to establish a two-level series comprehensive compensation model. The two-level series comprehensive compensation model includes a global rigid compensation matrix for compensating for overall deviations and a mapping model for compensating for local nonlinear residual errors. S4: Online error compensation. During robot operation, the comprehensive error compensation model is used to compensate for visual guidance commands in real time.

[0005] Further, step S1 specifically includes: fixing the structured light camera in an eye-in-hand mode, and fixing the standard calibration plate to the robot's end effector; controlling the robot to move the calibration plate to N different poses within the camera's field of view, where N≥3; at each pose, acquiring an image of the calibration plate and calculating its pose matrix in the camera coordinate system, while simultaneously recording the pose matrix of the end effector in the robot's base coordinate system; based on the hand-eye calibration equation... Solve for the transformation matrix from the camera coordinate system to the robot base coordinate system.

[0006] Furthermore, the acquisition of spatially corresponding point pairs in step S2 specifically includes: S2.1: Design an L-shaped three-dimensional calibration block. Its overall appearance is an L-shaped geometry, with each side perpendicular to the top and bottom surfaces, and the surface is matte. S2.2: Within the shared coverage area of ​​the structured light camera's field of view and the robot's workspace, L-shaped calibration blocks are placed on the planar worktable in various poses. For each pose, point clouds are acquired through the structured light camera and processed to obtain the three-dimensional coordinates of the feature corner points in the camera coordinate system. After transformation by the hand-eye calibration matrix, the visual observation coordinates in the robot's base coordinate system are obtained. Simultaneously, the center point of the robot's end effector is controlled to precisely touch the corresponding physical feature corner point, and the actual coordinates in the robot's base coordinate system are recorded. All poses are traversed to obtain a set of corresponding point pairs in space.

[0007] Furthermore, step S2.2, which involves acquiring point clouds using a structured light camera and processing them to obtain the three-dimensional coordinates of the feature corner points in the camera coordinate system, specifically includes the following steps: S2.2.1: Use a structured light camera to scan the worktable where the L-shaped calibration block is placed to obtain the initial scene point cloud. The equation of the worktable plane was obtained by fitting the RANSAC algorithm. S2.2.2: Place the L-shaped calibration block on the worktable in K different poses. For each pose, collect scene point clouds, calculate the distance from each point in the point cloud to the worktable plane, and determine the points whose distance is greater than a preset threshold as calibration block point clouds to achieve calibration block point cloud segmentation. S2.2.3: The RANSAC algorithm is used again to fit the upper surface plane of the calibration block point cloud, the set of interior points belonging to the plane is obtained and projected vertically onto the plane to obtain the two-dimensional projected point cloud; S2.2.4: The Alpha Shape algorithm is used to extract the boundary contour of the two-dimensional projected point cloud to obtain the two-dimensional contour point set. Then, the RANSAC algorithm is used to fit the straight line multiple times to extract multiple boundary line segments on the upper surface of the calibration block. S2.2.5: Project the interior points of each boundary line segment onto the corresponding straight line and sort them. Take the first and last endpoints as three-dimensional vertices. Based on the length ratio and topological connection relationship of each boundary line segment, determine the correspondence between the extracted line segments and the physical edge lines of the calibration block. By solving the intersection points of the intersecting line segments, calculate the three-dimensional coordinates of multiple feature corner points on the upper surface of the calibration block in the camera coordinate system.

[0008] Furthermore, the construction of the comprehensive error compensation model in step S3 specifically includes: S3.1: Define the spatial corresponding point pairs obtained in step S2 as the visual observation point set and the robot measured point set. Use the absolute orientation algorithm based on singular value decomposition to solve the optimal rigid transformation from the visual observation point set to the robot measured point set, and obtain the global rigidity error compensation matrix. S3.2: Use the global rigid error compensation matrix to perform preliminary correction on each visual observation coordinate to obtain the rigidly compensated coordinates; for each data point, calculate the difference between the rigidly compensated coordinates and the corresponding robot actual coordinates to obtain the residual error vector of that point; use all the obtained rigidly compensated coordinates as input features and the residual error vector as output targets to construct a training dataset; establish a nonlinear mapping function based on this dataset to predict the residual error vector at any position in space.

[0009] Furthermore, the nonlinear mapping function is implemented using a radial basis function network model, which decouples the prediction problem of the three-dimensional residual error vector into three independent scalar regression problems. The nonlinear mapping function consists of three radial basis function sub-networks with the same structure but independent weights, which are used to predict the residual error components in the X, Y and Z directions, respectively.

[0010] Furthermore, in the radial basis function network model, the number of hidden layer nodes is taken as the number of training samples, and each training sample point is used as the center of the Gaussian kernel function, which is defined as: in, Input coordinates, For the first The center of a Gaussian kernel function, This is the width parameter.

[0011] Furthermore, the width parameter The nearest neighbor heuristic is used to determine this, i.e., take... The output layers of each subnetwork are linearly weighted sums. Taking the X-direction subnetwork as an example, its output is: in, This represents the number of hidden layer nodes. The weighting coefficients are obtained using the least squares method.

[0012] Furthermore, the global rigid error compensation matrix The transformation matrix is ​​a 4×4 homogeneous transformation matrix containing the rotation matrix R and the translation vector t, and its expression is: This matrix is ​​used to perform a rigid transformation correction on the original visual guide coordinates.

[0013] Furthermore, the online error compensation in step S4 specifically includes: S4.1: Scan the target using a structured light camera, calculate the three-dimensional coordinates of the target feature points in the camera coordinate system, and use the hand-eye calibration matrix from step S1 to transform them to the robot base coordinate system to obtain the original visual guidance coordinates without any compensation. S4.2: Input the original visual guidance coordinates into the global rigid error compensation matrix to obtain the rigidly compensated coordinates; S4.3: Input the rigidly compensated coordinates into the nonlinear mapping model to predict the local nonlinear residual error vector; S4.4: Superimpose the residual error vector with the rigidly compensated coordinates to obtain the final corrected target coordinates, and send them to the robot controller for execution.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. The L-shaped 3D calibration block designed in this invention has a simple structure and well-defined features. It can be used in conjunction with algorithms such as point cloud segmentation, boundary extraction, and corner point calculation to stably obtain the 3D coordinates of multiple feature corner points on the calibration block in the camera coordinate system. By directly touching the corresponding physical corner points with the center point of the robot's end effector, the actual coordinates of these corner points in the robot's base coordinate system can be obtained simultaneously. In this way, a high-precision dataset of spatially corresponding point pairs can be established under multiple different poses within the workspace, providing a rich and reliable data foundation for subsequent error modeling.

[0015] 2. This invention constructs a two-stage error compensation model that combines global rigidity compensation and local nonlinearity compensation. The first stage calculates the optimal rigidity transformation matrix through singular value decomposition to eliminate the overall rigidity deviation between the visual observation coordinate system and the robot's actual execution coordinate system. The second stage establishes a nonlinear mapping model related to spatial position to address the residual error that still exists after rigidity compensation. This architecture can simultaneously handle the systematic deviations caused by hand-eye calibration itself and the complex nonlinear errors formed by the robot's mechanical and control errors, achieving high compensation accuracy and good robustness.

[0016] 3. The second-level compensation of this invention is implemented using a radial basis function (RBF) network. The RBF network has local activation characteristics, enabling it to automatically learn and fit the variation patterns of residual errors at different locations in the workspace, and predict the possible residual error value at any input location. The entire process does not require manual division of the workspace region, nor does it require prior assumptions about the error distribution. The model can adaptively complete the smooth and continuous error field modeling, making it more flexible and convenient.

[0017] 4. The online compensation process of this invention is clear and highly integrated. During normal robot operation, the target point coordinates acquired by the camera are first transformed using an initial hand-eye matrix, and then corrected in real time using a global rigid compensation matrix and the residual error predicted by the RBF network. Finally, the corrected target coordinates are sent to the robot controller. The entire process is completed automatically without human intervention, which can significantly improve the absolute positioning accuracy of vision-guided robots in high-precision machining or assembly tasks, and has high engineering practical value. Attached Figure Description

[0018] Figure 1 This is an overall flowchart of the present invention; Figure 2 This is an overall layout diagram in an embodiment of the present invention; Figure 3 This is a schematic diagram of an L-shaped calibration block in an embodiment of the present invention; Figure 4 This is a schematic diagram of the reconstructed calibration block point cloud and the extracted feature corner points in an embodiment of the present invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0020] It should be noted that, unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0021] This embodiment provides a method for compensating for hand-eye calibration errors in structured light cameras based on L-shaped calibration blocks, which is suitable for eye-out configurations. Figure 2 The diagram shows the overall layout of the invention. The structured light camera is fixedly installed with the eye outside the hand. The robotic arm is located next to the workbench. Within the common coverage area of ​​the structured light camera's field of view and the robot's workspace, the L-shaped calibration block is placed on the workbench in various different poses.

[0022] The method in this embodiment includes the following four core steps, such as Figure 1 As shown: S1: Calculate the hand-eye calibration matrix and establish the initial transformation relationship between the structured light camera coordinate system and the robot base coordinate system; S2: Collect corresponding point pairs in space, place the L-shaped calibration block on the worktable in various poses, obtain the visual observation coordinates of the feature corner points on the L-shaped calibration block and the actual coordinates of the robot, and construct a dataset for error compensation; S3: Construct a comprehensive error compensation model. Utilize the spatial corresponding point pairs to establish a two-level series comprehensive compensation model. The two-level series comprehensive compensation model includes a global rigid compensation matrix for compensating for overall deviations and a mapping model for compensating for local nonlinear residual errors. S4: Online error compensation. During robot operation, the comprehensive error compensation model is used to compensate for visual guidance commands in real time.

[0023] The following section provides a detailed explanation of each step in the specific implementation process.

[0024] S1: Hand-eye calibration matrix calculation This step establishes the initial transformation relationship between the structured light camera coordinate system and the robot's base coordinate system. The specific execution process is as follows: (1) Fix the structured light camera in the eye-in-hand mode and fix the standard calibration plate on the robot end effector; the standard calibration plate can be a checkerboard calibration plate or a dot array calibration plate, and its size and number of grid points are determined according to the camera field of view.

[0025] (2) Control the robot to move the calibration board to N (N≥3) different poses within the camera's field of view; in this step, N=10 is used to enhance the stability of the calibration results.

[0026] (3) At each pose i, the structured light camera acquires an image of the calibration board and calculates its pose matrix in the camera coordinate system. .

[0027] (4) Synchronously, the robot controller records the pose matrix of the end effector in the robot base coordinate system. .

[0028] (5) Based on the standard equation of hand-eye calibration Construct and solve the transformation matrix from the camera coordinate system to the robot base coordinate system. .in, , , This embodiment uses a classic two-step method combined with singular value decomposition to solve the matrix. Thus, the initial hand-eye transformation relationship is obtained.

[0029] S2: Collect spatial corresponding point pairs This step is used to obtain the visually observed coordinates and the robot's actual coordinates of the feature corner points on the L-shaped calibration block, and to construct a dataset for error compensation. Specifically, it includes the following sub-steps.

[0030] S2.1: L-shaped calibration block design: Design a custom 3D calibration block, such as... Figure 3 As shown in (a), the calibration block has an overall L-shaped three-dimensional geometric shape, with its top and bottom surfaces arranged in an L-shape, and each side perpendicular to the top and bottom surfaces. Furthermore, the surface of the calibration block has a matte finish to avoid specular reflection issues during 3D structured light projection. It should be noted that... Figure 3 (b) The dimensions of the upper surface of the calibration block shown are only one feasible embodiment. The specific dimensions of the L-shaped calibration block can vary depending on the application scenario and are not limited to the dimensions shown. In this embodiment, the L-shaped calibration block is made of metal material and the surface is sandblasted to achieve a matte effect.

[0031] S2.2: Construction of the corresponding point set for error compensation: as follows Figure 2 As shown, within the shared coverage area of ​​the structured light camera's field of view and the robot's workspace, the L-shaped calibration block is placed on a planar worktable in various poses. For each placement pose, visual data acquisition and processing, and robot actual coordinate acquisition are performed sequentially to simultaneously obtain the visual observation coordinates and robot actual coordinates of physical corner points. By traversing all placement poses, a set of spatial corresponding point pairs for error compensation modeling is finally obtained. In this embodiment, the L-shaped calibration block is placed on the worktable in K=10 different poses, and J=6 feature corner points are extracted under each pose, resulting in a total of 60 sets of corresponding point pairs.

[0032] Furthermore, the construction of the error compensation corresponding point set in step S2.2 specifically includes the following steps.

[0033] S2.2.1: Calibration of the worktable reference plane (1) Use a structured light camera to scan the worktable on which the L-shaped calibration block is placed to obtain the initial scene point cloud. .

[0034] (2) The RANSAC algorithm is used to... By performing plane fitting, the equation of the worktable plane is obtained. The distance threshold for RANSAC plane fitting is taken as the average point spacing of the point cloud. of times, of which This is obtained by calculating the mean distance between each point and its nearest neighbor. The value of is determined through experimental testing. In this embodiment... Take 1.5.

[0035] S2.2.2: Calibration Block Point Cloud Segmentation (1) The L-shaped calibration block is placed on the worktable in K different positions and orientations. For each orientation k, the structured light camera acquires scene point clouds. .

[0036] (2) Calculate each point in the point cloud to plane distance .

[0037] (3) Based on the distance threshold δ, the point cloud is divided into calibration blocks and a workbench. The points are determined to belong to the calibration block, thus segmenting the calibration block point cloud. δ is determined as follows: A preliminary scan of the empty workbench is performed, the distance from each point in the resulting point cloud to the fitting plane is calculated, and the standard deviation σ of the distance values ​​is obtained. Then, δ is taken as... ·σ, where The value of is determined through experimental testing. In this embodiment... Take 3.0.

[0038] S2.2.3: Extraction and Projection of the Plane on the Upper Surface of the Calibration Block (1) To The RANSAC algorithm is used again to fit the plane containing the upper surface of the calibration block. And obtain the set of interior points belonging to this plane. .

[0039] (2) Set the interior points Vertical projection onto a plane , thus obtaining a two-dimensional planar projection point cloud .

[0040] S2.2.4: Boundary contour extraction and line fitting (1) Applying the Alpha Shape algorithm to Boundary extraction is performed to obtain the two-dimensional contour point set on the upper surface of the calibration block. The parameter α in the Alpha Shape algorithm is taken as the average point spacing of the point cloud. of times, that is, α= · ,in The value of is determined through experimental testing. In this embodiment... Take 2.0.

[0041] (2) To The RANSAC algorithm was used to perform multiple line fittings, and M (M=6) boundary line segments were extracted from the upper surface of the calibration block. The equation of the straight line and its corresponding set of interior points. The distance threshold for RANSAC straight line fitting is taken as the average point spacing of the point cloud. of times, of which The value of is determined through experimental testing. In this embodiment... Take 1.5.

[0042] S2.2.5: Calculation of Characteristic Corner Points and Coordinate Transformation (1) For each boundary segment The set of interior points of a given line segment is projected onto the line containing that line segment.

[0043] (2) Sort the projection points along the straight line and take the first and last endpoints. and These are the two three-dimensional vertices of the line segment.

[0044] (3) Traverse all line segments and establish the correspondence between the extracted line segments and the physical edge of the calibration block based on the length ratio and topological connection relationship between the line segments.

[0045] (4) For line segments that intersect each other, calculate the J characteristic corner points on the upper surface of the calibration block by solving for the intersection points of the lines they belong to. ,in Let be the three-dimensional coordinates of the j-th corner point in the camera coordinate system at pose k. Figure 4 This is a schematic diagram of the reconstructed calibration block point cloud and the extracted feature corner points in this embodiment, demonstrating the effects of point cloud segmentation, boundary extraction, and feature corner point localization.

[0046] (5) Using the aforementioned hand-eye calibration matrix The coordinates of the feature corner points are transformed to the robot's base coordinate system to obtain the visual observation coordinates of the calibration block corner points in the robot's base coordinate system. .

[0047] S2.2.6: Robot actual coordinate acquisition (1) Control the robot so that the center point of its end tool moves and precisely touches the corresponding physical feature corner point on the L-shaped calibration block.

[0048] (2) Record the coordinates of the TCP in the base coordinate system read by the robot controller at the touch point. This coordinate is the actual coordinate of the point in the robot's base coordinate system.

[0049] S2.2.7: Corresponding point set construction By repeatedly changing the pose of the calibration block (k=1, 2, ..., K) and repeating steps S2.2.2 to S2.2.6, a set of spatially corresponding point pairs is finally obtained. ,in The coordinates of the corner points obtained through visual observation. These are the actual corner coordinates fed back by the robot.

[0050] S3: Construction of the Comprehensive Error Compensation Model This step, based on the spatially corresponding point set, employs a two-level serial compensation strategy to sequentially construct a global rigid compensation matrix for compensating for overall deviations and a mapping model for compensating for local nonlinear residual errors, thereby achieving high-precision compensation for hand-eye calibration errors. Specifically, it includes the following sub-steps.

[0051] S3.1: Calculation of the global rigidity error compensation matrix The spatially corresponding point set reflects a systematic rigid deviation between the visual observation coordinate system and the robot's actual execution coordinate system. This deviation mainly stems from hand-eye calibration errors, manifesting as a fixed rotation and translation relationship. To compensate for this overall deviation, a global rigid transformation matrix needs to be calculated, specifically by performing the following steps: (1) Define the spatial corresponding point set obtained in step S2 as two input point sets for the calculation of the rigid error compensation matrix: Visual observation point set: ; Robot test point set: ; in, This represents the total number of corresponding point pairs.

[0052] (2) An absolute orientation algorithm based on singular value decomposition (SVD) is used to solve the point set. Point set The optimal rigidity transformation is used to obtain the global rigidity error compensation matrix. This matrix is ​​a 4×4 homogeneous transformation matrix that contains the optimal rotation matrix. Translation vector .

[0053] (3) Calculate the visual observation point set separately. Robot measured point set The centroid is determined; this step is used to eliminate the effect of translation so that the rotation matrix can be solved separately. (4) Subtract the centroid of each point in the two point sets to obtain a decentralized point set, which is used to prepare for calculating the covariance matrix: (5) Using the decentralized point set, calculate the covariance matrix between the two point sets. : (6) The covariance matrix obtained in step (5) Perform Singular Value Decomposition (SVD): ,in, and It is a unitary matrix. It is a singular value matrix.

[0054] (7) Based on the results of SVD decomposition, calculate the value from the point set. Point set Optimal rotation matrix With translation vector : (8) The optimal rotation matrix obtained by solving Translation vector The global rigid error compensation matrix is ​​formed by combining the homogeneous transformation matrix. : S3.2: Residual Error Calculation and Nonlinear Mapping Model Establishment Through the global rigidity compensation matrix Even after eliminating the overall bias, local nonlinear residuals still exist between the vision system and the robot system. These residuals are primarily caused by inherent mechanical and control errors in the robot itself. These errors exhibit significant "spatial position correlation," meaning their magnitude and direction change continuously and smoothly with the robot's position within the workspace, and they show different patterns in different regions. These residuals cannot be compensated for by a single rigid transformation. To accurately compensate for these residuals, a high-precision spatial error mapping model is established, and the following steps are performed: (1) Using the global rigidity error compensation matrix calculated in step S3.1 For each visual observation coordinate After initial correction, its new coordinates after rigid alignment are obtained: This step aims to eliminate global biases, laying the foundation for subsequent modeling that focuses on local nonlinear errors.

[0055] (2) For each data point, calculate the coordinates after rigid compensation. Corresponding robot actual coordinates The difference between the two points yields the residual error vector at that point: The residual error vector The remaining positioning deviation after rigid compensation was quantified.

[0056] (3) Combine all the rigidly compensated coordinates obtained from step (1) As input features, all residual error vectors obtained from step (2) As output targets, they together constitute the training dataset used to train the spatial error field mapping model. .

[0057] (4) Based on the training dataset constructed in step (3), establish a nonlinear mapping function. This function can predict the residual error vector corresponding to any input position in space, that is: .

[0058] (5) As a preferred embodiment, the nonlinear mapping function A radial basis function (RBF) network model is used for implementation. Its prediction of spatial local residual errors is achieved through the following mechanism: the RBF network establishes three sub-networks with identical structures but independent weights in the X, Y, and Z directions respectively. The number of hidden layer nodes M is taken as the number of training samples N, and each training sample point is used as the center of the Gaussian kernel function, i.e., let... The Gaussian kernel function is defined as follows: Width parameter The nearest neighbor heuristic is used to determine the outcome, i.e., take... The output layer uses a linearly weighted summation. Taking the X-direction sub-network as an example: Weighting coefficient The least squares method is used to minimize the sum of squares between the predicted value and the actual residual error. The Y and Z direction sub-networks are trained independently in the same way. After the three sub-networks are trained, their outputs are combined to obtain the complete predicted residual error vector. .

[0059] Finally, the complete system integrated error compensation model is composed of the global rigid compensation matrix. Mapping function with the nonlinear residual error The common definition provides core algorithmic support for subsequent online high-precision compensation.

[0060] S4: Online Error Compensation This step improves the robot's absolute operational accuracy by executing the two-level cascaded compensation model to perform real-time, high-precision correction on the target point given by the robot's hand-eye system. Specifically, it includes the following sub-steps.

[0061] S4.1: For a target to be operated on, a 3D structured light camera is used to scan it. Based on the target's features, a corresponding point cloud processing algorithm is designed to calculate the three-dimensional coordinates of the target's feature points in the camera coordinate system. Subsequently, the hand-eye calibration matrix calibrated in step S1 is used. The target point coordinates are then transformed to the robot's base coordinate system to obtain the original visual guidance coordinates without any compensation. This coordinate This refers to the initial target instruction that includes system errors.

[0062] S4.2: Convert the original visual guidance coordinates obtained in step S4.1 Input the global rigidity error compensation matrix calculated in step S3.1. After initial overall deviation correction, the coordinates after rigid compensation are obtained: This step aims to eliminate the fixed rotational and translational deviations between the visual observation coordinate system and the robot's actual execution coordinate system.

[0063] S4.3: The rigidly compensated coordinates obtained in step S4.2 Input into the nonlinear residual error mapping model trained in step S3.2 In the process, the local nonlinear residual error vector that still exists at this specific spatial location is predicted: S4.4: The residual error vector predicted in step S4.3 The coordinates obtained after rigid compensation in step S4.2 The coordinates are superimposed to calculate the final, fully calibrated target coordinates sent to the robot controller: S4.5: The robot control system receives the final target coordinates. The robot performs path planning and drives the movement of each joint to move the end effector precisely to the compensated target position.

[0064] Through the above-mentioned series compensation process, the global systematic rigidity deviation is corrected, and the local nonlinear error that varies with spatial position is precisely compensated. This minimizes the error between the actual arrival position of the robot end and the real physical position of the target point, and ultimately realizes high-precision absolute positioning operation of the robot based on vision guidance.

[0065] Finally, it should be noted that any parts of this invention not described in detail are prior art. Those skilled in the art will understand that the above descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A method for compensating hand-eye calibration errors in a structured light camera based on an L-shaped calibration block, suitable for eye-to-hand configuration, characterized in that... Includes the following steps: S1: Calculate the hand-eye calibration matrix and establish the initial transformation relationship between the structured light camera coordinate system and the robot base coordinate system; S2: Collect corresponding point pairs in space, place the L-shaped calibration block on the worktable in various poses, obtain the visual observation coordinates of the feature corner points on the L-shaped calibration block and the actual coordinates of the robot, and construct a dataset for error compensation; S3: Construct a comprehensive error compensation model. Utilize the spatial corresponding point pairs to establish a two-level series comprehensive compensation model. The two-level series comprehensive compensation model includes a global rigid compensation matrix for compensating for overall deviations and a mapping model for compensating for local nonlinear residual errors. S4: Online error compensation: During robot operation, the comprehensive error compensation model is used to compensate for visual guidance commands in real time. Step S3, which involves constructing the comprehensive error compensation model, specifically includes: S3.1: Define the spatial corresponding point pairs obtained in step S2 as the visual observation point set and the robot measured point set. Use the absolute orientation algorithm based on singular value decomposition to solve the optimal rigid transformation from the visual observation point set to the robot measured point set, and obtain the global rigidity error compensation matrix. S3.2: Use the global rigid error compensation matrix to perform preliminary correction on each visual observation coordinate to obtain the rigidly compensated coordinates; for each data point, calculate the difference between the rigidly compensated coordinates and the corresponding robot actual coordinates to obtain the residual error vector of that point; use all the obtained rigidly compensated coordinates as input features and the residual error vector as output targets to construct a training dataset; establish a nonlinear mapping function based on this dataset to predict the residual error vector at any position in space.

2. The method for compensating hand-eye calibration error of a structured light camera based on an L-shaped calibration block according to claim 1, characterized in that, Step S1 specifically includes: fixing the structured light camera in an eye-in-hand mode, and fixing the standard calibration plate to the robot's end effector; controlling the robot to move the calibration plate to N different poses within the camera's field of view, where N≥3; at each pose, acquiring an image of the calibration plate and calculating its pose matrix in the camera coordinate system, while simultaneously recording the pose matrix of the end effector in the robot's base coordinate system; and based on the hand-eye calibration equation... Solve for the transformation matrix from the camera coordinate system to the robot base coordinate system.

3. The method for compensating hand-eye calibration error of a structured light camera based on an L-shaped calibration block according to claim 1, characterized in that, Step S2, which involves collecting spatially corresponding point pairs, specifically includes: S2.1: Design an L-shaped three-dimensional calibration block. Its overall appearance is an L-shaped geometry, with each side perpendicular to the top and bottom surfaces, and the surface is matte. S2.2: Within the shared coverage area of ​​the structured light camera's field of view and the robot's workspace, L-shaped calibration blocks are placed on the planar worktable in various poses. For each pose, point clouds are acquired through the structured light camera and processed to obtain the three-dimensional coordinates of the feature corner points in the camera coordinate system. After transformation by the hand-eye calibration matrix, the visual observation coordinates in the robot's base coordinate system are obtained. Simultaneously, the center point of the robot's end effector is controlled to precisely touch the corresponding physical feature corner point, and the actual coordinates in the robot's base coordinate system are recorded. All poses are traversed to obtain a set of corresponding point pairs in space.

4. The method for compensating hand-eye calibration error of a structured light camera based on an L-shaped calibration block according to claim 3, characterized in that, Step S2.2, which involves acquiring point clouds using a structured light camera and processing them to obtain the 3D coordinates of feature corner points in the camera coordinate system, specifically includes the following steps: S2.2.1: Use a structured light camera to scan the worktable where the L-shaped calibration block is placed to obtain the initial scene point cloud. The equation of the worktable plane was obtained by fitting the RANSAC algorithm. S2.2.2: Place the L-shaped calibration block on the worktable in K different poses. For each pose, collect scene point clouds, calculate the distance from each point in the point cloud to the worktable plane, and determine the points whose distance is greater than a preset threshold as calibration block point clouds to achieve calibration block point cloud segmentation. S2.2.3: The RANSAC algorithm is used again to fit the upper surface plane of the calibration block point cloud, the set of interior points belonging to the plane is obtained and projected vertically onto the plane to obtain the two-dimensional projected point cloud; S2.2.4: The Alpha Shape algorithm is used to extract the boundary contour of the two-dimensional projected point cloud to obtain the two-dimensional contour point set. Then, the RANSAC algorithm is used to fit the straight line multiple times to extract multiple boundary line segments on the upper surface of the calibration block. S2.2.5: Project the interior points of each boundary line segment onto the corresponding straight line and sort them. Take the first and last endpoints as three-dimensional vertices. Based on the length ratio and topological connection relationship of each boundary line segment, determine the correspondence between the extracted line segments and the physical edge lines of the calibration block. By solving the intersection points of the intersecting line segments, calculate the three-dimensional coordinates of multiple feature corner points on the upper surface of the calibration block in the camera coordinate system.

5. The method for compensating hand-eye calibration error of a structured light camera based on an L-shaped calibration block according to claim 1, characterized in that, The nonlinear mapping function is implemented using a radial basis function network model, which decouples the prediction problem of the three-dimensional residual error vector into three independent scalar regression problems. The nonlinear mapping function consists of three radial basis function sub-networks with the same structure but independent weights, which are used to predict the residual error components in the X, Y and Z directions, respectively.

6. The method for compensating hand-eye calibration error of a structured light camera based on an L-shaped calibration block according to claim 5, characterized in that, In the radial basis function network model, the number of hidden layer nodes is taken as the number of training samples, and each training sample point is used as the center of the Gaussian kernel function, which is defined as follows: in, Input coordinates, For the first The center of a Gaussian kernel function, This is the width parameter.

7. The method for compensating hand-eye calibration error of a structured light camera based on an L-shaped calibration block according to claim 6, characterized in that, The width parameter The nearest neighbor heuristic is used to determine this, i.e., take... The output layers of each subnetwork are linearly weighted sums. Taking the X-direction subnetwork as an example, its output is: in, This represents the number of hidden layer nodes. The weighting coefficients are obtained using the least squares method.

8. The method for compensating hand-eye calibration error of a structured light camera based on an L-shaped calibration block according to claim 1, characterized in that, The global rigid error compensation matrix The transformation matrix is ​​a 4×4 homogeneous transformation matrix containing the rotation matrix R and the translation vector t, and its expression is: This matrix is ​​used to perform a rigid transformation correction on the original visual guide coordinates.

9. The method for compensating hand-eye calibration error of a structured light camera based on an L-shaped calibration block according to claim 1, characterized in that, The online error compensation in step S4 specifically includes: S4.1: Scan the target using a structured light camera, calculate the three-dimensional coordinates of the target feature points in the camera coordinate system, and use the hand-eye calibration matrix from step S1 to transform them to the robot base coordinate system to obtain the original visual guidance coordinates without any compensation. S4.2: Input the original visual guidance coordinates into the global rigid error compensation matrix to obtain the coordinates after rigid compensation; S4.3: Input the rigidly compensated coordinates into the nonlinear mapping model to predict the local nonlinear residual error vector; S4.4: Superimpose the residual error vector with the rigidly compensated coordinates to obtain the final corrected target coordinates, and send them to the robot controller for execution.

Images