Method for self-calibration of mechanical arm drop precision based on visual feedback

Abstract

The present application relates to the technical field of image detection, and particularly relates to a mechanical arm landing precision self-calibration method based on visual feedback, comprising the following steps: converting a target coordinate to a mechanical arm base coordinate system according to a preset mechanical arm kinematic model, calculating corresponding first motion parameters when a mechanical arm end moves above a target landing area in combination with a preset observation height, obtaining a local image of a chess piece and a target landing point, obtaining an image space deviation amount, calculating second motion parameters, obtaining an actual position of the chess piece after landing, and updating the mechanical arm kinematic model parameters. The present application combines global positioning and local correction, significantly improves landing precision, and a closed-loop feedback mechanism can automatically correct model parameters according to landing results, so that self-calibration can effectively compensate for mechanical wear errors caused by long-term operation of the mechanical arm, and long-term landing precision is maintained.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, and more specifically to a self-calibration method for the precision of a robotic arm's placement of a piece based on visual feedback. Background Technology

[0002] With the development of intelligent manufacturing, robotic automation, and human-machine collaboration technologies, robotic arms are increasingly being applied in intelligent chess-playing equipment, automated operation platforms, and precision assembly. In chess-playing robot applications, the robotic arm needs to automatically grasp and precisely place pieces based on the analysis of the game situation. The accuracy of the placement directly affects the game's execution and system stability. Therefore, improving the positioning accuracy of the robotic arm's end effector and achieving error detection and automatic calibration during the piece-placing process has become a crucial technological direction for the development of intelligent chess-playing robots.

[0003] Existing robotic arm move-piece control methods typically rely on pre-calibrated kinematic models of the robotic arm, calculating the end effector trajectory through coordinate transformation between the chessboard coordinate system and the robotic arm's base coordinate system. However, in actual operation, factors such as joint clearances, transmission errors, end effector installation deviations, environmental temperature variations, and changes in the chessboard's installation position can easily lead to discrepancies between the pre-established kinematic model and the actual motion state. This results in a certain error between the actual placement point and the target position of the piece when the robotic arm moves according to the theoretical trajectory. Especially in applications where the chessboard grid size is small or continuous high-precision moves are required, accumulated errors can cause move-piece deviations, reducing the reliability of the move-piece placement. Summary of the Invention

[0004] (a) Purpose of the invention The purpose of this invention is to provide a self-calibration method for the placement accuracy of a robotic arm based on visual feedback. By combining global positioning with local correction, the placement accuracy is significantly improved. Its closed-loop feedback mechanism can automatically correct model parameters based on the placement results, effectively compensating for system errors caused by mechanical wear and environmental interference. The kinematic model parameters of the robotic arm are automatically corrected based on the actual physical position differences after each placement. Self-calibration can effectively compensate for errors generated by the robotic arm over long-term operation, maintaining placement accuracy over the long term.

[0005] (II) Technical Solution To address the above problems, this invention provides a self-calibration method for the precision of a robotic arm's placement based on visual feedback, comprising: Based on the global image of the chessboard, the chessboard grid points are identified and located, a chessboard coordinate system is established, and the target coordinates of the target placement point in the chessboard coordinate system are determined. Based on the preset kinematic model of the robotic arm, the target coordinates are transformed to the base coordinate system of the robotic arm, and combined with the preset observation height, the first motion parameters corresponding to the end effector of the robotic arm when it moves above the target dropping area are calculated. Based on the first motion parameters, the robotic arm is controlled to carry the chess piece to a preset observation height above the target placement point, and a local image of the chess piece and the target placement point is obtained. Based on the local image, the positional deviation between the center of the chess piece and the center of the target placement point is extracted to obtain the image spatial deviation. The second motion parameter is calculated based on the preset robotic arm kinematics model, the image spatial deviation, and the preset observation height. Based on the second motion parameters, the placing action is executed, and the actual position of the piece after the placement is completed is obtained; The kinematic model parameters of the robotic arm are updated based on the difference between the actual position of the chess piece and the target placement point.

[0006] In another aspect of the present invention, preferably, the step of identifying and locating chessboard grid points based on a global chessboard image, establishing a chessboard coordinate system, and determining the target coordinates of the target placement point in the chessboard coordinate system includes: The chessboard boundary of the global image of the chessboard is extracted by a preset edge detection algorithm, and the horizontal and vertical grid lines of the chessboard are identified by a preset Hough line detection algorithm. Based on the intersection of the horizontal and vertical grid lines, the coordinates of each grid point on the chessboard are extracted, and a chessboard coordinate system is established according to the topological relationship of the chessboard rows and columns. A mapping relationship between the image coordinate system and the chessboard coordinate system is established by perspective transformation, and the coordinates of the chessboard grid points are corrected. Based on the target move instruction, determine the target chessboard grid point corresponding to the target move point; Based on the target chessboard grid points, determine the target coordinates of the target placement point in the chessboard coordinate system.

[0007] In another aspect of the present invention, preferably, the step of transforming the target coordinates to the robot arm base coordinate system according to a preset robot arm kinematic model, and calculating the first motion parameters corresponding to the robot arm end effector moving above the target dropping area in combination with a preset observation height, includes: Based on the calibration relationship between the chessboard coordinate system and the robotic arm base coordinate system, a coordinate transformation matrix is ​​established to convert the target coordinates of the target placement point in the chessboard coordinate system into the target position coordinates in the robotic arm base coordinate system. Based on the target position coordinates and the preset observation height, the end-effector target pose is constructed, wherein the end-effector target pose includes end-effector position coordinates and end-effector attitude angles; The target pose of the robotic arm end is input into a preset robotic arm kinematics model. By solving the motion relationship between the joints of the robotic arm, the target joint angles corresponding to each joint of the robotic arm are obtained. Based on the target joint angle and the current joint state of the robotic arm, the first motion parameters of each joint are calculated. The first motion parameters include the joint motion direction, the joint rotation angle, and the motion speed.

[0008] In another aspect of the present invention, preferably, the step of controlling the robotic arm to move the chess piece to a preset observation height above the target placement point according to the first motion parameter, and acquiring a local image of the chess piece and the target placement point, includes: Based on the first motion parameters, a motion control command for the robotic arm is generated, and the motion control command is sent to the robotic arm control module. Drive each joint of the robotic arm to move in the corresponding direction of motion, rotation angle and speed, so that the end of the robotic arm carrying the chess piece moves to the preset observation height above the target placement point; The vision acquisition device is controlled to acquire local images containing chess pieces and target placement points.

[0009] In another aspect of the present invention, preferably, the step of extracting the positional deviation between the center of the chess piece and the center of the target placement point based on the local image to obtain the image spatial deviation includes: Based on a preset image recognition algorithm, the chess piece region and the target placement point region in the local image are detected, and the first position coordinates and the second position coordinates are determined respectively. The first position coordinates are the position coordinates of the chess piece outline center in the image coordinate system, and the second position coordinates are the position coordinates of the target placement point center in the image coordinate system. Based on the first and second position coordinates, the horizontal deviation between the chess piece and the target placement point is determined, and the horizontal deviation is the image space deviation.

[0010] In another aspect of the present invention, preferably, the calculation of the second motion parameter based on the preset robotic arm kinematic model, the image spatial deviation, and the preset observation height includes: The preset observation height is used to determine the vertical movement between the chess piece and the target placement point; The horizontal deviation and vertical motion are fused together to obtain the three-dimensional displacement of the robotic arm end effector. The three-dimensional displacement is input into the preset kinematic model of the robotic arm to obtain the second motion parameters.

[0011] In another aspect of the present invention, preferably, the step of fusing the horizontal deviation and the vertical motion to obtain the three-dimensional displacement of the robotic arm end effector includes: Based on the lateral and longitudinal offsets corresponding to the image spatial deviation, the first and second direction correction distances of the robotic arm end effector in the chessboard plane coordinate system are determined. Based on the height difference between the preset observation height and the current height of the robotic arm end effector, the third-direction movement distance of the robotic arm end effector along the vertical direction is determined; The first direction correction distance, the second direction correction distance, and the third direction motion distance are combined to generate the displacement of the robotic arm end effector in three-dimensional space.

[0012] In another aspect of the present invention, preferably, the step of performing the placing action based on the second motion parameter and obtaining the actual position of the piece after the placement is completed includes: Based on the second motion parameters, a motion control command for the robotic arm end effector is generated, and the robotic arm end effector is controlled to move to the target placement position; The robotic arm is controlled to release the chess piece. A visual acquisition device is used to acquire an image of the chessboard after the piece is placed. The positions of the chess pieces in the chessboard image are identified to determine the actual positions of the pieces after the piece is placed.

[0013] In another aspect of the present invention, preferably, updating the kinematic model parameters of the robotic arm based on the image spatial deviation and the positional difference between the actual position of the chess piece and the target placement point includes: Based on the image spatial deviation, a first error amount is obtained; The second error is obtained based on the difference between the actual position of the piece and the target placement point. Map the first error quantity and the second error quantity to a unified coordinate system, and calculate the vector correlation, wherein the vector correlation includes the magnitude deviation and the direction angle; The parameters of the robotic arm kinematic model are updated based on the vector correlation.

[0014] In another aspect of the present invention, preferably, updating the kinematic model parameters of the robotic arm based on the vector correlation includes: If the magnitude deviation and the direction angle are both within a preset consistency threshold, the error source is determined to be a systematic calibration deviation. The average value of the first error and the second error is used as a correction vector to update the kinematic model parameters of the robotic arm. If the magnitude deviation or directional angle exceeds a preset consistency threshold, the nonlinear execution deviation of the robotic arm on the dropping and pressing path is extracted by the vector difference between the first error and the second error. Based on the nonlinear execution deviation, the backlash error of each joint of the robotic arm or the flexible deformation of the link is identified, and the kinematic model parameters of the robotic arm are updated.

[0015] (III) Beneficial Effects The above-described technical solution of the present invention has the following beneficial technical effects: This invention employs a two-stage visual feedback mechanism of global coarse positioning and local fine correction, effectively eliminating cumulative errors and end effector jitter in long-distance robotic arm movements, ensuring sub-millimeter-level placement accuracy. Fine-tuning using local images further enhances placement accuracy. Dynamic updates to the kinematic model parameters based on measured position feedback after placement enable real-time compensation for mechanical deviations. The calibration process is integrated into the workflow, achieving online automatic calibration without the need for tedious offline maintenance by professional personnel, significantly improving the equipment's intelligence and practical value. Attached Figure Description

[0016] Figure 1 This is an overall flowchart of one embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0018] Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0019] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0020] Example 1 A self-calibration method for the placement accuracy of a robotic arm based on visual feedback. Figure 1 An overall flowchart of one embodiment of the present invention is shown, as follows: Figure 1 As shown, it includes: Before the robotic arm performs the move, an image containing the entire chessboard area is acquired using a vision acquisition device fixed above or to the side of the robotic arm's working area. Image preprocessing is performed, including grayscale conversion, noise filtering, edge enhancement, perspective distortion correction, and illumination equalization, to reduce the impact of ambient light variations, chessboard tilt, and camera angle changes on subsequent recognition processes, thus obtaining a global chessboard image. Based on this global image, chessboard grid points are identified and located, a chessboard coordinate system is established, and the target coordinates of the target move point within the chessboard coordinate system are determined, including: A preset edge detection algorithm is used to extract the chessboard boundary of the global chessboard image. This algorithm is then used to extract edge features from the preprocessed global chessboard image to obtain edge response information within the chessboard area. Specifically, the edge detection algorithm can be a gradient-based algorithm, which determines the chessboard boundary and candidate edge regions corresponding to the internal grid lines by calculating the degree of change in image pixel grayscale values ​​in the horizontal and vertical directions. After obtaining the edge image, based on the regular rectangular distribution of the chessboard, longer, continuous edge segments are selected as candidate chessboard boundary lines. A preset Hough line detection algorithm is used to identify the horizontal and vertical grid lines of the chessboard. The preset Hough line detection algorithm extracts line features from the edge image, transforming edge pixels in image space to parameter space, and determining the position of lines in the chessboard through cumulative voting. Since the horizontal and vertical grid lines in the chessboard have directional regularity, the candidate lines are classified by direction based on the detected line parameters. Lines with an angle within a preset range to the horizontal direction are designated as horizontal grid lines, and lines with an angle within a preset range to the vertical direction are designated as vertical grid lines. The positional parameters of all horizontal and vertical grid lines on the chessboard are obtained using the method described above.

[0021] Based on the intersections of horizontal and vertical grid lines, the coordinates of each chessboard grid point are extracted, and a chessboard coordinate system is established according to the chessboard's row and column topology. After obtaining the sets of horizontal and vertical grid lines, the intersection of each horizontal and vertical grid line is calculated to obtain the corresponding intersection point position. This intersection point represents the two-dimensional coordinates of the chessboard grid point in the image coordinate system. Due to potential errors in line detection, local occlusion, or noise interference during actual detection, some intersection point positions may deviate. Therefore, the obtained initial grid point coordinates are optimized and corrected. Specifically, outliers are removed based on the consistency constraint of distance between adjacent grid points, and surrounding valid grid points are used for fitting and correction to ensure that the obtained chessboard grid point coordinates satisfy the characteristics of a regular chessboard structure. After grid point extraction, each grid point is numbered according to the chessboard's row and column arrangement. Specifically, the top-left corner grid point is used as the origin, and grid point indices are established in both the horizontal and vertical directions. For example, the horizontal direction is defined as the X-axis of the chessboard coordinate system, and the vertical direction is defined as the Y-axis of the chessboard coordinate system. The chessboard coordinate system is established based on the row and column numbering relationship of the grid points in the chessboard.

[0022] A perspective transformation is used to establish a mapping relationship between the image coordinate system and the chessboard coordinate system, and coordinate correction is performed on the chessboard grid point coordinates. Since visual acquisition devices typically cannot guarantee a direct view of the chessboard when shooting, the acquired global image of the chessboard may suffer from perspective distortion, causing the actually evenly spaced grid points to appear non-uniformly distributed in the image. To eliminate this effect, a perspective mapping relationship between the image coordinate system and the chessboard coordinate system is established based on the detected chessboard boundary corner points. Furthermore, a perspective transformation matrix is ​​calculated based on the four corner points corresponding to the chessboard boundary region. Using this perspective transformation matrix, the chessboard grid point coordinates in the image coordinate system are transformed to the standard chessboard plane coordinate system, achieving geometric correction of the chessboard grid point positions. The perspective transformation relationship can be expressed as: mapping the coordinates of any point in the image coordinate system to its corresponding position in the chessboard coordinate system, so that the corrected chessboard grid points satisfy an evenly spaced arrangement.

[0023] Based on the target move instruction, the target chessboard grid corresponding to the target move point is determined. When the robotic arm receives the move instruction, it determines the area to be moved based on the move position parameters contained in the target move instruction. The target move instruction can be generated by user input or automatically generated by the game analysis module based on the current game state. Based on the target chessboard grid, the target coordinates of the target move point in the chessboard coordinate system are determined.

[0024] Based on a preset robotic arm kinematics model, the target coordinates are transformed to the robotic arm base coordinate system. Combined with a preset observation height, the first motion parameters corresponding to the robotic arm end effector moving above the target placement area are calculated. A spatial transformation relationship between the robotic arm base coordinate system and the chessboard coordinate system is pre-established. This spatial transformation relationship includes a rotation matrix and a translation vector, used to describe the positional relationship between the actual installation position of the chessboard and the robotic arm's workspace. Based on this spatial transformation relationship, the target coordinates in the chessboard coordinate system are transformed to the robotic arm base coordinate system to obtain the three-dimensional spatial coordinates of the target placement point in the robotic arm's workspace at the preset observation height. In this embodiment, the following is included: Based on the calibration relationship between the chessboard coordinate system and the robotic arm base coordinate system, a coordinate transformation matrix is ​​established to convert the target coordinates of the target placement point in the chessboard coordinate system to the target position coordinates in the robotic arm base coordinate system. Before the robotic arm performs the placement task, it is necessary to pre-establish the spatial correspondence between the chessboard coordinate system and the robotic arm base coordinate system to achieve data conversion between visual detection results and robotic arm motion control. Since the chessboard coordinate system is usually established based on visual recognition results, while the robotic arm base coordinate system is a fixed reference coordinate system defined by the robotic arm control system itself, the two differ in origin position, coordinate axis direction, and spatial scale. Therefore, a calibration process is needed to obtain the coordinate transformation relationship between the two. In the specific calibration process, multiple calibration points with known positions can be set in the chessboard area. The coordinates of each calibration point in the chessboard coordinate system are obtained through a visual acquisition device. At the same time, the robotic arm end effector moves to the corresponding position to obtain the actual coordinates of each calibration point in the robotic arm base coordinate system. Based on multiple sets of corresponding point data, the rotation matrix and translation vector between the chessboard coordinate system and the robotic arm base coordinate system are calculated using a rigid body transformation solution method, thereby establishing the coordinate transformation matrix. The coordinate transformation matrix describes the positional and orientational relationship between the chessboard coordinate system and the robot arm's base coordinate system. It includes rotation parameters describing the rotation of the coordinate axes and translation parameters describing the offset of the coordinate origin. Through this coordinate transformation matrix, the two-dimensional target coordinates in the chessboard coordinate system can be converted into three-dimensional spatial coordinates in the robot arm's base coordinate system.

[0025] Based on the target position coordinates and the preset observation height, the target pose of the robotic arm's end effector is constructed. This target pose includes the end effector's position coordinates and attitude angles. The observation height controls the robotic arm's end effector to move above the target placement area before the actual placement, enabling the visual acquisition device to clearly obtain the positional relationship between the piece and the target placement area. The target position coordinates include the robotic arm's end effector's position along the X-axis, Y-axis, and Z-axis in the base coordinate system. Simultaneously, the robotic arm's end effector attitude angle is determined based on the piece's gripping direction, the chessboard's plane orientation, and the structural parameters of the end effector. This attitude angle describes the spatial rotation state of the end effector relative to the robotic arm's base coordinate system, including rotation angles around the X-axis, Y-axis, and Z-axis. For example, when the robotic arm moves the piece above the chessboard, the end effector can be set to maintain a vertically downward attitude, ensuring the piece's release direction aligns with the normal direction of the chessboard surface, thereby reducing problems such as piece tilting, slippage, or placement deviation caused by attitude deviations. By combining the end-effector position coordinates and end-effector attitude angles, a complete end-effector target pose is formed, which serves as the input parameters for solving the kinematics of the robotic arm.

[0026] The target pose of the robotic arm's end effector is input into a preset robotic arm kinematics model. By solving the kinematic relationships between the joints of the robotic arm, the target joint angles corresponding to each joint are obtained. The robotic arm kinematics model describes the correspondence between the joint variables of the robotic arm and the spatial pose of the end effector, including the length parameters of the robotic arm links, joint connection methods, joint rotation directions, end effector installation offset parameters, and joint range of motion limitation parameters. It can be a DH-based robotic arm kinematics model. In this embodiment, the target joint angles corresponding to each joint are calculated using inverse kinematics based on the target pose of the robotic arm's end effector. Specifically, the end effector target position coordinates and attitude angles are used as constraints. By establishing a functional relationship between the robotic arm's end effector pose and the joint angles, a set of joint angles that meet the target pose requirements is solved. Further, during the inverse kinematics solution process, multiple feasible solutions can be screened by combining the current posture state of the robotic arm and the joint range of motion constraints. The joint angle combination with the shortest motion path, the smallest joint change, or the best obstacle avoidance performance is selected as the target joint angle. For example, for a robotic arm with multiple rotary joints, the same end position may correspond to multiple different joint angle combinations. By introducing the current joint state as a reference, the robotic arm can prioritize the solution with the smallest change in the current posture, thereby reducing vibration and positioning errors during the robotic arm's movement.

[0027] Based on the target joint angle and the current joint state of the robotic arm, the first motion parameters of each joint are calculated. These parameters include the joint motion direction, joint rotation angle, and motion speed. After obtaining the target joint angles for each joint of the robotic arm, the real-time angle state of each joint is acquired. By comparing the difference between the target joint angle and the current joint angle, the required motion direction and rotation angle for each joint are determined. Specifically, when the target joint angle is greater than the current joint angle, the corresponding joint is determined to rotate in the positive direction; when the target joint angle is less than the current joint angle, the corresponding joint is determined to rotate in the negative direction. The required rotation angle for each joint is determined based on the absolute value of the difference between the target angle and the current angle. Further, according to the robotic arm motion control strategy and preset motion constraints, a corresponding motion speed is assigned to each joint. The motion speed can be dynamically adjusted based on the distance of the joint from the target angle, the robotic arm load state, and motion stability requirements. For example, for joints far from the target position, a higher speed can be used for coarse positioning motion; as the joint gradually approaches the target angle, the motion speed is reduced to improve end-effector positioning accuracy and avoid position overshoot due to inertia. Finally, the first motion parameters of the robotic arm are generated based on the motion direction, rotation angle, and motion speed of each joint.

[0028] Based on the first motion parameters, the robotic arm is controlled to move with the chess piece to a preset observation height above the target placement point, and a local image of the chess piece and the target placement point is acquired. This includes: generating a robotic arm motion control command based on the first motion parameters, sending the motion control command to the robotic arm control module; driving each joint of the robotic arm to move according to the corresponding motion direction, rotation angle, and motion speed, so that the end of the robotic arm carrying the chess piece moves to a preset observation height above the target placement point, and controlling the vision acquisition device to acquire a local image containing the chess piece and the target placement point.

[0029] Based on the local image, the positional deviation between the center of the chess piece and the center of the target placement point is extracted to obtain the image spatial deviation. This includes: preprocessing the local image; detecting the chess piece region and the target placement point region in the local image based on a preset image recognition algorithm; and determining the first position coordinate and the second position coordinate, where the first position coordinate is the position coordinate of the chess piece outline center in the image coordinate system, and the second position coordinate is the position coordinate of the target placement point center in the image coordinate system. Specifically, the chess piece center position can be determined by calculating the centroid of the outline. First, multiple pixels on the chess piece outline boundary are extracted, and the area moment and first moment of the outline region are calculated based on the coordinates of each pixel, thereby obtaining the position of the chess piece outline center. The chess piece outline center is the first position coordinate of the chess piece in the image coordinate system. Alternatively, when using deep learning detection, a chess piece target detection model can be pre-trained, enabling the model to automatically identify the chess piece region based on the input local image and output the chess piece bounding box position. The chess piece center position is determined based on the coordinates of the center point of the bounding box obtained from the detection, improving the robustness of chess piece detection in complex environments. Furthermore, for the detection of the target placement area, the chessboard grid region corresponding to the target placement point can be determined based on the aforementioned established chessboard coordinate system information, and the target placement area can be extracted by combining the chessboard texture features in the local image. Since the target placement point is usually located at the intersection of the chessboard grid, the center of the target placement point, i.e., the second position coordinates, can be determined by chessboard line intersection detection, template matching, or coordinate mapping.

[0030] Based on the first and second position coordinates, the horizontal deviation between the piece and the target placement point is determined, whereby the horizontal deviation is an image spatial deviation. After obtaining the first position coordinates of the piece's center and the second position coordinates of the target placement point's center, the difference between the two coordinates is calculated to obtain the offset of the piece's current position relative to the target placement position. Since the robotic arm has now moved to an observation height above the target placement area, the main error between the piece and the target placement point manifests as a horizontal positional offset within the chessboard plane.

[0031] Based on the preset robotic arm kinematics model, the image spatial deviation, and the preset observation height, the second motion parameters are calculated, including: The preset observation height is used to determine the vertical motion between the chess piece and the target placement point. After the robotic arm completes its initial positioning, the end effector of the robotic arm, carrying the chess piece, is positioned at the preset observation height above the target placement area. Since the spatial deviation of the image acquired by the visual acquisition device mainly reflects the two-dimensional offset of the chess piece in the chessboard plane, and the actual placement process also requires controlling the end effector of the robotic arm to descend vertically to the placement height, it is necessary to determine the vertical motion of the end effector of the robotic arm in conjunction with the preset observation height. The observation height is the height difference between the current position of the end effector of the robotic arm and the chessboard placement plane.

[0032] The horizontal deviation and vertical motion are fused to obtain the three-dimensional displacement of the robotic arm's end effector. This includes: determining a first-direction correction distance and a second-direction correction distance of the robotic arm's end effector in the chessboard plane coordinate system based on the lateral and longitudinal offsets corresponding to the image space deviation; determining a third-direction motion distance of the robotic arm's end effector along the vertical direction based on the height difference between the preset observation height and the current height of the robotic arm's end effector; and combining the first-direction correction distance, the second-direction correction distance, and the third-direction motion distance to generate the displacement of the robotic arm's end effector in three-dimensional space. The three-dimensional displacement is input into the preset robotic arm kinematics model to obtain second motion parameters. The robotic arm kinematics model is the same as the model described above, and the calculation process is the same as the calculation process for the first motion parameters.

[0033] Based on the second motion parameters, the process of placing a piece and obtaining its actual position after placement includes: generating motion control commands for the robotic arm end effector based on the second motion parameters, and controlling the robotic arm end effector to move to the target placement position; controlling the robotic arm to release the piece; acquiring a chessboard image after placement using a vision acquisition device; identifying the positions of the pieces in the chessboard image; and determining the actual position of the piece after placement. This identification can be performed using a preset image recognition method.

[0034] Based on the image spatial deviation and the difference between the actual position of the chess piece and the target placement point, the kinematic model parameters of the robotic arm are updated, including: Based on the image spatial deviation, a first error is obtained. Before the robotic arm performs the placement action, the positional relationship between the center of the piece and the center of the target placement point is obtained through local visual feedback, and the image spatial deviation is obtained. The first error represents the spatial deviation between the theoretically predicted placement position and the target placement position before the robotic arm makes motion corrections based on visual feedback, and it includes the deviation along the X-axis and Y-axis directions of the robotic arm's base coordinate system.

[0035] The second error is obtained based on the difference between the actual position of the piece and the target placement point; the second error is used to represent the final positional deviation produced after the robotic arm actually performs the placement action.

[0036] The first and second error quantities are mapped to a unified coordinate system. Since the first error quantity originates from the visual feedback process, while the second error quantity originates from the actual placement result, they may be expressed in different coordinate forms. Therefore, the first and second error quantities are first uniformly transformed to the robot arm base coordinate system. Vector correlation is calculated, which includes magnitude deviation and direction angle. The magnitude deviation between the first and second error quantities is calculated based on the Euclidean norm formula, and the direction angle between them is calculated based on the vector cosine similarity formula. Error consistency is judged based on magnitude deviation and direction angle, and systematic calibration error and nonlinear execution error are extracted using the error vector averaging fusion formula or error residual analysis formula, respectively, to update the robot arm kinematic calibration model based on DH parameters.

[0037] Based on the vector correlation, the kinematic model parameters of the robotic arm are updated, including: If both the magnitude deviation and the directional angle are within a preset consistency threshold, the error source is determined to be a systematic calibration deviation. The average of the first error and the second error is used as a correction vector to update the kinematic model parameters of the robotic arm. When the magnitude deviation is small and the directional angle is within a preset range, it indicates that the first error and the second error have high consistency, meaning that the deviation direction obtained by visual detection is basically the same as the deviation direction generated by the actual placement of the piece, and the error magnitudes are close, such as the calibration error of the center point of the robotic arm's end effector. The preset consistency threshold can be set based on experience.

[0038] If the magnitude deviation or directional angle exceeds a preset consistency threshold, it indicates that the current error is not caused by fixed coordinate calibration deviation, but may originate from dynamic execution errors during the movement of the robotic arm. For example, during the process of the robotic arm descending to release a chess piece, due to joint clearance, transmission error, link elastic deformation, and load changes in the mechanical structure, the actual movement trajectory of the robotic arm's end effector deviates from the theoretical movement trajectory. The vector difference between the first error and the second error is obtained to extract the nonlinear execution deviation of the robotic arm on the placement path. Based on the nonlinear execution deviation and its variation with the robotic arm's movement stage, direction, and joint state, the location of the error is analyzed. For example, if the error mainly accumulates along the movement direction of a certain joint, it is determined that the corresponding joint has transmission clearance or backlash error; if the error increases with the extension length of the robotic arm, it is determined that the link has flexible deformation; if the error only occurs during the descent stage, it is determined that there is a dynamic response error on the placement path. Based on the nonlinear execution deviation data collected from multiple placement processes, a correspondence between the error and the robotic arm joint state is established, and the error parameters of each joint are identified. Specifically, for joint backlash error, the angle compensation amount of the corresponding joint can be estimated based on the positional deviation generated during the forward and reverse movements of the robotic arm, and this angle compensation amount is added to the joint zero-position correction parameter in the kinematic model. For link flexibility deformation error, a link deformation compensation model can be established based on the end-effector position deviation under different load states and extension postures of the robotic arm, and the corresponding deformation amount is used as the flexibility compensation parameter in the kinematic model. Furthermore, the identified joint backlash error parameters, link flexibility deformation parameters, and end-effector offset parameters are written into the robotic arm kinematic model, enabling the model to reflect the actual operating state of the robotic arm. Through the above parameter update process, the robotic arm kinematic model can not only compensate for fixed installation errors, but also adapt to mechanical wear, structural changes, and dynamic execution errors generated during long-term operation, achieving online self-calibration based on visual feedback.

[0039] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of the invention and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of the invention should be included within the protection scope of the invention. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

[0040] The present invention has been described above with reference to embodiments thereof. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. The scope of the invention is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

[0041] Although embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions, and modifications can be made to the embodiments of the present invention without departing from the spirit and scope of the invention.

[0042] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A self-calibration method for the placement accuracy of a robotic arm based on visual feedback, characterized in that, include: Based on the global image of the chessboard, the chessboard grid points are identified and located, a chessboard coordinate system is established, and the target coordinates of the target placement point in the chessboard coordinate system are determined. Based on the preset kinematic model of the robotic arm, the target coordinates are transformed to the base coordinate system of the robotic arm, and combined with the preset observation height, the first motion parameters corresponding to the end effector of the robotic arm when it moves above the target dropping area are calculated. Based on the first motion parameters, the robotic arm is controlled to carry the chess piece to a preset observation height above the target placement point, and a local image of the chess piece and the target placement point is obtained. Based on the local image, the positional deviation between the center of the chess piece and the center of the target placement point is extracted to obtain the image spatial deviation. The second motion parameter is calculated based on the preset robotic arm kinematics model, the image spatial deviation, and the preset observation height. Based on the second motion parameters, the placing action is executed, and the actual position of the piece after the placement is completed is obtained; The kinematic model parameters of the robotic arm are updated based on the image spatial deviation and the difference between the actual position of the chess piece and the target placement point.

2. The self-calibration method for robotic arm placement accuracy based on visual feedback according to claim 1, characterized in that, The process of identifying and locating chessboard grid points based on a global chessboard image, establishing a chessboard coordinate system, and determining the target coordinates of the target placement point within the chessboard coordinate system includes: The chessboard boundary of the global image of the chessboard is extracted by a preset edge detection algorithm, and the horizontal and vertical grid lines of the chessboard are identified by a preset Hough line detection algorithm. Based on the intersection of the horizontal and vertical grid lines, the coordinates of each grid point on the chessboard are extracted, and a chessboard coordinate system is established according to the topological relationship of the chessboard rows and columns. A mapping relationship between the image coordinate system and the chessboard coordinate system is established by perspective transformation, and the coordinates of the chessboard grid points are corrected. Based on the target move instruction, determine the target chessboard grid point corresponding to the target move point; Based on the target chessboard grid points, determine the target coordinates of the target placement point in the chessboard coordinate system.

3. The self-calibration method for robot arm placement accuracy based on visual feedback according to claim 2, characterized in that, The step involves transforming the target coordinates to the robot arm base coordinate system based on a preset robot arm kinematic model, and calculating the first motion parameters corresponding to the robot arm end effector moving above the target dropping area, in conjunction with a preset observation height. This includes: Based on the calibration relationship between the chessboard coordinate system and the robotic arm base coordinate system, a coordinate transformation matrix is ​​established to convert the target coordinates of the target placement point in the chessboard coordinate system into the target position coordinates in the robotic arm base coordinate system. Based on the target position coordinates and the preset observation height, the end-effector target pose is constructed, wherein the end-effector target pose includes end-effector position coordinates and end-effector attitude angles; The target pose of the robotic arm end is input into a preset robotic arm kinematics model. By solving the motion relationship between the joints of the robotic arm, the target joint angles corresponding to each joint of the robotic arm are obtained. Based on the target joint angle and the current joint state of the robotic arm, the first motion parameters of each joint are calculated. The first motion parameters include the joint motion direction, the joint rotation angle, and the motion speed.

4. The self-calibration method for robot arm placement accuracy based on visual feedback according to claim 3, characterized in that, The step of controlling the robotic arm to move the chess piece to a preset observation height above the target placement point according to the first motion parameters, and acquiring a local image of the chess piece and the target placement point, includes: Based on the first motion parameters, a motion control command for the robotic arm is generated, and the motion control command is sent to the robotic arm control module. Drive each joint of the robotic arm to move in the corresponding direction of motion, rotation angle and speed, so that the end of the robotic arm carrying the chess piece moves to the preset observation height above the target placement point; The vision acquisition device is controlled to acquire local images containing chess pieces and target placement points.

5. The self-calibration method for robot arm placement accuracy based on visual feedback according to claim 4, characterized in that, The step of extracting the positional deviation between the center of the chess piece and the center of the target placement point based on the local image to obtain the image spatial deviation includes: Based on a preset image recognition algorithm, the chess piece region and the target placement point region in the local image are detected, and the first position coordinates and the second position coordinates are determined respectively. The first position coordinates are the position coordinates of the chess piece outline center in the image coordinate system, and the second position coordinates are the position coordinates of the target placement point center in the image coordinate system. Based on the first and second position coordinates, the horizontal deviation between the chess piece and the target placement point is determined, and the horizontal deviation is the image space deviation.

6. The self-calibration method for robot arm placement accuracy based on visual feedback according to claim 5, characterized in that, The calculation of the second motion parameter based on the preset robotic arm kinematics model, the image spatial deviation, and the preset observation height includes: The preset observation height is used to determine the vertical movement between the chess piece and the target placement point; The horizontal deviation and vertical motion are fused together to obtain the three-dimensional displacement of the robotic arm end effector. The three-dimensional displacement is input into the preset kinematic model of the robotic arm to obtain the second motion parameters.

7. The self-calibration method for robot arm placement accuracy based on visual feedback according to claim 6, characterized in that, The process of fusing the horizontal deviation and vertical motion to obtain the three-dimensional displacement of the robotic arm's end effector includes: Based on the lateral and longitudinal offsets corresponding to the image spatial deviation, the first and second direction correction distances of the robotic arm end effector in the chessboard plane coordinate system are determined. Based on the height difference between the preset observation height and the current height of the robotic arm end effector, the third-direction movement distance of the robotic arm end effector along the vertical direction is determined; The first direction correction distance, the second direction correction distance, and the third direction motion distance are combined to generate the displacement of the robotic arm end effector in three-dimensional space.

8. The self-calibration method for robot arm placement accuracy based on visual feedback according to claim 7, characterized in that, The step of performing the placing action based on the second motion parameters and obtaining the actual position of the piece after the placement includes: Based on the second motion parameters, a motion control command for the robotic arm end effector is generated, and the robotic arm end effector is controlled to move to the target placement position; The robotic arm is controlled to release the chess piece. A visual acquisition device is used to acquire an image of the chessboard after the piece is placed. The positions of the chess pieces in the chessboard image are identified to determine the actual positions of the pieces after the piece is placed.

9. The self-calibration method for robot arm placement accuracy based on visual feedback according to claim 8, characterized in that, The step of updating the kinematic model parameters of the robotic arm based on the image spatial deviation and the position difference between the actual position of the chess piece and the target placement point includes: Based on the image spatial deviation, a first error amount is obtained; The second error is obtained based on the difference between the actual position of the piece and the target placement point. Map the first error quantity and the second error quantity to a unified coordinate system, and calculate the vector correlation, wherein the vector correlation includes the magnitude deviation and the direction angle; The parameters of the robotic arm kinematic model are updated based on the vector correlation.

10. The self-calibration method for robot arm placement accuracy based on visual feedback according to claim 9, characterized in that, The step of updating the kinematic model parameters of the robotic arm based on the vector correlation includes: If the magnitude deviation and the direction angle are both within a preset consistency threshold, the error source is determined to be a systematic calibration deviation. The average value of the first error and the second error is used as a correction vector to update the kinematic model parameters of the robotic arm. If the magnitude deviation or directional angle exceeds a preset consistency threshold, the nonlinear execution deviation of the robotic arm on the dropping and pressing path is extracted by the vector difference between the first error and the second error. Based on the nonlinear execution deviation, the backlash error of each joint of the robotic arm or the flexible deformation of the link is identified, and the kinematic model parameters of the robotic arm are updated.

Images