Automatic manipulator eye calibration method, device and equipment and medium

Abstract

The invention discloses a manipulator hand-eye automatic calibration method, device and equipment and a medium, and the method comprises the steps: obtaining a calibration plate image, recognizing a feature point and a camera view center, selecting a target feature point, and recording a first manipulator coordinate and an initial pixel coordinate; controlling the manipulator to move by a fixed step length in multiple directions to obtain tentative information, and determining coordinates of a target manipulator and a step length adjusting mode according to the tentative information; iteratively updating the manipulator coordinate and the fixed step length according to the alignment condition, and establishing a corresponding relationship between the final manipulator coordinate and the initial pixel coordinate; and repeating the operation on the other feature points, and calculating a hand-eye transformation relationship based on multiple groups of corresponding relationships. According to the method, the position of the manipulator is gradually corrected in a pixel error driven iterative alignment mode, and stable convergence is realized in combination with step length adjustment, so that the manual point alignment error is reduced, the calibration accuracy is improved, the repeated adjustment process is reduced, and the calibration efficiency is improved.

Description

Technical Field

[0001] This invention relates to the fields of robotics and machine vision technology, and in particular to a method, apparatus, equipment and medium for automatic hand-eye calibration of a robotic arm. Background Technology

[0002] In existing industrial assembly and automated production scenarios, SCARA locking robots typically require hand-eye calibration to establish the correspondence between the camera coordinate system and the robot's coordinate system, ensuring spatial consistency between the locking position and the target workstation. However, current engineering practices still largely rely on manual calibration methods, where operators manually match the robot's position with feature positions in the image. This method has significant limitations in terms of automation and calibration accuracy.

[0003] On the one hand, existing manual point calibration methods mainly rely on the operator's subjective judgment of the target position in the camera image. Since the pixel coordinates in the image themselves do not have intuitive spatial meaning, the human eye can hardly accurately perceive pixel-level positional differences. Operators can usually only move the robotic arm to an approximate position near the target area, and cannot precisely align it with the target point in the image. This method, which relies on experience and visual inspection, inevitably introduces random errors into the calibration results, which can easily lead to limited final operational accuracy, especially in locking applications where high positional accuracy is required.

[0004] On the other hand, existing calibration processes generally lack effective error convergence mechanisms. When the result of a single calibration does not reach the ideal state, it is usually necessary to repeat manual adjustments and recalibration, gradually approaching the optimal position through multiple trials and errors. This iterative operation not only significantly increases the time cost required for calibration, but also makes calibration efficiency heavily dependent on the operator's skill level, making it difficult to maintain consistent calibration quality across different equipment or batches. Summary of the Invention

[0005] The main objective of this invention is to provide a method, apparatus, device, and storage medium for automatic hand-eye calibration of a robotic arm. This invention aims to solve the technical problem that the existing calibration methods for locking robotic arms mainly rely on manual visual inspection for approximate point alignment, lacking an automatic alignment and iterative convergence mechanism based on pixel errors, resulting in difficulty in guaranteeing the accuracy of the calibration process and low overall efficiency.

[0006] To achieve the above objectives, the present invention provides an automatic hand-eye calibration method for a robotic arm, comprising: Place the calibration board within the camera's field of view, acquire an image of the calibration board, identify multiple feature points in the image, and obtain the position of the camera's field of view center in the image. Select one feature point from the plurality of feature points as the target feature point, and record the current first manipulator coordinates and the initial pixel coordinates of the target feature point; Using the first robotic arm coordinates as a reference, the robotic arm is controlled to move in multiple directions by a fixed step length. After each movement, the trial information related to the pixel coordinates of the target feature point and the robotic arm coordinates is recorded, and the robotic arm is controlled to return to the first robotic arm coordinates and continue to move in the next direction. Based on the trial information, the initial pixel coordinates, and the camera field of view center, the target robot's coordinates and the corresponding step size adjustment method are determined. Determine whether the target feature point and the center of the camera's field of view meet preset alignment conditions; If the preset alignment conditions are not met, the first robot coordinates are updated according to the target robot coordinates, and the fixed step size is updated according to the step size adjustment method. The steps of moving and recording trial information, determining the target robot coordinates and the corresponding step size adjustment method, and judging the alignment conditions are repeated with the updated fixed step size until the preset alignment conditions are met. If the preset alignment conditions are met, the current robot arm coordinates are recorded as the final robot arm coordinates, and a correspondence is established between the final robot arm coordinates and the initial pixel coordinates. For each feature point other than the target feature point among the plurality of feature points, the same alignment operation is performed to obtain multiple sets of correspondences, and the hand-eye transformation relationship is calculated based on the multiple sets of correspondences.

[0007] Furthermore, to achieve the above objectives, the present invention provides an automatic hand-eye calibration device for a robotic arm, comprising: The image feature acquisition module is used to place the calibration board within the camera's field of view, acquire an image of the calibration board, identify multiple feature points in the image, and acquire the position of the camera's field of view center in the image. The target point initialization module is used to select one feature point from the plurality of feature points as the target feature point, and to record the current first manipulator coordinates and the initial pixel coordinates of the target feature point. The step length probing control module is used to control the robot to move a fixed step length in multiple directions based on the coordinates of the first robot. After each movement, it records the probing information related to the pixel coordinates of the target feature point and the robot coordinates, and controls the robot to return to the first robot coordinates and continue moving in the next direction. The target coordinate determination module is used to determine the target robot coordinates and the corresponding step size adjustment method based on the trial information, the initial pixel coordinates, and the camera field of view center. The alignment condition determination module is used to determine whether the target feature point and the center of the camera's field of view meet the preset alignment conditions; The iterative update control module is used to update the first robot coordinates according to the target robot coordinates if the preset alignment conditions are not met, and update the fixed step size according to the step size adjustment method. The steps of moving and recording trial information, determining the target robot coordinates and the corresponding step size adjustment method, and judging the alignment conditions are repeated with the updated fixed step size until the preset alignment conditions are met. The single-point correspondence generation module is used to record the current robot arm coordinates as the final robot arm coordinates if the preset alignment conditions are met, and to establish a correspondence between the final robot arm coordinates and the initial pixel coordinates. The hand-eye transformation calculation module is used to perform the same alignment operation on each feature point other than the target feature point among the multiple feature points to obtain multiple sets of correspondences, and calculate the hand-eye transformation relationship based on the multiple sets of correspondences.

[0008] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and a robotic hand-eye automatic calibration program stored in the memory and executable on the processor, wherein when the robotic hand-eye automatic calibration program is executed by the processor, it implements the steps of the robotic hand-eye automatic calibration method as described above.

[0009] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a robotic hand-eye automatic calibration program, wherein when the robotic hand-eye automatic calibration program is executed by a processor, it implements the steps of the robotic hand-eye automatic calibration method as described above.

[0010] Beneficial Effects: This invention discloses an automatic hand-eye calibration method, apparatus, device, and medium for a robotic arm, comprising: acquiring a calibration board image and identifying feature points and the camera's field of view center; selecting target feature points and recording the first robotic arm coordinates and initial pixel coordinates; controlling the robotic arm to move along multiple directions with a fixed step length to acquire trial information, and determining the target robotic arm coordinates and step length adjustment method accordingly; iteratively updating the robotic arm coordinates and fixed step length based on alignment conditions to establish a correspondence between the final robotic arm coordinates and the initial pixel coordinates; repeating the above operations for the remaining feature points, and calculating the hand-eye transformation relationship based on multiple sets of correspondences. This invention gradually corrects the robotic arm position through an iterative alignment method driven by pixel errors, and achieves stable convergence by combining step length adjustment, thereby reducing manual point-to-point errors, improving calibration accuracy, reducing repetitive adjustment processes, and improving calibration efficiency. Attached Figure Description

[0011] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a schematic diagram of an application environment for the automatic hand-eye calibration method of a robotic arm according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating an embodiment of the automatic hand-eye calibration method for robotic arms according to the present invention. Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of the automatic hand-eye calibration device for robotic arms of the present invention; Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0012] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0013] The automatic hand-eye calibration method for robotic arms provided in this invention can be applied to, for example... Figure 1 In this application environment, the client communicates with the server via a network. The server can acquire calibration board images from the client, identify feature points and the camera's field of view center, select target feature points, and record the first robot arm coordinates and initial pixel coordinates. It controls the robot arm to move a fixed step length in multiple directions to acquire trial information and determines the target robot arm coordinates and step length adjustment method accordingly. Based on alignment conditions, iteratively updates the robot arm coordinates and fixed step length, establishing a correspondence between the final robot arm coordinates and the initial pixel coordinates. The above operations are repeated for the remaining feature points, and the hand-eye transformation relationship is calculated based on multiple sets of correspondences. This invention uses a pixel error-driven iterative alignment method to gradually correct the robot arm position and combines step length adjustment to achieve stable convergence, thereby reducing manual point-to-point errors, improving calibration accuracy, reducing repetitive adjustment processes, and increasing calibration efficiency. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0014] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the automatic hand-eye calibration method for robotic arms provided by the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0015] like Figure 2As shown, the automatic hand-eye calibration method for robotic arms proposed in this invention includes the following steps: S10, Place the calibration board within the camera's field of view, acquire an image of the calibration board, identify multiple feature points in the image, and obtain the position of the camera's field of view center in the image; In this embodiment, during the robotic hand-eye calibration process, placing the calibration plate within the camera's field of view aims to provide a stable and identifiable visual carrier for image coordinate acquisition. The calibration plate is positioned where the camera can fully image it, ensuring its surface structure is unobstructed and clearly defined in the image, thereby guaranteeing the effective representation of various spatial information during subsequent image processing. This placement is not limited to a fixed height or angle, but rather requires the calibration plate to form a complete projection within the imaging area, enabling its structural features to be accurately captured by the image acquisition unit.

[0016] With the calibration board within the camera's field of view, the camera acquires images of the area including the calibration board, forming image data for processing. The image acquisition process aims for pixel-level information integrity, ensuring that the edges, intersections, or structural changes of the calibration board are distinguishable in the image, providing a basis for subsequent feature point localization. The quality of the acquired image data, serving as input for all subsequent processing, directly affects the stability of pixel coordinate extraction.

[0017] After image acquisition, multiple feature points are identified within the image. This identification process focuses on extracting pixels that are spatially stable and structurally repeatable. The presence of multiple feature points allows the overall spatial distribution of the calibration board to be described without relying on a single location, thereby improving the reliability of image coordinate information. Each identified feature point corresponds to a pixel position in the image coordinate system, representing the set of discrete positions of the calibration board within the image.

[0018] Simultaneously, it is necessary to obtain the position of the camera's field of view center within the image. This position describes the geometric center relationship of the camera's imaging area. The camera's field of view center originates from the camera imaging model or image size relationship, and it corresponds to a specific position in the image coordinate system, serving as a unified reference for subsequent pixel position relationship calculations. By clearly defining this center position, the positional relationship of any feature point in the image can be described with the center as a reference, thereby avoiding reliance on manual visual judgment of image offset.

[0019] This embodiment obtains the pixel coordinates of multiple feature points in the image and the position of the camera's field of view center simultaneously, so that the calibration process has a clear pixel reference basis, avoiding reliance on manual visual judgment of the image center or position offset, thereby improving the stability and consistency of image coordinate acquisition.

[0020] S20, Select one feature point from the plurality of feature points as the target feature point, and record the current first robot arm coordinates and the initial pixel coordinates of the target feature point; In this embodiment, after identifying multiple feature points in the image, one feature point needs to be selected as the target feature point for establishing a reference in the subsequent alignment process. The purpose of this selection is to determine a clear reference object among multiple available pixel locations, ensuring that subsequent spatial adjustments and position determinations revolve around the same pixel. The target feature point originates from the previously identified feature point set; its position in the image coordinate system is deterministic and repeatable, thus serving as a stable reference in subsequent processing.

[0021] While selecting the target feature point, it is necessary to record the current first robot arm coordinates. The first robot arm coordinates describe the initial spatial position of the robot arm before alignment adjustments are made to the target feature point. These coordinates originate from the robot arm control system or position feedback unit. This recording process fixes the robot arm's spatial position at the current moment in numerical form, providing a basis for subsequent position updates and comparisons. The recording of the first robot arm coordinates does not involve position changes; it only reflects the robot arm's state at the moment the target feature point is selected.

[0022] Simultaneously, the initial pixel coordinates of the target feature points also need to be recorded. These initial pixel coordinates describe the original position of the target feature points in the current image; this position has not yet undergone any alignment adjustments, and its relative relationship with the camera's field of view center reflects the current initial offset. By recording the initial pixel coordinates, subsequent pixel position changes can be analyzed based on a unified starting reference, thereby avoiding positional drift caused by image updates or repeated recognition.

[0023] The target feature points, the coordinates of the first robotic arm, and the initial pixel coordinates are recorded together at the same time point, so that the spatial information of the image and the spatial information of the robotic arm correspond in the time dimension, providing a consistent data basis for position association in subsequent processing.

[0024] This embodiment selects a target feature point from multiple feature points and simultaneously records the coordinates of the first manipulator and the initial pixel coordinates of the target feature point, so that the image coordinate information and the spatial position of the manipulator form a stable correspondence at the same time, thereby providing a consistent data starting point for subsequent position adjustment and alignment judgment.

[0025] S30, using the first robot arm coordinates as a reference, control the robot arm to move a fixed step length in multiple directions, record the trial information related to the pixel coordinates of the target feature point and the robot arm coordinates after each movement, and control the robot arm to return to the first robot arm coordinates and continue moving in the next direction; In this embodiment, based on the determined coordinates of the first robotic arm and the target feature points, the first robotic arm coordinates are used as a spatial reference datum to perform controlled displacement operations on the robotic arm. The first robotic arm coordinates are used to define the center position of the current search area, and their function is to provide a unified starting position for multi-directional movements, making each movement comparable.

[0026] Moving in multiple directions with a fixed step size refers to applying displacement control to the robot in different spatial directions while keeping the step size value constant. The multiple directions are derived from the orientation definitions in the robot's planar coordinate system or workspace coordinate system, and their setting allows the robot to form discrete spatial sampling positions around the target feature point. The fixed step size is used to limit the amplitude of each displacement, thereby preventing the robot from making uncontrollable large movements.

[0027] After each movement and stabilization of the robotic arm, the pixel coordinates associated with the target feature point and the current robotic arm coordinates need to be recorded. The pixel coordinates reflect the positional change of the target feature point in the image, while the robotic arm coordinates reflect the corresponding spatial displacement. Both are recorded and correlated simultaneously to form exploratory information, used to describe the spatial-image correspondence under a specific direction and step length.

[0028] After completing a movement and recording of probe information in one direction, the robot arm needs to be controlled to return to the first robot arm coordinate. This return operation ensures that each movement in one direction starts from the same spatial origin, thus avoiding path overlap or position drift from interfering with the probe results. By continuing to execute the movement in the next direction after returning, probe information from multiple directions is acquired one by one under a unified reference condition.

[0029] This embodiment uses a unified first manipulator coordinate system as a reference to acquire trial information in multiple directions with a fixed step size, making the pixel changes of target feature points under different spatial displacement conditions comparable, thereby providing stable and symmetrical input data for subsequent position determination.

[0030] S40, based on the trial information, the initial pixel coordinates, and the camera field of view center, determine the target robot coordinates and the corresponding step size adjustment method; In this embodiment, after recording trial information in multiple directions, the acquired data needs to be comprehensively analyzed to determine the target manipulator coordinates used to guide subsequent movements and the corresponding step size adjustment method. The trial information includes the pixel coordinate changes of target feature points under different manipulator coordinate conditions. This information reflects the correspondence between the spatial displacement of the manipulator and the positional changes of target feature points in the image.

[0031] The initial pixel coordinates, representing the image position of the target feature point in the baseline state, are used for comparison with the pixel coordinates in each set of trial information. This comparison allows us to obtain the offset direction and magnitude of the target feature point within the image plane, thereby determining the error distribution of the robot's current spatial position relative to the target position.

[0032] The camera's field of view center represents the reference position used for alignment in the image coordinate system. Its function is to provide a unified criterion for judging pixel coordinate offsets. By correlating the pixel coordinates in the trial information with the camera's field of view center, the trend of target feature points moving closer to or further away from the camera's field of view center under different trial directions can be determined.

[0033] Based on the above analysis, the corresponding robot coordinates in the trial information are filtered and combined to determine the spatial position in pixel space that causes the target feature point to change towards the center of the camera's field of view, which is then used as the target robot coordinates. Simultaneously, based on the correspondence between pixel offset amplitude and fixed step size, it is determined whether the current step size meets the convergence requirements, and a step size adjustment method is generated accordingly to correct the subsequent robot movement amplitude.

[0034] This embodiment combines the analysis of trial information, initial pixel coordinates, and camera field of view center to establish the determination of the target robot's coordinates on a clear correspondence between spatial displacement and pixel changes, and simultaneously generates a matching step size adjustment method, thereby improving the directionality and convergence stability of position adjustment.

[0035] S50, determine whether the target feature point and the center of the camera's field of view meet the preset alignment conditions; In this embodiment, after obtaining the coordinates of the target robotic arm, it is necessary to determine the positional state of the target feature points in the image to confirm whether the current position of the robotic arm has reached an acceptable level of alignment. The relationship between the target feature points and the center of the camera's field of view reflects the result of the current spatial position adjustment, which is directly reflected in the deviation state at the pixel coordinate level.

[0036] Preset alignment conditions are used to limit the maximum allowable deviation range between the target feature point and the camera's field of view center within the image plane. These conditions can originate from system calibration accuracy requirements, robot arm repeatability accuracy, or the smallest resolvable unit corresponding to the image resolution. By comparing the current pixel coordinates of the target feature point with the camera's field of view center, the horizontal and vertical offsets can be calculated, or a comprehensive pixel distance can be calculated based on these offsets.

[0037] When the calculation result falls within the threshold range defined by the preset alignment conditions, it indicates that the current spatial position of the robot arm has established a stable correspondence between the target feature points and the center of the camera's field of view at the pixel level, and further position correction using trial information is no longer necessary. If it exceeds this range, it indicates that there are still unresolved spatial deviations, and further corrections are required based on the previously determined target robot arm coordinates and step size adjustment method.

[0038] This judgment process logically serves as a convergence determination, distinguishing between intermediate states and acceptable termination states during the adjustment process, thereby avoiding invalid iterations or premature termination.

[0039] This embodiment quantitatively determines the relationship between target feature points and the camera's field of view at the pixel coordinate level, freeing the alignment state determination from reliance on human experience and providing clear termination conditions for subsequent processing, thereby improving the stability and consistency of the overall adjustment process.

[0040] S60, if the preset alignment condition is not met, the first robot coordinate is updated according to the target robot coordinate, and the fixed step size is updated according to the step size adjustment method. The steps of moving and recording trial information, determining the target robot coordinate and the corresponding step size adjustment method, and judging the alignment condition are repeated with the updated fixed step size until the preset alignment condition is met. In this embodiment, when the target feature point and the center of the camera's field of view do not meet the preset alignment conditions, it indicates that the current position of the robotic arm has not yet brought the pixel-level deviation within the allowable range, and the spatial position and adjustment strategy of the robotic arm need to be updated. The target robotic arm coordinates are used to indicate a better spatial position direction under the analysis results of the existing trial information, and they are derived from a comprehensive judgment of the relationship between the trial robotic arm coordinates and the corresponding pixel deviation.

[0041] The process of updating the coordinates of the first robotic arm is used to switch the current reference position of the robotic arm to the coordinates of the target robotic arm. This allows subsequent movement operations to be performed based on a position closer to the alignment direction, thus avoiding unnecessary repeated adjustments near the original reference position. Logically, this update operation implements the dynamic migration of the search reference point.

[0042] The step size adjustment method controls the trend of the fixed step size, which is derived from the comparison results of previous pixel deviations, reflecting whether the current adjustment is close to the optimal area. When the pixel deviation has not improved, reducing the fixed step size can improve the resolution of the position adjustment; when the pixel deviation is still within the improvement stage, maintaining the fixed step size helps to maintain adjustment efficiency. The update of the fixed step size directly affects the magnitude and precision of subsequent movement operations.

[0043] After updating the coordinates and fixed step size of the first robot arm, the process of re-executing the movement and recording the trial information, determining the coordinates of the target robot arm and the corresponding step size adjustment method, and judging the alignment conditions is carried out. This process forms a closed-loop iterative structure for the entire adjustment process until the preset alignment conditions are met.

[0044] This embodiment dynamically updates the coordinates of the first robotic arm when the preset alignment conditions are not met, and adjusts the fixed step size in conjunction with the step size adjustment method. This enables the position adjustment process to adaptively balance between efficiency and accuracy, avoids invalid searches, and gradually guides the pixel deviation to converge.

[0045] S70, if the preset alignment conditions are met, the current robot arm coordinates are recorded as the final robot arm coordinates, and the final robot arm coordinates and the initial pixel coordinates are established as a correspondence; In this embodiment, when the position of the target feature point in the image meets the preset alignment conditions with the center of the camera's field of view, it indicates that the current spatial position of the robot has established a stable correspondence with the target position at the pixel level, and further position adjustments will no longer produce effective error improvement. In this state, the current spatial pose of the robot can be regarded as the convergence result for the target feature point.

[0046] The operation of recording the current robot arm coordinates is used to solidify this convergence state, and the robot arm pose at the moment the alignment conditions are met is used as spatial coordinate data that can be directly used in subsequent calibration calculations. These robot arm coordinates are no longer used as temporary reference points during the adjustment process, but are saved as data results with definite meaning.

[0047] A correspondence is established between the final robot coordinates and the initial pixel coordinates to express the mapping relationship between the target feature point positions in the image coordinate system and their corresponding spatial positions in the robot coordinate system. The initial pixel coordinates are derived from the image positions of the feature points before any robot adjustments. The pairing relationship between the initial pixel coordinates and the final robot coordinates reflects the actual correspondence from pixel space to robot space, providing the basic input for subsequent spatial transformation calculations based on multiple sets of data.

[0048] This embodiment solidifies the robot's coordinates and establishes a correspondence with the initial pixel coordinates in real time when the alignment conditions are met, so that the mapping data between the pixel space and the robot's space has a clear source and stable meaning, avoiding data drift caused by over-adjustment.

[0049] S80, perform the same alignment operation on each feature point other than the target feature point among the plurality of feature points to obtain multiple sets of correspondences, and calculate the hand-eye transformation relationship based on the multiple sets of correspondences.

[0050] In this embodiment, after aligning a single target feature point and establishing the correspondence between pixel coordinates and robot coordinates, a single set of data is insufficient to support a stable and solvable mapping relationship between spatial coordinate systems. Therefore, it is necessary to perform the same alignment operation process as the target feature point on each of the remaining feature points on the calibration board, so that each feature point obtains an independent and usable set of correspondence data.

[0051] Performing the same alignment operation on every feature point except the target feature point means that the feature point is reselected as the current processing object in each processing iteration, and the pixel-bias-based movement, judgment, and convergence process is repeated until the position of the feature point in the image meets the alignment conditions. After each alignment is completed, a set of correspondences consisting of initial pixel coordinates and final robot arm coordinates is obtained.

[0052] By performing the above process on multiple feature points, multiple sets of corresponding data distributed in the image space and the robot arm space can be formed. These correspondences are discrete and covert in spatial location, making the overall mapping relationship between the pixel coordinate system and the robot arm coordinate system computable. Based on these multiple sets of correspondences, the hand-eye transformation relationship between image coordinates and robot arm coordinates can be established by solving for spatial transformation parameters, which can be used to describe the geometric mapping between the two coordinate systems.

[0053] This embodiment establishes a stable correspondence between pixel coordinates and robot coordinates for multiple feature points, providing sufficient data support for spatial transformation calculations, thereby improving the stability of coordinate mapping relationships and the reliability of overall calibration results.

[0054] In one embodiment, step S10 above includes: S101 controls the rotation of each joint of the robotic arm and adjusts the relative angle between the upper arm and forearm of the robotic arm so that the robotic arm avoids the strange position of being fully extended or fully folded, so as to keep the robotic arm in a non-singular posture. S102, place the calibration board within the camera's field of view, control the camera to acquire raw image data containing the calibration board, preprocess the raw image data, and obtain an image of the calibration board; S103, Multiple feature points are identified from the image using a corner detection algorithm, and the pixel coordinates of each feature point are extracted; S104, based on the camera's imaging parameters, determines the coordinate position of the camera's field of view center in the image coordinate system.

[0055] In this embodiment, controlling the rotation of each joint of the manipulator and adjusting the relative angle between the upper and lower arms constitutes a preprocessing step for the manipulator's posture accessibility and motion stability. Joint rotation refers to the change in the angle of the driven joints, placing the end effector within a suitable posture range for observation and movement. The relative angle between the upper and lower arms refers to the angle formed by the two linkage configurations. When this angle approaches full extension or full folding, the mapping relationship between joint velocity and end effector displacement changes drastically, leading to unstable response of the end effector to small control quantities, thus affecting the accuracy of fixed-step movement and position reproduction in subsequent alignment processes. Avoiding singular positions of full extension or full folding is achieved by setting an angle range and constraining the target joint angle. This can be achieved through joint angle threshold judgment, posture constraints in motion planning, or singularity avoidance strategies within the controller. Maintaining the manipulator in a non-singular posture means restricting the manipulator's working point to a region with reversible kinematic mapping and favorable numerical conditions, making the end effector displacement during subsequent multi-directional movement closer to the commanded displacement and reducing the cumulative deviation when repeatedly returning to the first manipulator coordinates.

[0056] Placing the calibration board within the camera's field of view and controlling the camera to acquire raw image data containing the calibration board constitutes the image acquisition stage for obtaining input data for subsequent feature point recognition. Placing the calibration board within the camera's field of view ensures it is completely or sufficiently covered on the imaging plane, avoiding missing feature points or edge distortion. Camera acquisition is controlled by triggering acquisition commands and setting parameters such as exposure time, gain, frame rate, and resolution to achieve the required sharpness and contrast for feature point detection. Raw image data is the unprocessed or minimally processed data stream output from the camera sensor, containing noise, uneven illumination, lens distortion residue, and color channel differences. Preprocessing the raw image data to obtain the image of the calibration board involves performing a standardized processing procedure on the acquired data that can be used for detection. The goal of this processing is to improve the separability of corner structures at the pixel level and reduce unstructured interference. Preprocessing may include denoising filtering to suppress random noise, grayscale conversion or channel selection to reduce detection instability caused by color differences, histogram stretching or local contrast enhancement to strengthen the boundaries of black and white grids or dot arrays, binarization or edge enhancement to highlight geometric contours, distortion correction or image cropping to remove irrelevant areas and reduce the impact of edge distortion. The calibration board image output from preprocessing needs to be consistent with the input format for subsequent corner detection and maintain a consistent pixel coordinate system definition so that the extracted pixel coordinates can be compared with the coordinate position of the camera's field of view center in the same coordinate system.

[0057] Using corner detection algorithms to identify multiple feature points in an image and extract the pixel coordinates of each feature point is a crucial step in converting the calibration board's geometry into discrete pixel measurements. The function of corner detection algorithms is to extract the intersection points of local two-dimensional structures from locations with significant grayscale or gradient changes. Feature points can be checkerboard intersections, the center of a dot array, or other points with stable geometric constraints. Identifying multiple feature points means locating multiple candidate points across the entire image and filtering out false positives using response value thresholding, non-maximum suppression, connected component constraints, or geometric consistency checks. Extracting the pixel coordinates of each feature point means outputting the coordinate values ​​of that point in the image coordinate system. Pixel coordinates must include at least horizontal and vertical coordinates, and coordinate accuracy can be further improved through sub-pixel precision localization. Sub-pixel precision localization can be achieved based on local grayscale fitting, corner response function interpolation, or iterative error minimization, freeing pixel coordinates from the limitation of integer pixel grids, thus enabling higher resolution for subsequent pixel-distance-based alignment judgments. The pixel coordinates of multiple feature points output in this step are not only used for subsequent selection of target feature points, but also provide a repeatable image-side measurement benchmark for establishing multiple sets of correspondences. Therefore, it is necessary to ensure that the output coordinates are strictly consistent with the coordinate system of the preprocessed image to avoid the problem of asynchronous coordinate transformation caused by image cropping, scaling or distortion correction.

[0058] Determining the coordinates of the camera's field of view center in the image coordinate system based on the camera's imaging parameters is crucial for providing a unified reference point for subsequent alignment decisions. The coordinates of the camera's field of view center in the image coordinate system can be derived from imaging parameters, including image resolution, principal point parameters, or the definition of the effective imaging region. If the imaging parameters include principal point coordinates, the camera's field of view center can be directly derived from these coordinates; if the imaging parameters only include image width and height and the effective region boundary, the camera's field of view center can be calculated from the geometric center of the effective region. This determination process must be consistent with the preprocessing process. When preprocessing includes cropping, scaling, or distortion correction, the coordinates of the camera's field of view center need to be synchronously mapped to the preprocessed image coordinate system to ensure that it is on the same coordinate reference as the feature point pixel coordinates. As the reference point for subsequent pixel distance calculations and the determination of preset alignment conditions, the determination of the camera's field of view center's coordinates directly affects the numerical stability of the error measurement and the consistency of the alignment termination condition determination. Therefore, it is necessary to use reproducible parameter sources and solidify the calculation rules to ensure consistency in the definition of the reference point across different batches of images acquired.

[0059] This embodiment constrains the robot arm to a non-singular posture and performs image preprocessing for the calibration board before image acquisition, enabling the calibration board image acquired by the camera to have a more stable structural feature expression, thereby improving the detectability and positioning accuracy of multiple feature point pixel coordinates. At the same time, by determining the coordinate position of the camera's field of view center in the image coordinate system based on imaging parameters, it provides an error measurement reference point consistent with the feature point pixel coordinates, so that subsequent alignment judgment based on pixel distance has a consistent coordinate benchmark and reproducible calculation input, thereby reducing the accumulation of alignment errors caused by posture instability, image quality fluctuations, or inconsistent reference point definitions.

[0060] In one embodiment, step S30 above includes: S301, setting multiple movement directions including the positive horizontal axis, negative horizontal axis, positive vertical axis, and negative vertical axis in the robot's planar coordinate system; S302, for each direction of movement, control the robot arm to start from the first robot arm coordinates, move a fixed step distance along the direction of movement, and keep the robot arm stationary; S303, record the robot arm coordinates after the robot arm moves as the probe robot arm coordinates, and identify the position of the target feature point in the current image as the probe pixel coordinates; S304, The association between the probe manipulator coordinates and the probe pixel coordinates is recorded as probe information; S305, control the robot arm to return to the first robot arm coordinate and continue moving in the next direction until all trial operations in all directions of movement have been completed.

[0061] In this embodiment, the first manipulator coordinates serve as a reference point to solidify the manipulator's current position as a reference point for one trial cycle. Subsequent displacement calculations, backtracking control, and trial information archiving in each movement direction all revolve around the first manipulator coordinates, establishing the same reference. The first manipulator coordinates can originate from the real-time feedback pose of the manipulator controller or from the position confirmation readback value after the motion control command is issued. To avoid reference drift, the first manipulator coordinates need to be written into a cache and remain unchanged before the start of a trial cycle, until all trial operations in all movement directions are completed. Using the first manipulator coordinates as a reference also implies consistency in coordinate representation. The manipulator coordinates need to adopt a coordinate representation method consistent with the manipulator's planar coordinate system, including at least displacement components in the horizontal and vertical axes. If necessary, attitude components can be included, but during planar trials, they are used to constrain the attitude to remain constant, so as to avoid changes in image side projection caused by attitude changes interfering with the comparison of trial pixel coordinates.

[0062] Multiple movement directions are defined to construct a discrete search neighborhood around the first manipulator coordinate system, allowing the exploration to cover four independent directions: positive horizontal axis, negative horizontal axis, positive vertical axis, and negative vertical axis. The manipulator's planar coordinate system defines the space of movement directions, forming a pair of opposite direction vectors between the positive and negative horizontal axes, and another pair between the positive and negative vertical axes. The movement directions can originate from the axial definition projected from the manipulator's base coordinate system onto the working plane, or from the tooling coordinate system or the local coordinate system of the calibration plate plane. Once selected, the unit vectors of the four movement directions need to be fixed and remain unchanged within the same exploration cycle to avoid incomparable exploration information due to changes in direction definitions. To ensure the repeatability of the four-directional exploration, the movement directions can be written into the control program in a discrete enumeration form, for example, by binding direction vectors with direction identifiers, while retaining the target displacement after combining each direction with a fixed step size, facilitating the generation of motion control commands.

[0063] A fixed step size is used to uniformly constrain the displacement scale of each trial. A fixed step size means that the displacement modulus remains consistent across all movement directions, ensuring that the trial results are caused only by directional differences and not masked by step size differences. The fixed step size can be derived from preset configuration parameters or updated from the previous iteration. Maintaining a constant fixed step size throughout a four-directional trial ensures comparability within the same round of trials. Controlling the robot arm to move a fixed step size along the movement direction requires combining the fixed step size with the corresponding movement direction to generate candidate values ​​for the target robot arm coordinates. The motion controller then executes point-to-point motion or linear interpolation motion. To ensure a stable correlation between the trial robot arm coordinates and the trial pixel coordinates, the motion process needs to include positioning logic. For example, it might confirm that the robot arm has reached the target position based on a position error threshold, a velocity threshold, or a controller positioning flag, and then enter a state where the robot arm remains stationary. The purpose of keeping the robot arm stationary is to eliminate motion blur and time drift during image acquisition and target feature point localization, ensuring that the current image and the current robot arm coordinates are at the same moment. The holding time can be determined by the camera exposure time, the image frame synchronization period, and the controller stabilization time. Alternatively, image acquisition can be triggered by detecting that the robot arm's feedback speed approaches zero and continuously meets the stabilization condition.

[0064] The trial robot coordinates are recorded from the robot's position after movement. These trial robot coordinates must be read while the robot is stationary to avoid reading intermediate values ​​during movement. The recorded coordinates should retain the same coordinate format as the initial robot coordinates, along with the current movement direction and fixed step size, facilitating subsequent categorization and comparison of the trial information by direction. The position of the target feature point in the current image is identified to form trial pixel coordinates. These trial pixel coordinates can be obtained by performing target feature point localization on the current image. The localization process must maintain the same coordinate system as the initial pixel coordinates to ensure the pixel distance between the trial pixel coordinates and the camera's field of view center can be directly calculated. To improve the stability of the trial pixel coordinates, the localization process can use sub-pixel corner localization or center fitting based on a local window, and output an invalid flag when recognition fails to avoid contaminating the trial information. However, within this processing section, the key is to ensure that the trial pixel coordinates and the trial robot coordinates are acquired and recorded in pairs at the same trial moment.

[0065] The trial information is used to express the relationship between the trial robot coordinates and the trial pixel coordinates. The minimum structure of the trial information can contain a set of key-value pairs, where the key contains the movement direction identifier and the trial robot coordinates, and the value contains the trial pixel coordinates. Alternatively, it can be in the form of record entries, combining the trial robot coordinates, trial pixel coordinates, movement direction identifier, and fixed step size into an indivisible record. The organization of the trial information needs to support subsequent traversal to extract all trial pixel coordinates and support tracing the corresponding trial robot coordinates based on the trial pixel coordinates. Therefore, when writing the trial information, it should be ensured that the fields of each record are complete and consistent to avoid missing fields that would prevent the subsequent retrieval of the trial robot coordinates from the trial information. The storage medium for the trial information can be a memory structure, a cache queue, or a persistent table, but the focus of this processing is to write it immediately after each movement to ensure that the trial information completely covers all movement directions after the four-way trial is completed.

[0066] The control mechanism returns the robotic arm to its first coordinate and continues movement in the next direction. This ensures that each attempt is made under the same starting conditions, making the results comparable across different movement directions. The return control generates motion control instructions to return from the trial robotic arm coordinate to the first robotic arm coordinate, and performs a stability check again upon arrival to ensure that the next movement in the next direction originates from the same first robotic arm coordinate. Continuing movement in the next direction means executing a loop according to a preset sequence of movement directions until all directions—positive horizontal, negative horizontal, positive vertical, and negative vertical—are completed. The determination that all trial operations have been completed can be based on a direction counter, a direction identifier traversal completion flag, or a condition that the number of trial information entries reaches the required number of movement directions. This ensures that the exit condition corresponds one-to-one with the set of movement directions, preventing missed or repeated trials that could lead to inconsistent trial information.

[0067] This embodiment uses the coordinates of the first manipulator as a fixed reference and performs discrete probes with fixed step sizes in the positive horizontal direction, negative horizontal direction, positive vertical direction, and negative vertical direction. After each stationary position, the probe manipulator coordinates and probe pixel coordinates are recorded synchronously and written into the probe information. At the same time, the coordinates of the first manipulator are returned after each probe to unify the starting conditions. This makes the probe information comparable in terms of directional coverage, displacement scale consistency, and coordinate time consistency. This provides a structured, traceable, and lower-noise data input for subsequent determination of the target manipulator coordinates and step size adjustment method based on the probe information, initial pixel coordinates, and camera field of view center.

[0068] In one embodiment, step S40 above includes: S401, calculate the pixel distance between the initial pixel coordinates and the center of the camera's field of view, and mark the calculated distance value as the first pixel distance; S402, traverse the trial information, extract all the trial pixel coordinates contained in the trial information, calculate the pixel distance between each trial pixel coordinate and the center of the camera's field of view, and mark the calculated distance value as the trial pixel distance; S403, compare the first pixel distance with all the trial pixel distances in terms of numerical value, and select the minimum pixel distance with the smallest value; S404, if the minimum pixel distance is the first pixel distance, then the target robot coordinates are determined to be the first robot coordinates, and the corresponding step size adjustment method is determined to reduce the fixed step size; S405, if the minimum pixel distance is a certain trial pixel distance, then obtain the trial robot coordinates corresponding to the trial pixel distance from the trial information, determine the target robot coordinates as the trial robot coordinates, and determine the corresponding step size adjustment method as keeping the fixed step size unchanged.

[0069] In this embodiment, trial information, initial pixel coordinates, and the camera's field of view center together constitute the input set required for a single decision. The trial information originates from trial operations in multiple directions and includes at least a record linking the trial manipulator coordinates to the trial pixel coordinates, allowing the pixel-side deviation to be traced back to the trial manipulator coordinates that generated the deviation. The initial pixel coordinates are derived from the recorded results after selecting the target feature point, representing the image position of the target feature point before the current trial movement. The camera's field of view center is derived from camera imaging parameters or the image coordinate system definition, representing the reference position used for alignment determination within the image coordinate system. Combining these three elements to determine the target manipulator coordinates and the corresponding step size adjustment method essentially involves quantifying and comparing the deviations of multiple candidate positions in the pixel domain and mapping the optimal deviation back to the control decision in the manipulator domain.

[0070] The pixel distance between the initial pixel coordinates and the camera's field of view center is used to express the deviation in the current baseline state. The pixel distance means converting the difference between the initial pixel coordinates and the camera's field of view center into a single scalar, using the image coordinate system as the metric space, facilitating a unified comparison with the distances of other candidate states. The calculation of the pixel distance requires a clear distance measurement method. The pixel distance can be obtained by combining the horizontal and vertical pixel differences. This combination can use Euclidean distance to express two-dimensional geometric deviation, or Manhattan distance to express the linear superposition of axial deviations. Regardless of the combination method, the initial pixel coordinates and the coordinates of the camera's field of view center must be in the same image coordinate system and maintain the same pixel unit. Labeling the calculated distance value as the first pixel distance serves to name and solidify the deviation in the baseline state, enabling subsequent comparisons to establish a clear set of comparison objects between the first pixel distance and the trial pixel distances, avoiding ambiguity caused by unclear comparison object sources.

[0071] The purpose of traversing the trial information is to construct a set of candidate deviations within the same round of trials. The implementation of the traversal depends on the organizational structure of the trial information. When the trial information is stored as a set of entries, traversal can be performed sequentially by entry; when the trial information is stored as a direction index structure, traversal can be performed sequentially by the movement direction index. Extracting all trial pixel coordinates from the trial information requires that each record of the trial information carries a trial pixel coordinate field, and the coordinate expression of this field is consistent with the initial pixel coordinates. The pixel distance between each trial pixel coordinate and the camera's field of view center is calculated to form a set of trial pixel distances. The calculation process requires repeatedly performing the same distance metric on each set of trial pixel coordinates to ensure that the trial pixel distances can be directly compared. The calculated distance values ​​are marked as trial pixel distances, giving each candidate direction a clear deviation identifier and maintaining a traceable relationship with its source trial pixel coordinates. This facilitates subsequent tracing from the trial pixel distance to the trial robot coordinates that generated that distance.

[0072] The comparison between the first pixel distance and the trial pixel distances is used to select the state with the smallest deviation between the baseline state and each trial state. The input set for the comparison includes the first pixel distance and all trial pixel distances, and the output is the minimum pixel distance with the smallest value. The selection of the minimum pixel distance can be implemented using a linear scan method, taking the first pixel distance as the initial minimum and comparing it with each trial pixel distance to update it, or by constructing all distances into a unified array and performing minimum value extraction. Regardless of the implementation method, the source identifier of the minimum pixel distance needs to be retained to distinguish whether the minimum pixel distance comes from the first pixel distance or from a trial pixel distance. The role of the minimum pixel distance is to transform the two-dimensional deviation problem into a single optimality criterion, allowing the target robot's coordinates and step size adjustment method to be determined through branch conditions without introducing additional decision variables.

[0073] When the minimum pixel distance is the first pixel distance, it means that the state based on the first robot arm coordinates has the smallest deviation in the candidate set in this round. Determining the target robot arm coordinates as the first robot arm coordinates means keeping the target position in the next iteration at the first robot arm coordinates, without performing a reference migration towards any trial direction. The purpose of determining the corresponding step size adjustment method as reducing the fixed step size is to shrink the search scale without position migration, allowing subsequent trials to be performed near the first robot arm coordinates with a smaller fixed step size, reducing the risk of crossing the optimal position due to excessively large step sizes. The specific numerical update of reducing the fixed step size is not elaborated in this section, but the step size adjustment method, as a control variable, needs to be explicitly output so that there is a definite basis for subsequent actions to update the fixed step size.

[0074] When the minimum pixel distance is a certain trial pixel distance, it means that the deviation of a certain trial state is better than the baseline state. At this time, it is necessary to obtain the trial manipulator coordinates corresponding to the trial pixel distance from the trial information to realize the mapping from pixel domain optimality to manipulator domain control target. The logic depends on the association structure of the trial information. The trial pixel distance is calculated from the trial pixel coordinates and the camera's field of view center. Therefore, the trial information needs to support locating the corresponding record by the trial pixel coordinates that generated the trial pixel distance, and then reading the trial manipulator coordinates in that record to avoid a broken chain where only distance values ​​are available but the source of the coordinates cannot be traced. Determining the target manipulator coordinates as trial manipulator coordinates means migrating the baseline for the next iteration to these trial manipulator coordinates, forming a control result that advances along the optimal direction. Determining the corresponding step size adjustment method as maintaining a fixed step size means maintaining the search scale when a better direction has been found to maintain advancement efficiency and continue probing for a better position with the same fixed step size in the next round until a situation arises where the baseline is better than the four-way trial, at which point the fixed step size is reduced. The output of maintaining a fixed step size as a step size adjustment method also needs to be directly referenced in the subsequent fixed step size update stage to ensure that the decision output and the execution action are connected in a closed loop.

[0075] This embodiment marks the pixel distance between the initial pixel coordinates and the center of the camera's field of view as the first pixel distance. When traversing the trial information, the trial pixel distance is calculated for all trial pixel coordinates. Then, the first pixel distance and the trial pixel distance are compared to obtain the minimum pixel distance. This allows the target robot's coordinates and step size adjustment method to be determined from the source of the minimum pixel distance. Thus, when the baseline state is better, the fixed step size is reduced to shrink the search scale. When the trial state is better, the target robot's coordinates are migrated to the corresponding trial robot coordinates and the fixed step size is kept unchanged to continue the advance. This achieves a quantifiable mapping and traceable closed loop from pixel deviation to robot control decision.

[0076] In one embodiment, step S60 above includes: S601, if the preset alignment condition is not met, compare whether the coordinates of the target robot arm are the same as the coordinates of the first robot arm; S602, if the coordinates of the target robot are the same as the coordinates of the first robot, then keep the position of the robot unchanged; S603, if the target robot coordinates are not the same as the first robot coordinates, then control the robot to move to the target robot coordinates and update the target robot coordinates to the new first robot coordinates; S604, if the step size adjustment method is to decrease, then the fixed step size is reduced according to a preset ratio; if the step size adjustment method is to keep it unchanged, then the fixed step size is kept unchanged. S605, using the current first robot arm coordinates as a reference, and with the current fixed step size, repeatedly execute the steps of moving and recording trial information, determining the target robot arm coordinates and the corresponding step size adjustment method, and judging the alignment conditions, until the preset alignment conditions are met.

[0077] In this embodiment, failure to meet the preset alignment conditions indicates that the deviation between the target feature point and the camera's field of view is still unacceptable in the current iteration. Continuing to output the correspondence along the original state would introduce systematic errors. Therefore, it is necessary to enter the linkage update process of coordinates and step size so that the next round of alignment operation can continue to advance under the new spatial center or new search scale. Updating the first robot coordinates according to the target robot coordinates means re-anchoring the reference position of the next round of trial operation to the currently determined better position. Updating the fixed step size according to the step size adjustment method means binding the displacement scale of the next round of trial operation with the deviation comparison result of this round, so that the search can both advance in a better direction and shrink the search scale when it cannot continue to advance, avoiding repeated oscillations under a large fixed step size.

[0078] Comparing the target robot coordinates with the first robot coordinates distinguishes between two iteration scenarios. In one scenario, the target robot coordinates are the same as the first robot coordinates. This means the candidate position corresponding to the minimum pixel distance obtained from the trial information is still located at the first robot coordinates. This is manifested in the fact that no directional improvement closer to the camera's field of view center was found after trial movements in multiple directions. This result requires keeping the robot position unchanged to avoid invalid displacement without gain, while providing a stable spatial reference for subsequent fixed-step contraction. In the other scenario, the target robot coordinates are different from the first robot coordinates. This means that some trial robot coordinates in the trial information bring a smaller deviation in the pixel domain. This result requires controlling the robot to move to the target robot coordinates, migrating the robot's physical position to a candidate position with a smaller deviation, thus placing the search center for the next iteration in a better neighborhood.

[0079] Maintaining the robot's position is not only a constraint on physical execution but also on state variables. Keeping the robot's position unchanged ensures that the current reference position can still serve as a baseline for multi-directional exploration in the next round, preventing baseline drift if a better direction is not found. In this branch, the first robot's coordinates remain numerically unchanged, and the exploration information for the next round will be generated around the same first robot coordinates, making the exploration results from different rounds comparable and providing a clear effective baseline for fixed-step updates.

[0080] Controlling the robotic arm to move to the target robotic arm coordinates embodies a closed-loop mapping from pixel-domain optimality to robotic arm-domain motion. The movement command needs to be sent to the robotic arm controller using the target robotic arm coordinates as the target point parameter. This ensures the end effector reaches the target point and stabilizes. Subsequently, the target robotic arm coordinates are updated to the new first robotic arm coordinates, completing the replacement of the reference coordinates. This update ensures that the next round of exploration information is generated around the new first robotic arm coordinates, allowing the search center to iteratively advance in the optimal direction, reducing the probability of repeated explorations near the original first robotic arm coordinates that fail to converge. Updating the target robotic arm coordinates to the new first robotic arm coordinates also ensures that the subsequent "return to first robotic arm coordinates" action has a unique direction, avoiding incorrect return paths or incorrect attribution of exploration information due to inconsistencies between the reference coordinates and the actual position.

[0081] Updating the fixed step size using a step size adjustment method is used to adjust the displacement scale of the next round of exploration after updating or keeping it unchanged at the spatial center. The step size adjustment method involves reducing the corresponding fixed step size by a preset ratio, which can be a fractional ratio or other scaling factor less than one. This ensures the fixed step size is numerically smaller than the previous fixed step size, thereby reducing the search radius of the next round of exploration, improving the resolution of the alignment process within the local area, and reducing the risk of crossing the optimal position. Alternatively, maintaining the fixed step size without numerical update allows the next round of exploration to continue with the original fixed step size, maintaining the propulsion speed. This is suitable for situations where a better direction has been found and the robot has migrated to a new first manipulator coordinate system, allowing the search to continue along the better neighborhood without reducing the speed of approaching the camera's field of view too early.

[0082] Using the current first robot arm coordinates as a reference, and repeatedly executing the steps of moving and recording trial information, determining the target robot arm coordinates and the corresponding step size adjustment method, and judging alignment conditions, constitutes a closed-loop iterative structure. The current first robot arm coordinates are the values ​​after the target robot arm coordinates are updated to the new first robot arm coordinates or remain unchanged. The current fixed step size is the value after the fixed step size is reduced according to a preset ratio or remains unchanged. Using the current first robot arm coordinates as a reference ensures that the starting point of each round of trials is consistent and can be reused with return actions. The current fixed step size ensures that the displacement scale and step size adjustment method of each round of trials are consistent and take effect directly. In the repeated execution chain, moving and recording trial information provides correlation data between pixel-domain deviation and robot arm coordinates; determining the target robot arm coordinates and the corresponding step size adjustment method transforms the correlation data into iterative decisions; and judging alignment conditions provides convergence criteria. When the alignment conditions are still not met, this process is triggered again, realizing an alternating combination of spatial center migration and search scale contraction, allowing the iteration to dynamically switch between advancement and refinement until the alignment conditions are met.

[0083] This embodiment compares the coordinates of the target robot arm with the coordinates of the first robot arm when the preset alignment conditions are not met. It distinguishes between two branches: keeping the robot arm position unchanged and controlling the robot arm to move to the coordinates of the target robot arm. In the coordinate migration branch, the coordinates of the target robot arm are updated to the new coordinates of the first robot arm. At the same time, according to the step size adjustment method, the fixed step size is updated by reducing or keeping it unchanged according to a preset ratio. Then, the trial information generation, target robot arm coordinate determination and alignment condition judgment are repeated with the updated first robot arm coordinates and the updated fixed step size. This makes the iterative process have a clear spatial reference update mechanism and search scale adaptive mechanism, reducing invalid displacement and repeated trials, and improving the stability and efficiency of the alignment process converging towards the center of the camera's field of view.

[0084] In one embodiment, step S70 above includes: S701, if the preset alignment conditions are met, then the movement control and probing operation of the robotic arm are terminated. S702, Read the current robot arm coordinates and record the current robot arm coordinates as the final robot arm coordinates; S703, pair the final robot arm coordinates with the initial pixel coordinates of the target feature points to establish a correspondence; S704, save the set of correspondences to the calibration dataset.

[0085] In this embodiment, meeting the preset alignment conditions means that the relative deviation between the target feature point and the camera's field of view center has entered the allowable range. Continuing to execute motion control and probing operations would introduce new displacement disturbances and disrupt the currently achieved alignment state. Therefore, it is necessary to switch to the data solidification and correspondence generation process. Terminating the motion control and probing operations of the robot arm includes stopping the motion command channel and the probing trigger channel, keeping the robot arm stationary and freezing the control output of the current iteration to avoid positional changes during the reading of robot arm coordinates, which would lead to instability in the final robot arm coordinates. The termination action also includes switching the control state of this round of alignment process from the loop execution state to the recording state, so that subsequent reading and storage actions are completed in a consistent control context, ensuring that the pairing of the final robot arm coordinates and the initial pixel coordinates is based on the same alignment moment.

[0086] Reading the current robot arm coordinates means obtaining the robot arm's coordinate data in a stationary state from the robot arm controller or position feedback link. The robot arm coordinates can be obtained from the joint encoder feedback through forward kinematics calculation, or the controller can directly provide the end-effector pose coordinates output. The reading process needs to be performed after the movement control is terminated to ensure that the current robot arm coordinates correspond to the actual position when the preset alignment conditions are met, rather than a transitional position during movement. Recording the current robot arm coordinates as the final robot arm coordinates means writing these coordinates into the calibration data structure and assigning them an identifier attribute as the final robot arm coordinates. This ensures that they are referenced with this unique name in subsequent data pairing and dataset saving, thus avoiding confusion with intermediate variables such as the first robot arm coordinates and the target robot arm coordinates. The record of the final robot arm coordinates can include the coordinate value itself and a timestamp or sequence number field associated with that coordinate, used to maintain the traceability of data items in the calibration dataset. However, the core of recording the action is still to determine the current robot arm coordinates as the final robot arm coordinates.

[0087] Pairing the final robot coordinates with the initial pixel coordinates of the target feature point establishes a single-point binding relationship across coordinate domains, making the initial pixel coordinates in the image coordinate system and the final robot coordinates in the robot coordinate system the two endpoints of the same calibration sample. The initial pixel coordinates originate from the image position record of the target feature point at the start of the alignment iteration, while the final robot coordinates originate from the robot position record at the moment the preset alignment conditions are met. Both reflect the observation reference of the same target feature point in the image domain and its alignment position in the robot domain, respectively. Data pairing requires a clearly defined field structure, including at least the initial pixel coordinate field and the final robot coordinate field, organized as a single record, so that paired data can be directly obtained when reading the calibration dataset later. Establishing a set of correspondences means managing the paired data as an indivisible data unit, avoiding the loss of association due to separate storage of the initial pixel coordinates and the final robot coordinates, and also avoiding cross-pairing of coordinates of different target feature points in multi-point calibration scenarios.

[0088] Saving a set of mappings to a calibration dataset means writing those mappings into a persistent data container, allowing them to be repeatedly accessed across sessions and execution cycles. The calibration dataset can be a structured table, a file-based record, or a database entry. The key is its ability to store multiple mappings at the record level and support append-only writes. The save operation must include integrity checks during the write process, at least verifying the integrity of the mapping fields and returning a successful write status to prevent incomplete records where only initial pixel coordinates or only final robot coordinates are written. The organization of the calibration dataset needs to support subsequent traversal of records to extract the initial pixel coordinates and final robot coordinates. Therefore, consistent field naming and data format should be maintained during saving to ensure a unified data interface for each mapping within the same calibration dataset.

[0089] This embodiment terminates the movement control and probing operation of the robot arm when the preset alignment conditions are met, stabilizing the robot arm's position at the stationary state at the alignment moment. Then, the current robot arm coordinates are read and recorded as the final robot arm coordinates. The final robot arm coordinates are paired with the initial pixel coordinates of the target feature points to establish a set of correspondences, and the set of correspondences is saved to the calibration dataset. This allows each target feature point to form a persistent pair of data records after the alignment conditions are met, reducing coordinate drift and pairing misalignment caused by continued movement or inconsistent recording times, and improving the availability and consistency of the correspondence data.

[0090] In one embodiment, step S80 above includes: S801, determine the traversal order of the remaining feature points among the plurality of feature points, excluding the target feature point; S802, according to the traversal order, select one from the remaining feature points as the current target feature point; S803, for the current target feature point, with the current position of the robot arm as the reference, the following operations are performed: starting from selecting the current target feature point, recording the first robot arm coordinates and the initial pixel coordinates, moving and recording trial information, determining the target robot arm coordinates and step size adjustment method, performing iterative judgment and movement until the alignment conditions are met, and finally recording the final robot arm coordinates and establishing a correspondence with the initial pixel coordinates; S804, Collect a set of correspondences established after each execution of the operation; S805, repeat the selection, execution and collection steps until all remaining feature points have been processed and multiple sets of correspondences are obtained; S806, for each of the multiple sets of correspondences, extract the initial pixel coordinates and the final robot arm coordinates from the correspondence; S807, based on the extracted multiple sets of initial pixel coordinates and final robot coordinates, solves the transformation parameters from the image coordinate system to the robot coordinate system; S808, using the transformation parameters, construct the hand-eye transformation relationship.

[0091] In this embodiment, multiple feature points are derived from feature point recognition results of the calibration board image. The target feature point has already completed one alignment and formed a set of correspondences. Performing the same alignment operation on each feature point other than the target feature point means reusing the closed-loop process of single-point alignment point by point on the remaining feature point set, thereby expanding the single correspondence into multiple sets of correspondences. The same alignment operation here refers to the same set of action chains, including recording the coordinates of the first robot arm and the initial pixel coordinates, moving and recording trial information, determining the coordinates of the target robot arm and the step size adjustment method, judging whether the target feature point and the center of the camera's field of view meet the preset alignment conditions, and recording the final robot arm coordinates and establishing a correspondence with the initial pixel coordinates when the preset alignment conditions are met. The reuse of the same alignment operation requires that each round of processing maintain the same naming system and the same data field organization, so that the subsequent solution stage can read multiple sets of correspondences in a consistent format.

[0092] The purpose of determining the traversal order of the remaining feature points (excluding the target feature point) among multiple feature points is to constrain the processing order, avoid omitting or repeatedly selecting the same feature point, and ensure that the "current target feature point" has a unique orientation in each alignment process. The traversal order can be generated by pixel coordinate sorting rules, such as combining and sorting by horizontal and vertical coordinates in the image coordinate system from smallest to largest, or by the grid row and column index order of feature points on the calibration board, or by the pixel distance between the feature points and the camera's field of view center from closest to farthest. Once the order is generated, a stable index mapping needs to be maintained so that the set of "remaining feature points" can accurately eliminate processed feature points in each loop and locate unprocessed feature points in the next selection action according to the traversal order.

[0093] Selecting one feature point from the remaining feature points sequentially as the current target feature point means breaking down multi-point processing into several rounds of single-point processing, with each round using the current target feature point as the unique alignment object. The selection action requires writing the pixel coordinates and feature point identifier of the current target feature point into the runtime context, ensuring that subsequent recording of initial pixel coordinates, identification of trial pixel coordinates, and determination of whether preset alignment conditions are met all revolve around the same current target feature point. To avoid target drift during alignment, the referencing method of the current target feature point needs to be fixed after the selection action, for example, by using a feature point number or pixel coordinate index to lock the current target feature point, thus ensuring that the pixel coordinates recorded in the trial information all correspond to the same current target feature point.

[0094] The meaning of performing alignment operations on the current target feature point based on the current position of the robot arm is to use the current position of the robot arm as the source of the first robot arm coordinates, so that the current alignment round completes local search and convergence from the current physical state. The processing based on the current position of the robot arm requires first reading the robot arm coordinates and recording them as the first robot arm coordinates, and simultaneously reading the pixel coordinates of the current target feature point in the image and recording them as the initial pixel coordinates. Then, multi-directional fixed-step movement is performed around the first robot arm coordinates. After the movement is completed, trial information is recorded. The trial information includes at least the correlation between the trial robot arm coordinates and the trial pixel coordinates. The trial pixel coordinates come from the position recognition of the current target feature point in the corresponding image frame. The meaning of determining the target robot arm coordinates and the corresponding step size adjustment method based on the trial information, the initial pixel coordinates, and the camera's field of view center is to select the robot arm coordinates that minimize the pixel distance from the candidate coordinate set as the target robot arm coordinates, and use the branch rule corresponding to the minimum pixel distance to generate the step size adjustment method, thereby driving subsequent iterations. Determining whether the target feature point and the camera's field of view meet the preset alignment conditions means judging the convergence state. If the condition is not met, the first robot coordinates are updated based on the target robot coordinates, and the fixed step size is updated according to the step size adjustment method. The movement is repeated, and the trial information is recorded. The target robot coordinates and the corresponding step size adjustment method are determined. The alignment condition is judged again in a loop until the condition is met. After the condition is met, the current robot coordinates are recorded as the final robot coordinates, and a correspondence is established between the final robot coordinates and the initial pixel coordinates. This solidifies the image observation of the current target feature point and the robot alignment position into paired data that can be used for solving.

[0095] The meaning of collecting a set of correspondences established after each alignment operation is to summarize the initial pixel coordinates and final robot coordinates obtained from each round of processing into the same container in the form of records, and maintain their association with the current target feature points. The collection action needs to constrain the writing timing to after the final robot coordinates are confirmed, and maintain the consistency of the field structure during writing to ensure that multiple sets of correspondences have a unified data item format in the calibration dataset. Repeatedly selecting, executing, and collecting until all remaining feature points have been processed means driving the loop termination condition with the traversal order. When the set of remaining feature points is empty, the collection stops and multiple sets of correspondences are obtained. The key to this loop is to update the state of the remaining feature point set after each set of correspondences is completed, ensuring that the traversal order and termination judgment are consistent.

[0096] Extracting the initial pixel coordinates and final robot arm coordinates from each of multiple correspondences means reading the input data used for solving the problem from the calibration dataset record by record, forming two sets of data sequences of the same length. The initial pixel coordinates belong to the image coordinate system, and the final robot arm coordinates belong to the robot arm coordinate system; the two correspond one-to-one within each record. The extraction process must maintain the pairing order, ensuring that the first initial pixel coordinate and the first final robot arm coordinate come from the same correspondence, avoiding parameter offsets caused by cross-pairing in subsequent solutions.

[0097] The meaning of solving the transformation parameters from the image coordinate system to the robot coordinate system based on the extracted initial pixel coordinates and final robot coordinates is to construct a cross-coordinate system mapping model, so that points in the image coordinate system can be mapped to points in the robot coordinate system through the transformation parameters. The transformation parameters can be in the form of an affine model, a rigid body model including rotation and translation, or an extended linear model with scale and non-orthogonal terms. The solution process requires organizing the multiple sets of initial pixel coordinates and final robot coordinates into equation constraints, and fitting the transformation parameters through an error metric to minimize the overall residual between the mapped predicted robot coordinates and the corresponding final robot coordinates. To improve the stability of the fitting, consistency screening can be performed on multiple sets of correspondences during the solution stage. For example, outlier records can be removed based on a residual threshold, and the transformation parameters can be re-solved for the remaining records, reducing the sensitivity of the transformation parameters to abnormal pairings.

[0098] The meaning of constructing hand-eye transformation relationships using transformation parameters is to encapsulate the transformation parameters into a callable mapping relationship expression, enabling pixels in any subsequent image coordinate system to be converted into target pose or target position in the robot coordinate system through the hand-eye transformation relationship. Constructing the action requires clearly defined input and output interfaces: the input is the pixel coordinates in the image coordinate system, and the output is the robot coordinates in the robot coordinate system. The transformation parameters are bound to this interface as the same relation body, thus forming a stable and reusable hand-eye transformation relationship. To ensure consistency in invocation, the storage format of the hand-eye transformation relationship needs to be managed separately from the calibration dataset. The calibration dataset retains multiple sets of correspondences for verification and recalculation, while the hand-eye transformation relationship retains the transformation parameters for online invocation and deployment.

[0099] The hand-eye transformation relationship is obtained by solving multiple sets of initial pixel coordinates and final robot arm coordinates. Essentially, it establishes a stable mapping between the image coordinate system and the robot arm coordinate system. This mapping is not limited to the calibration stage; it can also serve as a fundamental correlation between image perception results and robot arm motion control in subsequent operation stages.

[0100] During actual operation or repetitive tasks, the camera acquires new images and identifies the pixel coordinates of the object to be processed within the image. These pixel coordinates are located in the same image coordinate system as during the calibration phase. By invoking the established hand-eye transformation relationship, the pixel coordinates are input into the transformation parameters corresponding to the transformation relationship, resulting in the robot's coordinates in the corresponding robot coordinate system. These robot coordinates can be directly used as the robot's motion target coordinates, driving the robot to move from its current position to a spatial position corresponding to the target position in the image, thereby achieving automatic conversion between visual perception results and robot motion commands.

[0101] In scenarios requiring further improvements in positioning accuracy, the robot's coordinates output by the hand-eye transformation can also serve as the starting reference position for the alignment process. The robot first moves to this coordinate, then combines it with real-time camera feedback to obtain new pixel coordinates. The distance between the pixel coordinates and the camera's field of view center is used to determine whether the preset alignment conditions are met. If the preset alignment conditions are not met, this position is used as the first robot coordinate to continue multi-directional probing and iterative updates with fixed step sizes. This allows the alignment process to revolve around a position closer to the real target, thereby reducing the probing range and the number of iterations.

[0102] In continuous operations involving multiple targets or feature points, the hand-eye transformation relationship can be repeatedly invoked. The same mapping operation is performed on each newly identified pixel coordinate to obtain the corresponding robot arm coordinate sequence. The robot arm then sequentially completes the positioning or operation task according to this sequence. If changes occur in ambient lighting, installation posture, or mechanical structure during long-term operation, causing mapping errors to gradually accumulate, new initial pixel coordinates and final robot arm coordinates can be re-acquired, and multiple sets of correspondences can be updated. The transformation parameters are then recalculated to update the hand-eye transformation relationship, ensuring that the mapping between the image coordinate system and the robot arm coordinate system remains consistent with the actual state.

[0103] This embodiment performs the same alignment operation on each feature point (excluding the target feature point) among multiple feature points, generating and collecting a set of correspondences point by point until multiple sets of correspondences are formed. Then, the initial pixel coordinates and the final robot coordinates are extracted from the multiple sets of correspondences, and the transformation parameters from the image coordinate system to the robot coordinate system are solved. The transformation parameters are then used to construct the hand-eye transformation relationship, so that the mapping relationship is established on the basis of multi-point constraints and supported by a unified data pairing format. This reduces the impact of single-point pairing errors on the mapping results, improves the stability and consistency of the transformation parameter fitting, and enhances the applicability of the hand-eye transformation relationship under different feature point position distributions.

[0104] In one embodiment, an automatic hand-eye calibration device for a robotic arm is provided, which corresponds one-to-one with the automatic hand-eye calibration method for a robotic arm described in the above embodiments. (Refer to...) Figure 3 , Figure 3This is a schematic diagram of the functional modules of a preferred embodiment of the automatic hand-eye calibration device for a robotic arm according to the present invention. The modules include: image feature acquisition module 10, target point initialization module 20, step size trial control module 30, target coordinate determination module 40, alignment condition determination module 50, iterative update control module 60, single-point correspondence generation module 70, and hand-eye transformation calculation module 80. Detailed descriptions of each functional module are as follows: The image feature acquisition module 10 is used to place the calibration board within the camera's field of view, acquire an image of the calibration board, identify multiple feature points in the image, and acquire the position of the camera's field of view center in the image. The target point initialization module 20 is used to select one feature point from the plurality of feature points as the target feature point, and to record the current first robot arm coordinates and the initial pixel coordinates of the target feature point; The step length probing control module 30 is used to control the robot to move a fixed step length in multiple directions based on the first robot coordinates. After each movement, it records the probing information related to the pixel coordinates of the target feature point and the robot coordinates, and controls the robot to return to the first robot coordinates and continue moving in the next direction. The target coordinate determination module 40 is used to determine the target robot coordinates and the corresponding step size adjustment method based on the trial information, the initial pixel coordinates and the camera field of view center; The alignment condition judgment module 50 is used to determine whether the target feature point and the center of the camera's field of view meet the preset alignment conditions. The iterative update control module 60 is used to update the first robot coordinates according to the target robot coordinates if the preset alignment conditions are not met, and update the fixed step size according to the step size adjustment method. The steps of moving and recording trial information, determining the target robot coordinates and the corresponding step size adjustment method, and judging the alignment conditions are repeated with the updated fixed step size until the preset alignment conditions are met. The single-point correspondence generation module 70 is used to record the current robot arm coordinates as the final robot arm coordinates if the preset alignment conditions are met, and to establish a correspondence between the final robot arm coordinates and the initial pixel coordinates. The hand-eye transformation calculation module 80 is used to perform the same alignment operation on each feature point other than the target feature point among the plurality of feature points to obtain multiple sets of correspondences, and to calculate the hand-eye transformation relationship based on the multiple sets of correspondences.

[0105] Specific limitations regarding the automatic hand-eye calibration device for robotic arms can be found in the aforementioned limitations on the automatic hand-eye calibration method for robotic arms, and will not be repeated here. Each module in the aforementioned automatic hand-eye calibration device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0106] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a robotic hand automatic hand-eye calibration method on the server side.

[0107] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements the functions or steps of a robotic hand automatic hand-eye calibration method on the client side.

[0108] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Place the calibration board within the camera's field of view, acquire an image of the calibration board, identify multiple feature points in the image, and obtain the position of the camera's field of view center in the image. Select one feature point from the plurality of feature points as the target feature point, and record the current first manipulator coordinates and the initial pixel coordinates of the target feature point; Using the first robotic arm coordinates as a reference, the robotic arm is controlled to move in multiple directions by a fixed step length. After each movement, the trial information related to the pixel coordinates of the target feature point and the robotic arm coordinates is recorded, and the robotic arm is controlled to return to the first robotic arm coordinates and continue to move in the next direction. Based on the trial information, the initial pixel coordinates, and the camera field of view center, the target robot's coordinates and the corresponding step size adjustment method are determined. Determine whether the target feature point and the center of the camera's field of view meet preset alignment conditions; If the preset alignment conditions are not met, the first robot coordinates are updated according to the target robot coordinates, and the fixed step size is updated according to the step size adjustment method. The steps of moving and recording trial information, determining the target robot coordinates and the corresponding step size adjustment method, and judging the alignment conditions are repeated with the updated fixed step size until the preset alignment conditions are met. If the preset alignment conditions are met, the current robot arm coordinates are recorded as the final robot arm coordinates, and a correspondence is established between the final robot arm coordinates and the initial pixel coordinates. For each feature point other than the target feature point among the plurality of feature points, the same alignment operation is performed to obtain multiple sets of correspondences, and the hand-eye transformation relationship is calculated based on the multiple sets of correspondences.

[0109] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, and a computer program is stored thereon, which, when executed by a processor, performs the following steps: Place the calibration board within the camera's field of view, acquire an image of the calibration board, identify multiple feature points in the image, and obtain the position of the camera's field of view center in the image. Select one feature point from the plurality of feature points as the target feature point, and record the current first manipulator coordinates and the initial pixel coordinates of the target feature point; Using the first robotic arm coordinates as a reference, the robotic arm is controlled to move in multiple directions by a fixed step length. After each movement, the trial information related to the pixel coordinates of the target feature point and the robotic arm coordinates is recorded, and the robotic arm is controlled to return to the first robotic arm coordinates and continue to move in the next direction. Based on the trial information, the initial pixel coordinates, and the camera field of view center, the target robot's coordinates and the corresponding step size adjustment method are determined. Determine whether the target feature point and the center of the camera's field of view meet preset alignment conditions; If the preset alignment conditions are not met, the first robot coordinates are updated according to the target robot coordinates, and the fixed step size is updated according to the step size adjustment method. The steps of moving and recording trial information, determining the target robot coordinates and the corresponding step size adjustment method, and judging the alignment conditions are repeated with the updated fixed step size until the preset alignment conditions are met. If the preset alignment conditions are met, the current robot arm coordinates are recorded as the final robot arm coordinates, and a correspondence is established between the final robot arm coordinates and the initial pixel coordinates. For each feature point other than the target feature point among the plurality of feature points, the same alignment operation is performed to obtain multiple sets of correspondences, and the hand-eye transformation relationship is calculated based on the multiple sets of correspondences.

[0110] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

Claims

1. A method for automatic hand-eye calibration of a robotic arm, characterized in that, Includes the following steps: Place the calibration board within the camera's field of view, acquire an image of the calibration board, identify multiple feature points in the image, and obtain the position of the camera's field of view center in the image. Select one feature point from the plurality of feature points as the target feature point, and record the current first manipulator coordinates and the initial pixel coordinates of the target feature point; Using the first robotic arm coordinates as a reference, the robotic arm is controlled to move in multiple directions by a fixed step length. After each movement, the trial information related to the pixel coordinates of the target feature point and the robotic arm coordinates is recorded, and the robotic arm is controlled to return to the first robotic arm coordinates and continue to move in the next direction. Based on the trial information, the initial pixel coordinates, and the camera field of view center, the target robot's coordinates and the corresponding step size adjustment method are determined. Determine whether the target feature point and the center of the camera's field of view meet preset alignment conditions; If the preset alignment conditions are not met, the first robot coordinates are updated according to the target robot coordinates, and the fixed step size is updated according to the step size adjustment method. The steps of moving and recording trial information, determining the target robot coordinates and the corresponding step size adjustment method, and judging the alignment conditions are repeated with the updated fixed step size until the preset alignment conditions are met. If the preset alignment conditions are met, the current robot arm coordinates are recorded as the final robot arm coordinates, and a correspondence is established between the final robot arm coordinates and the initial pixel coordinates. For each feature point other than the target feature point among the plurality of feature points, the same alignment operation is performed to obtain multiple sets of correspondences, and the hand-eye transformation relationship is calculated based on the multiple sets of correspondences.

2. The automatic hand-eye calibration method for a robotic arm as described in claim 1, characterized in that, The calibration board is placed within the camera's field of view, an image of the calibration board is acquired, multiple feature points in the image are identified, and the position of the camera's field of view center in the image is obtained, including: Control the rotation of each joint of the robotic arm and adjust the relative angle between the upper arm and forearm of the robotic arm to avoid the robotic arm from being in a strange position of being fully extended or fully folded, so as to keep the robotic arm in a non-singular posture. Place the calibration board within the camera's field of view, control the camera to acquire raw image data containing the calibration board, preprocess the raw image data to obtain an image of the calibration board; Multiple feature points are identified from the image using a corner detection algorithm, and the pixel coordinates of each feature point are extracted. Based on the camera's imaging parameters, determine the coordinate position of the camera's field of view center in the image coordinate system.

3. The automatic hand-eye calibration method for a robotic arm as described in claim 1, characterized in that, Using the first robotic arm coordinates as a reference, the robotic arm is controlled to move in fixed steps along multiple directions. After each movement, trial information related to the pixel coordinates of the target feature point and the robotic arm coordinates is recorded. The robotic arm is then controlled to return to the first robotic arm coordinates and continue moving in the next direction. This includes: Multiple movement directions are defined, including the positive horizontal axis, negative horizontal axis, positive vertical axis, and negative vertical axis in the robot's planar coordinate system. For each direction of movement, the robot arm is controlled to start from the first robot arm coordinates, move a fixed step distance along the direction of movement, and keep the robot arm stationary; The robot arm coordinates after movement are recorded as the probe robot arm coordinates, and the position of the target feature point in the current image is identified as the probe pixel coordinates. The association between the probe manipulator coordinates and the probe pixel coordinates is recorded as probe information; Control the robotic arm to return to the first robotic arm coordinate and continue moving in the next direction until all trial operations in all directions have been completed.

4. The automatic hand-eye calibration method for a robotic arm as described in claim 1, characterized in that, Based on the trial information, the initial pixel coordinates, and the camera's field of view center, the target robot's coordinates and the corresponding step size adjustment method are determined, including: Calculate the pixel distance between the initial pixel coordinates and the center of the camera's field of view, and mark the calculated distance value as the first pixel distance; Traverse the trial information, extract all trial pixel coordinates contained in the trial information, calculate the pixel distance between each trial pixel coordinate and the center of the camera's field of view, and mark the calculated distance value as the trial pixel distance; The first pixel distance is compared with the numerical values ​​of all the trial pixel distances, and the minimum pixel distance with the smallest value is selected. If the minimum pixel distance is the first pixel distance, then the target robot coordinates are determined to be the first robot coordinates, and the corresponding step size adjustment method is determined to be reducing the fixed step size; If the minimum pixel distance is a certain trial pixel distance, then the trial robot coordinates corresponding to the trial pixel distance are obtained from the trial information, the target robot coordinates are determined to be the trial robot coordinates, and the corresponding step size adjustment method is determined to keep the fixed step size unchanged.

5. The automatic hand-eye calibration method for a robotic arm as described in claim 1, characterized in that, If the preset alignment condition is not met, the first robot arm coordinates are updated according to the target robot arm coordinates, and the fixed step size is updated according to the step size adjustment method. The steps of moving and recording trial information, determining the target robot arm coordinates and the corresponding step size adjustment method, and judging the alignment condition are repeated with the updated fixed step size until the preset alignment condition is met, including: If the preset alignment conditions are not met, then compare whether the coordinates of the target robot arm are the same as the coordinates of the first robot arm; If the coordinates of the target robot are the same as the coordinates of the first robot, then the position of the robot remains unchanged; If the target robot coordinates are different from the first robot coordinates, then control the robot to move to the target robot coordinates and update the target robot coordinates to the new first robot coordinates; If the step size adjustment method is to decrease, then the fixed step size is reduced by a preset ratio; if the step size adjustment method is to keep it unchanged, then the fixed step size is kept unchanged. Using the current first robot arm coordinates as a reference, and with the current fixed step size, repeatedly execute the steps of moving and recording trial information, determining the target robot arm coordinates and the corresponding step size adjustment method, and judging the alignment conditions, until the preset alignment conditions are met.

6. The automatic hand-eye calibration method for a robotic arm as described in claim 1, characterized in that, If the preset alignment conditions are met, the current robot arm coordinates are recorded as the final robot arm coordinates, and a correspondence is established between the final robot arm coordinates and the initial pixel coordinates, including: If the preset alignment conditions are met, the movement control and probing operation of the robotic arm will be terminated. Read the current robot arm coordinates and record the current robot arm coordinates as the final robot arm coordinates; The final robot arm coordinates are paired with the initial pixel coordinates of the target feature points to establish a correspondence. Save the set of correspondences to the calibration dataset.

7. The automatic hand-eye calibration method for a robotic arm as described in claim 1, characterized in that, For each feature point other than the target feature point among the plurality of feature points, the same alignment operation is performed to obtain multiple sets of correspondences. Based on the multiple sets of correspondences, the hand-eye transformation relationship is calculated, including: Determine the traversal order of the remaining feature points among the plurality of feature points, excluding the target feature point; According to the traversal order, one of the remaining feature points is selected as the current target feature point in turn; For the current target feature point, with the current position of the robot arm as the reference, the following operations are performed: starting from selecting the current target feature point, recording the first robot arm coordinates and the initial pixel coordinates, moving and recording trial information, determining the target robot arm coordinates and step size adjustment method, performing iterative judgment and movement until the alignment conditions are met, and finally recording the final robot arm coordinates and establishing a correspondence with the initial pixel coordinates; Collect a set of correspondences established after each execution of the operation; Repeat the selection, execution, and collection steps until all remaining feature points have been processed and multiple sets of correspondences are obtained. For each of the multiple sets of correspondences, extract the initial pixel coordinates and the final robot arm coordinates from the correspondence; Based on the extracted initial pixel coordinates and final robot coordinates, the transformation parameters from the image coordinate system to the robot coordinate system are solved. Using the aforementioned transformation parameters, a hand-eye transformation relationship is constructed.

8. An automatic hand-eye calibration device for a robotic arm, characterized in that, The robotic arm automatic hand-eye calibration device includes: The image feature acquisition module is used to place the calibration board within the camera's field of view, acquire an image of the calibration board, identify multiple feature points in the image, and acquire the position of the camera's field of view center in the image. The target point initialization module is used to select one feature point from the plurality of feature points as the target feature point, and to record the current first manipulator coordinates and the initial pixel coordinates of the target feature point. The step length probing control module is used to control the robot to move a fixed step length in multiple directions based on the coordinates of the first robot. After each movement, it records the probing information related to the pixel coordinates of the target feature point and the robot coordinates, and controls the robot to return to the first robot coordinates and continue moving in the next direction. The target coordinate determination module is used to determine the target robot coordinates and the corresponding step size adjustment method based on the trial information, the initial pixel coordinates, and the camera field of view center. The alignment condition determination module is used to determine whether the target feature point and the center of the camera's field of view meet the preset alignment conditions; The iterative update control module is used to update the first robot coordinates according to the target robot coordinates if the preset alignment conditions are not met, and update the fixed step size according to the step size adjustment method. The steps of moving and recording trial information, determining the target robot coordinates and the corresponding step size adjustment method, and judging the alignment conditions are repeated with the updated fixed step size until the preset alignment conditions are met. The single-point correspondence generation module is used to record the current robot arm coordinates as the final robot arm coordinates if the preset alignment conditions are met, and to establish a correspondence between the final robot arm coordinates and the initial pixel coordinates. The hand-eye transformation calculation module is used to perform the same alignment operation on each feature point other than the target feature point among the multiple feature points to obtain multiple sets of correspondences, and calculate the hand-eye transformation relationship based on the multiple sets of correspondences.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a robotic hand-eye automatic calibration program stored in the memory and executable on the processor. When the robotic hand-eye automatic calibration program is executed by the processor, it implements the steps of the robotic hand-eye automatic calibration method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores an automatic hand-eye calibration program for a robotic arm, which, when executed by a processor, implements the steps of the automatic hand-eye calibration method for a robotic arm as described in any one of claims 1-7.

Images