Mechanical hand eye automatic calibration method, device, equipment and medium

Abstract

The application discloses a kind of mechanical hand eye automatic calibration method, device, equipment and medium, comprising: obtaining calibration board image and identifying feature point and camera field of view center, select target feature point and record first mechanical hand coordinates and initial pixel coordinates;Control mechanical hand moves fixed step along multiple directions and obtains tentative information, and determine target mechanical hand coordinates and step length adjustment mode accordingly;According to iterative update mechanical hand coordinates and fixed step according to alignment condition, establish the corresponding relationship of final mechanical hand coordinates and initial pixel coordinates;Repeat the above operation to the rest feature points, calculate hand-eye transformation relationship based on multiple sets of corresponding relationship.The application gradually corrects the position of mechanical hand by the iterative alignment mode driven by pixel error, and realizes stable convergence by combining step length adjustment, so as to reduce the point error of artificial, improve calibration accuracy, reduce repeated adjustment process, and improve calibration efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robots and machine vision, and in particular to a mechanical hand eye automatic calibration method, device, equipment and medium. BACKGROUND

[0002] In the existing industrial assembly and automated production scene, the SCARA locking mechanical hand usually needs to establish the corresponding relationship between the camera coordinate system and the mechanical hand coordinate system through hand-eye calibration to ensure the spatial consistency between the locking position and the target station. However, the current engineering practice still adopts a calibration method mainly relying on manual participation, which depends on the operator to complete the matching of the mechanical hand position and the feature position in the image by manually pointing. This method has obvious limitations in terms of automation degree and calibration accuracy.

[0003] On the one hand, the existing manual pointing calibration method mainly relies on the subjective judgment of the operator on the target position in the camera picture. Since the pixel coordinates in the image do not have intuitive spatial meaning, the human eye cannot accurately perceive the pixel-level position difference, and the operator can only move the mechanical hand to an approximate position near the target area, but cannot accurately align the target point in the image. This experience-dependent and visual method inevitably introduces random errors in the calibration results, especially in the locking application with high position accuracy requirements, which can easily lead to limited final operation accuracy.

[0004] On the other hand, the existing calibration process generally lacks effective error convergence mechanism. When the one-time pointing result does not reach the ideal state, manual adjustment and re-calibration need to be repeated, and the optimal position is gradually approached through multiple trial and error. This repeated operation not only significantly increases the time cost of calibration, but also makes the calibration efficiency seriously dependent on the proficiency of the operator, making it difficult to maintain consistent calibration quality in different devices or different batches. SUMMARY

[0005] The main purpose of the present application is to provide a mechanical hand eye automatic calibration method, device, equipment and storage medium, which aims to solve the technical problems that the existing locking mechanical hand calibration method mainly relies on manual visual approximation pointing, lacks automatic alignment and iterative convergence mechanism based on pixel error, and the accuracy of the calibration process is difficult to guarantee and the overall efficiency is low.

[0006] To achieve the above purpose, the present application provides a mechanical hand eye automatic calibration method, comprising:

[0007] placing a calibration board in the camera field of view, acquiring an image of the calibration board, identifying a plurality of feature points in the image and acquiring the position of the camera field of view center in the image;

[0008] select one feature point from the plurality of feature points as a target feature point, record a current first robot coordinate and an initial pixel coordinate of the target feature point;

[0009] take the first robot coordinate as a reference, control the robot to move a fixed step length in multiple directions, record the pixel coordinate of the target feature point and the robot coordinate related to the trial information after each movement, and control the robot to return to the first robot coordinate and continue the movement in the next direction;

[0010] determine the target robot coordinate and the corresponding step length adjustment mode based on the trial information, the initial pixel coordinate and the center of the camera field of view;

[0011] determine whether the target feature point and the center of the camera field of view satisfy a preset alignment condition;

[0012] If the preset alignment condition is not satisfied, update the first robot coordinate according to the target robot coordinate, update the fixed step length according to the step length adjustment mode, repeat the steps of moving and recording trial information, determining the target robot coordinate and the corresponding step length adjustment mode, and judging the alignment condition with the updated fixed step length until the preset alignment condition is satisfied;

[0013] If the preset alignment condition is satisfied, record the current robot coordinate as a final robot coordinate, and establish the final robot coordinate and the initial pixel coordinate as a corresponding relationship;

[0014] For each feature point in the plurality of feature points except the target feature point, perform the same alignment operation to obtain a plurality of corresponding relationships, and calculate a hand-eye transformation relationship based on the plurality of corresponding relationships.

[0015] Further, in order to achieve the above-mentioned purpose, the present application provides a robot hand-eye automatic calibration device, comprising:

[0016] An image feature acquisition module is configured to place a calibration board in a camera field of view range, acquire an image of the calibration board, identify a plurality of feature points in the image and acquire a position of a center of the camera field of view in the image;

[0017] A target point initialization module is configured to select one feature point from the plurality of feature points as a target feature point, record a current first robot coordinate and an initial pixel coordinate of the target feature point;

[0018] A step length trial control module is configured to control the robot to move a fixed step length in multiple directions based on the first robot coordinates, record trial information related to the pixel coordinates of the target feature point and the robot coordinates after each movement, and control the robot to return to the first robot coordinates and continue movement in the next direction;

[0019] A target coordinate determination module is configured to determine target robot coordinates and a corresponding step length adjustment mode based on the trial information, the initial pixel coordinates, and the center of the camera field of view.

[0020] An alignment condition determination module is configured to determine whether the target feature point and the center of the camera field of view satisfy a preset alignment condition.

[0021] An iterative update control module is configured to update the first robot coordinates according to the target robot coordinates if the preset alignment condition is not satisfied, and update the fixed step length according to the step length adjustment mode, so as to repeat the steps of moving and recording trial information, determining target robot coordinates and a corresponding step length adjustment mode, and determining an alignment condition with the updated fixed step length until the preset alignment condition is satisfied.

[0022] A single-point correspondence generation module is configured to record the current robot coordinates as final robot coordinates if the preset alignment condition is satisfied, and establish the final robot coordinates and the initial pixel coordinates as a set of correspondence.

[0023] A hand-eye transformation calculation module is configured to perform the same alignment operation on each feature point in the plurality of feature points except the target feature point to obtain a plurality of sets of correspondence, and calculate a hand-eye transformation relationship based on the plurality of sets of correspondence.

[0024] Further, to achieve the above object, the present application also provides a computer device, which comprises a memory, a processor, and a robot hand-eye automatic calibration program stored in the memory and executable on the processor, and the robot hand-eye automatic calibration program, when executed by the processor, implements the steps of the robot hand-eye automatic calibration method as described above.

[0025] Further, to achieve the above object, the present application also provides a computer readable storage medium, which stores a robot hand-eye automatic calibration program, and the robot hand-eye automatic calibration program, when executed by a processor, implements the steps of the robot hand-eye automatic calibration method as described above.

[0026] Beneficial effects: The application discloses a mechanical hand eye automatic calibration method, device, equipment and medium, comprising: acquiring a calibration board image and identifying feature points and a camera visual field center, selecting target feature points and recording first mechanical hand coordinates and initial pixel coordinates; controlling the mechanical hand to move a fixed step length in multiple directions to acquire trial information, and determining target mechanical hand coordinates and step length adjustment modes according to the trial information; iteratively updating the mechanical hand coordinates and the fixed step length according to alignment conditions, and establishing a corresponding relationship between final mechanical hand coordinates and initial pixel coordinates; repeating the above operation on the remaining feature points, and calculating a hand-eye transformation relationship based on multiple sets of corresponding relationships. The application gradually corrects the mechanical hand position through the pixel error driven iterative alignment mode, and realizes stable convergence in combination with step length adjustment, so that the manual point error is reduced, the calibration accuracy is improved, the repeated adjustment process is reduced, and the calibration efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0027] The application will be further described below in combination with the drawings and embodiments, wherein:

[0028] Figure 1 An application environment schematic diagram of the mechanical hand eye automatic calibration method in an embodiment of the application;

[0029] Figure 2 A flowchart of the mechanical hand eye automatic calibration method in an embodiment of the application;

[0030] Figure 3 A functional module schematic diagram of the mechanical hand eye automatic calibration device in a preferred embodiment of the application;

[0031] Figure 4 A structure schematic diagram of the computer equipment in an embodiment of the application;

[0032] Figure 5 Another structure schematic diagram of the computer equipment in an embodiment of the application. DETAILED DESCRIPTION

[0033] It should be understood that the specific embodiments described herein are merely intended to explain the application, and are not intended to limit the application.

[0034] The mechanical hand eye automatic calibration method provided by the embodiments of the application can be applied to, for example, Figure 1In the application environment, the client communicates with the server through the network. The server can obtain the calibration board image through the client, identify the feature points and the camera field of view center, select the target feature points and record the first robot coordinates and the initial pixel coordinates; control the robot to move in multiple directions by a fixed step to obtain the trial information, and determine the target robot coordinates and the step adjustment mode according to the trial information; iteratively update the robot coordinates and the fixed step according to the alignment condition, and establish the corresponding relationship between the final robot coordinates and the initial pixel coordinates; repeat the above operation for the remaining feature points, and calculate the hand-eye transformation relationship based on multiple sets of corresponding relationships. The present application gradually corrects the robot position through the pixel error driven iterative alignment method, and realizes stable convergence combined with step adjustment, thereby reducing the manual point error, improving the calibration accuracy, reducing the repeated adjustment process, and improving the calibration efficiency. The client can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The server can be realized by an independent server or a server cluster composed of multiple servers. The present application will be described in detail through specific embodiments.

[0035] Please refer to Figure 2 , Figure 2 The flowchart of an embodiment of the robot hand-eye automatic calibration method provided by the present application is shown. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0036] As Figure 2 shown, the robot hand-eye automatic calibration method provided by the present application includes the following steps:

[0037] S10, place the calibration board in the camera field of view range, obtain the image of the calibration board, identify multiple feature points in the image and obtain the position of the camera field of view center in the image;

[0038] In this embodiment, during the robot hand-eye calibration process, the purpose of placing the calibration board in the camera field of view range is to provide a stable and identifiable visual carrier for image coordinate acquisition. The calibration board is placed at a position where the camera can form a complete image, so that the surface structure is not blocked in the image and the shape is clear, thereby ensuring the effective expression of various spatial information in the subsequent image processing process. The placement relationship is not limited to a fixed height or a fixed angle, but requires that the calibration board forms a complete projection in the imaging area, so that its structural features can be accurately captured by the image acquisition unit.

[0039] Under the condition that the calibration board is located within the camera field of view, the camera carries out image acquisition on the picture containing the calibration board, and forms image data for processing. The image acquisition process aims at the integrity of pixel-level information, so that the edges, intersection points or structural changes of the calibration board are distinguishable in the image, providing a basis for subsequent feature point positioning. The acquisition result of the image data is the input of the subsequent processing, and the quality thereof directly affects the stability of the pixel coordinate extraction.

[0040] After the image is acquired, a plurality of feature points in the image are identified. The identification focuses on extracting pixel points that are stable in spatial position and can be repeatedly positioned in structure from the image. The presence of the plurality of feature points enables the overall spatial distribution of the calibration board to be described in the image without relying on a single position, thereby improving the reliability of the image coordinate information. Each identified feature point corresponds to a pixel position in the image coordinate system, which is used to express the discrete position set of the calibration board in the image.

[0041] At the same time, the position of the center of the camera field of view in the image needs to be acquired, which is used to describe the geometric center relationship of the camera imaging area. The center of the camera field of view is derived from the camera imaging model or the image size relationship, which corresponds to a certain position in the image coordinate system and is used as a unified reference for subsequent pixel position relationship calculation. By specifying the center position, the position relationship of any feature point in the image can be described with the center as a reference, thereby avoiding relying on artificial visual judgment of the image offset degree.

[0042] The present embodiment simultaneously acquires the pixel coordinates of a plurality of feature points and the position of the center of the camera field of view in the image, so that the calibration process has a clear pixel reference basis and avoids relying on artificial visual judgment of the image center or position offset, thereby improving the stability and consistency of the image coordinate acquisition.

[0043] S20, selecting one feature point from the plurality of feature points as a target feature point, recording the current first robot coordinate and the initial pixel coordinate of the target feature point;

[0044] In the present embodiment, on the basis of having identified a plurality of feature points in the image, one feature point needs to be selected as a target feature point for subsequent reference establishment in the alignment process. The purpose of this selection is to determine a clear reference object among the plurality of available pixel positions, so that the subsequent spatial adjustment and position judgment are carried out around the same pixel point. The target feature point is derived from the feature point set obtained by the previous identification, and its position in the image coordinate system is determined and can be repeatedly acquired, so as to serve as a stable reference in the subsequent processing.

[0045] When the target feature point is selected, the current first robot coordinate needs to be recorded. The first robot coordinate is used to describe the initial spatial position state of the robot when the target feature point has not been adjusted by alignment. The coordinate is derived from the robot control system or the position feedback unit, and its recording behavior fixes the spatial position of the robot at the current time in numerical form, providing a basis for subsequent position updating and comparison. The recording of the first robot coordinate does not involve position changes, and only reflects the state of the robot at the time when the target feature point is selected.

[0046] At the same time, the initial pixel coordinate of the target feature point also needs to be recorded. The initial pixel coordinate is used to describe the original position state of the target feature point in the current image, which has not been adjusted by any alignment. The relative relationship between the position and the center of the camera field of view reflects the current initial offset. By recording the initial pixel coordinate, subsequent pixel position changes can be analyzed based on a unified starting reference, thereby avoiding position drift caused by image updating or repeated identification.

[0047] The target feature point, the first robot coordinate and the initial pixel coordinate are recorded at the same time point, so that the image space information and the robot space information form a corresponding relationship in the time dimension, providing a consistent data basis for position association in subsequent processing.

[0048] The embodiment selects a target feature point from a plurality of feature points, and synchronously records the first robot coordinate and the initial pixel coordinate of the target feature point, so that the image coordinate information and the robot spatial position form a stable corresponding relationship at the same time, thereby providing a consistent data starting point for subsequent position adjustment and alignment judgment.

[0049] S30, taking the first robot coordinate as a reference, controlling the robot to move a fixed step length in multiple directions, recording the trial information related to the pixel coordinate of the target feature point and the robot coordinate after each movement, and controlling the robot to return to the first robot coordinate and continue moving in the next direction;

[0050] In the embodiment, on the basis of the determined first robot coordinate and target feature point, the first robot coordinate is taken as a spatial reference to perform a controlled displacement operation on the robot. The first robot coordinate is used to define the center position of the current search area, which functions to provide a unified starting position for multi-directional movement, so that each movement is comparable.

[0051] Moving fixed step length in multiple directions refers to exerting displacement control on the manipulator in different spatial directions respectively while keeping the step length value unchanged. Multiple directions are derived from the definition of the manipulator plane coordinate system or the workspace coordinate system, and the setting enables the manipulator to form discrete spatial sampling positions around the target feature point. The fixed step length is used to limit the amplitude of each displacement, thereby avoiding uncontrollable large movements of the manipulator.

[0052] After each movement is completed and the manipulator position is kept stable, the pixel coordinates related to the target feature point and the current manipulator coordinates need to be recorded. The pixel coordinates reflect the position change of the target feature point in the image, and the manipulator coordinates reflect the corresponding spatial displacement state. Both are recorded and associated at the same time to form the probe information, which is used to describe the spatial-image corresponding result under the condition of a certain direction and a certain step length.

[0053] After completing the movement in one direction and recording the probe information, the manipulator needs to be controlled to return to the first manipulator coordinate. This return operation ensures that each directional movement starts from the same spatial starting point, thereby avoiding interference of path superposition or position drift on the probe result. By continuing to perform the movement in the next direction after returning, the probe information of multiple directions is obtained one by one under the unified reference condition.

[0054] This embodiment obtains probe information in multiple directions with a fixed step length based on the unified first manipulator coordinate as a reference, so that the pixel changes of the target feature point under different spatial displacement conditions have comparability, thereby providing stable and symmetric input data for subsequent position judgment.

[0055] S40, based on the probe information, the initial pixel coordinates and the center of the camera field of view, determining the target manipulator coordinate and the corresponding step length adjustment mode;

[0056] In this embodiment, after completing the recording of probe information in multiple directions, the obtained data needs to be comprehensively analyzed to determine the target manipulator coordinate for guiding subsequent movement and the step length adjustment mode matching therewith. The probe information contains the pixel coordinate change of the target feature point under different manipulator coordinate conditions, and this information reflects the corresponding relationship between the spatial displacement of the manipulator and the position change of the target feature point in the image.

[0057] The initial pixel coordinates, as the image position of the target feature point in the reference state, are used for comparison with the pixel coordinates in each group of probe information. Through comparison, the offset direction and offset amplitude of the target feature point in the image plane can be obtained, thereby judging the error distribution of the current spatial position of the manipulator relative to the target position.

[0058] The camera visual field center represents a reference position in the image coordinate system for alignment, which serves as a unified judgment basis for the offset judgment of the pixel coordinates. By correlating and analyzing the pixel coordinates in the probe information with the camera visual field center, the trend of the target feature point moving towards or away from the camera visual field center in different probe directions can be determined.

[0059] On the basis of the above analysis, the corresponding robot coordinates in the probe information are screened and combined to determine the spatial position that makes the target feature point change towards the camera visual field center in the pixel space as the target robot coordinate. At the same time, according to the corresponding relationship between the pixel offset amplitude and the fixed step length, it is judged whether the current step length meets the convergence requirement, and the step length adjustment mode is generated accordingly to correct the subsequent robot movement amplitude.

[0060] The embodiment jointly analyzes the probe information, the initial pixel coordinates and the camera visual field center, so that the determination of the target robot coordinate is based on the explicit correspondence between the spatial displacement and the pixel change, and the matching step length adjustment mode is generated simultaneously, thereby improving the directionality and convergence stability of the position adjustment.

[0061] S50, judging whether the target feature point and the camera visual field center meet a preset alignment condition;

[0062] In this embodiment, after obtaining the target robot coordinate, the position state of the target feature point in the image needs to be judged to confirm whether the current robot position has reached an acceptable alignment degree. The relationship between the target feature point and the camera visual field center reflects the result of the current spatial position adjustment, which is directly embodied as the deviation state at the pixel coordinate level.

[0063] The preset alignment condition is used to limit the maximum deviation range allowed between the target feature point and the camera visual field center in the image plane, which can be derived from the system calibration accuracy requirement, the robot repeatability accuracy or the minimum resolvable unit corresponding to the image resolution. By numerically comparing the current pixel coordinates corresponding to the target feature point with the camera visual field center, the offset in the horizontal and vertical directions can be calculated, or the comprehensive pixel distance can be calculated based on the offset.

[0064] When the calculation result falls within the threshold range defined by the preset alignment condition, it indicates that the current robot spatial position has formed a stable corresponding relationship with the camera visual field center at the pixel level, and subsequent position correction through the probe information is no longer needed. If it exceeds the range, it means that there is still uneliminated spatial deviation, which needs to be corrected according to the target robot coordinate and the step length adjustment mode determined in the previous sequence.

[0065] The judgment process logically plays a role of convergence determination, which is used to distinguish the intermediate state in the adjustment process from the acceptable termination state, thereby avoiding invalid iteration or premature stop.

[0066] The embodiment makes quantitative judgment on the relationship between the target feature point and the center of the camera field of view at the pixel coordinate level, so that the determination of the alignment state is free from artificial experience dependence, and a clear termination condition is provided for subsequent processing, thereby improving the stability and consistency of the overall adjustment process.

[0067] S60, if the preset alignment condition is not met, updating the first robot coordinate according to the target robot coordinate, and updating the fixed step according to the step adjustment mode, repeating the steps of moving and recording the trial information, determining the target robot coordinate and the corresponding step adjustment mode, and judging the alignment condition with the updated fixed step, until the preset alignment condition is met;

[0068] In the embodiment, when the target feature point and the center of the camera field of view do not meet the preset alignment condition, it indicates that the current robot position has not yet made the deviation at the pixel level converge within the allowable range, and the robot spatial position and the adjustment strategy need to be updated. The target robot coordinate is used to indicate a more optimal spatial position direction based on the analysis result of the trial information, which is derived from the comprehensive judgment of the relationship between the trial robot coordinate and the corresponding pixel deviation.

[0069] The process of updating the first robot coordinate is used to switch the reference position of the current robot to the target robot coordinate, so that the subsequent moving operation is carried out based on a position closer to the alignment direction, thereby avoiding invalid repeated adjustment near the original reference position. The updating operation logically realizes the dynamic migration of the search reference point.

[0070] The step adjustment mode is used to control the change trend of the fixed step, which is derived from the previous pixel deviation comparison result, and is used to reflect whether the current adjustment has approached the optimal area. When the pixel deviation does not improve, reducing the fixed step can improve the resolution of position adjustment; when the pixel deviation is still in the improvable stage, maintaining the fixed step helps to maintain the adjustment efficiency. The update of the fixed step directly affects the amplitude and fineness of the subsequent moving operation.

[0071] After the update of the first robot coordinate and the fixed step is completed, the moving and recording of the trial information, the determination of the target robot coordinate and the corresponding step adjustment mode, and the judgment of the alignment condition are re-executed, so that the whole adjustment process forms a closed-loop iterative structure, until the preset alignment condition is met.

[0072] The embodiment can adaptively balance between efficiency and accuracy in the position adjustment process, avoid invalid search, and gradually guide the pixel deviation to converge by dynamically updating the first robot coordinates and adjusting the fixed step size in coordination with the step size adjustment mode when the preset alignment condition is not met.

[0073] S70, if the preset alignment condition is met, record the current robot coordinates as final robot coordinates, and establish the final robot coordinates and the initial pixel coordinates as a set of corresponding relationships;

[0074] In the embodiment, when the position of the target feature point in the image and the center of the camera field of view meet the preset alignment condition, it indicates that the current robot spatial position has established a stable corresponding relationship with the target position on the pixel level, and continuing to perform position adjustment no longer produces effective error improvement. In this state, the current spatial pose of the robot can be regarded as the convergence result for the target feature point.

[0075] The operation of recording the current robot coordinates is used to solidify this convergence state, and the robot pose at the moment when the alignment condition is met is used as spatial coordinate data that can be directly used for subsequent calibration calculation. The robot coordinates are no longer used as temporary reference points in the adjustment process, but are saved as data results with definite meaning.

[0076] The final robot coordinates and the initial pixel coordinates are established as a set of corresponding relationships, which are used to express the mapping relationship between the position of the target feature point in the image coordinate system and the corresponding spatial position in the robot coordinate system. The initial pixel coordinates are derived from the image position of the feature point before any robot adjustment, and the pairing relationship between the initial pixel coordinates and the final robot coordinates reflects the actual corresponding relationship from the pixel space to the robot space, providing basic input for subsequent spatial transformation calculation based on multiple sets of data.

[0077] The embodiment can make the mapping data between the pixel space and the robot space have clear source and stable meaning by solidifying the robot coordinates and establishing the corresponding relationship with the initial pixel coordinates when the alignment condition is met, thereby avoiding data drift caused by excessive adjustment.

[0078] S80, for each feature point in the plurality of feature points except the target feature point, the same alignment operation is performed to obtain multiple sets of corresponding relationships, and a hand-eye transformation relationship is calculated based on the multiple sets of corresponding relationships.

[0079] In the embodiment, after completing the alignment of a single target feature point and establishing the corresponding relationship between the pixel coordinates and the robot coordinates, only a single set of data cannot support a stable and solvable mapping relationship between the spatial coordinate systems. Therefore, the alignment operation process consistent with the target feature point needs to be performed on the remaining feature points on the calibration board one by one, so that each feature point obtains a set of independent and usable corresponding relationship data.

[0080] The same alignment operation is performed on each feature point except the target feature point, which means that the feature point is reselected as the current processing object at each time of processing, and the pixel deviation-based movement, judgment and convergence process are repeatedly performed until the position of the feature point in the image meets the alignment condition. After each alignment is completed, a set of corresponding relationships composed of initial pixel coordinates and final robot coordinates is obtained.

[0081] By completing the above process on multiple feature points, multiple sets of corresponding data sets distributed in the image space and the robot space can be formed. These corresponding relationships have discreteness and coverage in spatial position, so that the overall mapping relationship between the pixel coordinate system and the robot coordinate system has computability. Based on the multiple sets of corresponding relationships, the hand-eye transformation relationship between the image coordinates and the robot coordinates can be established by solving the spatial transformation parameters, which is used to describe the geometric mapping between the two coordinate systems.

[0082] The embodiment establishes stable pixel coordinate-robot coordinate corresponding relationships for multiple feature points, so that the spatial transformation calculation has sufficient data support, thereby improving the stability of the coordinate mapping relationship and the reliability of the overall calibration result.

[0083] In one embodiment, the above step S10 comprises:

[0084] S101, control the rotation of each joint of the robot, adjust the relative angle between the large arm and the small arm of the robot, and make the robot avoid the singular position of being fully stretched or fully folded, so as to keep the robot in a non-singular posture;

[0085] S102, place the calibration board in the field of view of the camera, control the camera to collect original image data containing the calibration board, and pre-process the original image data to obtain the image of the calibration board;

[0086] S103, identify multiple feature points from the image by using a corner detection algorithm, and extract the pixel coordinates of each feature point;

[0087] S104, determine the coordinate position of the center of the camera field of view in the image coordinate system based on the imaging parameters of the camera.

[0088] In the embodiment, the rotation of each joint of the manipulator and the adjustment of the relative angle between the upper arm and the lower arm belong to the pretreatment of the manipulator posture reachability and motion stability. The rotation of each joint refers to the driving joint executing angle change, so that the end effector is in a posture range suitable for observation and movement. The relative angle between the upper arm and the lower arm refers to the included angle state formed by the two linkage configurations. When the included angle is close to being fully stretched or fully folded, the mapping relationship between the joint speed and the end displacement will change dramatically, causing the end to respond unstably to a small control amount, and then affecting the fixed step movement accuracy and position reproduction accuracy in the subsequent alignment process. Avoiding the singular position of fully stretching or fully folding can be achieved by setting the included angle interval and constraining the target angle of the joint. It can be achieved by joint angle threshold judgment, posture constraint in motion planning or singularity avoidance strategy in the controller. Keeping the manipulator in a non-singular posture means limiting the manipulator working point to a region with reversible kinematic mapping and good numerical conditions, so that the end displacement is closer to the command displacement during subsequent multi-directional movement, and the cumulative deviation when returning to the first manipulator coordinate is reduced.

[0089] Placing the calibration board in the camera field of view and controlling the camera to collect raw image data containing the calibration board belong to the image acquisition link of obtaining subsequent feature point recognition input data. Placing the calibration board in the camera field of view emphasizes that the calibration board can be completely or sufficiently covered on the imaging plane, avoiding feature point missing or edge distortion. Controlling the camera to collect can be achieved by triggering the acquisition instruction, setting the exposure time, gain, frame rate and resolution parameters to obtain the clarity and contrast that meet the feature point detection. The raw image data is the unprocessed or low-processed data stream output by the camera sensor, which contains noise, uneven illumination, lens distortion residues, and color channel differences. Preprocessing the raw image data to obtain the image of the calibration board means performing a standardization processing procedure that can be used for detection on the collected data. The processing goal is to improve the separability of the corner structure at the pixel level and reduce non-structural interference. Preprocessing can include noise reduction filtering to suppress random noise, grayscale or channel selection to reduce detection instability caused by color differences, histogram stretching or local contrast enhancement to enhance the black and white grid or circle array boundary, binarization or edge enhancement to highlight the geometric contour, distortion correction or image cropping to exclude irrelevant areas and reduce the influence of edge distortion. The calibration board image output by preprocessing needs to be consistent with the input format of subsequent corner detection and keep the pixel coordinate system definition consistent, so that the extracted pixel coordinates can be compared with the coordinate position at the center of the camera field of view in the same coordinate system.

[0090] The key step of converting the geometric structure of the calibration board into discrete pixel measurements is to identify multiple feature points from the image and extract the pixel coordinates of each feature point using a corner detection algorithm. The function of the corner detection algorithm is to extract the intersection of local two-dimensional structures from positions with significant gray level or gradient changes. The feature points can be the intersection of a checkerboard, the center of a circular dot array, or other points with stable geometric constraints. Identifying multiple feature points means locating multiple candidate points within the entire image range and eliminating false positives through response value thresholding, non-maximum suppression, connected component constraints, or geometric consistency checks. Extracting the pixel coordinates of each feature point means outputting the coordinate values of the point in the image coordinate system. The pixel coordinates at least include horizontal and vertical coordinates, and the coordinate accuracy can be further improved through sub-pixel precision positioning. Sub-pixel precision positioning can be achieved based on local gray fitting, corner response function interpolation, or iterative minimization of error, so that the pixel coordinates are not limited by the integer pixel grid, thereby providing higher resolution for subsequent alignment judgments based on pixel distance. The multiple feature point pixel coordinates output in this step are not only used for subsequent selection of target feature points, but also provide repeatable image-side measurement references for subsequent establishment of multiple corresponding relationships. Therefore, it is necessary to ensure that the output coordinates are strictly consistent with the pre-processed image coordinate system, and avoid the problem of unsynchronized coordinate transformation caused by image cropping, scaling, or distortion correction.

[0091] Determining the coordinate position of the camera field of view center in the image coordinate system based on the camera's imaging parameters is a step that provides a unified reference point for subsequent alignment determination. The coordinate position of the camera field of view center in the image coordinate system can be derived from the imaging parameters, which include image resolution, principal point parameters, or effective imaging area definition. If the imaging parameters include the principal point coordinates, the camera field of view center can be directly used as the principal point coordinates. If the imaging parameters only include image width and height and effective area boundaries, the camera field of view center can be calculated from the effective area geometric center. The determination process needs to be consistent with the pre-processing process. When the pre-processing includes cropping, scaling, or distortion correction, the coordinate position of the camera field of view center needs to be mapped to the pre-processed image coordinate system synchronously to ensure consistency with the feature point pixel coordinates in the same coordinate reference. The camera field of view center serves as a reference point for subsequent pixel distance calculation and preset alignment condition determination. The determination of its coordinate position directly affects the numerical stability of error measurement and the consistency of alignment termination condition determination. Therefore, it is necessary to use reproducible parameter sources and solidify the calculation rules to ensure consistency in the definition of the reference point for images obtained from different batches.

[0092] The embodiment constrains the robot to be in a non-singular pose before image acquisition and completes image preprocessing for the calibration board, so that the image of the calibration board collected by the camera has more stable structural feature expression, thereby improving the detectability and positioning accuracy of multiple feature point pixel coordinates. Meanwhile, by determining the coordinate position of the center of the camera field of view in the image coordinate system based on the imaging parameters, a consistent error measurement reference point is provided with the feature point pixel coordinates, so that the subsequent alignment judgment based on pixel distance has consistent coordinate reference and reproducible calculation input, thereby reducing the accumulation of alignment errors caused by unstable pose, image quality fluctuations or inconsistent reference point definition.

[0093] In one embodiment, the above step S30 comprises:

[0094] S301, set a plurality of movement directions including the positive direction of the horizontal axis, the negative direction of the horizontal axis, the positive direction of the vertical axis and the negative direction of the vertical axis in the robot plane coordinate system;

[0095] S302, for each movement direction, control the robot to move a fixed step distance from the first robot coordinate along the movement direction and keep the robot stationary;

[0096] S303, record the robot coordinate of the position where the robot is located after moving as the tentative robot coordinate, and identify the position of the target feature point in the current image as the tentative pixel coordinate;

[0097] S304, record the association of the tentative robot coordinate and the tentative pixel coordinate as tentative information;

[0098] S305, control the robot to return to the first robot coordinate and continue the movement of the next direction until the tentative operation of all movement directions is completed.

[0099] In the embodiment, the meaning of the first robot coordinate as a reference is to fix the current position of the robot as a reference point for a tentative cycle, and the displacement calculation, return control and tentative information archiving of each movement direction are established around the first robot coordinate. The source of the first robot coordinate can come from the real-time feedback pose of the robot controller or from the read value after the motion control instruction is issued; to avoid reference drift, the first robot coordinate needs to be written into the cache before a tentative cycle starts and remains unchanged until the tentative operation of all movement directions is completed. Taking the first robot coordinate as a reference also implies the consistency of coordinate expression, which requires the robot coordinate to use the same coordinate expression as the robot plane coordinate system, at least including the displacement components of the horizontal axis and the vertical axis, and if necessary, the attitude component can be included but is kept constant during the plane tentative to avoid the disturbance of image side projection caused by attitude change.

[0100] The setting of multiple moving directions is used to construct a discrete search neighborhood around the first robot coordinate, so that the probe covers four independent directions, i.e., the positive horizontal direction, the negative horizontal direction, the positive vertical direction, and the negative vertical direction. The robot plane coordinate system defines the definition space of the moving directions, so that the positive horizontal direction and the negative horizontal direction form a pair of opposite direction vectors, and the positive vertical direction and the negative vertical direction form another pair of opposite direction vectors. The source of the moving direction can be the axial definition of the robot base coordinate system projected onto the work plane, or the local coordinate system of the jig coordinate system or the plane where the calibration plate is located. Once selected, the unit vectors of the four moving directions need to be fixed and remain unchanged within the same probe cycle to avoid changes in direction definition leading to incomparable probe information. In order to ensure the repeatability of four-direction probe, the moving direction can be written in the control program in the form of discrete enumeration, for example, binding the direction vector with the direction identifier, and at the same time, reserving the target displacement amount combined with the fixed step length for each direction, so as to facilitate the generation of motion control instructions.

[0101] The fixed step length is used to uniformly constrain the displacement scale of each probe. The meaning of the fixed step length is that the displacement modulus remains consistent in each moving direction, so that the probe result is caused only by the difference in direction and is not masked by the difference in step length. The source of the fixed step length can be a preconfigured parameter or obtained by updating the previous iteration; the fixed step length remains constant in a four-direction probe to ensure the comparability of the internal information of the same round of probe. In order to control the robot to move a fixed step length along the moving direction, the fixed step length needs to be combined with the corresponding moving direction to generate a target robot coordinate candidate value, and a point-to-point motion or linear interpolation motion is executed by a motion controller. In order to form a stable association between the probe robot coordinate and the probe pixel coordinate, the motion process needs to include a position determination logic, for example, based on a position error threshold, a speed threshold, or a controller to position flag to confirm that the robot has reached the target position, and then enters a state of keeping the robot stationary. The purpose of keeping the robot stationary is to eliminate motion blur and time drift when image acquisition and target feature point positioning are performed, so that the current image and the current robot coordinate are at the same time. The holding time can be determined by the camera exposure time, the image frame synchronization period, and the controller stable time, or the image acquisition can be triggered by detecting that the robot feedback speed approaches zero and continuously satisfies the stable condition.

[0102] The record of the tentative robot coordinates is derived from the robot coordinates of the position where the robot is located after moving. The tentative robot coordinates need to be read while the robot is stationary to avoid reading intermediate values in motion. The record should retain the same coordinate expression form as the first robot coordinates, and be accompanied by the current moving direction identifier and fixed step value, to facilitate subsequent classification of the tentative information by direction and comparison with other directions. The position recognition of the target feature point in the current image forms the tentative pixel coordinates. The source of the tentative pixel coordinates can be the pixel coordinate output obtained by performing target feature point positioning operation on the current image. The positioning process needs to be consistent with the coordinate system of the initial pixel coordinates to ensure that the pixel distance between the tentative pixel coordinates and the center of the camera field of view can be directly calculated. To improve the stability of the tentative pixel coordinates, the positioning process can use corner sub-pixel precise positioning or local window-based center fitting, and output an invalid mark when identification fails to avoid pollution of the tentative information; but within the scope of this paragraph, the key is to ensure that the tentative pixel coordinates and the tentative robot coordinates are collected at the same tentative time and recorded in pairs.

[0103] The tentative information is used to express the correlation between the tentative robot coordinates and the tentative pixel coordinates. The minimum structure of the tentative information can include a set of key-value pairs, where the key includes the moving direction identifier and the tentative robot coordinates, and the value includes the tentative pixel coordinates; or it can adopt the form of a record entry, combining the tentative robot coordinates, the tentative pixel coordinates, the moving direction identifier, and the fixed step into an indivisible record. The organization method of the tentative information needs to support subsequent traversal to extract all tentative pixel coordinates, and support tracing the corresponding tentative robot coordinates according to the tentative pixel coordinates, so when writing the tentative information, it should ensure that the fields of each record are complete and consistent, to avoid missing fields that prevent subsequent retrieval of tentative robot coordinates from the tentative information. The storage carrier of the tentative information can be a memory structure, a cache queue, or a persistent table, but the focus in this paragraph is to write immediately after each movement to ensure that the tentative information covers all moving directions after the four-way tentative exploration is completed.

[0104] The control robot returns to the first robot coordinate and continues the movement in the next direction, so as to place each trial in the same starting condition, and make the trial results between the movement directions comparable. The return control needs to generate a movement control instruction from the trial robot coordinate to the first robot coordinate, and perform the stability determination again after reaching the position, so as to ensure that the next direction movement starts from the same first robot coordinate. The continuous movement in the next direction means that the cycle is executed according to the preset movement direction sequence until the horizontal axis positive direction, the horizontal axis negative direction, the vertical axis positive direction and the vertical axis negative direction are all completed. The determination that the trial operation is all completed can be realized based on the direction counter, the direction identifier traversal completion flag or the condition that the number of trial information entries reaches the number of movement directions, so as to ensure that the exit condition corresponds to the movement direction set one by one, and avoid missing trial or repeated trial to cause inconsistent trial information.

[0105] The embodiment fixes the reference in the first robot coordinate, and performs discrete trials with fixed step lengths in the horizontal axis positive direction, the horizontal axis negative direction, the vertical axis positive direction and the vertical axis negative direction. Then, the trial robot coordinate and the trial pixel coordinate are recorded synchronously after each time reaching the position and being stationary, and are written into the trial information. Meanwhile, the first robot coordinate is returned after each trial to unify the starting condition, so that the trial information has a comparable basis in terms of direction coverage, displacement scale consistency and coordinate time consistency, thereby providing structured, traceable and lower-noise data input for subsequent determination of the target robot coordinate and the step length adjustment mode based on the trial information, the initial pixel coordinate and the camera field of view center.

[0106] In one embodiment, the step S40 described above comprises:

[0107] S401, calculating a pixel distance between the initial pixel coordinate and the camera field of view center, and marking the calculated distance value as a first pixel distance;

[0108] S402, traversing the trial information, extracting all trial pixel coordinates contained in the trial information, calculating a pixel distance between each trial pixel coordinate and the camera field of view center, and marking the calculated distance value as a trial pixel distance;

[0109] S403, comparing the first pixel distance with all the trial pixel distances in terms of numerical value, and screening out a minimum pixel distance with the smallest numerical value;

[0110] S404, if the minimum pixel distance is the first pixel distance, determining that the target robot coordinate is the first robot coordinate, and determining that the corresponding step length adjustment mode is to reduce the fixed step length;

[0111] S405, if the minimum pixel distance is a certain trial pixel distance, obtaining the trial robot coordinates corresponding to the trial pixel distance from the trial information, determining the target robot coordinates as the trial robot coordinates, and determining the corresponding step adjustment mode as keeping the fixed step unchanged.

[0112] In the present embodiment, the trial information, the initial pixel coordinates, and the camera field center together constitute the input set required for one-time decision. The source of the trial information comes from trial operations in multiple directions, which at least contains the association record of the trial robot coordinates and the trial pixel coordinates, so that the deviation amount on the pixel side can be traced back to the trial robot coordinates that generate the deviation amount. The source of the initial pixel coordinates comes from the recorded results after selecting the target feature point, which represents the image position of the target feature point when the present round of trial movement is not performed. The source of the camera field center comes from the camera imaging parameters or the image coordinate system definition, which represents the reference position in the image coordinate system for alignment determination. Combining the three to determine the target robot coordinates and the corresponding step adjustment mode is essentially to quantitatively compare the deviation amounts of multiple candidate positions in the pixel domain, and to map the optimal deviation amount back to the control decision in the robot domain.

[0113] The pixel distance between the initial pixel coordinates and the camera field center is used to express the deviation amount in the current reference state. The meaning of the pixel distance is to convert the coordinate difference between the initial pixel coordinates and the camera field center into a single scalar in the image coordinate system, so as to facilitate uniform comparison with the distances of other candidate states. The calculation of the pixel distance needs to clarify the distance measurement method. The pixel distance can be obtained by combining the horizontal pixel difference and the vertical pixel difference. The combination method can adopt the Euclidean distance to express the two-dimensional geometric deviation, or the Manhattan distance to express the linear superposition of axial deviation; no matter which combination method is adopted, the coordinate expression of the initial pixel coordinates and the camera field center must be in the same image coordinate system and maintain the same pixel unit. The role of marking the calculated distance value as the first pixel distance is to name and solidify the deviation amount of the reference state, so that the subsequent comparison link can establish a clear comparison object set between the first pixel distance and the trial pixel distance, avoiding ambiguity caused by unclear comparison object source.

[0114] The purpose of traversing the probe information is to construct a candidate deviation set in the same round of probing. The implementation of traversal action depends on the organization structure of the probe information. When the probe information is stored in the form of entry set, traversal can be read one by one in the order of entries; when the probe information is stored in the form of direction index structure, traversal can be read in the order of moving direction index. Extracting all the probe pixel coordinates contained in the probe information requires that each record of the probe information carries a probe pixel coordinate field, and the coordinate expression of this field is consistent with the initial pixel coordinate. Calculating the pixel distance between each probe pixel coordinate and the center of the camera field of view forms a set of probe pixel distances, and the same distance metric needs to be repeatedly executed for each set of probe pixel coordinates to ensure that the probe pixel distances can be directly compared. Marking the calculated distance value as a probe pixel distance makes each candidate direction correspond to an explicit deviation identifier, and maintains a traceable relationship with the source probe pixel coordinate, which facilitates subsequent tracing from the probe pixel distance to the probe robot coordinate that generates the distance.

[0115] The numerical size comparison of the first pixel distance and the probe pixel distance is used to select the state with the smallest deviation between the reference state and each probe state. The input set of the numerical size comparison contains the first pixel distance and all the probe pixel distances, and the output of the comparison is the minimum pixel distance with the smallest numerical value. The implementation of screening out the minimum pixel distance can adopt a linear scanning method, taking the first pixel distance as the initial minimum value and comparing and updating it with the probe pixel distance one by one, or constructing a unified array of all distances and performing minimum value extraction; no matter which implementation is adopted, the source identifier of the minimum pixel distance needs to be retained at the same time, which is used to distinguish whether the minimum pixel distance comes from the first pixel distance or from a certain probe pixel distance. The role of the minimum pixel distance is to convert the two-dimensional deviation problem into a single optimality criterion, so that the target robot coordinate and the step adjustment method can be determined through the branch condition without introducing additional decision variables.

[0116] When the minimum pixel distance is the first pixel distance, it means that the state taking the first robot coordinate as the reference has the smallest deviation in the current round of candidate set. The meaning of determining the target robot coordinate as the first robot coordinate is to keep the target position of the next iteration at the first robot coordinate without performing reference migration towards any probe direction. Determining the corresponding step adjustment method as reducing the fixed step size serves to shrink the search scale on the premise of no position migration, so that subsequent probing is performed near the first robot coordinate with a smaller fixed step size, reducing the risk of crossing the optimal position caused by a too large step size. The specific numerical update of the reduced fixed step size is not expanded in this section, but the step adjustment method as a control variable needs to be explicitly output to provide a definite basis for subsequent actions of updating the fixed step size.

[0117] When the minimum pixel distance is a certain trial pixel distance, it means that the deviation amount of a certain trial state is better than the reference state. At this time, the trial robot coordinates corresponding to the trial pixel distance need to be obtained from the trial information to realize the mapping from the pixel domain optimality to the robot control target. The acquisition logic depends on the association structure of the trial information. The trial pixel distance is calculated from the trial pixel coordinates and the center of the camera field of view, so the trial information needs to support positioning to the corresponding record through the trial pixel coordinates that generate the trial pixel distance, and then reading the trial robot coordinates in the record to avoid the broken chain of only having distance values but being unable to trace the coordinate source. The meaning of determining the target robot coordinates as the trial robot coordinates is to move the reference of the next iteration to the trial robot coordinates to form the control result of advancing along the optimal direction. The meaning of determining the corresponding step length adjustment mode as keeping the fixed step length unchanged is to maintain the search scale in the case of finding a better direction to maintain the advancing efficiency and continue to detect a better position with the same fixed step length in the next round until the reference is better than the four-way trial to trigger the reduction of the fixed step length. The output of keeping the fixed step length unchanged as the step length adjustment mode also needs to be directly referenced in the subsequent fixed step length update link to ensure the closed-loop connection of the decision output and the execution action.

[0118] The embodiment marks the pixel distance between the initial pixel coordinates and the center of the camera field of view as the first pixel distance, calculates the trial pixel distance for all trial pixel coordinates when traversing the trial information, and then compares the first pixel distance and the trial pixel distance to obtain the minimum pixel distance. In this way, the target robot coordinates and the step length adjustment mode can be determined from the source of the minimum pixel distance, so as to trigger the reduction of the fixed step length to shrink the search scale when the reference state is better, and to move the target robot coordinates to the corresponding trial robot coordinates and keep the fixed step length unchanged to continue the advancement when the trial state is better, realizing the quantifiable mapping and traceable closed loop from the pixel deviation amount to the robot control decision.

[0119] In one embodiment, the above step S60 comprises:

[0120] S601, if the preset alignment condition is not met, comparing whether the target robot coordinates and the first robot coordinates are the same;

[0121] S602, if the target robot coordinates and the first robot coordinates are the same, keeping the robot position unchanged;

[0122] S603, if the target robot coordinates and the first robot coordinates are not the same, controlling the robot to move to the target robot coordinates, and updating the target robot coordinates as the new first robot coordinates;

[0123] S604, if the step adjustment mode is to decrease, the fixed step is reduced by a preset ratio, and if the step adjustment mode is to remain unchanged, the fixed step remains unchanged;

[0124] S605, based on the current first robot coordinates, the current fixed step is used to repeatedly perform the steps of moving and recording the trial information, determining the target robot coordinates and the corresponding step adjustment mode, and judging the alignment condition, until the preset alignment condition is met.

[0125] In the present embodiment, the failure to meet the preset alignment condition represents that the deviation between the target feature point and the center of the camera field of view is still in an unacceptable range in the current iteration state, and continuing to output the corresponding relationship in the original state will introduce systematic errors, so it is necessary to enter the joint update process of coordinates and steps to enable the next round of alignment operation to continue in a new spatial center or a new search scale. The meaning of updating the first robot coordinates according to the target robot coordinates is to re-anchor the reference position of the next round of trial operation to the more optimal position determined in the current round, and the meaning of updating the fixed step according to the step adjustment mode is to bind the displacement scale of the next round of trial operation to the comparison result of the deviation in the current round, so that the search can not only advance in a more optimal direction, but also shrink the search scale when it cannot continue to advance, thereby avoiding repeated oscillation in a larger fixed step.

[0126] Comparing whether the target robot coordinates and the first robot coordinates are the same is used to distinguish between two types of iteration situations. In one type of situation, the target robot coordinates and the first robot coordinates are the same, which means that the candidate position corresponding to the minimum pixel distance based on the trial information is still located at the first robot coordinates, which shows that no directional improvement closer to the center of the camera field of view is found in the trial movement in multiple directions, which requires the robot position to remain unchanged to avoid invalid displacement in the absence of benefits, and at the same time provides a stable spatial reference for the shrinkage of the fixed step in the subsequent iteration. In another type of situation, the target robot coordinates and the first robot coordinates are not the same, which means that there is a trial robot coordinates in the trial information that brings a smaller deviation in the pixel domain, which requires the robot to move to the target robot coordinates to migrate the physical position of the robot to a candidate position with a smaller deviation, thereby placing the search center of the next iteration in a more optimal neighborhood.

[0127] Keeping the robot position unchanged is not only a constraint on physical execution, but also a constraint on state variables. Keeping the robot position unchanged enables the current reference position to still serve as a reference point for participating in multi-directional trial in the next round, thereby avoiding reference drift when no more optimal direction is obtained. The first robot coordinates remain unchanged in value in this branch, and the next round of trial information will be generated around the same first robot coordinates, so that the trial results of different rounds are comparable and provide a clear effective reference for the update of the fixed step.

[0128] The movement of the robot to the target robot coordinates embodies the mapping closed loop from the pixel domain optimality to the robot domain action. The movement instruction needs to be issued to the robot controller with the target robot coordinates as the target point parameter, so that the end effector reaches the target point and is in a stable state, and then the target robot coordinates are updated to the new first robot coordinates, completing the replacement of the reference coordinates. The role of this update action is to make the next round of exploration information around the new first robot coordinates, so that the search center iteratively advances along the optimal direction, reducing the probability of repeated exploration near the original first robot coordinates without convergence. Updating the target robot coordinates to the new first robot coordinates at the same time ensures that the subsequent "return to the first robot coordinates" action has a unique direction, avoiding the inconsistency between the reference coordinates and the actual position leading to the error of the return path or the error of the exploration information.

[0129] The fixed step size is updated according to the step size adjustment mode. After the spatial center is updated or remains unchanged, the displacement scale of the next round of exploration movement is adjusted. The step size adjustment mode is to reduce the corresponding fixed step size by a preset proportion, which can be a fractional proportion or other scaling proportion less than one, so that the fixed step size is strictly less than the last round of fixed step size in value, thereby reducing the search radius of the next round of exploration, improving the resolution of the alignment process in the local area, and reducing the risk of crossing the optimal position. The step size adjustment mode is to keep the corresponding fixed step size unchanged without numerical update, so that the next round of exploration is still with the original fixed step size, maintaining the advancing speed, which is suitable for the situation where a better direction has been found and migrated to a new first robot coordinates, so that the search can continue to move along the better neighborhood, without reducing the speed of approaching the center of the camera field of view due to the premature reduction of the fixed step size.

[0130] The steps of moving and recording the trial information, determining the target robot coordinates and the corresponding step adjustment mode, and judging the alignment condition, which are repeated with the current first robot coordinates as the reference and the current fixed step, together constitute a closed-loop iterative structure. The current first robot coordinates are the value after the target robot coordinates are updated to the new first robot coordinates or the value when the first robot coordinates remain unchanged. The current fixed step is the value after the fixed step is reduced by a preset proportion or the value when the fixed step remains unchanged. The current first robot coordinates as the reference ensure that the starting point of each trial is consistent and can be reused by the return action. The current fixed step ensures that the displacement scale of each trial is consistent with the step adjustment mode and takes effect directly. In the repeated execution link, moving and recording the trial information provides associated data between the pixel domain deviation and the robot coordinates, determining the target robot coordinates and the corresponding step adjustment mode converts the associated data into iterative decisions, and judging the alignment condition provides convergence criteria. When the alignment condition is still not met, the process is triggered again to realize the alternative combination of spatial center migration and search scale contraction, so that the iteration can dynamically switch between promotion and refinement until the alignment condition is met.

[0131] The embodiment compares the target robot coordinates with the first robot coordinates when the preset alignment condition is not met, distinguishes between the two branches of keeping the robot position unchanged and controlling the robot to move to the target robot coordinates, and updates the target robot coordinates to the new first robot coordinates in the coordinate migration branch. At the same time, the fixed step is updated by reducing it by a preset proportion or keeping it unchanged according to the step adjustment mode. Then, the trial information generation, target robot coordinates determination, and alignment condition judgment are repeated with the updated first robot coordinates and the updated fixed step, so that the iterative process has a clear spatial reference updating mechanism and search scale adaptive mechanism, reduces invalid displacement and repeated trials, and improves the stability and efficiency of the alignment process converging to the center of the camera field of view.

[0132] In one embodiment, the above step S70 includes:

[0133] S701, if the preset alignment condition is met, terminating the movement control and trial operation of the robot;

[0134] S702, reading the current robot coordinates of the robot and recording the current robot coordinates as the final robot coordinates;

[0135] S703, data pairing the final robot coordinates and the initial pixel coordinates of the target feature point to establish a set of corresponding relationships;

[0136] S704, saving the set of corresponding relationships to the calibration data set.

[0137] In the present embodiment, the satisfaction of the preset alignment condition means that the relative deviation of the target feature point from the center of the camera field of view has entered the allowable range, and further execution of the movement control and the trial operation will introduce new displacement disturbances and destroy the current alignment state that has been achieved, thus the need to switch to the data solidification and correspondence generation process. The termination of the movement control and the trial operation of the robot includes the stop processing of the motion instruction channel and the trial trigger channel, so that the robot remains in a static state and freezes the current iteration of the control output, avoiding position changes during reading of the robot coordinates, resulting in unstable final robot coordinates. The termination action also includes switching the control state of the current alignment process from the loop execution state to the record state, so that the subsequent reading and storage actions are completed in a consistent control context, ensuring that the pairing of the final robot coordinates and the initial pixel coordinates is based on the same alignment time.

[0138] The reading of the current robot coordinates of the robot means obtaining the coordinate data of the robot in the static state from the robot controller or the position feedback link. The robot coordinates can be obtained by joint encoder feedback through forward kinematics solving, or can be directly provided by the controller as the end pose coordinate output. The reading process needs to be executed after the termination of the movement control, to ensure that the current robot coordinates of the robot correspond to the actual position when the preset alignment condition is satisfied, rather than the transitional position during the movement process. The recording of the current robot coordinates as the final robot coordinates means writing the robot coordinates into the data structure used for calibration, and giving it the identification attribute as the final robot coordinates, so that it is referred to as the final robot coordinates in subsequent data pairing and data set saving, thereby avoiding confusion with intermediate variables such as the first robot coordinates, the target robot coordinates, etc. The recording of the final robot coordinates can include the coordinate value itself and the time stamp or serial number field associated with the coordinate, for maintaining the traceability of data items in the calibration data set, but the core of the recording action is to determine the current robot coordinates as the final robot coordinates.

[0139] The meaning of data pairing the final robot coordinates with the initial pixel coordinates of the target feature points is to establish a single-point binding relationship across the coordinate domain, so that the initial pixel coordinates in the image coordinate system and the final robot coordinates in the robot coordinate system become two endpoints of the same calibration sample. The initial pixel coordinates are derived from the image position record of the target feature point at the beginning of the alignment iteration, and the final robot coordinates are derived from the robot position record at the time when the preset alignment condition is met. Both of them reflect the observation reference of the same target feature point in the image domain and the alignment position in the robot domain, respectively. The data pairing needs to clarify the field structure of the corresponding relationship, at least including the initial pixel coordinate field and the final robot coordinate field, and is organized by the same record, so that the paired data can be directly obtained when reading the calibration data set subsequently. The meaning of establishing a set of corresponding relationships is to manage the paired data as an indivisible data unit, avoiding the loss of association caused by the separation of the initial pixel coordinates and the final robot coordinates, and avoiding the cross-pairing of coordinates of different target feature points in the multi-point calibration scene.

[0140] The meaning of saving a set of corresponding relationships to the calibration data set is to write the corresponding relationship into a persistent data container, so that it can be repeatedly called across sessions and across execution cycles. The calibration data set can be a structured table, a file record or a database entry, the key is to be able to store multiple sets of corresponding relationships in record granularity and support additional writing. The saving action needs to include the integrity check at the time of writing, at least including the field integrity check of the corresponding relationship and the return of the writing success state, to prevent the occurrence of half-record state of writing only initial pixel coordinates or final robot coordinates. The organization method of the calibration data set needs to support subsequent traversal and extraction of initial pixel coordinates and final robot coordinates by record, so the field naming and data format should be kept consistent when saving, so that each set of corresponding relationships in the same calibration data set has a unified data interface.

[0141] The embodiment terminates the movement control and trial operation of the robot when the preset alignment condition is met, so that the robot position is stabilized in the static state at the alignment time, and then the current robot coordinates of the robot are read and recorded as the final robot coordinates. The final robot coordinates are data paired with the initial pixel coordinates of the target feature points to establish a set of corresponding relationships, and a set of corresponding relationships is saved to the calibration data set, so that each target feature point can form a persistent paired data record after reaching the alignment condition, reducing the coordinate drift and pairing misplacement caused by continuous movement or inconsistent recording time, and improving the usability and consistency of the corresponding relationship data.

[0142] In one embodiment, the above step S80 includes:

[0143] S801, determine the traversal order of the remaining feature points except the target feature point in the plurality of feature points;

[0144] S802, selecting one from the remaining feature points as a current target feature point in turn according to the traversal order;

[0145] S803, performing, for the current target feature point, an operation of taking the current position of the robot as a reference, recording the first robot coordinate and the initial pixel coordinate from the selection of the current target feature point, moving and recording the trial information, determining the target robot coordinate and the step adjustment mode, making iterative judgment and moving until the alignment condition is met, and finally recording the final robot coordinate and establishing the corresponding relationship with the initial pixel coordinate;

[0146] S804, collecting a set of corresponding relationships established after each execution of the operation;

[0147] S805, repeating the selection, execution and collection steps until all the remaining feature points are processed to obtain multiple sets of corresponding relationships;

[0148] S806, for each set of corresponding relationships in the multiple sets of corresponding relationships, extracting the initial pixel coordinate and the final robot coordinate in the corresponding relationship;

[0149] S807, based on the multiple sets of extracted initial pixel coordinates and final robot coordinates, solving the transformation parameters from the image coordinate system to the robot coordinate system;

[0150] S808, using the transformation parameters to construct the hand-eye transformation relationship.

[0151] In the embodiment, the multiple feature points come from the feature point recognition result of the calibration board image, and the target feature point has completed a time of alignment and formed a set of corresponding relationships. The meaning of performing the same alignment operation for each feature point in the multiple feature points except the target feature point is to reuse the closed-loop process of single-point alignment on the set of remaining feature points, so as to expand the single corresponding relationship into multiple sets of corresponding relationships. The same alignment operation here refers to the same action chain, including recording the first robot coordinate and the initial pixel coordinate, moving and recording the trial information, determining the target robot coordinate and the step adjustment mode, judging whether the target feature point and the center of the camera field of view meet the preset alignment condition, and recording the final robot coordinate and establishing the corresponding relationship with the initial pixel coordinate when the preset alignment condition is met. The reuse of the same alignment operation requires that each round of processing maintains the same naming system and the same data field organization, so that the subsequent solving stage can read the multiple sets of corresponding relationships in a consistent format.

[0152] The purpose of determining the traversal order of the remaining feature points in the plurality of feature points except the target feature point is to constrain the processing order, avoid missing or repeatedly selecting the same feature point, and make the "current target feature point" have a unique direction in each alignment process. The traversal order can be generated by a pixel coordinate sorting rule, for example, combined sorting by the horizontal coordinate from small to large and the vertical coordinate from small to large in the image coordinate system, or by the grid row and column index order of the feature points on the calibration board, or by the pixel distance from the center of the camera field of view, from near to far. Once the order is generated, a stable index mapping is needed to keep the "remaining feature points" set accurately exclude the feature points that have completed processing in each loop, and position the unprocessed feature points to the next selection action according to the traversal order.

[0153] The meaning of selecting one from the remaining feature points as the current target feature point according to the traversal order is to split the multi-point processing into several rounds of single-point processing, and each round takes the current target feature point as the only alignment object. The selection action needs to write the pixel coordinates and feature point identification of the current target feature point into the running context, so that the subsequent recording of the initial pixel coordinates, the identification of the trial pixel coordinates, and the determination of the satisfaction of the preset alignment conditions are all carried out around the same current target feature point. In order to avoid target drift in the alignment process, the reference method of the current target feature point needs to be fixed after the selection action, for example, locking the current target feature point by feature point number or pixel coordinate index, so as to ensure that the pixel coordinates recorded in the trial information all correspond to the same current target feature point.

[0154] The meaning of performing alignment operation on the current target feature point with the current position of the manipulator as the reference is to take the current position of the manipulator as the source of the first manipulator coordinates, so that the local search and convergence of the current round of alignment are completed from the current physical state. The processing with the current position of the manipulator as the reference needs to read the manipulator coordinates first and record them as the first manipulator coordinates, and read the pixel coordinates of the current target feature point in the image and record them as the initial pixel coordinates. Then, multi-direction fixed-step movement is performed around the first manipulator coordinates, and after the movement is completed, the trial information is recorded, which at least contains the association between the trial manipulator coordinates and the trial pixel coordinates, and the trial pixel coordinates come from the position recognition of the current target feature point in the corresponding image frame. The meaning of determining the target manipulator coordinates and the corresponding step adjustment mode based on the trial information, the initial pixel coordinates and the center of the camera field of view is to select the manipulator coordinates with the smallest pixel distance in the candidate coordinate set as the target manipulator coordinates, and to generate the step adjustment mode by the branch rule corresponding to the smallest pixel distance, so as to drive the subsequent iteration. The meaning of judging whether the target feature point and the center of the camera field of view satisfy the preset alignment condition is to determine the convergence state. When the condition is not satisfied, the first manipulator coordinates are updated according to the target manipulator coordinates, and the fixed step is updated according to the step adjustment mode, the movement is performed again, and the cycle of recording the trial information, determining the target manipulator coordinates and the corresponding step adjustment mode, and judging the alignment condition is performed again, until the condition is satisfied. After the condition is satisfied, the current manipulator coordinates are recorded as the final manipulator coordinates, and the final manipulator coordinates and the initial pixel coordinates are established as a set of corresponding relationships, so that the image observation of the current target feature point and the alignment position of the manipulator are fixed as the paired data that can be used for solving.

[0155] The meaning of collecting a set of corresponding relationships established after each execution of the above alignment operation is to collect the initial pixel coordinates and the final manipulator coordinates obtained by each round of processing in a record form into the same container, and maintain the association with the current target feature point. The collection action needs to constrain the writing time after the final manipulator coordinates are confirmed, and keep the field structure consistent during writing, to ensure that the multiple sets of corresponding relationships have a uniform data item format in the calibration data set. The meaning of repeating selection, execution and collection until all remaining feature points are processed is to drive the loop termination condition in traversal order, and stop collecting when the remaining feature point set is empty to obtain multiple sets of corresponding relationships. The key of the loop is to update the remaining feature point set state after each set of corresponding relationships is completed, to ensure that the traversal order advances and the termination condition is consistent.

[0156] The meaning of extracting the initial pixel coordinates and the final robot coordinates in each group of correspondence in the multiple groups of correspondence is to read the input data for solving from the calibration dataset by record to form two data sequences of the same length. The initial pixel coordinates belong to the image coordinate system, and the final robot coordinates belong to the robot coordinate system, which are one-to-one corresponding in each record. The extraction process needs to keep the pairing order from being disturbed, ensure that the first initial pixel coordinate and the first final robot coordinate come from the same group of correspondence, and avoid parameter deviation caused by cross pairing in subsequent solving.

[0157] The meaning of solving the transformation parameters from the image coordinate system to the robot coordinate system based on the extracted multiple groups of initial pixel coordinates and final robot coordinates is to construct a cross-coordinate system mapping model, so that the points in the image coordinate system can be mapped to the points in the robot coordinate system through the transformation parameters. The transformation parameters can be in the form of affine model parameters, rigid body model parameters containing rotation and translation, or extended linear model parameters with scale terms and non-orthogonal terms. The solving process needs to organize multiple groups of initial pixel coordinates and final robot coordinates into equation constraints, and fit the transformation parameters through error measurement, so that the residual error between the predicted robot coordinates after mapping and the corresponding final robot coordinates is minimized as a whole. To improve the fitting stability, consistency screening can be performed on the multiple groups of correspondence during the solving stage, such as removing outlier records according to the residual error threshold, and then re-solving the transformation parameters for the remaining records to reduce the sensitivity of the transformation parameters to abnormal pairing.

[0158] The meaning of constructing the hand-eye transformation relationship using the transformation parameters is to encapsulate the transformation parameters as a callable mapping relationship expression, so that any pixel point in the image coordinate system can be converted to the target pose or target position in the robot coordinate system through the hand-eye transformation relationship. The construction action needs to clarify the input and output interfaces, the input is the pixel coordinate in the image coordinate system, and the output is the robot coordinate in the robot coordinate system, and the transformation parameters are bound to the same relationship as the interface, thereby forming a stable and reusable hand-eye transformation relationship. In order to ensure consistency, the saving form of the hand-eye transformation relationship needs to be managed separately from the calibration dataset, the calibration dataset retains multiple groups of correspondence for review and recalculation, and the hand-eye transformation relationship retains the transformation parameters for online calling and deployment.

[0159] The hand-eye transformation relationship is solved by multiple groups of initial pixel coordinates and final robot coordinates, and its essence is to establish a stable mapping between the image coordinate system and the robot coordinate system. This mapping is not limited to the calibration stage, but can also be used as a basic association relationship between image perception results and robot motion control in the subsequent running stage.

[0160] When entering actual operation or repeated operation, the camera collects a new image and identifies the pixel coordinates of the object to be processed in the image, which are in the image coordinate system consistent with the calibration stage. By calling the established hand-eye transformation relationship, the pixel coordinates are input into the transformation parameters corresponding to the transformation relationship, and the mechanical hand coordinates in the mechanical hand coordinate system corresponding thereto are obtained. The mechanical hand coordinates can be directly used as the movement target coordinates of the mechanical hand, for driving the mechanical hand to move from the current position to the spatial position corresponding to the target position in the image, so as to realize automatic conversion between the visual perception result and the mechanical hand movement instruction.

[0161] In a scene where it is necessary to further improve the positioning accuracy, the mechanical hand coordinates output by the hand-eye transformation relationship can also be used as the starting reference position of the alignment process. The mechanical hand first moves to the mechanical hand coordinates, and then obtains new pixel coordinates in combination with the real-time feedback of the camera. Whether the preset alignment condition is met is judged by the distance between the pixel coordinates and the center of the camera field of view. When the preset alignment condition is not met, the first mechanical hand coordinates are taken as the position to continue to execute the fixed-step multi-directional exploration and iterative update, so that the alignment process is carried out around a position closer to the real target, thereby reducing the exploration range and the number of iterations.

[0162] In the continuous operation process of multiple targets or multiple feature points, the hand-eye transformation relationship can be repeatedly called. The same mapping operation is performed on each newly identified pixel coordinate to obtain a corresponding sequence of mechanical hand coordinates, and the mechanical hand sequentially completes the positioning or operation task according to the sequence. If changes in environmental illumination, installation posture or mechanical structure occur during long-term operation, resulting in gradual accumulation of mapping errors, new initial pixel coordinates and final mechanical hand coordinates can be collected and multiple corresponding relationships are updated, and the transformation parameters are recalculated to update the hand-eye transformation relationship, so that the mapping between the image coordinate system and the mechanical hand coordinate system remains consistent with the actual state.

[0163] The embodiment generates and collects a group of corresponding relationships point by point by performing the same alignment operation on each feature point in the multiple feature points except the target feature point until multiple corresponding relationships are formed, extracts the initial pixel coordinates and the final mechanical hand coordinates from the multiple corresponding relationships, and solves the transformation parameters from the image coordinate system to the mechanical hand coordinate system, and then uses the transformation parameters to construct the hand-eye transformation relationship, so that the mapping relationship is established on the basis of multiple point constraints and supported by a unified data pairing format, the influence of single-point pairing error on the mapping result is reduced, the stability and consistency of the transformation parameter fitting are improved, and the applicability of the hand-eye transformation relationship under different feature point position distributions is enhanced.

[0164] In an embodiment, a mechanical hand eye automatic calibration device is provided, which corresponds to the mechanical hand eye automatic calibration method in the above embodiment. Referring to Figure 3 , Figure 3Fig. 1 is a schematic diagram of the function modules of a preferred embodiment of the hand-eye automatic calibration device of the present application. The image feature acquisition module 10, the target point initialization module 20, the step trial control module 30, the target coordinate determination module 40, the alignment condition judgment module 50, the iterative update control module 60, the single point correspondence generation module 70, and the hand-eye transformation calculation module 80. The detailed description of each function module is as follows:

[0165] The image feature acquisition module 10 is used to place the calibration board in the camera's field of view, acquire the image of the calibration board, identify a plurality of feature points in the image, and acquire the position of the center of the camera's field of view in the image.

[0166] The target point initialization module 20 is used to select one feature point from the plurality of feature points as a target feature point, and record the current first robot coordinates and the initial pixel coordinates of the target feature point.

[0167] The step trial control module 30 is used to control the robot to move a fixed step in multiple directions based on the first robot coordinates, record the trial information related to the pixel coordinates of the target feature point and the robot coordinates after each movement, and control the robot to return to the first robot coordinates and continue the movement in the next direction.

[0168] The target coordinate determination module 40 is used to determine the target robot coordinates and the corresponding step adjustment method based on the trial information, the initial pixel coordinates, and the center of the camera's field of view.

[0169] The alignment condition judgment module 50 is used to determine whether the target feature point and the center of the camera's field of view satisfy the preset alignment condition.

[0170] The iterative update control module 60 is used to update the first robot coordinates according to the target robot coordinates if the preset alignment condition is not satisfied, and update the fixed step according to the step adjustment method. The steps of moving and recording trial information, determining target robot coordinates and corresponding step adjustment method, and judging alignment condition are repeated with the updated fixed step until the preset alignment condition is satisfied.

[0171] The single point correspondence generation module 70 is used to record the current robot coordinates as the final robot coordinates if the preset alignment condition is satisfied, and establish a set of correspondence between the final robot coordinates and the initial pixel coordinates.

[0172] The hand-eye transformation calculation module 80 is used to perform the same alignment operation on each feature point in the plurality of feature points except the target feature point to obtain a plurality of sets of correspondence, and calculate the hand-eye transformation relationship based on the plurality of sets of correspondence.

[0173] The specific limitations of the robot hand-eye automatic calibration device can refer to the aforementioned limitations of the robot hand-eye automatic calibration method, which will not be repeated here. Each module in the robot hand-eye automatic calibration device described above can be realized by software, hardware, and combinations thereof, in whole or in part. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor calls and executes the operations corresponding to each of the above modules.

[0174] In one embodiment, a computer device is provided, which can be a server, and its internal structure diagram can be as shown in Figure 4 The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. The processor of the computer device is used to provide determination and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external client through the network connection. The computer program is executed by the processor to implement the functions or steps of the robot hand-eye automatic calibration method on the server side.

[0175] In one embodiment, a computer device is provided, which can be a client, and its internal structure diagram can be as shown in Figure 5 The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. The processor of the computer device is used to provide determination and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external server through the network connection. The computer program is executed by the processor to implement the functions or steps of the robot hand-eye automatic calibration method on the client side.

[0176] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the computer program to implement the following steps:

[0177] Place the calibration board in the camera's field of view, acquire the image of the calibration board, identify a plurality of feature points in the image and acquire the position of the center of the camera's field of view in the image;

[0178] select one feature point as a target feature point from the plurality of feature points, record a current first robot coordinate and an initial pixel coordinate of the target feature point;

[0179] control the robot to move a fixed step length in multiple directions based on the first robot coordinate, record the pixel coordinate of the target feature point and the robot coordinate related to the trial information after each movement, and control the robot to return to the first robot coordinate and continue moving in the next direction;

[0180] determine a target robot coordinate and a corresponding step length adjustment mode based on the trial information, the initial pixel coordinate and the center of the camera field of view;

[0181] determine whether the target feature point and the center of the camera field of view satisfy a preset alignment condition;

[0182] If the preset alignment condition is not satisfied, update the first robot coordinate according to the target robot coordinate, update the fixed step length according to the step length adjustment mode, and repeat the steps of moving and recording trial information, determining a target robot coordinate and a corresponding step length adjustment mode, and judging an alignment condition with the updated fixed step length until the preset alignment condition is satisfied;

[0183] If the preset alignment condition is satisfied, record the current robot coordinate as a final robot coordinate, and establish the final robot coordinate and the initial pixel coordinate as a corresponding relationship;

[0184] For each feature point in the plurality of feature points except the target feature point, perform the same alignment operation to obtain a plurality of corresponding relationships, and calculate a hand-eye transformation relationship based on the plurality of corresponding relationships.

[0185] In one embodiment, a computer readable storage medium is provided, which can be non-volatile or volatile, and a computer program is stored on the computer readable storage medium, and the computer program is executed by a processor to implement the following steps:

[0186] Place a calibration board in the camera field of view range, obtain an image of the calibration board, identify a plurality of feature points in the image and obtain the position of the center of the camera field of view in the image;

[0187] select one feature point as a target feature point from the plurality of feature points, record a current first robot coordinate and an initial pixel coordinate of the target feature point;

[0188] With the first robot coordinate as a reference, the robot is controlled to move along multiple directions by a fixed step length, after each movement, tentative information related to pixel coordinates and robot coordinates of the target feature point is recorded, and the robot is controlled to return to the first robot coordinate and continue movement in the next direction;

[0189] Based on the tentative information, the initial pixel coordinates and the center of the camera field of view, a target robot coordinate and a corresponding step length adjustment mode are determined;

[0190] It is judged whether the target feature point and the center of the camera field of view satisfy a preset alignment condition;

[0191] If the preset alignment condition is not satisfied, the first robot coordinate is updated according to the target robot coordinate, the fixed step length is updated according to the step length adjustment mode, and the steps of moving and recording tentative information, determining a target robot coordinate and a corresponding step length adjustment mode, and judging an alignment condition are repeated with the updated fixed step length until the preset alignment condition is satisfied;

[0192] If the preset alignment condition is satisfied, the current robot coordinate is recorded as a final robot coordinate, and the final robot coordinate and the initial pixel coordinates are established as a corresponding relationship;

[0193] For each feature point in the multiple feature points except the target feature point, the same alignment operation is performed to obtain multiple corresponding relationships, and based on the multiple corresponding relationships, a hand-eye transformation relationship is calculated.

[0194] It should be noted that the functions or steps that the computer readable storage medium or the computer device can achieve as described above can correspond to the related descriptions of the server side and the client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

Claims

1. A robot hand-eye automatic calibration method, characterized in that, The method comprises the following steps: placing a calibration board in the field of view of a camera, obtaining an image of the calibration board, identifying a plurality of feature points in the image and obtaining the position of the center of the field of view of the camera in the image; selecting one of the plurality of feature points as a target feature point, recording the current first robot coordinate and the initial pixel coordinate of the target feature point; taking the first robot coordinate as a reference, controlling the robot to move a fixed step length in multiple directions, recording the trial information related to the pixel coordinate of the target feature point and the robot coordinate after each movement, and controlling the robot to return to the first robot coordinate and continue moving in the next direction; based on the trial information, the initial pixel coordinate and the center of the field of view of the camera, determining the target robot coordinate and the corresponding step length adjustment mode; determining whether the target feature point and the center of the field of view of the camera satisfy a preset alignment condition; if the preset alignment condition is not satisfied, updating the first robot coordinate according to the target robot coordinate, updating the fixed step length according to the step length adjustment mode, and repeating the steps of moving and recording trial information, determining the target robot coordinate and the corresponding step length adjustment mode, and judging the alignment condition with the updated fixed step length until the preset alignment condition is satisfied; if the preset alignment condition is satisfied, recording the current robot coordinate as the final robot coordinate, and establishing the final robot coordinate and the initial pixel coordinate as a corresponding relationship; for each feature point in the plurality of feature points except the target feature point, performing the same alignment operation to obtain a plurality of corresponding relationships, and calculating the hand-eye transformation relationship based on the plurality of corresponding relationships.

2. The robot hand-eye automatic calibration method of claim 1, wherein, placing a calibration board in the field of view of a camera, obtaining an image of the calibration board, identifying a plurality of feature points in the image and obtaining the position of the center of the field of view of the camera in the image, comprising: controlling the joints of the robot to rotate, adjusting the relative angle between the large arm and the small arm of the robot, so that the robot avoids the singular position of being fully stretched or fully folded, and keeps the robot in a non-singular posture; placing a calibration board in the field of view of a camera, controlling the camera to collect raw image data containing the calibration board, preprocessing the raw image data to obtain an image of the calibration board; using a corner detection algorithm to identify a plurality of feature points from the image and extract the pixel coordinates of each feature point; based on the imaging parameters of the camera, determining the coordinate position of the center of the field of view of the camera in the image coordinate system.

3. The robot hand-eye automatic calibration method of claim 1, wherein, taking the first robot coordinate as a reference, controlling the robot to move a fixed step length in multiple directions, recording the trial information related to the pixel coordinate of the target feature point and the robot coordinate after each movement, and controlling the robot to return to the first robot coordinate and continue moving in the next direction, comprising: setting a plurality of movement directions including the positive direction of the horizontal axis, the negative direction of the horizontal axis, the positive direction of the vertical axis and the negative direction of the vertical axis in the plane coordinate system of the robot; for each movement direction, controlling the robot to move a fixed step length from the first robot coordinate along the movement direction and keep the robot stationary; record the robot coordinates of the positions where the robot hand is located after moving as tentative robot coordinates, and identify the positions of the target feature points in the current image as tentative pixel coordinates; record the association between the tentative robot coordinates and the tentative pixel coordinates as tentative information; control the robot hand to return to the first robot coordinates and continue moving in the next direction until the tentative operation in all moving directions is completed.

4. The robot hand-eye automatic calibration method of claim 1, wherein, based on the tentative information, the initial pixel coordinates and the center of the camera field of view, determine the target robot coordinates and the corresponding step adjustment mode, including: calculate the pixel distance between the initial pixel coordinates and the center of the camera field of view, and mark the calculated distance value as the first pixel distance; traverse the tentative information, extract all tentative pixel coordinates contained in the tentative information, calculate the pixel distance between each tentative pixel coordinate and the center of the camera field of view, and mark the calculated distance value as the tentative pixel distance; compare the first pixel distance with all tentative pixel distances in value size, and filter out the minimum pixel distance with the smallest value; if the minimum pixel distance is the first pixel distance, determine that the target robot coordinates are the first robot coordinates, and determine that the corresponding step adjustment mode is to reduce the fixed step size; if the minimum pixel distance is a tentative pixel distance, obtain the tentative robot coordinates corresponding to the tentative pixel distance from the tentative information, determine that the target robot coordinates are the tentative robot coordinates, and determine that the corresponding step adjustment mode is to keep the fixed step size unchanged.

5. The robot hand-eye automatic calibration method of claim 1, wherein, if the preset alignment condition is not met, update the first robot coordinates according to the target robot coordinates, update the fixed step size according to the step adjustment mode, and repeat the steps of moving and recording tentative information, determining target robot coordinates and corresponding step adjustment mode, and judging alignment condition with the updated fixed step size until the preset alignment condition is met, including: if the preset alignment condition is not met, compare whether the target robot coordinates are the same as the first robot coordinates; if the target robot coordinates are the same as the first robot coordinates, keep the robot hand position unchanged; if the target robot coordinates are not the same as the first robot coordinates, control the robot hand to move to the target robot coordinates, and update the target robot coordinates as new first robot coordinates; if the step adjustment mode is to reduce, reduce the fixed step size by a preset proportion, and if the step adjustment mode is to keep unchanged, keep the fixed step size unchanged; repeat the steps of moving and recording tentative information, determining target robot coordinates and corresponding step adjustment mode, and judging alignment condition with the current first robot coordinates as the reference and the current fixed step size until the preset alignment condition is met.

6. The robot hand-eye automatic calibration method of claim 1, wherein, if the preset alignment condition is met, record the current robot coordinates as the final robot coordinates, and establish a corresponding relationship between the final robot coordinates and the initial pixel coordinates, including: If the preset alignment condition is met, the movement control and the trial operation of the robot are terminated. The current robot coordinates of the robot are read and recorded as final robot coordinates. The final robot coordinates are paired with the initial pixel coordinates of the target feature point to establish a set of corresponding relationships. The set of corresponding relationships is saved in a calibration dataset.

7. The robot hand-eye automatic calibration method of claim 1, wherein, For each feature point in the plurality of feature points except the target feature point, the same alignment operation is performed to obtain a plurality of sets of corresponding relationships. Based on the plurality of sets of corresponding relationships, a hand-eye transformation relationship is calculated, including: determining a traversal order of the remaining feature points in the plurality of feature points except the target feature point; selecting one of the remaining feature points as a current target feature point in the traversal order; for the current target feature point, taking the current position of the robot as a reference, performing the operation of recording the first robot coordinates and the initial pixel coordinates from the selection of the current target feature point, moving and recording the trial information, determining the target robot coordinates and the step adjustment mode, performing the iterative judgment and movement until the alignment condition is met, and finally recording the final robot coordinates and establishing the corresponding relationship with the initial pixel coordinates; collecting a set of corresponding relationships established after each execution of the operation; repeating the selection, execution, and collection steps until all remaining feature points are processed to obtain a plurality of sets of corresponding relationships; for each set of corresponding relationships in the plurality of sets of corresponding relationships, extracting the initial pixel coordinates and the final robot coordinates in the corresponding relationship; based on the extracted plurality of sets of initial pixel coordinates and final robot coordinates, solving the transformation parameters from the image coordinate system to the robot coordinate system; using the transformation parameters, constructing a hand-eye transformation relationship.

8. A robot hand-eye automatic calibration device, characterized by comprising: The robot hand-eye automatic calibration device comprises: an image feature acquisition module configured to place a calibration board in the field of view of a camera, acquire an image of the calibration board, identify a plurality of feature points in the image, and acquire the position of the center of the field of view of the camera in the image; a target point initialization module configured to select one feature point from the plurality of feature points as a target feature point, and record the current first robot coordinates and the initial pixel coordinates of the target feature point; a step trial control module configured to take the first robot coordinates as a reference, control the robot to move a fixed step in multiple directions, record the trial information related to the pixel coordinates and robot coordinates of the target feature point after each movement, and control the robot to return to the first robot coordinates and continue the movement in the next direction; a target coordinate determination module configured to determine the target robot coordinates and the corresponding step adjustment mode based on the trial information, the initial pixel coordinates, and the center of the field of view of the camera; an alignment condition determination module configured to determine whether the target feature point and the center of the field of view of the camera meet a preset alignment condition. The iterative updating control module is configured to, if the preset alignment condition is not met, update the first robot coordinate according to the target robot coordinate, and update the fixed step according to the step adjustment mode, and repeat the steps of moving and recording the trial information, determining the target robot coordinate and the corresponding step adjustment mode, and judging the alignment condition with the updated fixed step until the preset alignment condition is met. The single-point correspondence generation module is configured to, if the preset alignment condition is met, record the current robot coordinate as a final robot coordinate, and establish the final robot coordinate and the initial pixel coordinate as a set of correspondence. The hand-eye transformation calculation module is configured to perform the same alignment operation on each feature point in the plurality of feature points except the target feature point to obtain a plurality of sets of correspondence, and calculate a hand-eye transformation relationship based on the plurality of sets of correspondence.

9. A computer device, comprising: The computer device comprises a memory, a processor, and a robot hand-eye automatic calibration program stored in the memory and executable on the processor, and the robot hand-eye automatic calibration program, when executed by the processor, implements the steps of the robot hand-eye automatic calibration method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a robot hand-eye automatic calibration program, and the robot hand-eye automatic calibration program, when executed by the processor, implements the steps of the robot hand-eye automatic calibration method according to any one of claims 1-7.

Images