Hand-eye calibration error quantification method based on eye on hand

Abstract

The invention provides a hand-eye calibration error quantification method based on an eye on a hand, and the method comprises the steps: obtaining a calibration image of a calibration plate at a designated position and an actual mechanical arm pose corresponding to the calibration image, the calibration image is acquired by a camera arranged at the tail end of the mechanical arm based on a preset mechanical arm pose; first coordinates of the feature points are determined according to the calibration image and the actual mechanical arm pose; determining reference coordinates of the feature points according to the first coordinates corresponding to the feature points in the different calibration images; and determining an error quantification result according to the first coordinate and the reference coordinate corresponding to each feature point. By implementing the method disclosed by the invention, the quantification processing of the hand-eye calibration error can be accurately and intuitively realized, so that the quantification effect of the eye calibration error is effectively improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of hand-eye calibration, and in particular to a hand-eye calibration error quantification method based on eyes on hands. Background Art

[0002] Hand-eye calibration is a key technology in robotic vision and automation. It is primarily used to determine the relative position and posture between a robot's end effector (e.g., a manipulator) and its sensors (e.g., cameras, lidar, etc.). However, errors can occur during the hand-eye calibration process for various reasons, affecting the robot's control accuracy. Therefore, it is necessary to promptly determine these errors to facilitate error correction.

[0003] In related art, when determining the hand-eye calibration error, the quantification effect of the error is poor. Summary of the Invention

[0004] The present disclosure aims to solve one of the technical problems in the related art at least to a certain extent.

[0005] To this end, the purpose of the present disclosure is to propose a hand-eye calibration error quantification method, device, computer equipment and storage medium based on the eye on the hand, which can accurately and intuitively realize the quantification processing of the hand-eye calibration error, thereby effectively improving the quantification effect of the eye calibration error.

[0006] To achieve the above objectives, the first embodiment of the present disclosure proposes a hand-eye calibration error quantification method based on eye-on-hand, including:

[0007] Acquire a calibration image of a calibration plate at a specified position and an actual robotic arm posture corresponding to the calibration image, wherein the calibration plate includes a plurality of feature points, and the calibration image is acquired by a camera configured at the end of the robotic arm based on a preset robotic arm posture;

[0008] Determining the first coordinates of the feature point according to the calibration image and the actual robotic arm posture;

[0009] determining a reference coordinate of the feature point according to the first coordinate corresponding to the feature point in different calibration images;

[0010] An error quantization result is determined according to the first coordinate and the reference coordinate corresponding to each feature point.

[0011] To achieve the above-mentioned purpose, a second embodiment of the present disclosure provides a hand-eye calibration error quantification device based on eye-on-hand, comprising:

[0012] An acquisition module is configured to acquire a calibration image of a calibration plate at a specified position and an actual robotic arm posture corresponding to the calibration image, wherein the calibration plate includes a plurality of feature points, and the calibration image is acquired by a camera configured at the end of the robotic arm based on a preset robotic arm posture;

[0013] A first determination module is used to determine the first coordinates of the feature point according to the calibration image and the actual robotic arm posture;

[0014] a second determining module, configured to determine the reference coordinates of the feature point according to the first coordinates corresponding to the feature point in different calibration images;

[0015] The third determining module is configured to determine an error quantization result according to the first coordinate and the reference coordinate corresponding to each feature point.

[0016] The computer device proposed in the third embodiment of the present disclosure includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the hand-eye calibration error quantification method based on the eye on the hand proposed in the first embodiment of the present disclosure is implemented.

[0017] The fourth embodiment of the present disclosure proposes a non-temporary computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the hand-eye calibration error quantification method based on the eye on the hand as proposed in the first embodiment of the present disclosure is implemented.

[0018] A fifth aspect embodiment of the present disclosure proposes a computer program product. When the instructions in the computer program product are executed by a processor, the hand-eye calibration error quantification method based on the eye on the hand proposed in the first aspect embodiment of the present disclosure is executed.

[0019] The present disclosure provides a hand-eye calibration error quantification method, device, computer equipment, and storage medium based on eye-on-hand calibration. The method comprises obtaining a calibration image of a calibration plate at a specified position and the actual robotic arm posture corresponding to the calibration image, wherein the calibration plate includes multiple feature points, and the calibration image is acquired by a camera configured at the end of the robotic arm based on a preset robotic arm posture. The method determines the first coordinates of the feature points based on the calibration image and the actual robotic arm posture. The method also determines the reference coordinates of the feature points based on the first coordinates corresponding to the feature points in different calibration images. The method also determines the error quantification result based on the first coordinates and reference coordinates corresponding to each feature point. Thus, the method can accurately and intuitively implement quantification of hand-eye calibration errors, thereby effectively improving the quantification effect of eye calibration errors.

[0020] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present disclosure will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1 1 is a flow chart of a method for quantifying hand-eye calibration errors based on eye-on-hand proposed in one embodiment of the present disclosure;

[0023] Figure 2 is a flowchart of a hand-eye calibration error quantification method based on eye on hand proposed in another embodiment of the present disclosure;

[0024] Figure 3 1 is a flow chart of a method for quantifying and compensating hand-eye calibration errors of a depth camera based on eye-on-hand calibration proposed in the present disclosure;

[0025] Figure 4 2 is a schematic diagram of a hand-eye calibration error quantification device based on eye-on-hand proposed in one embodiment of the present disclosure;

[0026] Figure 5 A block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0027] The following describes in detail embodiments of the present disclosure, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present disclosure and are not to be construed as limiting the present disclosure. On the contrary, the embodiments of the present disclosure include all variations, modifications, and equivalents that fall within the spirit and scope of the appended claims.

[0028] Figure 1 4 is a flowchart of a method for quantifying hand-eye calibration errors based on eye-on-hand proposed in one embodiment of the present disclosure.

[0029] Among them, it should be noted that the executor of the hand-eye calibration error quantification method based on eyes on hands in this embodiment is a hand-eye calibration error quantification device based on eyes on hands, which can be implemented by software and / or hardware. The device can be configured in a computer device, and the computer device can include but is not limited to a terminal, a server side, etc. For example, the terminal can be a mobile phone, a handheld computer, etc.

[0030] like Figure 1 As shown, the hand-eye calibration error quantification method based on the eye on the hand includes:

[0031] S101: Acquire a calibration image of a calibration plate at a specified position and an actual robotic arm posture corresponding to the calibration image, wherein the calibration plate includes multiple feature points, and the calibration image is acquired by a camera configured at the end of the robotic arm based on a preset robotic arm posture.

[0032] The calibration plate may be, for example, a checkerboard, a dot array, etc., and there is no limitation to this.

[0033] The designated position refers to the position where the calibration plate is placed in the embodiment of the present disclosure.

[0034] The calibration image refers to an image containing the calibration plate.

[0035] The actual robotic arm posture may refer to the actual posture of the robotic arm when carrying a camera to capture the calibration image.

[0036] The feature points may be pre-marked points in the index plate.

[0037] The preset robotic arm posture may refer to a pre-configured posture for instructing the robotic arm to acquire the calibration plate image.

[0038] Optionally, in some embodiments, the preset manipulator poses are determined by: obtaining manipulator work requirement information; determining a manipulator calibration space; and setting multiple preset manipulator poses in the manipulator calibration space based on the manipulator work requirement information. This ensures that the resulting preset manipulator poses can meet the individual work requirements of the manipulator, effectively improving the practicality of the preset manipulator poses.

[0039] Among them, the robot arm work requirement information can be used to indicate the work requirements of the robot arm.

[0040] The robotic arm calibration space refers to the set range of all possible positions (positions and postures) that the robotic arm end (or tool) can move to when performing hand-eye calibration or robot system calibration.

[0041] In the embodiment of the present disclosure, the number of preset robot arm postures can be determined based on the robot arm work requirement information, and then multiple different postures (positions and postures) can be evenly distributed in the robot arm calibration space to ensure that the calibration can cover the full range of motion of the robot arm.

[0042] It can be understood that although the calibration image is acquired by the camera configured at the end of the robotic arm based on the preset robotic arm posture, there may be differences between the actual robotic arm posture corresponding to the calibration image and the preset robotic arm posture due to environmental factors or system errors. Therefore, in the embodiments of the present disclosure, it is necessary to determine the actual robotic arm posture corresponding to the calibration image.

[0043] In the embodiment of the present disclosure, when a calibration image of the calibration plate at a specified position and the actual robotic arm posture corresponding to the calibration image are obtained, reliable data support can be provided for subsequent hand-eye calibration error quantification processing.

[0044] S102: Determine the first coordinates of the feature point based on the calibration image and the actual robot arm posture.

[0045] The first coordinate may refer to the coordinate of the feature point in the robotic arm coordinate system.

[0046] It is understood that, in the embodiment of the present disclosure, each calibration plate may include multiple feature points. Therefore, the first coordinate corresponding to each feature point can be determined.

[0047] In the embodiment of the present disclosure, when determining the first coordinate of the feature point based on the calibration image and the actual robotic arm posture, the calibration image and the actual robotic arm posture can be used to determine the first coordinate corresponding to each feature point in the calibration plate, or the first coordinate of the feature point can be determined based on the calibration image and the actual robotic arm posture based on any other possible method, and there is no limitation to this.

[0048] That is to say, in the embodiment of the present disclosure, after obtaining the calibration image of the calibration plate at the specified position and the actual robotic arm posture corresponding to the calibration image, the first coordinates of the feature points can be determined based on the calibration image and the actual robotic arm posture, thereby realizing real-time measurement of the feature point coordinates, thereby providing reliable reference information for subsequent error quantification.

[0049] S103: Determine the reference coordinates of the feature points according to the first coordinates corresponding to the feature points in different calibration images.

[0050] The reference coordinates refer to the coordinates of the feature points used as a reference. The reference coordinates and the first coordinates should be in the same coordinate system.

[0051] In the embodiment of the present disclosure, when determining the reference coordinates of the feature points based on the first coordinates corresponding to the feature points in different calibration images, the first coordinates corresponding to the feature points in the different calibration images can be marked in the coordinate system based on a method combining numbers and shapes, and then the reference coordinates of the feature points can be determined based on the marking results. Alternatively, the first coordinates corresponding to the feature points in the different calibration images can be input into a pre-trained machine learning model to obtain the corresponding reference coordinates of the feature points. There is no restriction on this.

[0052] In the disclosed embodiment, when the reference coordinates of the feature points are determined based on the first coordinates corresponding to the feature points in different calibration images, a reliable comparison standard can be provided for the subsequent determination of the hand-eye calibration error.

[0053] S104: Determine an error quantization result according to the first coordinate and the reference coordinate corresponding to each feature point.

[0054] The error quantization result can be used to quantify the hand-eye calibration error corresponding to the calibration image.

[0055] In the embodiment of the present disclosure, when determining the error quantization result based on the first coordinate and the reference coordinate corresponding to each feature point, the first coordinate and the reference coordinate corresponding to each feature point can be input into a pre-trained machine learning model to obtain the corresponding error quantization result, or the first coordinate and the reference coordinate corresponding to each feature point can be processed based on a third-party device to obtain the error quantization result, and there is no limitation on this.

[0056] In this embodiment, a calibration image of a calibration plate at a specified position and the actual robotic arm posture corresponding to the calibration image are obtained, wherein the calibration plate includes multiple feature points, and the calibration image is acquired by a camera configured at the end of the robotic arm based on a preset robotic arm posture; the first coordinates of the feature points are determined based on the calibration image and the actual robotic arm posture; the reference coordinates of the feature points are determined based on the first coordinates corresponding to the feature points in different calibration images; and the error quantification result is determined based on the first coordinates and reference coordinates corresponding to each feature point. In this way, the hand-eye calibration error can be accurately and intuitively quantified, thereby effectively improving the quantification effect of the eye calibration error.

[0057] Figure 2 It is a flowchart of a hand-eye calibration error quantification method based on the eye on the hand proposed in another embodiment of the present disclosure.

[0058] like Figure 2 As shown, the hand-eye calibration error quantification method based on the eye on the hand includes:

[0059] S201: Acquire a calibration image of a calibration plate at a specified position and an actual robotic arm posture corresponding to the calibration image, wherein the calibration plate includes multiple feature points, and the calibration image is acquired by a camera configured at the end of the robotic arm based on a preset robotic arm posture.

[0060] The description of S201 can be found in the above embodiment and will not be repeated here.

[0061] S202: Determine the second coordinate of the feature point in the calibration plate coordinate system.

[0062] The calibration plate coordinate system refers to the coordinate system built based on the calibration plate.

[0063] The second coordinate can be used to indicate the position of the feature point in the calibration plate coordinate system.

[0064] That is, in the embodiment of the present disclosure, the second coordinate of the feature point in the calibration plate coordinate system can be determined first, thereby providing reliable reference information for the subsequent determination of the first coordinate of the feature point in the robot arm coordinate system.

[0065] S203: Determine camera extrinsic parameters based on the calibration image.

[0066] Among them, the camera extrinsic parameters can be used to describe the transformation relationship between the camera coordinate system and the world coordinate system.

[0067] In the disclosed embodiment, when the camera extrinsic parameters are determined based on the calibration image, reliable data support can be provided for the subsequent acquisition of the hand-eye calibration results.

[0068] S204: Obtain hand-eye calibration results based on camera extrinsic parameters and actual robotic arm posture.

[0069] Among them, the hand-eye calibration result can be the fixed spatial transformation relationship between the camera (or visual sensor) and the end effector of the robotic arm solved through the calibration process.

[0070] In the embodiment of the present disclosure, when the hand-eye calibration result is obtained based on the camera external parameters and the actual robotic arm posture, reliable reference information can be provided for the subsequent determination of the first coordinate of the feature point in the robotic arm coordinate system.

[0071] S205: Process the second coordinate according to the camera extrinsic parameters and the hand-eye calibration result to determine the first coordinate of the feature point in the robotic arm coordinate system.

[0072] That is, in the disclosed embodiment, after obtaining a calibration image of the calibration plate at a specified position and the actual robotic arm posture corresponding to the calibration image, the second coordinate of the feature point in the calibration plate coordinate system can be determined; the camera extrinsic parameters are determined based on the calibration image; the hand-eye calibration result is obtained based on the camera extrinsic parameters and the actual robotic arm posture; and the second coordinate is processed based on the camera extrinsic parameters and the hand-eye calibration result to determine the first coordinate of the feature point in the robotic arm coordinate system. Thus, the second coordinate can be converted by combining the camera extrinsic parameters and the hand-eye calibration result, thereby ensuring the accuracy of the obtained first coordinate.

[0073] S206: Perform mean filtering and optimal estimation on the first coordinates corresponding to the feature points in different calibration images to obtain the reference coordinates of the feature points.

[0074] That is, after determining the first coordinates, in the disclosed embodiments, the first coordinates corresponding to the feature points in different calibration images can be subjected to mean filtering and optimal estimation to obtain the reference coordinates of the feature points. Thus, the practicality and reliability of the obtained reference coordinates can be effectively improved through mean filtering and optimal estimation.

[0075] S207: Determine the Euclidean distance between each first coordinate corresponding to the feature point and the reference coordinate.

[0076] The Euclidean distance can be used to indicate the error between the first coordinate and the reference coordinate corresponding to the feature point.

[0077] S208: Determine an average of the multiple Euclidean distances as an error quantization result.

[0078] Optionally, in some embodiments, the mode or median of multiple Euclidean distances may be determined as the error quantization result, which is not limited.

[0079] That is, in the disclosed embodiment, after determining each first coordinate and reference coordinate corresponding to a feature point, the Euclidean distance between each first coordinate and the reference coordinate corresponding to the feature point can be determined; and the average of the multiple Euclidean distances can be determined as the error quantization result. This effectively improves the indicative effect of the obtained error quantization result.

[0080] S209: Constructing an error compensation matrix corresponding to the calibration image based on the first coordinates and the reference coordinates.

[0081] In the embodiment of the present disclosure, when constructing an error compensation matrix corresponding to the calibration image based on the first coordinates and the reference coordinates, the error compensation matrix corresponding to the calibration image can be constructed based on the least squares method, or the first coordinates and the reference coordinates can be input into a pre-trained machine learning model to obtain the corresponding error compensation matrix.

[0082] Optionally, in some embodiments, the scope of the error compensation matrix is the neighborhood of the calibration image corresponding to the actual robotic arm posture. In this way, the compensation effect of the obtained error compensation matrix can be guaranteed.

[0083] In the embodiment of the present disclosure, a corresponding error compensation matrix may be constructed for each calibration image.

[0084] S210: Generate a posture correction instruction for the end effector of the robot arm based on the error compensation matrix.

[0085] Among them, the posture correction instruction refers to the control instruction generated based on the error compensation matrix for controlling the end effector of the robot arm to achieve posture correction.

[0086] That is, after determining the feature points corresponding to each first coordinate and reference coordinate, the disclosed embodiment can also construct an error compensation matrix corresponding to the calibration image based on the first coordinate and reference coordinate. Based on the error compensation matrix, the position correction instructions for the robot end effector are generated. This can correct the hand-eye calibration errors, thereby effectively improving the reliability and robustness of the robot control process.

[0087] In this embodiment, the second coordinate of a feature point in the calibration plate coordinate system is determined; camera extrinsic parameters are determined based on the calibration image; a hand-eye calibration result is obtained based on the camera extrinsic parameters and the actual robotic arm posture; and the second coordinate is processed based on the camera extrinsic parameters and the hand-eye calibration result to determine the first coordinate of the feature point in the robotic arm coordinate system. Thus, the second coordinate can be transformed by combining the camera extrinsic parameters and the hand-eye calibration result, thereby ensuring the accuracy of the obtained first coordinate. The first coordinates corresponding to the feature point in different calibration images are mean filtered and optimally estimated to obtain the reference coordinates of the feature point. This mean filtering and optimal estimation effectively improve the practicality and reliability of the obtained reference coordinates. The Euclidean distance between each first coordinate corresponding to the feature point and the reference coordinate is determined; and the mean of multiple Euclidean distances is determined as the error quantification result. This effectively improves the indication effect of the obtained error quantification result. An error compensation matrix corresponding to the calibration image is constructed based on the first coordinates and the reference coordinates; and based on the error compensation matrix, a posture correction instruction for the robotic arm end effector is generated. In this way, the hand-eye calibration error can be corrected, thereby effectively improving the reliability and robustness of the robotic arm control process.

[0088] In summary of the above embodiments, the present disclosure proposes a method for quantifying and compensating hand-eye calibration errors of a depth camera based on eye-on-hand. Figure 3 As shown, Figure 3 The following is a flow chart of the method for quantifying and compensating the hand-eye calibration error of a depth camera based on the eye-on-hand method proposed in this disclosure.

[0089] 1. Posture setting and data collection:

[0090] Evenly set the hand-eye calibration poses in the robot arm calibration space, and the number of poses is n (n≥5);

[0091] The robot is driven to move with the camera to n different positions to collect the calibration plate image and its corresponding robotic arm position.

[0092] 2. Hand-eye calibration solution:

[0093] Estimate camera extrinsic parameters Rc and tc based on the calibration image;

[0094] Nonlinearly optimize the AX=XB equation to obtain the hand-eye calibration results R and t.

[0095] 3. Establishment of standard coordinate system:

[0096] Calculate the coordinates current_base of the calibration plate corresponding to each calibration image in the base coordinate system:

[0097] current_base=[R|t]*[Rc|tc]*P

[0098] Where P is the coordinate of the feature point of the calibration plate in the calibration plate coordinate system.

[0099] Perform mean filtering and optimal estimation on multiple sets of current_base to generate the reference coordinate standard_base:

[0100] standard_base=1 / n*Σcurrent_base

[0101] Where n is the number of calibration plate images.

[0102] ||standard_base(i)-standard_base(i+1)||2=s

[0103] standard_base(i) is the i-th feature point in standard_base, and s is the actual physical length of the feature point in the calibration plate.

[0104] 4. Error quantification:

[0105] Calculate the error between current_base and standard_base for each calibration image:

[0106] error=1 / m*Σ||current_base(i)-standard_base(i)|| 2

[0107] Where m is the number of feature points in the calibration plate.

[0108] The error value obtained is the coordinate error of the current image that unifies the translation and rotation errors to the robot base coordinate system.

[0109] 5. Linear compensation matrix calculation:

[0110] Construct the error compensation matrix error_mat = [R|t] and solve it by the least squares method:

[0111] Σ||error_mat*current_base-standard_base||2→min

[0112] The error_mat matrix parameters are optimized using SVD decomposition or pseudo-inverse method to obtain the error compensation matrix of the robotic arm posture corresponding to the current calibration image.

[0113] Traverse all calibration images, and the scope of the error compensation matrix of each calibration image is the neighborhood of its corresponding pose.

[0114] 6. Error compensation implementation:

[0115] Embed the error_mat matrix into the robot controller to correct the end effector pose instructions in real time:

[0116] P_corrected=error_mat*P_original

[0117] Figure 4 Schematic diagram of the structure of a hand-eye calibration error quantification device based on eye on hand proposed in one embodiment of the present disclosure.

[0118] like Figure 4 As shown, the hand-eye calibration error quantification device 40 based on eye-on-hand includes:

[0119] An acquisition module 401 is configured to acquire a calibration image of a calibration plate at a specified position and an actual robotic arm posture corresponding to the calibration image, wherein the calibration plate includes multiple feature points and the calibration image is acquired by a camera configured at the end of the robotic arm based on a preset robotic arm posture;

[0120] A first determination module 402 is used to determine the first coordinates of the feature point according to the calibration image and the actual robot arm posture;

[0121] A second determining module 403 is configured to determine the reference coordinates of the feature point based on the first coordinates corresponding to the feature point in different calibration images;

[0122] The third determining module 404 is configured to determine an error quantization result according to the first coordinate and the reference coordinate corresponding to each feature point.

[0123] It should be noted that the aforementioned explanation of the hand-eye calibration error quantization method based on eyes on hands is also applicable to the hand-eye calibration error quantization device based on eyes on hands in this embodiment, and will not be repeated here.

[0124] In this embodiment, a calibration image of a calibration plate at a specified position and the actual robotic arm posture corresponding to the calibration image are obtained, wherein the calibration plate includes multiple feature points, and the calibration image is acquired by a camera configured at the end of the robotic arm based on a preset robotic arm posture; the first coordinates of the feature points are determined based on the calibration image and the actual robotic arm posture; the reference coordinates of the feature points are determined based on the first coordinates corresponding to the feature points in different calibration images; and the error quantification result is determined based on the first coordinates and reference coordinates corresponding to each feature point. In this way, the hand-eye calibration error can be accurately and intuitively quantified, thereby effectively improving the quantification effect of the eye calibration error.

[0125] Figure 5 A block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure is shown. Figure 5The computer device 12 shown is only an example and should not bring any limitation to the functionality and scope of use of the embodiments of the present disclosure.

[0126] like Figure 5 As shown, computer device 12 is implemented as a general-purpose computing device. Components of computer device 12 may include, but are not limited to, one or more processors or processing units 16, system memory 28, and a bus 18 that connects various system components (including system memory 28 and processing unit 16).

[0127] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of such architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnection (PCI) bus.

[0128] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0129] The memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be configured to read and write non-removable, non-volatile magnetic media ( Figure 5 Not shown, often called a "hard drive").

[0130] although Figure 5Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a Compact Disc Read Only Memory (hereinafter referred to as: CD-ROM), a Digital Video Disc Read Only Memory (hereinafter referred to as: DVD-ROM), or other optical media) may be provided. In these cases, each drive can be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the various embodiments of the present disclosure.

[0131] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 42 generally implement the functions and / or methods of the embodiments described herein.

[0132] The computer device 12 can also communicate with one or more external devices 14 (e.g., a keyboard, pointing device, display 24, etc.), one or more devices that enable human interaction with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). This communication can occur via an input / output (I / O) interface 22. Furthermore, the computer device 12 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with the other modules of the computer device 12 via a bus 18. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with the computer device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0133] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the hand-eye calibration error quantification method based on the eye on the hand mentioned in the above embodiment.

[0134] In order to implement the above embodiments, the present disclosure also proposes a non-temporary computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it implements the hand-eye calibration error quantification method based on the eye on the hand as proposed in the above embodiments of the present disclosure.

[0135] In order to implement the above embodiments, the present disclosure also proposes a computer program product. When the instruction processor in the computer program product executes, it performs the hand-eye calibration error quantization method based on eye on hand proposed in the above embodiments of the present disclosure.

[0136] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow from the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0137] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0138] It should be noted that, in the description of this disclosure, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In addition, in the description of this disclosure, unless otherwise specified, "plurality" means two or more.

[0139] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0140] It should be understood that various parts of the present disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0141] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0142] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0143] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0144] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0145] Although the embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are illustrative and are not to be construed as limitations on the present disclosure. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present disclosure.

Claims

1. A hand-eye calibration error quantification method based on eye-on-hand, characterized in that: include: Acquire a calibration image of a calibration plate at a specified position and an actual robotic arm posture corresponding to the calibration image, wherein the calibration plate includes a plurality of feature points, and the calibration image is acquired by a camera configured at the end of the robotic arm based on a preset robotic arm posture; Determining the first coordinates of the feature point according to the calibration image and the actual robotic arm posture; determining a reference coordinate of the feature point according to the first coordinate corresponding to the feature point in different calibration images; An error quantization result is determined according to the first coordinate and the reference coordinate corresponding to each feature point.

2. The method according to claim 1, wherein The preset robotic arm posture is determined based on the following method: Obtaining the work requirement information of the robotic arm; Determine the robotic arm calibration space; According to the working requirement information of the robotic arm, a plurality of preset robotic arm postures are set in the robotic arm calibration space.

3. The method according to claim 1, wherein The determining the first coordinate of the feature point according to the calibration image and the actual robotic arm posture includes: Determining a second coordinate of the feature point in the calibration plate coordinate system; Determining camera extrinsics based on the calibration image; Obtaining a hand-eye calibration result based on the camera extrinsic parameters and the actual robotic arm posture; The second coordinate is processed according to the camera extrinsic parameter and the hand-eye calibration result to determine the first coordinate of the feature point in the robotic arm coordinate system.

4. The method according to claim 1, wherein The determining the reference coordinates of the feature point according to the first coordinates corresponding to the feature point in different calibration images includes: Mean filtering and optimal estimation are performed on the first coordinates corresponding to the feature points in different calibration images to obtain the reference coordinates of the feature points.

5. The method according to claim 1, wherein The determining the error quantization result according to the first coordinate and the reference coordinate corresponding to each feature point includes: Determine the Euclidean distance between each of the first coordinates corresponding to the feature point and the reference coordinate; An average of the plurality of Euclidean distances is determined as the error quantization result.

6. The method according to claim 1, wherein The method further comprises: constructing an error compensation matrix corresponding to the calibration image based on the first coordinates and the reference coordinates; Based on the error compensation matrix, a posture correction instruction for the end effector of the robotic arm is generated.

7. The method according to claim 6, wherein in, The scope of the error compensation matrix is the neighborhood of the calibration image corresponding to the actual robotic arm posture.

8. A hand-eye calibration error quantification device based on eye-on-hand, characterized in that: include: An acquisition module is configured to acquire a calibration image of a calibration plate at a specified position and an actual robotic arm posture corresponding to the calibration image, wherein the calibration plate includes a plurality of feature points, and the calibration image is acquired by a camera configured at the end of the robotic arm based on a preset robotic arm posture; A first determination module is used to determine the first coordinates of the feature point according to the calibration image and the actual robotic arm posture; a second determining module, configured to determine the reference coordinates of the feature point according to the first coordinates corresponding to the feature point in different calibration images; The third determining module is configured to determine an error quantization result according to the first coordinate and the reference coordinate corresponding to each feature point.

9. A computer device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: in, The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

Images