Robot and vision matching calibration method and robot

Abstract

The invention provides a robot and vision matching calibration method and a robot, which can be widely applied to the field of automatic calibration, and the robot and vision matching calibration method comprises the following steps: obtaining the initial position of the robot; determining a preset pose of the robot according to the initial position; wherein the preset pose is the position and the pose of the robot during calibration; the preset pose is transmitted to the robot through a communication module, so that the robot moves to the preset pose; collecting image information of the robot at the preset pose, and preprocessing the image information to obtain a first image; and calibrating the robot through the first image. According to the method, automatic calibration of the robot can be realized, the problem that the calibration process depends on manpower in the related technology is relieved, and the calibration efficiency and accuracy are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic calibration, in particular to a robot and a vision matching calibration method and robot. BACKGROUND

[0002] With the development of science and technology, robots are increasingly widely used in various industries. For example, in the manufacturing industries of automobiles, electronics, furniture, etc., robots are used for repetitive and high-precision work such as carrying, assembling, welding, and painting. In the logistics field, robots can perform sorting, carrying, and boxing work. The main reason for robot calibration is to ensure that the robot can accurately perceive the environment, locate itself, and perform tasks, which is the basis for the work of the robot. Through robot calibration, the motion accuracy and repeatability of the robot can be ensured, and the production efficiency and product quality can be improved.

[0003] In related technologies, the calibration of the robot is usually performed by manually adjusting the motion of the robot with a manual teach pendant and by copying data. SUMMARY

[0004] The main purpose of the present application is to provide a robot and a vision matching calibration method and robot to at least solve the problem of low detection accuracy in the prior art.

[0005] To achieve the above-mentioned purpose, according to one aspect of the present application, a robot and a vision matching calibration method are provided, which comprises: obtaining an initial position of a robot; determining a preset pose of the robot according to the initial position; wherein the preset pose is the position and attitude of the robot at the time of calibration; transmitting the preset pose to the robot through a communication module to make the robot move to the preset pose; collecting image information of the robot at the preset pose, and pre-processing the image information to obtain a first image; and calibrating the robot through the first image. The present application determines the preset pose of the robot through the initial position of the robot, transmits the preset pose to the robot, makes the robot move to the preset pose, collects the image information of the robot, and calibrates the robot. The present application can realize automatic calibration of the robot, which is beneficial to alleviate the problem that the calibration process in related technologies depends on manual work, and is beneficial to improve the calibration efficiency and accuracy.

[0006] Optionally, the calibration of the robot through the first image comprises:

[0007] calibrating the internal parameters of the image acquisition unit through the first image to obtain calibrated internal parameters;

[0008] Transforming a position of the fixed point in the first image by the internal reference to obtain three-dimensional position information;

[0009] According to the three-dimensional position information, a first coordinate system is constructed;

[0010] According to the preset pose and the first coordinate system, the robot and the image acquisition unit are calibrated.

[0011] Optionally, the pre-processing of the image information to obtain a first image comprises:

[0012] First corner detection and / or sub-pixel first corner detection are performed on the image information;

[0013] If no first corner is detected in a second image, the second image is removed from the image information to obtain a first image.

[0014] Optionally, the initial position comprises a first position, a second position and a third position, and the initial position of the robot comprises:

[0015] A plurality of second corners of the image acquisition unit on the projection surface are acquired;

[0016] The robot is adjusted to move the calibration board to reach the three second corners respectively, and the first position, the second position and the third position are obtained.

[0017] Optionally, the initial position comprises a first position, a second position and a third position, and the determination of the preset pose of the robot according to the initial position comprises:

[0018] A first vector axis is determined according to the first position and the second position, and a second vector axis is determined according to the third position and the second position;

[0019] Position sampling is performed along the first vector axis and / or the second vector axis to obtain a plurality of preset positions;

[0020] A preset pose is obtained by rotating a second coordinate system by a preset angle based on each preset position in the plurality of preset positions;

[0021] The preset pose is determined according to the preset position and the preset pose.

[0022] Optionally, before the determination of the preset pose according to the preset position and the preset pose, the method further comprises:

[0023] Inverse kinematics is used to inversely solve the preset position and the preset pose to obtain a joint angle value;

[0024] update the preset position and the preset pose according to the joint angle value.

[0025] Optionally, the updating the preset position and the preset pose according to the joint angle value comprises:

[0026] if the joint angle value exceeds a preset joint limit, taking the first preset joint or the second preset joint as a new joint angle value, and solving the new joint angle value based on forward kinematics to obtain a new preset position and a new preset pose;

[0027] if the joint angle value does not exceed the preset joint limit, maintaining the preset position and the preset pose.

[0028] Optionally, the inverse solving the preset position and the preset pose based on inverse kinematics to obtain a joint angle value comprises:

[0029] generating a rotation matrix according to the preset pose, and generating a position vector according to the preset position;

[0030] inverse solving the rotation matrix and the position vector based on an analytical method to obtain a joint angle value.

[0031] Optionally, the collecting the image information of the robot at the preset pose comprises:

[0032] receiving an arrival signal of the robot moving to the preset pose, and obtaining an accumulated time length;

[0033] if the accumulated time length is equal to a preset time length, clearing the accumulated time length, and collecting the image information of the robot at the preset pose by an image acquisition unit; wherein the accumulated time length is a time length elapsed since the arrival signal is received, and the preset time length is a time length of image stabilization.

[0034] To achieve the above object, according to another aspect of the present application, a robot is provided, which is calibrated by the robot and the visual matching calibration method described above.

[0035] By applying the technical solution of the present application, the preset pose of the robot is determined through the initial position of the robot, and the preset pose is transmitted to the robot, so that the robot moves to the preset pose, the image information of the robot is collected, and the robot is calibrated. The present application can realize automatic calibration of the robot, which is conducive to alleviating the problem that the calibration process in the related art depends on manual operation, and is conducive to improving the calibration efficiency and accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0036] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0037] Figure 1 A schematic diagram illustrating an application scenario of a robot and vision matching calibration method provided in an embodiment of this application is shown.

[0038] Figure 2 A flowchart illustrating a robot and vision matching calibration method provided in an embodiment of this application is shown.

[0039] Figure 3 A schematic diagram of the system structure of a robot calibration process provided according to an embodiment of this application is shown;

[0040] Figure 4 A flowchart illustrating a robot calibration process according to an embodiment of this application is shown;

[0041] The above figures include the following reference numerals:

[0042] 102, Processor; 104, Memory; 106, Transmission device; 108, Input / output device; 301, Camera; 302, Robot body; 303, Base coordinate system. Detailed Implementation

[0043] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0044] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0045] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0046] As described in the background section, related technologies involve placing a calibration board within the robot's working area under predetermined constraints, determining the positional components of image acquisition points, acquiring images of the calibration board and laser stripes, and calculating camera intrinsic parameters and hand-eye relationships from the acquired images. However, calibration based on manual adjustments suffers from low efficiency.

[0047] To address the aforementioned issues, embodiments of this application provide a calibration method for robot and vision matching, enabling automatic calibration and improving system calibration efficiency. It utilizes a hand-eye calibration method incorporating 3D sensor information to enhance calibration accuracy. This application provides an automatic calibration scheme for robot and vision matching, eliminating the need for manual teaching calibration and data transcription, thus improving the efficiency of hand-eye calibration. It constructs a description of the camera's spatial pose by combining the camera's 3D information, and achieves automatic calibration of the 3D sensor and the robot arm by combining this description with received robot arm pose information, thereby improving calibration accuracy.

[0048] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0049] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a robot and vision matching calibration method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more...Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0050] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the robot and vision matching calibration method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-described networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the above-described networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0051] This embodiment provides a calibration method for robot and vision matching that runs on a mobile terminal, computer terminal or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although the logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0052] Figure 2 This is a flowchart illustrating a robot and vision matching calibration method according to an embodiment of this application. The robot and vision matching calibration method includes:

[0053] Step S100: Obtain the robot's initial position; determine the robot's preset pose based on the initial position; wherein, the preset pose is the robot's position and orientation during calibration;

[0054] Step S200: The preset pose is transmitted to the robot through the communication module so that the robot moves to the preset pose;

[0055] At step S300, image information of the robot at the preset pose is collected, and the image information is preprocessed to obtain a first image.

[0056] At step S400, the robot is calibrated based on the first image.

[0057] The initial position in the present application is a plurality of first positions of the robot corresponding to the corner points, and the preset pose is determined according to the relationship between the first positions. The preset pose is a position reached by the robot through posture adjustment and movement, and the preset pose is the pose of the robot during calibration. In some embodiments, the preset pose can be the pose of the robot during calibration. The preprocessing of the image information in the present application can be normalization processing or rejection processing, and the specific processing manner of the preprocessing is not limited in the present application. The robot is calibrated based on the position of the calibration board in the first image. The determination of the preset pose of the robot, the data sending, the image data receiving and the processing can be realized by the host computer, and the automatic calibration of the robot is realized by the host computer, which is beneficial to alleviate the low efficiency of manual teaching calibration in the related art, and improve the calibration accuracy and precision.

[0058] Optionally, the robot is calibrated based on the first image, including:

[0059] The intrinsic parameters of the image acquisition unit are calibrated based on the first image to obtain calibrated intrinsic parameters;

[0060] The positions of the calibration points in the first image are converted into point cloud based on the intrinsic parameters to obtain three-dimensional position information;

[0061] A first coordinate system is constructed based on the three-dimensional position information;

[0062] The robot and the image acquisition unit are calibrated based on the preset pose and the first coordinate system.

[0063] The first image in the present application includes a picture and a depth map. The camera is calibrated based on the intrinsic parameters, the positions of the calibration points in the first image are corresponded to the pixel positions on the depth map based on the calibrated intrinsic parameters, the point cloud is converted, and the three-dimensional position information is obtained. The calibration points in the present application are the positions of the corner points of the calibration board in the first image acquired by the image acquisition unit. The image acquisition unit can be a camera. The first coordinate system in the present application is used to represent the relationship between the image acquisition unit and the calibration board. The robot and the image acquisition unit can be calibrated based on the preset pose and the first coordinate system by a calibration system, or can be calibrated by Tsai’s method. The robot and the image acquisition unit are automatically calibrated based on the first image and the actual three-dimensional position, and the accuracy and efficiency of the calibration are improved.

[0064] Optionally, the image information is preprocessed to obtain a first image, including:

[0065] performing first corner detection and / or sub-pixel first corner detection on the image information;

[0066] if no first corner is detected in the second image, eliminating the second image from the image information to obtain the first image.

[0067] It can be understood that, since the variation range of some poses may be too large, the camera cannot detect the first corner, and therefore it is necessary to eliminate the picture in which the first corner is not detected and the pose data at the corresponding position in the list. Specifically, the first corner is a point in a local neighborhood of an image that has significant gray level changes in two directions, usually appearing at the corners of an object, intersections or places with dramatic changes in texture. The result of ordinary corner detection is integer pixel coordinates, but the actual corner point may be located between pixels. In order to improve the accuracy (especially for high-precision tasks such as camera calibration), it is necessary to locate the corner point to 0.1 pixel level or even higher, which is sub-pixel first corner detection. On the other hand, the present application eliminates the preset pose corresponding to the first corner in the sequence. Of course, the first in the first corner of the present application is used to distinguish different corners in the present application, and is not a limitation on the data or properties of the corner. The second image in the present application is any one of the image information and no first corner is detected in the image, that is, the image information of the present application includes a plurality of images, and the first image in the present application is all the images remaining after the second image is eliminated from the image information. The present application eliminates the picture in which the corner point cannot be detected, uses the remaining second image for robot calibration, and improves the calibration accuracy.

[0068] Optionally, the initial position includes a first position, a second position and a third position, and obtaining the initial position of the robot includes:

[0069] obtaining a plurality of second corner points of the image acquisition unit on the projection surface;

[0070] adjusting the robot to drive the calibration board to move so that the calibration board reaches the three second corner points respectively, and obtaining the first position, the second position and the third position.

[0071] In some embodiments, the initial positions include a first position, a second position and a third position, of course, the first, second and third in the present application do not limit the specific number of initial positions. In some embodiments, the image acquisition unit in the present application has four second corner points on the projection surface, which can be used for subsequent position marker reference points. The present application adjusts the robot to drive the calibration board to move, so that the calibration board reaches three second corner points and the calibration board is completely in the field of view of the image acquisition unit at the second corner point. That is, the calibration board reaches the first second corner point and the calibration board is completely in the field of view of the image acquisition unit at the second corner point, to obtain the first position; the calibration board reaches the second second corner point and the calibration board is completely in the field of view of the image acquisition unit at the second corner point, to obtain the second position; the calibration board reaches the third second corner point and the calibration board is completely in the field of view of the image acquisition unit at the second corner point, to obtain the third position. In some embodiments, the calibration board is at the end flange of the robot, and the calibration board is moved by moving the robot. The present application sets multiple positions of the initial position, which is convenient for subsequent determination of the preset pose, and then calibrates the robot through the preset pose.

[0072] Optionally, the initial positions include a first position, a second position and a third position, according to the initial positions, determining a preset pose of the robot, comprising:

[0073] According to the first position and the second position, a first vector axis is determined; according to the third position and the second position, a second vector axis is determined;

[0074] Position sampling is performed along the first vector axis and / or the second vector axis to obtain a plurality of preset positions;

[0075] On the basis of each preset position in the plurality of preset positions, a preset angle is rotated along the second coordinate system to obtain a preset attitude;

[0076] According to the preset position and the preset attitude, a preset pose is determined.

[0077] The present application determines a first vector axis and a second vector axis through any two positions of the three initial positions, and the two positions corresponding to the first vector axis and the second vector axis are not completely consistent. The present application can uniformly sample along the first vector axis and uniformly sample along the second vector axis to obtain a plurality of preset positions. In some embodiments, 16 preset positions can be set. At each preset position, a preset angle is rotated along the three axes of the second coordinate system to obtain a preset attitude. The present application stores the preset position and the preset attitude in a list, which is convenient for subsequent related calculation and calibration.

[0078] Optionally, before determining the preset pose according to the preset position and the preset attitude, the method further comprises:

[0079] The preset position and the preset pose are inversely solved based on inverse kinematics to obtain a joint angle value;

[0080] The preset position and the preset pose are updated according to the joint angle value.

[0081] The application determines the joint angle value through inverse kinematics analysis. The joint angle value is the joint angle required by the robot to reach based on the preset pose. The joint angle is screened according to the specific hardware limitations of the robot. If the joint angle is unattainable by the robot, the joint angle is updated, and the preset pose is further updated. Through the updating operation of the preset position and the preset pose, the application realizes the purpose that the robot can reach, which is convenient for subsequent image acquisition and calibration.

[0082] Optionally, updating the preset position and the preset pose according to the joint angle value comprises:

[0083] If the joint angle value exceeds the preset joint limit, the first preset joint or the second preset joint is taken as a new joint angle value, and the new joint angle value is solved based on forward kinematics to obtain a new preset position and a new preset pose.

[0084] If the joint angle value does not exceed the preset joint limit, the preset position and the preset pose are maintained.

[0085] The preset joint limit is related to the hardware of the robot itself. Different types of robots correspond to different preset joint limits. The application does not limit the specific value of the preset joint limit. In the application, the first preset joint is the lower limit of the joint that the robot can reach, and the second preset joint is the upper limit of the joint that the robot can reach. In the application, the preset joint limit can be regarded as an interval composed of the first preset joint and the second preset joint. Specifically, if the joint angle value exceeds the preset joint limit and the joint angle value is at the end of the first preset joint, the first preset joint is taken as a new joint angle value. If the joint angle value exceeds the preset joint limit and the joint angle value is at the end of the second preset joint, the second preset joint is taken as a new joint angle value. The application solves the new joint angle value through forward kinematics to obtain a new preset pose. The application adjusts the preset pose to ensure that the preset pose is a pose that the robot can reach, which is convenient for subsequent image acquisition and calibration and improves the calibration accuracy.

[0086] Optionally, the preset position and the preset pose are inversely solved based on inverse kinematics to obtain a joint angle value, comprising:

[0087] A rotation matrix is generated according to the preset pose, and a position vector is generated according to the preset position.

[0088] The rotation matrix and the position vector are inversely solved based on an analytical method to obtain a joint angle value.

[0089] Optionally, the image information of the robot in the preset pose is collected, comprising:

[0090] receiving a reaching signal of the robot moving to the preset pose, and acquiring an accumulated time length;

[0091] if the accumulated time length is equal to a preset time length, clearing the accumulated time length, and collecting image information of the robot at the preset pose by the image acquisition unit; wherein the accumulated time length is a time length elapsed since the reaching signal is received, and the preset time length is a time length of image stabilization.

[0092] In the present application, the accumulated time length is a time length elapsed since the reaching signal is received. By setting the preset time length, the present application improves the definition and accuracy of the collected image information, and further improves the accuracy of the calibration.

[0093] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a robot is provided, which is calibrated by the robot and the visual matching calibration method described above.

[0094] In order to enable those skilled in the art to more clearly understand the technical solutions of the present application, the implementation process of the robot and the visual matching calibration method of the present application will be described in detail below with specific embodiments.

[0095] The present application realizes the pre-planning of the robot motion position by the host computer, realizes the automatic image acquisition at the calibration position sending and the arrival position by the communication module, realizes the automatic information interaction of the robot and the 3D camera, and completes the determination of the system calibration relationship; the description method of the camera space 3D pose is established by detecting the specific marker of the calibration board, and the accurate system calibration is realized by combining the corresponding mechanical arm accurate position solution. Referring to the calibration system structure shown in Figure 3 and the calibration process shown in Figure 4 , specifically comprising the following steps:

[0096] Firstly, for the application scene of the six-degree-of-freedom industrial robot combined with 3D vision, the relative position of the robot and the 3D camera in space is determined, the field of view range of the camera is projected onto a plane as a rectangular region, and the positions of the four corner points of the rectangular region are taken as reference points, as shown in Figure 3 .

[0097] Further, the calibration board is fixed at the end flange of the 6-degree-of-freedom industrial robot 302, the robot is moved by the teach pendant to make the calibration board reach three corner points in the rectangular region of the camera 301 field of view, and it is ensured that the calibration board is completely in the camera field of view.

[0098] In the present application, the movement is manual teaching, the three corner points refer to any three corner points in the camera projection region, the teach pendant is a tool for the user to control the robot motion, the calibration board, the teach pendant and the projection region points are shown in the attached Figure 3 .

[0099] Further, the positions of the three corner points p1, p2, p3 (i.e. initial positions) of the mechanical arm are recorded, and the positions of the three corner points are input into the calibration system, 16 positions in 4 rows and 4 columns are obtained by uniform sampling method, and the positions are stored in a list;

[0100] The region corresponding to the uniform sampling method is and The region determined by the two vector axes, and the region is uniformly sampled along the two determined axes, and the positions obtained by uniform sampling are stored in a data list.

[0101] Further, system calibration requires the robot to move to different positions and different postures. For the posture part, the robot base coordinate system 303 is taken as the reference, and for each position corresponding to the previous step, the Z axis is rotated by a certain angle a, the X axis is rotated by a certain angle β, and the Y axis is rotated by a certain angle γ. 16 posture information of the robot is generated in turn, and the position and posture information is stored in the system list (i.e. preset position and posture).

[0102] Further, in order to ensure that the 16 pre-generated postures of the robot can be run without triggering the limit, the calibration system converts the posture into a matrix before the robot runs, obtains the required 6 joint angle values of the robot through inverse kinematics, compares the joint angle with the robot limit value input into the system in advance, for the joint value that will exceed the limit, the system will adjust the joint value to a new joint value greater than the minimum threshold or (less than the threshold) greater than the maximum threshold. Degree, for the joint value that does not exceed the limit, keep it unchanged, update the posture that the robot needs to move to through forward kinematics of the robot again, so as to ensure that the pre-generated position of the robot can be reached.

[0103] The base coordinate system mentioned in the present application is the reference system of the system, which generally refers to the world coordinate system selected in advance. On the other hand, the base coordinate system refers to the coordinate system established at the base of the robot. The 16 posture information in the present application is assigned in turn based on the position in the foregoing, for example, the i( )th posture, , , .

[0104] The inverse solution process of inverse kinematics is as follows:

[0105] The position and attitude angle are converted into a matrix, taking the rx, ry, and rz type attitude angle in the present application as an example:

[0106] ;

[0107] ;

[0108] ;

[0109] ;

[0110] ;

[0111] ;

[0112] wherein R is a 3 3 rotation matrix, t is a position vector, which are obtained from the above-mentioned pose information.

[0113] The inverse solution method can be summarized as solving wherein the right side of the equation is a positive kinematics solution matrix, and the specific steps of the inverse solution method are as follows:

[0114] Inverse kinematics analysis is a process of solving joint angles through mathematical operation under the condition that the end pose in the base coordinate system is known. Current inverse kinematics mathematical solution methods include: analytical method, geometric method, algebraic method (note: the sufficient condition for the existence of closed solution of the operating arm with 6 rotary joints is that the axes of three adjacent joints intersect at a point, and currently six degrees of freedom industrial robots basically have this feature).

[0115] Therefore, the input of the inverse kinematics model is , and the output is Theta1~Theta6; it is known that the GR35 robot is a ZYX type right-hand system serial robot, and it is assumed that the data read from the teach pendant is XYZABC, and the numerical value of can be obtained as follows:

[0116] ;

[0117] wherein , , , , and the obtained result is in the form of a homogeneous transformation matrix:

[0118] ;

[0119] At this point, the solution problem is transformed into solving the equation: , and the solving process is omitted here, and the inverse solution result of GR35 (self-made system) is directly given as follows:

[0120] ;

[0121] ;

[0122] wherein, ;

[0123] ;

[0124] Where K2 is in polynomial fraction form, expressed as: ;

[0125] Since the GR35's two-axis zero-position offset angle is -90 degrees, the actual angle displayed on the teach pendant is... The solution is as follows:

[0126] ;

[0127] Parametric observation of the solutions for the first three joint axes of the GR35 reveals that the fractional polynomial parameters are only related to the link parameters and... , , (The position of the robot arm end) is related to the angles of the first three axes. It can be understood that the position of the robot arm end determines the angles of the first three axes. And the structure of the solution can be used to deduce that there are four sets of solutions for the robot's first three axes (solutions that do not meet the conditions need to be eliminated).

[0128] ;

[0129] in, Theta23 in the formula is known when solving for Theta3 above;

[0130] Similarly, Theta5 and Theta6 can be obtained. The specific solution steps are omitted here. Observing the structure of the solutions for the last three axes, it can be seen that the parameters of the solution only relate to... The rotational part of the matrix is ​​relevant; the end effector posture of the robotic arm determines the angles of the three rear axes.

[0131] The reason why this application updates the preset position and the preset posture based on the joint angle value is that the mechanical structure of the robot has hard limits, so the joint value has a solution but the actual limit constraint makes it not necessarily reachable.

[0132] Furthermore, the calibration system software includes a socket communication module, which acts as a server to send the pose information of the list to the robot controller. After receiving the data, the robot parses it and moves to the corresponding preset position. After reaching the position, the robot teach pendant program sets the robot to send an arrival signal to the server and pause for a time t. After receiving the signal, the calibration system will open the 3D camera sensor to acquire the data stream. When it detects that the pixel position of the calibration board remains unchanged in the adjacent frame, it will capture RGB images and depth maps (i.e., image information).

[0133] Furthermore, this method uses a checkerboard calibration board. For the image input from the camera, the system first performs corner detection and subpixel detection. Images in which corners cannot be detected are discarded, and the corresponding depth map and position information are also discarded accordingly to obtain the first image.

[0134] Further, for the reserved RGB picture first input system, the camera's internal parameters and distortion calibration are performed, the internal parameters are used to convert the position of the table point on the 2D image to the pixel position on the depth map, and the spatial three-dimensional position information of the table point is used to construct a coordinate system, which is used to describe the relationship between the camera and the calibration board. The purpose of generating the point cloud is to obtain the pose in the 3D space.

[0135] Finally, the preset mechanical arm pose and the relationship between the camera and the calibration board are input into the calibration system, and the TASI method is used to calibrate the relationship between the sensor and the robot.

[0136] It should be noted that the above is an example and does not specifically limit the implementation logic.

[0137] Obviously, those skilled in the art should understand that the modules or steps of the present application described above can be realized by general computing devices, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices, and can be realized by program codes executable by the computing device, so that they can be stored in the storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in different order, or they can be manufactured into individual integrated circuit modules, or multiple modules or steps can be manufactured into a single integrated circuit module. Thus, the present application is not limited to any specific hardware and software combination.

[0138] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer usable storage media containing computer usable program codes (including but not limited to disk storage, CD-ROM, optical storage, etc.).

[0139] The present application is described with reference to flowcharts and / or block diagrams according to the method, device (system), and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The device that implements the functions specified in one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0140] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0141] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that are executed on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0142] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0143] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory. The memory is an example of computer readable media.

[0144] Computer readable media includes permanent and non-permanent, moveable and non- moveable media that can be implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disks (DVDs) or other optical storage, magnetic cassettes, magnetic disks storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that is accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0145] Any technical features in the above-described embodiments can be combined in any manner, and for the sake of brevity, not all possible combinations are described, however, it is to be understood that the application encompasses all possible combinations.

[0146] It should also be noted that the terms "comprising", "comprises", "including", "includes" or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0147] The preferred embodiments of the application are described above in detail. The application is not limited to the embodiments described above, but can be modified and changed by those skilled in the art without departing from the scope of the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the scope of the application.

Claims

1. A calibration method for robot and vision matching, characterized in that, The calibration method for robot and vision matching includes: Obtain the robot's initial position; Based on the initial position, the preset pose of the robot is determined; wherein, the preset pose is the position and attitude of the robot during calibration; The preset pose is transmitted to the robot via a communication module, so that the robot moves to the preset pose. The robot is captured in a preset pose, and the image information is preprocessed to obtain a first image; The robot is calibrated using the first image.

2. The calibration method for robot and vision matching according to claim 1, characterized in that, The calibration of the robot using the first image includes: The image acquisition unit is calibrated using the first image to obtain the calibrated intrinsic parameters. The positions of the fixed points in the first image are converted into point cloud processing using the intrinsic parameters to obtain three-dimensional position information. Based on the three-dimensional position information, a first coordinate system is constructed; The robot and the image acquisition unit are calibrated according to the preset pose and the first coordinate system.

3. The calibration method for robot and vision matching according to claim 1, characterized in that, The step of preprocessing the image information to obtain the first image includes: Perform first corner detection and / or subpixel first corner detection on the image information; If the first corner point is not detected in the second image, the second image is removed from the image information to obtain the first image.

4. The calibration method for robot and vision matching according to claim 1, characterized in that, The initial position includes a first position, a second position, and a third position. Obtaining the robot's initial position includes: The image acquisition unit acquires multiple second corner points on the projection plane; Adjust the robot to move the calibration board so that the calibration board reaches the three second corner points respectively, thus obtaining the first position, the second position and the third position.

5. The calibration method for robot and vision matching according to claim 1, characterized in that, The initial position includes a first position, a second position, and a third position. Determining the robot's preset pose based on the initial position includes: A first vector axis is determined based on the first position and the second position; a second vector axis is determined based on the third position and the second position. Position sampling is performed along the first vector axis and / or the second vector axis to obtain multiple preset positions; Based on each of the plurality of preset positions, rotate along the second coordinate system by a preset angle to obtain a preset posture; The preset pose is determined based on the preset position and the preset posture.

6. The calibration method for robot and vision matching according to claim 5, characterized in that, Before determining the preset pose based on the preset position and the preset pose, the method further includes: The joint angle values ​​are obtained by inverse kinematics solution of the preset position and preset posture. The preset position and preset posture are updated based on the joint angle values.

7. The calibration method for robot and vision matching according to claim 6, characterized in that, The step of updating the preset position and the preset posture based on the joint angle value includes: If the joint angle value exceeds the preset joint limit, the first preset joint or the second preset joint is taken as the new joint angle value, and the new joint angle value is solved based on the forward kinematics to obtain the new preset position and the new preset posture. If the joint angle value does not exceed the preset joint limit, maintain the preset position and the preset posture.

8. The calibration method for robot and vision matching according to claim 6, characterized in that, The step of performing inverse kinematics on the preset position and preset posture to obtain joint angle values ​​includes: A rotation matrix is ​​generated based on the preset posture, and a position vector is generated based on the preset position. The joint angle values ​​are obtained by inversely solving the rotation matrix and the position vector using an analytical method.

9. The calibration method for robot and vision matching according to claim 1, characterized in that, The acquisition of image information of the robot in the preset pose includes: Receive the arrival signal of the robot moving to the preset pose, and obtain the cumulative duration; If the cumulative duration equals the preset duration, the cumulative duration is cleared to zero, and the image information of the robot in the preset pose is acquired by the image acquisition unit; wherein, the cumulative duration is the duration elapsed since the arrival signal was received, and the preset duration is the duration during which the image is stable.

10. A robot, characterized in that, The robot is calibrated using the robot and vision matching calibration method as described in any one of claims 1 to 9.

Images