Robot vision calibration device and method

Abstract

According to the robot vision calibration device and method, through collaborative design of the industrial personal computer, the vision system, the robot, the calibration jig and the calibration plate, the problems caused by complex structure, low calibration precision and poor adaptability in the prior art are solved. A floating mechanism of the calibration jig is combined with a displacement sensor to realize multi-degree-of-freedom dynamic displacement compensation, so that mechanical positioning errors are remarkably reduced; the conical surface guide structure and the reference hole layout of the calibration plate optimize the stability of the positioning reference, and ensure the accurate embedding of the calibration needle. The flexible installation of the visual system adapts to different scene requirements. According to the calibration method, the generation of a high-precision visual calibration matrix is realized through the processes of multi-reference hole coordinate recording, calibration ring coordinate calculation and visual image acquisition and matching. According to the whole scheme, through a dynamic compensation mechanism and intelligent process design, the calibration precision, efficiency and robustness are improved, meanwhile, the operation complexity is reduced, and reliable technical support is provided for an industrial automation scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robot vision, and in particular to a robot vision calibration device and method. BACKGROUND

[0002] The current robot vision calibration method has the following problems, resulting in low calibration accuracy and complex structure. For example, on the one hand, for a feasible implementation, the vision calibration is performed by moving the camera and fixing the calibration board. This method only supports the "eye on hand" calibration method and cannot adapt to the "eye outside hand" scene. The system design does not consider the coordinate system conversion requirement of the fixed camera, and the rigid connection between the robot and the camera limits the application scenario. For another feasible implementation, a fixed vision camera is used and the calibration board is moved. This method has no dynamic calibration capability and is only suitable for static working environment and cannot process real-time calibration of dynamic scenes such as conveyer belts. The reason is that the calibration algorithm is based on the collection of pre-set fixed positions and lacks a motion compensation mechanism. For another feasible implementation, the calibration method is performed by controlling the motion of the calibration board through an external mechanical device and the camera is stationary. The mechanical device used in this method, such as a rotating device or a linear device, will increase the gap if used too many times, thereby affecting the actual calibration accuracy.

[0003] In summary, in some feasible implementations, the structure is complex, resulting in high difficulty in calibration operation, low calibration efficiency and accuracy. Therefore, a robot vision calibration device and corresponding calibration method are needed to solve the corresponding problems. SUMMARY

[0004] The present application provides a robot vision calibration device and method to solve the problem of complex structure of current robot vision calibration, resulting in high difficulty in calibration operation, low calibration efficiency and accuracy.

[0005] In a first aspect, the present application provides a robot vision calibration device, characterized in that it comprises:

[0006] an industrial computer;

[0007] a vision system connected with the industrial computer, used for collecting images of a calibration board;

[0008] a robot in communication connection with the industrial computer, the end of which is used for installing a calibration jig;

[0009] a calibration jig installed at the end of the robot, comprising a floating mechanism, a displacement sensor and a calibration needle, the floating mechanism being used to drive the calibration needle to slide in a first direction and a second direction, and the displacement sensor being used to detect the offset value of the calibration needle;

[0010] A calibration plate is placed on a workbench, and three reference holes and nine calibration rings are arranged on the calibration plate. The reference holes have a tapered guide structure that gradually shrinks towards the inside of the calibration plate, which is used to guide the calibration needle to be embedded.

[0011] As an optional implementation, the floating mechanism of the calibration jig comprises:

[0012] A first guide rail is fixed at the end of the robot;

[0013] A first sliding block is slidably connected to the first guide rail;

[0014] A second guide rail is fixed to the first sliding block, and the extension direction of the second guide rail is perpendicular to the first guide rail;

[0015] A second sliding block is slidably connected to the second guide rail;

[0016] A first sliding plate is fixed to the second sliding block;

[0017] A third guide rail is fixed to the first sliding plate, and the extension direction of the third guide rail is perpendicular to the second guide rail;

[0018] A third sliding block is slidably connected to the third guide rail;

[0019] A second sliding plate is fixed to the third sliding block, and the calibration needle is installed on the second sliding plate.

[0020] As an optional implementation, the displacement sensor comprises a first displacement sensor and a second displacement sensor. The first displacement sensor is used to detect the offset value of the first sliding plate in the first direction, and the second displacement sensor is used to detect the offset value of the second sliding plate in the second direction.

[0021] As an optional implementation, the three reference holes of the calibration plate are distributed in a triangular shape, and the nine calibration rings are arranged in a 3x3 matrix within the area surrounded by the three reference holes.

[0022] The tapered guide structure of the reference hole comprises a tapered surface that gradually shrinks towards the inside of the calibration plate, which is used to guide the calibration needle to slide along the tapered surface to the center of the reference hole.

[0023] As an optional implementation, the vision system is installed at the end of the robot or fixedly installed above the workbench.

[0024] As an optional implementation, when the vision system is installed at the end of the robot, it is used alternately with the calibration jig; when the vision system is fixedly installed above the workbench, the robot carries the calibration jig to complete the positioning of the reference hole, and then moves to a position that does not block the field of view of the vision system.

[0025] In a second aspect, the present application provides a robot vision calibration method, the method comprising:

[0026] initializing position information of the calibration plate on the workbench;

[0027] the robot carrying the calibration fixture down to the calibration plate, the calibration needle of the calibration fixture contacting the conical surface of the reference hole of the calibration plate, sliding along the first direction and the second direction through the floating mechanism to compensate for the offset until the calibration needle is embedded in the reference hole, and recording the mechanical coordinates of the robot at this time;

[0028] repeatedly performing the calibration needle embedding in the three reference holes of the calibration plate in turn, and recording the corresponding mechanical coordinates of the robot respectively;

[0029] calculating the second mechanical coordinates of the nine calibration circles on the calibration plate according to the first mechanical coordinates of the three reference holes;

[0030] acquiring the image of the calibration plate through the vision system, identifying the image coordinates of the nine calibration circles and automatically sorting them;

[0031] matching the corresponding second mechanical coordinates of the nine calibration circles with the sorted image coordinates, and calculating the vision calibration matrix.

[0032] As an optional implementation, the robot carrying the calibration fixture down to the calibration plate, the calibration needle of the calibration fixture contacting the conical surface of the reference hole of the calibration plate, sliding along the first direction and the second direction through the floating mechanism to compensate for the offset until the calibration needle is embedded in the reference hole, and recording the mechanical coordinates of the robot at this time, specifically comprising:

[0033] the robot carrying the calibration fixture down along the vertical direction, so that the tip of the calibration needle contacts the conical surface of the reference hole;

[0034] the reaction force applied by the conical surface pushing the calibration needle to drive the second sliding plate to slide along the third guide rail, the second displacement sensor detecting the offset value of the second sliding plate along the second direction, and at the same time, the sliding of the second sliding plate driving the first sliding plate to slide along the second guide rail, the first displacement sensor detecting the offset value of the first sliding plate along the first direction;

[0035] when the detection values of the first displacement sensor and the second displacement sensor are both less than a preset threshold, it is determined that the calibration needle is embedded in the center of the reference hole, the down is stopped and the mechanical coordinates of the robot at this time are recorded.

[0036] As an optional implementation, the step of calculating the robot mechanical coordinates of the nine calibration circles according to the robot mechanical coordinates of the three reference holes specifically comprises:

[0037] The robot mechanical coordinates of the three reference holes are respectively set as a first reference point, a second reference point and a third reference point;

[0038] A first unit vector from the first reference point to the second reference point and a second unit vector from the first reference point to the third reference point are calculated, and a rotation transformation matrix of the calibration board to a robot coordinate system is constructed;

[0039] According to the rotation transformation matrix, the original coordinates of the nine calibration circles on the calibration board based on a calibration board coordinate system are converted into mechanical coordinates in a robot coordinate system, so as to obtain the robot mechanical coordinates of the nine calibration circles.

[0040] As an optional implementation, the step of identifying the image coordinates of the nine calibration circles and automatically sorting by the vision system specifically comprises:

[0041] The vision system extracts the center coordinates of the nine calibration circles by using a sub-pixel edge detection algorithm, so as to obtain an initial image coordinate set;

[0042] According to a preset second direction sorting direction and a second direction error threshold, the initial image coordinate set is second direction grouped, and the coordinates with a coordinate difference less than the second direction error threshold are grouped into the same group;

[0043] For the coordinates in each group, a first direction sorting is performed according to a preset first direction sorting direction, so as to obtain an ordered image coordinate sequence;

[0044] The ordered image coordinate sequence is outputted and used for matching with the robot mechanical coordinates of the nine calibration circles.

[0045] The robot vision calibration device and method provided by the application solves the problems caused by the complex structure, low calibration accuracy and poor adaptability in the prior art through the cooperative design of the industrial computer, the vision system, the robot, the calibration fixture and the calibration board. The floating mechanism of the calibration fixture realizes dynamic displacement compensation with multiple degrees of freedom in combination with the displacement sensor, significantly reducing the mechanical positioning error; the conical surface guiding structure and the reference hole layout of the calibration board optimize the stability of the positioning reference, ensuring the accurate embedding of the calibration needle. The flexible installation mode of the vision system supports two modes of "eye on hand" and "eye off hand", adapting to different scene requirements. The calibration method realizes the generation of a high-precision vision calibration matrix through the automatic process of multi-reference hole coordinate recording, calibration circle coordinate calculation, vision image acquisition and matching. The overall scheme improves the calibration accuracy, efficiency and robustness through the dynamic compensation mechanism and intelligent process design, reduces the operation complexity, and provides reliable technical support for industrial automation scenes. BRIEF DESCRIPTION OF DRAWINGS

[0046] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0047] Figure 1 is the overall architecture schematic diagram of a robot vision calibration device disclosed by the embodiments of the application;

[0048] Figure 2 is the architecture schematic diagram of a vision calibration board disclosed by the embodiments of the application;

[0049] Figure 3 is the architecture schematic diagram of a vision calibration fixture disclosed by the embodiments of the application;

[0050] Figure 4 is the flow schematic diagram of a robot vision calibration method disclosed by the embodiments of the application.

[0051] Through the above drawings, the specific embodiments of the application have been shown, and there will be more detailed descriptions in the following, and these drawings and textual descriptions are not intended to limit the scope of the concept of the application by any means, but to illustrate the concept of the application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0052] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is only one of the specific embodiments consistent with the present application, and is not intended to represent all the embodiments consistent with the present application. Rather, they are only examples of devices and methods consistent with some aspects of the present application, as detailed in the appended claims.

[0053] The present application is to realize 2D vision to quickly complete hand-eye calibration, reduce the difficulty of manual operation, and improve the efficiency of calibration. The present application comprises a calibration needle, a vision system, a robot, and a calibration board. Specifically, it embodies a set of devices and systems that reduce manual labor intensity, improve calibration efficiency, and improve the calibration accuracy of vision.

[0054] At present, similar technologies include a general robot vision automatic calibration device and method. The method includes converting robot motion control, parameter setting, camera calibration, and grabbing new coordinate calculation functions into an external independent third-party software and hardware-independent device that only transmits necessary result data to the robot and camera through a standard communication interface, making the robot hand-eye calibration process completely automated without relying on third-party software and hardware, greatly improving the universality of the hand-eye calibration device and method, improving the calibration speed and efficiency, and reducing the technical requirements for on-site operators. The robot moves the camera to search for the position of the calibration board, and records the pixel coordinates and corresponding robot coordinates by taking pictures of the feature points of the calibration board through multiple translations and rotations. The device of this patent has high integration and high automation, but is only suitable for eye-on-hand calibration methods and cannot be used for eye-outside-hand fixed vision calibration methods. Another feasible implementation includes a vision automatic calibration system, method, and storage medium. The movable calibration device (including a rotating support rod and a moving base) works with the vision measuring instrument to automatically adjust the position and angle of the calibration board, completely replacing manual operation. The degree of automation is high, but the mechanical structure of the device used is relatively complex and inconvenient to maintain, which may affect the calibration accuracy, such as rotation device jamming and universal wheel failure.

[0055] In summary, on the one hand, there is a vision rapid calibration method that moves the camera and fixes the calibration board. This method only supports the "eye-on-hand" calibration method and cannot be adapted to the "eye-outside-hand" scene. The system design does not consider the coordinate system conversion requirements of the fixed camera, and the rigid connection between the robot and the camera limits the application scenarios. On the other hand, there is a method that uses a fixed vision camera and moves the calibration board. This method has no dynamic calibration capability and is only suitable for static working environments, and cannot handle real-time calibration of dynamic scenes such as conveyors. The reason is that the calibration algorithm is based on pre-set fixed position collection and lacks a motion compensation mechanism. On the other hand, there is a calibration method that controls the motion of the calibration board through an external mechanical device and the camera is stationary. The mechanical device used in this method, such as a rotating device or a linear device, will increase the gap if used too many times, thereby affecting the actual calibration accuracy.

[0056] The application aims to solve the core problems of single mode, complex mechanical structure, insufficient dynamic calibration capability and the like of existing 2D vision calibration technology, and a calibration system with high efficiency, high precision and high reliability is constructed by innovatively using a calibration needle floating mechanism, a calibration plate, a 2D vision system and robot cooperative control technology. Compared with the prior art, the calibration efficiency and precision are improved, and manual intervention is reduced by the application through the innovative points of self-adaptive compensation positioning of the calibration fixture, automatic image coordinate acquisition and sorting, modular and expandable design of the calibration device and the like.

[0057] Specifically, compared with the traditional vision calibration process, the application optimizes the calibration process, and the calibration process is simplified into four simplified operation steps: coarse positioning, first placing the calibration plate to a tolerance of ±2 mm; fine positioning, obtaining three reference points and calculating the robot coordinates of nine calibration circles (the floating mechanism automatically compensates); image coordinate acquisition and sorting; and calculating the vision calibration matrix.

[0058] The application supports robot camera calibration in the "eye on hand" mode and fixed camera calibration in the "eye outside hand" scene. The system uses a floating compensation calibration needle, and only needs to roughly place the calibration plate under the calibration needle (±2 mm tolerance), and the system can automatically complete fine positioning through the conical guide surface and the displacement sensor, eliminating the tedious manual fine adjustment steps in the traditional method. The vision processing algorithm can automatically identify the nine calibration circles in the calibration plate image, extract the center coordinates by using the sub-pixel edge detection technology, and automatically sort by using the preset sorting method, output the sorted coordinate sequence, completely avoiding the sorting errors caused by manual intervention.

[0059] The application is a robot vision rapid calibration device and method, mainly used for realizing 2D vision rapid hand-eye calibration, reducing the difficulty of manual operation and improving the efficiency of calibration. The purpose of the application is to solve the problem of low calibration efficiency, such as the traditional robot vision 9-point calibration method, which needs to rely on manual teaching, multiple image acquisition and manual parameter adjustment, and is time-consuming; to improve the calibration precision and robustness in complex environments, to solve the problem of large manual point error for the traditional nine-point calibration using manual operation; to reduce the operation threshold and safety risk, the traditional calibration needs professional technical personnel to operate the robot multiple times, which has a collision risk and is highly dependent on personnel experience.

[0060] In summary, the technical concept of the present application is that the robot vision calibration device and method provided by the present application solve the problems caused by the complex structure, low calibration accuracy and poor adaptability in the prior art through the cooperative design of the industrial computer, the vision system, the robot, the calibration jig and the calibration plate. The floating mechanism of the calibration jig combines with the displacement sensor to realize dynamic displacement compensation with multiple degrees of freedom, significantly reducing the mechanical positioning error; the conical surface guiding structure and the layout optimization of the reference hole of the calibration plate optimize the stability of the positioning reference, ensuring the accurate embedding of the calibration needle. The flexible installation mode of the vision system supports two modes of "eye on hand" and "eye off hand", which adapts to different scene requirements. The calibration method realizes the generation of a high-precision vision calibration matrix through the automatic process of multi-reference hole coordinate recording, calibration circle coordinate calculation, vision image acquisition and matching. The overall scheme improves the calibration accuracy, efficiency and robustness through the dynamic compensation mechanism and intelligent process design, while reducing the operation complexity, providing reliable technical support for industrial automation scenes.

[0061] Firstly, the present application provides a robot vision calibration device, comprising:

[0062] an industrial computer;

[0063] a vision system connected with the industrial computer, used for acquiring images of a calibration plate;

[0064] a robot in communication connection with the industrial computer, the end of which is used for installing a calibration jig;

[0065] a calibration jig installed at the end of the robot, comprising a floating mechanism, a displacement sensor and a calibration needle, the floating mechanism being used to drive the calibration needle to slide in a first direction and a second direction, and the displacement sensor being used to detect the offset value of the calibration needle;

[0066] a calibration plate placed on a workbench, which is provided with three reference holes and nine calibration circles, the reference holes having a conical surface guiding structure gradually shrinking towards the inside of the calibration plate, used for guiding the embedding of the calibration needle.

[0067] The embodiment provides a robot vision calibration device, which realizes high-precision vision calibration by coordinating the cooperative work of a vision system, a robot, a calibration jig and a calibration board through an industrial computer. The floating mechanism of the calibration jig allows the calibration needle to slide in a first direction and a second direction, and the displacement sensor is used to detect the offset value in real time, so that the mechanical error generated by the robot in the positioning process can be dynamically compensated. The reference hole of the calibration board is designed as a conical surface guide structure, the calibration needle is guided to be accurately embedded in the center position through the gradually shrinking conical surface, and the calibration error caused by mechanical vibration or positioning deviation is avoided. In addition, the vision system can be flexibly installed at the end of the robot or above the workbench, supporting two calibration modes of "eye on hand" and "eye off hand", and the applicability of the device is improved. The overall structure design simplifies the calibration process, reduces manual intervention, and significantly improves the calibration precision and efficiency.

[0068] As an optional embodiment, the floating mechanism of the calibration jig comprises:

[0069] A first guide rail is fixed at the end of the robot;

[0070] A first sliding block is slidably connected to the first guide rail;

[0071] A second guide rail is fixed on the first sliding block, and the extension direction of the second guide rail is perpendicular to the first guide rail;

[0072] A second sliding block is slidably connected to the second guide rail;

[0073] A first sliding plate is fixed on the second sliding block;

[0074] A third guide rail is fixed on the first sliding plate, and the extension direction of the third guide rail is perpendicular to the second guide rail;

[0075] A third sliding block is slidably connected to the third guide rail;

[0076] A second sliding plate is fixed on the third sliding block, and the calibration needle is installed on the second sliding plate.

[0077] The embodiment further provides a specific structure of the floating mechanism, which comprises a combination of the first guide rail, the first sliding block, the second guide rail, the second sliding block, the first sliding plate, the third guide rail, the third sliding block and the second sliding plate. The nested layout of the multi-stage vertical guide rails allows the calibration needle to slide independently in the first direction and the second direction, realizing high-precision displacement compensation with multiple degrees of freedom. The compact design of the floating mechanism reduces mechanical clearance and cumulative error, ensuring that the calibration needle can quickly respond and adaptively adjust the position when contacting the conical surface of the reference hole, thereby further improving the calibration precision. At the same time, the modular structure reduces the assembly complexity, is convenient to maintain and replace, and enhances the reliability and practicality of the device.

[0078] As an optional implementation, the displacement sensor comprises a first displacement sensor and a second displacement sensor, the first displacement sensor is used to detect the offset value of the first sliding plate in the first direction, and the second displacement sensor is used to detect the offset value of the second sliding plate in the second direction.

[0079] The first displacement sensor and the second displacement sensor are arranged to detect the offset value of the first sliding plate in the first direction and the offset value of the second sliding plate in the second direction respectively. The cooperation of the two sensors realizes real-time monitoring of the displacement of the floating mechanism, ensures that the offset amount of the calibration needle during embedding into the reference hole is accurately captured and fed back to the industrial computer. This design avoids the defect of limited detection range of a single sensor, covers the multi-direction displacement compensation demand, and significantly improves the accuracy of the calibration data. In addition, the real-time processing capability of the sensor data reduces the delay in the calibration process, further improving the overall calibration efficiency.

[0080] As an optional implementation, the three reference holes of the calibration plate are in a triangular distribution, and the nine calibration circles are arranged in a 3x3 matrix in the area surrounded by the three reference holes.

[0081] The conical surface guiding structure of the reference hole comprises a conical surface gradually shrinking towards the inside of the calibration plate, used to guide the calibration needle to slide along the conical surface to the center of the reference hole.

[0082] The present embodiment provides a triangular distribution of three reference holes on the calibration plate and a 3x3 matrix arrangement of nine calibration circles. The triangular distribution of reference holes provides a reliable positioning reference for calibration through geometric stability, while the nine calibration circles are evenly distributed in the area surrounded by the reference holes, ensuring that the coverage range of the calibration points is comprehensive and has no blind area. The conical surface guiding structure of the reference hole guides the calibration needle to slide to the center position through the gradually shrinking conical surface, avoiding positioning failure caused by initial contact deviation. This layout design optimizes the spatial distribution of calibration data, providing sufficient data support for subsequent calculation of the visual calibration matrix, thereby improving the calibration accuracy and robustness.

[0083] As an optional implementation, the visual system is installed at the end of the robot or fixedly installed above the workbench.

[0084] The embodiment provides two installation modes of the vision system: the robot end or above the workbench. When installed at the robot end, the vision system can be alternately used with the calibration jig, simplifying the equipment switching process; when fixed above the workbench, the robot can be moved to a position not blocking the field of view after completing the reference hole positioning, ensuring that the vision system can collect the calibration plate image without interference. This flexible installation mode meets the needs of different working scenes, avoids the problem of limited field of view caused by fixed camera in the traditional calibration device, and improves the universality and calibration efficiency of the device.

[0085] As an optional embodiment, when the vision system is installed at the robot end, the vision system is alternately used with the calibration jig; when the vision system is fixedly installed above the workbench, the robot carries the calibration jig to complete the reference hole positioning, and then moves to a position not blocking the field of view of the vision system.

[0086] The embodiment further refines the use logic of the vision system. When the vision system is installed at the robot end, the alternation of the vision system and the calibration jig realizes seamless connection of function switching; when fixed above the workbench, the robot actively avoids the field of view area after completing the reference hole positioning, ensuring that the vision system can completely collect the image. This design avoids image collection failure caused by the robot blocking the field of view, and at the same time, through the industrial computer coordinating the action timing of the robot and the vision system, the automation degree of the calibration process is optimized, and the calibration efficiency and reliability are significantly improved.

[0087] The application also provides a robot vision calibration method, the method comprising:

[0088] Initializing the position information of the calibration plate on the workbench;

[0089] The robot carries the calibration jig to dive to the calibration plate, the calibration needle of the calibration jig contacts the conical surface of the reference hole of the calibration plate, slides along the first direction and the second direction through the floating mechanism to compensate for the offset, until the calibration needle is embedded in the reference hole, and the mechanical coordinates of the robot at this time are recorded;

[0090] Repeating the steps of embedding the calibration needle into the three reference holes of the calibration plate in sequence, and recording the corresponding mechanical coordinates of the robot respectively;

[0091] According to the first mechanical coordinates of the robot of the three reference holes, the second mechanical coordinates of the nine calibration circles on the calibration plate are calculated;

[0092] An image of the calibration plate is collected through the vision system, image coordinates of the nine calibration circles are identified and automatically sorted;

[0093] The second robot mechanical coordinates of the nine calibration circles are matched with the sorted image coordinates to obtain a visual calibration matrix.

[0094] The embodiment provides a robot visual calibration method. By initializing a calibration plate position, embedding a reference hole of a robot and recording mechanical coordinates, calculating calibration circle coordinates, collecting images by a visual system and matching coordinates, full-automatic visual calibration is realized. Multi-directional sliding of a floating mechanism compensates for mechanical positioning errors. A displacement sensor detects offset values in real time to ensure that a calibration needle is accurately embedded into the center of the reference hole. The coordinates of three reference holes provide a geometric reference for calibration circle coordinate calculation. High-precision matching of mechanical coordinates and image coordinates is realized by combining image recognition and sorting of the nine calibration circles by the visual system. The method significantly improves calibration accuracy and efficiency by dynamic compensation and automatic processes, and supports real-time calibration requirements in complex working environments.

[0095] As an optional embodiment, the robot carries a calibration jig to the calibration plate. A calibration needle of the calibration jig contacts a conical surface of the reference hole of the calibration plate. The floating mechanism slides in a first direction and a second direction to compensate for the offset until the calibration needle is embedded into the reference hole. The mechanical coordinates of the robot at this time are recorded. Specifically, the method comprises the following steps.

[0096] The robot carries the calibration jig to vertically descend so that the tip of the calibration needle contacts the conical surface of the reference hole.

[0097] The reaction force applied by the conical surface pushes the calibration needle to drive the second sliding plate to slide along the third guide rail. The second displacement sensor detects the offset value of the second sliding plate in the second direction. Meanwhile, the sliding of the second sliding plate drives the first sliding plate to slide along the second guide rail. The first displacement sensor detects the offset value of the first sliding plate in the first direction.

[0098] When the detection values of the first displacement sensor and the second displacement sensor are both less than a preset threshold value, it is determined that the calibration needle is embedded into the center of the reference hole. The vertical descent is stopped, and the mechanical coordinates of the robot at this time are recorded.

[0099] The embodiment provides a specific process of embedding the calibration needle into the reference hole. The robot vertically descends so that the calibration needle contacts the conical surface. The reaction force pushes the floating mechanism to slide along the guide rail. The displacement sensor detects the offset value in real time. When the offset value is less than the preset threshold value, it is determined that the calibration needle is embedded into the center position. The process realizes real-time compensation for the micro-displacement by the dynamic response of the floating mechanism and the accurate feedback of the sensor, avoiding embedding failure caused by mechanical vibration or positioning deviation. Meanwhile, the setting of the preset threshold value ensures the controllability of the calibration accuracy, further improving the reliability of the calibration result.

[0100] As an optional implementation, the step of calculating the robot mechanical coordinates of the nine calibration circles according to the robot mechanical coordinates of the three reference holes specifically comprises:

[0101] The robot mechanical coordinates of the three reference holes are respectively set as a first reference point, a second reference point, and a third reference point;

[0102] A first unit vector from the first reference point to the second reference point and a second unit vector from the first reference point to the third reference point are calculated, and a rotation transformation matrix of the calibration board to the robot coordinate system is constructed;

[0103] According to the rotation transformation matrix, the original coordinates of the nine calibration circles on the calibration board based on the calibration board coordinate system are converted into mechanical coordinates in the robot coordinate system, and robot mechanical coordinates of the nine calibration circles are obtained.

[0104] The embodiment provides a specific logic for calculating the coordinates of the nine calibration circles according to the coordinates of the three reference holes. The rotation transformation matrix is constructed by the unit vectors of the reference points, and the original coordinates of the calibration circles in the calibration board coordinate system are converted into mechanical coordinates in the robot coordinate system. This conversion method is based on the rigid body transformation principle of geometric space, ensuring the accuracy of coordinate mapping. Through the simplified processing of the mathematical model, the computational burden brought by complex algorithms is avoided, and the efficiency and accuracy of coordinate calculation are improved, providing a reliable data basis for the generation of subsequent visual calibration matrices.

[0105] As an optional implementation, the step of identifying the image coordinates of the nine calibration circles and automatically sorting them by the vision system specifically comprises:

[0106] The vision system uses a sub-pixel edge detection algorithm to extract the center coordinates of the nine calibration circles, obtaining an initial image coordinate set;

[0107] According to a preset second direction sorting direction and a second direction error threshold, the initial image coordinate set is second direction grouped, and coordinates with a coordinate difference less than the second direction error threshold in the second direction are grouped into the same group;

[0108] For the coordinates in each group, a first direction sorting is performed according to a preset first direction sorting direction, and an ordered image coordinate sequence is obtained;

[0109] The ordered image coordinate sequence is outputted and used for matching with the robot mechanical coordinates of the nine calibration circles.

[0110] The embodiment provides a sorting logic of a vision system for identifying image coordinates of a calibration circle. The center coordinates are extracted by a sub-pixel edge detection algorithm, and grouping and sorting rules in a preset direction are combined to realize automatic sorting of nine image coordinates of the calibration circle. The sub-pixel algorithm improves the accuracy of coordinate extraction, the grouping and sorting rules ensure the logical consistency of the coordinate sequence, and errors caused by manual intervention are avoided. The design optimizes the matching efficiency of the image coordinates and the mechanical coordinates, and further improves the calculation accuracy and automation level of the vision calibration matrix.

[0111] Therefore, the present application aims to solve the core problems of single mode, complex mechanical structure, and insufficient dynamic calibration capability of existing 2D vision calibration technology. By innovatively using a calibration needle floating mechanism, a calibration plate, a 2D vision system, and a robot cooperative control technology, a calibration system with high efficiency, high precision, and high reliability is constructed. Compared with the prior art, the present application improves the calibration efficiency and precision and reduces manual intervention through mechanical positioning, vision calculation verification, adaptive compensation, and modular expandable design.

[0112] The present application will be described based on actual application scenarios. First, refer to Figures 1 to 3 , Figure 1 is a structural schematic diagram of a robot vision calibration device disclosed by an embodiment of the present application, Figure 2 is a structural schematic diagram of a vision calibration plate disclosed by an embodiment of the present application; Figure 3 is a structural schematic diagram of a vision calibration jig disclosed by an embodiment of the present application;

[0113] As shown in Figures 1 to 3 , the device structure comprises:

[0114] 1 - industrial computer; 2 - 2D vision system; 3 - robot; 4 - robot base; 5 - calibration jig; 6 - calibration plate; 7 - suction jig; 8 - workbench; 9 - product; 501 - first displacement sensor; 502 - first sliding block; 503 - first guide rail; 504 - second displacement sensor; 505 - second guide rail; 506 - second sliding block; 507 - first sliding plate; 508 - third sliding block; 509 - third guide rail; 510 - calibration needle; 511 - second sliding plate; 512 - shell; 601 - first reference hole; 602 - second reference hole; 603 - third reference hole; 604 - suction module; 605 - calibration circle.

[0115] The main working principle of the present application is:

[0116] The robot 2D vision rapid calibration system is composed of the industrial computer 1, the 2D vision system 2, the robot 3, the workbench 8, the calibration board 6 and the calibration jig 5, and is used for realizing rapid vision calibration of the robot.

[0117] The software system of the 2D vision system 2 is loaded in the industrial computer 1, and the industrial computer 1 is connected with the robot 3 through TCP communication. The 2D vision software is used for processing calculation of nine-point robot coordinates, sorting of nine-point image coordinates and calculation of a nine-point calibration matrix.

[0118] The first displacement sensor 501, the first sliding block 502 and the first guide rail 503 on the calibration jig 5 can slide along the X direction together with the second sliding block 506 and the first sliding plate 507. The first sliding plate 507 also moves along the X direction with the calibration needle 510. The movement of the first sliding plate 507 compresses or stretches the second displacement sensor, and the offset value of the calibration needle in the X direction is obtained. The second sliding block 508 and the second sliding plate 511 move the calibration needle 510 along the Y direction. The movement of the second sliding plate 511 compresses or stretches the first displacement sensor, and the offset value of the calibration needle in the Y direction is obtained.

[0119] The calibration board 6 has three reference point holes, i.e., a first reference hole 601, a second reference hole 602 and a third reference hole 603. The calibration needle 510 of the calibration jig 5 is embedded into the three reference holes for three times, three reference points are recorded and obtained, and are sent to the industrial computer 1.

[0120] Please refer to Figure 4 , Figure 4 is a flowchart of a robot vision calibration method disclosed in the embodiment of the application. The method comprises the following steps:

[0121] The installation of the calibration jig 5 is completed, and the calibration jig 5 is installed on the robot 3.

[0122] The calibration jig 5 and the robot 3 perform serial communication, and the 2D vision system 2 and the robot 3 perform TCP communication.

[0123] The calibration board 6 is placed on the workbench 8, and the first reference hole is roughly aligned with the calibration jig 5.

[0124] The robot 3 carries the calibration jig 5 and is lowered to the calibration board 6. The calibration needle of the calibration jig 5 is deeply embedded into the first reference hole of the calibration board 6. The calibration needle jig is floated due to the influence of the first reference hole taper surface, and is embedded into the first reference hole of the calibration board 6. At this time, the current mechanical coordinates of the robot are recorded.

[0125] The robot 3 is controlled to align the calibration jig 5 with the second reference hole, embed the calibration jig 5 into the second reference hole of the calibration board 6, record the current mechanical coordinates of the robot, and then move the robot to embed the calibration jig 5 into the third reference hole of the calibration board 6 and record the current mechanical coordinates of the robot.

[0126] According to the robot coordinates of the first, second and third reference holes, the first reference point is recorded as PBASE1(X, Y), the second reference point is recorded as PBASE2(X, Y), and the third reference point is recorded as PBASE3(X, Y). The nine-point mechanical coordinates are recorded as Probot(Xn, Yn). The distance between each point on the calibration plate 6 is c. The mechanical coordinates corresponding to the positions of the nine points on the calibration plate 6 are automatically calculated;

[0127] The distance between PBASE1 and PBASE2 is calculated as d, and the distance between PBASE1 and PBASE3 is calculated as e.

[0128] The rotation transformation matrix R of the calibration plate to the robot is composed of column vectors after the reference point vectors are unitized.

[0129]

[0130] Through the rotation transformation matrix R, the calibration plate coordinates are converted into robot coordinates.

[0131]

[0132] The X and Y coordinates of the robot coordinates are calculated respectively

[0133]

[0134] P robot =[X n ,Y n ]

[0135] The robot 3 moves to a position that does not block the camera's view. The 2D vision system 2 takes a picture, identifies the image coordinates of the nine points on the calibration plate 6, and automatically sorts the nine image coordinates Pimage. The sorting direction Dir_y, Dir_x, and the Y sorting error Ierr need to be manually set.

[0136] The sorting function σY is defined for Y-direction sorting of the image coordinates.

[0137]

[0138] The ordered points P' = {Pimage, Pimage, …, Pimage} are obtained by applying σY to Pimage. According to the Y error grouping, the grouping function Γ is defined, and the grouping is performed according to the grouping rule

[0139] G1={P image1}

[0140] If ∣Y′ i -Y′ i-1 ∣≤I err

[0141] G j ={P′ imagei If |Y′ i -Y′ i-1 |>I err At this point, start a new group G. j+1

[0142] The final grouped set is G = {G1, G2, ..., Gk}

[0143] Define a sorting function σx to sort the image coordinates in the Y direction.

[0144]

[0145] Finally, the sorted P′ is obtained image .

[0146] Nine mechanical coordinates are obtained on industrial computer 1, with the X and Y coordinates denoted as P. robot-x P robot-y And the image coordinates are P. image Then convert the image coordinate matrix into a square matrix P. image-s Calculate the calibration matrix, complete the calibration, and export the calibration data.

[0147] Convert the image coordinate matrix into a square matrix:

[0148] P image-s =P image T ·P image

[0149] Calculate the transformation matrices for X and Y respectively:

[0150] T cal_x =P image-s -1 ·P image T ·P robot-x

[0151] T cal_y =P image-s -1 ·P image T ·P robot-x

[0152] T cal =[T cal_x T cal_y ]

[0153] Switch calibration fixture 5 to suction cup fixture 7, place the product in the camera's field of view, and the industrial computer 1 will adjust the calibration data T according to the calibration data. cal Calculate the mechanical coordinates P of the product.robot The robot 3 reaches the coordinate position to grab the product.

[0154] P robot = T cal · P' image

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not limited thereto; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A robot vision calibration device, characterized in that, include: Industrial control computers; The vision system, connected to the industrial control computer, is used to acquire images of the calibration board; The robot is connected to the industrial control computer, and its end effector is used to install calibration fixtures. A calibration fixture, installed at the end of the robot, includes a floating mechanism, a displacement sensor, and a calibration needle. The floating mechanism is used to drive the calibration needle to slide along a first direction and a second direction, and the displacement sensor is used to detect the offset value of the calibration needle. A calibration plate is placed on a workbench and has three reference holes and nine calibration rings. The reference holes have a tapered guide structure that gradually tapers inward to guide the calibration needles into place.

2. The robot vision calibration device according to claim 1, characterized in that, The floating mechanism of the calibration fixture includes: The first guide rail is fixed to the end of the robot; The first slider is slidably connected to the first guide rail; The second guide rail is fixed on the first slider, and its extension direction is perpendicular to the first guide rail. The second slider is slidably connected to the second guide rail; The first sliding plate is fixed to the second slider; The third guide rail is fixed on the first slide plate, and its extension direction is perpendicular to the second guide rail; The third slider is slidably connected to the third guide rail; The second slide plate is fixed on the third slider, and the calibration pin is mounted on the second slide plate.

3. The robot vision calibration device according to claim 1, characterized in that, The displacement sensor includes a first displacement sensor and a second displacement sensor. The first displacement sensor is used to detect the offset value of the first slide plate along a first direction, and the second displacement sensor is used to detect the offset value of the second slide plate along a second direction.

4. The robot vision calibration device according to claim 1, characterized in that, The calibration plate has three reference holes arranged in a triangle, and the nine calibration rings are arranged in a 3×3 matrix within the area enclosed by the three reference holes. The reference hole's tapered guide structure includes a tapered surface that gradually tapers inward toward the calibration plate, used to guide the calibration needle to slide along the tapered surface to the center of the reference hole.

5. The robot vision calibration device according to claim 1, characterized in that, The vision system is installed at the end of the robot or fixedly mounted above the worktable.

6. The robot vision calibration device according to claim 5, characterized in that, When the vision system is installed at the end of the robot, it is used alternately with the calibration fixture; when the vision system is fixedly installed above the worktable, the robot carries the calibration fixture to complete the reference hole positioning and then moves to a position that does not obstruct the field of view of the vision system.

7. A robot vision calibration method, characterized in that, The method includes: Initialize the position information of the calibration board on the worktable; The robot carrying the calibration fixture descends to the calibration plate. The calibration needle of the calibration fixture contacts the conical surface of the reference hole of the calibration plate. It slides along the first and second directions through the floating mechanism to compensate for the offset until the calibration needle is embedded in the reference hole. The mechanical coordinates of the robot at this time are recorded. Repeat the process to sequentially insert the calibration pin into the three reference holes of the calibration plate, and record the corresponding robot mechanical coordinates respectively; Based on the first robot mechanical coordinates of the three reference holes, the second robot mechanical coordinates of the nine calibration rings on the calibration plate are calculated; The vision system acquires images of the calibration plate, identifies the image coordinates of the nine calibration circles, and automatically sorts them. The visual calibration matrix is ​​calculated by matching the corresponding mechanical coordinates of the second robot in the nine calibration circles with the sorted image coordinates.

8. The robot vision calibration method according to claim 7, characterized in that, The robot, carrying a calibration fixture, descends to the calibration plate. The calibration needle of the calibration fixture contacts the conical surface of the reference hole on the calibration plate. A floating mechanism slides along a first and second direction to compensate for offset until the calibration needle is embedded in the reference hole. The robot's mechanical coordinates at this point are recorded, specifically including: The robot carries the calibration fixture and descends vertically, so that the tip of the calibration needle contacts the conical surface of the reference hole; The reaction force applied by the conical surface pushes the calibration needle to slide the second slide along the third guide rail. The second displacement sensor detects the offset value of the second slide along the second direction. At the same time, the sliding of the second slide causes the first slide to slide along the second guide rail. The first displacement sensor detects the offset value of the first slide along the first direction. When the detection values ​​of the first displacement sensor and the second displacement sensor are both less than a preset threshold, it is determined that the calibration needle is embedded in the center of the reference hole, the downward probe is stopped, and the mechanical coordinates of the robot at this time are recorded.

9. The robot vision calibration method according to claim 7, characterized in that, The steps for calculating the robot's mechanical coordinates for nine calibration cycles based on the robot's mechanical coordinates from three reference holes specifically include: The robot's mechanical coordinates of the three reference holes are respectively set as the first reference point, the second reference point, and the third reference point; Calculate the first unit vector from the first reference point to the second reference point and the second unit vector from the first reference point to the third reference point, and construct the rotation transformation matrix from the calibration plate to the robot coordinate system; Based on the rotation transformation matrix, the original coordinates of the nine calibration rings on the calibration plate are converted from the original coordinates of the calibration plate itself to the mechanical coordinates in the robot coordinate system, thus obtaining the robot mechanical coordinates of the nine calibration rings.

10. The robot vision calibration method according to claim 7, characterized in that, The specific steps of the vision system identifying and automatically sorting the image coordinates of the nine calibration circles include: The vision system uses a sub-pixel edge detection algorithm to extract the center coordinates of the nine calibration circles to obtain an initial image coordinate set. Based on the preset second direction sorting direction and second direction error threshold, the initial image coordinate set is grouped in the second direction, and coordinates with coordinate differences in the second direction less than the second direction error threshold are grouped into the same group; For the coordinates within each group, sort them according to the preset first direction sorting direction to obtain an ordered sequence of image coordinates; The ordered sequence of image coordinates is output and used to match the robot's mechanical coordinates with the nine calibration circles.

Images