A vision robot system

Abstract

The present invention discloses a vision robot system, aiming to overcome the problem of inaccurate positioning in the prior art. It is used to execute the processes of establishing the workpiece coordinate system and the tool coordinate system, the N-point calibration process, the template making process, the feature image processing process, the pose adaptation process, and the height automatic adjustment process. The system includes a robot, a two-dimensional camera, a workbench, a workpiece, and a feature pattern. The robot includes an execution mechanism. The two-dimensional camera is arranged at the end of the execution mechanism. The workbench is used for placing and clamping the workpiece. The feature pattern is used for attaching to the workpiece and includes feature points and feature line segments.

Description

Technical Field

[0001] The present invention belongs to the field of machine vision technology, and particularly relates to a vision robot system. Background Art

[0002] The positioning accuracy of a robot is an important indicator to measure its working performance. At present, due to manufacturing, installation and other factors, most of the vision robots produced by domestic and foreign manufacturers have low positioning accuracy and cannot meet the needs of high-precision processing. Therefore, analyzing various factors causing robot positioning errors and maximizing the absolute positioning accuracy of the robot have become the core research content of machine vision technology.

[0003] At present, the commonly used robot calibration methods at home and abroad usually rely on advanced external measurement equipment to complete, but this leads to problems such as high cost, complex measurement process and the need for professional operation. At the same time, in the process of coordinate system conversion, coordinate system conversion errors are easily introduced, resulting in large measurement errors.

[0004] In view of the above situation, the present invention proposes a vision robot system to solve the problem of positioning and calibration. Summary of the Invention

[0005] To overcome the deficiencies and problems of the prior art, the present invention provides a vision robot system and its fusion method.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] On the one hand, the present invention provides a vision robot system for performing the processes of establishing a workpiece coordinate system and a tool coordinate system, performing a template making process, a template making process, a feature point position approximation process, a feature image processing process, an attitude adaptive process, and a height automatic adjustment process. The system includes:

[0008] A robot, which includes an execution mechanism;

[0009] A two-dimensional camera, which is arranged at the end of the execution mechanism;

[0010] A workbench, on which workpieces are positioned;

[0011] A feature pattern, which includes feature points and feature line segments.

[0012] Preferably, the process of establishing the workpiece coordinate system and the tool coordinate system includes the process of the vision robot system establishing the workpiece coordinate system and the tool coordinate system, wherein the tool coordinate system includes the TCP position.

[0013] Preferably, the N-point calibration process includes the vision robot system establishing a transformation matrix between the image coordinate system and the physical coordinate system of the robot, where N is a positive integer.

[0014] Preferably, the template manufacturing process includes the vision robot system adjusting the shooting pose of the two-dimensional camera for the feature pattern, triggering the two-dimensional camera to take a picture and obtain an image, processing the image to obtain the pixel contour of the feature pattern, using the pixel contour of the feature pattern as the positioning template, and calculating the physical geometric parameters of the positioning template. Among them, the feature pattern includes feature points and feature line segments.

[0015] Preferably, the feature point position approximation process includes the vision robot system setting the feature pattern on the workpiece, triggering the camera to take a picture and obtain an image, performing a feature image processing process on the image to obtain the physical geometric parameters of the feature contour, and controlling the camera to translate on the X-axis and Y-axis of the robot physical coordinate system so that the physical geometric parameters of the feature contour approximate the physical geometric parameters of the positioning template.

[0016] Preferably, the feature image processing process includes the vision robot system processing the image to obtain the contour of the feature pattern, determining whether the contour of the feature pattern matches the positioning template. If so, taking the contour of the feature pattern as the feature contour and calculating the physical geometric parameters of the feature contour according to the feature contour. If not, readjusting the shooting pose of the two-dimensional camera for the feature pattern, triggering the two-dimensional camera to take a picture and obtain an image. Among them, the feature contour is the pixel contour of the feature pattern.

[0017] Preferably, the pose adaptation process includes the vision robot system respectively controlling the two-dimensional camera to rotate around the X-axis, Y-axis and Z-axis of the robot physical coordinate system so that the physical geometric parameters of the feature contour approximate the physical geometric parameters of the positioning template.

[0018] Preferably, the height automatic adjustment process includes the vision robot system controlling the two-dimensional camera to translate on the Z-axis of the robot physical coordinate system so that the physical geometric parameters of the feature contour approximate the physical geometric parameters of the positioning template.

[0019] On the other hand, the present invention also provides a vision robot fusion method, which is executed by using the above-mentioned vision robot system, and includes the following steps:

[0020] Workpiece coordinate system and tool coordinate system establishment process: Establish the workpiece coordinate system and the tool coordinate system process, where the tool coordinate system includes the TCP position;

[0021] N-point calibration process: Establish the conversion matrix between the image coordinate system and the robot physical coordinate system, where N is a positive integer;

[0022] Template making process: Adjust the shooting pose of the 2D camera for the feature pattern, trigger the 2D camera to take a photo and obtain an image, process the image to obtain the pixel contour of the feature pattern, use the pixel contour of the feature pattern as the positioning template, and calculate the physical geometric parameters of the positioning template. Among them, the feature pattern includes feature points and feature line segments;

[0023] Feature point position approximation process: Set the feature pattern on the workpiece, trigger the camera to take a photo and obtain an image, execute the feature image processing process on the image to obtain the physical geometric parameters of the feature contour, and control the camera to translate on the X-axis and Y-axis of the robot physical coordinate system so that the physical geometric parameters of the feature contour approach the physical geometric parameters of the positioning template;

[0024] Feature image processing process: Process the image to obtain the contour of the feature pattern, determine whether the contour of the feature pattern matches the positioning template. If so, use the contour of the feature pattern as the feature contour and calculate the physical geometric parameters of the feature contour according to the feature contour. If not, readjust the shooting pose of the 2D camera for the feature pattern, trigger the 2D camera to take a photo and obtain an image. Among them, the feature contour is the pixel contour of the feature pattern;

[0025] Pose self-adaptation process: Control the 2D camera to rotate around the X-axis, Y-axis and Z-axis of the robot physical coordinate system respectively so that the physical geometric parameters of the feature contour approach the physical geometric parameters of the positioning template;

[0026] Height automatic adjustment process: Control the 2D camera to translate on the Z-axis of the robot physical coordinate system so that the physical geometric parameters of the feature contour approach the physical geometric parameters of the positioning template.

[0027] Preferably, the workpiece coordinate system and tool coordinate system establishment process specifically includes the following steps:

[0028] Step 111: Establish the tool coordinate system and determine the TCP position, and set the TCP position at the visual center point. Among them, the visual center refers to the center of the lower end face of the 2D camera lens;

[0029] Step 112: Determine the working surface of the robot through the workpiece coordinate system. The XY plane of the workpiece coordinate system is established on the machining surface. Among them, the machining surface is the cutting plane on the feature pattern, and the common point of the machining surface and the feature pattern is the feature point.

[0030] Preferably, the N-point calibration process specifically includes the following process:

[0031] Step 121: Set the TCP position at the visual center;

[0032] Step 122: Control the 2D camera to face the machining surface and control the object distance to reach the set value;

[0033] Step 123: Set N calibration points on the machining surface. The physical coordinates of the N calibration points are (X1, Y1, Z1), (X2, Y2, Z2)…(X N , Y N , Z N ). Control the TCP to move to the N calibration points and obtain calibration images. Determine the pixel coordinates of the TCP at the N calibration points according to the calibration images. The pixel coordinates of the TCP at the N calibration points are (x1, y1), (x2, y2)…(x N , y N );

[0034] Step 124: Substitute the physical coordinates of the N calibration points and the pixel coordinates of the TCP at the N calibration points into the following formula and calculate the transformation matrix according to the least squares method:

[0035]

[0036] In the formula, the transformation matrix is a, d, b, and e are respectively rotation components, and c and f are respectively translation components.

[0037] Preferably, the template manufacturing process specifically includes the following steps:

[0038] Step 131: Adjust the shooting pose of the 2D camera for the feature pattern, trigger the 2D camera to take a photo and obtain an image. The feature pattern includes feature points and feature line segments; Step 132: Obtain the pixel contour of the feature pattern and use the pixel contour of the feature pattern as the positioning template; Step 133: Calculate the pixel geometric parameters of the positioning template according to the positioning template, and convert the pixel geometric parameters of the positioning template into physical geometric parameters of the positioning template through the transformation matrix. The geometric parameters of the positioning template include the position parameters of the feature points and the dimension parameters of the feature line segments.

[0039] Preferably, the feature point position approximation process specifically includes the following steps:

[0040] Step 211: Set the feature pattern on the workpiece and position the workpiece on the workbench;

[0041] Step 212: Trigger the 2D camera to take a photo and obtain an image;

[0042] Step 213: Perform the feature image processing process on the image to obtain the physical geometric parameters of the feature contour;

[0043] Step 214: Calculate the offset according to the physical geometric parameters of the positioning template and the physical geometric parameters of the feature contour. The offset includes ΔT X and ΔT Y , ΔT xΔT is the offset of the relative positioning template of the feature profile on the X-axis of the robot's physical coordinate system Y ΔT is the offset of the relative positioning template of the feature profile on the Y-axis of the robot's physical coordinate system;

[0044] Step 215: Determine whether ΔT X is less than the threshold ΔX i , if ΔT X is not less than the threshold ΔX i then determine whether ΔT X exceeds the threshold If ΔT X exceeds the threshold then control the two-dimensional camera to translate on the X-axis of the robot's physical coordinate system to ΔT X not exceeding the threshold If ΔT X does not exceed the threshold then control the two-dimensional camera to iteratively translate on the X-axis of the robot's physical coordinate system by the distance until ΔT X is less than If ΔT X is less than then control the two-dimensional camera to translate by the distance of ΔT on the X-axis of the robot's physical coordinate system X , where the threshold is greater than 4 times the threshold ΔX i ;

[0045] Step 216: If ΔT X is less than the threshold ΔX i then determine whether ΔT Y is less than the threshold ΔY i , if ΔT Y is not less than the threshold ΔY i then determine whether ΔT Y exceeds the threshold If ΔT Y exceeds the threshold then control the two-dimensional camera to translate on the Y-axis of the robot's physical coordinate system to ΔT Y not exceeding the threshold If ΔT Y does not exceed the threshold then control the two-dimensional camera to iteratively translate on the Y-axis of the robot's physical coordinate system by the distance until ΔT Y is less than If ΔT Y is less than then control the two-dimensional camera to translate by the distance of ΔT on the Y-axis of the robot's physical coordinate system Y , where the threshold is greater than 4 times the threshold ΔYi 。

[0046] Preferably, the feature image processing process specifically includes the following steps:

[0047] S11: The vision robot system extracts the edge points of the pattern in the image according to the Canny algorithm;

[0048] S12: The vision robot system generates a pattern contour based on the extracted edge points of the pattern, where the pattern contour is composed of a number of edge points;

[0049] S13: The vision robot system calculates the number of edge points and the aspect ratio of the pattern contour based on the pattern contour, and determines whether the number of edge points of the pattern contour is within the safe number range and whether the aspect ratio of the pattern contour is within the safe aspect ratio range. If so, the corresponding pattern contour is retained; otherwise, the corresponding pattern contour is removed;

[0050] S14: The vision robot system determines whether the difference between an edge point of the retained pattern contour and an adjacent edge point is within the gradient change threshold. If so, the edge point is determined as a contour intersection point, and it is determined whether the number of contour intersection points of the pattern contour is within the safe intersection number range. If so, the contour of the feature pattern is reconstructed based on the contour intersection points of the pattern contour;

[0051] S15: The vision robot system calculates the pixel geometric parameters of the feature pattern based on the contour of the feature pattern, converts the pixel geometric parameters of the feature pattern into physical geometric parameters of the feature pattern according to the transformation matrix, and determines whether the pixel geometric parameters of the feature pattern and the physical geometric parameters of the positioning template form a similar relationship. If so, it is determined that the feature pattern and the positioning template match successfully, and the contour of the feature pattern is used as the feature contour; otherwise, it is determined that the feature pattern and the positioning template do not match successfully, and the shooting pose of the two-dimensional camera for the feature pattern is readjusted and an image is acquired.

[0052] Preferably, the attitude adaptive process specifically includes the following steps:

[0053] Step 221: Trigger the two-dimensional camera to take a picture and acquire an image;

[0054] Step 222: Perform the feature image processing process on the image to obtain the physical geometric parameters of the feature contour;

[0055] Step 223: Calculate the offset based on the physical geometric parameters of the feature contour and the physical geometric parameters of the positioning template. The offset includes θ Z1 、θ X1 and θ Y1 ,where θ Z1 is the offset angle of the feature contour relative to the positioning template on the Z axis of the robot physical coordinate system, and θ X1is the offset angle of the feature profile relative positioning template on the X-axis of the robot's physical coordinate system, θ Y1 is the offset angle of the feature profile relative positioning template on the Y-axis of the robot's physical coordinate system;

[0056] Step 224: Determine whether θ Z1 is less than the threshold θ Z , if not, control the two-dimensional camera to rotate around the Z-axis of the robot's physical coordinate system until it is less than the threshold θ Z ;

[0057] Step 225: If θ Z1 is less than the threshold θ Z then determine whether θ X1 is less than the threshold θ X , if θ X1 is not less than the threshold θ X then determine whether θ X1 is greater than the threshold If θ X1 is greater than the threshold then control the two-dimensional camera to rotate around the X-axis of the robot's physical coordinate system until θ X1 is not greater than the threshold If θ X1 is not greater than the threshold then control the two-dimensional camera to iteratively rotate around the X-axis of the robot's physical coordinate system by the angle until θ X1 is less than If θ Z1 is less than then control the two-dimensional camera to rotate around the X-axis of the robot's physical coordinate system by the angle of θ X1 , where the threshold θ X is greater than 4 times the threshold

[0058] Step 226: If θ X1 is less than the threshold θ X then execute the feature point approximation process, and then determine whether θ Y1 is less than the threshold θ Y , if θ Y1 is not less than the threshold θ Y then determine whether θ Y1 is greater than the threshold If θ Y1 is greater than the threshold then control the two-dimensional camera to rotate around the Y-axis of the robot's physical coordinate system until θ Y1 is not greater than the threshold If θ Y1 is not greater than the threshold then control the two-dimensional camera to iteratively rotate around the Y-axis of the robot's physical coordinate system by the angle until θY1 Less than If θ Y1 Less than Then control the two - dimensional camera to rotate around the Y - axis of the robot's physical coordinate system by an angle of θ Y1 where the threshold θ Y Is greater than 4 times the threshold

[0059] Step 227: If θ Y1 Is less than the threshold θ Y Then execute the feature point position approximation process and record the current pose information of the robot.

[0060] Preferably, the height automatic adjustment process includes the following steps:

[0061] Step 241: Trigger the two - dimensional camera to take a picture and obtain an image;

[0062] Step 242: Execute the feature image processing process on the image to obtain the physical geometric parameters of the feature contour;

[0063] Step 243: Calculate the offset according to the physical geometric parameters of the feature contour and the physical geometric parameters of the positioning template. The offset includes ΔH, and ΔH is the height offset of the feature contour relative to the positioning template;

[0064] Step 244: Determine whether ΔH is less than the threshold ΔH1. If ΔH is less than the threshold ΔH1, then determine whether ΔH is greater than the threshold ΔH2. If ΔH is greater than the threshold ΔH2, then control the two - dimensional camera to translate on the Z - axis of the robot's physical coordinate system until ΔH is not greater than the threshold ΔH2. If ΔH is not greater than the threshold ΔH2, then control the two - dimensional camera to iteratively translate by a distance until ΔH is less than If ΔH is less than Then control the two - dimensional camera to translate on the Z - axis of the robot's physical coordinate system by a distance of ΔH, where the threshold ΔH2 is greater than 4 times the threshold ΔH1.

[0065] The prominent and beneficial technical effects of the present invention compared with the prior art are:

[0066] (1) In the present invention, by processing the image captured by the two - dimensional camera, the position adjustment of the two - dimensional camera, the workpiece position positioning and the robot pose adjustment are realized, thus achieving the effect of positioning and processing the workpiece by the actuator at the feature position. It is suitable for positioning and processing workpieces with complex surfaces. Therefore, the present invention has the advantages of high positioning efficiency and accurate positioning.

[0067] (2) In the present invention, during the positioning process, a method combining successive approximation and coarse and fine adjustment is adopted to gradually approximate the pixel contour to the positioning template. On the one hand, the efficiency of adjusting the position of the two-dimensional camera is improved, and on the other hand, the positioning error of the two-dimensional camera is reduced.

[0068] (3) In the present invention, the postures of the robot and the actuator are strictly positioned to avoid abnormal postures interfering with the subsequent machining of the workpiece by the robot, ensuring the machining accuracy and quality of the workpiece. Description of the Drawings

[0069] Figure 1 is a schematic structural diagram of the vision robot system of the present invention;

[0070] Figure 2 is a schematic structural diagram of the characteristic pattern of the present invention;

[0071] Figure 3 is a schematic overall flow diagram of the vision robot fusion method of the present invention;

[0072] Figure 4 is a schematic diagram of the distribution of 9 calibration points on the workpiece in the vision robot fusion method of the present invention;

[0073] Figure 5 is a schematic diagram of the template manufacturing process in the vision robot fusion method of the present invention;

[0074] Figure 6 is a schematic diagram of the characteristic point position approximation process in the vision robot fusion method of the present invention;

[0075] Figure 7 is a schematic diagram of the characteristic point position in the coarse and fine adjustment processes in the X direction of the present invention;

[0076] Figure 8 is a schematic diagram of the posture self-adaptation process in the vision robot fusion method of the present invention;

[0077] Figure 9 is a schematic diagram of the height automatic adjustment process in the vision robot fusion method of the present invention;

[0078] In the figure: 1 - robot, 2 - two-dimensional camera, 3 - workbench, 4 - workpiece, 5 - characteristic pattern, 6 - tool, 11 - actuator. Detailed Embodiments

[0079] For the convenience of those skilled in the art to understand, the present invention will be further described below in conjunction with the drawings and specific embodiments.

[0080] As Figure 1 shown, a vision robot system includes a robot, a two-dimensional camera, a workbench, a workpiece, a characteristic pattern, a photoelectric switch, and a touch screen.

[0081] The robot includes an actuating mechanism. The actuating mechanism is a robotic arm that converts the control signal of the robot into corresponding posture actions and is the main entity for the robot to complete work tasks. In this embodiment, the actuating mechanism adopted belongs to the prior art and is also a robotic arm composed of a series of connecting rods, mechanical joints, pose sensors, etc. A base is installed at the bottom of the actuating mechanism, and a tool is installed at the end of the actuating mechanism. The tool is the main implement for machining the workpiece during work. In this embodiment, the tool is a cutting tool.

[0082] The two-dimensional camera is used to take images. The robot generates a control signal for controlling the actuating mechanism by processing the images. The two-dimensional camera is also installed at the end of the actuating mechanism. There is an end flange at the end of the actuating mechanism. The tool and the two-dimensional camera are respectively arranged on the end flange, and the axial direction of the tool and the shooting direction of the two-dimensional camera are parallel to each other to facilitate subsequent positioning and calibration.

[0083] The workbench is used to place and clamp the workpiece and plays a role in positioning the workpiece. The workbench is arranged near the robot. In this embodiment, the workbench includes several movable positioning blocks. In actual use, when the workpiece is correctly placed on the workbench, the positioning blocks move to abut against the workpiece, and the positioning blocks position the workpiece on the workbench.

[0084] The photoelectric switch (not shown in the figure) is used to detect the position of the workpiece on the workbench to help the workpiece be correctly positioned on the workbench. The photoelectric switch is electrically connected to the robot. The photoelectric switch is arranged on the workbench, and the detection range of the photoelectric switch is set on the workbench, so as to realize the photoelectric switch detecting the position of the workpiece on the workbench.

[0085] The touch screen (not shown in the figure) is used to realize human-machine interaction. In actual use, the touch screen can real-time display the current pose state of the actuating mechanism.

[0086] The workpiece is the part to be machined by this robot system. The workpiece has a machining surface, and the tool can start machining on the machining surface. In this embodiment, the workpiece is a car, and the machining surface is a plane tangent to the feature pattern.

[0087] The feature pattern is used to be attached to the workpiece. In actual use, the feature pattern is attached to a specific position of the workpiece. The specific position of the workpiece can be determined in advance and manually. The camera takes pictures of the feature pattern, and the robot realizes the recognition and positioning of the workpiece by recognizing and positioning the feature pattern. In this embodiment, the specific position on the workpiece can be the machining surface of the workpiece.

[0088] In this embodiment, the feature pattern is a soft film with a specific pattern on its surface. Such as Figure 2As shown in the figure, the specific pattern includes a rectangle and an equilateral triangle intersecting with each other. One end point of the equilateral triangle is set at the center of the rectangle. The feature pattern includes feature points and feature line segments. The end points of the feature line segments are feature points. The feature points are represented by g, h, i, j, and k, and the feature line segments are represented by A1, A2, B1, and B2. The feature point g is located at the intersection of the equilateral triangle and the rectangle and is also located at the midpoint of the side of the equilateral triangle. The feature points h and k are located at the end points of the rectangle. The feature points i and j are located at the corners of the equilateral triangle. The feature line segment A1 is the line segment between the feature points i and g. The feature line segment A2 is the line segment between the feature points g and j. The feature line segment B1 is the line segment between the feature points h and g. The feature line segment B2 is the line segment between the feature points g and k. In actual use, the feature pattern can be fully attached to the processing surface of the workpiece, so as to have an accurate positioning effect on the curved surface.

[0089] This vision robot system is used for the processes of establishing the workpiece coordinate system and the tool coordinate system, executing the template making process, the template making process, the feature point position approximation process, the feature image processing process, the attitude adaptation process, and the height automatic adjustment process.

[0090] The process of establishing the workpiece coordinate system and the tool coordinate system includes the process of the vision robot system establishing the workpiece coordinate system and the tool coordinate system. Among them, the tool coordinate system includes the TCP position.

[0091] The N-point calibration process includes the vision robot system establishing the transformation matrix between the image coordinate system and the physical coordinate system of the robot. Among them, N is a positive integer.

[0092] The template making process includes the vision robot system adjusting the shooting pose of the two-dimensional camera for the feature pattern, triggering the two-dimensional camera to take pictures and obtaining images, processing the images to obtain the pixel contour of the feature pattern, using the pixel contour of the feature pattern as the positioning template, and calculating the physical geometric parameters of the positioning template. Among them, the feature pattern includes feature points and feature line segments;

[0093] In the template making process, the two-dimensional camera can obtain images at multiple shooting poses of the feature pattern and make multiple positioning templates. In the subsequent feature image processing process, the contour of the feature pattern can be matched with the positioning templates at multiple shooting poses. The two-dimensional camera can also accurately match the corresponding positioning template at different poses, thus increasing the matching range.

[0094] The feature point position approximation process includes the vision robot system setting the feature pattern on the workpiece, triggering the camera to take pictures and obtaining images, performing the feature image processing process on the images to obtain the physical geometric parameters of the feature contour, and controlling the camera to translate on the X-axis and Y-axis of the physical coordinate system of the robot so that the physical geometric parameters of the feature contour approach the physical geometric parameters of the positioning template;

[0095] The described feature image processing process includes the vision robot system processing the image to obtain the contour of the feature pattern, determining whether the contour of the feature pattern matches the positioning template. If so, the contour of the feature pattern is used as the feature contour and the physical geometric parameters of the feature contour are calculated based on the feature contour. If not, the shooting pose of the two-dimensional camera for the feature pattern is readjusted, the two-dimensional camera is triggered to take a picture, and an image is obtained. Among them, the feature contour is the pixel contour of the feature pattern.

[0096] The described pose self-adaptation process includes the vision robot system respectively controlling the two-dimensional camera to rotate around the X-axis, Y-axis, and Z-axis of the physical coordinate system of the robot so that the physical geometric parameters of the feature contour approach the physical geometric parameters of the positioning template.

[0097] The described height automatic adjustment process includes the vision robot system controlling the two-dimensional camera to translate on the Z-axis of the physical coordinate system of the robot so that the physical geometric parameters of the feature contour approach the physical geometric parameters of the positioning template.

[0098] On the other hand, the present invention also provides a vision robot fusion method, which is executed by using the above-mentioned vision robot system. The steps of this vision robot fusion method include a preprocessing process and a real-time processing process. The specific steps of the preprocessing include a workpiece coordinate system and a tool coordinate system establishment process, an N-point calibration process, and a template making process. The real-time processing process includes a feature point position approximation process, a feature image processing process, a pose self-adaptation process, and a height automatic adjustment process.

[0099] As Figure 3 shown, in the actual positioning process, the vision robot system starts. The vision robot system determines whether it is necessary to execute the preprocessing process. If not, the vision robot system enters the determination of whether it is necessary to execute the real-time processing process. If so, the vision robot system displays the interface of the preprocessing process on the touch screen. The vision robot system determines whether it is necessary to establish the workpiece coordinate system and the tool coordinate system. If so, the workpiece coordinate system and the tool coordinate system establishment process is executed. If not, it is determined whether it is necessary to execute the N-point calibration process. If so, the N-point calibration process is executed. If not, it is determined whether it is necessary to execute the template making process. If so, the template making process is executed and it returns to the determination of whether it is necessary to execute the preprocessing process. If not, it returns to the determination of whether it is necessary to execute the preprocessing process. The vision robot system determines whether it is necessary to execute the real-time processing process. If so, the feature point position approximation process and the pose self-adaptation process are executed in sequence. After the execution of the pose self-adaptation process, the vision robot system determines whether the pose adjustment angles (i.e., the offsets θ Z1 、θ X1 and θ Y1 ) exceed the thresholds (i.e., the threshold θ Z 、the threshold θ X and the threshold θ Y) If so, re - execute the feature point position approximation process and the pose adaptation process; if not, execute the height automatic adjustment process, then determine whether the total height adjustment value is less than the threshold ΔH3. If so, the vision robot system ends; if not, execute the feature point approximation process and then end. If the real - time processing process does not need to be executed, then determine whether to exit the system. If so, the vision robot system ends; if not, return to determine whether the pre - processing process needs to be executed.

[0100] Workpiece coordinate system and tool coordinate system establishment process: The vision robot system establishes the workpiece coordinate system and the tool coordinate system process. Among them, the tool coordinate system includes the TCP position. In this embodiment, as Figure 2 shown, the X - axis and Y - axis in the figure respectively represent the X - axis and Y - axis of the workpiece coordinate system. By establishing the workpiece coordinate system, the X - axis and Y - axis of the workpiece coordinate system can be set on the processing surface of the workpiece, so as to accurately position the feature pattern on the processing surface.

[0101] N - point calibration process: The vision robot system establishes the transformation matrix between the image coordinate system and the robot physical coordinate system, where N is a positive integer. In this embodiment, the value of N is 9. Through the N - point calibration process, the transformation between the image coordinate system and the robot physical coordinate system is realized.

[0102] Template making process: The vision robot system adjusts the shooting pose of the two - dimensional camera for the feature pattern, triggers the two - dimensional camera to take pictures and obtains images, processes the images to obtain the pixel contour of the feature pattern, uses the pixel contour of the feature pattern as the positioning template, and calculates the physical geometric parameters of the positioning template. Among them, the feature pattern includes feature points and feature line segments. In this embodiment, the positioning template is used to adjust the poses of the two - dimensional camera, the actuator and the robot relative to the workpiece after the workpiece is loaded onto the workbench.

[0103] In the template making process, the two - dimensional camera can obtain images at multiple shooting poses of the feature pattern and make multiple positioning templates. In the subsequent feature image processing process, the contour of the feature pattern can be matched with the positioning templates at multiple shooting poses, and the two - dimensional camera can accurately match the corresponding positioning template at different poses, thereby increasing the matching range.

[0104] Feature point position approximation process: The vision robot system sets the feature pattern on the workpiece, triggers the camera to take pictures and obtains images, executes the feature image processing process on the images to obtain the physical geometric parameters of the feature contour, and controls the two - dimensional camera to translate on the X - axis and Y - axis of the robot physical coordinate system so that the physical geometric parameters of the feature contour approach the physical geometric parameters of the positioning template.

[0105] Feature image processing process: The vision robot system processes the image to obtain the contour of the feature pattern, determines whether the contour of the feature pattern matches the positioning template. If so, the contour of the feature pattern is used as the feature contour, and the physical geometric parameters of the feature contour are calculated based on the feature contour. If not, the shooting pose of the 2D camera for the feature pattern is readjusted, the 2D camera is triggered to take a picture, and an image is obtained. Among them, the feature contour is the pixel contour of the feature pattern.

[0106] Pose self - adaptation process: The vision robot system controls the 2D camera to rotate around the X - axis, Y - axis, and Z - axis of the robot's physical coordinate system respectively, so that the physical geometric parameters of the feature contour approach the physical geometric parameters of the positioning template.

[0107] Height automatic adjustment process: The vision robot system controls the 2D camera to translate along the Z - axis of the robot's physical coordinate system, so that the physical geometric parameters of the feature contour approach the physical geometric parameters of the positioning template.

[0108] The process of establishing the workpiece coordinate system and the tool coordinate system specifically includes the following steps:

[0109] Step 111: The vision robot system establishes the tool coordinate system and determines the TCP position, and sets the TCP position at the vision center. Among them, the vision center refers to the center of the lower end face of the 2D camera lens.

[0110] Step 112: The lower end face of the 2D camera lens is the end face exposed outside the 2D camera. The vision robot system determines the position of the machining surface of the workpiece through the workpiece coordinate system. The XY plane of the workpiece coordinate system is established on the machining surface. Among them, the machining surface is the section plane on the feature pattern, and the common point of the machining surface and the feature pattern is the feature point.

[0111] Preferably, for the N - point calibration process, the value of N is 9, and it specifically includes the following process:

[0112] Step 121: The vision robot system sets the TCP position at the vision center. Among them, the vision center refers to the center of the lower end face of the 2D camera lens.

[0113] Step 122: The vision robot system controls the 2D camera to face the machining surface and controls the object distance to reach the set value.

[0114] In the above steps, the position of the 2D camera can be adjusted by the robot. When the 2D camera faces the machining surface, the lower end face of the 2D camera lens is parallel to the machining surface. The set value can be preset in the vision robot system. When the object distance reaches the set value, the focal length between the 2D camera and the machining surface is appropriate, and the 2D camera is in clear focus.

[0115] Step 123: The vision robot system sets N calibration points on the machining surface. The physical coordinates of the N calibration points are (X1, Y1, Z1), (X2, Y2, Z2)…(X N ,Y N ,Z N ), respectively. Control the TCP to move to the N calibration points and obtain calibration images. Determine the pixel coordinates of the TCP at the N calibration points according to the calibration images. The pixel coordinates of the TCP at the N calibration points are (x1, y1), (x2, y2)…(x N ,y N ), respectively;

[0116] In the above steps, as Figure 4 shown, it is a schematic diagram of the distribution of 9 calibration points on the machining surface. The points from left to right and from top to bottom are the 1st, 2nd…the Nth calibration points. In this embodiment, the calibration points can be set on the feature point a of the feature pattern. (X1, Y1, Z1), (X2, Y2, Z2)…(X N ,Y N ,Z N ) can be expressed as (x1, y1), (x2, y2)…(x N ,y N ) can be expressed as and

[0117] Step 124: The vision robot system substitutes the physical coordinates of the N calibration points and the pixel coordinates of the TCP at the N calibration points into the following formula and calculates the transformation matrix according to the least square method:

[0118]

[0119] In the formula, the transformation matrix is a, d, b, and e are respectively rotation components, and c and f are respectively translation components;

[0120] In the above steps, the physical coordinates of the calibration points, the pixel coordinates of the TCP at the calibration points, and the transformation matrix satisfy the following formula:

[0121]

[0122] Expand the above formula to obtain the following formula:

[0123] ax + by + c = X (1),

[0124] dx + ey + f = Y (2),

[0125] In this embodiment, N is 9. Substituting the physical coordinates of the 9 calibration points and the pixel coordinates of the TCP at the 9 calibration points into the above formula (1), the conversion formula between the X-axis of the physical coordinate system and the X-axis of the pixel coordinate system is obtained as follows:

[0126] ax1 + by1 + c = X1

[0127] ax2 + by2 + c = X2

[0128] ax3 + by3 + c = X3

[0129] ax4 + by4 + c = X4

[0130] ax5 + by5 + c = X5

[0131] ax6 + by6 + c = X6

[0132]

[0133] Similarly, substituting the physical coordinates of the 9 calibration points and the pixel coordinates of the TCP at the 9 calibration points into the above formula (2), the conversion formula between the Y-axis of the physical coordinate system and the Y-axis of the pixel coordinate system is obtained.

[0134] Minimize the variance on both sides of the conversion formula between the X-axis of the physical coordinate system and the X-axis of the pixel coordinate system according to the least squares method, and the following formula is obtained:

[0135]

[0136] In the formula, S(a, b, c) is the variance on the X-axis;

[0137] Solve the minimum value of S(a, b, c) by calculating the partial derivatives of S(a, b, c) and making the value of the first-order derivative 0, and then obtain a system of linear equations of three variables for a, b, and c. By calculating this system of linear equations of three variables, the values of a, b, and c can be obtained.

[0138] Similarly, minimize the variance on both sides of the conversion formula between the Y-axis of the physical coordinate system and the Y-axis of the pixel coordinate system according to the least squares method, and the following formula is obtained:

[0139]

[0140] In the formula, S(d, e, f) is the variance on the Y-axis;

[0141] Similarly, the values of d, e, and f can be obtained by calculating S(d, e, f), which will not be repeated here. According to the values of a, b, c, d, e, and f, the transformation matrix is further obtained.

[0142] The template production process specifically includes the following steps:

[0143] Step 131: The vision robot system adjusts the shooting pose of the two-dimensional camera for the feature pattern, triggers the two-dimensional camera to take a photo and obtains an image, where the feature pattern includes feature points and feature line segments;

[0144] In the above steps, the feature pattern is placed in a background without other scenes for shooting, and the obtained image has only the feature pattern as the scene, so as to facilitate obtaining the pixel contour of the feature pattern subsequently. In this embodiment, the two-dimensional camera can take photos and obtain images at multiple poses relative to the feature pattern, and subsequently, the images obtained by the two-dimensional camera at multiple poses can be processed to obtain multiple positioning templates.

[0145] Step 132: The vision robot system obtains the pixel contour of the feature pattern and uses the pixel contour of the feature pattern as a positioning template;

[0146] In the above steps, the vision robot system can obtain the pixel contour of the feature pattern in the image according to the Canny algorithm.

[0147] Step 133: The vision robot system calculates the pixel geometric parameters of the positioning template according to the positioning template, and converts the pixel geometric parameters of the positioning template into physical geometric parameters of the positioning template through a transformation matrix, where the geometric parameters of the positioning template include the position parameters of the feature points and the dimension parameters of the feature line segments.

[0148] As Figure 5 shown, it is a schematic diagram of the template making process. The vision robot system starts to enter the template making process. The actuator of the vision robot system adjusts the shooting pose of the two-dimensional camera for the feature pattern, triggers the two-dimensional camera to take a photo and obtains an image, then obtains the pixel contour of the feature pattern, and then judges whether the contour acquisition is successful. If not, the touch screen of the vision robot system prompts: Template matching fails, please adjust the position. If so, it is set as the positioning template, and the position of the feature points of the positioning template and the pixel dimensions of the feature line segments are obtained and converted into physical dimensions. The feature points and feature line segments of the positioning template correspond to the feature points and feature line segments of the feature pattern. Finally, it is judged whether each dimension value (i.e., the feature line segment in the positioning template) is within the threshold range of the actual dimension (i.e., the feature line segment of the feature pattern). If so, the position of the feature points of the positioning template and the physical dimensions of the feature line segments are recorded as the physical geometric parameters of the positioning template. If not, it is prompted that the physical geometric parameters of the positioning template are incorrect, please adjust the position (i.e., adjust the shooting pose of the two-dimensional camera for the feature pattern).

[0149] The specific steps of the feature point position approximation process are as follows:

[0150] Step 211: The vision robot system sets the feature pattern on the workpiece and positions the workpiece on the workbench;

[0151] In the above steps, the vision robot system can monitor the position of the workpiece on the workbench through an optoelectronic switch. When the workpiece reaches the predetermined position on the workbench, the optoelectronic switch emits a trigger signal, and the vision robot system determines that the workpiece is positioned on the workbench.

[0152] Step 212: The vision robot system triggers the two-dimensional camera to take a picture and obtain an image;

[0153] Step 213: The vision robot system performs a feature image processing process on the image to obtain the physical geometric parameters of the feature contour;

[0154] In the above steps, after matching, the deflection angle of the pixel contour relative to the positioning template can also be recorded, and the two-dimensional camera is rotated according to the deflection angle so that the pixel contour can be aligned with the positioning template, facilitating subsequent correction of the offset.

[0155] Step 214: The vision robot system calculates the offset based on the physical geometric parameters of the positioning template and the physical geometric parameters of the feature contour, where the offset includes ΔT X and ΔT Y , ΔT x is the offset of the feature contour relative to the positioning template on the X-axis of the robot physical coordinate system, and ΔT Y is the offset of the feature contour relative to the positioning template on the Y-axis of the robot physical coordinate system;

[0156] Step 215: The vision robot system determines whether ΔT X is less than the threshold ΔX i . If ΔT X is not less than the threshold ΔX i , then it is determined whether ΔT X exceeds the threshold . If ΔT X exceeds the threshold , then the two-dimensional camera is controlled to translate on the X-axis of the robot physical coordinate system to ΔT X not exceeding the threshold . If ΔT X does not exceed the threshold , then the two-dimensional camera is controlled to iteratively translate on the X-axis of the robot physical coordinate system by the distance until ΔT X is less than . If ΔT X is less than , then the two-dimensional camera is controlled to translate by the distance of ΔT X on the X-axis of the robot physical coordinate system, where the threshold is greater than 4 times the threshold ΔX i ;

[0157] Step 216: If ΔT XLess than the threshold ΔX i When it is, the vision robot system determines ΔT Y Whether it is less than the threshold ΔY i , if ΔT Y Is not less than the threshold ΔY i Then it determines ΔT Y Whether it exceeds the threshold If ΔT Y Exceeds the threshold Then control the two - dimensional camera to translate along the Y - axis of the robot's physical coordinate system to ΔT Y Does not exceed the threshold If ΔT Y Does not exceed the threshold Then control the two - dimensional camera to iteratively translate along the Y - axis of the robot's physical coordinate system By the distance until ΔT Y Is less than If ΔT Y Is less than When it is, then control the two - dimensional camera to translate by the distance of ΔY along the Y - axis of the robot's physical coordinate system, where the threshold i Is greater than 4 times the threshold ΔY i .

[0158] Figure 6 As Shown, it is a schematic diagram of the feature point position approximation process. The vision robot system starts to enter the feature point position approximation process, locates the workpiece in place, that is, step 211. The vision robot system then triggers the two - dimensional camera to take a picture and obtain an image, that is, step 212. The vision robot system then performs the feature image processing process and obtains the offset, that is, steps 213 and 214. The vision robot system then determines whether the image deviation in the X - direction (i.e., ΔT X ) is less than ΔX i , if not, then determine whether the image deviation in the X - direction exceeds If it is, then control the robot to make a rough adjustment in the X - direction and record the adjustment value, if not, then control the robot to make a fine adjustment in the X - direction and record the adjustment value, that is, step 215. The robot making a rough adjustment in the X - direction means controlling the two - dimensional camera to translate along the X - axis of the robot's physical coordinate system to ΔT X Does not exceed the threshold Controlling the robot to make a fine adjustment in the X - direction means controlling the two - dimensional camera to iteratively translate along the X - axis of the robot's physical coordinate system By the distance until ΔT X Is less than If the image deviation in the X - direction is less than ΔX i Then the vision robot system determines whether the image deviation in the Y - direction (i.e., ΔY i ) is less than ΔY i , if not, then determine whether the image deviation in the Y - direction exceeds If so, control the robot to perform coarse adjustment in the Y direction and record the adjustment value. If not, control the robot to perform fine adjustment in the Y direction and record the adjustment value, that is, step 216. Controlling the robot to perform coarse adjustment in the Y direction means controlling the two-dimensional camera to translate along the Y-axis of the robot's physical coordinate system to ΔT Y not exceeding the threshold Controlling the robot to perform fine adjustment in the Y direction means controlling the two-dimensional camera to iteratively translate along the Y-axis of the robot's physical coordinate system by a distance of until ΔT Y is less than If the deviation in the Y direction of the image (i.e., ΔY i ) is less than ΔY i Or after step 216 is completed, the visual robot system ends the feature point position approximation process. The adjustment process that combines coarse and fine adjustments in the X and Y directions takes into account both adjustment efficiency and positioning accuracy.

[0159] As Figure 7 shown, the schematic diagram of the feature point position in the coarse and fine adjustment processes in the X direction. In the figure, X i is the real-time value of ΔT X , T X represents the target deviation value of the adjustment in the X direction, ΔX i is the threshold for the X direction position adjustment, is also the threshold for the X direction position adjustment, and is still the critical point of the coarse and fine adjustment processes. The detailed processes of the coarse and fine adjustments are as follows:

[0160] The visual robot system determines whether X i is greater than If so, perform coarse adjustment and control the robot to adjust in the X direction to ensure that the distance between the robot and the target position after adjustment is within the range;

[0161] The visual robot system then determines whether X i is less than If so, perform fine adjustment. The fine adjustment area is Divide this area into 4 equal parts and continuously determine which area the current position is in. If it is within the last three area position intervals, control the robot to run in the X direction by a distance of to make it enter the previous area until it enters the range; If it is within the range, control the robot to run the current offset X i . In the feature point position approximation process, the threshold range setting must be greater than 4 times of ΔX i .

[0162] Similarly, according to the coarse and fine adjustment processes in the X direction, the coarse and fine adjustment processes in the Y direction can be known.

[0163] The feature image processing process

[0164] Specifically includes the following steps:

[0165] S11: The vision robot system extracts the edge points of the pattern in the image according to the Canny algorithm;

[0166] In the actual processing process, in the image captured by the two-dimensional camera, it not only contains the feature pattern but also the patterns of other scenes. Using the Canny algorithm, not only the edge points of the feature pattern are extracted, but also the edge points of the patterns of other scenes are extracted.

[0167] S12: The vision robot system generates a pattern contour according to the extracted edge points of the pattern, where the pattern contour is composed of several continuous edge points;

[0168] In the above steps, the vision robot generates not only the contour of the feature pattern but also the contours of the patterns of other scenes according to the edge points. In this embodiment, the number of pattern contours is at least two, one of which is the contour of the feature pattern, and the rest are the contours of the patterns of other scenes.

[0169] S13: The vision robot system calculates the number of edge points and the aspect ratio of the pattern contour according to the pattern contour, and judges whether the number of edge points of the pattern contour is within the safe number range and whether the aspect ratio of the pattern contour is within the safe aspect ratio range. If so, the corresponding pattern contour is retained, otherwise the corresponding pattern contour is removed.

[0170] In the above steps, in order to be able to distinguish the contour of the feature pattern in the pattern contour, in this embodiment, a safe number range and a safe aspect ratio range are pre-set in the vision robot system. The safe number range represents the possible value range of the number of edge points that make up the contour of the feature pattern. If the number of edge points of the pattern contour is outside the safe number range, then the corresponding pattern contour cannot be the contour of the feature pattern, so this pattern contour is removed; the safe aspect ratio range refers to the possible value range of the aspect ratio of the feature pattern. If the aspect ratio of the pattern contour is outside the safe aspect ratio range, then the corresponding pattern contour cannot be the contour of the feature pattern, so this pattern contour is removed. If the number of edge points of the pattern contour is within the safe number range and the aspect ratio of the pattern contour is within the safe aspect ratio range, then the corresponding pattern contour is very likely to be the contour of the feature pattern, so this pattern contour is retained.

[0171] S14: When the vision robot determines whether the difference between an edge point of the remaining pattern contour and an adjacent edge point lies within the gradient change threshold, if so, it determines that this edge point is a contour intersection point, and judges whether the contour intersection points of the pattern contour are within the range of the safe number of intersection points. If so, it reconstructs the contour of the feature pattern based on the contour intersection points of this pattern contour;

[0172] In the above steps, since the direction of the contour line segment changes near the contour intersection point, it is determined whether there is an intersection point of the contour line segment by judging the vector change trend of adjacent edge points. In this embodiment, a gradient change threshold is preset. The gradient change threshold represents the value range of the vector difference between adjacent edge points when they are on two adjacent contour line segments respectively. If the difference between an edge point of the remaining pattern contour and an adjacent edge point lies within the gradient change threshold, it proves that this edge point is a contour intersection point.

[0173] By the above method, the vision robot system can determine the number and position of the contour intersection points of the pattern contour. In order to further ensure that the remaining pattern contour is the contour of the feature pattern, the vision robot system also presets a range of the safe number of intersection points. The range of the safe number of intersection points is the possible value range of the contour intersection points of the feature pattern. In this embodiment, the range of the safe number of intersection points is 9. The contour intersection points of the feature pattern include the three vertices of an equilateral triangle, the four vertices of a rectangle, and the intersection points of the equilateral triangle and the rectangle. If the contour intersection points of the pattern contour are outside the range of the safe number of intersection points, the corresponding pattern contour is removed; if the contour intersection points of the pattern contour are within the range of the safe number of intersection points, the contour intersection points of the corresponding pattern contour are retained and the contour of the feature pattern is reconstructed based on the contour intersection points of this pattern contour;

[0174] S15: The vision robot system calculates the pixel geometric parameters of the feature pattern based on the contour of the feature pattern, converts the pixel geometric parameters of the feature pattern into physical geometric parameters of the feature pattern according to the transformation matrix, and judges whether the pixel geometric parameters of the feature pattern and the physical geometric parameters of the positioning template form a similar relationship. If so, it determines that the feature pattern and the positioning template are successfully matched and takes the contour of the feature pattern as the feature contour. If not, it determines that the feature pattern and the positioning template are not successfully matched and readjusts the shooting pose of the two-dimensional camera for the feature pattern and acquires an image.

[0175] In the above steps, the similarity relationship formed by the pixel geometric parameters of the feature pattern and the physical geometric parameters of the positioning template in the vision robot system means that when the similarity between the feature pattern and the positioning template is less than the similarity threshold, the shooting pose of the camera for the feature pattern is adjusted and a new photo is taken. In this embodiment, the similarity between the feature pattern and the positioning template is determined according to the ratio of different feature line segments of the feature pattern and different feature line segments of the positioning template. For example, the similarity between the feature pattern and the positioning template is equal to the ratio of A1 of the feature pattern to A1 of the positioning template minus the ratio of B1 of the feature pattern to B1 of the positioning template.

[0176] In the feature image processing process, the vision robot system matches the contour of the feature pattern in the image with multiple positioning templates to quickly and accurately match the two successfully, and then quickly and accurately calibrate the actuator.

[0177] As Figure 8 shown, it is a schematic diagram of the pose adaptation process, which is used to realize the pose adjustment of the actuator and the two-dimensional camera on the X-axis, Y-axis and Z-axis of the physical coordinate system, and ensure that the position of the feature points remains unchanged. The pose adaptation process specifically includes the following steps:

[0178] Step 221: The vision robot system triggers the two-dimensional camera to take a photo and obtains an image;

[0179] Step 222: The vision robot system performs a feature image processing process on the image to obtain the physical geometric parameters of the feature contour;

[0180] In the above steps, the physical geometric parameters of the feature contour include the physical values of A1, A2, B1 and B2.

[0181] Step 223: The vision robot system calculates the offset according to the physical geometric parameters of the feature contour and the physical geometric parameters of the positioning template. The offset includes θ Z1 、θ X1 and θ Y1 ,θ Z1 is the offset angle of the feature contour relative to the positioning template on the Z-axis of the robot physical coordinate system, θ X1 is the offset angle of the feature contour relative to the positioning template on the X-axis of the robot physical coordinate system, θ Y1 is the offset angle of the feature contour relative to the positioning template on the Y-axis of the robot physical coordinate system;

[0182] In the above steps, the offsets θ X1 and θ Y1 are calculated through the proportional relationship formula of A1, A2 and B1, B2. The above proportional relationship formula is as follows:

[0183]

[0184]

[0185] where f(θ Y1 ) is the offset θ Y1 , are respectively the ratios of A1 and A2 obtained at the nth time... the ratios of A1 and A2 obtained at the ith time, a n ...a i ...a0 are respectively the weight coefficients at the nth time... the ith time... the 0th time, f(θ X1 ) is the offset θ X1 , are respectively the ratios of A1 and A2 obtained at the nth time... the ratios of A1 and A2 obtained at the ith time, b n ...b i ...b0 are respectively the weight coefficients at the nth time... the ith time... the 0th time.

[0186] Step 224: The vision robot system determines whether θ Z1 is less than the threshold θ Z . If not, control the two-dimensional camera to rotate around the Z-axis of the physical coordinate system of the robot until it is less than the threshold θ Z ;

[0187] In the above steps, the vision robot system controls the two-dimensional camera to rotate around the Z-axis of the physical coordinate system of the robot until θ Z is less than the threshold θ Z1 using a one-step adjustment method.

[0188] Step 225: If θ Z1 is less than the threshold θ Z then the vision robot system determines whether θ X1 is less than the threshold θ X . If θ X1 is not less than the threshold θ X then determine whether θ X1 is greater than the threshold If θ X1 is greater than the threshold then control the two-dimensional camera to rotate around the X-axis of the physical coordinate system of the robot until θ X1 is not greater than the threshold If θ X1 is not greater than the threshold then control the two-dimensional camera to iteratively rotate around the X-axis of the physical coordinate system of the robot by an angle until θ X1 is less than If θ Z1 is less than then control the two-dimensional camera to rotate around the X-axis of the physical coordinate system of the robot by an angle of θ X1 , where the threshold θX Greater than 4 times the threshold

[0189] In the above steps, the visual robot system controls the movement of the two-dimensional camera on the X-axis of the robot's physical coordinate system in a coarse and fine adjustment manner.

[0190] Step 226: If θ X1 Less than the threshold θ X Then the visual robot system executes the feature point approximation process, and then determines whether θ Y1 Is less than the threshold θ Y , if θ Y1 Is not less than the threshold θ Y Then determine whether θ Y1 Is greater than the threshold If θ Y1 Is greater than the threshold Then control the two-dimensional camera to rotate around the Y-axis of the robot's physical coordinate system until θ Y1 Is not greater than the threshold If θ Y1 Is not greater than the threshold Then control the two-dimensional camera to iteratively rotate around the Y-axis of the robot's physical coordinate system By the angle until θ Y1 Is less than If θ Y1 Is less than Then control the two-dimensional camera to rotate around the Y-axis of the robot's physical coordinate system by the angle of θ Y1 , where the threshold θ Y Is greater than 4 times the threshold

[0191] In the above steps, the visual robot system controls the movement of the two-dimensional camera on the Y-axis of the robot's physical coordinate system in a coarse and fine adjustment manner.

[0192] Step 227: If θ Y1 Is less than the threshold θ Y Then the visual robot system executes the feature point position approximation process and records the current pose information of the robot;

[0193] In the above steps, the above θ Z1 、θ X1 、θ Y1 、ΔT x And ΔT Y After the adjustment is completed, the visual robot system can record the pose of the robot as the final pose when positioning and processing the workpiece.

[0194] As Figure 9 Shown, it is a schematic diagram of the height automatic adjustment process. The height automatic adjustment process includes the following steps:

[0195] Step 241: The vision robot system triggers the two-dimensional camera to take a picture and obtains an image;

[0196] Step 242: The vision robot system performs a feature image processing process on the image to obtain the physical geometric parameters of the feature contour;

[0197] In the above steps, the physical geometric parameters of the feature contour further include the height of the feature contour, and the height of the feature contour is the vertical distance between the camera and the feature pattern.

[0198] In the above steps, the height in the physical geometric parameters of the feature contour satisfies the following formula:

[0199] f(A1) = c n (A1) n +...+ c i (A1) i +...+ c0,

[0200] f(A2) = d n (A2) n +...+ d i (A2) i +...+ d0

[0201] f(B1) = e n (B1) n +...+ e i (B1) i +...+ e0

[0202] f(B2) = f n (B2) n +...+ f i (B2) i +...+ f0

[0203] In the formula, f(A1) is the height of the feature contour calculated based on A1, c n ...c i ...c0 are the weight coefficients of the nth... the ith... the 0th time respectively, (A1) n ...(A1) i are the physical values of A1 obtained at the nth time... the physical values of A1 obtained at the ith time respectively, f(A2) is the height of the feature contour calculated based on A2, d n ...d i ...d0 are the weight coefficients of the nth... the ith... the 0th time respectively, (A2) n ...(A2) iare respectively the physical value of A2 obtained at the nth time... the physical value of A2 obtained at the ith time, f(B1) is the characteristic contour height calculated according to B1, e n ...e i ...e0 are respectively the weight coefficients at the nth time... the ith time... the 0th time, (B1) n ...(B1) i are respectively the physical value of B1 obtained at the nth time... the physical value of B1 obtained at the ith time, f(B2) is the characteristic contour height calculated according to B2, f n ...f i ...f0 are respectively the weight coefficients at the nth time... the ith time... the 0th time, (B2) n ...(B2) i are respectively the physical value of B2 obtained at the nth time... the physical value of B2 obtained at the ith time. The height of the characteristic contour can be obtained according to the above formula.

[0204] Step 243: The vision robot system calculates the offset according to the physical geometric parameters of the characteristic contour and the physical geometric parameters of the positioning template. The offset includes ΔH, and ΔH is the height offset of the characteristic contour relative to the positioning template;

[0205] In the above steps, the physical geometric parameters of the positioning template include the height of the positioning template. The height of the positioning template is the vertical distance between the pre-calculated characteristic pattern and the camera, and the height of the positioning template can also be calculated using the formula in Step 242.

[0206] Step 244: The vision robot system determines whether ΔH is less than the threshold ΔH1. If ΔH is less than the threshold ΔH1, it then determines whether ΔH is greater than the threshold ΔH2. If ΔH is greater than the threshold ΔH2, it controls the two-dimensional camera to translate on the Z-axis of the robot's physical coordinate system until ΔH is not greater than the threshold ΔH2. If ΔH is not greater than the threshold ΔH2, it controls the two-dimensional camera to iteratively translate by a distance until ΔH is less than If ΔH is less than it controls the two-dimensional camera to translate by a distance of ΔH on the Z-axis of the robot's physical coordinate system, where the threshold ΔH2 is greater than 4 times the threshold ΔH1;

[0207] Step 245: The vision robot system determines whether the total height adjustment value is less than ΔH3. If so, it determines that the robot's height adjustment is excessive, resulting in an offset of the feature point position, and then executes the feature point position approximation process again to adjust the feature point position. The above total height adjustment value is equal to the difference between the final height of the two-dimensional camera and the initial height of the two-dimensional camera.

[0208] The above embodiments are only preferred embodiments of the present invention, and do not limit the protection scope of the present invention accordingly. Therefore, all equivalent changes made according to the structure, shape, and principle of the present invention shall be covered within the protection scope of the present invention.

Claims

1. A visual robot system, characterized in that, For performing the processes of establishing the workpiece coordinate system and the tool coordinate system, executing the template making process, the template making process, the feature point position approximation process, the feature image processing process, the attitude adaptation process, the N-point calibration process, and the height automatic adjustment process, the system includes: A robot, which includes an execution mechanism; A two-dimensional camera, which is arranged at the end of the execution mechanism; A workbench, on which the workpiece is positioned; A feature pattern, which includes feature points and feature line segments; The template making process includes the vision robot system adjusting the shooting pose of the two-dimensional camera for the feature pattern, triggering the two-dimensional camera to take a picture and obtaining an image, processing the image to obtain the pixel contour of the feature pattern, using the pixel contour of the feature pattern as the positioning template, and calculating the physical geometric parameters of the positioning template, where the feature pattern includes feature points and feature line segments; The feature point position approximation process includes the vision robot system setting the feature pattern on the workpiece, triggering the camera to take a picture and obtaining an image, performing the feature image processing process on the image to obtain the physical geometric parameters of the feature contour, and controlling the camera to translate on the X-axis and Y-axis of the robot physical coordinate system so that the physical geometric parameters of the feature contour approach the physical geometric parameters of the positioning template; The feature image processing process includes the vision robot system processing the image to obtain the contour of the feature pattern, judging whether the contour of the feature pattern matches the positioning template, if so, using the contour of the feature pattern as the feature contour and calculating the physical geometric parameters of the feature contour according to the feature contour, if not, readjusting the shooting pose of the two-dimensional camera for the feature pattern, triggering the two-dimensional camera to take a picture and obtaining an image, where the feature contour is the pixel contour of the feature pattern; The attitude adaptation process includes the vision robot system respectively controlling the two-dimensional camera to rotate around the X-axis, Y-axis, and Z-axis of the robot physical coordinate system so that the physical geometric parameters of the feature contour approach the physical geometric parameters of the positioning template; During the actual positioning process, the vision robot system starts, and the vision robot system judges whether it is necessary to execute the preprocessing process; If not, the vision robot system enters the judgment of whether it is necessary to execute the real-time processing process. If so, the vision robot system displays the interface of the preprocessing process on the touch screen. The vision robot system judges whether it is necessary to establish the workpiece coordinate system and the tool coordinate system. If so, it executes the process of establishing the workpiece coordinate system and the tool coordinate system. If not, it judges whether it is necessary to execute the N-point calibration process. If so, it executes the N-point calibration process. If not, it judges whether it is necessary to execute the template making process. If so, it executes the template making process and returns to the judgment of whether it is necessary to execute the preprocessing process. If not, it returns to the judgment of whether it is necessary to execute the preprocessing process; The vision robot system determines whether to execute the real-time processing process. If so, it sequentially executes the feature point position approximation process and the pose adaptation process. After the pose adaptation process is completed, the vision robot system determines the pose adjustment offset θ Z1 , θ X1 and θ Y1 whether they exceed the threshold. If so, it re-executes the feature point position approximation process and the pose adaptation process. If not, it executes the height automatic adjustment process, and then determines whether the total height adjustment value is less than the threshold ΔH3. If so, the vision robot system ends. If not, it executes the feature point approximation process and then ends; if it does not need to execute the real-time processing process, it determines whether to exit the system. If so, the vision robot system ends. If not, it returns to determine whether to execute the preprocessing process; The vision robot system calculates the offset, θ, based on the physical geometric parameters of the feature contour and the physical geometric parameters of the positioning template. Z1 θ is the offset angle of the feature contour relative to the positioning template on the Z-axis of the robot's physical coordinate system. X1 θ is the offset angle of the feature contour relative to the positioning template on the X-axis of the robot's physical coordinate system. Y1 θ is the offset angle of the feature contour relative to the positioning template on the Y-axis of the robot's physical coordinate system. The physical geometric parameters of the feature contour include the physical values of A1, A2, B1, and B2; Offset θ X1 and θ Y1 are calculated through the proportional relationship formulas of A1, A2 and B1, B2; the above proportional relationship formulas are as follows: ； ； where f(θ Y1 ) is the offset θ Y1 , are the ratios of A1 and A2 obtained at the nth time... the ratios of A1 and A2 obtained at the ith time, a n ... a i ... a0 are the weight coefficients at the nth time... the ith time... the 0th time respectively, and f(θ X1 ) is the offset θ X1 , are the ratios of A1 and A2 obtained at the nth time... the ratios of A1 and A2 obtained at the ith time, b n ... b i ... b0 are the weight coefficients at the nth time... the ith time... the 0th time respectively.

2. The visual robot system according to claim 1, characterized in that, The process of establishing the workpiece coordinate system and the tool coordinate system includes the vision robot system establishing the process of the workpiece coordinate system and the tool coordinate system, where the tool coordinate system includes the TCP position.

3. A vision robot system according to claim 1, characterized in that, The N-point calibration process includes the vision robot system establishing the conversion matrix between the image coordinate system and the robot physical coordinate system, where N is a positive integer.

4. A vision robot system according to claim 1, wherein The height automatic adjustment process includes the vision robot system controlling the two-dimensional camera to translate along the Z-axis of the robot physical coordinate system so that the physical geometric parameters of the feature contour approximate the physical geometric parameters of the positioning template.

Images