Calibration Method, Device and Computer Equipment for Machine Vision

Abstract

The present application provides a calibration method, device and computer equipment for machine vision. A gripper camera is deployed at the end of the robotic arm. During calibration, the system respectively constructs a robotic arm coordinate system, a gripper camera coordinate system and a fixed positioning camera coordinate system. During the movement process, the system takes the vision of the gripper camera as a reference, and through the calibration method, calibrates the first mapping relationship between the robotic arm coordinate system and the gripper camera coordinate system. Then, according to the first mapping relationship, the central position coordinates of the end of the robotic arm in the gripper camera coordinate system are determined. Finally, the system calculates the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system through the central position coordinates. In the present application, by using the machine vision of the gripper camera, the entire calibration process is automatically controlled and completed by the system, which not only avoids the calibration error caused by manual intervention, but also can greatly improve the calibration efficiency and realize the rapid installation and debugging of the machine vision recognition system.

Description

Technical Field

[0001] The present application relates to the technical field of vision calibration, and in particular to a machine vision calibration method, device and computer equipment. Background Art

[0002] A robotic arm is a device that automatically locates and grasps products through a machine vision recognition system. The machine vision recognition system needs to be calibrated before locating the product to achieve the conversion between the robotic arm coordinate system and the camera coordinate system and to confirm the coordinate values ​​of key points, thereby achieving accurate identification, positioning and grasping of the product.

[0003] The existing machine vision recognition system adopts manual calibration in the calibration process, that is, for a set of existing positions in the plane or space, the end of the robot arm is manually controlled to move to these points in turn, and the corresponding position points in the robot arm coordinate system are recorded. Then, the pixel positions of these points in the camera coordinate system are identified by positioning the camera to take pictures, so as to establish a mapping relationship between the pixel points of the positioning camera coordinate system and the position points of the robot arm coordinate system, and finally complete the calibration process. However, due to the unstable factors such as edge distortion that are common in positioning cameras, in order to improve the accuracy of the mapping relationship, the mapping samples of the position must not be less than a certain value, so it is necessary to manually control the robot arm to collect point samples multiple times. Not only is the calibration efficiency extremely low, but also human intervention will introduce uncontrollable errors. Different people operating the robot arm to a marked point will have more or less fixed-point errors, resulting in insufficient accuracy of the calibrated mapping relationship. Summary of the invention

[0004] In order to solve the above-mentioned problems, the present application provides a machine vision calibration method, device and computer equipment.

[0005] To achieve the above purpose, the present application provides a machine vision calibration method, wherein a gripper camera is deployed at the end of a robotic arm, and the method comprises:

[0006] A robot arm coordinate system, a gripper camera coordinate system and a fixed positioning camera coordinate system are respectively constructed, wherein the robot arm coordinate system represents a coordinate system constructed with a preset point of the mounting base of the robot arm as the origin, the gripper camera coordinate system represents a coordinate system constructed with a preset point of the gripper camera as the origin, and the fixed positioning camera coordinate system represents a coordinate system constructed with a preset point of the fixed positioning camera as the origin;

[0007] Using the vision of the gripper camera as a reference, calibrating a first mapping relationship between the robot arm coordinate system and the gripper camera coordinate system through a calibration method;

[0008] Determine the central position coordinates of the end center of the robotic arm in the coordinate system of the gripper camera according to the first mapping relationship;

[0009] Calculate the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system through the central position coordinates.

[0010] This application also provides a calibration device for machine vision. A gripper camera is deployed at the end of the robotic arm. The device includes:

[0011] A construction module for respectively constructing a robotic arm coordinate system, a gripper camera coordinate system, and a fixed positioning camera coordinate system. Among them, the robotic arm coordinate system represents a coordinate system constructed with a preset point of the mounting base of the robotic arm as the origin, the gripper camera coordinate system represents a coordinate system constructed with a preset point of the gripper camera as the origin, and the fixed positioning camera coordinate system represents a coordinate system constructed with a preset point of the fixed positioning camera as the origin;

[0012] A first calibration module for calibrating the first mapping relationship between the robotic arm coordinate system and the gripper camera coordinate system based on the vision of the gripper camera through a calibration method;

[0013] A second calibration module for determining the central position coordinates of the end center of the robotic arm in the gripper camera coordinate system according to the first mapping relationship;

[0014] A calculation module for calculating the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system through the central position coordinates.

[0015] This application also provides a computer device, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the steps of the method described in any one of the above are implemented.

[0016] This application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in any one of the above are implemented.

[0017] A calibration method, device, and computer device for machine vision provided in this application, compared with the prior art, not only have a fixed-positioning camera, but also a gripper camera is deployed at the end of the robotic arm. During the calibration process, the machine vision recognition system (hereinafter referred to as the system) respectively constructs a robotic arm coordinate system, a gripper camera coordinate system, and a fixed-positioning camera coordinate system; during the movement, the system takes the vision of the gripper camera as a reference, and through the calibration method, calibrates the first mapping relationship between the robotic arm coordinate system and the gripper camera coordinate system. Then, according to the first mapping relationship, the central position coordinates of the end of the robotic arm in the gripper camera coordinate system are determined. Finally, the system calculates the second mapping relationship between the robotic arm coordinate system and the fixed-positioning camera coordinate system through the central position coordinates, completing the calibration process. In this application, by using the machine vision of the gripper camera at the end of the robotic arm, there is no need for manual intervention when moving to the marked points, and the entire calibration process is automatically controlled and completed by the system, which not only avoids the calibration errors caused by manual intervention, but also can greatly improve the calibration efficiency and realize the rapid installation and debugging of the machine vision recognition system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a schematic diagram of the steps of the calibration method for machine vision in an embodiment of this application;

[0019] Figure 2 is a block diagram of the overall structure of the calibration device for machine vision in an embodiment of this application;

[0020] Figure 3 is a schematic block diagram of the structure of a computer device in an embodiment of this application.

[0021] The implementation, functional features, and advantages of the objectives of this application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] In order to make the objectives, technical solutions, and advantages of this application clearer, the following further details this application with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0023] Refer to Figure 1 , in an embodiment of this application, a calibration method for machine vision is provided, and a gripper camera is deployed at the end of the robotic arm. The method includes:

[0024] S1: Construct the robotic arm coordinate system, the gripper camera coordinate system, and the fixed positioning camera coordinate system respectively. Among them, the robotic arm coordinate system represents the coordinate system constructed with a preset point on the mounting base of the robotic arm as the origin, the gripper camera coordinate system represents the coordinate system constructed with a preset point on the gripper camera as the origin, and the fixed positioning camera coordinate system represents the coordinate system constructed with a preset point on the fixed positioning camera as the origin;

[0025] S2: Based on the vision of the gripper camera, calibrate the first mapping relationship between the robotic arm coordinate system and the gripper camera coordinate system through a calibration method;

[0026] S3: According to the first mapping relationship, determine the central position coordinates of the end center of the robotic arm in the gripper camera coordinate system;

[0027] S4: Calculate the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system through the central position coordinates.

[0028] In this embodiment, compared with the prior art, the machine vision recognition system (hereinafter referred to as the system) not only has a fixed-position camera (the fixed-position camera is fixedly installed at a preset position), but also includes a gripper camera, and the gripper camera is deployed at the end of the robotic arm. The system respectively constructs a robotic arm coordinate system with a preset point on the mounting base of the robotic arm (preferably the center point of the mounting base) as the origin, constructs a gripper camera coordinate system with a preset point on the gripper camera (preferably the end point at the upper left corner of the gripper camera) as the origin, and establishes a fixed-position camera coordinate system with a preset point on the fixed-position camera (preferably the end point at the upper left corner of the fixed-position camera) as the origin. During the process of moving the robotic arm, the system uses the image collected by the gripper camera installed at the end of the robotic arm as the recognition reference (i.e., based on the vision of the gripper camera), and controls the end of the robotic arm to move above the teaching board. When the image collected by the gripper camera contains the teaching board (multiple locatable marking points are printed on the teaching board), the system determines that the end of the robotic arm is already above the teaching board, and records the first pixel coordinates of the first marking point of the teaching board in the gripper camera coordinate system. Then, it controls the end of the robotic arm to move a preset distance along the X-axis direction of the robotic arm coordinate system, and records the second pixel coordinates of the current first marking point in the gripper camera coordinate system. The system calculates the first mapping relationship between the robotic arm coordinate system and the gripper camera coordinate system through affine transformation according to the first pixel coordinates, the second pixel coordinates corresponding to the same marking point (i.e., the first marking point) collected before and after the movement of the robotic arm, and the preset distance moved, which is used to represent the second mapping relationship between the two coordinate systems. The system obtains the third pixel coordinates of the image center point of the gripper camera in the gripper camera coordinate system, and a certain marking point on the teaching board, that is, the second marking point (the second marking point can be any marking point on the teaching board, as long as it can be captured by the gripper camera, that is, the second marking point needs to appear in the image captured by the gripper camera so as to obtain the fourth pixel coordinates of the second marking point; preferably, in order to reduce the subsequent moving distance and improve the calibration speed, the second marking point is preferably the one with the shortest distance from the image center point in the image captured this time) in the gripper camera coordinate system. The system controls the end of the robotic arm to move to align the image center point of the gripper camera with the second marking point according to the first mapping relationship between the robotic arm coordinate system and the gripper camera coordinate system and the fourth pixel coordinates of the current second marking point. Then, it controls the rotation axis at the end of the robotic arm to drive the gripper camera to rotate 180 degrees, and records the fifth pixel coordinates of the current second marking point in the gripper camera coordinate system after the rotation. The system calculates the sixth pixel coordinates of the midpoint of the straight line between the second marking point and the image center point according to the fifth pixel coordinates corresponding to the second marking point respectively after the movement and rotation of the gripper camera, and the third pixel coordinates corresponding to the image center point of the gripper camera.The midpoint of this straight line is the rotation center of the gripper camera image when the rotation axis at the end of the robotic arm rotates. At this time, the center of the rotation axis at the end of the robotic arm is coaxial with the rotation center of the gripper camera image, that is, the center of the end of the robotic arm is coaxial with the rotation center of the gripper camera image. Therefore, the system directly uses the sixth pixel coordinate corresponding to the rotation center of the gripper camera image as the center position coordinate of the end of the robotic arm in the gripper camera coordinate system. After determining the center position coordinate of the end of the robotic arm in the gripper camera coordinate system, based on this center position coordinate, the system controls the end of the robotic arm to align with multiple third marking points on the teaching board, and sequentially records the first position coordinates corresponding to each third marking point in the robotic arm coordinate system, and then controls the robotic arm to move to the waiting position. Then, the system uses a fixed positioning camera to collect an image of the teaching board, and obtains the seventh pixel coordinates corresponding to each third marking point in the fixed positioning camera coordinate system. The system calculates the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system based on the second mapping relationship between each first position coordinate and each seventh pixel coordinate with the same third marking point as the corresponding reference, and completes the calibration process of the robotic arm and the fixed positioning camera.

[0029] In this embodiment, the system uses the machine vision of the gripper camera at the end of the robotic arm, and no manual intervention is required when moving to the marking point position for mapping sample collection; and during the calibration calculation process, the gripper camera coordinate system corresponding to the gripper camera is used as an intermediate quantity for conversion mapping, so that the entire calibration process is automatically controlled and completed by the system, which not only avoids the calibration error caused by manual intervention, but also can greatly improve the calibration efficiency and realize the rapid installation and debugging of the machine vision recognition system.

[0030] Further, the step of calibrating the first mapping relationship between the robotic arm coordinate system and the gripper camera coordinate system based on the vision of the gripper camera by a calibration method includes:

[0031] S201: Based on the vision of the gripper camera, control the end of the robotic arm to move above the teaching board, and record the first pixel coordinate of the first marking point on the teaching board in the gripper camera coordinate system;

[0032] S202: Control the end of the robotic arm to move a preset distance along the X-axis direction of the robotic arm coordinate system, and record the second pixel coordinate of the current first marking point in the gripper camera coordinate system;

[0033] S203: Calculate the first mapping relationship through affine transformation based on the first pixel coordinate, the second pixel coordinate, and the preset distance.

[0034] In this embodiment, after the calibration process starts, the system controls the end of the robotic arm to move above the teaching board. Specifically, the robotic arm first moves in a preset direction. At the same time, the gripper camera collects images directly below the end of the robotic arm at a preset frequency and feeds the collected images back to the system. The system determines whether the end of the robotic arm has moved above the teaching board based on whether the images captured by the gripper camera contain the teaching board (since the gripper camera is installed at the end of the robotic arm, the viewing angle of the gripper camera is actually equivalent to the viewing angle of the end of the robotic arm). After determining that the end of the robotic arm has moved above the teaching board, the system randomly selects a marked point on the teaching board as the first marked point and determines the first pixel coordinates of the first marked point in the gripper camera coordinate system based on the position of the first marked point in the image collected by the gripper camera. At the same time, the system records the second position coordinates of the end of the robotic arm at this time. The system uses the second position coordinates of the end of the robotic arm as the movement reference and controls the end of the robotic arm to move a preset distance along the X-axis direction of the robotic arm coordinate system (the end of the robotic arm can move along the positive axis or the negative axis of the X-axis of the robotic arm coordinate system, which is specifically set by the calibration personnel and is not specifically limited here, but there is no change in direction during the movement, that is, the current movement always moves in one direction and does not reverse midway). After the movement is completed, the system obtains the second pixel coordinates of the first marked point in the gripper camera coordinate system at the current moment through the image collected by the gripper camera. The system calculates the angle parameter between the robotic arm coordinate system and the gripper camera coordinate system based on the first pixel coordinates and the second pixel coordinates corresponding to the same marked point (i.e., the first marked point) collected before and after the movement of the robotic arm; and calculates the scaling ratio parameter between the robotic arm coordinate system and the gripper camera coordinate system based on the first pixel coordinates, the second pixel coordinates, and the preset distance. The angle parameter and the scaling ratio parameter form the first mapping relationship, which represents the second mapping relationship between the two coordinate systems.

[0035] Further, the first mapping relationship includes an angle parameter and a scaling ratio parameter. The step of calculating the first mapping relationship through affine transformation according to the first pixel coordinates, the second pixel coordinates, and the preset distance includes:

[0036] S2031: Substitute the first pixel coordinates and the second pixel coordinates into the first calculation formula to calculate the angle parameter. The first calculation formula is: angle_base = atan2((C2.X - C1.X), (C2.Y - C1.Y)), where angle_base is the angle parameter, C1.X and C1.Y are the abscissa and ordinate of the first pixel coordinates in sequence, and C2.X and C2.Y are the abscissa and ordinate of the second pixel coordinates in sequence;

[0037] S2032: Substitute the first pixel coordinate and the second pixel coordinate into the second calculation formula to calculate the square of the pixel length. The second calculation formula is: visionLen_power = (C2.X - C1.X) 2 +(C2.Y - C1.Y) 2 , where visionLen_power is the square of the pixel length;

[0038] S2033: Substitute the square of the pixel length and the preset distance into the third calculation formula to calculate the scaling ratio parameter. The third calculation formula is: scale = x_offset / sqrt(visionLen_power), where scale is the scaling ratio parameter and x_offset is the preset distance.

[0039] In this embodiment, the system first substitutes the first pixel coordinate and the second pixel coordinate into the first calculation formula angle_base = atan2((C2.X - C1.X), (C2.Y - C1.Y)) to calculate the included angle parameter between the robotic arm coordinate system and the gripper camera coordinate system. This included angle parameter represents the second angle mapping relationship between the two coordinate axes. In the first calculation formula, angle_base is the included angle parameter, C1.X and C1.Y are the abscissa and ordinate of the first pixel coordinate respectively, and C2.X and C2.Y are the abscissa and ordinate of the second pixel coordinate respectively; the meaning of atan2(y, x) is the angle between the ray starting from the coordinate origin and pointing to (x, y) and the positive x-axis direction in the coordinate plane. The system then substitutes the first pixel coordinate and the second pixel coordinate into the second calculation formula visionLen_power = (C2.X - C1.X) × (C2.X - C1.X) + (C2.Y - C1.Y) × (C2.Y - C1.Y) to calculate the square of the pixel length; where visionLen_power is the square of the pixel length. Finally, the system substitutes the obtained square of the pixel length and the preset distance into the third calculation formula scale = x_offset / sqrt(visionLen_power) to calculate the scaling ratio parameter. This scaling ratio parameter represents the second ratio mapping relationship between the robotic arm coordinate system and the gripper camera coordinate system; sqrt represents the square root of a non-negative real number. Thus, after combining the calculated included angle parameter and scaling ratio parameter, the system represents the second mapping relationship between the two coordinate systems. With the help of the included angle parameter and the scaling ratio parameter, coordinate conversion between the two coordinate systems can be achieved.

[0040] Further, the step of determining the central position coordinate of the end center of the robotic arm in the gripper camera coordinate system according to the first mapping relationship includes:

[0041] S301: Obtain the third pixel coordinate of the image center point of the gripper camera and the fourth pixel coordinate of the second marking point on the teaching board;

[0042] S302: Using the first mapping relationship and the fourth pixel coordinate as the corresponding reference, control the end of the robotic arm to move to align the image center point of the gripper camera with the second marking point;

[0043] S303: Control the end of the robotic arm to drive the gripper camera to rotate 180 degrees, and record the fifth pixel coordinate of the second marking point in the gripper camera coordinate system;

[0044] S304: Calculate the sixth pixel coordinate of the midpoint of the straight line between the second marking point and the image center point based on the third pixel coordinate and the fifth pixel coordinate, and use the sixth pixel coordinate as the center position coordinate.

[0045] In this embodiment, the system obtains the picture size captured by the gripper camera, and then determines the third pixel coordinate of the image center point of the gripper camera in the gripper camera coordinate system according to the picture size. Among them, assuming the picture size is w*h, the third pixel coordinate of the image center point is Q(w / 2, h / 2). And, the system selects any marking point on the teaching board as the second marking point, and obtains the fourth pixel coordinate of the second marking point in the gripper camera coordinate system through the image captured by the gripper camera. Then, using the first mapping relationship representing the relationship between the gripper camera coordinate system and the robotic arm coordinate system and the fourth pixel coordinate as the corresponding reference, obtain the position coordinate of the end of the robotic arm in the robotic arm coordinate system when the image center point of the gripper camera is aligned with the second marking point through coordinate transformation, and control the end of the robotic arm to perform corresponding movements according to this position coordinate, so that the image center point of the gripper camera is aligned with the second marking point. Then, the system controls the rotation axis of the end of the robotic arm to rotate 180 degrees, thereby driving the gripper camera at the end to rotate 180 degrees as well. The system records the fifth pixel coordinate of the second marking point in the gripper camera coordinate system in the image captured by the rotated gripper camera at this time. At this time, connect the second marking point and the image center point, and the midpoint of the straight line between the two is the rotation center of the camera image of the gripper camera when the rotation axis of the end of the robotic arm rotates, and at this time, the center of the end of the robotic arm is collinear with this rotation center. Specifically, the system calculates the sixth pixel coordinate corresponding to the midpoint of the straight line between the second marking point obtained after rotation and the image center point based on the third pixel coordinate and the fifth pixel coordinate, and uses this sixth pixel coordinate as the center position coordinate of the end of the robotic arm in the gripper camera coordinate system.

[0046] Further, the step of controlling the end of the robotic arm to move to the image center point of the gripper camera to align with the second marking point based on the first mapping relationship and the fourth pixel coordinates includes:

[0047] S3021: Substitute the third pixel coordinates and the fourth pixel coordinates into the fourth calculation formula to calculate the offset angle. The fourth calculation formula is: cur_angle = atan2((C0.X - Q.X), (C0.Y - Q.Y)), where cur_angle is the offset angle, Q.X and Q.Y are the abscissa and ordinate of the third pixel coordinates respectively, and C0.x and C0.y are the abscissa and ordinate of the fourth pixel coordinates respectively;

[0048] S3022: Substitute the third pixel coordinates and the fourth pixel coordinates into the fifth calculation formula to calculate the moving distance. The fifth calculation formula is: distance = sqrt((C0.X - Q.X) * (C0.X - Q.X) + (C0.Y - Q.Y) * (C0.Y - Q.Y)), where distance is the moving distance;

[0049] S3023: Substitute the offset angle, the moving distance, and the first mapping relationship into the sixth calculation formula and the seventh calculation formula respectively, and sequentially calculate the moving distances of the end of the robotic arm in the X-axis and Y-axis directions of the robotic arm coordinate system. The sixth calculation formula is: offset_x = -cos(cur_angle - angle_base) * distance * scale, and the seventh calculation formula is: offset_y = sin(cur_angle - angle_base) * distance * scale, where offset_x is the moving distance of the end of the robotic arm in the X-axis direction of the robotic arm coordinate system, offset_y is the moving distance of the end of the robotic arm in the Y-axis direction of the robotic arm coordinate system, angle_base is the included angle parameter in the first mapping relationship, and scale is the scaling ratio parameter in the first mapping relationship;

[0050] S3024: Control the end of the robotic arm to move respectively according to the moving distances in the X-axis and Y-axis directions of the robotic arm coordinate system to achieve the alignment of the image center point of the gripper camera with the second marking point.

[0051] In this embodiment, it is assumed that the third pixel coordinates are (Q.X, Q.Y), and the fourth pixel coordinates are C0.x, C0.y. The system substitutes the third pixel coordinates and the fourth pixel coordinates into the fourth calculation formula to calculate the offset angle. This offset angle represents the offset angle between the front and rear positions of the end of the robotic arm when the end of the robotic arm moves from the current position to the image center point of the gripper camera to align with the second marking point. Among them, the fourth calculation formula is: cur_angle = atan2((C0.X - Q.X), (C0.Y - Q.Y)), where cur_angle is the offset angle. The system then substitutes the third pixel coordinates and the fourth pixel coordinates into the fifth calculation formula distance = sqrt((C0.X - Q.X) * (C0.X - Q.X) + (C0.Y - Q.Y) * (C0.Y - Q.Y)) to calculate the offset distance. This offset distance represents the straight-line distance between the front and rear positions of the end of the robotic arm when the end of the robotic arm moves from the current position to the image center point of the gripper camera to align with the second marking point; among them, distance is the moving distance. Finally, the system substitutes the offset angle, the moving distance, and the first mapping relationship into the sixth calculation formula and the seventh calculation formula respectively, and calculates in sequence the moving distances of the end of the robotic arm in the X-axis and Y-axis directions of the robotic arm coordinate system. Among them, the sixth calculation formula is: offset_x = -cos(cur_angle - angle_base) * distance * scale, and the seventh calculation formula is: offset_y = sin(cur_angle - angle_base) * distance * scale; offset_x is the moving distance of the end of the robotic arm in the X-axis direction of the robotic arm coordinate system, offset_y is the moving distance of the end of the robotic arm in the Y-axis direction of the robotic arm coordinate system, angle_base is the included angle parameter in the first mapping relationship, and scale is the scaling ratio parameter in the first mapping relationship. The system controls the end of the robotic arm to perform corresponding movements according to the calculated moving distances in the X-axis direction and the Y-axis direction of the robotic arm coordinate system. After completing the movement, the image center point of the gripper camera can be aligned with the second marking point.

[0052] Further, after the end of the robotic arm moves to the image center point of the gripper camera to align with the second marking point and rotates 180 degrees, the fifth pixel coordinates corresponding to the second marking point are recorded as C3(x1, y1). According to the third pixel coordinates and the fifth pixel coordinates of the image center point, the calculated central position coordinates (i.e., the sixth pixel coordinates) are O((w / 2 + x1) / 2, (h / 2 + y1) / 2).

[0053] Further, the step of calculating the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system through the central position coordinates includes:

[0054] S401: Taking the central position coordinates as a reference, control the end center of the robotic arm to align with multiple third marking points on the teaching board, and sequentially record the first position coordinates corresponding to each of the third marking points in the robotic arm coordinate system;

[0055] S402: Use the fixed positioning camera to collect an image of the teaching board to obtain the seventh pixel coordinates corresponding to each of the third marking points in the fixed positioning camera coordinate system;

[0056] S403: Taking the same third marking point as the corresponding reference, calculate the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system respectively according to the mapping relationship between each of the first position coordinates and each of the seventh pixel coordinates.

[0057] In this embodiment, the system takes the central position coordinates of the end of the robotic arm as a reference, controls the end center of the robotic arm to align with multiple third marking points on the teaching board (since the end center of the robotic arm and the rotation center of the camera image of the gripper camera are actually to move the end of the robotic arm so that the rotation center of the camera image aligns with each third marking point), and sequentially records the first position coordinates corresponding to each of the third marking points in the robotic arm coordinate system according to the order of movement. These first position coordinates represent the position coordinates of the end of the robotic arm in the robotic arm coordinate system when the end of the robotic arm moves to these third marking points. After sampling multiple first position coordinates, the system controls the robotic arm to move to the waiting position and uses the fixed positioning camera to collect an image of the teaching board. According to the position distribution of each third marking point on the teaching board in the collected image, the seventh pixel coordinates corresponding to each of the third marking points in the fixed positioning camera coordinate system are obtained. The system takes the same third marking point as the corresponding relationship, and calculates the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system respectively according to the mapping relationship between the first position coordinates and the seventh pixel coordinates of the same third marking point (the calculation of this part of the second mapping relationship is the same as the calculation logic in the existing calibration process and will not be elaborated here), completing the calibration of the entire machine vision.

[0058] Further, a dispensing tool is installed at the end of the robotic arm. After the step of calculating the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system through the central position coordinates, it includes:

[0059] S5: Use the fixed positioning camera to collect the dispensing pixel coordinates of the dispensing position of the workpiece in the fixed positioning camera coordinate system;

[0060] S6: Convert the dispensing pixel coordinates into dispensing position coordinates on the robotic arm coordinate system according to the second mapping relationship;

[0061] S7: Control the end of the robotic arm to move to the dispensing position of the workpiece according to the dispensing position coordinates, and use the dispensing tool to perform the dispensing operation.

[0062] In this embodiment, the machine vision recognition system after calibration can be applied to the dispensing process. Specifically, a dispensing tool is installed at the end of the robotic arm. After the workpiece to be dispensed is transported to the station where the robotic arm is located, the system captures the workpiece image through a fixed positioning camera, and identifies the dispensing position and the dispensing pixel coordinates of each dispensing position in the fixed positioning camera coordinate system from the workpiece image. Then, according to the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system, convert each dispensing pixel coordinate into the dispensing position coordinate on the robotic arm coordinate system. The system uses each dispensing position coordinate as the movement reference to control the end of the robotic arm to accurately move to each dispensing position on the workpiece, and uses the dispensing tool to perform the dispensing operation at the dispensing position, completing the dispensing process.

[0063] Refer to Figure 2 , in an embodiment of the present application, a calibration device for machine vision is further provided. A gripper camera is deployed at the end of the robotic arm. The device includes:

[0064] A construction module 1 for respectively constructing a robotic arm coordinate system, a gripper camera coordinate system, and a fixed positioning camera coordinate system. Among them, the robotic arm coordinate system represents a coordinate system constructed with a preset point of the mounting base of the robotic arm as the origin, the gripper camera coordinate system represents a coordinate system constructed with a preset point of the gripper camera as the origin, and the fixed positioning camera coordinate system represents a coordinate system constructed with a preset point of the fixed positioning camera as the origin;

[0065] A first calibration module 2 for calibrating the first mapping relationship between the robotic arm coordinate system and the gripper camera coordinate system by a calibration method with the vision of the gripper camera as the reference;

[0066] A second calibration module 3 for determining the central position coordinates of the end of the robotic arm in the gripper camera coordinate system according to the first mapping relationship;

[0067] A calculation module 4 for calculating the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system through the central position coordinates.

[0068] Further, the first calibration module 2 includes:

[0069] The first moving unit is used to control the end of the robotic arm to move above the teaching board based on the vision of the gripper camera, and record the first pixel coordinates of the first marking point of the teaching board in the gripper camera coordinate system;

[0070] The second moving unit is used to control the end of the robotic arm to move a preset distance along the X-axis direction of the robotic arm coordinate system, and record the second pixel coordinates of the current first marking point in the gripper camera coordinate system;

[0071] The first calculation unit is used to calculate the first mapping relationship through affine transformation according to the first pixel coordinates, the second pixel coordinates and the preset distance.

[0072] Further, the first mapping relationship includes an angle parameter and a scaling ratio parameter. The first calculation unit includes:

[0073] The first calculation subunit is used to substitute the first pixel coordinates and the second pixel coordinates into the first calculation formula to calculate the angle parameter. The first calculation formula is: angle_base = atan2((C2.X - C1.X), (C2.Y - C1.Y)), where angle_base is the angle parameter, C1.X and C1.Y are the abscissa and ordinate of the first pixel coordinates respectively, and C2.X and C2.Y are the abscissa and ordinate of the second pixel coordinates respectively;

[0074] The second calculation subunit is used to substitute the first pixel coordinates and the second pixel coordinates into the second calculation formula to calculate the square of the pixel length. The second calculation formula is: visionLen_power = (C2.X - C1.X) × (C2.X - C1.X) + (C2.Y - C1.Y) × (C2.Y - C1.Y), where visionLen_power is the square of the pixel length;

[0075] The third calculation subunit is used to substitute the square of the pixel length and the preset distance into the third calculation formula to calculate the scaling ratio parameter. The third calculation formula is: scale = x_offset / sqrt(visionLen_power), where scale is the scaling ratio parameter and x_offset is the preset distance.

[0076] Further, the second calibration module 3 includes:

[0077] The first acquisition unit is used to acquire the third pixel coordinates of the image center point of the gripper camera and the fourth pixel coordinates of the second marking point of the teaching board;

[0078] A third moving unit, configured to control the end of the robotic arm to move to the image center point of the gripper camera to align with the second marking point based on the first mapping relationship and the fourth pixel coordinates;

[0079] A rotating unit, configured to control the end of the robotic arm to drive the gripper camera to rotate 180 degrees and record the fifth pixel coordinates of the second marking point in the gripper camera coordinate system;

[0080] A second calculation unit, configured to calculate the sixth pixel coordinates of the midpoint of the straight line between the second marking point and the image center point according to the third pixel coordinates and the fifth pixel coordinates, and use the sixth pixel coordinates as the central position coordinates.

[0081] Further, the third moving unit includes:

[0082] A fourth calculation sub-unit, configured to substitute the third pixel coordinates and the fourth pixel coordinates into a fourth calculation formula to calculate an offset angle, and the fourth calculation formula is: cur_angle = atan2((C0.X - Q.X), (C0.Y - Q.Y)), where cur_angle is the offset angle, Q.X and Q.Y are the abscissa and ordinate of the third pixel coordinates in sequence, and C0.x and C0.y are the abscissa and ordinate of the fourth pixel coordinates in sequence;

[0083] A fifth calculation sub-unit, configured to substitute the third pixel coordinates and the fourth pixel coordinates into a fifth calculation formula to calculate a moving distance, and the fifth calculation formula is: distance = sqrt((C0.X - Q.X)*(C0.X - Q.X)+(C0.Y - Q.Y)*(C0.Y - Q.Y)), where distance is the moving distance;

[0084] The sixth calculation subunit is configured to substitute the offset angle, the moving distance, and the first mapping relationship into a sixth calculation formula and a seventh calculation formula respectively, and sequentially calculate the moving distances of the end of the robotic arm in the X-axis and Y-axis directions of the robotic arm coordinate system. The sixth calculation formula is: offset_x = -cos(cur_angle - angle_base) * distance * scale, and the seventh calculation formula is: offset_y = sin(cur_angle - angle_base) * distance * scale, where offset_x is the moving distance of the end of the robotic arm in the X-axis direction of the robotic arm coordinate system, offset_y is the moving distance of the end of the robotic arm in the Y-axis direction of the robotic arm coordinate system, angle_base is the included angle parameter in the first mapping relationship, and scale is the scaling ratio parameter in the first mapping relationship;

[0085] The moving subunit is configured to control the end of the robotic arm to move respectively according to the moving distances in the X-axis and Y-axis directions of the robotic arm coordinate system, so as to align the image center point of the gripper camera with the second marking point.

[0086] Further, the calculation module 4 includes:

[0087] The fourth moving unit is configured to use the central position coordinates as a reference to control the center of the end of the robotic arm to align with a plurality of third marking points on the teaching board, and sequentially record the first position coordinates corresponding to each of the third marking points in the robotic arm coordinate system;

[0088] The second acquisition unit is configured to use a fixed positioning camera to acquire an image of the teaching board, and obtain the seventh pixel coordinates corresponding to each of the third marking points in the fixed positioning camera coordinate system;

[0089] The third calculation unit is configured to use the same third marking point as a corresponding reference, and respectively calculate a second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system according to the mapping relationship between each of the first position coordinates and each of the seventh pixel coordinates;

[0090] Further, a dispensing tool is installed at the end of the robotic arm, and the calibration device further includes:

[0091] The acquisition module 5 is configured to acquire the dispensing pixel coordinates of the dispensing position of the workpiece in the fixed positioning camera coordinate system through a fixed positioning camera;

[0092] The conversion module 6 is configured to convert the dispensing pixel coordinates into the dispensing position coordinates on the robotic arm coordinate system according to the second mapping relationship;

[0093] The dispensing module 7 is used to control the end of the robotic arm to move to the dispensing position of the workpiece according to the dispensing position coordinates, and perform a dispensing operation using the dispensing tool.

[0094] In this embodiment, each module, unit, and subunit in the calibration device is used to correspondingly execute each step in the above-mentioned calibration method of machine vision, and its specific implementation process will not be elaborated here.

[0095] A calibration device for machine vision provided in this embodiment, compared with the prior art, not only has a fixed positioning camera, but also has a gripper camera deployed at the end of the robotic arm. During the calibration process, the machine vision recognition system respectively constructs a robotic arm coordinate system, a gripper camera coordinate system, and a fixed positioning camera coordinate system; during the movement process, the system takes the vision of the gripper camera as the benchmark, and calibrates the first mapping relationship between the robotic arm coordinate system and the gripper camera coordinate system through a calibration method. Then, according to the first mapping relationship, the central position coordinates of the end of the robotic arm in the gripper camera coordinate system are determined. Finally, the system calculates the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system through the central position coordinates, completing the calibration process. In this application, the system uses the machine vision of the gripper camera at the end of the robotic arm, and there is no need for manual intervention when moving to the marked point positions. The entire calibration process is automatically controlled and completed by the system, which not only avoids the calibration errors caused by manual intervention, but also can greatly improve the calibration efficiency and realize the rapid installation and debugging of the machine vision recognition system.

[0096] Refer to Figure 3 , a computer device is also provided in the embodiment of the present application. This computer device can be a server, and its internal structure can be as Figure 3 shown. This computer device includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of this computer design is used to provide computing and control capabilities. The memory of this computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of this computer device is used to store data such as the first calculation formula. The network interface of this computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it realizes a calibration method for machine vision, and a gripper camera is deployed at the end of the robotic arm.

[0097] The above processor executes the steps of the above-mentioned calibration method for machine vision:

[0098] S1: Respectively construct a robotic arm coordinate system, a gripper camera coordinate system, and a fixed positioning camera coordinate system. Among them, the robotic arm coordinate system represents a coordinate system constructed with a preset point on the mounting base of the robotic arm as the origin, the gripper camera coordinate system represents a coordinate system constructed with a preset point on the gripper camera as the origin, and the fixed positioning camera coordinate system represents a coordinate system constructed with a preset point on the fixed positioning camera as the origin;

[0099] S2: Based on the vision of the gripper camera, calibrate the first mapping relationship between the robotic arm coordinate system and the gripper camera coordinate system through a calibration method;

[0100] S3: According to the first mapping relationship, determine the central position coordinates of the end of the robotic arm in the gripper camera coordinate system;

[0101] S4: Calculate the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system through the central position coordinates.

[0102] Further, the step of calibrating the first mapping relationship between the robotic arm coordinate system and the gripper camera coordinate system based on the vision of the gripper camera through a calibration method includes:

[0103] S201: Based on the vision of the gripper camera, control the end of the robotic arm to move above the teaching board, and record the first pixel coordinates of the first marking point on the teaching board in the gripper camera coordinate system;

[0104] S202: Control the end of the robotic arm to move a preset distance along the X-axis direction of the robotic arm coordinate system, and record the second pixel coordinates of the current first marking point in the gripper camera coordinate system;

[0105] S203: Calculate the first mapping relationship through affine transformation according to the first pixel coordinates, the second pixel coordinates, and the preset distance.

[0106] Further, the first mapping relationship includes an included angle parameter and a scaling ratio parameter. The step of calculating the first mapping relationship through affine transformation according to the first pixel coordinates, the second pixel coordinates, and the preset distance includes:

[0107] S2031: Substitute the first pixel coordinates and the second pixel coordinates into the first calculation formula to calculate the included angle parameter. The first calculation formula is: angle_base = atan2((C2.X - C1.X), (C2.Y - C1.Y)), where angle_base is the included angle parameter, C1.X and C1.Y are the abscissa and ordinate of the first pixel coordinates respectively, and C2.X and C2.Y are the abscissa and ordinate of the second pixel coordinates respectively;

[0108] S2032: Substitute the first pixel coordinates and the second pixel coordinates into the second calculation formula to calculate the square of the pixel length. The second calculation formula is: visionLen_power = (C2.X - C1.X) × (C2.X - C1.X) + (C2.Y - C1.Y) × (C2.Y - C1.Y), where visionLen_power is the square of the pixel length;

[0109] S2033: Substitute the square of the pixel length and the preset distance into the third calculation formula to calculate the scaling ratio parameter. The third calculation formula is: scale = x_offset / sqrt(visionLen_power), where scale is the scaling ratio parameter and x_offset is the preset distance.

[0110] Further, the step of determining the central position coordinates of the end of the robotic arm in the hand - held camera coordinate system according to the first mapping relationship includes:

[0111] S301: Obtain the third pixel coordinates of the image center point of the hand - held camera and the fourth pixel coordinates of the second marking point on the teaching board;

[0112] S302: Taking the first mapping relationship and the fourth pixel coordinates as the corresponding reference, control the end of the robotic arm to move to align the image center point of the hand - held camera with the second marking point;

[0113] S303: Control the end of the robotic arm to drive the hand - held camera to rotate 180 degrees, and record the fifth pixel coordinates of the second marking point in the hand - held camera coordinate system;

[0114] S304: Calculate the sixth pixel coordinates of the mid - point of the straight line between the second marking point and the image center point according to the third pixel coordinates and the fifth pixel coordinates, and use the sixth pixel coordinates as the central position coordinates.

[0115] Further, the step of controlling the end of the robotic arm to move to the image center point of the gripper camera to align with the second marking point based on the first mapping relationship and the fourth pixel coordinates includes:

[0116] S3021: Substitute the third pixel coordinates and the fourth pixel coordinates into the fourth calculation formula to calculate the offset angle. The fourth calculation formula is: cur_angle = atan2((C0.X - Q.X), (C0.Y - Q.Y)), where cur_angle is the offset angle, Q.X and Q.Y are the abscissa and ordinate of the third pixel coordinates respectively, and C0.x and C0.y are the abscissa and ordinate of the fourth pixel coordinates respectively;

[0117] S3022: Substitute the third pixel coordinates and the fourth pixel coordinates into the fifth calculation formula to calculate the moving distance. The fifth calculation formula is: distance = sqrt((C0.X - Q.X) * (C0.X - Q.X) + (C0.Y - Q.Y) * (C0.Y - Q.Y)), where distance is the moving distance;

[0118] S3023: Substitute the offset angle, the moving distance, and the first mapping relationship into the sixth calculation formula and the seventh calculation formula respectively, and calculate the moving distances of the end of the robotic arm in the X-axis and Y-axis directions of the robotic arm coordinate system in sequence. The sixth calculation formula is: offset_x = -cos(cur_angle - angle_base) * distance * scale, and the seventh calculation formula is: offset_y = sin(cur_angle - angle_base) * distance * scale, where offset_x is the moving distance of the end of the robotic arm in the X-axis direction of the robotic arm coordinate system, offset_y is the moving distance of the end of the robotic arm in the Y-axis direction of the robotic arm coordinate system, angle_base is the included angle parameter in the first mapping relationship, and scale is the scaling ratio parameter in the first mapping relationship;

[0119] S3024: Control the end of the robotic arm to move according to the moving distances in the X-axis and Y-axis directions of the robotic arm coordinate system respectively, so as to achieve the alignment of the image center point of the gripper camera with the second marking point.

[0120] Further, the step of calculating the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system through the central position coordinates includes:

[0121] S401: Based on the central position coordinates, control the center of the end of the robotic arm to align with multiple third marking points on the teaching board, and sequentially record the first position coordinates corresponding to each of the third marking points in the robotic arm coordinate system;

[0122] S402: Use a fixed positioning camera to collect an image of the teaching board, and obtain the seventh pixel coordinates corresponding to each of the third marking points in the fixed positioning camera coordinate system;

[0123] S403: Taking the same third marking point as the corresponding reference, respectively calculate the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system according to the mapping relationship between each of the first position coordinates and each of the seventh pixel coordinates.

[0124] Further, a dispensing tool is installed at the end of the robotic arm. After the step of calculating the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system through the central position coordinates, it includes:

[0125] S5: Use the fixed positioning camera to collect the dispensing pixel coordinates of the dispensing position of the workpiece in the fixed positioning camera coordinate system;

[0126] S6: According to the second mapping relationship, convert the dispensing pixel coordinates into the dispensing position coordinates in the robotic arm coordinate system;

[0127] S7: According to the dispensing position coordinates, control the end of the robotic arm to move to the dispensing position of the workpiece, and use the dispensing tool to perform a dispensing operation.

[0128] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements a calibration method for machine vision. A gripper camera is deployed at the end of the robotic arm. The calibration method for machine vision is specifically as follows:

[0129] S1: Respectively construct a robotic arm coordinate system, a gripper camera coordinate system, and a fixed positioning camera coordinate system. Among them, the robotic arm coordinate system represents a coordinate system constructed with a preset point on the mounting base of the robotic arm as the origin, the gripper camera coordinate system represents a coordinate system constructed with a preset point on the gripper camera as the origin, and the fixed positioning camera coordinate system represents a coordinate system constructed with a preset point on the fixed positioning camera as the origin;

[0130] S2: Based on the vision of the gripper camera, calibrate the first mapping relationship between the robotic arm coordinate system and the gripper camera coordinate system through a calibration method;

[0131] S3: Determine the central position coordinates of the end center of the robotic arm in the hand gripper camera coordinate system according to the first mapping relationship;

[0132] S4: Calculate the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system through the central position coordinates.

[0133] Further, the step of calibrating the first mapping relationship between the robotic arm coordinate system and the hand gripper camera coordinate system by a calibration method based on the vision of the hand gripper camera includes:

[0134] S201: Based on the vision of the hand gripper camera, control the end of the robotic arm to move above the teaching board, and record the first pixel coordinates of the first marking point of the teaching board in the hand gripper camera coordinate system;

[0135] S202: Control the end of the robotic arm to move a preset distance along the X-axis direction of the robotic arm coordinate system, and record the second pixel coordinates of the current first marking point in the hand gripper camera coordinate system;

[0136] S203: Calculate the first mapping relationship through affine transformation according to the first pixel coordinates, the second pixel coordinates, and the preset distance.

[0137] Further, the first mapping relationship includes an included angle parameter and a scaling ratio parameter. The step of calculating the first mapping relationship through affine transformation according to the first pixel coordinates, the second pixel coordinates, and the preset distance includes:

[0138] S2031: Substitute the first pixel coordinates and the second pixel coordinates into the first calculation formula to calculate the included angle parameter. The first calculation formula is: angle_base = atan2((C2.X - C1.X), (C2.Y - C1.Y)), where angle_base is the included angle parameter, C1.X and C1.Y are the abscissa and ordinate of the first pixel coordinates in sequence, and C2.X and C2.Y are the abscissa and ordinate of the second pixel coordinates in sequence;

[0139] S2032: Substitute the first pixel coordinates and the second pixel coordinates into the second calculation formula to calculate the square of the pixel length. The second calculation formula is: visionLen_power = (C2.X - C1.X) × (C2.X - C1.X) + (C2.Y - C1.Y) × (C2.Y - C1.Y), where visionLen_power is the square of the pixel length;

[0140] S2033: Substitute the sum of the squares of the pixel lengths and the preset distance into the third calculation formula to calculate the scaling ratio parameter. The third calculation formula is: scale = x_offset / sqrt(visionLen_power), where scale is the scaling ratio parameter and x_offset is the preset distance.

[0141] Further, the step of determining the central position coordinates of the end center of the robotic arm in the gripper camera coordinate system according to the first mapping relationship includes:

[0142] S301: Obtain the third pixel coordinates of the image center point of the gripper camera and the fourth pixel coordinates of the second marking point of the teaching board;

[0143] S302: Based on the first mapping relationship and the fourth pixel coordinates as the corresponding reference, control the end of the robotic arm to move the image center point of the gripper camera to align with the second marking point;

[0144] S303: Control the end of the robotic arm to drive the gripper camera to rotate 180 degrees, and record the fifth pixel coordinates of the second marking point in the gripper camera coordinate system;

[0145] S304: Calculate the sixth pixel coordinates of the midpoint of the straight line between the second marking point and the image center point according to the third pixel coordinates and the fifth pixel coordinates, and use the sixth pixel coordinates as the central position coordinates.

[0146] Further, the step of controlling the end of the robotic arm to move the image center point of the gripper camera to align with the second marking point based on the first mapping relationship and the fourth pixel coordinates as the corresponding reference includes:

[0147] S3021: Substitute the third pixel coordinates and the fourth pixel coordinates into the fourth calculation formula to calculate the offset angle. The fourth calculation formula is: cur_angle = atan2((C0.X - Q.X), (C0.Y - Q.Y)), where cur_angle is the offset angle, Q.X and Q.Y are the abscissa and ordinate of the third pixel coordinates in sequence, and C0.x and C0.y are the abscissa and ordinate of the fourth pixel coordinates in sequence;

[0148] S3022: Substitute the third pixel coordinate and the fourth pixel coordinate into the fifth calculation formula to calculate the moving distance. The fifth calculation formula is: distance = sqrt((C0.X - Q.X)*(C0.X - Q.X)+(C0.Y - Q.Y)*(C0.Y - Q.Y)), where distance is the moving distance;

[0149] S3023: Substitute the offset angle, the moving distance, and the first mapping relationship into the sixth calculation formula and the seventh calculation formula respectively, and calculate in sequence the moving distances of the end of the robotic arm in the X-axis and Y-axis directions of the robotic arm coordinate system. The sixth calculation formula is: offset_x = -cos(cur_angle - angle_base)*distance*scale, and the seventh calculation formula is: offset_y = sin(cur_angle - angle_base)*distance*scale, where offset_x is the moving distance of the end of the robotic arm in the X-axis direction of the robotic arm coordinate system, offset_y is the moving distance of the end of the robotic arm in the Y-axis direction of the robotic arm coordinate system, angle_base is the included angle parameter in the first mapping relationship, and scale is the scaling ratio parameter in the first mapping relationship;

[0150] S3024: Control the end of the robotic arm to move respectively according to the moving distances in the X-axis and Y-axis directions of the robotic arm coordinate system, so as to align the image center point of the gripper camera with the second marking point.

[0151] Further, the step of calculating the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system through the center position coordinate includes:

[0152] S401: Taking the center position coordinate as a reference, control the center of the end of the robotic arm to align with multiple third marking points on the teaching board, and sequentially record the first position coordinates corresponding to each of the third marking points on the robotic arm coordinate system;

[0153] S402: Use the fixed positioning camera to collect the image of the teaching board to obtain the seventh pixel coordinates corresponding to each of the third marking points on the fixed positioning camera coordinate system;

[0154] S403: Taking the same third marking point as the corresponding reference, calculate the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system respectively according to the mapping relationship between each of the first position coordinates and each of the seventh pixel coordinates.

[0155] Further, a dispensing tool is installed at the end of the robotic arm. After the step of calculating the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system through the central position coordinates, the following steps are included:

[0156] S5: Collect the dispensing pixel coordinates of the dispensing position of the workpiece in the fixed positioning camera coordinate system through the fixed positioning camera;

[0157] S6: Convert the dispensing pixel coordinates into the dispensing position coordinates on the robotic arm coordinate system according to the second mapping relationship;

[0158] S7: Control the end of the robotic arm to move to the dispensing position of the workpiece according to the dispensing position coordinates, and use the dispensing tool to perform a dispensing operation.

[0159] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium provided in this application and used in the embodiments can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0160] It should be noted that in this text, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, device, first object or method comprising a series of elements not only includes those elements but also other elements not expressly listed, or further includes elements inherent in such a process, device, first object or method. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, device, first object or method comprising that element.

[0161] The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present application.

Claims

1. A calibration method for machine vision, characterized in that, A gripper camera is deployed at the end of the robotic arm, and the method includes: Construct a robotic arm coordinate system, a gripper camera coordinate system, and a fixed positioning camera coordinate system respectively. Among them, the robotic arm coordinate system represents a coordinate system constructed with a preset point of the mounting base of the robotic arm as the origin, the gripper camera coordinate system represents a coordinate system constructed with a preset point of the gripper camera as the origin, and the fixed positioning camera coordinate system represents a coordinate system constructed with a preset point of the fixed positioning camera as the origin; Based on the vision of the gripper camera, calibrate the first mapping relationship between the robotic arm coordinate system and the gripper camera coordinate system through a calibration method; According to the first mapping relationship, determine the central position coordinates of the end of the robotic arm in the gripper camera coordinate system; Through the central position coordinates, calculate the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system; The step of determining the central position coordinates of the end of the robotic arm in the gripper camera coordinate system according to the first mapping relationship includes: Obtain the third pixel coordinates of the image center point of the gripper camera and the fourth pixel coordinates of the second marking point of the teaching board; Based on the first mapping relationship and the fourth pixel coordinates as the corresponding reference, control the end of the robotic arm to move to the image center point of the gripper camera to align with the second marking point; Control the end of the robotic arm to drive the gripper camera to rotate 180 degrees, and record the fifth pixel coordinates of the second marking point in the gripper camera coordinate system; According to the third pixel coordinates and the fifth pixel coordinates, calculate the sixth pixel coordinates of the midpoint of the straight line between the second marking point and the image center point, and use the sixth pixel coordinates as the central position coordinates.

2. The calibration method of machine vision according to claim 1, characterized in that, The step of calibrating the first mapping relationship between the robotic arm coordinate system and the gripper camera coordinate system through a calibration method based on the vision of the gripper camera includes: Based on the vision of the gripper camera, control the end of the robotic arm to move above the teaching board, and record the first pixel coordinates of the first marking point of the teaching board in the gripper camera coordinate system; Control the end of the robotic arm to move a preset distance along the X-axis direction of the robotic arm coordinate system, and record the second pixel coordinates of the first marking point in the gripper camera coordinate system at present; According to the first pixel coordinates, the second pixel coordinates, and the preset distance, calculate the first mapping relationship through affine transformation.

3. The calibration method of machine vision according to claim 2, characterized in that, The first mapping relationship includes an included angle parameter and a scaling ratio parameter. The step of calculating the first mapping relationship through affine transformation according to the first pixel coordinates, the second pixel coordinates, and the preset distance includes: Substitute the first pixel coordinate and the second pixel coordinate into the first calculation formula to calculate the included angle parameter. The first calculation formula is: angle_base = atan2((C2.X - C1.X), (C2.Y - C1.Y)), where angle_base is the included angle parameter, C1.X and C1.Y are the abscissa and ordinate of the first pixel coordinate respectively, and C2.X and C2.Y are the abscissa and ordinate of the second pixel coordinate respectively; Substitute the first pixel coordinate and the second pixel coordinate into the second calculation formula to calculate the square of the pixel length. The second calculation formula is: visionLen_power = (C2.X - C1.X) × (C2.X - C1.X) + (C2.Y - C1.Y) × (C2.Y - C1.Y), where visionLen_power is the square of the pixel length; Substitute the square of the pixel length and the preset distance into the third calculation formula to calculate the scaling ratio parameter. The third calculation formula is: scale = x_offset / sqrt(visionLen_power), where scale is the scaling ratio parameter and x_offset is the preset distance.

4. The calibration method of machine vision according to claim 1, characterized in that, The step of controlling the end of the robotic arm to move to the image center point of the gripper camera to align with the second marking point with the first mapping relationship and the fourth pixel coordinate as the corresponding reference includes: Substitute the third pixel coordinate and the fourth pixel coordinate into the fourth calculation formula to calculate the offset angle. The fourth calculation formula is: cur_angle = atan2((C0.X - Q.X), (C0.Y - Q.Y)), where cur_angle is the offset angle, Q.X and Q.Y are the abscissa and ordinate of the third pixel coordinate respectively, and C0.x and C0.y are the abscissa and ordinate of the fourth pixel coordinate respectively; Substitute the third pixel coordinate and the fourth pixel coordinate into the fifth calculation formula to calculate the moving distance. The fifth calculation formula is: distance = sqrt((C0.X - Q.X) * (C0.X - Q.X) + (C0.Y - Q.Y) * (C0.Y - Q.Y)), where distance is the moving distance; Substitute the offset angle, the moving distance, and the first mapping relationship into the sixth calculation formula and the seventh calculation formula respectively, and calculate successively to obtain the moving distances of the end of the robotic arm in the X-axis and Y-axis directions of the robotic arm coordinate system. The sixth calculation formula is: offset_x = -cos(cur_angle - angle_base) * distance * scale, and the seventh calculation formula is: offset_y = sin(cur_angle - angle_base) * distance * scale, where offset_x is the moving distance of the end of the robotic arm in the X-axis direction of the robotic arm coordinate system, offset_y is the moving distance of the end of the robotic arm in the Y-axis direction of the robotic arm coordinate system, angle_base is the included angle parameter in the first mapping relationship, and scale is the scaling ratio parameter in the first mapping relationship; Control the end of the robotic arm to move respectively according to the moving distances in the X-axis and Y-axis directions of the robotic arm coordinate system, so as to align the image center point of the gripper camera with the second marking point.

5. The calibration method of machine vision according to claim 1, characterized in that, The step of calculating the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system through the center position coordinates includes: Taking the center position coordinates as a reference, control the center of the end of the robotic arm to align with multiple third marking points on the teaching board, and successively record the first position coordinates corresponding to each of the third marking points in the robotic arm coordinate system; Use the fixed positioning camera to collect the image of the teaching board to obtain the seventh pixel coordinates corresponding to each of the third marking points in the fixed positioning camera coordinate system; Taking the same third marking point as the corresponding reference, calculate the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system respectively according to the mapping relationship between each of the first position coordinates and each of the seventh pixel coordinates.

6. The calibration method of machine vision according to claim 1, characterized in that, After the step of calculating the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system through the center position coordinates, the end of the robotic arm is equipped with a dispensing tool, and it includes: Collect the dispensing pixel coordinates of the dispensing position of the workpiece in the fixed positioning camera coordinate system through the fixed positioning camera; According to the second mapping relationship, convert the dispensing pixel coordinates into the dispensing position coordinates on the robotic arm coordinate system; According to the dispensing position coordinates, control the end of the robotic arm to move to the dispensing position of the workpiece, and use the dispensing tool to perform a dispensing operation.

7. A calibration device for machine vision, characterized in that, A gripper camera is deployed at the end of the robotic arm, which is used to implement the method according to any one of claims 1 to 6. The device includes: A building module is used to respectively build a robotic arm coordinate system, a gripper camera coordinate system, and a fixed positioning camera coordinate system. Among them, the robotic arm coordinate system represents a coordinate system built with a preset point of the mounting base of the robotic arm as the origin, the gripper camera coordinate system represents a coordinate system built with a preset point of the gripper camera as the origin, and the fixed positioning camera coordinate system represents a coordinate system built with a preset point of the fixed positioning camera as the origin; A first calibration module is used to calibrate the first mapping relationship between the robotic arm coordinate system and the gripper camera coordinate system based on the vision of the gripper camera through a calibration method; A second calibration module is used to determine the central position coordinates of the end center of the robotic arm in the gripper camera coordinate system according to the first mapping relationship; A calculation module is used to calculate the second mapping relationship between the robotic arm coordinate system and the fixed positioning camera coordinate system through the central position coordinates.

8. A computer device, comprising a memory and a processor, wherein a computer program is stored in the memory, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 6.

Images