Method, system, device and storage medium for calibrating flow direction of conveyor belt

Abstract

The present invention discloses a conveyor belt flow direction calibration method, system, device and storage medium, comprising: obtaining a first coordinate angle and a scaling parameter between a robot arm and a camera device in response to a calibration instruction; calculating the rotation center coordinates of the rotation axis of the end of the robot arm in the camera field of view coordinate system; recording the first calibration coordinates and the second calibration coordinates of the end of the robot arm in the mobile reference coordinate system respectively; calculating the flow direction of the conveyor belt relative to the mobile reference coordinate system according to the first calibration coordinates and the second calibration coordinates to complete the flow direction calibration of the conveyor belt, thereby avoiding the configuration differences between the robot arm and the camera device, and the contact calibration method easily causing the calibration point and the conveyor belt to slide relative to each other during the calibration process, which ultimately leads to the invalidation of the calibration parameters, thereby improving the calibration accuracy of the conveyor belt flow direction and reducing the cumulative error of the robot arm tracking operation.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent technology, for example, to a method, system, computer device and storage medium for calibrating the flow direction of a conveyor belt. Background Art

[0002] In the process of industrial automation application, the application scope of machine vision is becoming more and more extensive. There is a conveyor belt tracking application scenario in current industrial production: a fixed positioning camera cooperates with an industrial robot arm, that is, the end of the robot arm is manually controlled to align with a mark point placed on the conveyor belt as much as possible, and the current position P1 of the robot arm is recorded at this time. Then the conveyor belt is started, and the conveyor belt carrying the mark point flows for a distance and then stops. The end of the robot arm is manually controlled to align with the mark point again, and the current position P2 of the robot arm is recorded at this time. The flow direction of the conveyor belt in the moving reference coordinate system is calculated based on P1 and P2. However, when the robot arm is manually controlled to align with the mark point position, there is often an aiming error, resulting in a large deviation in the calibrated parameters. The robot arm is tracked during work, and a large deviation in the flow direction of the conveyor belt will cause excessive cumulative errors after multiple tracking operations of the robot arm. Summary of the invention

[0003] The purpose of the present disclosure is to provide a method, system, computer device and storage medium for calibrating the flow direction of a conveyor belt, which has a high calibration accuracy and reduces the cumulative error of the robot arm tracking operation.

[0004] To achieve the above objectives, this application adopts the following technical solutions:

[0005] The present disclosure provides a method for calibrating the flow direction of a conveyor belt, comprising the following steps:

[0006] In response to the calibration instruction, obtaining a first coordinate angle and a scaling parameter between a moving reference coordinate system of the robot arm and a camera field coordinate system of a camera device, wherein the camera device is fixed to the end of the robot arm;

[0007] Calculating the rotation center coordinates of the rotation axis of the end of the robotic arm in the camera field of view coordinate system according to the first coordinate angle and the scaling parameter;

[0008] Controlling the movement of the robotic arm so that the rotation center coordinates overlap with the calibration point on the conveyor belt in the field of view of the camera device, and recording the first calibration coordinates of the end of the robotic arm in the moving reference coordinate system at this time;

[0009] Control the conveyor belt to move a certain distance and then stop, re-move the robotic arm so that the rotation center coordinates overlap with the calibration point on the conveyor belt again in the field of view of the camera device, and record the second calibration coordinates of the end of the robotic arm in the moving reference coordinate system at this time;

[0010] The flow direction of the conveyor belt relative to the moving reference coordinate system is calculated according to the first calibration coordinates and the second calibration coordinates to complete the flow direction calibration of the conveyor belt.

[0011] The present disclosure also provides a conveyor belt flow direction calibration system, comprising:

[0012] a parameter acquisition unit, configured to acquire, in response to a calibration instruction, a first coordinate angle between a moving reference coordinate system of the robot arm and a camera field coordinate system of a camera device and a scaling parameter, wherein the camera device is fixed to the end of the robot arm;

[0013] A rotation center acquisition unit, used to calculate the rotation center coordinates of the rotation axis of the end of the robotic arm in the camera field of view coordinate system according to the first coordinate angle and the scaling parameter;

[0014] A first calibration coordinate acquisition unit, used for controlling the movement of the robotic arm so that the rotation center coordinates overlap with the calibration point on the conveyor belt in the field of view of the camera device, and recording the first calibration coordinates of the end of the robotic arm in the mobile reference coordinate system at this time;

[0015] A second calibration coordinate acquisition unit is used to control the conveyor belt to stop after moving a certain distance, re-move the robotic arm so that the rotation center coordinates overlap with the calibration point on the conveyor belt again in the field of view of the camera device, and record the second calibration coordinates of the end of the robotic arm in the mobile reference coordinate system at this time;

[0016] A flow direction calculation unit is used to calculate the flow direction of the conveyor belt relative to the moving reference coordinate system according to the first calibration coordinates and the second calibration coordinates, so as to complete the flow direction calibration of the conveyor belt.

[0017] The present disclosure also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method for calibrating the flow direction of a conveyor belt described in any one of the above are implemented.

[0018] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method for calibrating the flow direction of a conveyor belt as described above are implemented.

[0019] The calibration method, system, device and storage medium for the conveyor belt flow direction disclosed in the present invention obtain the first coordinate angle and the scaling parameter between the robot arm and the camera device, thereby correcting the rotation axis of the end of the robot arm and the field of view center of the camera device, avoiding the problem of the calibration accuracy of the robot arm affected by the configuration difference between the robot arm and the camera device; the calibration points pre-marked on the conveyor belt are aligned and tracked by the camera device, so that the robot arm does not need to contact the calibration points to locate the calibration points, avoiding the contact calibration method, which is easy to cause the calibration points and the conveyor belt to slide relative to each other during the calibration process, and finally causing the calibration parameters to become invalid, thereby improving the calibration accuracy of the conveyor belt flow direction and reducing the cumulative error of the robot arm tracking operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of the steps of a method for calibrating the flow direction of a conveyor belt;

[0021] Figure 2 It is a schematic diagram of the moving trajectory of the robot arm in the moving reference coordinate system;

[0022] Figure 3 is a schematic diagram of the coordinate change of the first reference point in the camera field of view coordinate system;

[0023] Figure 4 It is a schematic diagram of the moving trajectory of the robot arm in the moving reference coordinate system;

[0024] Figure 5 It is a schematic diagram of the conveyor belt flow direction;

[0025] Figure 6 It is a structural block diagram of a conveyor belt flow direction calibration system;

[0026] Figure 7 The figure is a schematic block diagram of the structure of a computer device.

[0027] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present disclosure more clear, the present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not used to limit the present disclosure.

[0029] Reference Figure 1 , is a schematic diagram of the steps of a method for calibrating the flow direction of a conveyor belt in an embodiment disclosed in the present invention, the method comprising:

[0030] S1. In response to a calibration instruction, obtaining a first coordinate angle and a scaling parameter between a moving reference coordinate system of the robot arm and a camera field coordinate system of a camera device, wherein a camera device is fixed at the end of the robot arm;

[0031] In one embodiment, for step S1, the above-mentioned mechanical arm can be a four-axis Scara mechanical arm, and the above-mentioned camera device can be a gripper camera, which is fixed on the mechanical arm to be close to one end of the conveyor belt. Since in the specific application process, the positioning system of the mechanical arm and the image acquisition system of the camera device often use independent coordinate systems, and there will be certain differences in the scaling ratio and the direction of the coordinate system relative to the conveyor belt, in order to control the mechanical arm through the image acquisition result of the camera device and avoid the situation where the movement efficiency is low and the error is large due to manual movement of the mechanical arm, the above-mentioned first coordinate angle and scaling ratio parameters are obtained before the calibration point recognition to facilitate the calibration of the difference between the moving reference coordinate system of the mechanical arm and the camera field coordinate system of the camera device.

[0032] S2. Calculate the rotation center coordinates of the rotation axis of the end of the robotic arm in the camera field of view coordinate system according to the first coordinate angle and the scaling parameter;

[0033] In one embodiment, for step S2, although the camera device is fixed at the end of the robot arm, in actual applications, the fixed position of the camera device cannot be completely located at the center of the end position of the robot arm, and the fixed position of the camera device cannot ensure complete horizontality. Therefore, there is often a certain distance between the center of the field of view of the camera device and the rotation axis of the robot arm. In order to obtain more accurate robot arm control results based on the field of view of the camera device, it is necessary to first calculate the rotation center coordinates of the rotation axis of the robot arm corresponding to the camera field of view coordinate system, that is, calculate the position of the rotation axis of the robot arm in the field of view of the camera device, so that the camera device can mark and reference this position.

[0034] S3, controlling the movement of the robotic arm so that the rotation center coordinates overlap with the calibration point on the conveyor belt in the field of view of the camera device, and recording the first calibration coordinates of the end of the robotic arm in the moving reference coordinate system at this time;

[0035] In one embodiment, for step S3, the movement of the robotic arm is controlled so that the rotation center coordinates in the camera field of view coordinate system overlap with the calibration point collected in the camera field of view, that is, the rotation axis at the end of the robotic arm is physically located just above the calibration point, thereby reducing the error between the robotic arm and the calibration point.

[0036] S4, controlling the conveyor belt to move a certain distance and then stop, re-moving the robotic arm so that the rotation center coordinates and the calibration point on the conveyor belt overlap again in the field of view of the camera device, and recording the second calibration coordinates of the end of the robotic arm in the moving reference coordinate system at this time;

[0037] In one embodiment, for step S4, after the first calibration coordinate is recorded, the conveyor belt is turned on to make it work normally, and the calibration point on the conveyor belt is driven to move together to simulate the movement of items on the conveyor belt during assembly line production. After the conveyor belt moves, when the rotation center coordinates overlap with the calibration point on the conveyor belt again in the field of view of the camera device, the position of the rotation axis at the end of the robot arm is the position of the calibration point.

[0038] In one embodiment, for step S4, in order to prepare for calibrating the flow direction of the conveyor belt, it may be necessary to move the conveyor belt forward a sufficiently long distance during the actual calibration to avoid the problem of inaccurate flow direction judgment caused by too short a moving distance. Therefore, after the conveyor belt moves, the calibration point may move out of the field of view of the camera device. At this time, the mechanical arm can be manually moved until the calibration point appears in the field of view of the camera device. A search range can also be preset. For example, if there is no calibration point in the field of view of the camera device, the mechanical arm is controlled to automatically search a certain range. For example, the mechanical arm is controlled to use the first calibration coordinate as the reference point, and move a distance of X' along the positive and negative directions of the X-axis of the moving reference coordinate system, and then move a distance of Y' along the positive and negative directions of the Y-axis of the moving reference coordinate system. If the camera device still fails to recognize the calibration point within the above range, the calibration point is searched again along the directions with an angle of 45° with the positive direction of the X-axis, the negative direction of the X-axis, the positive direction of the Y-axis, and the negative direction of the Y-axis, until the calibration point appears in the field of view of the camera device.

[0039] S5. Calculate the flow direction of the conveyor belt relative to the moving reference coordinate system according to the first calibration coordinate and the second calibration coordinate to complete the flow direction calibration of the conveyor belt.

[0040] In one embodiment, for step S5, since the first calibration coordinate and the second calibration coordinate are both the position coordinates of the robot arm when the rotation axis of the end of the robot arm is above the calibration point, the line connecting the first calibration coordinate and the second calibration coordinate in the moving reference coordinate system, that is, the flow direction of the conveyor belt relative to the moving reference coordinate system of the robot arm, is calculated by calculating the angle between the above-mentioned line and the Y-axis or X-axis in the moving reference coordinate system, so as to quantify the flow direction of the conveyor belt for the robot arm.

[0041] In one embodiment, after step S5, when the angle of the conveyor belt flow direction is quantified in the moving reference coordinate system of the robot arm, the objects on the conveyor belt can be tracked according to the angle to realize automated assembly line production. For example, the robot arm is controlled to align with an object on the conveyor belt so that the robot arm processes the object. After the processing is completed, the robot arm can move a certain distance along the flow direction to process the next object. The distance is the preset placement distance between two objects on the conveyor belt.

[0042] In summary, by acquiring the first coordinate angle and scaling parameters between the robot arm and the camera device, the rotation axis of the end of the robot arm and the center of the field of view of the camera device are corrected, thereby avoiding the problem of the calibration accuracy of the robot arm affected by the configuration difference between the robot arm and the camera device; the calibration points pre-marked on the conveyor belt are aligned and tracked by the camera device, so that the robot arm does not need to contact the calibration points to locate the calibration points, avoiding the contact calibration method, which is easy to cause the calibration points and the conveyor belt to slide relative to each other during the calibration process, and ultimately leading to the invalidity of the calibration parameters, thereby improving the calibration accuracy of the conveyor belt flow direction and reducing the cumulative error of the robot arm tracking operation.

[0043] Further, for step S1, the step of obtaining a first coordinate angle between the moving reference coordinate system of the robot arm and the camera field coordinate system of the camera device and a scaling parameter includes:

[0044] Control the robot arm to move to above the teaching panel marked with a first reference point, and record the current position of the end of the robot arm as the first reference coordinate, wherein the end of the robot arm is an end close to the teaching panel, and a camera device is fixed to the end;

[0045] Controlling the camera device to take a picture of the first reference point to obtain a first reference image;

[0046] Acquire a first reference pixel position of a first reference point in the first reference image;

[0047] Controlling the robotic arm to move a first reference distance along the horizontal axis direction of the moving reference coordinate system, and recording the current position of the end of the robotic arm as a second reference coordinate;

[0048] Calculate the first coordinate angle according to the first reference coordinate and the second reference coordinate;

[0049] A scaling parameter is calculated using the first reference coordinate, the second reference coordinate, and the first reference distance.

[0050] In one embodiment, the teaching panel is a plane marked with a first reference point, and its purpose is to serve as a reference example to obtain in advance the coordinate conversion relationship between the camera device and the robot arm.

[0051] In one embodiment, referring to Figure 2 , is a schematic diagram of the movement trajectory of the robot arm in the mobile reference coordinate system in an embodiment disclosed in the present invention. When the robot arm is controlled to move above the teaching panel marked with the first reference point, the position of the robot arm in the mobile reference coordinate system is P1'. After the robot arm is controlled to move the first reference distance along the horizontal axis direction of the mobile reference coordinate system, the position of the robot arm in the mobile reference coordinate system is P2'; refer to Figure 3 , is a schematic diagram of the coordinate change of the first reference point in the camera field of view coordinate system in an embodiment disclosed in the present scheme. At this time, since the installation orientation of the camera device is often difficult to be exactly parallel to the X-axis or Y-axis in the moving reference coordinate system of the robot arm, after the robot arm moves along the X-axis of the moving reference coordinate system, relative to the camera device, the moving direction of the first reference point is not parallel to the X-axis of the camera field of view coordinate system. At this time, the angle difference between the camera device and the robot arm, that is, the first coordinate angle, can be obtained according to the moving direction of the first reference point and the X-axis direction of the camera field of view coordinate system.

[0052] In one embodiment, since the pixel coordinate size of the camera device is often different from the size of the moving reference coordinate system of the robotic arm, the proportional relationship between them is also different. Therefore, it is indispensable to calculate the scaling ratio parameters between the two. The coordinate conversion between the camera device and the robotic arm can be achieved based on the first coordinate angle and the scaling ratio relationship.

[0053] Further, for step S1, the step of calculating the first coordinate angle according to the first reference coordinate and the second reference coordinate includes:

[0054] The first coordinate angle is calculated by the following formula:

[0055] angle_base=atan2((C2.X-C1.X),(C2.Y-C1.Y))

[0056] Where angle_base is the first coordinate angle, C1.X and C1.Y are the X coordinate value and Y coordinate value of the first reference coordinate, respectively, and C2.X and C2.Y are the X coordinate value and Y coordinate value of the second reference coordinate.

[0057] In one embodiment, since the robot arm moves along the X-axis direction in the moving reference coordinate system, it is only necessary to establish a camera field of view coordinate system whose X-axis direction is the same as the X-axis direction in the moving reference coordinate system. At this time, the first coordinate angle can be calculated with the X-axis as one side. The other side of the first coordinate angle can be obtained according to the first reference coordinate and the second reference coordinate of the above formula. The angle between the other side and the X-axis is calculated as the first coordinate angle. The parameters between the two coordinate systems are converted according to the angle, thereby improving the accuracy of the control of the robot arm.

[0058] Further, for step S1, the step of calculating the scaling parameter by using the first reference coordinate, the second reference coordinate and the first reference distance includes:

[0059] The scaling parameter is calculated by the following formula:

[0060] visionLen_power=(C2.X-C1.X)*(C2.X-C1.X)+(C2.Y-C1.Y)*(C2.Y-C1.Y)

[0061] scale=x_offset / sqrt(visionLen_power)

[0062] Wherein, visionLen_power is the field of view movement parameter, scale is the scaling parameter, and x_offset is the first reference distance.

[0063] In one embodiment, the scaling parameters between the two coordinate systems are calculated by acquiring the first reference distance of the robot arm movement and the distance that the first reference point moves in the camera field of view coordinate system due to the first reference distance, thereby further improving the accuracy of controlling the robot arm through the camera device.

[0064] Further, for step S2, the step of calculating the rotation center coordinates of the rotation axis of the end of the robot arm in the camera field of view coordinate system according to the first coordinate angle and the scaling parameter includes:

[0065] Calculate the reference movement parameters of the robot arm according to the coordinates of the center of the field of view of the camera device, the first coordinate angle and the scaling parameters;

[0066] Controlling the movement of the robotic arm according to the reference movement parameter so that the center of the field of view of the camera device overlaps with the first reference point in the field of view of the camera device;

[0067] Control the end of the robotic arm to rotate 180° along the rotation axis, and control the camera device to take a picture of the first reference point to obtain an offset reference image;

[0068] Obtaining a third reference pixel position of the first reference point in the offset reference image;

[0069] The rotation center coordinates of the rotation axis of the end of the robot arm in the camera field of view coordinate system are calculated according to the third reference pixel position.

[0070] In one embodiment, the reference movement parameters of the robotic arm are calculated based on the coordinates of the field of view center Q of the camera device, the first coordinate angle and the scaling ratio parameters, specifically including: first obtaining the pixel coordinates of the field of view center Q of the camera device in the camera field of view coordinate system, that is, the coordinates of the field of view center Q mentioned above, and identifying the pixel coordinates of the first reference point A in the camera field of view coordinate system through the image captured by the camera device, that is, the reference point coordinates; when the coordinates of the field of view center Q and the reference point coordinates do not overlap, calculating the distance between the coordinates of the field of view center Q and the reference point coordinates and the direction relative to the coordinate axis, calculating the distance that the robotic arm needs to move when moving the field of view center Q of the camera device to the first reference point A through the scaling ratio parameters, and calculating the direction that the robotic arm needs to move when moving the field of view center Q of the camera device to the first reference point A through the first coordinate angle, thereby realizing automatic control of the robotic arm based on the camera device and avoiding manual intervention to move the robotic arm.

[0071] In one embodiment, referring to Figure 4 , is a schematic diagram of the moving trajectory of the robot arm in the mobile reference coordinate system in an embodiment disclosed in the present scheme. When the coordinates of the center of view Q overlap with the coordinates of the reference point, the robot arm is rotated 180° along the U axis, that is, the vertical rotation axis, and the third reference pixel position of the rotated first reference point A in the camera field of view coordinate system is recorded. The third reference pixel position is connected with the coordinates of the center of view Q, and the midpoint O of the connecting line is the rotation center coordinate of the rotation axis of the end of the robot arm in the camera field of view coordinate system.

[0072] It should be noted that the sizes of the field of view center Q, rotation center O and first reference point A shown in the figure are only exemplary and are not actual measured dimensions. In actual applications, the first reference point A is an entity, which can be a dot painted with paint or a circular sticker. Therefore, the first reference point A has a certain radius and area. At this time, the midpoint of the shape of the first reference point A can be taken as the theoretical value of the first reference point A mentioned above in this embodiment, and the coordinates of the midpoint can be used as the reference point coordinates to avoid the problem of alignment error caused by the physical area of ​​the first reference point A.

[0073] Further, for step S2, the step of calculating the reference movement parameters of the robot arm according to the center coordinates of the field of view of the camera device, the first coordinate angle and the scaling parameter includes:

[0074] Obtaining the length and width of the first reference image, and calculating the X and Y coordinates of the center of the field of view according to the length and width of the first reference image;

[0075] Controlling the camera device to take a picture of the first reference point to obtain an initial reference image;

[0076] Obtaining an initial reference pixel position of a first reference point in an initial reference image;

[0077] The offset angle is calculated by the following formula:

[0078] cur_angle=atan2((C0.XQ.X),(C0.YQ.Y))

[0079] Where cur_angle is the offset angle, QX and QY are the X and Y coordinates of the center of the field of view, and C0.X and C0.Y are the X and Y coordinates of the initial reference pixel position.

[0080] The offset distance is calculated by the following formula:

[0081] distance=sqrt((C0.XQ.X)*(C0.XQ.X)+(C0.YQ.Y)*(C0.YQ.Y))

[0082] Where, distance is the offset distance;

[0083] The X-axis reference moving distance and Y-axis reference moving distance of the robot arm are calculated by the following formula:

[0084] offset_x=-cos(cur_angle-angle_base)*distance*scale

[0085] offset_y=sin(cur_angle-angle_base)*distance*scale

[0086] Where, offset_x is the reference moving distance of the X axis, and offset_y is the reference moving distance of the Y axis;

[0087] The combination of the X-axis reference movement distance and the Y-axis reference movement distance is used as a reference movement parameter;

[0088] The step of calculating the rotation center coordinates of the rotation axis of the end of the robot arm in the camera field of view coordinate system according to the third reference pixel position includes:

[0089] Calculate the rotation center coordinates O((w / 2+x3) / 2,(h / 2+y3) / 2), where w is the length value, h is the width value, and the third reference pixel position is (x3,y3).

[0090] In one embodiment, the length value w and the width value h of the first reference image may be obtained first. Since the size of the image captured by the camera device each time is usually consistent, there is no need to re-acquire the size of the remaining reference images.

[0091] In one embodiment, the reference movement parameters of the robotic arm are obtained through the coordinates of the center of the field of view, the angle of the first coordinate and the scaling parameters, which can realize automatic control of the robotic arm based on the camera device, so that it can automatically move a certain distance according to the above-mentioned reference movement parameters, thereby realizing the next alignment at the physical position, improving the calibration efficiency and accuracy, and avoiding errors caused by human intervention.

[0092] Further, for step S6, the step of calculating the flow direction of the conveyor belt relative to the moving reference coordinate system according to the first calibration coordinate and the second calibration coordinate includes:

[0093] The moving angle between the flow direction and the X-axis of the moving reference coordinate system is calculated by the following formula:

[0094] beltAngleToRobot=atan2(P1.Y-P2.Y,P1.X-P2.X)

[0095] Wherein, beltAngleToRobot is the moving angle between the flow direction and the X-axis of the moving reference coordinate system, P1.X and P1.Y are the X-coordinate value and Y-coordinate value of the first calibration coordinate, and P2.X and P2.Y are the X-coordinate value and Y-coordinate value of the second calibration coordinate;

[0096] The direction of the extension line of the line between the first calibration coordinate and the second calibration coordinate is used as the flow direction of the conveyor belt.

[0097] In one embodiment, referring to Figure 5 , is a schematic diagram of the flow direction of a conveyor belt disclosed in the present invention, and the first calibration coordinate P1 of the calibration point before the conveyor belt flows in the camera field of view coordinate system and the second calibration coordinate P2 of the calibration point after the conveyor belt flows in the camera field of view coordinate system are obtained respectively, so as to obtain the precise degree of the moving angle through trigonometric calculation, and the moving angle is the angle of the flow direction relative to the moving reference coordinate system; specifically, one side of the moving angle is the X-axis of the moving reference coordinate system, and the other side is the extension line of the line connecting the first calibration coordinate P1 to the second calibration coordinate P2.

[0098] Reference Figure 6 , is a structural block diagram of a conveyor belt flow direction calibration system disclosed in this solution, the system includes:

[0099] The parameter acquisition unit 100 is used to acquire a first coordinate angle and a scaling parameter between a moving reference coordinate system of the robot arm and a camera field coordinate system of a camera device in response to a calibration instruction, wherein the camera device is fixed to the end of the robot arm;

[0100] A rotation center acquisition unit 200, used to calculate the rotation center coordinates of the rotation axis of the end of the robotic arm in the camera field of view coordinate system according to the first coordinate angle and the scaling parameter;

[0101] A first calibration coordinate acquisition unit 300, used to control the movement of the robot arm so that the rotation center coordinates overlap with the calibration point on the conveyor belt in the field of view of the camera device, and record the first calibration coordinates of the end of the robot arm in the mobile reference coordinate system at this time;

[0102] The second calibration coordinate acquisition unit 400 is used to control the conveyor belt to stop after moving a certain distance, re-move the robotic arm so that the rotation center coordinates overlap with the calibration point on the conveyor belt again in the field of view of the camera device, and record the second calibration coordinates of the end of the robotic arm in the mobile reference coordinate system at this time;

[0103] The flow direction calculation unit 500 is used to calculate the flow direction of the conveyor belt relative to the moving reference coordinate system according to the first calibration coordinates and the second calibration coordinates, so as to complete the flow direction calibration of the conveyor belt.

[0104] Furthermore, the parameter acquisition unit 100 is specifically used for:

[0105] Controlling the robotic arm to move to above a teaching panel marked with a first reference point, and recording the current position of the end of the robotic arm as a first reference coordinate, wherein the end of the robotic arm is an end close to the teaching panel, and a camera device is fixed to the end;

[0106] Controlling the camera to take a picture of the first reference point to obtain a first reference image;

[0107] Acquire a first reference pixel position of the first reference point in the first reference image;

[0108] Controlling the robotic arm to move a first reference distance along the horizontal axis direction of the moving reference coordinate system, and recording the current position of the end of the robotic arm as a second reference coordinate;

[0109] Calculate the first coordinate angle according to the first reference coordinate and the second reference coordinate;

[0110] The scaling parameter is calculated by using the first reference coordinate, the second reference coordinate and the first reference distance.

[0111] Furthermore, the parameter acquisition unit 100 is also used for:

[0112] The first coordinate angle is calculated by the following formula:

[0113] angle_base=atan2((C2.X-C1.X),(C2.Y-C1.Y))

[0114] Wherein, angle_base is the first coordinate angle, C1.X and C1.Y are the X coordinate value and Y coordinate value of the first reference coordinate, respectively, and C2.X and C2.Y are the X coordinate value and Y coordinate value of the second reference coordinate.

[0115] Furthermore, the parameter acquisition unit 100 is also used for:

[0116] The scaling parameter is calculated by the following formula:

[0117] visionLen_power=(C2.X-C1.X)*(C2.X-C1.X)+(C2.Y-C1.Y)*(C2.Y-C1.Y)

[0118] scale=x_offset / sqrt(visionLen_power)

[0119] Wherein, visionLen_power is the field of view movement parameter, scale is the scaling parameter, and x_offset is the first reference distance.

[0120] Furthermore, the rotation center acquisition unit 200 is specifically used for:

[0121] Calculate the reference movement parameters of the robotic arm according to the coordinates of the center of the field of view of the camera device, the first coordinate angle and the scaling parameter;

[0122] Controlling the movement of the robotic arm according to the reference movement parameter so that the center of the field of view of the camera device overlaps with the first reference point in the field of view of the camera device;

[0123] Controlling the end of the robotic arm to rotate 180° along the rotation axis, and controlling the camera device to take a picture of the first reference point to obtain an offset reference image;

[0124] Acquire a third reference pixel position of the first reference point in the offset reference image;

[0125] The rotation center coordinates of the rotation axis of the end of the robotic arm in the camera field of view coordinate system are calculated according to the third reference pixel position.

[0126] Furthermore, the rotation center acquisition unit 200 is also used for:

[0127] Acquire the length value and the width value of the first reference image, and calculate the X coordinate value and the Y coordinate value of the center coordinate of the field of view according to the length value and the width value of the first reference image;

[0128] Controlling the camera to take a picture of the first reference point to obtain an initial reference image;

[0129] Acquire an initial reference pixel position of the first reference point in the initial reference image;

[0130] The offset angle is calculated by the following formula:

[0131] cur_angle=atan2((C0.XQ.X),(C0.YQ.Y))

[0132] Wherein, cur_angle is the offset angle, QX and QY are the X coordinate value and Y coordinate value of the center coordinate of the field of view, and C0.X and C0.Y are the X coordinate value and Y coordinate value of the initial reference pixel position;

[0133] The offset distance is calculated by the following formula:

[0134] distance=sqrt((C0.XQ.X)*(C0.XQ.X)+(C0.YQ.Y)*(C0.YQ.Y))

[0135] Wherein, distance is the offset distance;

[0136] The X-axis reference moving distance and the Y-axis reference moving distance of the robot arm are calculated by the following formula:

[0137] offset_x=-cos(cur_angle-angle_base)*distance*scale

[0138] offset_y=sin(cur_angle-angle_base)*distance*scale

[0139] Wherein, offset_x is the reference moving distance of the X-axis, and offset_y is the reference moving distance of the Y-axis;

[0140] Using a combination of the X-axis reference movement distance and the Y-axis reference movement distance as the reference movement parameter;

[0141] The step of calculating the rotation center coordinates of the rotation axis of the end of the robotic arm in the camera field of view coordinate system according to the third reference pixel position comprises:

[0142] Calculate the rotation center coordinates O((w / 2+x3) / 2,(h / 2+y3) / 2), where w is the length value, h is the width value, and the third reference pixel position is (x3, y3).

[0143] Furthermore, the flow direction calculation unit 500 is specifically used for:

[0144] The moving angle between the flow direction and the X-axis of the moving reference coordinate system is calculated by the following formula:

[0145] beltAngleToRobot=atan2(P1.Y-P2.Y,P1.X-P2.X)

[0146] Wherein, beltAngleToRobot is the moving angle between the flow direction and the X-axis of the moving reference coordinate system, P1.X and P1.Y are the X-coordinate value and Y-coordinate value of the first calibration coordinate, and P2.X and P2.Y are the X-coordinate value and Y-coordinate value of the second calibration coordinate;

[0147] The direction of the extension line of the line between the first calibration coordinate and the second calibration coordinate is used as the flow direction of the conveyor belt.

[0148] Reference Figure 7 In the embodiment of the present application, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store calibration data of the flow direction of the conveyor belt, etc. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for calibrating the flow direction of the conveyor belt is implemented.

[0149] Those skilled in the art will understand that Figure 7 The structure shown in is merely a block diagram of a portion of the structure related to the present application solution and does not constitute a limitation on the computer device to which the present application solution is applied.

[0150] An embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, a method for calibrating the flow direction of a conveyor belt is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0151] In summary, the calibration method, system, device and storage medium for the flow direction of the conveyor belt provided in the embodiments of the present application obtain the first coordinate angle between the robot arm and the camera device and the scaling ratio parameter, thereby correcting the rotation axis of the robot arm and the center of the field of view of the camera device, thereby avoiding the problem of the calibration accuracy of the robot arm affected by the configuration difference between the robot arm and the camera device; the calibration points pre-marked on the conveyor belt are aligned and tracked by the camera device, so that the robot arm does not need to contact the calibration points to locate the calibration points, thereby avoiding the contact calibration method, which is easy to cause the calibration points and the conveyor belt to slide relative to each other during the calibration process, and ultimately causing the calibration parameters to fail. The problem of large calibration error caused by manual intervention in moving the robot arm is avoided. The precise angle, i.e. the angle of the flow direction relative to the moving reference coordinate system, is obtained by calculating the trigonometric function, so that the robot arm is controlled to automatically track the items on the conveyor belt according to the angle, thereby improving the accuracy of the robot arm tracking operation.

[0152] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0153] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0154] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for calibrating the flow direction of a conveyor belt, characterized in that: The following steps are involved: In response to the calibration instruction, obtaining a first coordinate angle and a scaling parameter between a moving reference coordinate system of the robot arm and a camera field coordinate system of a camera device, wherein the camera device is fixed to the end of the robot arm; Calculating the rotation center coordinates of the rotation axis of the end of the robotic arm in the camera field of view coordinate system according to the first coordinate angle and the scaling parameter; Controlling the movement of the robotic arm so that the rotation center coordinates overlap with the calibration point on the conveyor belt in the field of view of the camera device, and recording the first calibration coordinates of the end of the robotic arm in the moving reference coordinate system at this time; Control the conveyor belt to move a certain distance and then stop, re-move the robotic arm so that the rotation center coordinates overlap with the calibration point on the conveyor belt again in the field of view of the camera device, and record the second calibration coordinates of the end of the robotic arm in the moving reference coordinate system at this time; Calculating the flow direction of the conveyor belt relative to the moving reference coordinate system according to the first calibration coordinate and the second calibration coordinate to complete the flow direction calibration of the conveyor belt; Furthermore, the step of obtaining a first coordinate angle and a scaling parameter between the moving reference coordinate system of the robot arm and the camera field coordinate system of the camera device includes: Control the robot arm to move to above the teaching panel marked with a first reference point, and record the current position of the end of the robot arm as the first reference coordinate, wherein the end of the robot arm is an end close to the teaching panel, and a camera device is fixed to the end; Controlling the camera device to take a picture of the first reference point to obtain a first reference image; Acquire a first reference pixel position of a first reference point in the first reference image; Controlling the robotic arm to move a first reference distance along the horizontal axis direction of the moving reference coordinate system, and recording the current position of the end of the robotic arm as a second reference coordinate; Calculate the first coordinate angle according to the first reference coordinate and the second reference coordinate; Calculate a scaling parameter using the first reference coordinate, the second reference coordinate and the first reference distance; Further, the step of calculating the scaling parameter by using the first reference coordinate, the second reference coordinate and the first reference distance includes: The scaling parameter is calculated by the following formula: visionLen_power=(C2.X-C1.X)*(C2.X-C1.X)+(C2.Y-C1.Y)*(C2.Y-C1.Y) scale=x_offset / sqrt(visionLen_power) Where, visionLen_power is the field of view movement parameter, scale is the scaling parameter, and x_offset is the first reference distance; Further, the step of calculating the rotation center coordinates of the rotation axis of the end of the robot arm in the camera field of view coordinate system according to the first coordinate angle and the scaling parameter includes: Calculate reference movement parameters of the robot arm according to the coordinates of the center of the field of view of the camera device, the first coordinate angle and the scaling parameter; Controlling the movement of the robotic arm according to the reference movement parameter so that the center of the field of view of the camera device overlaps with the first reference point in the field of view of the camera device; Control the end of the robotic arm to rotate 180° along the rotation axis, and control the camera device to take a picture of the first reference point to obtain an offset reference image; Obtaining a third reference pixel position of the first reference point in the offset reference image; Calculate the rotation center coordinates of the rotation axis of the end of the robot arm in the camera field of view coordinate system according to the third reference pixel position; Calculating the reference movement parameters of the robotic arm based on the coordinates of the field of view center Q of the camera device, the first coordinate angle and the scaling ratio parameters specifically includes: first obtaining the pixel coordinates of the field of view center Q of the camera device in the camera field of view coordinate system, that is, the coordinates of the field of view center Q mentioned above, and identifying the pixel coordinates of the first reference point A in the camera field of view coordinate system through the image captured by the camera device, that is, the reference point coordinates; when the coordinates of the field of view center Q and the reference point coordinates do not overlap, calculating the distance between the coordinates of the field of view center Q and the reference point coordinates and the direction relative to the coordinate axis, calculating the distance that the robotic arm needs to move when moving the field of view center Q of the camera device to the first reference point A through the scaling ratio parameters, and calculating the direction that the robotic arm needs to move when moving the field of view center Q of the camera device to the first reference point A through the first coordinate angle, thereby realizing automatic control of the robotic arm based on the camera device and avoiding manual intervention to move the robotic arm.

2. The method for calibrating the flow direction of a conveyor belt according to claim 1, wherein: The step of calculating the first coordinate angle according to the first reference coordinate and the second reference coordinate comprises: The first coordinate angle is calculated by the following formula: angle_base=atan2((C2.X-C1.X),(C2.Y-C1.Y)) Wherein, angle_base is the first coordinate angle, C1.X and C1.Y are the X coordinate value and Y coordinate value of the first reference coordinate, respectively, and C2.X and C2.Y are the X coordinate value and Y coordinate value of the second reference coordinate.

3. The method for calibrating the flow direction of a conveyor belt according to claim 1, wherein: The step of calculating the reference movement parameters of the robotic arm according to the center coordinates of the field of view of the camera device, the first coordinate angle and the scaling parameter comprises: Acquire the length value and the width value of the first reference image, and calculate the X coordinate value and the Y coordinate value of the center coordinate of the field of view according to the length value and the width value of the first reference image; Controlling the camera to take a picture of the first reference point to obtain an initial reference image; Acquire an initial reference pixel position of the first reference point in the initial reference image; The offset angle is calculated by the following formula: cur_angle=atan2((C0.XQ.X),(C0.YQ.Y)) Wherein, cur_angle is the offset angle, QX and QY are the X coordinate value and Y coordinate value of the center coordinate of the field of view, and C0.X and C0.Y are the X coordinate value and Y coordinate value of the initial reference pixel position; The offset distance is calculated by the following formula: distance=sqrt((C0.XQ.X)*(C0.XQ.X)+(C0.YQ.Y)*(C0.YQ.Y)) Wherein, distance is the offset distance; The X-axis reference moving distance and the Y-axis reference moving distance of the robot arm are calculated by the following formula: offset_x=-cos(cur_angle-angle_base)*distance*scale offset_y=sin(cur_angle-angle_base)*distance*scale Wherein, offset_x is the reference moving distance of the X-axis, and offset_y is the reference moving distance of the Y-axis; Using a combination of the X-axis reference movement distance and the Y-axis reference movement distance as the reference movement parameter; The step of calculating the rotation center coordinates of the rotation axis of the end of the robotic arm in the camera field of view coordinate system according to the third reference pixel position comprises: Calculate the rotation center coordinates O((w / 2+x3) / 2,(h / 2+y3) / 2), where w is the length value, h is the width value, and the third reference pixel position is (x3, y3).

4. The method for calibrating the flow direction of a conveyor belt according to claim 3, wherein: The step of calculating the flow direction of the conveyor belt relative to the moving reference coordinate system according to the first calibration coordinate and the second calibration coordinate comprises: The moving angle between the flow direction and the X-axis of the moving reference coordinate system is calculated by the following formula: beltAngleToRobot=atan2(P1.Y-P2.Y,P1.X-P2.X) Wherein, beltAngleToRobot is the moving angle between the flow direction and the X-axis of the moving reference coordinate system, P1.X and P1.Y are the X-coordinate value and Y-coordinate value of the first calibration coordinate, and P2.X and P2.Y are the X-coordinate value and Y-coordinate value of the second calibration coordinate; The direction of the extension line of the line between the first calibration coordinate and the second calibration coordinate is used as the flow direction of the conveyor belt.

5. A conveyor belt flow direction calibration system, characterized in that: include: a parameter acquisition unit, configured to acquire, in response to a calibration instruction, a first coordinate angle between a moving reference coordinate system of the robot arm and a camera field coordinate system of a camera device and a scaling parameter, wherein the camera device is fixed to the end of the robot arm; A rotation center acquisition unit, used to calculate the rotation center coordinates of the rotation axis of the end of the robotic arm in the camera field of view coordinate system according to the first coordinate angle and the scaling parameter; A first calibration coordinate acquisition unit, used for controlling the movement of the robotic arm so that the rotation center coordinates overlap with the calibration point on the conveyor belt in the field of view of the camera device, and recording the first calibration coordinates of the end of the robotic arm in the mobile reference coordinate system at this time; A second calibration coordinate acquisition unit is used to control the conveyor belt to stop after moving a certain distance, re-move the robotic arm so that the rotation center coordinates overlap with the calibration point on the conveyor belt again in the field of view of the camera device, and record the second calibration coordinates of the end of the robotic arm in the mobile reference coordinate system at this time; a flow direction calculation unit, used for calculating the flow direction of the conveyor belt relative to the moving reference coordinate system according to the first calibration coordinate and the second calibration coordinate, so as to complete the flow direction calibration of the conveyor belt; Furthermore, the step of obtaining a first coordinate angle and a scaling parameter between the moving reference coordinate system of the robot arm and the camera field coordinate system of the camera device includes: Control the robot arm to move to above the teaching panel marked with a first reference point, and record the current position of the end of the robot arm as the first reference coordinate, wherein the end of the robot arm is an end close to the teaching panel, and a camera device is fixed to the end; Controlling the camera device to take a picture of the first reference point to obtain a first reference image; Acquire a first reference pixel position of a first reference point in the first reference image; Controlling the robotic arm to move a first reference distance along the horizontal axis direction of the moving reference coordinate system, and recording the current position of the end of the robotic arm as a second reference coordinate; Calculate the first coordinate angle according to the first reference coordinate and the second reference coordinate; Calculate a scaling parameter using the first reference coordinate, the second reference coordinate and the first reference distance; Further, the step of calculating the scaling parameter by using the first reference coordinate, the second reference coordinate and the first reference distance includes: The scaling parameter is calculated by the following formula: visionLen_power=(C2.X-C1.X)*(C2.X-C1.X)+(C2.Y-C1.Y)*(C2.Y-C1.Y) scale=x_offset / sqrt(visionLen_power) Where, visionLen_power is the field of view movement parameter, scale is the scaling parameter, and x_offset is the first reference distance; Further, the step of calculating the rotation center coordinates of the rotation axis of the end of the robot arm in the camera field of view coordinate system according to the first coordinate angle and the scaling parameter includes: Calculate reference movement parameters of the robot arm according to the coordinates of the center of the field of view of the camera device, the first coordinate angle and the scaling parameter; Controlling the movement of the robotic arm according to the reference movement parameter so that the center of the field of view of the camera device overlaps with the first reference point in the field of view of the camera device; Control the end of the robotic arm to rotate 180° along the rotation axis, and control the camera device to take a picture of the first reference point to obtain an offset reference image; Obtaining a third reference pixel position of the first reference point in the offset reference image; Calculate the rotation center coordinates of the rotation axis of the end of the robot arm in the camera field of view coordinate system according to the third reference pixel position; Calculating the reference movement parameters of the robotic arm based on the coordinates of the field of view center Q of the camera device, the first coordinate angle and the scaling ratio parameters specifically includes: first obtaining the pixel coordinates of the field of view center Q of the camera device in the camera field of view coordinate system, that is, the coordinates of the field of view center Q mentioned above, and identifying the pixel coordinates of the first reference point A in the camera field of view coordinate system through the image captured by the camera device, that is, the reference point coordinates; when the coordinates of the field of view center Q and the reference point coordinates do not overlap, calculating the distance between the coordinates of the field of view center Q and the reference point coordinates and the direction relative to the coordinate axis, calculating the distance that the robotic arm needs to move when moving the field of view center Q of the camera device to the first reference point A through the scaling ratio parameters, and calculating the direction that the robotic arm needs to move when moving the field of view center Q of the camera device to the first reference point A through the first coordinate angle, thereby realizing automatic control of the robotic arm based on the camera device and avoiding manual intervention to move the robotic arm.

6. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the method for calibrating the flow direction of the conveyor belt according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for calibrating the flow direction of a conveyor belt according to any one of claims 1 to 4 are implemented.

Images