Robot joint deformation sensor, mechanical arm and method based on machine vision

Abstract

The present invention discloses a robot joint deformation sensor, a robot arm and a method based on machine vision. The sensor is installed in the robot joint. The robot joint includes a first mounting part and a second mounting part that are relatively arranged, and a supporting part that connects and supports the first mounting part and the second mounting part. The sensor includes a camera and a marker; the camera is fixed on a side of the first mounting part facing the second mounting part; the marker is arranged on a side of the second mounting part facing the first mounting part. The camera is electrically connected to the controller of the robot, and the camera takes a picture of the marker to record the position change of the marker. In the robot joint deformation sensor based on machine vision of the present invention, a camera and a marker are arranged in the robot joint. The camera can take an image of the marker by the camera, and the position change of the marker before and after the collision can be compared, so as to determine the deformation data of the robot joint after the collision, and then determine the position after the collision.

Description

Technical Field

[0001] The present invention belongs to the field of robot sensors, and in particular relates to a robot joint deformation sensor, a mechanical arm and a method based on machine vision. Background Art

[0002] Robots are widely used in industrial production. In the process of using robots, human error, parts failure, control system failure and other accidents may cause deviation of the motion trajectory and workpiece offset, which may cause collisions in the robot during work. If the collision is not detected and the movement of the robot is not stopped in time, it may cause damage to the robot system, expensive tools at the end or expensive workpieces, and even cause more serious safety problems. The reason for the above accidents is that the existing robots have certain deficiencies in the anti-collision system and have not established an effective active prevention mechanism. It is difficult for the robot to predict or detect the collision in the first time, and take the initiative to interrupt or avoid measures. It is also impossible to restore the working position in time after the collision is resolved, so as to continue the interrupted operation. In response to this, more and more industrial robot companies have invested in the research and development of automatic collision detection and recognition and posture self-recovery systems to try to solve the problem of robot collision.

[0003] Existing solutions mainly focus on the prevention and protection of the collision itself. After the collision is resolved, it is also a very important requirement that the robot arm can quickly return to the position before the collision and continue the interrupted operation. By controlling the operating status and records of the control system, the theoretical position of the robot arm when the collision signal is triggered can be retrieved. However, the actual stop position of the robot arm is affected by factors such as signal delay and joint deformation, making it difficult to accurately locate. Summary of the invention

[0004] In order to solve the problem in the prior art that it is difficult to accurately position a robotic arm after a collision, the present invention provides a robot joint deformation sensor, a robotic arm and a method based on machine vision.

[0005] The present invention adopts the following technical solutions:

[0006] A robot joint deformation sensor based on machine vision is installed in a robot joint, wherein the robot joint comprises a first mounting portion and a second mounting portion arranged opposite to each other and a supporting portion connecting and supporting the first mounting portion and the second mounting portion, and the robot joint deformation sensor comprises one or more camera devices and one or more markers;

[0007] The camera device is fixed on a side of the first mounting part facing the second mounting part; the marker is arranged on a side of the second mounting part facing the first mounting part, the camera device is electrically connected to the controller of the robot, and the camera device takes a picture of the marker to record the position change of the marker.

[0008] In some embodiments, the change in position includes a change in six degrees of freedom.

[0009] In some embodiments, the marker is provided with a marking origin and at least one target point, and the distance between the target point and the marking origin is a known value.

[0010] A robot arm comprises at least two robot arm segments and a robot joint connected between the two robot arm segments, wherein each of the robot joints is provided with the robot joint deformation sensor based on machine vision as described above.

[0011] In some embodiments, the robotic joint further comprises a housing disposed around the first mounting portion and the second mounting portion.

[0012] In some embodiments, the robotic arm includes three robotic arm segments connected in sequence, and adjacent robotic arm segments are connected through the robot joints.

[0013] A method for determining the position of the end of a robotic arm after a collision, using the robotic arm as described above, the method comprising the following steps:

[0014] S1. Before the collision, the camera device photographs the marker to obtain a first image, and the controller obtains first position data of the marker according to the first image;

[0015] S2. The robot stops moving after the collision;

[0016] S3. The camera device photographs the marker to obtain a second image, and the controller obtains second position data of the marker according to the second image;

[0017] S4. Compare the second position data with the first position data to obtain a displacement change of the marker after the collision relative to that before the collision;

[0018] S5. Calculate and obtain the post-collision position data of the end of the robot arm according to the pre-stored length data of each robot arm segment and the displacement change of the marker in the robot joint connected to the robot arm segment.

[0019] In some embodiments, step S6 is also included. The post-collision position data is used as the position data of the end of the robotic arm to reset the robotic arm.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] In the robot joint deformation sensor based on machine vision of the present invention, a camera device and a marker are set in the robot joint. The image of the marker can be captured by the camera device, and the position change of the marker before and after the collision can be compared to determine the deformation data of the robot joint after the collision, and then determine the position after the collision.

[0022] In the method for determining the position of the robot arm and the end of the robot arm after a collision of the present invention, the above-mentioned robot joint deformation sensor based on machine vision is adopted. The deformation amount of the joint can be determined by relying on the image taken by the camera device. Combined with the pre-stored length data of the robot arm segment, the position of the end of the robot arm after the collision of the robot arm can be determined. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The technology of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0024] Figure 1 is a schematic diagram of a robot joint deformation sensor of the present invention installed in a robot joint;

[0025] Figure 2 It is a schematic diagram of the robot arm before and after the collision with the obstacle;

[0026] Figure 3 is a schematic diagram of a first image, a second image and a comparison between the first image and the second image taken by a camera device;

[0027] Figure 4 It is a step diagram of the method of the present invention.

[0028] Reference numerals:

[0029] 1-robot joint; 11-first mounting portion; 12-second mounting portion; 13-support portion; 14-housing;

[0030] 2- Camera device;

[0031] 3-marker; 31-mark origin; 32-target point;

[0032] 4-Robotic arm segment;

[0033] 5-Obstacles. DETAILED DESCRIPTION

[0034] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention, so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments in this application and the features in the embodiments can be combined with each other without conflict. The same reference numerals used throughout the drawings indicate the same or similar parts.

[0035] It should be noted that, unless otherwise specified, when a feature is referred to as being "fixed" or "connected" to another feature, it may be directly fixed or connected to the other feature, or it may be indirectly fixed or connected to the other feature. In addition, the descriptions of up, down, left, right, etc. used in the present invention are only relative to the relative positional relationship of the components of the present invention in the drawings.

[0036] Reference Figures 1 to 3 , a robot joint deformation sensor based on machine vision is installed in a robot joint 1. The robot joint 1 deformation sensor includes one or more camera devices 2 and one or more markers 3. In one embodiment, the camera device 2 and the marker 3 are both one. In another embodiment, the camera device 2 and the marker 3 are both three.

[0037] The robot joint 1 includes a first mounting portion 11 and a second mounting portion 12 arranged opposite to each other and a support portion 13 connecting and supporting the first mounting portion 11 and the second mounting portion 12. In one embodiment, the first mounting portion 11 and the second mounting portion 12 are two flanges of the robot joint 1, and the support portion 13 is a support shaft. It can be understood that the robot joint 1 can be a common joint or a flexible joint.

[0038] The camera device 2 is fixed on the side of the first mounting portion 11 facing the second mounting portion 12; the marker 3 is arranged on the side of the second mounting portion 12 facing the first mounting portion 11. That is, the camera device 2 and the marker 3 are both located inside the joint and arranged opposite to each other, so that the camera device 2 can take pictures of the marker 3.

[0039] The camera device 2 is electrically connected to the controller of the robot, and the camera device 2 takes a picture of the marker 3 to record the position change of the marker 3. If a collision occurs, the controller compares the pictures taken before and after the collision, and can obtain the position change of the marker 3 after the collision relative to before the collision. The offset and / or deflection of the marker 3 relative to the camera device 2 can be calculated based on the size change of the marker 3, and the final position data (including movement data and rotation data) can be obtained. For example, the controller obtains the data of the position of the marker 3 from the pictures taken before the collision, and after the collision, takes a picture to obtain the position of the marker 3 in the picture after the collision, and compares the two positions to obtain the offset, thereby determining the size of the deformation, thereby providing a calculation basis for determining the position of the robot arm after the collision.

[0040] The position change includes changes in six degrees of freedom. Since the collision occurs in three-dimensional space, various collisions may occur, and the deformation after the collision is also various. It is possible that the collision is offset or rotated in a single direction, or it may be offset and rotated in multiple directions.

[0041] The marker 3 is provided with a marking origin 31 ( Figure 3 a circle) and at least one target point 32 ( Figure 3 The distance between the target point 32 and the mark origin 31 is a known value. The translation in the plane direction can be determined by the position of the mark origin 31 before and after the collision, the change in the distance from the camera device 2 can be determined by the change in the overall size relationship, and the overall rotation can be determined by the change in the angle of the line between the mark origin 31 and the target point 32. By setting the mark image, the calculation of the position data change can be further facilitated. Among them, when the number of target points 32 is multiple, the shapes of the target points 32 can be all the same, all different, or partially the same.

[0042] A robotic arm comprises at least two robotic arm segments 4 and a robot joint 1 connected between the two robotic arm segments 4, wherein each of the robotic joints 1 is provided with a robot joint 1 deformation sensor based on machine vision as described above. Each joint on the robotic arm uses the above-mentioned robot joint 1 deformation sensor, so that after a collision, the deformation amount of the activities of multiple robot joint 1 deformation sensors is calculated and processed, thereby calculating the displacement of the entire robotic arm after the collision, thereby determining the position of the robotic arm after the collision. In one embodiment, the robotic arm has two robotic arm segments 4, and the robot joint 1 is between the two. In another embodiment, the robotic arm has three robotic arm segments 4, with a total of two robot joints 1.

[0043] In some embodiments, the robot joint 1 further includes a housing 14 disposed around the first mounting portion 11 and the second mounting portion 12 to protect the internal camera device 2 and the marker 3 to prevent the two from being damaged during a collision and causing failure. The housing 14 is tubular and the entire robot joint 1 is cylindrical.

[0044] In one embodiment, Figure 2 As shown, Figure 2 The middle dotted line part is the robot arm before it hits obstacle 5. Figure 2 The solid line portion is the robot arm segment 4 that is deformed and stopped after the collision. The robot arm includes three robot arm segments 4 connected in sequence. Adjacent robot arm segments 4 are connected by the robot joint 1. One end of the robot arm is fixed and the other end is the working end. After the collision, the position of the end of the robot arm (i.e., the working end) changes, and the angle relationship between adjacent robot arms also changes. By comparing the position of each joint before and after the collision, the angle change and offset can be calculated, so as to finally determine the position of the end of the robot arm.

[0045] A method for determining the position of the end of a robotic arm after a collision, using the robotic arm as described above, the method comprising the following steps:

[0046] S1. Before the collision, the camera device 2 photographs the marker 3 to obtain a first image, and the controller obtains the first position data of the marker 3 based on the first image. In actual use, it is difficult to predict when the collision will occur. Therefore, the camera device 2 can continue to shoot when the robot arm is running, and the photographs taken correspond to the time records. When a collision occurs, the robot joints are deformed, and the position data of the markers in the photographs taken are compared with the previous photographs. Based on this, it is determined that a collision has occurred, and an alarm is issued. Of course, the robot arm's own collision detection sensor (pressure or other forms of sensors) can also be relied upon to detect the collision and issue an alarm, using the photograph taken before the alarm time as the first image, such as Figure 3 Middle (a).

[0047] S2. The robot stops moving after the collision. After the collision, the robot joints are deformed, which is detected by the robot joint deformation sensor based on machine vision. The controller controls each driver to stop working to avoid further damage.

[0048] S3. The camera device 2 captures the marker 3 to obtain a second image, and the controller obtains second position data of the marker 3 based on the second image. During the collision and after the robot arm stops moving, the camera device 2 continues to capture photos, and data at different times during the entire joint deformation process can be obtained. By comparing the position data of the later photo with the previous photo, the movement process of the marker during deformation can be known. Figure 3 It can be understood that when the image captured later does not change any more, it means that the robot arm has stopped running, and the photo at this time can be used as the second image.

[0049] S4. Compare the second position data with the first position data, such as Figure 3 In (c), the displacement change of the marker 3 after the collision relative to that before the collision is obtained. The position data before and after the collision are compared to determine the position change.

[0050] S5. Calculate and obtain the post-collision position data of the end of the robot arm according to the pre-stored length data of each robot arm segment 4, combined with the displacement change of the marker 3 in the robot joint 1 connected to the robot arm segment 4. When the first joint is deformed, the length of the robot arm segment 4 connected to it will further amplify the deformation, so it is necessary to determine the overall offset data of the second joint behind the first joint after the deformation of the first joint in combination with the length of the robot arm segment 4, and then determine the offset of the third arm segment according to the deformation of the second joint, and finally obtain the position data of its end.

[0051] S6. Using the post-collision position data as the position data of the end of the robotic arm, the robotic arm is reset to perform subsequent work.

[0052] For other contents of the robot joint deformation sensor, mechanical arm and method based on machine vision described in the present invention, please refer to the prior art and will not be repeated here.

[0053] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Therefore, any modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the technical solution of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A robot joint deformation sensor based on machine vision, installed in a robot joint, the robot joint comprising a first mounting portion and a second mounting portion arranged opposite to each other and a supporting portion connecting and supporting the first mounting portion and the second mounting portion, characterized in that: The robot joint deformation sensor includes one or more camera devices and one or more markers; The camera device is fixed on a side of the first mounting portion facing the second mounting portion; the marker is arranged on a side of the second mounting portion facing the first mounting portion, the camera device is electrically connected to the controller of the robot, and the camera device takes a picture of the marker to record the position change of the marker; The robot joint deformation sensor based on machine vision is applied in a robot arm, the robot arm comprises at least two robot arm segments and a robot joint connected between the two robot arm segments, each of the robot joints is provided with the robot joint deformation sensor based on machine vision; The mechanical arm is applied to a method for determining the position of the end of the mechanical arm after a collision, the method comprising the following steps: S1. Before the collision, the camera device photographs the marker to obtain a first image, and the controller obtains first position data of the marker according to the first image; S2. The robot stops moving after the collision; S3. The camera device photographs the marker to obtain a second image, and the controller obtains second position data of the marker according to the second image; S4. Compare the second position data with the first position data to obtain a displacement change of the marker after the collision relative to that before the collision; S5. Calculate and obtain the post-collision position data of the end of the robot arm according to the pre-stored length data of each robot arm segment and the displacement change of the marker in the robot joint connected to the robot arm segment.

2. The robot joint deformation sensor based on machine vision according to claim 1, characterized in that: The position change includes changes in six degrees of freedom.

3. The robot joint deformation sensor based on machine vision according to claim 1, characterized in that: The marker is provided with a marking origin and at least one target point, and the distance between the target point and the marking origin is a known value.

4. The robot joint deformation sensor based on machine vision according to claim 1, characterized in that: The robot joint also includes a housing disposed around the first mounting portion and the second mounting portion.

5. The robot joint deformation sensor based on machine vision according to claim 1, characterized in that: The robotic arm comprises three robotic arm segments connected in sequence, and adjacent robotic arm segments are connected via the robot joints.

6. The robot joint deformation sensor based on machine vision according to claim 1, characterized in that: The determination method further includes step S6. using the post-collision position data as the position data of the end of the robotic arm to reset the robotic arm.

Images