End effector guidance system and method based on 2D camera and laser rangefinder

Abstract

This invention provides an end effector guidance system and method based on a 2D camera and a laser rangefinder, including a robot, a measurement system, and an end effector. Both the end effector and the measurement system are mounted on the robot's end effector. The measurement system acquires spatial information of the work plane, and the robot adjusts the pose of the end effector according to the position information, making the end effector's feed direction perpendicular to the work plane, and moves the end effector to the work plane to complete the work command. The system provided by this invention, based on a 2D camera and laser rangefinder to guide the end effector, can measure depth information and allows end effector operation on planes with different heights and inclinations.

Description

Technical Field

[0001] The present invention relates to the technical field of robot vision guidance, and in particular to an end effector guidance system and method based on a 2D camera and a laser rangefinder. Background Art

[0002] In the field of automated visual guidance, the methods for end effectors to implement operations are generally divided into repetitive operations of fixed workpieces and sensor recognition operations of non-fixed workpieces. Among them, the repetitive operations of fixed workpieces can achieve stable end effector operations by setting the path of the execution module. The accuracy of this method depends on the repeat positioning accuracy of the actuator itself, but if the workpiece is replaced, the path needs to be replanned to adapt to the operation requirements of different products; while the sensor recognition guidance end effector operation of non-fixed workpieces is to perform corresponding operations based on the characteristics of the operation location. The accuracy of this method is determined by the actuator and the sensor, and this method can perform end effector operations on different workpieces without changing the program.

[0003] Sensor recognition operations for non-fixed workpieces can use 2D cameras and 3D cameras depending on the sensor. If a 2D camera is used as a sensor, the distance between the workpiece and the 2D camera shooting point must remain consistent when the end effector is working, and the workpiece cannot be rotated except in the normal direction of the operated plane, otherwise the accuracy of the end effector operation cannot be guaranteed. At the same time, the height of the end effector guiding the workpiece must be fixed because the monocular camera cannot measure the height information. If a 3D camera is used as a sensor, it can meet the needs of the workpiece being placed in any position, but the price of the corresponding 3D camera will be more expensive due to the need to achieve a relatively high end effector operation accuracy. Summary of the invention

[0004] In view of the defects in the prior art, an object of the present invention is to provide an end effector guidance system and method based on a 2D camera and a laser rangefinder.

[0005] According to the present invention, an end effector guidance system based on a 2D camera and a laser rangefinder comprises: a robot, a measuring system and an end effector; the end effector and the measuring system are both installed at the end of the robot;

[0006] The measurement system acquires spatial information of the working plane, and the robot adjusts the posture of the end effector according to the position information, so that the feeding direction of the end effector is perpendicular to the working plane, and moves the end effector to the working plane to complete the working instruction.

[0007] Preferably, the measurement system includes a 2D camera and a laser rangefinder, and the method for acquiring the spatial information of the working plane is: collecting the two-dimensional plane information of three non-collinear points in the working plane through the 2D camera, acquiring the height information of the three points through the laser rangefinder, obtaining the three-dimensional coordinate information of the three points, and obtaining the spatial information of the working plane according to the three-dimensional coordinate information.

[0008] According to the present invention, an end effector guidance method based on a 2D camera and a laser rangefinder comprises:

[0009] Step S1: determining a reference plane of the end effector, and obtaining a feed direction vector OA of the end effector relative to the reference plane;

[0010] Step S2: determine the working plane of the end effector, obtain the coordinates of any three points in the working plane, and calculate the normal vector OA' of the working plane;

[0011] Step S3: Calculate the angle between the normal vector OA' of the working plane and the feed direction vector OA of the end effector ;

[0012] Step S4: According to the angle Adjust the feeding direction of the end effector to obtain the feeding direction vector of the end effector after adjustment;

[0013] Step S5: measuring the shooting height of the 2D camera, if it exceeds the reference height range, adjusting the shooting height of the 2D camera, and repeating steps S2 to S4 until the reference height requirement is met;

[0014] The reference height is custom-set according to the operation content;

[0015] Step S6: determine the feature points on the working plane, obtain the position information of the feature points, control the end effector to move to the feature points, and execute the working instructions.

[0016] Preferably, the reference plane is an initial calibration plane of the end effector, and in an original state, the feeding direction of the end effector is perpendicular to the reference plane.

[0017] Preferably, the coordinates of any three points in the working plane are obtained by photographing the working plane with a 2D camera to determine the two-dimensional coordinate information (x1, y1), (x2, y2) and (x3, y3) of the three points, and then obtaining the coordinate values ​​z1, z2, z3 of the three points in the Z direction based on the distance from the laser rangefinder to any reference point on the reference plane and the distance from the laser rangefinder to the three points in the working plane.

[0018] Preferably, the angle Methods for adjusting the feed direction of the end effector include:

[0019] The rotation of the end effector is represented by quaternion: , where q is the rotation quaternion, is the angle between OA' and OA, , and are the components of the vectors (a, b, c) orthogonal to OA' and OA, , and Is an imaginary unit.

[0020] Preferably, step S4 further includes: setting an angle range within which the end effector does not need to rotate.

[0021] Preferably, the method for measuring the shooting height of the 2D camera includes: selecting any point in the reference plane as a reference point, measuring the distance to the reference point by a laser rangefinder, and converting the distance from the 2D camera to the reference point based on the positional relationship between the laser rangefinder and the 2D camera, which is regarded as the distance from the 2D camera to the working plane.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] 1. Compared with the existing 2D camera solution, the system based on a 2D camera and a laser rangefinder to guide the end effector provided by the present invention can measure depth information and perform end effector operations on planes of different heights and different inclinations.

[0024] 2. Compared with the current solution using 3D cameras, the present invention provides a system based on 2D cameras and laser rangefinders to guide the end effector, which can significantly reduce the purchase cost of equipment under the same accuracy requirements.

[0025] 3. The present invention provides a method for guiding an end effector based on a 2D camera and a laser rangefinder, which can adapt to planes of different heights and inclinations and allow the end effector to operate precisely perpendicular to the operating plane. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0027] Figure 1 It is a schematic diagram of the framework of the control system of the present invention;

[0028] Figure 2 It is a schematic diagram of the control principle of the control system of the present invention;

[0029] Figure 3 It is a flow chart of the control method of the present invention.

[0030] Description of reference numerals:

[0031] DETAILED DESCRIPTION

[0032] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0033] The present invention provides an end effector 300 guidance system based on a 2D camera 201 and a laser rangefinder 202, referring to Figure 1 and Figure 2 As shown, it includes: a robot 100, a measuring system 200 and an end effector 300. The end effector 300 and the measuring system 200 are both installed at the end of the robot 100. The measuring system 200 includes a 2D camera 201 and a laser rangefinder 202. The three-dimensional coordinate information of the target point is obtained by measuring and calculating the 2D camera 201 and the laser rangefinder 202. The robot 100 controls the end effector 300 to complete the set operation instruction according to the three-dimensional coordinate information.

[0034] In a specific embodiment, the end effector 300 includes but is not limited to a dispensing machine, a welding machine, a hole making machine, etc.

[0035] The 2D camera 201 acts as a server. The 2D camera 201 communicates with the robot 100 via TCP / IP. The robot 100 sends a program execution instruction to the 2D camera 201. After the 2D camera 201 recognizes the feature point, it sends the two-dimensional coordinates of the feature point to the robot 100.

[0036] The laser rangefinder 202 acts as a server, and the robot 100 acts as a client to perform network communication via TCP / IP. The robot 100 sends a program execution instruction to the laser rangefinder 202, and the laser rangefinder 202 sends the depth data at this time to the robot 100.

[0037] The end effector 300 performs the final execution operation. The end effector 300 is controlled by the robot 100 through the DI / DO module. After the robot 100 sends a program execution signal, the end effector 300 starts the corresponding operation.

[0038] The working principle of the end effector 300 guidance system disclosed in the present invention is:

[0039] First, the 2D camera 201 identifies the feature point and sends its two-dimensional plane information to the robot 100, wherein the feature point is a point in the operation plane, and the feature point can be the operation starting point of the end effector 300 or the center point of the operation area. The two-dimensional plane information is the two-dimensional coordinate information of the feature point in the reference plane, and the reference plane is the initial calibration plane of the robot 100. In the initial stage, the feeding direction of the robot 100 is perpendicular to the reference plane.

[0040] Secondly, the TCP of the robot 100 is changed according to the inclination of the operation plane so that the feeding direction of the robot 100 is perpendicular to the operation plane. In a specific embodiment, the TCP of the robot 100 is the working end point at the end of the end effector 300. The height of the robot 100 is changed according to the height of the operation plane from the TCP;

[0041] Finally, the robot 100 controls the end effector 300 to move to the vicinity of the feature point according to the information sent by the 2D camera 201 and the laser rangefinder 202, and operates the end effector 300. The method provided by the present invention can adapt to the needs of high-precision positioning of the end effector 300 on different planes. The operating plane is the plane on which the end effector 300 operates, and there is a certain angle and height difference with the reference plane.

[0042] The present invention also provides a method for guiding an end effector 300 based on a 2D camera 201 and a laser rangefinder 202. The method is based on the above-mentioned end effector 300 guidance system based on a 2D camera 201 and a laser rangefinder 202, and is used to guide the end effector 300 to execute an operation instruction. Figure 3 As shown, the boot process includes the following steps:

[0043] Step S1: Determine a reference plane of the end effector 300 and obtain a feed direction vector OA of the end effector 300 relative to the reference plane.

[0044] Specifically, the reference plane is the initial calibration plane of the end effector 300. In the original state, the feeding direction of the end effector 300 is perpendicular to the reference plane. In the initial state, the feeding direction vector OA of the end effector 300 relative to the reference plane is (0, 0, 1)

[0045] Step S2: determine the working plane of the end effector 300, obtain the coordinates of any three points in the working plane, and calculate the normal vector OA' of the working plane;

[0046] In a specific embodiment, the coordinates of any three points in the working plane are obtained by photographing the working plane with a 2D camera 201 to determine the two-dimensional coordinate information (x1, y1), (x2, y2) and (x3, y3) of the three points, and then obtaining the coordinate values ​​z1, z2, z3 of the three points in the Z direction based on the distance from the laser rangefinder 202 to any reference point on the reference plane and the distance from the laser rangefinder 202 to the three points in the working plane.

[0047] Step S3: Calculate the angle between the normal vector OA' of the working plane and the feed direction vector OA of the end effector 300 ;

[0048] Step S4: According to the angle The feeding direction of the end effector 300 is adjusted to obtain the adjusted feeding direction vector of the end effector 300 .

[0049] The specific adjustment method is: use quaternion to represent the rotation, that is: , the current quaternion of the robot 100 (100) is multiplied by the rotation quaternion q, so that the end effector 300 can be rotated to a new end effector feed direction vector, where, is the rotation quaternion, is the angle between OA and OA', , and are the components of the vectors (a, b, c) orthogonal to OA and OA', , and Is an imaginary unit.

[0050] In a preferred embodiment, an angle range within which the end effector 300 does not need to rotate is set, and the smaller the angle range, the better the perpendicularity between the end effector 300 and the plane, but the number of rotations required for the end effector 300 will increase, and the larger the angle range, the worse the perpendicularity between the end effector 300 and the plane, but the number of rotations required for the end effector 300 will decrease.

[0051] Step S5: Use the laser rangefinder 202 to measure the shooting height of the 2D camera 201. If it exceeds the reference height range, adjust the height of the 2D camera 201 and repeat steps S2 to S4 until the angle between the normal vector OA' of the working plane and the feed direction vector OA of the end effector 300 is equal to The requirements are met and the adjusted height is within the reference height range.

[0052] The feature point is a point in the working plane, which can be the starting point of the end-operation operation or the center point of the working area. The reference height range is a range set artificially according to the actual working conditions.

[0053] When the 2D camera 201 is at different shooting heights, the camera's imaging angle of view is different, and the ratio of the obtained image changes according to the perspective principle. Therefore, after adjusting the posture of the end effector 300, it is necessary to measure the distance value between the 2D camera 201 and the working plane. If the distance value exceeds the reference height range, it is determined that there is an error between the posture of the end effector 300 at this time and the final ideal posture. At this time, after adjusting the distance from the 2D camera 201 to the working plane, re-measure the normal vector of the working plane to further calibrate the posture of the end effector 300.

[0054] In a specific embodiment, the method for measuring the distance value from the 2D camera 201 to the working plane is: select any point in the reference plane as a reference point, measure the distance to the reference point by the laser rangefinder 202, and based on the positional relationship between the laser rangefinder 202 and the 2D camera 201, calculate the distance from the 2D camera 201 to the reference point, which is regarded as the distance from the 2D camera 201 to the working plane.

[0055] Step S7: Acquire the coordinate information and height information of the feature point, control the end effector 300 to move to the feature point, and execute the work instruction.

[0056] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0057] In the description of the present application, it should be understood that the terms "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0058] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. An end effector guidance method based on a 2D camera and a laser rangefinder, characterized in that: include: Step S1: determining a reference plane of the end effector (300) and obtaining a feed direction vector OA of the end effector (300) relative to the reference plane; Step S2: determining the working plane of the end effector (300), obtaining the coordinates of any three points in the working plane, and calculating the normal vector OA' of the working plane; The spatial information of the working plane is determined by obtaining the three-dimensional coordinates of three non-collinear points in the working plane through a 2D camera (201) and a laser rangefinder (202); Step S3: Calculate the angle between the normal vector OA' of the working plane and the feed direction vector OA of the end effector (300) ; Step S4: According to the angle Adjusting the feeding direction of the end effector (300) to obtain the adjusted feeding direction vector of the end effector (300); Step S5: measuring the shooting height of the 2D camera (201); if the shooting height exceeds the reference height range, adjusting the shooting height of the 2D camera (201), and repeatedly executing steps S2 to S4 until the reference height requirement is met; The reference height is custom-set according to the operation content; Step S6: determining a feature point on the working plane, obtaining position information of the feature point, controlling the end effector (300) to move to the feature point, and executing the working instruction; The coordinates of any three points in the working plane are obtained by photographing the working plane with a 2D camera (201) to determine the two-dimensional coordinate information (x1, y1), (x2, y2) and (x3, y3) of the three points, and then obtaining the coordinate values ​​z1, z2 and z3 of the three points in the Z direction based on the distance from the laser rangefinder (202) to any reference point on the reference plane and the distance from the laser rangefinder (202) to the three points in the working plane.

2. The end effector guidance method based on a 2D camera and a laser rangefinder according to claim 1, characterized in that: The reference plane is an initial calibration plane of the end effector (300); in an original state, the feeding direction of the end effector (300) is perpendicular to the reference plane.

3. The end effector guidance method based on a 2D camera and a laser rangefinder according to claim 1, characterized in that: The angle The method for adjusting the feeding direction of the end effector (300) comprises: The rotation of the end effector (300) is represented by quaternion: , where q is the rotation quaternion, is the angle between OA' and OA, , and are the components of the vectors (a, b, c) orthogonal to OA' and OA, , and Is an imaginary unit.

4. The end effector guidance method based on a 2D camera and a laser rangefinder according to claim 1, characterized in that: Step S4 also includes: setting an angle range within which the end effector (300) does not need to rotate.

5. The end effector guidance method based on a 2D camera and a laser rangefinder according to claim 1, characterized in that: The method for measuring the shooting height of the 2D camera (201) comprises: selecting any point in a reference plane as a reference point, measuring the distance to the reference point by means of a laser rangefinder (202), and converting the distance from the 2D camera (201) to the reference point based on the positional relationship between the laser rangefinder (202) and the 2D camera (201) to obtain the distance as the distance from the 2D camera (201) to the working plane.

6. An end effector guidance system based on a 2D camera and a laser rangefinder, using the end effector guidance method based on a 2D camera and a laser rangefinder according to any one of claims 1 to 5, characterized in that: include: A robot (100), a measuring system (200) and an end effector (300); the end effector (300) and the measuring system (200) are both installed at the end of the robot (100); The measurement system (200) acquires spatial information of the working plane, and the robot (100) adjusts the position and posture of the end effector (300) according to the position information, so that the feeding direction of the end effector (300) is perpendicular to the working plane, and moves the end effector (300) to the working plane to complete the working instruction.

7. The end effector guidance system based on a 2D camera and a laser rangefinder according to claim 6, characterized in that: The measurement system (200) comprises a 2D camera (201) and a laser rangefinder (202), and the method for acquiring the spatial information of the working plane comprises: collecting two-dimensional plane information of three non-collinear points in the working plane by means of the 2D camera (201), acquiring height information of the three points by means of the laser rangefinder (202), obtaining three-dimensional coordinate information of the three points, and obtaining the spatial information of the working plane according to the three-dimensional coordinate information.

Images