A method for quickly restoring the working function of a three-dimensional vision sensor

Abstract

The present invention provides a method for quickly restoring the working function of a three-dimensional vision sensor. When the position of the sensor changes, the robot drives the three-dimensional vision sensor to a specified position, collects a point cloud on the target surface, and records it as a measured point cloud. The target includes at least three planes, and any two planes are not parallel to each other. The measured point cloud and the original point cloud are aligned to obtain a rotation and translation matrix between the sensor before the position change and the sensor after the position change. The rotation and translation matrix is ​​compensated to the hand-eye relationship between the three-dimensional vision sensor and the robot end, and then the hand-eye relationship after the sensor position change is obtained, so as to realize the restoration of the working function of the three-dimensional vision sensor. The method does not need to find and record a standard trajectory, and uses a special three-dimensional target to reflect the position and posture changes of the sensor in the six degrees of freedom directions. By obtaining the point cloud information of the target, the conversion relationship before and after the sensor position change is solved, so as to realize the rapid restoration of the three-dimensional vision sensor function.

Description

Technical Field

[0001] The invention relates to the field of visual measurement, and in particular to a method for quickly restoring the working function of a three-dimensional visual sensor. Background Art

[0002] Robot vision refers to a system that enables robots to have visual perception functions. Usually, the visual sensor is fixed on the end flange of the robot; the hand-eye relationship between the visual sensor and the end flange of the robot is obtained through hand-eye calibration. The information of the object to be measured obtained by the visual sensor is transferred to the robot base coordinate system through the hand-eye relationship and the robot posture to complete the positioning of the object to be measured, and the robot is guided to complete the automated operation based on the positioning. Therefore, the hand-eye relationship is an indispensable part of this type of application. During the working process, the robot works along a highly repetitive motion trajectory. During the entire motion trajectory, it is necessary to ensure that the relative position relationship between the visual sensor and the end of the robot remains unchanged; due to the complex industrial site environment, the position of the visual sensor will move due to collisions, equipment maintenance, etc., causing the relative position of the visual sensor and the end of the robot to change, that is, the hand-eye relationship changes. The traditional method is to perform hand-eye calibration on the visual sensor after the position change, and redo the record of the standard trajectory for the robot measurement trajectory. This process requires continuous attempts and verifications of the robot's measurement trajectory until the robot's measurement trajectory is complete and correct, and then saves and records the position information of the standard trajectory in the visual sensor coordinate system. On an automated production line, a blocked workstation will affect the output of the entire line. Existing methods have seriously slowed down the progress of industrial production. Summary of the invention

[0003] In order to solve the above technical problems, the present invention provides a method for quickly restoring the working function of a three-dimensional vision sensor. The method proposes a special three-dimensional target according to the working characteristics of the three-dimensional vision sensor. The target can reflect the position change of the sensor in the six degrees of freedom. By acquiring the point cloud information of the target, the conversion relationship before and after the sensor position change is calculated, and then it is added to the original hand-eye relationship, so as to realize the rapid recovery of the three-dimensional vision sensor function.

[0004] On the other hand, this method also sets marker points on the target to reflect whether the sensor has undergone a large position change by the coordinate change of the marker points, guides coarse adjustment to restore to the approximate position, and then uses the target point cloud to repair the hand-eye relationship, thereby improving system accuracy and achieving rapid hand-eye relationship recovery; the hand-eye relationship that originally took 60 minutes to recover is controlled within 10 minutes, thereby improving work efficiency.

[0005] A method for quickly restoring the working function of a three-dimensional vision sensor, wherein the three-dimensional vision sensor is installed at the end of a robot, and the hand-eye relationship between the two has been calibrated;

[0006] When the sensor position changes, first determine whether coarse adjustment is needed based on the degree of position change of the sensor, so that the position change is within the allowable range; then quickly restore the sensor function through the following steps:

[0007] 1) The robot drives the 3D vision sensor to the specified position and collects the point cloud of the target surface, which is recorded as the measured point cloud;

[0008] The target is fixed in the workstation and fixed relative to the robot, and includes at least three planes, any two of which are not parallel to each other;

[0009] 2) Perform point cloud registration on the measured point cloud and the original point cloud to obtain the rotation and translation matrix between the sensor before and after the position change. c T s ;

[0010] The original point cloud is: before the sensor position is changed, the robot drives the three-dimensional vision sensor to run to the specified position and obtains the point cloud from the target surface;

[0011] 3) Use the following formula to obtain the hand-eye relationship between the 3D vision sensor and the robot end after the position change: f T c :

[0012] f T c = f T s ·[ c T s ] -1

[0013] f T s Indicates the hand-eye relationship between the 3D vision sensor and the robot end before the position change;

[0014] Realize the restoration of the working function of the three-dimensional vision sensor.

[0015] Furthermore, the target morphology is presented in the first or second manner:

[0016] Mode 1: The target is concave, including a polygonal bottom surface and side planes connected to each side of the polygonal bottom surface; the angle between a single side plane and the bottom surface is 110° to 160°;

[0017] Mode 2: The target includes a polygonal bottom surface and side planes respectively connected to at least two sides of the polygonal bottom surface.

[0018] Further, the target is any one of the following forms:

[0019] Form 1: The target includes a rectangular bottom surface, side planes connected to four sides of the rectangular bottom surface respectively, the angles between each side plane and the bottom surface are consistent, and the target is concave as a whole;

[0020] Form 2: The target includes a triangular base and side planes connected to any two sides of the triangular base;

[0021] Form 3: The target includes a parallelogram bottom surface and side planes connected to any two sides of the parallelogram bottom surface;

[0022] Form 4: The target includes a rectangular bottom surface and side planes connected to any three sides of the rectangular bottom surface;

[0023] Form 5: The target includes a pentagonal bottom surface and side planes connected to any three sides of the pentagonal bottom surface.

[0024] Preferably, the following method is used to determine whether the sensor position needs to be roughly adjusted:

[0025] Set at least 3 non-collinear markers on the target;

[0026] Before step 1), perform the following steps:

[0027] The two-dimensional coordinates of the mark point obtained by the sensor during normal operation are recorded as the original two-dimensional coordinates, and the two-dimensional coordinates of the mark point obtained by the sensor after the position change are recorded as the measured two-dimensional coordinates;

[0028] Compare the original two-dimensional coordinates and the measured two-dimensional coordinates of a single marker point to obtain the coordinate difference;

[0029] Obtain the coordinate difference of each mark point, and determine whether the maximum value of the difference exceeds the preset threshold A. If not, the sensor position does not need to be roughly adjusted, and directly proceed to step 1);

[0030] If so, it is necessary to roughly adjust the sensor position first so that the maximum value of the coordinate difference between the two-dimensional coordinate mark of each marker point obtained by the adjusted sensor and the original two-dimensional coordinate does not exceed the preset threshold A.

[0031] Furthermore, the acquisition process of the original point cloud is as follows:

[0032] After the sensor and robot complete the hand-eye calibration, first teach the robot trajectory so that it can drive the sensor to collect the complete target image, and record this position as the designated position;

[0033] At the specified position, the sensor obtains the three-dimensional point cloud of the target and records it as the original point cloud.

[0034] Furthermore, the sensor is a three-dimensional scanning vision sensor or a multi-line structured light vision sensor, which can obtain a point cloud of the surface of the object being measured at one time.

[0035] Preferably, the point cloud registration method in step 2) is an ICP registration method.

[0036] Advantages of this method: This method does not require redoing the standard motion trajectory, saving the time required for the original method to repeatedly verify the robot measurement trajectory and find and record the standard trajectory, effectively improving work efficiency.

[0037] When this method is used in the measurement process of the automobile production line, the time consumption comparison table of different workstations is as follows:

[0038]

[0039] It can be seen from the table that the time to restore a single sensor is greatly reduced with this method compared to the existing technology. At the same time, for workstations with a large number of sensors and many types of vehicles to be tested, the adjustment time will increase exponentially because the traditional method also requires teaching the robot trajectory and recalibrating the hand-eye relationship. At this time, the advantages of this method are more significant. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1a to Figure 1e Schematic diagram of various forms of target structures in specific implementation methods. DETAILED DESCRIPTION

[0041] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0042] A method for quickly restoring the working function of a three-dimensional vision sensor, wherein the three-dimensional vision sensor is installed at the end of a robot, and the hand-eye relationship between the two has been calibrated;

[0043] When the sensor position changes, first determine whether coarse adjustment is needed based on the degree of position change of the sensor, so that the position change is within the allowable range; then quickly restore the sensor function through the following steps:

[0044] 1) The robot drives the 3D vision sensor to the specified position and collects the point cloud of the target surface, which is recorded as the measured point cloud;

[0045] The target is fixed in the workstation and fixed relative to the robot, and includes at least three planes, and any two planes are not parallel to each other;

[0046] 2) Perform point cloud registration on the measured point cloud and the original point cloud to obtain the rotation and translation matrix between the sensor before and after the position change. c T s ; In this embodiment, the point cloud registration method is the ICP registration method;

[0047] The original point cloud is: before the sensor position changes, the robot drives the 3D vision sensor to run to the specified position and obtains the point cloud from the target surface;

[0048] 3) Use the following formula to obtain the hand-eye relationship between the 3D vision sensor and the robot end after the position change: f T c :

[0049] f T c = f T s ·[ c T s ] -1

[0050] f T s Indicates the hand-eye relationship between the 3D vision sensor and the robot end before the position change;

[0051] Realize the restoration of the working function of the three-dimensional vision sensor.

[0052] Among them, the target morphology is presented in mode 1 or mode 2:

[0053] Method 1: The target is concave, including a polygonal bottom surface and side planes connected to each side of the polygonal bottom surface; the angle between a single side plane and the bottom surface is 110° to 160°;

[0054] Mode 2: The target includes a polygonal bottom surface and side planes respectively connected to at least two sides of the polygonal bottom surface.

[0055] In specific implementation, the target is any of the following forms:

[0056] Form 1: Figure 1a As shown, the target includes a rectangular bottom surface, side planes connected to the four sides of the rectangular bottom surface respectively, the angles between each side plane and the bottom surface are consistent, and the target is concave as a whole;

[0057] Form 2: Figure 1b , Figure 1d As shown, the target includes a triangular base and side planes respectively connected to any two sides of the triangular base;

[0058] Form 3: Figure 1c As shown, the target includes a parallelogram bottom surface and side planes respectively connected to any two sides of the parallelogram bottom surface;

[0059] Form 4: Figure 1e As shown, the target includes a rectangular bottom surface and side planes respectively connected to any three sides of the rectangular bottom surface;

[0060] Form 5: The target includes a pentagonal base and side planes connected to any three sides of the pentagonal base.

[0061] As a preferred embodiment of the present invention, the following method is used to determine whether the sensor position needs to be roughly adjusted:

[0062] Set at least 3 non-collinear markers on the target;

[0063] Before step 1), perform the following steps:

[0064] The two-dimensional coordinates of the mark point obtained by the sensor during normal operation (the sensor before the position change or after the function is restored) are recorded as the original two-dimensional coordinates, and the two-dimensional coordinates of the mark point obtained by the sensor after the position change are recorded as the measured two-dimensional coordinates;

[0065] Compare the original two-dimensional coordinates and the measured two-dimensional coordinates of a single marker point to obtain the coordinate difference;

[0066] Obtain the coordinate difference of each mark point, and determine whether the maximum value of the difference exceeds the preset threshold A. If not, the sensor position does not need to be roughly adjusted, and directly proceed to step 1);

[0067] If so, it is necessary to roughly adjust the sensor position first so that the maximum value of the coordinate difference between the two-dimensional coordinate mark of each marker point obtained by the adjusted sensor and the original two-dimensional coordinate does not exceed the preset threshold A.

[0068] Specifically, the process of obtaining the original point cloud is as follows:

[0069] After the sensor and robot complete the hand-eye calibration, first teach the robot trajectory so that it can drive the sensor to collect the complete target image, and record this position as the designated position;

[0070] At the specified position, the sensor obtains the three-dimensional point cloud of the target and records it as the original point cloud.

[0071] Among them, the sensor is a three-dimensional scanning vision sensor or a multi-line structured light vision sensor, which can obtain the point cloud of the surface of the object being measured at one time.

[0072] Taking the four-door visual guidance installation station of a car as an example, the specific implementation process is as follows:

[0073] After the sensor and robot complete the hand-eye calibration, first teach the robot trajectory so that it can drive the sensor to collect the complete target image and record the position as the designated position; select Figure 1a The target of form 1 shown has three non-collinear marking points set on the bottom surface of the target;

[0074] At the specified position, the sensor obtains the three-dimensional point cloud of the target and records it as the original point cloud. It also collects a two-dimensional image of the target and obtains the two-dimensional coordinates of the marker points, which are recorded as the original two-dimensional coordinates:

[0075] {(530.54,546.99),(1530.27,548.87),(1029.01,1230.15)};

[0076] When the position of the sensor changes, the two-dimensional coordinates of the marker point are obtained again using the sensor after the position change, which are recorded as the measured two-dimensional coordinates {(505.24, 520.66), (1505.77, 522.78), (1001.23, 1200.26)};

[0077] Compare the original two-dimensional coordinates and the measured two-dimensional coordinates of a single marker point to obtain the coordinate difference;

[0078] Obtain the coordinate difference of each mark point, and determine whether the maximum difference value 40.806piexl exceeds the preset threshold A. If not, the sensor position does not need to be roughly adjusted, and step 1) is directly performed;

[0079] If so, it is necessary to roughly adjust the sensor position first so that the maximum value of the coordinate difference between the two-dimensional coordinate mark of each marker point obtained by the adjusted sensor and the original two-dimensional coordinate does not exceed the preset threshold A.

[0080] The preset threshold value A is set according to the imaging position of the object to be measured in the sensor during actual operation of the sensor. The preset threshold value A needs to ensure that the adjusted sensor can normally collect information of the object to be measured.

[0081] For example: In the actual operation of the sensor, the surface feature points of the object being measured collected are the points to be solved;

[0082] In the point cloud image, find the horizontal distance and vertical distance between the point to be solved and the edge of the image, and record them as the horizontal component and vertical component of the preset threshold A respectively.

[0083] Then quickly restore the sensor function through the following steps:

[0084] 1) The robot drives the 3D vision sensor to the specified position and collects the point cloud of the target surface, which is recorded as the measured point cloud;

[0085] The target is fixed in the workstation and its relative position to the robot is fixed;

[0086] 2) Perform point cloud registration on the measured point cloud and the original point cloud to obtain the rotation and translation matrix between the sensor before and after the position change. c T s ; In this embodiment, the point cloud registration method is the ICP registration method;

[0087] The original point cloud is: before the sensor position changes, the robot drives the 3D vision sensor to run to the specified position and obtains the point cloud from the target surface;

[0088] 3) Use the following formula to obtain the hand-eye relationship between the 3D vision sensor and the robot end after the position change: f T c :

[0089] f T c = f T s ·[ c T s ] -1

[0090] f T s Indicates the hand-eye relationship between the 3D vision sensor and the robot end before the position change;

[0091] Realize the restoration of the working function of the three-dimensional vision sensor.

[0092] This method does not need to redo the standard motion trajectory, which saves the time required for the original method to repeatedly verify the robot measurement trajectory and find and record the standard trajectory, effectively improving work efficiency.

[0093] The foregoing description of the specific exemplary embodiments of the present invention is presented for the purpose of illustration and description. The foregoing description is not intended to be exhaustive, nor is it intended to limit the present invention to the precise form disclosed, and it is apparent that many changes and variations are possible in light of the above teachings. The exemplary embodiments are selected and described to explain the specific principles of the present invention and its practical application, so that other technicians in the field can realize and utilize the various exemplary embodiments of the present invention and its different selected forms and modified forms. The scope of the present invention is intended to be limited by the appended claims and their equivalents.

Claims

1. A method for quickly restoring the working function of a three-dimensional vision sensor, characterized in that: The three-dimensional visual sensor is installed at the end of the robot, and the hand-eye relationship between the two has been calibrated; When the sensor position changes, first determine whether coarse adjustment is needed based on the degree of position change of the sensor, so that the position change is within the allowable range; then quickly restore the sensor function through the following steps: 1) The robot drives the 3D vision sensor to the specified position and collects the point cloud of the target surface, which is recorded as the measured point cloud; The target is fixed in the workstation and fixed relative to the robot, and includes at least three planes, any two of which are not parallel to each other; 2) Perform point cloud registration on the measured point cloud and the original point cloud to obtain the rotation and translation matrix between the sensor before and after the position change. c T s ; The original point cloud is: before the sensor position is changed, the robot drives the three-dimensional vision sensor to run to the specified position and obtains the point cloud from the target surface; 3) Use the following formula to obtain the hand-eye relationship between the 3D vision sensor and the robot end after the position change: f T c : f T c = f T s ·[ c T s ] -1 f T s Indicates the hand-eye relationship between the 3D vision sensor and the robot end before the position change; Realize the restoration of the working function of the three-dimensional vision sensor.

2. The method according to claim 1, characterized in that The target morphology is presented in the first or second manner: Mode 1: The target is concave, including a polygonal bottom surface and side planes connected to each side of the polygonal bottom surface; the angle between a single side plane and the bottom surface is 110° to 160°; Mode 2: The target includes a polygonal bottom surface and side planes respectively connected to at least two sides of the polygonal bottom surface.

3. The method according to claim 1, characterized in that The target is any of the following forms: Form 1: The target includes a rectangular bottom surface, side planes connected to four sides of the rectangular bottom surface respectively, the angles between each side plane and the bottom surface are consistent, and the target is concave as a whole; Form 2: The target includes a triangular base and side planes connected to any two sides of the triangular base; Form 3: The target includes a parallelogram bottom surface and side planes connected to any two sides of the parallelogram bottom surface; Form 4: The target includes a rectangular bottom surface and side planes connected to any three sides of the rectangular bottom surface; Form 5: The target includes a pentagonal bottom surface and side planes connected to any three sides of the pentagonal bottom surface.

4. The method according to claim 1, characterized in that: Use the following methods to determine whether the sensor position needs to be roughly adjusted: Set at least 3 non-collinear markers on the target; Before step 1), perform the following steps: The two-dimensional coordinates of the mark point obtained by the sensor during normal operation are recorded as the original two-dimensional coordinates, and the two-dimensional coordinates of the mark point obtained by the sensor after the position change are recorded as the measured two-dimensional coordinates; Compare the original two-dimensional coordinates and the measured two-dimensional coordinates of a single marker point to obtain the coordinate difference; Obtain the coordinate difference of each mark point, and determine whether the maximum value of the difference exceeds the preset threshold A. If not, the sensor position does not need to be roughly adjusted, and directly proceed to step 1); If so, it is necessary to roughly adjust the sensor position first so that the maximum value of the coordinate difference between the two-dimensional coordinate mark of each marker point obtained by the adjusted sensor and the original two-dimensional coordinate does not exceed the preset threshold A.

5. The method according to claim 1, characterized in that: The acquisition process of the original point cloud is as follows: After the sensor and robot complete the hand-eye calibration, first teach the robot trajectory so that it can drive the sensor to collect the complete target image, and record this position as the designated position; At the specified position, the sensor obtains the three-dimensional point cloud of the target and records it as the original point cloud.

6. The method according to claim 1, characterized in that: The sensor is a three-dimensional scanning vision sensor or a multi-line structured light vision sensor, which can obtain a point cloud of the surface of the object being measured at one time.

7. The method according to claim 1, characterized in that: The point cloud registration method in step 2) is the ICP registration method.

Images