A dual-arm collaborative assembly method with double calibration based on vision
Through the coordinated operation of the binocular camera and the robotic arm, two calibrations are performed to solve the blind spots and mechanical structure occlusion problems of the single-camera vision system, achieving high-precision assembly alignment and is suitable for assembly tasks in complex environments.
Patent Information
- Application Number
- CN202110946161.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-18
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-08-18
AI Technical Summary
The single-camera vision system has blind spots in the field of view or mechanical structure occlusion problems after approaching the object to be assembled, resulting in low assembly accuracy; it is difficult for the multi-camera vision system to find the observation position for high-precision assembly.
Using a two-arm collaborative assembly method based on vision, the binocular camera is used for initialization setting, image preprocessing and corner point detection, and the robotic arm posture is adjusted to parallel the center axis of the jaw and the center axis of the target to be assembled, and through Calibration is performed twice from the position adjustment of the robot arm to ensure high-precision alignment.
It realizes the positioning of the object to be assembled under initial conditions, effectively helping the main robot arm to align the center axis of the jaw and the center axis of the object to be assembled, suitable for high-precision, fast and robust assembly tasks in complex environments .
Smart Images

Figure CN113804106B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of robot vision, and in particular relates to a double-arm collaborative assembly method based on vision and double calibration. Background Art
[0002] In industrial production, assembly costs account for more than 50% of the total cost. With the development of industrial robots, robots are expected to replace manual labor to complete assembly tasks. However, there are many types of parts in industrial production, which are small and difficult to locate. The application of robots in precision assembly work is a challenge. Take the example of the precise clamping of screws by the clamp. When the center axis of the clamp is not aligned with the center axis of the bolt, the clamp may cause assembly failure when applying force, and the assembled workpiece or the robot may be damaged.
[0003] At present, in simple application scenarios, the initial and target poses of the workpiece are predetermined, and the robot arm only performs fixed operations through the teaching system or according to fixed procedures. However, in most real application scenarios, the initial or target poses of the workpiece are not strictly fixed. In this case, vision-guided positioning is an ideal solution. The robot arm senses the working environment through the vision system and obtains the initial pose of the workpiece. The existing single-camera vision system has blind spots in the field of view or occlusion problems of its own mechanical structure after approaching the object to be assembled, resulting in low assembly accuracy; multi-camera vision systems are difficult to find observation positions suitable for high-precision assembly. Summary of the invention
[0004] The purpose of the present invention is to provide a dual-arm collaborative assembly method with two calibrations based on vision, so as to solve the problem that a single-camera vision system has a blind spot in the field of vision or is blocked by its own mechanical structure after approaching the object to be assembled, resulting in low assembly accuracy; and the problem that a multi-camera vision system is difficult to find an observation position suitable for high-precision assembly, so as to achieve high-precision calibration of the assembly target.
[0005] The technical solution to achieve the purpose of the present invention is: a dual-arm collaborative assembly method based on double calibration of vision, comprising the following steps:
[0006] Step 1: Initialize and set the binocular cameras on the mechanical arms of the dual-arm robot respectively;
[0007] Step 2: Use a binocular camera to collect images, preprocess and detect corner points of the images, extract feature points of the target to be assembled, and determine the orientation of the target to be assembled;
[0008] Step 3: Adjust the posture of the main robot arm so that the center axis of the gripper is parallel to the center axis of the object to be assembled;
[0009] Step 4: Adjust the main robotic arm so that it is close to the object to be assembled until it enters the blind spot of the main camera or the information of the target object is significantly reduced;
[0010] Step 5: Adjust the slave robot arm to scan half a circle in a counterclockwise or clockwise direction with the center point of the target to be assembled as the center of the circle, and collect real-time image data stream to obtain grayscale information of the target to be assembled;
[0011] Step 6: Select the slave robot arm posture with the smallest grayscale change in the target frame corresponding to the target to be assembled for image acquisition, and adjust the master robot arm gripper according to the viewing angle to complete the left-right direction calibration;
[0012] Step 7: Rotate the robot arm clockwise or counterclockwise for 1 / 4 circle with the center point of the target object as the center, and observe from the side to complete the calibration of the front-to-back direction and vertical angle.
[0013] Compared with the prior art, the present invention has the following significant advantages:
[0014] (1) The present invention uses two binocular cameras, which can obtain the position and posture of the object to be assembled with high accuracy under initial conditions before the first calibration;
[0015] (2) The two-time calibration method of the present invention can effectively help the main robot arm align the center axis of the gripper with the center axis of the object to be assembled;
[0016] (3) The present invention is applied to assist the main robot arm in completing the assembly task by adjusting the posture of the slave robot arm and performing two calibrations when the target object is in a dangerous environment and it is difficult for humans to perform teaching operations;
[0017] (4) The present invention is applied to complex environments. When the target object is blocked, deformed, or changes in scale, the present invention can still automatically adjust the robot arm to a posture suitable for observation to obtain visual information, and has high precision, rapidity, and robustness.
[0018] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 It is a flow chart of the dual-arm collaborative assembly method with double calibration based on vision of the present invention. DETAILED DESCRIPTION
[0020] A dual-arm collaborative assembly method based on vision and double calibration, comprising the following steps:
[0021] Step 1: Initialize the binocular cameras on the dual-arm robot's arms, specifically:
[0022] The initial working position of the binocular camera is set so that the optical axes of the two binocular cameras are parallel and perpendicular to the line connecting the optical centers of the two binocular cameras. The transfer matrix from the binocular camera to the end of the robotic arm equipped with the binocular camera, the transformation matrix from the end of the robotic arm to the base, and the intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficient of the two cameras are calibrated respectively.
[0023] Step 2: Use a binocular camera to collect images, preprocess and detect corners of the images, extract feature points of the target to be assembled, and determine the orientation of the target to be assembled, specifically:
[0024] Step 2-1: Use the binocular cameras on the master and slave robotic arms to collect images of the target to be assembled;
[0025] Step 2-2: Using the intrinsic parameter matrix and extrinsic parameter matrix obtained in step 1 and the distortion coefficient, based on the co-polarity constraint in epipolar geometry, correct the image pair captured by the master and slave cameras in step 2-1;
[0026] Step 2-3: Based on the Harris operator, perform corner detection on the image taken by the binocular camera of the robot arm after correction in step 2-2, extract the six feature points of the target to be assembled, namely u1, u2, u3, u4, u5, and u6, and use the center coordinates of the six feature points as the center coordinates u0 of the target to be assembled:
[0027]
[0028] Step 2-4: Based on the NCC (Normalized Cross Correlation) matching algorithm, the disparity of the image pair corrected in step 2-3 is calculated on the basis of the alignment of the two views, and a disparity map is obtained;
[0029] Step 2-5, using the image coordinates obtained in step 2-3 to find the disparity value of the corresponding point in the disparity map obtained in step 2-4, based on the principle of triangulation, determine the depth information, and based on the depth value, obtain the six feature points of the target to be assembled, the world coordinates of the center point of the target to be assembled, and the grayscale average value Δ0 in the target frame corresponding to the target to be assembled;
[0030] Step 2-6: Define the plane of the robot base as a horizontal plane, select three feature points with the smallest horizontal coordinates to determine the normal vector of the target to be assembled, and determine the orientation of the target to be assembled.
[0031] Step 3: Adjust the posture of the main robot arm so that the center axis of the gripper is parallel to the center axis of the object to be assembled. Specifically:
[0032] According to the normal vector of the target to be assembled and the center point of the target to be assembled obtained in step 2, the center axis of the target to be assembled is determined, and the posture of the main robot arm is adjusted so that the center axis of the gripper is parallel to the center axis of the target to be assembled.
[0033] Step 4: Adjust the main robot arm to get close to the object to be assembled until it enters the blind spot of the main camera or the target object information is significantly reduced. Specifically:
[0034] Step 4-1: Adjust the main robotic arm to approach the target to be assembled, and at the same time, the binocular camera on the main robotic arm scans the target to be assembled;
[0035] Step 4-2: When the number of feature points in the image changes by Δn≥1 between the two frames before and after, it is considered that the information of the target to be assembled is significantly reduced and enters the blind spot of the main camera’s field of view;
[0036] Step 4-3: Obtain the distance d between the binocular camera on the main robot arm and the central axis of the target to be assembled.
[0037] Step 5: Adjust the slave robot arm, take the center point of the target to be assembled as the center of the circle, scan half a circle in a counterclockwise or clockwise direction (counterclockwise when the slave robot arm is the left arm, clockwise when the slave robot arm is the right arm), and collect real-time image data stream to obtain the grayscale information of the target to be assembled, specifically:
[0038] Step 5-1: Adjust the position of the slave robot arm so that the optical axis of the binocular camera on the slave robot arm coincides with the intersection line L of the plane where the gripper is located and the plane where the object to be assembled is located, and the binocular camera on the slave robot arm faces the operating area. Specifically:
[0039] Step 5-1-1: Adjust the position of the slave robot arm so that the center point of the binocular camera on the robot arm reaches the intersection line L of the plane where the gripper is located and the plane where the object to be assembled is located;
[0040] Step 5-1-2: Adjust the posture of the slave robot arm so that the optical axis of the binocular camera on the slave robot arm coincides with the intersection line L of the plane where the gripper is located and the plane where the object to be assembled is located, and the slave camera is facing the operating area.
[0041] Step 5-2, move the slave robot arm along the intersection line L close to the center point of the target to be assembled, until the distance between the binocular camera on the slave robot arm and the center point of the target to be assembled is 2d;
[0042] Step 5-3: The robot arm scans half a circle with the center point of the target to be assembled as the center. At the same time, the real-time image data stream is collected from the binocular camera on the robot arm. The image is pre-processed, i.e., Gaussian smoothing is performed to eliminate the noise in the local area and then the grayscale information of the target object is extracted.
[0043] Step 6: Select the slave robot arm posture with the smallest grayscale change in the target frame corresponding to the target to be assembled for image acquisition, and adjust the master robot arm gripper according to the viewing angle to complete the left and right direction calibration, specifically:
[0044] Step 6-1, according to the gray value extracted in step 5, calculate the gray average value Δi in the target frame corresponding to each posture, i=1, 2, 3...10t, t is the scanning time, and the difference between the gray average value |Δi-Δ0| in the target frame corresponding to the initial posture, and take the posture with the smallest absolute value of the difference as the best posture B;
[0045] Step 6-2, move the robot arm to the best posture B recorded in step 6-1;
[0046] Step 6-3: Use the binocular camera image acquisition on the robot arm to obtain the two-dimensional information of the gripper and the object to be assembled in posture B, and move the gripper horizontally until the projection of the center axis of the gripper in the two-dimensional plane obtained from this perspective coincides with the center axis of the object to be assembled, completing the left-right calibration.
[0047] Step 7: The robot arm will rotate 1 / 4 circle clockwise or counterclockwise with the center point of the target object as the center to complete the calibration of the front and back directions and vertical angles. Specifically:
[0048] Step 7-1, the robot arm is rotated 1 / 4 circle clockwise or counterclockwise (clockwise when the robot arm is the left arm, counterclockwise when the robot arm is the right arm) with the center point u0 of the target to be assembled as the center of the circle, and the image is collected from the side;
[0049] Step 7-2, adjust the gripper posture so that the projection of the gripper center axis at this viewing angle is parallel to the center axis of the target object, completing the vertical angle calibration;
[0050] Step 7-3: Move the gripper horizontally so that the center axis of the gripper coincides with the center axis of the target object to complete the calibration in the front-to-back direction.
[0051] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0052] Example
[0053] like Figure 1 As shown, a dual-arm collaborative assembly method based on vision and double calibration includes the following steps:
[0054] Step 1: Initialize the binocular cameras on the dual-arm robot's arms, specifically:
[0055] The initial working position of the binocular camera is set so that the optical axes of the two binocular cameras are parallel and perpendicular to the line connecting the optical centers of the two binocular cameras. The transfer matrix from the binocular camera to the end of the robotic arm equipped with the binocular camera, the transformation matrix from the end of the robotic arm to the base, and the intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficient of the two cameras are calibrated respectively.
[0056] Step 2: Use a binocular camera to collect images, preprocess and detect corner points of the images, extract the feature points of the bolts to be assembled, and determine the orientation of the bolts to be assembled, specifically:
[0057] Step 2-1: Use the binocular cameras on the master and slave robotic arms to collect images of the bolts to be assembled;
[0058] Step 2-2: Using the intrinsic parameter matrix and extrinsic parameter matrix obtained in step 1 and the distortion coefficient, based on the co-polarity constraint in epipolar geometry, correct the image pair captured by the master and slave cameras in step 2-1;
[0059] Step 2-3: Based on the Harris operator, perform corner detection on the image taken by the binocular camera of the robot arm after correction in step 2-2, and extract the six feature points of the bolt to be assembled, namely u1, u2, u3, u4, u5, and u6. Use the center coordinates of the six feature points as the center coordinates u0 of the bolt to be assembled:
[0060]
[0061] Step 2-4: Based on the NCC (Normalized Cross Correlation) matching algorithm, the disparity of the image pair corrected in step 2-3 is calculated on the basis of the alignment of the two views, and a disparity map is obtained;
[0062] Step 2-5, using the image coordinates obtained in step 2-3 to find the disparity value of the corresponding point in the disparity map obtained in step 2-4, based on the principle of triangulation, determine the depth information, and based on the depth value, obtain the six feature points of the bolt to be assembled, the world coordinates of the center point of the bolt to be assembled, and the grayscale average value Δ0 in the target frame corresponding to the bolt to be assembled;
[0063] Step 2-6: Define the base plane of the robot arm as a horizontal plane, select three feature points with the smallest horizontal coordinates to determine the normal vector of the bolt to be assembled, and determine the direction of the bolt to be assembled.
[0064] Step 3: Adjust the posture of the main robot arm so that the center axis of the gripper is parallel to the center axis of the bolt to be assembled. Specifically:
[0065] According to the normal vector of the bolt to be assembled and the center point of the bolt to be assembled obtained in step 2, the center axis of the bolt to be assembled is determined, and the posture of the main robot arm is adjusted so that the center axis of the gripper is parallel to the center axis of the bolt to be assembled.
[0066] Step 4: Adjust the main robot arm to get closer to the object to be assembled until it enters the blind spot of the main camera or the bolt object information is significantly reduced. Specifically:
[0067] Step 4-1: Adjust the main robotic arm to approach the bolt to be assembled, and at the same time, the binocular camera on the main robotic arm scans the bolt to be assembled;
[0068] Step 4-2: When the number of feature points in the image changes by Δn≥1 between the two frames before and after, it is considered that the information of the bolts to be assembled is significantly reduced and enters the blind spot of the main camera's field of view;
[0069] Step 4-3: Obtain the distance d between the binocular camera on the main robot arm and the central axis of the bolt to be assembled.
[0070] Step 5: Adjust the slave robot arm, take the center point of the bolt to be assembled as the center of the circle, scan half a circle in a counterclockwise or clockwise direction (counterclockwise when the slave robot arm is the left arm, clockwise when the slave robot arm is the right arm), and collect real-time image data stream to obtain the grayscale information of the bolt to be assembled, specifically:
[0071] Step 5-1: Adjust the position of the slave robot arm so that the optical axis of the binocular camera on the slave robot arm coincides with the intersection line L of the plane where the gripper is located and the plane where the bolts to be assembled are located, and the binocular camera on the slave robot arm faces the operation area. Specifically:
[0072] Step 5-1-1: Adjust the position of the slave robot arm so that the center point of the binocular camera on the robot arm reaches the intersection line L of the plane where the gripper is located and the plane where the bolt to be assembled is located;
[0073] Step 5-1-2: Adjust the posture of the slave robot arm so that the optical axis of the binocular camera on the slave robot arm coincides with the intersection line L of the plane where the gripper is located and the plane where the bolts to be assembled are located, and the slave camera is facing the operating area.
[0074] Step 5-2, move the slave robot arm along the intersection line L close to the center point of the bolt to be assembled until the distance between the binocular camera on the slave robot arm and the center point of the bolt to be assembled is 2d;
[0075] Step 5-3, the robot arm takes the center point of the bolt to be assembled as the center of the circle, scans half a circle, and uses the binocular camera on the robot arm to collect real-time image data stream, pre-processes the image, i.e., Gaussian smoothing, eliminates the noise in the local area, and extracts the grayscale information of the bolt object, where the Gaussian coefficient δ = 0.8.
[0076] Step 6: Select the slave robot arm posture with the smallest grayscale change in the bolt frame corresponding to the bolt to be assembled for image acquisition, and adjust the master robot arm gripper according to the viewing angle to complete the left and right direction calibration, specifically:
[0077] Step 6-1, according to the gray value extracted in step 5, calculate the gray average value Δi in the target frame corresponding to each posture, i=1, 2, 3...10t, t is the scanning time, and the difference between the gray average value |Δi-Δ0| in the target frame corresponding to the initial posture, and take the posture with the smallest absolute value of the difference as the best posture B;
[0078] Step 6-2, move the robot arm to the best posture B recorded in step 6-1;
[0079] Step 6-3: Use the binocular camera image acquisition on the robotic arm to obtain the two-dimensional information of the gripper and the bolt to be assembled in posture B, and move the gripper horizontally until the projection of the center axis of the gripper in the two-dimensional plane obtained from this perspective coincides with the center axis of the bolt to be assembled, completing the left-right calibration.
[0080] Step 7: Rotate the robot arm clockwise or counterclockwise for 1 / 4 circle with the center point of the bolt object as the center to complete the calibration of the front and back directions and vertical angles. Specifically:
[0081] Step 7-1, the robot arm is rotated 1 / 4 circle clockwise or counterclockwise (clockwise when the robot arm is the left arm, counterclockwise when the robot arm is the right arm) with the center point u0 of the bolt to be assembled as the center of the circle, and the image is collected from the side;
[0082] Step 7-2, adjust the posture of the clamping jaws so that the projection of the central axis of the clamping jaws at this viewing angle is parallel to the central axis of the bolt object, completing the calibration of the vertical angle;
[0083] Step 7-3: Move the clamp horizontally so that the center axis of the clamp coincides with the center axis of the bolt object to complete the calibration in the front-to-back direction.
[0084] The method of the present invention can be applied to complex environments. When the target object is blocked, deformed, or changes in scale, the present invention can still automatically adjust the robot arm to a posture suitable for observation to acquire visual information, combining high precision, rapidity, and robustness.
Claims
1. A dual-arm collaborative assembly method based on vision and double calibration, characterized in that: The following steps are involved: Step 1: Initialize and set the binocular cameras on the mechanical arms of the dual-arm robot respectively; Step 2: Use a binocular camera to collect images, preprocess and detect corner points of the images, extract feature points of the target to be assembled, and determine the orientation of the target to be assembled; Step 3: Adjust the posture of the main robot arm so that the center axis of the gripper is parallel to the center axis of the object to be assembled; Step 4: Adjust the main robotic arm so that it is close to the object to be assembled until it enters the blind spot of the main camera or the information of the target object is significantly reduced; Step 5: Adjust the slave robot arm to scan half a circle in a counterclockwise or clockwise direction with the center point of the target to be assembled as the center of the circle, and collect real-time image data stream to obtain grayscale information of the target to be assembled; Step 6: Select the slave robot arm posture with the smallest grayscale change in the target frame corresponding to the target to be assembled for image acquisition, and adjust the master robot arm gripper according to the viewing angle to complete the left-right direction calibration; Step 7: Rotate the robot arm clockwise or counterclockwise for 1 / 4 circle with the center point of the target object as the center, and observe from the side to complete the calibration of the front-to-back direction and vertical angle.
2. The dual-arm collaborative assembly method based on vision and double calibration according to claim 1, characterized in that: The initialization settings in step 1 are specifically: The initial working position of the binocular camera is set so that the optical axes of the two binocular cameras are parallel and perpendicular to the line connecting the optical centers of the two binocular cameras. The transfer matrix from the binocular camera to the end of the robotic arm equipped with the binocular camera, the transformation matrix from the end of the robotic arm to the base, and the intrinsic parameter matrix, extrinsic parameter matrix and distortion coefficient of the two cameras are calibrated respectively.
3. The dual-arm collaborative assembly method based on vision and double calibration according to claim 1, characterized in that: Determining the orientation of the target to be assembled in step 2 specifically includes the following steps: Step 2-1: Use the binocular cameras on the master and slave robotic arms to collect images of the target to be assembled; Step 2-2: Using the intrinsic parameter matrix and extrinsic parameter matrix obtained in step 1 and the distortion coefficient, based on the co-polarity constraint in epipolar geometry, correct the image pair captured by the master and slave cameras in step 2-1; Step 2-3: Based on the Harris operator, the image taken by the binocular camera of the robot arm and corrected in step 2-2 is detected at the corner points, and the six feature points of the target to be assembled are extracted, namely u1, u2, u3, u4, u5, and u6. The center coordinates of the six feature points are used as the center coordinates u0 of the target to be assembled; Step 2-4: Based on the NCC matching algorithm, the disparity of the image pair corrected in step 2-3 is calculated on the basis of the alignment of the two views, and a disparity map is obtained; Step 2-5, using the image coordinates obtained in step 2-3 to find the disparity value of the corresponding point in the disparity map obtained in step 2-4, based on the principle of triangulation, determine the depth information, and based on the depth value, obtain the six feature points of the target to be assembled, the world coordinates of the center point of the target to be assembled, and the grayscale average value Δ0 in the target frame corresponding to the target to be assembled; Step 2-6: Define the plane of the robot base as a horizontal plane, select three feature points with the smallest horizontal coordinates to determine the normal vector of the target to be assembled, and determine the orientation of the target to be assembled.
4. The dual-arm collaborative assembly method based on vision and double calibration according to claim 3 is characterized in that: The adjustment of the main robot arm posture in step 3 is specifically as follows: According to the normal vector of the target to be assembled and the center point of the target to be assembled obtained in step 2, the center axis of the target to be assembled is determined, and the posture of the main robot arm is adjusted so that the center axis of the gripper is parallel to the center axis of the target to be assembled.
5. The dual-arm collaborative assembly method based on vision and double calibration according to claim 1, characterized in that: The adjustment of the main robotic arm in step 4 specifically includes the following steps: Step 4-1: Adjust the main robotic arm to approach the target to be assembled, and at the same time, the binocular camera on the main robotic arm scans the target to be assembled; Step 4-2: When the number of feature points in the image changes by Δn≥1 between the two frames before and after, it is considered that the information of the target to be assembled is significantly reduced and enters the blind spot of the main camera’s field of view; Step 4-3: Obtain the distance d between the binocular camera on the main robot arm and the central axis of the target to be assembled.
6. The dual-arm collaborative assembly method based on vision and double calibration according to claim 5, characterized in that: The adjustment in step 5 obtains the grayscale information of the target to be assembled from the mechanical arm, and specifically includes the following steps: Step 5-1: Adjust the posture of the slave robot arm so that the optical axis of the binocular camera on the slave robot arm coincides with the intersection line L of the plane where the gripper is located and the plane where the object to be assembled is located, and at this time the binocular camera on the slave robot arm faces the operation area; Step 5-2, move the slave robot arm along the intersection line L close to the center point of the target to be assembled, until the distance between the binocular camera on the slave robot arm and the center point of the target to be assembled is 2d; Step 5-3: The robot arm scans half a circle with the center point of the target to be assembled as the center. At the same time, the real-time image data stream is collected from the binocular camera on the robot arm. The image is pre-processed, i.e., Gaussian smoothing is performed to eliminate the noise in the local area and then the grayscale information of the target object is extracted.
7. The dual-arm collaborative assembly method based on vision and double calibration according to claim 6, characterized in that: The adjustment of the slave robot arm posture in step 5-1 is specifically as follows: Step 5-1-1: Adjust the position of the slave robot arm so that the center point of the binocular camera on the robot arm reaches the intersection line L of the plane where the gripper is located and the plane where the object to be assembled is located; Step 5-1-2: Adjust the posture of the slave robot arm so that the optical axis of the binocular camera on the slave robot arm coincides with the intersection line L of the plane where the gripper is located and the plane where the object to be assembled is located, and the slave camera is facing the operating area.
8. The dual-arm collaborative assembly method based on vision and double calibration according to claim 1, characterized in that: The step 6 of completing the left-right direction calibration specifically includes the following steps: Step 6-1, according to the gray value extracted in step 5, calculate the gray average value Δi in the target frame corresponding to each posture, i=1, 2, 3...10t, t is the scanning time, and the difference between the gray average value |Δi-Δ0| in the target frame corresponding to the initial posture, and take the posture with the smallest absolute value of the difference as the best posture B; Step 6-2, move the robot arm to the best posture B recorded in step 6-1; Step 6-3: Use the binocular camera image acquisition on the robotic arm to obtain the two-dimensional information of the gripper and the target to be assembled in position B, and move the gripper horizontally until the projection of the center axis of the gripper in the two-dimensional plane obtained from the viewing angle coincides with the center axis of the target to be assembled, thus completing the left-right calibration.
9. The dual-arm collaborative assembly method based on vision and double calibration according to claim 1, characterized in that: The calibration of the front-to-back direction and the vertical angle in step 7 specifically includes the following steps: Step 7-1: The robot arm is rotated 1 / 4 circle clockwise or counterclockwise with the center point u0 of the object to be assembled as the center, and the image is collected from the side; Step 7-2, adjust the gripper posture so that the projection of the gripper center axis under the viewing angle is parallel to the center axis of the target object, and complete the vertical angle calibration; Step 7-3: Move the gripper horizontally so that the center axis of the gripper coincides with the center axis of the target object to complete the calibration in the front-to-back direction.
