Visual positioning method of high-environment self-adaptive mobile composite robot

Abstract

The invention discloses a visual positioning method for a high-environment self-adaptive mobile composite robot, which comprises an AGV trolley, a feature plate and a 2D camera, the AGV trolley is used for moving among stations and enabling a multi-axis mechanical arm located on the AGV trolley to pick and place a predetermined tool, the feature plate is arranged on the stations, the feature plate 1 is arranged on the tool, and the 2D camera is arranged on the multi-axis mechanical arm. The 2D camera is used for acquiring a 2D image; the visual positioning process comprises the steps that S1, a mechanical arm is controlled to shoot multiple sets of pictures at different angles (preferably 20 sets), the pose of the mechanical arm is obtained, then an internal parameter matrix and an external parameter matrix of a camera are calculated according to picture set data, and the parameter matrixes are used for 3D position calibration calculation. 3D poses (X, Y, Z, RX, RY and RZ) of material taking and placing points are obtained through a 2D camera; and the low-cost 2D camera is used for realizing partial functions of acquiring 3D poses by the 3D camera.

Description

Technical Field

[0001] The invention relates to the field of robot training systems, and in particular to a visual positioning method for a highly environmentally adaptive mobile composite robot. Background Art

[0002] A manipulator is an automatic device that can imitate certain movements of human hands and arms, and is used to grab, carry objects or operate tools according to a fixed program. It is characterized by being able to complete various expected operations through programming, and its structure and performance combine the advantages of both human and mechanical machines.

[0003] The design of existing mobile robots is generally an AGV car (mobile car), a multi-axis mechanical arm and a manipulator (of course, it is not limited to a gripper, and can also be a spraying mechanism).

[0004] Each time the AGV moves to a workstation, it needs to use a scanning device (such as a 3D camera or 3D laser scanner) to take a picture of the feature and calculate the AGV's docking deviation value. The deviation value is then compensated to the point where the robotic arm picks up the material for correction in order to accurately pick up the material.

[0005] Each workstation needs to perform a hand-eye calibration before taking photos of features and calculating deviation values ​​to obtain the conversion matrix between pixel coordinates and robot coordinates.

[0006] However, this type of scanning device (i.e. 3D imaging) is generally expensive (the cost of general precision is generally more than 10,000), and the calculation of deviations is also required to be high (i.e. the higher the precision, the more expensive it is). That is, it mainly relies on the quality of the 3D imaging point cloud, and has high requirements for the quality and accuracy of the point cloud. For example, for some workpieces with complex materials and surfaces, the imaging quality of ordinary 3D cameras is often poor, which seriously affects the positioning accuracy of the workpiece surface trajectory, resulting in limited application scenarios for the use of 3D cameras for visual positioning and high hardware costs.

[0007] At the same time, generally speaking, the size of a 3D scanning device is relatively large. When it is installed at the end of a multi-axis robotic arm, it has a greater impact on the gripping or processing of the robot when the processing space is small.

[0008] Especially when an AGV corresponds to multiple workstations, the relative position of the feature objects and material picking points at each workstation is inconsistent, and there are deviations in position, height, and angle (so the environmental factors are more complicated). The existing method is to teach the material picking point of the robot arm and perform hand-eye calibration for each workstation, but this method requires more time and manpower.

[0009] However, this type of scanning device is generally expensive, and has high requirements for deviation calculation, and its operation logic is relatively complex, so the debugging time cost and production cost are high. Assuming that a processed product is replaced, its time cost in secondary production is still high, so the equipment cost is relatively high. Summary of the invention

[0010] The main purpose of the present invention is to propose a visual positioning method for a highly environmentally adaptive mobile compound robot, which aims to use a 2D visual device to obtain the 3D posture of a predetermined target position, thereby enabling the mobile compound robot to accurately obtain objects. At the same time, it can achieve the mobile compound robot's adaptability to high environments (such as uneven ground, tilted machine and other problems), and can achieve the mobile compound robot's.

[0011] To achieve the above object, the present invention proposes a visual positioning method for a highly environmentally adaptive mobile composite robot, comprising:

[0012] An AGV trolley, which is used to move between various stations and enable a multi-axis robotic arm located on the AGV trolley to pick up and place predetermined tooling;

[0013] Characteristic plate, the characteristic plate n is arranged at the station, and the characteristic plate 1 is arranged on the tooling;

[0014] A 2D camera, wherein the 2D camera is used to acquire a 2D image;

[0015] The visual positioning process includes:

[0016] S1: Control the manipulator to take multiple groups of pictures (preferably 20 groups) at different angles, and obtain the position and posture of the manipulator, and then calculate the intrinsic parameter matrix and extrinsic parameter matrix of the camera according to the picture group data, and the parameter matrix is ​​used for 3D position calibration calculation;

[0017] S2: Place the tooling with feature plate 1 on the site to be taught, and the robot arm teaches the pick-up and placement posture of the tooling. Then the 2D camera takes pictures of the feature plate on the tooling for identification and calibration, and obtains the position P1 (X1, Y1, Z1, RX1, RY1, RZ1) of the feature plate 1 on the tooling in the camera coordinate system.

[0018] Calculate the three-dimensional transformation matrix T1 between the tooling grasping posture and the feature plate;

[0019] S3: Place the tooling in the material placement position of the machine that needs to be calibrated and taught, and use the 2D camera to shoot the feature plate on the tooling to obtain the pose P2 (X2, Y2, Z2, RX2, RY2, RZ2) of the feature plate 1 on the tooling in the camera coordinate system;

[0020] S4: Combining the intrinsic parameter matrix and extrinsic parameter matrix after hand-eye calibration, and the three-dimensional transformation matrix T1 between the tooling grasping posture and the feature plate, the station material pick-up and placement position P3 (X3, Y3, Z3, RX3, RY3, RZ3) can be calculated;

[0021] S5: The 2D camera shoots the feature plate on the site to obtain the pose P4 (X4, Y4, Z4, RX4, RY4, RZ4) of the site feature plate in the relative camera coordinate system. Combined with the site material placement position P3 of S4, the three-dimensional transformation matrix T2 between the site feature plate and the site material placement position is calculated. The calibration process of the target site is now completed.

[0022] S6: When the AGV leaves the station and comes back to the station to take a picture of the feature plate on the station,

[0023] That is, the material picking and placing posture of the site at this time can be calculated based on T2, and the picking and placing position of the mobile composite robot on the site material can be known;

[0024] S7: The parameters obtained are used to calculate the 3D posture of the material to be placed.

[0025] When the environment is a non-horizontal plane, the mobile composite robot is tilted at an angle, and the 2D camera reacquires the position P5 of the feature plate, and can reacquire the position P6 of the material through calculation.

[0026] Beneficial effects:

[0027] 1. The relative position of the material is obtained through the calculation of S4. Through the calculation of the image group data and the postures P1, P2, and P3, there is no need to manually use the robot arm to perform tedious teaching and positioning of the material pick-up and placement points (generally, manual teaching requires multiple debugging to reduce errors), and the application of rapid calibration teaching is realized. In the actual 3D model, it is a better reference model;

[0028] 2. In the actual design, the present application has two reference points, namely the feature plate and the tooling, with the relative position of the feature plate and the matrix molding of the 2D camera (i.e., 2D image). When the tooling is replaced or the material is replaced, the 3D pose of the new material can be quickly obtained through the known three-dimensional transformation matrix T1;

[0029] Therefore, in the subsequent debugging of the same site (i.e., there is no need to debug other sites one by one), it is only necessary to re-equip the tool to obtain the characteristic plate pose P1 to obtain the pose of the same type of site. For example, if the tool is tilted or the height is inconsistent, it is equivalent to that the relative position of the tool and the material is determined. It is only necessary to re-acquire the characteristic plate in the site to know whether there is a change in its height direction, X direction or Y direction. Then the multi-axis robot calculates the error and corrects the pose parameters.

[0030] 3. It can be understood that the relative position can be obtained through the three-dimensional position of the robot arm and the 2D position of the feature plate and the 2D position of the tooling. It can be further understood that the existing 3D scanning is for outer contour molding (it can be understood that the model is hollow), and the basis of this application is to set a predetermined plane at the site, and the 3D position of the tooling can be determined through the relative position of the robot arm, that is, it is not necessary to accurately obtain the product shape of the tooling, but only to obtain the shape of its clamping position, which reduces the calculation process;

[0031] When there are more sites, the efficiency is higher, that is, the location of the second site can be determined faster and does not need to be taught one by one. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is the block diagram of this application;

[0033] Figure 2 This is a schematic diagram of this application;

[0034] Figure 3 This is a schematic diagram of an existing 3D scan.

[0035] In the figure, a is the feature plate, b is the tooling, and c is the material. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, top, bottom, inside, outside, vertical, horizontal, longitudinal, counterclockwise, clockwise, circumferential, radial, axial...), then the directional indication is only used to explain the relative position relationship, movement status, etc. between the components in a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0038] In addition, if there are descriptions involving "first" or "second" etc. in the embodiments of the present invention, the descriptions of "first" or "second" etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0039] like Figures 1 to 3 As shown, a visual positioning method for a highly environmentally adaptive mobile composite robot includes:

[0040] An AGV trolley, which is used to move between various stations and enable a multi-axis robotic arm located on the AGV trolley to pick up and place predetermined tooling;

[0041] Characteristic plate, the characteristic plate n is arranged at the station, and the characteristic plate 1 is arranged on the tooling;

[0042] A 2D camera, wherein the 2D camera is used to acquire a 2D image;

[0043] The visual positioning process includes:

[0044] S1: Control the manipulator to take multiple groups of pictures (preferably 20 groups) at different angles, and obtain the position and posture of the manipulator, and then calculate the intrinsic parameter matrix and extrinsic parameter matrix of the camera according to the picture group data, and the parameter matrix is ​​used for 3D position calibration calculation;

[0045] S2: Place the tooling with feature plate 1 on the site to be taught, and the robot arm teaches the pick-up and placement posture of the tooling. Then the 2D camera takes pictures of the feature plate on the tooling for identification and calibration, and obtains the position P1 (X1, Y1, Z1, RX1, RY1, RZ1) of the feature plate 1 on the tooling in the camera coordinate system.

[0046] Calculate the three-dimensional transformation matrix T1 between the tooling grasping posture and the feature plate;

[0047] S3: Place the tooling in the material placement position of the machine that needs to be calibrated and taught, and use the 2D camera to shoot the feature plate on the tooling to obtain the pose P2 (X2, Y2, Z2, RX2, RY2, RZ2) of the feature plate 1 on the tooling in the camera coordinate system;

[0048] S4: Combining the intrinsic parameter matrix and extrinsic parameter matrix after hand-eye calibration, and the three-dimensional transformation matrix T1 between the tooling grasping posture and the feature plate, the station material pick-up and placement position P3 (X3, Y3, Z3, RX3, RY3, RZ3) can be calculated;

[0049] S5: The 2D camera shoots the feature plate on the site to obtain the pose P4 (X4, Y4, Z4, RX4, RY4, RZ4) of the site feature plate in the relative camera coordinate system. Combined with the site material placement position P3 of S4, the three-dimensional transformation matrix T2 between the site feature plate and the site material placement position is calculated. The calibration process of the target site is now completed.

[0050] S6: When the AGV leaves the station and comes back to the station to take a picture of the feature plate on the station,

[0051] That is, the material picking and placing posture of the site at this time can be calculated based on T2, and the picking and placing position of the mobile composite robot on the site material can be known;

[0052] S7: The parameters obtained are used to calculate the 3D posture of the material to be placed.

[0053] When the environment is a non-horizontal plane, the mobile composite robot is tilted at an angle, and the 2D camera reacquires the position P5 of the feature plate, and can reacquire the position P6 of the material through calculation.

[0054] Beneficial effects:

[0055] 1. The relative position of the material is obtained through the calculation of S4. Through the calculation of the image group data and the postures P1, P2, and P3, there is no need to manually use the robot arm to perform tedious teaching and positioning of the material pick-up and placement points (generally, manual teaching requires multiple debugging to reduce errors), and the application of rapid calibration teaching is realized. In the actual 3D model, it is a better reference model;

[0056] 2. In the actual design, the present application has two reference points, namely the feature plate and the tooling, with the relative position of the feature plate and the matrix molding of the 2D camera (i.e., 2D image). When the tooling is replaced or the material is replaced, the 3D pose of the new material can be quickly obtained through the known three-dimensional transformation matrix T1;

[0057] Therefore, in the subsequent debugging of the same site (i.e., there is no need to debug other sites one by one), it is only necessary to re-equip the tool to obtain the characteristic plate pose P1 to obtain the pose of the same type of site. For example, if the tool is tilted or the height is inconsistent, it is equivalent to that the relative position of the tool and the material is determined. It is only necessary to re-acquire the characteristic plate in the site to know whether there is a change in its height direction, X direction or Y direction. Then the multi-axis robot calculates the error and corrects the pose parameters.

[0058] 3. It can be understood that the relative position can be obtained through the three-dimensional position of the robot arm and the 2D position of the feature plate and the 2D position of the tooling. It can be further understood that the existing 3D scanning is for outer contour molding (it can be understood that the model is hollow), and the basis of this application is to set a predetermined plane at the site, and the 3D position of the tooling can be determined through the relative position of the robot arm, that is, it is not necessary to accurately obtain the product shape of the tooling, but only to obtain the shape of its clamping position, which reduces the calculation process;

[0059] When there are more sites, the efficiency is higher, that is, the location of the second site can be determined faster and does not need to be taught one by one.

[0060] That is, the 3D posture of the material being picked up and placed is calculated, so even if the composite robot is tilted at an angle due to uneven ground, the position of the material can still be determined after taking a picture of the reference plate.

[0061] Specifically, the 3D pose (X, Y, Z, RX, RY, RZ) of the material pick-up and placement points is acquired through a 2D camera; a low-cost 2D camera is used to implement some of the functions of a 3D camera to acquire 3D pose; the calibration deployment of the site can be completed by photographing the feature plates on the calibration tooling to achieve manual teaching of the site pick-up and placement positions, reducing the original teaching and calibration time for each site from 15 minutes to 1 minute.

[0062] Solve the problem that the compound robot takes a long time to teach and calibrate the site and consumes a lot of manpower. Solve the problem that the compound robot's 2D visual plane positioning causes material picking and placing accuracy deviation due to uneven ground and tilted sites.

[0063] Specifically, the 3D hand-eye calibration formula is:

[0064] Known: The position of the robot when taking pictures is converted into a matrix: Base_H_tool,

[0065] The pose obtained by the camera feature plate is converted into a matrix: Cam_H_Cal,

[0066] The position relationship matrix between the feature plate and the robot base: Base_H_Cal, fixed;

[0067] Solution:

[0068] The relationship matrix between the robot tool coordinate system and the camera: Cam_H_Tool;

[0069] The relationship matrix between the feature plate and the robot base: Base_H_Cal;

[0070] The S1 picture has 20 groups. Substituting the known 20 groups of Base_H_tool and Cam_H_Cal into the following formula to establish the equation group can obtain the solution result.

[0071] Specifically, the conversion matrix formula between the tooling grasping posture and the feature plate on the tooling is obtained:

[0072] Known:

[0073] Tooling grabbing posture P0 conversion matrix Base_H_Grab;

[0074] The tooling recognized by the camera obtains the characteristic plate pose P1 conversion matrix Cam_H_Cal;

[0075] Solution:

[0076] The conversion matrix T1 between the tooling grabbing posture and the feature plate on the tooling: Cam_H_Grab;

[0077] formula:

[0078] Cam_H_Grab=Base_H_Grab*Base_H_Cal^(-1)*Cam_H_Cal.

[0079] Specifically, the formula for shooting tooling to calculate the material placement posture at the site is:

[0080] Known:

[0081] The tooling identified by the camera obtains the characteristic plate pose P2 conversion matrix Cam_H_Cal;

[0082] Solution:

[0083] Station material picking and placing posture P3;

[0084] formula:

[0085] Base_H_Grab1=Cam_H_Cal^(-1)*Base_H_Cal*Cam_H_Grab;

[0086] Site material pick-up and placement posture P3: Base_H_Grab1 matrix conversion posture;

[0087] Abbreviation:

[0088] P3=P2*T1.

[0089] Specifically, the conversion matrix formula between the shooting site feature plate and the site feature plate is calculated:

[0090] Known:

[0091] The site features identified by the camera obtain the feature plate pose P4 conversion matrix Cam_H_Cal;

[0092] Solution:

[0093] The transformation matrix T2 between the material picking and placing posture of the site and the feature plate of the site;

[0094] formula:

[0095] Cam_H_Grab1=Base_H_Grab1*Base_H_Cal^(-1)*Cam_H_Cal.

[0096] Specifically, the formula for calculating the material placement posture of the site by shooting the site feature plate is:

[0097] Known:

[0098] The camera recognizes the site feature plate and obtains the pose P5 conversion matrix Cam_H_Cal;

[0099] Solution:

[0100] Station material placement posture P6

[0101] formula:

[0102] Base_H_Grab2=Cam_H_Cal^(-1)*Base_H_Cal*Cam_H_Grab1;

[0103] Site material pick-and-place posture P6: Base_H_Grab2 matrix conversion posture;

[0104] Abbreviation:

[0105] P6=P5*T2.

[0106] It also includes a storage device, which is used to store the parameters of the characteristic plate n and call the parameters of the predetermined characteristic plate n according to the different materials and tooling.

[0107] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A visual positioning method for a highly environmentally adaptive mobile composite robot, characterized in that: include, An AGV trolley, which is used to move between various stations and enable a multi-axis robotic arm located on the AGV trolley to pick up and place predetermined tooling; Characteristic plate, the characteristic plate n is arranged at the station, and the characteristic plate 1 is arranged on the tooling; A 2D camera, wherein the 2D camera is used to acquire a 2D image; The visual positioning process includes: S1: Control the manipulator to take multiple groups of pictures (preferably 20 groups) at different angles, and obtain the position and posture of the manipulator, and then calculate the intrinsic parameter matrix and extrinsic parameter matrix of the camera according to the picture group data, and the parameter matrix is ​​used for 3D position calibration calculation; S2: Place the tooling with feature plate 1 on the site to be taught, and the robot arm teaches the pick-up and placement posture of the tooling. Then the 2D camera takes pictures of the feature plate on the tooling for identification and calibration, and obtains the position P1 (X1, Y1, Z1, RX1, RY1, RZ1) of the feature plate 1 on the tooling in the camera coordinate system. Calculate the three-dimensional transformation matrix T1 between the tooling grasping posture and the feature plate; S3: Place the tooling in the material placement position of the machine that needs to be calibrated and taught, and use the 2D camera to shoot the feature plate on the tooling to obtain the pose P2 (X2, Y2, Z2, RX2, RY2, RZ2) of the feature plate 1 on the tooling in the camera coordinate system; S4: Combining the intrinsic parameter matrix and extrinsic parameter matrix after hand-eye calibration, and the three-dimensional transformation matrix T1 between the tooling grasping posture and the feature plate, the station material pick-up and placement position P3 (X3, Y3, Z3, RX3, RY3, RZ3) can be calculated; S5: The 2D camera shoots the feature plate on the site to obtain the pose P4 (X4, Y4, Z4, RX4, RY4, RZ4) of the site feature plate in the relative camera coordinate system. Combined with the site material placement position P3 of S4, the three-dimensional transformation matrix T2 between the site feature plate and the site material placement position is calculated. The calibration process of the target site is now completed. S6: When the AGV leaves the station and comes back to the station to take a picture of the feature plate on the station, That is, the material picking and placing posture of the site at this time can be calculated based on T2, and the picking and placing position of the mobile composite robot on the site material can be known; S7: The parameters obtained are used to calculate the 3D posture of the material to be placed. When the environment is a non-horizontal plane, the mobile composite robot is tilted at an angle, and the 2D camera reacquires the position P5 of the feature plate, and can reacquire the position P6 of the material through calculation.

2. The visual positioning method of the highly environment-adaptive mobile composite robot according to claim 1, characterized in that: 3D hand-eye calibration formula: Known: The position of the robot when taking pictures is converted into a matrix: Base_H_tool, The pose obtained by the camera feature plate is converted into a matrix: Cam_H_Cal, The position relationship matrix between the feature plate and the robot base: Base_H_Cal, fixed; Solution: The relationship matrix between the robot tool coordinate system and the camera: Cam_H_Tool; The relationship matrix between the feature plate and the robot base: Base_H_Cal; The S1 picture has 20 groups. Substituting the known 20 groups of Base_H_tool and Cam_H_Cal into the following formula to establish the equation group, the solution can be obtained. Cam_H_Tool=Cam_H_Cal*Base_H_Cal^(-1)*Base_H_Tool.

3. The visual positioning method of the highly environment-adaptive mobile composite robot according to claim 2, characterized in that: Get the conversion matrix formula between the tooling grasping posture and the feature plate on the tooling: Known: Tooling grabbing posture P0 conversion matrix Base_H_Grab; The tooling recognized by the camera obtains the characteristic plate pose P1 conversion matrix Cam_H_Cal; Solution: The conversion matrix T1 between the tooling grabbing posture and the feature plate on the tooling: Cam_H_Grab; formula: Cam_H_Grab=Base_H_Grab*Base_H_Cal^(-1)*Cam_H_Cal.

4. The visual positioning method of the highly environment-adaptive mobile composite robot according to claim 3, characterized in that: The formula for shooting the tooling calculation station material placement posture: Known: The tooling identified by the camera obtains the characteristic plate pose P2 conversion matrix Cam_H_Cal; Solution: Station material picking and placing posture P3; formula: Base_H_Grab1=Cam_H_Cal^(-1)*Base_H_Cal*Cam_H_Grab; Site material pick-up and placement posture P3: Base_H_Grab1 matrix conversion posture; Abbreviation: P3=P2*T1.

5. The visual positioning method of the highly environment-adaptive mobile composite robot according to claim 4, characterized in that: Shooting site feature plate to calculate the conversion matrix formula between the site material placement posture and the site feature plate: Known: The site features identified by the camera obtain the feature plate pose P4 conversion matrix Cam_H_Cal; Solution: The transformation matrix T2 between the material picking and placing posture of the site and the feature plate of the site; formula: Cam_H_Grab1=Base_H_Grab1*Base_H_Cal^(-1)*Cam_H_Cal.

6. The visual positioning method of the highly environment-adaptive mobile composite robot according to claim 5, characterized in that: Photograph the site feature plate to calculate the formula for the site material placement posture; Known: The camera recognizes the site feature plate and obtains the pose P5 conversion matrix Cam_H_Cal; Solution: Station material placement posture P6 formula: Base_H_Grab2=Cam_H_Cal^(-1)*Base_H_Cal*Cam_H_Grab1; Site material pick-and-place posture P6: Base_H_Grab2 matrix conversion posture; Abbreviation: P6=P5*T2.

7. The visual positioning method of the highly environment-adaptive mobile composite robot according to claim 1, characterized in that: It also includes a storage device, which is used to store the parameters of the characteristic plate n and call the parameters of the predetermined characteristic plate n according to the different materials and tooling.

Images