Mechanical arm follow-up hand-eye calibration method, system and equipment without public visual field

Abstract

The invention relates to the technical field of visual calibration, and discloses a follow-up hand-eye calibration method, system and equipment for a mechanical arm without a common visual field. A robot provided with a scanning camera and a positioning camera are arranged on the two sides of a production line, and first point cloud data of a calibration block in a coordinate system of a target positioning camera are obtained; second point cloud data of the calibration block under the coordinate system of the scanning camera is obtained, and the mechanical arm posture of the robot is obtained; and the first point cloud data and the second point cloud data are aligned, and the pose relation of the scanning camera and the target positioning camera in the space coordinate system is obtained in combination with the posture of the mechanical arm. Therefore, a set of effective calibration mode is provided by utilizing the high-precision calibration block and comprehensively utilizing the calibration technology that the eye is on the hand and the eye is on the hand, and the problem that a mechanical arm and a positioning camera do not have a public operation area on a plate chain production line can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of visual calibration, and in particular to a method, system and device for follow-up hand-eye calibration of a robotic arm without a public field of view. Background Art

[0002] With the increasing requirements for automobile assembly processes, more and more OEMs have begun to use automatic gap and flushness measurement systems instead of manual measurement methods in welding production lines, which has improved detection efficiency, accuracy, and stability. However, most OEMs still use manual detection methods for general assembly production lines, because the working conditions of general assembly lines are more complicated, and it is difficult to provide a large enough inspection station for the gap and flushness measurement system. In addition, the random appearance of personnel and the uncertainty of the shape of vehicles when entering the inspection station have brought considerable difficulties to the implementation of automatic gap and flushness detection applications on the general assembly line.

[0003] At present, the automatic gap and flushness measurement system implemented in the final assembly line in the industry generally requires the production line to stop moving and detect when the vehicle is static. Or a robot follow-up solution is used, but no systematic hand-eye calibration is performed, assuming that the robot's follow-up coordinate system is aligned with the motion coordinate system of the production line. In this way, the use of static detection on the final assembly line can greatly reduce the accuracy of vehicle positioning and correction, but the final assembly line needs to stop during detection to wait for the gap and flushness measurement system to be completed, which will reduce the production cycle. If a solution assuming that the coordinate system is aligned is used, the positioning and correction accuracy will be reduced. It can be seen that in the prior art, when the final assembly line vehicle is moving, the robot arm must adjust the robot arm scanning posture according to the vehicle posture, and the positioning guidance camera and the robot arm do not have a common field of view, and hand-eye calibration cannot be achieved through conventional methods. Summary of the invention

[0004] The present invention provides a method, system and equipment for follow-up hand-eye calibration of a robotic arm without a common field of view, so as to solve the problem in the prior art that during the movement of vehicles on the final assembly production line, the robotic arm needs to adjust the robotic arm scanning posture accordingly according to the vehicle posture, and the positioning guidance camera and the robotic arm have no common field of view, so hand-eye calibration cannot be achieved by conventional methods.

[0005] In order to achieve the above object, the present invention is implemented by the following technical solutions:

[0006] In a first aspect, the present invention provides a method for following hand-eye calibration of a robotic arm without a public field of view, comprising:

[0007] S1: Robots with scanning cameras and positioning cameras are arranged on both sides of the production line, and the plate chain is started after the calibration blocks are placed according to the set requirements for scanning; the number of robots is N, and the positioning cameras include the first positioning camera and the second positioning camera;

[0008] S2: Acquire first point cloud data of the calibration block in the coordinate system of the target positioning camera, where the target positioning camera is any one of the first positioning camera and the second positioning camera;

[0009] S3: Obtain the second point cloud data of the calibration block in the coordinate system of the scanning camera, and obtain the robot arm posture;

[0010] S4: Align the first point cloud data with the second point cloud data, and obtain the position and posture relationship between the scanning camera and the target positioning camera in the spatial coordinate system in combination with the robot arm posture.

[0011] Optionally, before starting the plate chain line, the method further comprises:

[0012] Hand-eye calibration is performed on each robot’s robotic arm and scanning camera respectively.

[0013] Optionally, set requirements, including:

[0014] Set the calibration block at a distance of M meters from the target positioning camera.

[0015] Optionally, the value of M satisfies the calibration block passing through the target positioning camera at a uniform speed stage after the plate chain line is started.

[0016] Optionally, a plurality of circular holes are provided on the calibration block.

[0017] Optionally, S4 includes:

[0018] Taking the circular hole on the calibration block as a feature, determining the circular hole center coordinate point set of the calibration block under the target camera and the circular hole center coordinate point set of the calibration block under the scanning camera;

[0019] Based on the circular hole center coordinate point set under the target camera and the circular hole center coordinate point set under the scanning camera, the position and posture relationship between the scanning camera and the target positioning camera in the spatial coordinate system is obtained.

[0020] Optionally, the diameter of the circular hole is 30 cm, the thickness of the circular hole is 15 cm, and the number of the circular holes is 6, including a first circular hole, a second circular hole, a third circular hole, a fourth circular hole, a fifth circular hole and a sixth circular hole.

[0021] Optionally, when scanning, the calibration block first passes through the target camera and then passes through the measurement camera.

[0022] In a second aspect, the present application provides a robotic arm follow-up hand-eye calibration system without a public field of view, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of the first aspect when executing the computer program.

[0023] In a third aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method of the first aspect when executing the computer program.

[0024] Beneficial effects:

[0025] The present invention provides a method for following hand-eye calibration of a robotic arm without a common field of view. Robots and positioning cameras equipped with scanning cameras are arranged on both sides of the production line. First, the first point cloud data of the calibration block in the coordinate system of the target positioning camera is obtained; then the second point cloud data of the calibration block in the coordinate system of the scanning camera is obtained, and the posture of the robot's robotic arm is obtained; the first point cloud data and the second point cloud data are aligned, and the position relationship between the scanning camera and the target positioning camera in the spatial coordinate system is obtained in combination with the posture of the robotic arm; the scanning camera and the target positioning camera on the same side are calibrated. In this way, a high-precision calibration block is used, and the calibration technology of eye outside the hand and eye on the hand is comprehensively utilized to propose an effective calibration method, which can solve the problem that the robotic arm and the positioning camera on the plate chain production line have no common operating area.

[0026] In a further solution, all scanning cameras are turned on after the calibration block has passed a certain distance from the positioning camera, which can reduce the waiting time of the scanning cameras. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of a method for following hand-eye calibration of a robotic arm without a common field of view according to a preferred embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram of the layout of both sides of the production line of a preferred embodiment of the present invention;

[0029] Figure 3 Schematic diagram of a calibration block according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] Unless otherwise defined, the technical terms or scientific terms used in the present invention shall have the usual meanings understood by persons with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "one" do not indicate quantity restrictions, but indicate the existence of at least one. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship also changes accordingly.

[0032] It should be understood that in the prior art, static detection is used in the final assembly line, which can greatly reduce the accuracy of vehicle positioning and correction. However, the final assembly line needs to be stopped during detection to wait for the gap and flushness measurement system to be completed, which will reduce the production cycle. If the coordinate system alignment scheme is assumed, the positioning and correction accuracy will be reduced. Based on this, the present application provides a method for calibrating a robotic arm with hand-eye tracking without public view.

[0033] See also Figure 1 The present application provides a method for calibrating a robotic arm with hand-eye contact without a public field of view, comprising:

[0034] S1: Robots equipped with scanning cameras and positioning cameras are arranged on both sides of the production line, and the plate chain line is started after the calibration blocks are placed based on the set requirements for scanning; the number of robots is N, and the positioning cameras include the first positioning camera and the second positioning camera.

[0035] In this step, the scanning camera can be installed on the robot arm, and N can be 4, that is, in one example, Figure 2 As shown, the first robot, the third robot and the first positioning camera can be arranged on one side of the production line, and the second robot, the fourth robot and the second positioning camera can be arranged on the other side of the production line. Among them, each robot's mechanical arm is equipped with a 3D line laser scanning camera to measure the gap and flushness. The positioning camera is used to position the car body. When fixing the mechanical arm, it is necessary to ensure that the scanning cameras on all mechanical arms can scan the calibration block.

[0036] S2: Acquire first point cloud data of the calibration block in the coordinate system of the target positioning camera, where the target positioning camera is any one of the first positioning camera and the second positioning camera.

[0037] In this step, after the plate chain line is started, the positioning camera starts scanning, and after the scanning is completed, the first point cloud data of the calibration block under the target positioning camera system can be obtained.

[0038] S3: Obtain the second point cloud data of the calibration block in the coordinate system of the scanning camera, and obtain the robot arm posture.

[0039] In this step, all scanning cameras are turned on after the calibration block has passed a certain distance from the positioning camera, thus reducing the waiting time of the scanning cameras.

[0040] S4: Align the first point cloud data with the second point cloud data, and obtain the position and posture relationship between the scanning camera and the target positioning camera in the spatial coordinate system in combination with the robot arm posture.

[0041] In this step, the alignment may be performed by using the SVD principle.

[0042] In this embodiment, during the calibration process, the scanning camera and the target positioning camera located on the same side are calibrated. Located on the same side means that, in one example, the first positioning camera is located on the same side as the first robot and the third robot; the second positioning camera is located on the same side as the second robot and the fourth robot. Then, the first positioning camera can be calibrated with the first robot or the third robot; the second positioning camera can be calibrated with the second robot or the fourth robot. Specifically, it is calibrated with the scanning camera on the robot arm.

[0043] The above-mentioned chain skipping identification method based on visual detection connects the targets of at least two chain links identified in pairs and calculates the angle of the connection to obtain the chain angle; if the chain angle is lower than the angle threshold, it is in a normal state, and if the chain angle is higher than the angle threshold, it is in a chain skipping state. In this way, precise algorithms and models can be used to provide highly accurate judgment and analysis when processing large amounts of data. Compared with manual judgment, it can maintain higher consistency and stability and reduce the possibility of human error.

[0044] Below, taking the number of robots as 4 and the number of positioning cameras as 2 as an example, the steps of the above-mentioned method for following hand-eye calibration of the robotic arm without a public field of view are described as follows:

[0045] First, a 3D line laser scanning camera is installed on each of the four robotic arms to measure the gap and face difference, and a 3D line laser camera is installed on each side of the production line for body positioning. Before calibration, hand-eye calibration is performed on each robotic arm and scanning camera. The hand-eye calibration relationship is assumed to be Where S2 represents the gap and face difference measurement camera (the measurement camera installed on the mechanical arm of the second robot), and E represents the end of the mechanical arm. Start the line body, and the second positioning camera C2 scans as follows Figure 3After scanning the calibration block, the point cloud data of the calibration block in the C2 system can be obtained. After the calibration block is scanned and enters the detection area, each measuring camera scans the calibration block with a fixed posture. After scanning, the point cloud data of the calibration block in the scanning camera coordinate system can be obtained, and the current posture of the robotic arm can be obtained from the robotic arm system. Then, the holes on the calibration block are used as features to align the calibration blocks in the two camera coordinate systems using SVD, and the pose relationship between the scanning camera (referred to as camera S2 in one example) and the second positioning camera C2 in the spatial coordinate system is calculated.

[0046] Specifically, The calculation principle is obtained based on the invariant characteristics of rigid body space transformation. For example, the coordinate point set of the center of the circular hole of the calibration block under the scanning camera is Assume that the center coordinate point set of the circular hole of the calibration block under the second positioning camera C2 is Then there is in It is decomposed by SVD principle. According to the above relationship integration, the relationship between the second positioning camera C2 and the current robot arm Base system can be obtained: Similarly, the position relationship between other positioning cameras and the robot arm, and the relationship between the positioning camera and the scanning camera can be obtained. In this process, it is best to calibrate with the camera on the same side, such as calibrating positioning camera 1 with robot 1 / 3, and calibrating positioning camera 2 with robot 2 / 4, because the characteristic holes scanned by the positioning camera and the scanning camera under the same-side calibration are the same group, and the calibration error will be smaller than the calibration error on the opposite side.

[0047] It is worth noting that during the scanning process, the positioning camera scans the calibration block first, and then the measuring camera scans the calibration block. The calibration block is placed at a certain distance from the positioning camera. This distance needs to satisfy the standard block passing the positioning camera at a constant speed after the plate chain line is started, rather than in an accelerated stage. After the plate chain line is started, the left and right positioning line scanning cameras start scanning. In addition, turning on all scanning cameras after the calibration block has passed the positioning camera for a certain distance can reduce the waiting time of the scanning camera.

[0048] The present application also provides a robotic arm follow-up hand-eye calibration system without a public field of view, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the steps of the above method are implemented when the processor executes the computer program.

[0049] The robot arm follow-up hand-eye calibration system without public field of view can implement various embodiments of the above-mentioned robot arm follow-up hand-eye calibration method without public field of view, and can achieve the same beneficial effects, which will not be elaborated here.

[0050] The present application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0051] The computer device can implement various embodiments of the above-mentioned method for following hand-eye calibration of a robotic arm without a public field of view, and can achieve the same beneficial effects, which will not be described in detail here.

[0052] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for follow-up hand-eye calibration of a robotic arm without public field of view, characterized in that: include: S1: Robots with scanning cameras and positioning cameras are arranged on both sides of the production line, and the plate chain line is started after the calibration block is placed according to the set requirements for scanning; the number of the robots is N, and the positioning cameras include a first positioning camera and a second positioning camera; S2: Acquire first point cloud data of the calibration block in the coordinate system of a target positioning camera, where the target positioning camera is any one of the first positioning camera and the second positioning camera; S3: Obtain the second point cloud data of the calibration block in the coordinate system of the scanning camera, and obtain the robot arm posture; S4: Align the first point cloud data and the second point cloud data, and obtain the position and posture relationship between the scanning camera and the target positioning camera in the spatial coordinate system in combination with the robot arm posture.

2. The method for following hand-eye calibration of a robotic arm without public view according to claim 1, characterized in that: Before the start-up board chain line, the method further comprises: Hand-eye calibration is performed on each robot’s robotic arm and scanning camera respectively.

3. The method for following hand-eye calibration of a robotic arm without public view according to claim 1, characterized in that: The setting requirements include: The calibration block is set at a distance of M meters from the target positioning camera.

4. The method for following hand-eye calibration of a robotic arm without public view according to claim 3 is characterized in that: The value of M satisfies the requirement that the calibration block passes through the target positioning camera at a uniform speed after the plate chain line is started.

5. The method for following hand-eye calibration of a robotic arm without public field of view according to claim 1, characterized in that: The calibration block is provided with a plurality of circular holes.

6. The method for following hand-eye calibration of a robotic arm without public view according to claim 5, characterized in that: The S4 includes: Taking the circular hole on the calibration block as a feature, determining the circular hole center coordinate point set of the calibration block under the target camera and the circular hole center coordinate point set of the calibration block under the scanning camera; Based on the circular hole center coordinate point set under the target camera and the circular hole center coordinate point set under the scanning camera, the position and posture relationship between the scanning camera and the target positioning camera in the space coordinate system is obtained.

7. The method for following hand-eye calibration of a robotic arm without public view according to claim 5, characterized in that: The diameter of the circular hole is 30 cm, the thickness of the circular hole is 15 cm, and the number of the circular holes is 6, including a first circular hole, a second circular hole, a third circular hole, a fourth circular hole, a fifth circular hole and a sixth circular hole.

8. The method for following hand-eye calibration of a robotic arm without public field of view according to claim 4, characterized in that: When scanning, the calibration block passes through the target camera and then the measurement camera.

9. A robotic arm follow-up hand-eye calibration system without public field of view, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 8 are implemented.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images