Path generation method, system and equipment for hand-eye calibration and medium

Abstract

The invention relates to a path generation method, system and equipment for hand-eye calibration and a medium, and the method comprises the steps: obtaining image data of a calibration plate shot by a camera through a mechanical arm at a plurality of preset positions; and according to the image data, the conversion relation between the camera and the mechanical arm is obtained. The method comprises the steps of determining a height movement range and a horizontal movement range of a camera through a plurality of preset positions, and obtaining a camera movement space range based on the height movement range and the horizontal movement range. And on the basis of the camera motion space range, multiple target camera poses are obtained, and the multiple target camera poses are converted into multiple target mechanical arm poses through the conversion relation. And based on the multiple target mechanical arm poses, a motion path of the mechanical arm is generated. The problem that the safety of path generation of the middle mechanical arm is low is solved. The safety of path generation during mechanical arm hand-eye calibration is improved.

Description

Technical Field

[0001] The present application relates to the technical field of robot hand-eye calibration, and in particular to a path generation method, system, device and medium for hand-eye calibration. Background Art

[0002] In recent years, with the rapid development of science and technology, robotic arm products have been continuously updated and widely used in various production fields, accelerating industrial upgrading and improving productivity. In particular, the application of visual technology has made automation more extensive and practical. When the robotic arm is working, it first performs visual analysis and positioning of the target object, obtains the position of the target object in the workspace, and then performs subsequent operations. This process requires the visual camera and the robotic arm to unify the reference system, that is, to establish the transformation relationship between the camera and the robotic arm coordinate system. This process is called robotic arm hand-eye calibration.

[0003] Robotic arm hand-eye calibration is usually divided into two categories according to the camera installation method: one is that the camera is installed outside the robotic arm, and the camera does not need to move when the robotic arm moves, which is called "eye-outside hand" (Eye-to-Hand); the other is that the camera is installed on the robotic arm, and the robotic arm moves with the camera, which is called "eye-on-hand". Robotic arms can be divided into many types according to the motion structure, and the more common ones are six-axis, four-axis and truss robotic arms. Among them, six-axis robotic arms are more widely used because of their higher degrees of freedom. In the "eye-on-hand" mode of the six-axis robotic arm with a camera installed, its working range can be expanded, flexible and easy to use, so it accounts for a relatively large proportion in actual applications.

[0004] In the "eyes on hands" mode, the hand-eye calibration of the robot arm usually adopts the method of collecting multi-angle rotation and translation calibration. The robot arm is manually moved or automatically generated to confirm several photo points. The more calibration data collected, the more accurate the result. In this process, it is necessary to ensure that the calibration plate is always within the camera's field of view when taking pictures, and safety issues must also be considered. Since the robot arm is equipped with a camera and an end effector, the camera will be connected to a cable, and the end effector may also be connected to a cable or air tube. These cables and air tubes will limit the movable position of part of the robot arm, especially the cables and air tubes of the software will change shape with the movement of the robot arm, and collision detection cannot be performed through the simulation model established in the early stage. Therefore, the existing robot arm path generation has the problem of low safety. Summary of the invention

[0005] The embodiments of the present application provide a path generation method, system, device and medium for hand-eye calibration, so as to at least solve the problem of low security in robot arm path generation in the related art.

[0006] In a first aspect, an embodiment of the present application provides a path generation method for hand-eye calibration, wherein a camera is fixedly mounted on a robotic arm and moves with the robotic arm, and the method comprises:

[0007] Acquire image data of a calibration plate captured by the camera at multiple preset positions by the robotic arm;

[0008] According to the image data, acquiring a conversion relationship between the camera and the robotic arm;

[0009] Determine the height motion range and the horizontal motion range of the camera through the plurality of preset positions, and obtain the camera motion space range based on the height motion range and the horizontal motion range;

[0010] Based on the camera motion space range, a plurality of target camera poses are acquired, and the plurality of target camera poses are converted into a plurality of target robotic arm poses through the conversion relationship;

[0011] Based on the multiple target robot arm positions, a motion path of the robot arm is generated.

[0012] In one embodiment, the mechanical arm is controlled to move to a first position above the calibration plate, and the calibration plate is photographed by the camera to obtain first image data;

[0013] Based on the first position, the robotic arm is controlled to move vertically to a second position, and the calibration plate is photographed by the camera to obtain second image data, wherein the second position is the highest position of the robotic arm;

[0014] On the basis of the second position, the mechanical arm is moved around the transverse axis to a third position, and the calibration plate is photographed by the camera to obtain third image data;

[0015] On the basis of the third position, the mechanical arm is moved around the transverse axis to a fourth position, and the calibration plate is photographed by the camera to obtain fourth image data;

[0016] The first image data, the second image data, the third image data and the fourth image data are used as image data of the calibration plate photographed by the robot arm through the camera at multiple preset positions.

[0017] In one embodiment, acquiring the conversion relationship between the camera and the robotic arm according to the image data includes:

[0018] According to the image data, a conversion relationship is obtained by the formula AX=XB, wherein A is the conversion relationship between the first posture and the second posture of the camera relative to the calibration plate, X is the conversion relationship between the camera and the end of the manipulator, and B is the conversion relationship between the first posture and the second posture of the end of the manipulator relative to the base of the manipulator;

[0019] The conversion relationship is used to convert the camera posture into the robot arm posture.

[0020] In one embodiment, the plurality of preset positions include a first position, a second position, a third position, and a fourth position;

[0021] The step of determining the height motion range and the horizontal motion range of the camera through the plurality of preset positions, and acquiring the camera motion space range based on the height motion range and the horizontal motion range, comprises:

[0022] Determining a height motion range of the camera through the first position and the second position;

[0023] Taking the second position as the center, and based on the third position and the fourth position, determining a horizontal motion range of the camera;

[0024] The camera motion space range is acquired based on the height motion range and the horizontal motion range.

[0025] In one embodiment, during hand-eye calibration, according to the height motion range and the horizontal motion range, the motion path of the robotic arm is in a shuttle shape.

[0026] In one embodiment, the acquiring of a plurality of target camera poses based on the camera motion space range, and the conversion of the plurality of target camera poses into a plurality of target robotic arm poses through the conversion relationship, comprises:

[0027] Acquire multiple camera poses within the camera motion space;

[0028] In the plurality of camera poses, obtaining an angle between the z-axis of the calibration plate coordinate system and the camera optical axis, adjusting the camera pose based on the angle, and obtaining a plurality of target camera poses so that the camera optical axis is aligned with the center of the calibration plate;

[0029] Based on the multiple target camera postures, the multiple target camera postures are converted into multiple target robot arm postures according to the conversion relationship.

[0030] In one embodiment, the acquiring of multiple camera poses within the camera motion space includes:

[0031] The height motion range includes multiple horizontal motion ranges; wherein,

[0032] The horizontal motion range is centered at the second position and is divided into a plurality of equally divided points, each of the equally divided points is used as a camera pose, and a plurality of camera poses within the camera motion space range are obtained.

[0033] In a second aspect, an embodiment of the present application provides a path generation system for hand-eye calibration, wherein a camera is fixedly mounted on a robotic arm and moves with the robotic arm, and the system comprises an image data module, a conversion relationship module, a camera motion space range module, a target robotic arm posture module, and a motion path module, wherein:

[0034] An image data module, used for acquiring image data of a calibration plate photographed by the camera at a plurality of preset positions by the robotic arm;

[0035] A conversion relationship module, used to obtain a conversion relationship between the camera and the robotic arm according to the image data;

[0036] A camera motion space range module, used for the camera motion space range module, used to determine the height motion range and the horizontal motion range of the camera through the plurality of preset positions, and obtain the camera motion space range based on the height motion range and the horizontal motion range;

[0037] A target manipulator arm posture module is used to obtain a plurality of target camera postures based on the camera motion space range, and convert the plurality of target camera postures into a plurality of target manipulator arm postures through the conversion relationship;

[0038] The motion path module generates a motion path of the robot arm based on the multiple target robot arm postures.

[0039] In a third aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a path generation method for hand-eye calibration as described in the first aspect above is implemented.

[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a path generation method for hand-eye calibration as described in the first aspect above.

[0041] The embodiments of the present application provide a path generation method, system, device and medium for hand-eye calibration, which have at least the following technical effects.

[0042] By obtaining the image data of the calibration plate taken by the robot arm through the camera at multiple preset positions. According to the image data, the conversion relationship between the camera and the robot arm is obtained. Through multiple preset positions, the height movement range and horizontal movement range of the camera are determined, and the camera movement space range is obtained based on the height movement range and the horizontal movement range. Based on the camera movement space range, multiple target camera postures are obtained, and the multiple target camera postures are converted into multiple target robot arm postures through the conversion relationship. Based on multiple target robot arm postures, the movement path of the robot arm is generated. By teaching the preset positions (four limited positions), the safety problem of being unable to predict some dangerous positions when the existing fully automatic path is generated is avoided, and then the automatic data collection path is generated according to the limited positions taught, avoiding the time-consuming and laborious problem of manually moving the robot arm to collect data. The problem of low safety in the robot arm path generation in the related technology is solved. The safety of path generation during robot hand-eye calibration is improved.

[0043] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0045] Figure 1 is a flow chart of a path generation method for hand-eye calibration according to an embodiment of the present application;

[0046] Figure 2 is a flowchart of step S101 according to an exemplary embodiment;

[0047] Figure 3 is a schematic structural diagram of a camera and a calibration plate according to an exemplary embodiment;

[0048] Figure 4 is a flowchart of step S103 according to an exemplary embodiment.

[0049] Figure 5 is a flowchart of step S104 according to an exemplary embodiment.

[0050] Figure 6 is a structural block diagram of a path generation system for hand-eye calibration according to an embodiment of the present application;

[0051] Figure 7 is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0053] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0054] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0055] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0056] In a first aspect, an embodiment of the present application provides a path generation method for hand-eye calibration. Figure 1 is a flow chart for path generation for hand-eye calibration, such as Figure 1 As shown, the camera is fixedly mounted on the robotic arm and moves with the robotic arm. A path generation method for hand-eye calibration includes:

[0057] Step S101, obtaining image data of a calibration plate captured by a robot arm at multiple preset positions through a camera.

[0058] Step S102: Obtain the conversion relationship between the camera and the robotic arm according to the image data.

[0059] Step S103: determine the height motion range and horizontal motion range of the camera through multiple preset positions, and obtain the camera motion space range based on the height motion range and the horizontal motion range.

[0060] Step S104: based on the camera motion space range, multiple target camera poses are acquired, and the multiple target camera poses are converted into multiple target robotic arm poses through a conversion relationship.

[0061] Step S105: Generate a motion path of the robot arm based on multiple target robot arm postures.

[0062] In summary, the embodiment of the present application provides a path generation method for hand-eye calibration, by obtaining image data of a calibration plate taken by a camera at multiple preset positions of a robotic arm. According to the image data, the conversion relationship between the camera and the robotic arm is obtained. Through multiple preset positions, the height motion range and horizontal motion range of the camera are determined, and the camera motion space range is obtained based on the height motion range and the horizontal motion range. Based on the camera motion space range, multiple target camera postures are obtained, and through the conversion relationship, the multiple target camera postures are converted into multiple target robotic arm postures. Based on multiple target robotic arm postures, the motion path of the robotic arm is generated. By teaching the preset positions (four limited positions), the safety problem of being unable to predict some dangerous positions when the existing fully automatic path is generated is avoided, and then the automatic data collection path is generated according to the limited positions taught, avoiding the time-consuming and laborious problem of manually moving the robotic arm to collect data. The problem of low safety in the path generation of the robotic arm in the related art is solved. The safety of path generation during the hand-eye calibration of the robotic arm is improved.

[0063] Figure 2 is a flowchart of step S101 according to an exemplary embodiment. Figure 2 As shown, step S101, obtaining image data of a calibration plate captured by a camera at multiple preset positions by a robotic arm. Specifically, the following steps are included:

[0064] Step S1011: control the robot arm to move to a first position above the calibration plate, and photograph the calibration plate through a camera to obtain first image data.

[0065] Step S1012: Based on the first position, control the robotic arm to move vertically to a second position, and photograph the calibration plate through a camera to obtain second image data, wherein the second position is the highest position of the robotic arm.

[0066] Step S1013: on the basis of the second position, move the robot arm around the transverse axis to a third position, and photograph the calibration plate through the camera to obtain third image data.

[0067] Step S1014: Based on the third position, the robot arm is moved around the transverse axis to a fourth position, and the calibration plate is photographed by a camera to obtain fourth image data.

[0068] Step S1015: Use the first image data, the second image data, the third image data and the fourth image data as image data of the calibration plate captured by the robot arm at multiple preset positions through the camera.

[0069] Optionally, Figure 3 is a schematic diagram of the structure of a camera and a calibration board according to an exemplary embodiment. Figure 3 As shown, the positions of the camera and the calibration plate are distributed in an umbrella-shaped space. Figure 3The middle circle is the position that the camera needs to reach, and the black line is the camera motion path. Since the camera and the end flange of the robot are rigidly mounted together, the camera's motion path or posture can be approximately equivalent to the end flange motion path and posture of the robot. When teaching, you only need to control the robot to move the solid circle position, because the two solid circles of the first and second positions limit the vertical space and direction, and then teach the solid circles of the third and fourth positions to limit the horizontal space and direction.

[0070] Specifically, the first position is the point where the camera is located at the closest distance to the depth of field in front of the calibration plate or the lowest point of the manipulator movement. When any of the limiting conditions is met, the manipulator will no longer continue to move, and the first position is determined (that is, the camera at the closest distance point of the depth of field and the lowest point of the manipulator movement can take a clear and complete calibration plate); the second position is the point where the camera is located at the farthest distance point of the depth of field behind the calibration plate or the highest point of the manipulator movement. When any of the limiting conditions is met, the manipulator will no longer continue to move, and the second position is determined (that is, the camera at the farthest distance point of the depth of field and the highest point of the manipulator movement can take a clear and complete calibration plate). The third position is within the extreme posture range that the manipulator can reach under the shooting conditions; the fourth position is also within the extreme posture range in another direction. The first position to the fourth position determines the camera working range and the manipulator movement range, thereby generating a path within these limiting conditions. In addition, the solid circles at the third position and the fourth position can be used to determine the axis, and the solid circles can be symmetrically centered to obtain two other hollow circles, and then spatial interpolation can be performed between any two circles (including solid circles and hollow circles) in the horizontal direction to obtain a circle of points. It is worth noting that the height movement range includes multiple first positions to fourth positions, that is, the height movement range includes multiple horizontal movement ranges.

[0071] Control the robot arm to move the camera to the first position of the calibration plate, and shoot the calibration plate to obtain the first image data, then lift the robot arm vertically to the second position, shoot the calibration plate, obtain the second image data, and then change the posture of the robot arm (i.e., move around the axis) to Figure 3 The third position on the side, and shoot the calibration plate, obtain the third image data, and finally change the posture to Figure 3 The fourth position on the side, and shoot the calibration plate to obtain the fourth image data. In this process, the camera optical axis always passes through the center of the calibration plate, which means that the calibration plate is always in the center of the camera frame. Among them, the third and fourth teaching positions are the limit positions of the robot arm, that is, further changes will cause collision or pulling of the cable.

[0072] Step S101: By acquiring images at multiple positions and postures, high-precision relative position and posture calibration between the camera and the robot can be achieved. The changes in multiple positions and postures ensure that the robot's workspace is fully covered during the calibration process. The extreme position of the robot is selected as the teaching position to ensure that no collision or cable pulling occurs during the calibration process, thereby improving the safety of path generation during the hand-eye calibration of the robot. The calibration results can be directly applied to the actual operation of the robot to improve its positioning accuracy and work efficiency.

[0073] In one embodiment, step S102, obtaining a conversion relationship between the camera and the robotic arm according to the image data, specifically includes:

[0074] According to the image data, the conversion relationship is obtained through the formula AX=XB, where A is the conversion relationship between the first and second poses of the camera relative to the calibration plate, X is the conversion relationship between the camera and the end of the robotic arm, and B is the conversion relationship between the first and second poses of the end of the robotic arm relative to the base of the robotic arm;

[0075] The transformation relationship is used to transform the camera pose into the robot arm pose.

[0076] Optionally, in the formula AX=XB, A represents the first and second pose conversion relationships of the camera relative to the calibration plate. Specifically, A is a plurality of sets of position conversion relationships from the first position to the second position, from the second position to the third position, from the third position to the fourth position, etc., X represents the fixed conversion relationship (i.e., conversion relationship) of the camera relative to the end of the manipulator, and B represents a plurality of sets of position conversion relationships from the first position to the second position, from the second position to the third position, from the third position to the fourth position, etc., of the end of the manipulator relative to the base of the manipulator. Pose transformation: By collecting image data at different positions, the transformation matrix A of the camera relative to the calibration plate and the transformation matrix B of the end of the manipulator relative to the base of the manipulator can be calculated. Through multiple sets of A and B data, the fixed conversion relationship X between the camera and the end of the manipulator can be solved.

[0077] Step S102 uses the formula AX=XB to solve the fixed transformation relationship X between the camera and the end of the robotic arm from multiple sets of posture data. Since it is calculated based on multiple position and posture data points, this transformation relationship is highly accurate. Based on the obtained transformation relationship X, the posture of the camera can be directly converted to the posture of the robotic arm. That is, the movement of the robotic arm can be guided by the image data of the camera.

[0078] Figure 4 is a flowchart of step S103 according to an exemplary embodiment. Figure 4As shown, step S103, through multiple preset positions, determine the height movement range and horizontal movement range of the camera, and obtain the camera movement space range based on the height movement range and the horizontal movement range. Specifically, it includes the following steps:

[0079] Step S1031, determining the height movement range of the camera through the first position and the second position.

[0080] Step S1032: Determine the horizontal motion range of the camera based on the third position and the fourth position, with the second position as the center.

[0081] Step S1033: Acquire the camera motion space range based on the height motion range and the horizontal motion range.

[0082] Optionally, the camera's height motion range in the vertical direction can be determined by the first position and the second position (the highest position of the robotic arm). The camera's motion range in the horizontal direction is determined based on the third position and the fourth position (the third position and the fourth position rotated around the transverse axis) with the second position as the center.

[0083] Step S103 specifies the maximum and minimum heights that the camera can move, ensuring that the robot arm does not exceed these limits in the vertical direction. This prevents the robot arm from overextending or pressing down in the vertical direction, and avoids collision or damage with other objects in the environment. The horizontal range of motion allows the camera to move freely within a certain range, avoiding collisions or cable pulling caused by excessive rotation of the robot arm in the horizontal direction. This provides clear boundary conditions for the path planning of the robot arm.

[0084] Figure 5 is a flowchart of step S104 according to an exemplary embodiment. Figure 5 As shown, step S104, based on the camera motion space range, obtain multiple target camera poses, and convert the multiple target camera poses into multiple target robot arm poses through conversion relationships. Specifically, it includes the following steps:

[0085] Step S1041, obtaining multiple camera poses within the camera motion space;

[0086] Step S1042: among the multiple camera poses, obtain the angle between the z-axis of the calibration plate coordinate system and the camera optical axis, adjust the camera pose based on the angle, and obtain multiple target camera poses so that the camera optical axis is aligned with the center of the calibration plate;

[0087] Step S1043: Based on the multiple target camera poses, the multiple target camera poses are converted into multiple target robotic arm poses according to the conversion relationship.

[0088] Optionally, multiple camera poses are obtained within the determined camera motion space range (height motion range and horizontal shuttle motion range). These poses can be randomly distributed or evenly distributed according to specific task requirements. For each selected camera pose, the angle between the Z axis of the calibration plate coordinate system and the camera optical axis is calculated. Based on the calculated angle, the camera pose is adjusted so that the camera optical axis is aligned with the center of the calibration plate. Adjusting the camera pose includes rotation and translation operations. Using the conversion relationship X obtained in step S103 (the fixed conversion relationship between the camera and the end of the robotic arm calculated by the formula AX=XB), the adjusted target camera pose is converted to the corresponding target robotic arm pose.

[0089] Step S104, by selecting multiple postures within the range of motion space, it can be ensured that the subsequent adjustment and calibration process can cover the entire working area. Providing rich posture data helps to more accurately calculate and verify the conversion relationship between the camera and the robotic arm. By adjusting the camera posture, the camera optical axis is always aligned with the center of the calibration plate, which improves the accuracy and consistency of image acquisition. Based on the converted target robotic arm posture, effective path planning can be performed to ensure that the robotic arm can move along the predetermined path.

[0090] In one embodiment, obtaining multiple camera poses within the camera motion space specifically includes:

[0091] The height range of motion includes multiple horizontal ranges of motion; among them,

[0092] The horizontal motion range is centered at the second position and is divided into a plurality of equally divided points. Each equally divided point is used as a camera pose, and a plurality of camera poses within the camera motion space range are obtained.

[0093] Optionally, the height motion range (from the first position to the second position) is divided into several height levels. For example, the height range can be divided into 5 or more levels. Divide the horizontal motion range: For each height level, a horizontal motion range is divided in the horizontal direction with the second position as the center. Within the horizontal motion range of each height level, it is divided into a plurality of equally divided points. These equally divided points are evenly distributed in the shuttle-shaped area to ensure that the entire horizontal range is covered. Each equally divided point represents a camera pose. In this way, multiple camera poses can be generated within the entire camera motion space. It is worth noting that since the first position and the second position are the height motion ranges, and the third position and the fourth position are the horizontal motion ranges obtained by rotating around the axis, therefore, during the hand-eye calibration, according to the height motion range and the horizontal motion range, the motion path shape of the robotic arm is shuttle-shaped, presenting a shuttle-shaped area.

[0094] By selecting multiple poses within the motion space, it can be ensured that the subsequent adjustment and calibration process can cover the entire working area, providing rich pose data.

[0095] In summary, the embodiment of the present application provides a path generation method for hand-eye calibration, by obtaining image data of a calibration plate taken by a camera at multiple preset positions of a robotic arm. According to the image data, the conversion relationship between the camera and the robotic arm is obtained. Through multiple preset positions, the height motion range and horizontal motion range of the camera are determined, and the camera motion space range is obtained based on the height motion range and the horizontal motion range. Based on the camera motion space range, multiple target camera postures are obtained, and through the conversion relationship, the multiple target camera postures are converted into multiple target robotic arm postures. Based on multiple target robotic arm postures, the motion path of the robotic arm is generated. By teaching the preset positions (four limited positions), the safety problem of being unable to predict some dangerous positions when the existing fully automatic path is generated is avoided, and then the automatic data collection path is generated according to the limited positions taught, avoiding the time-consuming and laborious problem of manually moving the robotic arm to collect data. The problem of low safety in the path generation of the robotic arm in the related art is solved. The safety of path generation during the hand-eye calibration of the robotic arm is improved.

[0096] In a second aspect, an embodiment of the present application provides a system for path generation for hand-eye calibration. Figure 6 FIG. 1 is a block diagram of a system for path generation for hand-eye calibration according to an exemplary embodiment. Figure 6 As shown, the system includes an image data module 610, a conversion relationship module 620, a camera motion space range module 630, a target manipulator position module 640 and a motion path module 650, wherein:

[0097] The image data module 610 is used to obtain image data of the calibration plate photographed by the robot arm through a camera at multiple preset positions;

[0098] The conversion relationship module 620 is used to obtain the conversion relationship between the camera and the robotic arm according to the image data;

[0099] The camera motion space range module 630 is used for the camera motion space range module, which is used to determine the height motion range and horizontal motion range of the camera through multiple preset positions, and obtain the camera motion space range based on the height motion range and the horizontal motion range;

[0100] The target manipulator pose module 640 is used to obtain multiple target camera poses based on the camera motion space range, and convert the multiple target camera poses into multiple target manipulator poses through a conversion relationship;

[0101] The motion path module 650 generates a motion path of the robot arm based on multiple target robot arm postures.

[0102] In summary, the embodiment of the present application provides a path generation system for hand-eye calibration, which solves the problem of low safety of the robot arm path generation in the related art through the image data module 610, the conversion relationship module 620, the camera motion space range module 630, the target robot arm posture module 640 and the motion path module 650. Specifically, the image data of the calibration plate taken by the robot arm at multiple preset positions is obtained through the camera. According to the image data, the conversion relationship between the camera and the robot arm is obtained. Through multiple preset positions, the height motion range and the horizontal motion range of the camera are determined, and the camera motion space range is obtained based on the height motion range and the horizontal motion range. Based on the camera motion space range, multiple target camera postures are obtained, and through the conversion relationship, the multiple target camera postures are converted into multiple target robot arm postures. Based on multiple target robot arm postures, the motion path of the robot arm is generated. By teaching the preset positions (four limited positions), the safety problem of being unable to predict some dangerous positions when the existing fully automatic path is generated is avoided, and then the automatic data acquisition path is generated according to the limited positions taught, which avoids the time-consuming and laborious problem of manually moving the robot arm to collect data. The invention solves the problem of low security in the path generation of the robot arm in the related art and improves the security of the path generation during the hand-eye calibration of the robot arm.

[0103] It should be noted that the system for path generation for hand-eye calibration provided in this embodiment is used to implement the above-mentioned implementation mode, and the descriptions that have been made will not be repeated. As used above, the terms "module", "unit", "subunit", etc. can implement a combination of software and / or hardware for a predetermined function. Although the device described in the above embodiment is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0104] In a third aspect, an embodiment of the present application provides an electronic device, Figure 7 FIG. 1 is a block diagram of an electronic device according to an exemplary embodiment. Figure 7 As shown, the electronic device may include a processor 71 and a memory 72 storing computer program instructions.

[0105] Specifically, the processor 71 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0106] Among them, the memory 72 may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory 72 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 72 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 72 may be inside or outside the data processing device. In a specific embodiment, the memory 72 is a non-volatile memory. In a specific embodiment, the memory 72 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable ProgrammableRead-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable ProgrammableRead-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0107] The memory 72 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 71 .

[0108] The processor 71 implements any one of the path generation methods for hand-eye calibration in the above embodiments by reading and executing computer program instructions stored in the memory 72 .

[0109] In one embodiment, a device for path generation for hand-eye calibration may further include a communication interface 73 and a bus 70. Figure 7 As shown, the processor 71, the memory 72, and the communication interface 73 are connected via a bus 70 and communicate with each other.

[0110] The communication interface 73 is used to implement communication between the modules, devices, units and / or equipment in the embodiment of the present application. The communication port 73 can also implement data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0111] The bus 70 includes hardware, software or both, coupling components of a device for path generation for hand-eye calibration to each other. The bus 70 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 70 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of the above. Where appropriate, bus 70 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0112] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a program stored thereon, and when the program is executed by a processor, a path generation method for hand-eye calibration provided in the first aspect is implemented.

[0113] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0114] In a possible implementation, the present invention can also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of a path generation method for hand-eye calibration provided in the first aspect.

[0115] Among them, the program code for executing the present invention can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or completely on the remote device.

[0116] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A path generation method for hand-eye calibration, characterized in that: The camera is fixedly mounted on the robotic arm and moves with the robotic arm. The method includes: Acquire image data of a calibration plate captured by the camera at multiple preset positions by the robotic arm; According to the image data, acquiring a conversion relationship between the camera and the robotic arm; Determine the height motion range and the horizontal motion range of the camera through the plurality of preset positions, and obtain the camera motion space range based on the height motion range and the horizontal motion range; Based on the camera motion space range, a plurality of target camera poses are acquired, and the plurality of target camera poses are converted into a plurality of target robotic arm poses through the conversion relationship; Based on the multiple target robot arm positions, a motion path of the robot arm is generated.

2. The method according to claim 1, characterized in that The step of obtaining image data of a calibration plate captured by the camera at a plurality of preset positions by the robotic arm includes: Controlling the mechanical arm to move to a first position above the calibration plate, and photographing the calibration plate through the camera to obtain first image data; Based on the first position, the robotic arm is controlled to move vertically to a second position, and the calibration plate is photographed by the camera to obtain second image data, wherein the second position is the highest position of the robotic arm; On the basis of the second position, the mechanical arm is moved around the transverse axis to a third position, and the calibration plate is photographed by the camera to obtain third image data; On the basis of the third position, the mechanical arm is moved around the transverse axis to a fourth position, and the calibration plate is photographed by the camera to obtain fourth image data; The first image data, the second image data, the third image data and the fourth image data are used as image data of the calibration plate photographed by the robot arm through the camera at multiple preset positions.

3. The method according to claim 1, characterized in that The step of obtaining a conversion relationship between the camera and the robotic arm according to the image data includes: According to the image data, a conversion relationship is obtained by the formula AX=XB, wherein A is the conversion relationship between the first posture and the second posture of the camera relative to the calibration plate, X is the conversion relationship between the camera and the end of the manipulator, and B is the conversion relationship between the first posture and the second posture of the end of the manipulator relative to the base of the manipulator; The conversion relationship is used to convert the camera posture into the robot arm posture.

4. The method according to claim 1, characterized in that The plurality of preset positions include a first position, a second position, a third position and a fourth position; The step of determining the height motion range and the horizontal motion range of the camera through the plurality of preset positions, and acquiring the camera motion space range based on the height motion range and the horizontal motion range, comprises: Determining a height motion range of the camera through the first position and the second position; Taking the second position as the center, and based on the third position and the fourth position, determining a horizontal motion range of the camera; The camera motion space range is acquired based on the height motion range and the horizontal motion range.

5. The method according to claim 4, characterized in that The step of acquiring a plurality of target camera poses based on the camera motion space range, and converting the plurality of target camera poses into a plurality of target robotic arm poses through the conversion relationship comprises: Acquire multiple camera poses within the camera motion space; In the plurality of camera poses, obtaining an angle between the z-axis of the calibration plate coordinate system and the camera optical axis, adjusting the camera pose based on the angle, and obtaining a plurality of target camera poses so that the camera optical axis is aligned with the center of the calibration plate; Based on the multiple target camera postures, the multiple target camera postures are converted into multiple target robot arm postures according to the conversion relationship.

6. The method according to claim 5, characterized in that The acquiring of multiple camera poses within the camera motion space includes: The height motion range includes a plurality of the horizontal motion ranges; wherein, The horizontal motion range is centered at the second position and is divided into a plurality of equally divided points, each of the equally divided points is used as a camera pose, and a plurality of camera poses within the camera motion space range are obtained.

7. The method according to claim 6, characterized in that During hand-eye calibration, according to the height motion range and the horizontal motion range, the motion path of the robotic arm is in the shape of a shuttle.

8. A path generation system for hand-eye calibration, characterized in that: The camera is fixedly mounted on the robot arm and moves with the robot arm. The system includes an image data module, a conversion relationship module, a camera motion space range module, a target robot arm position module and a motion path module, wherein: The image data module is used to obtain image data of the calibration plate photographed by the camera at multiple preset positions by the robotic arm; The conversion relationship module is used to obtain the conversion relationship between the camera and the mechanical arm according to the image data; The camera motion space range module is used for the camera motion space range module, which is used to determine the height motion range and the horizontal motion range of the camera through the plurality of preset positions, and obtain the camera motion space range based on the height motion range and the horizontal motion range; The target manipulator arm posture module is used to obtain multiple target camera postures based on the camera motion space range, and convert the multiple target camera postures into multiple target manipulator arm postures through the conversion relationship; The motion path module generates a motion path of the robot arm based on the multiple target robot arm postures.

9. An electronic device, characterized in that: The invention comprises a memory and a processor, a computer program stored in the memory and executable on the processor, and the processor implements a path generation method for hand-eye calibration as claimed in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a path generation method for hand-eye calibration as described in any one of claims 1 to 7 is implemented.

Images