111 results about "Hand eye calibration" patented technology

The invention discloses a robot grasping pose estimation method based on an object recognition depth learning model, which relates to the technical field of computer vision. The method is based on anRGBD camera and depth learning. The method comprises the following steps: S1, camera parameter calibration and hand-eye calibration being carried out; S2, training an object detection object model; S3, establishing a three-dimensional point cloud template library of the target object; 4, identifying the types and position of each article in the area to be grasped; 5, fusing two-dimensional and three-dimensional vision information and obtaining a point cloud of a specific target object; 6, completing the pose estimation of the target object; S7: adopting an error avoidance algorithm based on sample accumulation to avoid errors; S8: Steps S4 to S7 being continuously repeated by the vision system in the process of moving the robot end toward the target object, so as to realize iterative optimization of the pose estimation of the target object. The algorithm of the invention utilizes a target detection YOLO model to carry out early-stage fast target detection, reduces the calculation amount of three-dimensional point cloud segmentation and matching, and improves the operation efficiency and accuracy.

The embodiment of the invention provides a mechanical arm tail end camera hand-eye calibration method and system, and the method comprises the steps: obtaining a calibration image collected by a calibration area when a mechanical arm performs translation and rotation for multiple times at the same time; Obtaining a hand-eye calibration matrix based on the mechanical arm pose information and the calibration image external parameter information; And calibrating the hand-eye relationship of the tail end of the mechanical arm by using the hand-eye calibration matrix. According to the scheme, the calibration of the camera is completed by utilizing spatial motion and image acquisition at a plurality of different positions; According to the principle that the hand-eye relationship is constrainedby multiple spatial relative positions, only the positions of at least three spatial positions relative to the base and external parameters of the camera need to be obtained in the calibration process, and the calibration process can be universal for different mechanical arm models, the number of freedom degrees and camera models; According to the scheme, translation and rotation matrix transformation is utilized, the mechanical arm tail end coordinates are projected to the pixel coordinates, and calibration of the hand-eye relation between the camera and the mechanical arm tail end when the mechanical arm tail end has rotation and translation movement at the same time is obtained.

The invention relates to a robot hand eye calibration method and a robot hand eye calibration device based on a standard ball. The method comprises the steps of acquiring point cloud information of the standard ball based on a camera coordinate system, and position information of a robot TCP in a robot-based coordinate system; positioning a ball surface of the standard ball according to the pointcloud information, determining a ball center position of the standard ball, and determining a three-dimensional coordinate value of the ball center of the standard ball in the camera coordinate systembased on a maximum likelihood estimation algorithm according to the ball center position; and determining an overdetermined transformation matrix equation of the three-dimensional coordinate value from the camera coordinate system to the robot-based coordinate system, and utilizing a least square optimization algorithm for determining a homogeneous transformation matrix of the camera coordinate system in the robot-based coordinate system, so that robot hand eye calibration is realized. According to the robot hand eye calibration method and the robot hand eye calibration device based on the standard ball provided by the invention, the robot hand eye calibration is realized by adopting the standard ball, no internal parameters of a camera is required to be calibrated, and the calibration method is simple, high in efficiency and suitable for hand eye calibration operation of various robots.

The invention provides a full-automatic hand-eye calibration method and device and relates to the technical field of robot calibration. The full-automatic hand-eye calibration method comprises the steps that the current pose of a mechanical arm is obtained; the next pose of the mechanical arm is obtained according to the current pose of the mechanical arm and the preset movement path; calibration board data under all the poses of the mechanical arm are obtained; and according to the poses of the mechanical arm and the calibration board data, the hand-eye calibration result of a camera and the mechanical arm is obtained. According to the full-automatic hand-eye calibration method and device, a simpler, more accurate and collisionless automatic calibration scheme is provided for hand-eye calibration of a robot; a calibration system does not need extra auxiliary hardware equipment except a calibration board, and inner reference of the camera and calibration of a transformation matrix of the camera and the mechanical arm can be simultaneously completed only by automatically controlling the mechanical arm to move with the calibration board or the camera as the center.

The invention provides a hand-eye calibration method for a laser line structured light sensor. The hand-eye calibration method includes the steps that a sensor coordinate system is established on the sensor, a structured light stripe is generated from a structured light plane projected by a mid-laser device of the sensor to a target, the pose of a robot is adjusted, the stripe and a straight line on a calibration template are intersectant to form feature points, and a pose parameter of the robot and output data of the sensor of this moment are recorded; outputting of the sensor is combined with geometrical parameters of the sensor, and three-dimensional coordinates, under the sensor coordinate system, of all the points on the laser stripe can be obtained; the feature points generated when the stripe and the straight line on the calibration template are intersectant are extracted. The process is repeated, N sets of different robot pose parameters and the feature points corresponding to all the parameters are obtained, the calibration problem is converted into the least square problem through N sets of data collected under a constraint of space straight lines, and the problem is solved through the optimization algorithm. The hand-eye calibration method is simple and convenient and easy to conduct, the position and the pose are accurately controlled, the efficiency is high, the quality is good, and the hand-eye calibration method has the high practical value.

The invention discloses a camera and robot hand-eye calibration method based on an ROS. According to the camera and robot hand-eye calibration method based on the ROS, through setting up of a visual system, a mechanical arm and an operation object of the mechanical arm are within the view field of the camera; then, communication mechanisms between the camera and the ROS and between the mechanicalarm and the ROS are established, on one hand, motion control over the mechanical arm is achieved, on the other hand, the images of the camera and the state of the mechanical arm are obtained, and datacollection is conducted; calibration of internal reference and external reference of the camera is conducted, and parameters of the camera are obtained; and finally, according to different installation manners of the camera, hand-eye calibration of the camera and the mechanical arm is conducted, and a hand-eye calibration matrix is obtained. By the adoption of the camera and robot hand-eye calibration method based on the ROS, automatic hand-eye calibration of the camera and the mechanical arm is achieved, manual intervention is reduced, terminal commands are just called twice in the whole calibration process, the autonomous degree is high, the limitation of the type and number of the camera is avoided, the limitation of the type of mechanical arm is avoided, the expandability is high, itis just needed to print a piece of A4 checkerboard paper in the calibration process, no mark is needed, and the camera and robot hand-eye calibration method based on the ROS is convenient to perform and practical.

The invention belongs to the technical field of automatic welding, and provides an intelligent three-dimensional weld joint autonomous tracking method which comprises the following steps: S1, camera calibration: a camera shoots and acquires image information of calibration plates at different spatial positions, establishes a corresponding equation set according to the shot image information, and obtains an internal parameter matrix and an external parameter matrix through analysis; s2, line laser plane calibration; s3, hand-eye calibration; s4, determination of a welding seam starting point and a welding seam ending point; and s5, welding seam track planning. According to the intelligent three-dimensional autonomous welding seam tracing method, any type of weld grooves is identified by utilizing an image algorithm; a linear laser triangulation method and a vision and robot calibration algorithm are adopted to autonomously plan a motion path, so that autonomous tracking of a welding line in any three-dimensional space is realized, three-dimensional orientation information of a welding line groove is obtained, and tracking of a right-angle or acute-angle inflection point is realized in a self-adaptive pre-swing manner.

The invention discloses a hand-eye calibration method of an industrial robot; the hand-eye calibration method can deploy a camera and a calibration plate, a plurality of calibration data collection points are set in a working plane area of the industrial robot and the coordinates in the industrial robot base coordinate system are recorded; and then the tail end of the operating arm of the industrial robot is controlled to drive the calibration plate to rotate around a central shaft at the tail end of the operating arm; during this process, the camera is controlled to collect the image of the calibration plate at different positions extract the calibration point from the image; the coordinates of the center of the circle are calculated according to the coordinates of the same calibration point in the calibration plate image at different positions; and the coordinate is the coordinate of the tail end of the operating arm in the camera image coordinate system, so the coordinate mapping data of the industrial robot coordinate system and the camera image coordinate system is obtained; the conversion matrix between the industrial robot base coordinate system and the camera image coordinate system can be calculated according to the data, so the automatic hand-eye calibration of the industrial robot is realized.

The invention discloses a calibration system and calibration method for assisting laser osteotomy robots. The method includes obtaining a transfer matrix of a mechanical arm end coordinate system relative to a robot base coordinate system and the transfer matrix of a tracking device coordinate system relative to a camera coordinate system; solving the translation vector of a tool coordinate systemrelative to the mechanical arm end coordinate system, and establishing the transfer matrix of the tool coordinate system relative to the mechanical arm end coordinate system; obtaining the transfer matrix of the tool coordinate system relative to the robot base coordinate system so that the tool calibration of a robot can be realized; and solving the translation vector of the tool coordinate system relative to the tracking device coordinate system, obtaining two sets of three-dimensional point sets of tool coordinate system original points, and solving the transfer matrix of the robot base coordinate system relative to the camera coordinate system so that the hand-eye calibration of the robot can be realized.

The invention provides an automatic calibration device of a robot monocular vision guiding system. The automatic calibration device of the robot monocular vision guiding system comprises a mechanical arm driving module, a camera communication module, an image acquisition module, a camera calibration module, a hand-eye calibration module and an interactive interface, wherein the image acquisition module is used for managing movement of a mechanical arm during calibration and acquiring calibration plate images shot by a camera; the camera calibration module transfers a calibration plate image set in the image acquisition module, so that corresponding camera internal reference, external reference and lens distortion coefficients are generated; and the hand-eye calibration module transfers the calibration plate image set in the image acquisition module and the corresponding camera internal reference, external reference and lens distortion coefficients in the camera calibration module, and a transformation relation between a coordinate system of the camera and a coordinate system of the tail end of the mechanical arm is obtained by calculation.

The invention provides a high-precision method for hand-eye calibration of a fixed camera vision system of an SCARA mechanical arm. The high-precision method comprises the following steps that a target is adhered to one end of a strip-shaped tool, the other end of the strip-shaped tool is fixed to the end of the mechanical arm, a camera is fixed at a fixed position of the working range of the mechanical arm, and the camera shooting surface and the target are parallel to the working plane; rough calibration is carried out, wherein the mechanical arm drives the target to move and rotate within apreset small range, and an initial mapping relation between an image coordinate system and a mechanical arm coordinate system is established according to the position of the mechanical arm and the identification result of the target at each position; and fine calibration is carried out, wherein a plurality of target image positions are preset in the image coordinate system, the moving position corresponding to the mechanical arm is calculated on the basis of the initial mapping relation, the corresponding target images are automatically recognized, affine transformation models are constructed, an affine transformation model is converted into the optimal solution problem, and the affine transformation matrix of the image coordinate system to the mechanical arm coordinate system is obtained. The method is high in calibration precision, simple and convenient to operate and capable of realizing one-key calibration.

The embodiment of the invention discloses a joint automatic calibration method and device for a robot vision servo system, and the method comprises the steps: carrying out the camera calibration of the robot vision servo system, and determining the internal parameters and distortion parameters of a camera; performing line structured light plane calibration, and determining line structured light plane parameters; performing hand-eye calibration, and determining a coordinate conversion relation between a second robot end effector and a camera; performing positioning system calibration, and determining a transformation relation between a robot base coordinate system and an infrared laser positioning base station coordinate system; and determining a joint automatic calibration result of the robot vision servo system according to the camera internal parameters, the distortion parameters, the linear structure light plane parameters, the coordinate conversion relationship and the transformation relationship. Firstly, joint calibration of multiple sensing and positioning modes is realized, so that an industrial robot vision servo system is not limited to structured light vision sensing orbinocular vision sensing; secondly, the automation of the calibration process is realized, the manual operation is removed, and the working efficiency is improved.

The invention provides a robot extended hand-eye calibration method. In the method, an assistant video camera is arranged, and a space transition relation between an assistant video camera coordinate system and an extended hand coordinate system and a space transition relation between the assistant video camera coordinate system and a medium-target coordinate system are respectively calculated. And a space transition relation between an eye video camera coordinate system and the medium-target coordinate system is calculated. Then, a space transition relation between the eye video camera and an extended hand coordinates system is calculated according to the space transition relations calculated so as to realize the robot extended hand-eye calibration. The robot extended hand-eye calibration method provided by the invention has low application cost and is easy for operation. And the method can enhance the reliability and convenience of on-site calibration.

The invention discloses a robot dynamic capture method and system. The method comprises the steps that S1, a collected image transmitted from an industrial camera is received in real time, and the obtained image is preprocessed; S2, a template tracking algorithm based on multiple clues is utilized to determine a foreground target region; S3, a template matching algorithm based on an edge gradientis adopted to precisely recognize a foreground target; S4, a random sampling consensus algorithm is adopted to perform modeling on motion information of an object in a view field; and S5, the information is fed back to a robot in real time for dynamic capture in combination with camera calibration and hand-eye calibration parameters. According to the robot dynamic capture method and system, by theadoption of a target tracking algorithm based on multiple clues, the motion target foreground region can be extracted efficiently and accurately; and under the application scene of robot sorting, therunning speed of a visual motion target tracking algorithm can be effectively increased, high-frame-rate real-time recognition is realized, and therefore the precision of visual servo is improved.

The invention discloses an autonomous path planning method of a live-line working robot based on a point cloud. The method comprises the following steps: installing depth cameras at the ends of doublerobot arms; performing hand-eye calibration and base coordinate system calibration of the double robot arms to realize view sharing of the two cameras; controlling base joints of the double robot arms to rotate 360 DEG to acquire multiple frames of images; splicing the acquired images to obtain the complete environmental information of the workspace of the double robot arms; and planning the optimal non-collision movement path of the robot arms in real time by using an improved path planning algorithm, and adding the path to a database. The depth cameras are installed at the ends of the robotarms by adopting an eye-to-hand installation method, and combined with the kinematic model of the robot arms, a three-dimensional point cloud image of the entire operating environment is built, so that the system has the ability to sense environmental obstacles; and the improved path planning algorithm ensures that the robot arms do not collide with obstacles, and also the optimal movement path of the robot arms can be planned in real time.

The invention provides a method for obtaining three-dimensional coordinates of a light bar center based on binocular line structured light in welding detection. The method comprises the steps of (1) building a binocular line structured light visual sensing system, (2) calibrating the binocular line structured light visual sensing system, (3) obtaining images with structural light stripes at the same time during welding by using calibrated binocular line structured light device, (4) mapping pixel coordinates of pixels with non-zero grayscale values in two images to a plane in a three-dimensional space and selecting a light plane or an imaging plane of one camera, (5) judging whether a point is a noise point or a point in a light strip through three-dimensional coordinate points of the pixels in the two images in the same plane in a space coordinate system, and (6) obtaining a pose relationship between a camera and a robot end by hand-eye calibration, and mapping the coordinate valuesof a center line on the light plane to a robot coordinate system to obtain the surface shape of a weld seam in robot coordinates. According to the method, the visual inspection of welding is more accurate.

The invention provides a hand-eye calibration method based on an infrared stereoscopic vision positioning system and a mechanical arm. The method comprises the following steps that 1, a transformationrelation between a mechanical arm tail end flange plate coordinate system {TCP} and a mechanical arm coordinate system {Base} is built; 2, a transformation relation between {TCP} and an external marker local coordinate system {DRF} is built; 3, a transformation relation between {TCP} and an infrared stereoscopic vision positioning system coordinate system {Tracker} is built; and 4, by means of the transformation relation between {TCP} and {Base} and the transformation relation between {TCP} and {Tracker}, a transformation matrix of {Base} relative to {Tracker} is obtained, and therefore hand-eye calibration based on the infrared stereoscopic vision positioning system and the mechanical arm is achieved. The hand-eye calibration method has the advantages of being easy and convenient to implement, practical, flexible, high in precision and the like.

The invention relates to a hand-eye calibration method for a mechanical arm, a calibration plate, a device, equipment and a storage medium. The method comprises the steps that the calibration plate issequentially put at different positions in the view of a scanner, and the tool center point position of a mechanical arm tail tool coincides with the geometric figure center of the calibration platein a world coordinate; and the tool center point positions in a mechanical arm-based coordinate system and the geometric figure center positions in a scanner camera coordinate system are collected simultaneously, a hand-eye relational matrix of the mechanical arm and a calibration error corresponding to the hand-eye relational matrix are calculated and output according to all the collected tool center point positions and all the collected geometric figure center positions, and the calibration error can be used for judging whether the hand-eye relational matrix is accurate or not. By adopting the method, the mechanical arm hand-eye calibration convenience, efficiency and accuracy can be improved, and the mechanical arm hand-eye calibration cost is reduced.

This invention provides a system and method that automatically decides motion parameters of hand-eye calibration, and conducts the calibration procedure with minimal human intervention. It automatically computes hand-eye calibration motion parameters and spatial positions during pre-calibration, whereby the calibration pattern can continuously remain inside, and covering, the entire field of view FOV of the vision system camera, providing a fully automatic hand-eye calibration process. This system and method advantageously operates in a robot-based hand-eye calibration environment and can compensate for a cantilever effect between the calibration target and the FOV. A hand-eye calibration computes the transformation from the robot coordinate space to the camera coordinate space. This generally avoids a need to manually move the robot through a set of spatial positions in which the camera then acquires images and locates calibration features on the object at each position to establish the relationship between the robot and camera.

The invention provides an indoor robot vision hand-eye relation calibration method based on a 3D image sensor. The method comprises the following steps: S1, marking a plurality of marking points on hand grabbing tail end joints of a robot, acquiring point cloud image information of the hand grabbing tail end joints through the 3D vision sensor of the robot, and acquiring a plurality of sets of three-dimensional coordinate values relative to a visual sensor coordinate system; S2, acquiring the three-dimensional coordinate values of the multiple marking points of the hand grabbing tail end joints under the world coordinate system through an external three-dimensional measurement device, wherein the values are acquired through an arm-base coordinate system of the robot; S3, acquiring the coordinate values through the step S1 and the step S2 and obtaining a hand-eye calibration array. The indoor robot vision hand-eye relation calibration method based on the 3D image sensor simplifies the calibration process, the measurement precision is high, and the requirement for indoor robot hand-eye calibration can be effectively met.

The invention relates to a single-camera hand-eye calibration method of a manipulator end. The above-mentioned single-camera hand-eye calibration method of the manipulator end includes: providing a calibration plate; placing the calibration plate in a visual field of a camera, controlling the camera for photographing, and controlling the manipulator end to be rotated to obtain a plurality of firstpixel coordinates of a center of a mark; wherein the first pixel coordinates of the center of the mark are the first of the first pixel coordinates before the manipulator end is rotated; and fittingthe plurality of first pixel coordinates into a circle, and calculating circle center coordinates corresponding to the circle, namely pixel coordinates of the manipulator end. According to the above-mentioned single-camera hand-eye calibration method of the manipulator end, artificial and manual operations of manipulator movement is not required, an entire calibration process is simple, calibration time is shorter, and impacts of artificial factors on calibration precision are smaller; and because the calibration time of the above-mentioned calibration method is shorter, calibration efficiencyof the calibration method is enabled to be higher.