A control device and method for an eye-hand robot

Abstract

The present invention relates to a control device and method for an eye-hand robot, the device comprising a robot, a host computer, a target, a measuring component and a calibration plate, the target being fixedly mounted in front of a camera, and the crosshairs of the target completely coincide with the central line of the image in the camera; the measuring component comprising a prism component and a camera, which are arranged around the robot, the prism component being arranged between the target and the camera, and the visual axis of the camera and the optical axis of the prism component being kept coaxial; the calibration plate being arranged at the end of the robot's mechanical arm and being located within the camera's field of view; the host computer being connected to the robot, the camera and the prism component, respectively. Compared with the prior art, the present invention controls the rotation of the prism component through the host computer when the camera is fixed, thereby driving the adjustment of the camera's field of view position, effectively expanding the working range of a single camera and the robot, and realizing the real-time dynamic measurement of the end of the robot's mechanical arm by the measuring component.

Description

Technical Field

[0001] The present invention relates to the technical field of robot hand-eye systems, and in particular to a control device and method for an eye-hand robot. Background Art

[0002] At this stage, with the development of a new round of scientific and technological revolution, higher requirements are put forward for the intelligence and automation of industrial robots. At the same time, with the rapid progress of computer image processing and pattern recognition technology, machine vision has been widely used in various fields such as industrial inspection. Robots are widely used as high-freedom and high-precision working equipment in industrial production, and in most applications, visual guidance is required to control the robot to complete various movements. Therefore, guiding and tracking the working status of the robot through the visual system has important application value and practical significance. The common eye-to-hand robot vision system often adopts the mode of placing a fixed camera to observe the robot to collect image data, emphasizing that the relative posture relationship between the robot and the visual system cannot be changed, and the camera's field of view is fixed.

[0003] In the prior art, for example, Chinese invention patent CN201710485973.2 discloses a hybrid binocular industrial robot system synchronous calibration system, method and other devices. The invention uses local cameras and global cameras for synchronous calibration at the same time. By adjusting the end posture of the calibration plate and the industrial robot, and recording the transformation relationship Ai, Bi, Ci after each adjustment, the transformation matrix X, Y after each adjustment in the posture transformation chain AXB=YC is solved by an iterative method until the preset iteration conditions are met, and the transformation matrix X, Y obtained after the last adjustment is used as the final transformation matrix to obtain the final posture transformation matrix. The device can meet the hand-eye synchronous calibration requirements of an industrial robot system with two types of cameras, eye-to-hand and eye-to-hand, without the aid of additional measuring tools, and has good calibration accuracy and efficiency. However, the device uses two cameras, which increases the cost and has a more complex structure.

[0004] Chinese invention patent CN202111325994.0 discloses a remote-operated visual-guided imaging device, including a fixed frame and a robot, and a 3D camera device fixedly connected to the lower surface of the top of the fixed frame, and also includes a motor box and a conveyor. The output shaft of the motor is fixedly connected to the bottom end of the vertical axis, and the outer surface of the vertical axis is fixedly connected with an upper cross arm and a lower cross arm in the vertical direction. Through structural features such as the upper cross arm, the upper tray and the reference workpiece, the device has a global field of view in the fixed imaging eye-hand system, and has good seismic resistance, relatively stable posture estimation, and the robot will not block the imaging field of view. However, the device has the disadvantages of a fixed and limited global field of view and poor flexibility. Summary of the invention

[0005] The purpose of the present invention is to provide an eye-hand robot control device and method in order to overcome the defects of the above-mentioned prior art.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] An eye-hand robot control device comprises a robot, a host computer, a target, a measuring component and a calibration plate;

[0008] The measuring component is arranged around the robot;

[0009] The measuring assembly includes a camera and a prism assembly, the target is fixed in front of the camera, and the prism assembly is arranged between the camera and the target;

[0010] The host computer is connected to the robot, the camera and the prism assembly respectively;

[0011] The prism assembly comprises a prism, a rotating mirror assembly and a driving assembly, wherein the prism is arranged on the rotating mirror assembly, and the rotating mirror assembly is connected to the driving assembly; the driving assembly is used to drive the rotating mirror assembly to rotate;

[0012] The calibration plate is arranged at the end of the mechanical arm of the robot.

[0013] Furthermore, the visual axis of the camera is kept coaxial with the optical axis of the prism assembly.

[0014] Furthermore, the crosshairs of the target coincide with the center line of the image in the camera.

[0015] Furthermore, the calibration plate is located within the field of view of the camera.

[0016] Furthermore, the calibration plate is connected to the robot arm in a rigid connection manner.

[0017] Furthermore, the host computer is provided with an image acquisition module and a prism position control module, the image acquisition module is used to control the camera to acquire images, and the prism position control module is used to control the rotation of the prism assembly.

[0018] A method for controlling an eye-hand robot is used to control an eye-hand robot control device as described above, so as to realize real-time dynamic measurement of the end of the robot's mechanical arm by a measuring component, and comprises the following steps:

[0019] Step 1, respectively establish a world coordinate system, a robot coordinate system, a prism assembly coordinate system, and a camera coordinate system matching the prism assembly;

[0020] Step 2, calibrate the camera and prism assembly by using the target, determine the position of the main cross section of the prism in the prism assembly and complete the axial alignment;

[0021] Step 3, the host computer controls the prism assembly so that the visual axis of the camera passes through the prism assembly without deflection. At this time, the camera and the prism assembly are matched; keep the camera still, add multiple postures to the robot, take pictures at each posture and identify the corner points of the calibration plate to calibrate the intrinsic parameters of the camera; establish the relationship between the calibration plate, the camera and the robot to calibrate the external parameters of the camera;

[0022] Step 4, set the target point, the host computer adjusts the prism assembly and the camera visual axis matched with it, the camera collects multiple sets of images containing the target point at different rotation angles of the prism assembly, and obtains the three-dimensional coordinates of the target point in the camera coordinate system by the variable visual axis visual measurement technology; obtains the spatial coordinates of the target point in the robot coordinate system by the coordinate conversion relationship; and guides the end of the robot arm to the target point position through the host computer control according to the robot's own system parameters;

[0023] Step 5, setting the working path of the end of the robot arm, the host computer adjusts the prism assembly and the camera visual axis matched with it through the visual tracking algorithm, so as to realize the real-time dynamic measurement of the end of the robot arm by the measurement assembly.

[0024] Furthermore, in step 2, determining the position of the main cross section of the prism in the prism assembly and completing the axial alignment specifically includes the following steps:

[0025] Step 201, without connecting the prism assembly, adjusting the relative position of the camera and the target so that the crosshairs of the target completely coincide with the center line of the image in the camera;

[0026] Step 202, connect a prism assembly between the camera and the target, and make the plane side of the prism perpendicular to the camera visual axis, adjust the prism angle until the vertical directrix of the target in the image captured by the camera coincides with the vertical directrix of the image in the camera, and at the same time, the horizontal directrix of the target in the image captured by the camera is located above the horizontal directrix of the image in the camera; that is, determine the position of the main cross-section of the prism and ensure the alignment relationship between the camera, prism and target.

[0027] Furthermore, the step 4 specifically includes the following steps:

[0028] Step 401: the host computer controls the rotation angle of the prism assembly and adjusts the visual axis of the camera matched with the prism assembly so that the target point falls into the center of the camera's field of view. The camera collects an image of the target point and transmits the image to the host computer.

[0029] Step 402, the host computer controls to change the rotation angle of the prism assembly multiple times so that the target point is always within the field of view of the camera, collects multiple target point images, and transmits the images to the host computer in sequence;

[0030] Step 403: The host computer obtains the prism incident point coordinates F in the camera coordinate system from the measurement component parameters according to the i-th group of images corresponding to different rotation angles of the prism component, i>0. i And the space vector S between it and the target point i Based on the relationship between the prism incident point coordinates, the space vector and the target point, i straight line equations are combined and the three-dimensional coordinates of the target point in the camera coordinate system are calculated by the least squares method.

[0031] Furthermore, in step 403, the prism incident point coordinate F in the camera coordinate system is i Recorded as The corresponding incident light space vector S i Recorded as Then the three-dimensional coordinates (X, Y, Z) of the target point in the camera coordinate system are T The following relations are satisfied:

[0032]

[0033] Then the i groups of target images are combined to list 2i equality constraints, and their matrix form is:

[0034]

[0035] Based on the least squares method, the linear least squares solution of the constraint condition is used as the three-dimensional coordinates (X, Y, Z) of the target point in the camera coordinate system. T .

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. The present invention adds a rotatable prism assembly that can adjust the camera imaging visual axis in real time in front of the camera lens. The prism assembly is controlled by a host computer and the imaging visual axis of the camera matched therewith is adjusted. The camera itself does not need to perform any form of movement. The imaging range of a single camera can be effectively expanded, and the visual working space of the robot can be improved.

[0038] 2. The present invention can realize large-scale real-time online tracking of the robot's working position by the measuring component according to the visual tracking algorithm through the rapid response of the rotatable prism component.

[0039] 3. The present invention can achieve accurate tracking and monitoring of the working state of the end of the robot's mechanical arm at a high working speed of the robot by adjusting the rotation speed of the prism assembly, and has good flexibility and environmental adaptability.

[0040] 4. The present invention uses only a single fixed camera, which is combined with a rotatable prism assembly to achieve an expansion of the field of view, and has the advantages of simple structure, convenient use and low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a structural schematic diagram of the present invention;

[0042] Figure 2 A schematic diagram of the field of view of a camera, a prism assembly, and a combination of the two;

[0043] Figure 3 Schematic diagram of the refraction of a light beam through a prism;

[0044] Figure 4 is a schematic diagram of the structure of a prism;

[0045] Figure 5 Flowchart of the visual tracking algorithm.

[0046] The corresponding relationship between the numbers and the components in the drawings is: 1-camera; 2-prism assembly; 3-target; 4-calibration plate; 5-robot; 6-host computer; 7-imaging field of view; 8-optical scanning field. DETAILED DESCRIPTION

[0047] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0048] The present invention is based on the fixed camera of the eye-hand robot vision system. By placing a rotatable prism assembly in front of the camera visual axis, the upper computer drives the motor to control the speed and angle of the prism assembly, and can accurately control the direction of the camera's imaging visual axis, effectively expand the camera's field of view, and can achieve dynamic tracking of the robot's working state by the camera's field of view. Compared with the prior art, under the same camera fixed condition, the present invention expands the imaging range of a single camera, and can achieve the camera's field of view following the robot's movement, and has good flexibility and environmental adaptability.

[0049] like Figure 1 As shown, an eye-hand robot control device includes a robot 5, a host computer 6, a target 3, a measurement component and a calibration plate 4;

[0050] The measuring components are arranged around the robot 5;

[0051] The measuring assembly includes a camera 1 and a prism assembly 2. The target 3 is fixed in front of the camera 1. The prism assembly 2 is arranged between the camera 1 and the target 3. The crosshairs of the target 3 completely coincide with the center line of the image in the camera. The visual axis of the camera 1 and the optical axis of the prism assembly 2 remain coaxial.

[0052] The host computer 6 is respectively connected to the robot 5, the camera 1 and the prism assembly 2 and communicates with each other;

[0053] The calibration plate 4 is arranged at the end of the mechanical arm of the robot 5 and is located in the field of view of the camera. The calibration plate 4 is connected to the mechanical arm of the robot 5 using a rigid connection method and will not interfere with the robot.

[0054] The prism assembly 2 includes a prism, a rotating mirror assembly and a driving assembly. The prism is arranged on the rotating mirror assembly, and the rotating mirror assembly and the driving assembly are connected; the driving assembly is used to drive the rotating mirror assembly to rotate; in specific implementation, the types of prisms in the rotating mirror assembly include rotating single prism, rotating double prism, tilting double prism and rotating polygonal prism, and their optical parameters and combination forms can be matched and adjusted according to the field of view requirements; the driving assembly can adopt any one of gear transmission, worm gear transmission, synchronous belt transmission and chain transmission or a combination thereof.

[0055] like Figure 2 The figure shows a schematic diagram of the field of view of a camera, a prism assembly and a combination of the two. The optical scanning field 8 formed by light passing through the camera 1 and the rotating mirror assembly 2 is not smaller than the imaging field of view 7 when there is only an industrial camera.

[0056] like Figure 3 The figure shows the refraction of a light beam through a prism. When the plane of the prism is the incident plane of the light, it is defined as the front arrangement. When the inclined surface of the prism is the incident plane of the light, it is defined as the reverse arrangement. The degree of refraction of the light can be adjusted by rotating the angle of the prism.

[0057] like Figure 4 As shown, the prism used in this embodiment has a flat surface at one end and an inclined surface at the other end, a wedge-shaped cross section, and is made of K9 glass.

[0058] This embodiment also provides an eye-hand robot control method, which is used to control an eye-hand robot control device as described above, and realize the real-time dynamic measurement of the end of the robot arm by the measurement component. First, the industrial camera 1 and the prism assembly 2 are systematically calibrated through the target 3, the position of the main cross section of the prism in the prism assembly 2 is determined and the axial alignment is completed, and then the robot is controlled by the host computer 6 to perform multiple posture changes, take pictures at each posture and identify the corner points of the calibration plate 4, and calibrate the internal and external parameters of the industrial camera 1. The variable visual axis visual measurement technology obtains the target point in the camera coordinate system O C -X C Y C Z C The three-dimensional coordinates of the target point in the robot coordinate system Or-XrYrZr are obtained by the coordinate conversion relationship. According to the system parameters of the robot 5 itself, the upper computer 6 controls and guides the end of the robot 5 to the target point. The upper computer 6 controls the prism assembly 2 to adjust the imaging visual axis direction of the industrial camera 1 matched with it through the visual tracking algorithm to realize the tracking of the robot 5 mechanical arm.

[0059] Specifically, the steps include:

[0060] Step 1, respectively establish a world coordinate system, a robot coordinate system, a prism assembly coordinate system, and a camera coordinate system matching the prism assembly;

[0061] Step 2, calibrate the camera and prism assembly by using the target, determine the position of the main cross section of the prism in the prism assembly and complete the axial alignment;

[0062] Step 3, the host computer controls the prism assembly so that the visual axis of the camera passes through the prism assembly without deflection. At this time, the camera and the prism assembly are matched; keep the camera still, add multiple postures to the robot, take pictures at each posture and identify the corner points of the calibration plate to calibrate the intrinsic parameters of the camera; establish the relationship between the calibration plate, the camera and the robot to calibrate the external parameters of the camera;

[0063] Step 4, set the target point, the host computer adjusts the prism assembly and the camera axis of sight with it through the axis pointing algorithm, the camera collects multiple sets of images of the prism assembly containing the target point at different angles, and obtains the three-dimensional coordinates of the target point in the camera coordinate system by the variable axis vision measurement technology; obtains the spatial coordinates of the target point in the robot coordinate system by the coordinate conversion relationship; and guides the end of the robot arm to the target point position through the host computer control according to the robot's own system parameters;

[0064] Step 5, setting the working path of the end of the robot mechanical arm, the host computer adjusts the prism assembly and the camera visual axis matched with it through the visual tracking algorithm, so as to realize the real-time dynamic measurement of the end of the robot mechanical arm by the measurement assembly; in the specific implementation, the visual tracking algorithm can adopt the table lookup method, the coarse-fine coupling tracking algorithm or the iterative method.

[0065] like Figure 5 FIG. 1 is a flowchart of the implementation of the visual tracking algorithm used in this embodiment, which includes the following steps:

[0066] 1) Set the upper limit of the number of iterations CountMax, the initial iteration step Step and the tracking accuracy BiasThresh, and set the number of iterations Count = 0;

[0067] 2) Adjust the camera’s visual axis azimuth;

[0068] 3) If the current deviation Bias>BiasThresh, adjust the camera's visual axis pitch angle, execute Count++, and proceed to the next step; otherwise, the tracking is successful and all steps are completed.

[0069] 4) Determine whether Bias>BiasThresh and Count≤CountMax are satisfied. If so, set Step=Step / 2 and return to step (2); otherwise, execute the next step;

[0070] 5) Determine whether Bias>BiasThresh is satisfied. If so, the number of iterations is exceeded and all steps are terminated; otherwise, the tracking is successful and all steps are terminated.

[0071] For step 2, determining the position of the main cross section of the prism in the prism assembly and completing the axial alignment specifically includes the following steps:

[0072] Step 201, without connecting the prism assembly, adjusting the relative position of the camera and the target so that the crosshairs of the target completely coincide with the center line of the image in the camera, that is, ensuring the alignment relationship between the camera and the target;

[0073] Step 202, connect a prism assembly between the camera and the target, and make the prism plane side perpendicular to the camera visual axis, adjust the prism rotation angle until the vertical directrix of the target in the camera captured image coincides with the vertical directrix of the image in the camera, and the horizontal directrix of the target in the camera captured image is located above the horizontal directrix of the image in the camera; that is, determine the position of the prism main section and ensure the alignment relationship between the camera, prism and target. Wherein, the image in the camera contains crosshairs, which are divided into vertical directrix and horizontal directrix.

[0074] For step 4, the specific steps include:

[0075] Step 401: the host computer controls the rotation angle of the prism assembly and adjusts the visual axis of the camera matched with the prism assembly so that the target point falls into the center of the camera's field of view. The camera collects an image of the target point and transmits the image to the host computer.

[0076] Step 402, the host computer controls to change the prism assembly angle multiple times so that the target point is always within the field of view of the camera, collects multiple target point images, and transmits the images to the host computer in sequence;

[0077] Step 403: The host computer obtains the prism incident point coordinates F in the camera coordinate system from the measurement component parameters according to the i-th group of images corresponding to different rotation angles of the prism component, i>0. i And the space vector S between it and the target point i Based on the relationship between the prism incident point coordinates, the space vector and the target point, i straight line equations are combined and the three-dimensional coordinates of the target point in the camera coordinate system are calculated by the least squares method.

[0078] Specifically, the prism incident point coordinates F in the camera coordinate system are i Recorded as The corresponding incident light space vector S i Recorded as Then the three-dimensional coordinates (X, Y, Z) of the target point in the camera coordinate system are T The following relations are satisfied:

[0079]

[0080] Then the i groups of target images are combined to list 2i equality constraints, and their matrix form is:

[0081]

[0082] Based on the least squares method, the linear least squares solution of the constraint condition is used as the three-dimensional coordinates (X, Y, Z) of the target point in the camera coordinate system. T .

[0083] In this embodiment, taking the prism assembly 2 as an example, the composition parameters of each component are:

[0084] Camera 1: It is an industrial camera, using MV-EM200M black and white camera, AFT-2514MP lens, resolution 1200 (V) × 1600 (H), lens focal length 25mm;

[0085] Prism assembly 2: It includes an inner prism and an outer prism. The two prisms are exactly the same, with a prism wedge angle α = 10°, a refractive index n = 1.517, and a diameter D p =80mm, thin end thickness d 0 =5mm, the distance between the two prisms is D 1 =100mm;

[0086] Target 3: Use a cross target with adjustable position and related fixing tools;

[0087] Calibration plate 4: a checkerboard calibration plate with a side length of 12.5 mm;

[0088] Robot 5: It uses the Yuejiang DOBOT Magician robot, which can carry different tools at the end of the robotic arm to achieve different applications, communicate with the PC via USB, and connect to the Bluetooth or WIFI module via UART;

[0089] Host computer 6: It is equipped with an image acquisition module and a prism position control module. The image acquisition module is used to control the camera to acquire images, and the prism position control module is used to control the rotation of the prism assembly 2. The image acquisition module includes image acquisition software, and the prism position control module includes rotating prism position control software. The imaging visual axis adjustment inverse algorithm is used to calculate the prism angle, and the speed and angle information are sent to the rotating prism controller through RS485 communication, and the target image acquired by the camera is stored.

[0090] In specific implementation, a method for controlling an eye-hand robot is as follows:

[0091] S1. Select a suitable location to place the camera 1 and the robot 5, and establish the world coordinate system O. w -X w Y w Z w , establish camera 1 coordinate system O C -X C Y C Z C , establish robot 5 coordinate system O R -X R Y R Z R .

[0092] S2, install a cross target 3 in front of the camera 1, and adjust the position of the target 3 so that the crosshairs of the target 3 completely coincide with the center line of the camera 1 image;

[0093] S3, place an inner prism in the prism assembly between the camera 1 and the target 3, and make the plane side of the inner prism perpendicular to the camera visual axis, adjust the prism assembly angle until the vertical directrix of the target 3 in the camera captured image coincides with the vertical directrix of the image in the camera 1, and the horizontal directrix of the target 3 is located above the horizontal directrix of the image in the camera 1; then rotate the prism 90°, adjust the horizontal position of the prism, make the horizontal directrix of the target 3 coincide with the horizontal directrix of the image in the camera 1, and the camera 1 and the inner prism are aligned;

[0094] S4, set the inner prism rotation angle to 0°, place the outer prism, adjust the outer prism position, make the horizontal directrix of target 3 coincide with the horizontal directrix of camera 1 image, and at the same time, the vertical directrix of target 3 coincide with the vertical directrix of camera 1 image. At this time, the center of the outer prism is aligned and the rotation angle is 180°. The visual axis of the camera passes through the prism assembly without deflection. The prism assembly coordinate system is established with the center of the inner prism plane side as the origin. P -X P Y P Z P .

[0095] S5. Keep the camera and the rotating dual prism stationary, and control the robot 5 to change multiple postures by the host computer 6. Take pictures in each posture and identify the corner points of the calibration plate 4. Use a suitable calibration method to calibrate the intrinsic parameters of the camera 1. The suitable calibration method includes the direct linear method, the Tsai two-step calibration method or the Zhang Zhengyou calibration method, and record the intrinsic parameters of the camera 1.

[0096] S6. According to the hand-eye calibration principle, the position relationship between the end of the robot 5 and the calibration plate 4 remains unchanged. The relationship between the calibration plate 4, the camera 1 and the robot 5 is established, and the following formula is obtained:

[0097] End T Base2 × Base2 T Camera2 × Camera2 T Object = End T Base1 × Base1 T Camera1 × Camera1 T Object

[0098] The coordinate system O of camera 1 can be calculated by the least squares method: C -X C Y C Z C With the robot 5 base coordinate system O R -X R Y R Z R The pose relationship between them records the external parameters of camera 1;

[0099] S7, set the end feature of the robot mechanical arm as the target point, the host computer 6 adjusts the prism assembly 2 and the visual axis of the camera 1 matched therewith through the visual axis pointing algorithm, so that the target point falls into the center position of the field of view of the camera 1, the camera 1 collects the image of the target and transmits the image to the host computer 6; change the prism angle in steps of 5°, so that the target point is always within the field of view of the camera 1, and collect i groups (i≥2) of target images;

[0100] S8, camera coordinate system O C -X C Y C Z C With prism 2 coordinate system O P -X P Y P Z P The geometric spatial relationship between them is known. According to the pixel coordinates of the target point in the image and the corresponding prism parameters, combined with the law of vector refraction, the prism incident point coordinates of the target point in the i-th group of images in the camera coordinate system can be obtained. and the corresponding incident light space vector Then the target point is in the camera coordinate system O C -X C Y C Z C The three-dimensional coordinates (X, Y, Z) of T The following relations are satisfied:

[0101]

[0102] Then, the i groups of target images can list 2i equality constraints, which can be written in matrix form as follows:

[0103]

[0104] That is, based on the least squares method, the linear least squares solution of the equation is used as the target point in the camera coordinate system O C -X C Y C Z C The three-dimensional coordinates (X, Y, Z) of T ;

[0105] S9, according to the external parameters of the camera 1 obtained by the hand-eye calibration in step S6, the coordinate transformation relationship is used to obtain the coordinate system of the target point in the robot 5. R -X R Y R Z R The spatial coordinates (X R ,Y R ,Z R ) T ; According to the system parameters of robot 5 itself and through the inverse solution of robot motion, the movement vector of the end working device of robot 5 is calculated, and the upper computer 6 controls and guides the end working device of robot 5 to the target point position;

[0106] S10, manually set the working path of the end of the robot arm of the robot 5, and the host computer 6 performs the following steps: Figure 5 The visual tracking algorithm shown adjusts the prism assembly 2 in real time, that is, adjusts the imaging visual axis of the camera 1 matched thereto in real time, so as to realize dynamic tracking of the target point at the end of the robot 5 mechanical arm.

[0107] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for controlling an eye-hand robot, characterized in that: Used to control the eye-hand robot control device to achieve real-time dynamic measurement of the end of the robot's mechanical arm by the measurement component; The eye-hand robot control device comprises a robot (5), a host computer (6), a target (3), a measuring component and a calibration plate (4); The measuring component is arranged around the robot (5); The measuring assembly comprises a camera (1) and a prism assembly (2), the target (3) is fixedly mounted in front of the camera (1), and the prism assembly (2) is arranged between the camera (1) and the target (3); The host computer (6) is connected to the robot (5), the camera (1) and the prism assembly (2) respectively; The prism assembly (2) comprises a prism, a rotating mirror assembly and a driving assembly, wherein the prism is arranged on the rotating mirror assembly, and the rotating mirror assembly is connected to the driving assembly; the driving assembly is used to drive the rotating mirror assembly to rotate; The calibration plate (4) is arranged at the end of the mechanical arm of the robot (5); The method comprises the following steps: Step 1, respectively establish a world coordinate system, a robot coordinate system, a prism assembly coordinate system, and a camera coordinate system matching the prism assembly; Step 2, calibrating the camera and prism assembly by using the target, determining the position of the main cross section of the prism in the prism assembly and completing the axial alignment, specifically including the following steps: Step 201, without connecting the prism assembly, adjusting the relative position of the camera and the target so that the crosshairs of the target completely coincide with the center line of the image in the camera; Step 202, connect a prism assembly between the camera and the target, and make the plane side of the prism perpendicular to the camera visual axis, and adjust the prism rotation angle until the vertical directrix of the target in the image captured by the camera coincides with the vertical directrix of the image in the camera, and the horizontal directrix of the target in the image captured by the camera is located above the horizontal directrix of the image in the camera; that is, determine the position of the main cross section of the prism and ensure the alignment relationship between the camera, prism and target; Step 3, the host computer controls the prism assembly so that the visual axis of the camera passes through the prism assembly without deflection. At this time, the camera and the prism assembly are matched; keep the camera still, add multiple postures to the robot, take pictures at each posture and identify the corner points of the calibration plate to calibrate the intrinsic parameters of the camera; establish the relationship between the calibration plate, the camera and the robot to calibrate the external parameters of the camera; Step 4, set the target point, the host computer adjusts the prism assembly and the camera visual axis matched therewith, the camera collects multiple sets of images containing the target point at different rotation angles of the prism assembly, and obtains the three-dimensional coordinates of the target point in the camera coordinate system by the variable visual axis visual measurement technology; obtains the spatial coordinates of the target point in the robot coordinate system by the coordinate conversion relationship; guides the end of the robot arm to the target point position through the host computer control according to the robot's own system parameters; specifically includes the following steps: Step 401: the host computer controls the rotation angle of the prism assembly and adjusts the visual axis of the camera matched with the prism assembly so that the target point falls into the center of the camera's field of view. The camera collects an image of the target point and transmits the image to the host computer. Step 402, the host computer controls to change the prism assembly angle multiple times so that the target point is always within the field of view of the camera, collects multiple target point images, and transmits the images to the host computer in sequence; Step 403: The host computer obtains the prism incident point coordinates F in the camera coordinate system from the measurement component parameters according to the i-th group of images corresponding to different rotation angles of the prism component, i>0. i And the space vector S between it and the target point i , based on the relationship between the prism incident point coordinates, the space vector and the target point, i straight line equations are combined, and the three-dimensional coordinates of the target point in the camera coordinate system are calculated by the least squares method; Step 5, setting the working path of the end of the robot arm, the host computer adjusts the prism assembly and the camera visual axis matched with it through the visual tracking algorithm, so as to realize the real-time dynamic measurement of the end of the robot arm by the measurement assembly.

2. The method according to claim 1, characterized in that The visual axis of the camera (1) remains coaxial with the optical axis of the prism assembly (2).

3. The method according to claim 1, characterized in that The crosshairs of the target (3) coincide with the center line of the image in the camera.

4. The method according to claim 1, characterized in that: The calibration plate (4) is located within the field of view of the camera (1).

5. The method according to claim 4, characterized in that The calibration plate (4) is connected to the mechanical arm of the robot (5) in a rigid connection manner.

6. The method according to claim 1, characterized in that The host computer (6) is provided with an image acquisition module and a prism position control module, wherein the image acquisition module is used to control the camera to acquire images, and the prism position control module is used to control the rotation of the prism assembly (2).

7. The method according to claim 1, characterized in that In step 403, the prism incident point coordinate F in the camera coordinate system is i Recorded as The corresponding incident light space vector S i Recorded as Then the three-dimensional coordinates (X, Y, Z) of the target point in the camera coordinate system are T The following relations are satisfied: Then the i groups of target images are combined to list 2i equality constraints, and their matrix form is: Based on the least squares method, the linear least squares solution of the constraint condition is used as the three-dimensional coordinates (X, Y, Z) of the target point in the camera coordinate system. T .

Images