Robot virtual programming demonstrator, system and method

Abstract

The invention relates to a robot virtual programming demonstrator, system and method, the robot virtual programming demonstrator comprises a handle, the handle is provided with a tool seat, and the tool seat is sequentially provided with a positioning area, an equipment area and a signal area along the length direction; the positioning area is provided with a calibration rod, and the equipment area is used for fixing end equipment needing teaching programming; the signal area is provided with a connecting part, at least two positions of the connecting part are provided with shells, each shell is internally provided with at least one gyroscope and at least one lamp bead, and a size interval is formed between every two adjacent shells so that light spots can be captured in real time by the camera shooting assembly during posture swing. Through the symmetrically arranged lamp beads, it is guaranteed that the handle can always collect information of one lamp bead in the movement process, the accuracy of camera positioning is guaranteed, meanwhile, the two gyroscopes are arranged to guarantee the posture information of the handle, the posture information can be transmitted in real time, the accuracy of system posture obtaining is improved, the error between a movement simulation track and a demonstrator track is reduced, and the system performance is improved. And the final robot motion precision is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot teaching, in particular to a robot virtual programming teaching device, system and method. BACKGROUND

[0002] The existing robot virtual teaching system mostly uses a single camera and an infrared signal lamp. During the movement of the handle, the infrared signal lamp is easily blocked, which causes the camera to fail to collect positioning data or the collected point position to deviate greatly, thereby affecting the generation of the final trajectory. With the development of laser radar, laser radar is also used for scanning positioning in the prior art. However, the laser radar has a scanning dead angle and is large in size and not easy to carry. Meanwhile, many noise points are easily generated during the collection process, and the collected data needs to be denoised by using a special algorithm, thereby increasing the use cost. In order to solve the above problems, the present application designs and develops a robot virtual programming teaching device, system and method. SUMMARY

[0003] In order to solve the above technical problems, the present application provides a robot virtual programming teaching device, system and method. The technical problems to be solved by the present application are realized by using the following technical solutions. The robot virtual programming teaching device comprises a handle, wherein a tool seat is arranged on the handle, and the tool seat is sequentially provided with a positioning area, a device area and a signal area along the length direction. The positioning area is provided with a calibration rod, and the device area is used to fix an end device that needs to be taught and programmed. The signal area is provided with a connecting part, at least two housings are arranged at the connecting part, at least one gyroscope and at least one lamp bead are arranged in each housing, and each housing has a size interval between the housings so that the light points can be captured by the camera assembly in real time when the posture is swung.

[0004] The housings are symmetrically arranged about the handle as the axis.

[0005] The signal area is distributed on one side of the handle, and the positioning area and the device area are distributed on the side of the handle away from the signal area.

[0006] The housings have a pyramid or cone structure.

[0007] The gyroscope is distributed in the area surrounded by the housings.

[0008] The lamp beads are distributed at the apex of the pyramid or cone structure of the housings.

[0009] The robot virtual teaching programming system comprises a robot virtual programming teaching device, a camera assembly and a data processing assembly. The camera assembly comprises a camera capable of walking. The data processing assembly comprises a workbench, and a signal synchronizer is arranged on the workbench and connected with a data terminal; The camera recognizes the light bead in the robot virtual programming demonstrator to obtain position information of the robot virtual programming demonstrator.

[0010] The robot virtual programming demonstration method comprises the following steps: First step: starting the camera component and the robot virtual programming demonstrator to complete calibration, and then the camera component recognizes the robot virtual programming demonstrator; Second step: sending the recognition signal to the signal synchronizer, receiving the signal by the signal synchronizer and activating the signal synchronizer, and reading the attitude information given by the gyroscope and the position information given by the light bead; Third step: aligning the time stamps of the camera component, the robot virtual programming demonstrator and the data processing assembly; Fourth step: transmitting the attitude information and the position information of the robot virtual programming demonstrator in the movement process to the data terminal, converting the attitude information of the robot virtual programming demonstrator to the position information coordinate system of the camera component by the data terminal, and fusing the camera position information and the robot virtual programming demonstrator attitude information; Fifth step: generating control instructions according to the data in the fourth step by the data processing assembly, forming a motion trajectory simulation, and transmitting to the robot.

[0011] Sixth step: the robot executes the simulation trajectory.

[0012] The camera component calculates the three-dimensional coordinates of the light-emitting marker according to the internal and external parameters of the binocular camera and the marker center parallax, using a stereo vision measurement model, and the formula is: Wherein, is the camera focal length, is the binocular baseline distance, and are the pixel coordinates of the marker center in the left and right images respectively; through the above formula, the two-dimensional image coordinates can be converted into three-dimensional space coordinates, realizing the three-dimensional positioning of the target.

[0013] The attitude information conversion of the robot virtual programming demonstrator to the position information coordinate system of the camera component is completed through the following steps: First step: the attitude information of the robot virtual programming demonstrator and the position of the corresponding point in the camera coordinate system ; Second step: attitude conversion, converting the attitude quaternion of the robot virtual programming demonstrator into a rotation matrix ; the conversion formula of quaternion to rotation matrix is: ; ​wherein, is a component of the quaternion; Third step: matrix alignment, using the formula Solving That is: ; wherein, is the corresponding rotation matrix in the camera coordinate system, is a rotation matrix; Fourth step: coordinate conversion, first convert into a rotation matrix , then convert back to the quaternion for data fusion, wherein is any one attitude information, is the attitude information in the camera coordinate system.

[0014] The beneficial effects of the present application are: the present application ensures that the handle can always collect the information of one lamp bead during the movement process through the symmetrically arranged lamp beads, to ensure the accuracy of camera positioning, and the two gyroscopes are arranged to ensure the attitude information of the handle and can be transmitted in real time, improve the accuracy of system attitude acquisition, reduce the error between the motion simulation trajectory and the trajectory of the teach pendant, and ensure the precision of the final robot movement. BRIEF DESCRIPTION OF DRAWINGS

[0015] The present application will be further described below in conjunction with the drawings and examples.

[0016] Figure 1 is a perspective structural schematic of the robot virtual programming teach pendant of the present application Figure 1 ; Figure 2 is a perspective structural schematic of the robot virtual programming teach pendant of the present application Figure 2 ; Figure 3 is a perspective structural schematic of the robot virtual programming teach pendant of the present application Figure 3 ; Figure 4 is an enlarged view of I in the present application Figure 2 ; Figure 5 is a perspective structural schematic of the robot virtual teach programming system of the present application Figure 6 is a flowchart of the robot virtual teach programming method of the present application.

[0017] The diagram shows: 1. Robot virtual programming teach pendant; 2. Camera assembly; 3. Data processing assembly; 11. Handle; 12. Tool holder; 13. Calibration rod; 14. Connecting part; 15. Housing; 16. LED beads; 121. Positioning area; 122. Equipment area; 123. Signal area; 21. Camera; 22. Walking unit; 23. Stand; 31. Workbench; 32. Signal synchronizer; 33. Data terminal. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be described more clearly and completely below with reference to the accompanying drawings in the embodiments. Of course, the described embodiments are only a part of the present invention and not all of it. Based on this embodiment, other embodiments obtained by those skilled in the art without creative effort are all within the protection scope of the present invention.

[0019] like Figures 1 to 4 As shown, the robot virtual programming teach pendant 1 includes a handle 11, a tool holder 12 on the handle 11, and a positioning area 121, a device area 122 and a signal area 123 arranged sequentially along the length direction of the tool holder 12. The positioning area 121 is equipped with a calibration rod 13, and the equipment area 122 is used to fix the end device that needs to be taught and programmed. When the device is started, the calibration rod 13 and the calibration plate work together to perform point-to-point teaching and calibration of the position. At the same time, the calibration rod 13 can also be used to visually see whether the teaching position and the position of the robot after being uploaded to the robot are consistent. The signal area 123 is provided with a connecting part 14, and the connecting part 14 is provided with a housing 15 at least at two locations. Each housing 15 is provided with at least one gyroscope and at least one LED 16. There is a dimensional gap between each housing 15 so that the light spot can be captured in real time by the camera component 2 when the posture is swaying.

[0020] like Figure 2 As shown, the housing 15 is symmetrically arranged with the handle 11 as the axis. During the swinging of the handle 11, there may be a situation where one side of the LED bead 16 cannot be recognized by the camera component 2. The LED bead 16 is symmetrically arranged with two LED beads to ensure that the camera component 2 can always collect the accurate position information of one LED bead 16. Each LED bead 16 is equipped with a gyroscope. When one side of the LED bead 16 is blocked, the accurate position information and attitude information of the other side of the LED bead 16 can be collected in real time, and the situation where only attitude information can be collected but position information cannot be collected will not occur.

[0021] The signal area 123 is located on one side of the handle 11, and the positioning area 121 and the device area 122 are located on the side of the handle 11 away from the signal area 123; for example Figure 3As shown, this design can prevent the signal area 123 from being blocked by the equipment on the equipment area 122, thus affecting the camera component 2's acquisition of the LED bead 16 signal.

[0022] The housing 15 has a pyramidal or conical structure. The housing 15 is pyramidal or conical, and its circumferential shape will not generate projected shadows that obstruct the camera assembly 2 from recognizing the LED beads 16 and collecting information during the movement of the handle 11.

[0023] The gyroscopes are distributed within the area enclosed by the housing 15.

[0024] The LED beads 16 are distributed at the apex of the pyramidal or conical structure of the housing 15. Distributing them at the apex of the pyramidal or conical structure can effectively avoid the obstruction of the light source of the LED beads 16 by the equipment on the equipment area 122 or the housing 15, thereby improving the recognition accuracy of the camera assembly 2 for the LED beads 16.

[0025] like Figure 5 As shown, the robot virtual teaching programming system includes a robot virtual programming teaching pendant 1, a camera component 2, and a data processing component 3; The camera assembly 2 includes a walkable camera 21; such as Figure 5 As shown, the camera assembly 2 also includes a walking part 22 and a stand 23. The camera 21 is hinged to the stand 23 and the angle of the camera 21 can be adjusted according to different devices to ensure that the coverage area of ​​the camera 21 can completely include the movement trajectory of the handle 11 during the teaching process.

[0026] The data processing component 3 includes a workbench 31, on which a signal synchronizer 32 is provided. The signal synchronizer 32 is connected to a data terminal 33, which can be an industrial control computer or other data processing system. Camera 21 identifies LED beads 16 in the robot virtual programming teach pendant 1 to obtain the position information of the robot virtual programming teach pendant 1.

[0027] like Figure 6 As shown, a robot virtual teaching programming method includes the following steps: Step 1: Start camera component 2 and robot virtual programming teach pendant to complete calibration, then camera component 2 recognizes robot virtual programming teach pendant; Step 2: Send the identification signal to the signal synchronizer 32. The signal synchronizer 32 receives the signal and is activated. At the same time, it reads the attitude information given by the gyroscope and the position information given by the LED 16. Image captured by camera component 2 is denoised and enhanced using median filtering and Gaussian filtering algorithms. The formula for calculating the template coefficients of the Gaussian kernel function is as follows: ; in, The standard deviation determines the width of the Gaussian function. The larger the standard deviation, the stronger the filtering effect, and the smoother the image will be, but the image details may be affected to some extent. By reasonably selecting the standard deviation, a balance can be struck between removing noise and preserving image details.

[0028] Third step: Align the timestamps of the camera assembly 2, the robot virtual programming teach pendant, and the data processing assembly 3; Fourth step: Transfer the posture information and position information data of the robot virtual programming teach pendant during the movement to the data terminal 33, which converts the posture information of the robot virtual programming teach pendant to the position information coordinate system of the camera assembly 2, and fuses the position information of the camera assembly 2 and the posture information of the robot virtual programming teach pendant; Fifth step: The data processing assembly 3 generates control instructions according to the data in the fourth step, forms a motion trajectory simulation, and transmits it to the robot.

[0029] Sixth step: The robot executes the simulation trajectory.

[0030] If the posture information and position information given by the gyroscope and the lamp bead 16 in the second step are abnormal, the robot virtual programming teach pendant 1 in the first step will be restarted.

[0031] If the data terminal 33 in the fourth step occurs an abnormality during analysis and processing, it needs to be removed by a removal algorithm after removing the mutation point, and then generate the control instruction.

[0032] The camera assembly 2 calculates the three-dimensional coordinates of the light-emitting marker based on the internal and external parameters of the binocular camera and the marker center disparity, using a stereo vision measurement model. The formula is: ; Where, is the camera focal length, is the binocular baseline distance, and are the pixel coordinates of the marker center in the left and right images respectively. Through the above formula, the two-dimensional image coordinates can be converted into three-dimensional space coordinates to realize the three-dimensional positioning of the target.

[0033] The conversion of the posture information of the robot virtual programming teach pendant to the position information coordinate system of the camera assembly 2 is completed through the following steps: First step: The posture information of the robot virtual programming teach pendant and the position of the corresponding point in the camera coordinate system ; Second step: Perform posture conversion to convert the robot virtual programming teach pendant posture quaternion to a rotation matrix ; The conversion formula from quaternion to rotation matrix is: ; where, These are the components of a quaternion; Step 3: Align the matrix using the formula. Solve ,Right now: ; in, It is the rotation matrix in the camera coordinate system. It is a rotation matrix; Step 4: Coordinate transformation, first convert the coordinates... Convert to rotation matrix Then, the third step of matrix alignment is used to... Convert back to quaternion To perform data fusion in For any pose information, This refers to the attitude information in the camera coordinate system.

[0034] The calibration of camera component 2 is completed through the following steps: Step 1: Based on the shape, size, and brightness of LED 16, create a template graphic using high-precision graphics software, and add details of brightness gradient to the graphic to enhance the accuracy of matching; The second step is to use block matching to divide the acquired image into several equal-sized blocks. These blocks are then matched in the left and right images. Based on the block matching method, a semi-global block matching method is further used to combine the matching results in the left, right, top, bottom, and diagonal directions to obtain a more accurate disparity value. Block matching compares the differences in grayscale values ​​of different blocks to find the block with the highest similarity between the left and right images, thereby determining the corresponding pixel position of the marker center in the left and right images and calculating the disparity. Block matching has the advantages of being computationally simple and easy to implement. Semi-local block matching, building upon block matching, further considers matching information from multiple directions, integrating matching results from left-right, top-bottom, and diagonal directions to obtain a more accurate disparity value. This method can, to some extent, handle occlusion problems in images, improving the robustness of disparity calculation.

[0035] Step 3: Use the normalized cross-correlation coefficient to accurately measure the similarity between the template image and the target image regions; The calculation formula is: ; in, Represents a template image. Indicates the target image region. These represent the average grayscale values ​​of the template image and the target image region, respectively. The value ranges from -1 to 1. The closer the value is to 1, the higher the matching degree, thus accurately determining the position of the marker in the image.

[0036] The attitude information collected by the gyroscope is calibrated through the following steps: Step 1: Create a calibration board with nine bright light-emitting points; Step 2: Place the calibration board within the combined measurement range of the camera assembly 2 and the gyroscope, and keep it stationary; Step 3: Use camera component 2 to acquire multiple images containing the light-emitting points of the calibration board, and at the same time use gyroscope to record the current attitude data to ensure that the attitude of the calibration board is diverse, covering different pitch, roll and yaw angles, so as to obtain comprehensive rotation information. Step 4: Construct a cost function to minimize the error between the predicted and actual observed positions of the light source in the coordinate system of camera component 2, thereby solving for the rotation matrix. ; Step 5: Using the nonlinear least squares method and gradient descent strategy, we gradually approach the optimal solution and finally achieve accurate rotation calibration of the gyroscope to the camera coordinate system.

[0037] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely prisms of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claims. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A robot virtual programming teach pendant, characterized in that: Includes a handle (11), on which a tool seat (12) is provided, and the tool seat (12) is provided with a positioning area (121), an equipment area (122) and a signal area (123) in sequence along the length direction. The positioning area (121) is provided with a calibration rod (13), and the equipment area (122) is used to fix the end device that needs to be taught and programmed; The signal area (123) is provided with a connecting part (14), and the connecting part (14) is provided with a housing (15) at least in two places. Each housing (15) is provided with at least one gyroscope and at least one LED (16). There is a size gap between each housing (15) so that the light spot can be captured in real time by the camera component (2) when the posture is swaying.

2. The robot virtual programming teach pendant according to claim 1, characterized in that: The housing (15) is symmetrically arranged with the handle (11) as the axis.

3. The robot virtual programming teach pendant according to claim 1, characterized in that: The signal area (123) is located on one side of the handle (11), and the positioning area (121) and the device area (122) are located on the side of the handle (11) away from the signal area (123).

4. The robot virtual programming teach pendant according to claim 3, characterized in that: The shell (15) has a pyramidal or conical structure.

5. The robot virtual programming teach pendant according to claim 3, characterized in that: The gyroscopes are distributed within the area enclosed by the housing (15).

6. The robot virtual programming teach pendant according to claim 3, characterized in that: The lamp beads (16) are distributed at the apex of the pyramidal or conical structure of the shell (15).

7. A robot virtual programming pendant system utilizing the robot virtual programming teach pendant according to any one of claims 1 to 6, characterized in that: It includes a robot virtual programming teach pendant, a camera component (2) and a data processing component (3); The camera assembly (2) includes a walking camera (21); The data processing component (3) includes a workbench (31), on which a signal synchronizer (32) is provided, and the signal synchronizer (32) is connected to a data terminal (33). The camera (21) identifies the LEDs (16) in the robot virtual programming teach pendant to obtain the position information of the robot virtual programming teach pendant.

8. The robot virtual teaching programming method using the robot virtual teaching programming system of claim 7, characterized in that: This method Includes the following steps: Step 1: Start the camera component (2) and the robot virtual programming teach pendant to complete the calibration, and then the camera component (2) recognizes the robot virtual programming teach pendant; Step 2: Send the identification signal to the signal synchronizer (32). The signal synchronizer (32) receives the signal and activates the signal synchronizer (32). At the same time, it reads the attitude information given by the gyroscope and the position information given by the LED (16). Step 3: Align the timestamps of the camera component (2), the robot virtual programming teach pendant, and the data processing component (3); Step 4: The posture and position information data of the robot virtual programming teach pendant during its movement are transmitted to the data terminal (33). The data terminal (33) converts the posture information of the robot virtual programming teach pendant into the position information coordinate system of the camera component (2) and fuses the position information of the camera component (2) with the posture information of the robot virtual programming teach pendant. Step 5: The data processing component (3) generates control instructions based on the data in Step 4, forms a motion trajectory simulation, and transmits it to the robot; Step 6: The robot executes the simulated trajectory.

9. The robot virtual teaching programming method according to claim 8, characterized in that: The camera component (2) calculates the three-dimensional coordinates of the luminous marker using a stereo vision measurement model based on the intrinsic and extrinsic parameters of the binocular camera and the parallax of the marker center. The formula is as follows: ; in, For camera focal length, The binocular baseline distance. and These are the pixel coordinates of the center of the marker in the left and right images, respectively. Using the above formula, the two-dimensional image coordinates are converted into three-dimensional spatial coordinates, thus achieving three-dimensional positioning of the target.

10. The robot virtual teaching programming method according to claim 8, characterized in that: The conversion of the robot virtual programming teach pendant's posture information into the position coordinate system of the camera component (2) is accomplished through the following steps: Step 1: Robot Virtual Programming Teach Pendant Attitude Information The position of the corresponding point in the camera coordinate system ; Step 2: Perform attitude transformation, converting the robot virtual programming teach pendant attitude quaternions into rotation matrices. The conversion formula from quaternions to rotation matrices is: ； in, These are the components of a quaternion; Step 3: Align the matrix using the formula. Solve ,Right now: ； in, It is the rotation matrix in the camera coordinate system. It is a rotation matrix; Step 4: Coordinate transformation, first convert the coordinates... Convert to rotation matrix Then, the third step of matrix alignment is used to... Convert back to quaternion To perform data fusion; in For any pose information, This refers to the attitude information in the camera coordinate system.

Images