Robotic vision-guided grasping device

Abstract

The application discloses a kind of robot vision guidance grabbing devices, it is related to industrial automation and robot technical field, including: robot main body, visual perception module, control module, end execution module and calibration module.The robot vision guidance grabbing device of the embodiment of the application, through the collaborative design of each module, effectively improve the precision of the grabbing of the application, anti-interference ability and working condition adaptability, with the advantages of convenient operation, stable operation, can adapt to workpiece grabbing demand in different scenes.

Description

Technical Field

[0001] This invention relates to the field of industrial automation and robotics, and in particular to a robot vision-guided grasping device. Background Technology

[0002] In industrial automated production, robotic grasping is a key step in automating processes such as material handling, assembly, and sorting. Visual guidance technology, acting as its "eyes," directly determines the accuracy, efficiency, and flexibility of the grasping process. Currently, existing visual guidance grasping devices still face multiple technical bottlenecks and struggle to adapt to the production demands of complex scenarios.

[0003] First, the point cloud data quality is severely lacking. Reflective and transparent objects (such as metal gears and glass bottles) can easily cause missing depth data, forming holes in the point cloud. The point clouds of stacked objects are prone to sticking together and are difficult to effectively separate. Differences in the color of the object surface and the tilt of the angle can also lead to uneven point cloud density, which seriously affects the positioning accuracy.

[0004] Secondly, the system lacks the ability to resist interference in complex environments. Highly reflective and light-absorbing materials, as well as fluctuations in illumination, can all interfere with the imaging effect. Dynamic occlusion caused by the movement of the robotic arm and changes in material stacking makes it difficult for the system to obtain complete target information in real time.

[0005] Third, the real-time performance of data processing is insufficient. The large volume of high-resolution point cloud data makes it difficult for traditional processing architectures to match the production line pace, which can easily lead to capture delays.

[0006] Fourth, the pose estimation accuracy of irregularly shaped parts is low, the sensor synchronization and calibration deviation is large, and the generalization ability is insufficient when there is a lack of prior knowledge about new objects, which limits the success rate of grasping. At the same time, the system is highly dependent on the accuracy of hand-eye calibration, and the initial calibration error is easy to accumulate and be transmitted, resulting in inaccurate positioning of the end effector.

[0007] Therefore, developing a robot vision-guided grasping device with high precision and high flexibility has become an urgent need in the field of industrial automation. Summary of the Invention

[0008] The present invention aims to at least partially solve one of the technical problems in the above-mentioned technologies.

[0009] To achieve the above objectives, the present invention adopts the following technical solution.

[0010] This invention provides a robot vision-guided grasping device, comprising: a robot body, a vision perception module, a control module, an end effector module, and a calibration module. The robot body includes a robotic arm and a base. The robotic arm adopts a multi-degree-of-freedom serial structure, with a flange at its end for connecting to the end effector module. The base integrates a servo drive motor and a braking unit. The vision perception module includes a 3D vision component, a multispectral illumination component, and a data preprocessing unit. The 3D vision component is composed of a multi-line laser structured light camera and a ToF camera. The data preprocessing unit integrates an FPGA hardware acceleration chip. The control module includes a main controller, a path planning unit, and an error compensation unit. The main controller adopts a CPU+GPU+FPGA heterogeneous computing architecture. The calibration module includes a calibration board and an automatic calibration unit.

[0011] Furthermore, the multispectral supplementary lighting component includes a blue light supplementary lighting unit and an infrared supplementary lighting unit. The blue light supplementary lighting unit is used to suppress reflective interference from highly reflective materials. The infrared supplementary lighting unit is used to improve the imaging clarity of light-absorbing materials. The multispectral supplementary lighting component automatically switches the supplementary lighting mode according to the ambient light and the material of the workpiece.

[0012] Furthermore, the data preprocessing unit uses an adaptive filtering algorithm and... PointNet ++Deep learning segmentation algorithm to achieve denoising, completion and separation of adhering objects in point cloud data.

[0013] Furthermore, the path planning unit uses an improved A* algorithm to plan the optimal grasping path and has a dynamic obstacle avoidance function. The error compensation unit uses a Kalman filter prediction algorithm to compensate for the pose deviation in real time and combines force sensor feedback data to achieve adaptive adjustment of the grasping force.

[0014] Furthermore, the 3D vision component has a resolution of no less than 1280×1024, a scanning frequency of ≥30Hz, a depth accuracy of ≤0.05mm, and a working distance of 300–1500mm.

[0015] Furthermore, the error compensation unit controls the pose error within ±0.05mm.

[0016] Furthermore, the control module also integrates a human-machine interaction unit, which includes a touch screen for parameter setting, status monitoring and fault alarm, as well as an emergency stop button for emergency shutdown.

[0017] Furthermore, the end effector module adopts a modular flexible gripper with an opening angle range of 0–90° and a clamping force adjustment range of 5–500N.

[0018] Furthermore, the visual perception module also integrates an image enhancement unit, which employs a multi-frame fusion algorithm.

[0019] Furthermore, the control module communicates with the robot body, the vision perception module, and the end effector module using the EtherCAT high-speed communication protocol, with a communication delay of ≤50ms.

[0020] According to the present invention, a robot vision-guided grasping device, through the collaborative design of various modules, effectively improves the grasping accuracy, anti-interference ability and working condition adaptability of the present invention, and has the advantages of convenient operation and stable operation, and can be adapted to the workpiece grasping needs in different scenarios. Attached Figure Description

[0021] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0022] Figure 1 This is a flowchart of a robot vision-guided grasping device according to an embodiment of the present invention;

[0023] Figure 2 This is an overall structural block diagram of a robot vision-guided grasping device according to an embodiment of the present invention;

[0024] Figure 3 This is a structural block diagram of the visual perception module of a robot vision-guided grasping device according to an embodiment of the present invention;

[0025] Figure 4 This is a structural block diagram of the control module of a robot vision-guided grasping device according to an embodiment of the present invention;

[0026] Figure 5 This is a three-dimensional structural diagram of a robot vision-guided grasping device according to an embodiment of the present invention;

[0027] As shown in the figure:

[0028] 1. Robot body; 11. Robotic arm; 12. Base; 2. Visual perception module; 21. 3D vision component; 22. Multispectral illumination component; 221. Blue light illumination unit; 222. Infrared illumination unit; 23. Data preprocessing unit; 24. Image enhancement unit; 3. Control module; 31. Main controller; 32. Path planning unit; 33. Error compensation unit; 34. Human-machine interaction unit; 341. Touch screen; 342. Emergency stop button; 4. End effector module; 41. Gripper; 5. Calibration module; 51. Calibration board; 52. Automatic calibration unit. Detailed Implementation

[0029] Embodiments of the present invention are described in detail below, with examples of the embodiments illustrated in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0030] The following description, in conjunction with the accompanying drawings, describes a robot vision-guided grasping device according to an embodiment of the present invention.

[0031] like Figures 1 to 5 As shown, this embodiment of the invention provides a robot vision-guided grasping device, including: a robot body 1, a vision perception module 2, a control module 3, an end effector module 4, and a calibration module 5. The modules work together to achieve accurate identification, path planning, and stable grasping of the workpiece.

[0032] It should be noted that the control module 3 communicates with the robot body 1, the vision perception module 2, and the end effector module 4 using the EtherCAT high-speed communication protocol, with a communication delay of ≤50ms. This enables microsecond-level synchronization of vision perception, force sensing, and the movement of the robotic arm 11, ensuring the timeliness of multi-sensor data fusion and the real-time response of control commands, thereby improving the grasping and positioning accuracy, dynamic obstacle avoidance reliability, and production line cycle time adaptability.

[0033] The robot body 1 includes a robotic arm 11 and a base 12.

[0034] The robotic arm 11 adopts a multi-degree-of-freedom serial structure, and its end is provided with a flange for connecting the end effector module 4. The base 12 integrates a servo drive motor and a braking unit.

[0035] Specifically, by adopting a multi-degree-of-freedom serial structure, the robotic arm 11 can achieve multi-posture and multi-angle movement in space.

[0036] Specifically, by setting a flange, the end effector module 4 can be quickly connected, making it easy to change different types of grippers 41 according to gripping requirements.

[0037] Specifically, the servo drive motor provides power for the movement of the robotic arm 11, and the braking unit can quickly lock the robotic arm 11 when the equipment stops or an abnormality occurs, ensuring the safety of the equipment and the workpiece.

[0038] The visual perception module 2 includes a 3D vision component 21, a multispectral illumination component 22, and a data preprocessing unit 23.

[0039] The 3D vision component 21 is composed of a multi-line laser structured light camera and a ToF camera, and the data preprocessing unit 23 integrates an FPGA hardware acceleration chip.

[0040] It should be noted that the 3D vision component 21 has a resolution of no less than 1280×1024, a scanning frequency of ≥30Hz, a depth accuracy of ≤0.05mm, and a working distance of 300–1500mm, which can meet the grasping needs in different scenarios.

[0041] It should be noted that by leveraging the advantages of both cameras, the multi-line laser structured light camera achieves high-precision imaging, while the ToF camera enables rapid depth measurement. The fusion of the two can improve the accuracy and speed of 3D information acquisition of workpieces.

[0042] The multispectral supplementary lighting component 22 includes a blue light supplementary lighting unit 221 and an infrared supplementary lighting unit 222.

[0043] Among them, the blue light supplement unit 221 is used to suppress the reflection interference of highly reflective materials, the infrared supplement unit 222 is used to improve the imaging clarity of light-absorbing materials, and the multi-spectral supplement component 22 automatically switches the supplement mode according to the ambient light and the material of the workpiece.

[0044] Specifically, the blue light supplement unit 221 is designed to avoid image blurring caused by reflections.

[0045] Specifically, the automatic switching of the supplementary lighting mode ensures that clear workpiece images can still be obtained under complex lighting conditions.

[0046] It should be noted that the data preprocessing unit 23 uses an adaptive filtering algorithm and the PointNet++ deep learning segmentation algorithm to achieve denoising, completion and separation of adhering objects in point cloud data. It can also remove environmental noise and image interference, restore the true three-dimensional shape of the workpiece, and provide accurate data support for subsequent path planning and grasping control.

[0047] It should be noted that the visual perception module 2 also integrates an image enhancement unit 24. The image enhancement unit 24 adopts a multi-frame fusion algorithm, which can further improve the image clarity and contrast and optimize the workpiece feature extraction effect.

[0048] The control module 3 includes a main controller 31, a path planning unit 32, and an error compensation unit 33.

[0049] Among them, the main controller 31 adopts a CPU+GPU+FPGA heterogeneous computing architecture, which balances data processing speed and control accuracy, and realizes the coordinated control of each module.

[0050] Specifically, the CPU is responsible for overall logic control and instruction distribution, the GPU is responsible for the rapid processing of image data and the running of deep learning algorithms, and the FPGA is responsible for the output of real-time control signals and data acquisition. The three work together to improve the device's response speed and control accuracy.

[0051] It should be noted that the path planning unit 32 uses an improved A* algorithm to plan the optimal grasping path and has a dynamic obstacle avoidance function, which can respond to sudden obstacles that occur during the grasping process in real time and avoid collisions between the robotic arm 11 and obstacles.

[0052] The error compensation unit 33 uses a Kalman filter prediction algorithm to compensate for the position deviation in real time, and combines the force sensor feedback data to realize the adaptive adjustment of the gripping force. The error compensation unit 33 controls the position error within ±0.05mm, and can realize the adaptive adjustment of the gripping force to avoid damage to the workpiece due to excessive gripping force or workpiece falling off due to insufficient gripping force.

[0053] It should be noted that the control module 3 also integrates a human-machine interaction unit 34. The human-machine interaction unit 34 includes a touch screen 341 for parameter setting, status monitoring and fault alarm, and an emergency stop button 342 for emergency shutdown. When the equipment malfunctions, it can promptly issue an alarm signal and display the cause of the fault. The emergency stop button 342 is used for emergency shutdown. In case of an emergency, pressing the emergency stop button 342 can quickly cut off the power supply to the equipment, ensuring the safety of the equipment, personnel and workpiece.

[0054] It should be noted that the end effector module 4 adopts a modular flexible gripper 41. The opening angle of the gripper 41 is 0–90° and the clamping force is adjustable from 5–500N. The opening angle and clamping force can be flexibly adjusted according to the size, shape and material of the workpiece to adapt to different types of workpiece gripping. At the same time, the flexible design can avoid damage to the workpiece surface and improve the stability of gripping.

[0055] The calibration module 5 includes a calibration plate 51 and an automatic calibration unit 52.

[0056] Specifically, the automatic calibration unit 52 is used to calibrate the visual perception module 2 and the robot body 1, ensuring accurate alignment between the visual coordinate system and the robot coordinate system, guaranteeing grasping accuracy. The automatic calibration unit 52 can automatically complete the calibration process periodically or according to user needs, without manual intervention, simplifying the operation process and improving the working efficiency and long-term stability of the device.

[0057] Specifically, the workflow of this invention is as follows:

[0058] First, the visual perception module 2 and the robot body 1 are automatically calibrated by the calibration module 5 to ensure that the coordinate system is aligned.

[0059] Secondly, the visual perception module 2 is activated, the multispectral supplementary lighting component 22 automatically switches the supplementary lighting mode according to the ambient light and the material of the workpiece, the 3D vision component 21 collects the three-dimensional image information of the workpiece, the image enhancement unit 24 enhances the image, and the data preprocessing unit 23 completes the denoising, completion and separation of adhering objects of the point cloud data through algorithms, and extracts the position, posture and other feature information of the workpiece.

[0060] Secondly, the main controller 31 of the control module 3 receives the data transmitted by the visual perception module 2, the path planning unit 32 uses the improved A* algorithm to plan the optimal grasping path, and the error compensation unit 33 predicts and compensates for the pose deviation in real time.

[0061] Next, the main controller 31 controls the robotic arm 11 of the robot body 1 to move along the planned path, and the flexible gripper 41 of the end effector module 4 adjusts the gripping force according to the characteristics of the workpiece to complete the workpiece gripping.

[0062] It should be noted that during the grasping process, the human-machine interaction unit 34 monitors the operating status of the equipment in real time. If any abnormality occurs, an alarm signal will be issued in a timely manner. Users can view the fault information through the touch screen 341 or press the emergency stop button 342 to stop the machine in an emergency.

[0063] Finally, after the gripping is completed, the robotic arm 11 moves the workpiece to the designated position, completing the gripping task. The device then resets and prepares for the next gripping operation.

[0064] In summary, through the collaborative design of various modules, this invention effectively improves the grasping accuracy, anti-interference ability, and adaptability to working conditions. It has the advantages of convenient operation and stable operation, and can be adapted to the workpiece grasping needs in different scenarios.

[0065] In the description of this specification, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0066] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0067] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A robot vision-guided grasping device, characterized by, include: The robot consists of a main body (1), a vision perception module (2), a control module (3), an end effector module (4), and a calibration module (5), among which, The robot body (1) includes a robotic arm (11) and a base (12). The robotic arm (11) adopts a multi-degree-of-freedom serial structure and has a flange at its end for connecting the end effector module (4). The base (12) integrates a servo drive motor and a braking unit. The visual perception module (2) includes a 3D vision component (21), a multispectral supplementary lighting component (22), and a data preprocessing unit (23). The 3D vision component (21) is composed of a multi-line laser structured light camera and a ToF camera. The data preprocessing unit (23) integrates an FPGA hardware acceleration chip. The control module (3) includes a main controller (31), a path planning unit (32) and an error compensation unit (33). The main controller (31) adopts a CPU+GPU+FPGA heterogeneous computing architecture. The calibration module (5) includes a calibration plate (51) and an automatic calibration unit (52).

2. The robot vision-guided grasping device according to claim 1, characterized in that, The multispectral supplementary lighting component (22) includes a blue light supplementary lighting unit (221) and an infrared supplementary lighting unit (222), wherein, The blue light supplement unit (221) is used to suppress the reflective interference of highly reflective materials; The infrared supplementary light unit (222) is used to improve the imaging clarity of the light-absorbing material, and the multispectral supplementary light component (22) automatically switches the supplementary light mode according to the ambient light and the material of the workpiece.

3. The robot vision-guided grasping device according to claim 1, characterized in that, The data preprocessing unit (23) uses an adaptive filtering algorithm and a PointNet++ deep learning segmentation algorithm to achieve noise reduction, completion and separation of sticky objects in point cloud data.

4. The robot vision-guided grasping device according to claim 1, characterized in that, The path planning unit (32) uses an improved A* algorithm to plan the optimal grasping path and has a dynamic obstacle avoidance function. The error compensation unit (33) uses a Kalman filter prediction algorithm to compensate for the pose deviation in real time and combines the force sensor feedback data to realize the adaptive adjustment of the grasping force.

5. The robot vision-guided grasping device according to claim 1, characterized in that, The 3D vision component (21) has a resolution of not less than 1280×1024, a scanning frequency of ≥30Hz, a depth accuracy of ≤0.05mm, and a working distance of 300–1500mm.

6. The robot vision-guided grasping device according to claim 1, characterized in that, The error compensation unit (33) controls the pose error within ±0.05mm.

7. The robot vision-guided grasping device according to claim 1, characterized in that, The control module (3) also integrates a human-machine interaction unit (34), which includes a touch screen (341) for parameter setting, status monitoring and fault alarm, and an emergency stop button (342) for emergency shutdown.

8. The robot vision-guided grasping device according to claim 1, characterized in that, The end effector module (4) adopts a modular flexible gripper (41), the opening and closing angle of the gripper (41) is 0–90°, and the clamping force is adjustable from 5–500N.

9. A robot vision-guided grasping device according to claim 1, characterized in that, The visual perception module (2) also integrates an image enhancement unit (24), which employs a multi-frame fusion algorithm.

10. A robot vision-guided grasping device according to claim 1, characterized in that, The control module (3) communicates with the robot body (1), the vision perception module (2), and the end effector module (4) using the EtherCAT high-speed communication protocol, and the communication delay is ≤50ms.

Images