A general experimental platform suitable for visual grasping system
By designing a general experimental platform suitable for vision-based grasping and releasing systems, and utilizing a benchmark platform and precise positioning algorithms, the problem of lacking simulation of real working conditions during the development of vision-based grasping and releasing systems was solved, enabling efficient training and verification, and improving the robustness and versatility of the system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUIZHOU AEROSPACE TIANMA ELECTRICAL TECH
- Filing Date
- 2023-11-14
- Publication Date
- 2026-06-16
Smart Images
Figure CN117484500B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a general experimental platform suitable for vision-based grasping and releasing systems, belonging to the field of automation control technology. Background Technology
[0002] With the development of modern industrial technology, automated production lines have become widely used in industries such as industry, agriculture, logistics, and medicine, including automated inspection, automated installation, automated sorting and delivery, and logistics depalletizing and palletizing.
[0003] Machine vision-based grasping and releasing systems, as the core and key components of the aforementioned automated production lines, have always been a focus of research and development. In practical applications, various vision-based grasping and releasing systems need to be developed for different fields, application scenarios, and working conditions. These systems share two common needs: first, training and testing during the development process; and second, verification before system deployment. Training and testing during development help system developers quickly master system functions and status, rapidly correct system errors, improve the system, and enhance its robustness. Verification before deployment is essential before system commercialization; any untested grasping and releasing system cannot be directly applied to avoid unforeseen losses. Therefore, given the numerous vision-based grasping and releasing systems available, a universal experimental platform is urgently needed to simulate the grasping and releasing environment, closely resembling real-world working conditions, and assisting developers in training, testing, and verification before deployment. Summary of the Invention
[0004] To address the aforementioned technical problems, this invention provides a universal experimental platform suitable for vision-based grasping and releasing systems. This platform can fulfill the common needs of training, testing, and pre-launch verification for numerous vision-based grasping and releasing systems during their development process.
[0005] The present invention is achieved through the following technical solutions.
[0006] The present invention provides a universal experimental platform suitable for vision-based grasping and releasing systems, comprising a reference platform; a controller and an execution device are provided on the end face of the reference platform; a detection device is connected to the execution device via an adapter frame, and a grasping and releasing device is connected to one side of the adapter frame; the reference platform is provided with millimeter-level XY plane coordinates and radius data markings with the center of the execution device as the origin;
[0007] The platform is used as follows:
[0008] ① Obtain the actual position: Place the target object on the reference platform, and read the plane coordinates and radius of the target object on the reference platform through the plane coordinates and radius data markers of the reference platform to obtain the actual position of the target object;
[0009] ② Obtaining the calculated position: The controller runs the grasping and releasing algorithm. During grasping, the controller adjusts the position of the execution device to bring the target object into the working range of the detection device. Then, the detection device acquires an image of the target object and transmits the image data to the controller. The controller then runs the grasping algorithm to calculate the pose information of the target object. Finally, the controller adjusts the execution device according to the calculated pose information of the target object, inserts the grasping and releasing device into the target object, rotates it 90°, locks it, and grasps the target object to obtain the calculated position of the target object.
[0010] ③ Correct the grasping algorithm: Obtain the visual detection error by using the actual position and the calculated position, and correct the grasping algorithm by using the visual detection error until the calculated position obtained by the detection device is the same as the actual position or the visual detection error meets the requirements.
[0011] ④ Obtain confirmed location: Use the revised grasping algorithm to lock and grasp the target object again, and obtain the confirmed location of the target object;
[0012] ⑤ Correct the grasping algorithm again: Obtain the control error of the robotic arm by using the actual position and the confirmed position, and correct the grasping algorithm obtained in step ③ by using the control error of the robotic arm until the confirmed position controlled by the execution device is the same as the actual position or the control error of the robotic arm meets the requirements.
[0013] ⑥ Complete the grasping of the target object: During placement, input the coordinates to be placed in the placement algorithm, run the placement algorithm through the controller and control the execution device to transfer the target object to the designated position on the reference platform, rotate the grasping and placing device again by 90°, unlock and place the target object.
[0014] The detection device is one or two vision cameras, the execution device is a six-degree-of-freedom robotic arm, the gripping and releasing device is a mushroom-shaped lifting device, and the target object is a docking base.
[0015] The adapter frame includes a crossbeam with mounting plates connected to both ends; one side of the crossbeam is connected to the actuator via an adapter flange, and the other side is connected to the gripping and releasing device via an installation interface.
[0016] The mounting plate is connected to the crossbeam by locking screws, the adapter is installed at the end of the actuator by an adapter flange, and the detection device is connected to the crossbeam by the mounting plate.
[0017] The mounting plate is provided with fixing holes and arc-shaped rotating grooves, and the locking screws are set in the arc-shaped rotating grooves.
[0018] The actuator is installed at the origin of the coordinate system on the reference platform.
[0019] The crawling algorithm is as follows:
[0020] The contour information of the target object is identified using a YOLOv8 deep learning model. Then, the quadrilateral of the docking hole is fitted using the approxPolyDP polygon fitting algorithm to obtain the information of the four vertices. Finally, the PnP algorithm is used to obtain the pose information of the gripping device relative to the docking hole of the target object.
[0021] The placement algorithm is as follows:
[0022] The PID control model is used, and the model parameters are optimized using a neural network algorithm.
[0023] The mushroom head lifting device has a 5mm guide bevel at its end and the upper end of its docking base.
[0024] The visual detection error and the robotic arm control error both meet the requirement that the error between the gripping and releasing device and the target object is less than ±5mm.
[0025] The beneficial effects of this invention are as follows: it can intuitively and quickly obtain the precise planar coordinates and radius of the target object through a reference platform, and quickly obtain the experimental error through comparison, which can assist in correcting the grasping and releasing algorithm and improve the algorithm training efficiency; it can quickly replace different grasping and releasing devices to adapt to different grasping and releasing targets, thereby improving the versatility of the experimental platform; it can select to install one or two vision cameras as needed, and is compatible with monocular vision systems and binocular vision systems; the detection device can be freely adjusted in angle to adapt to various working conditions. Attached Figure Description
[0026] Figure 1 This is a schematic diagram of the structure of the present invention;
[0027] Figure 2 yes Figure 1 Schematic diagram of the intermediate transfer frame;
[0028] Figure 3 yes Figure 2 Schematic diagram of the intermediate transition flange;
[0029] Figure 4 yes Figure 2 Schematic diagram of the mounting plate in the middle;
[0030] Figure 5 A schematic diagram of the mushroom-head lifting device in this invention;
[0031] Figure 6 A schematic diagram of the docking base in this invention;
[0032] In the diagram: 1-Base platform, 2-Adapter frame, 3-Detection device, 4-Grabbing and releasing device, 5-Target object, 6-Actuating device, 7-Controller, 8-Adapter flange, 9-Crossbeam, 10-Mounting plate, 11-Locking screw, 12-Mounting interface. Detailed Implementation
[0033] The technical solution of the present invention is further described below, but the scope of protection is not limited to what is described.
[0034] Example 1
[0035] like Figures 1-5 As shown, the actuator 6 is a six-degree-of-freedom robotic arm, which is installed at the origin of the coordinate system on the reference platform 1. The adapter 2 is installed at the end of the robotic arm through the adapter flange 8. The detection device 3 is a vision camera, which is installed on both sides of the crossbeam 9 through the mounting plate 10. The gripping and placing device 4 is a mushroom-shaped lifting device, which is installed in the middle of the crossbeam 9 through the mounting interface 12. The target object 5 is placed within the coordinate range of the reference platform 1.
[0036] Preferably, the mushroom-head lifting device has the advantages of high reliability and good stability. Target object 5 uses a docking base specifically designed for the mushroom-head lifting device, such as... Figure 6 As shown.
[0037] Furthermore, the mushroom-shaped lifting device has a 5mm guide bevel at the end and the upper end of the docking base, respectively, to ensure that the mushroom-shaped lifting device can successfully connect to the docking base through the docking hole within a ±5mm error range, thus achieving gripping.
[0038] Specifically, the reference platform 1 is equipped with millimeter-level XY plane coordinates and radius data markers with the center of the actuator 6 as the origin. It can quickly obtain the precise plane coordinates and radius of the target object 5 on the reference platform 1. By comparing the actual position with the calculated position, the visual detection error can be quickly obtained to correct the grasping algorithm and improve the training efficiency of the grasping algorithm.
[0039] Specifically, the gripping and releasing device 4 is installed on the crossbeam 9 using a universal screw connection method, and different gripping and releasing devices 4 can be quickly replaced as needed to adapt to different gripping and releasing targets 5.
[0040] Specifically, the detection device 3 is mounted on the crossbeam 9 via the mounting plate 10. It can be equipped with one or two vision cameras as needed, and is compatible with monocular vision systems and binocular vision systems. It has the advantages of high measurement accuracy and the ability to measure three-dimensional space.
[0041] Specifically, the mounting plate 10 is designed with fixing holes and an arc-shaped rotating slide, with locking screws 11 installed in the slide. Loosening the locking screws 11 allows the mounting plate 10 to move the vision camera to freely adjust its angle around the fixing holes as the origin and the arc-shaped slide as the radius, adapting to different working conditions; tightening the locking screws 11 will fix the vision camera in the current position.
[0042] Example 2
[0043] The method of using this invention is as follows:
[0044] First, the target object 5 is placed on the reference platform 1. The precise planar coordinates and radius of the target object 5 on the reference platform 1 are quickly read using planar coordinate and radius data markers and recorded as the actual position. Then, the grab-and-release algorithm is run in the controller 7, which controls the detection device 3 to acquire images of the target object 5 and calculate the spatial position of the target object 5, which is recorded as the calculated position. Finally, by comparing the actual position with the calculated position, the visual detection error can be quickly obtained. The grab-and-release algorithm is corrected using the visual detection error until the calculated position obtained by the detection device 3 is the same as the actual position or the visual detection error meets the requirements.
[0045] Once the visual detection error meets the requirements, the six-degree-of-freedom robotic arm is controlled to move directly above the target object 5, and the mushroom-head lifting device is adjusted to rotate vertically downwards to the gripping hole of the target object 5. The detection device 3 then acquires an image of the target object 5 again and calculates the spatial position of the target object 5, which is recorded as the confirmed position. Then, by comparing the actual position with the confirmed position, the robotic arm control error can be quickly obtained. The gripping and releasing algorithm is corrected using the robotic arm control error until the confirmed position controlled by the execution device 6 is the same as the actual position or the robotic arm control error meets the requirements.
[0046] Furthermore, when both the visual inspection error and the robotic arm control error meet the requirements, that is, the error between the current mushroom head lifting device and the docking base is less than ±5mm, the mushroom head lifting device moves downward to insert into the docking hole and rotates 90 degrees to lock, thus completing the grasping of the target object.
[0047] Preferably, during the grasping process, firstly, the controller 7 adjusts the execution device 6 to bring the target 5 into the working range of the detection device 3; secondly, the detection device 3 acquires an image of the target 5 and transmits the image data to the controller 7; then, the controller 7 runs the grasping algorithm to calculate the pose information of the target 5; finally, the controller 7 adjusts the execution device 6 according to the calculated pose information of the target 5, inserts the mushroom-head lifting device into the docking hole and rotates it 90° to lock and grasp the target.
[0048] Preferably, during placement, the coordinates to be placed are first input into the placement algorithm, and then the placement algorithm is run by the controller 7 and the execution device 6 is controlled to transfer the target object 5 to the designated position on the reference platform 1; finally, the mushroom head hanger is rotated 90° again to unlock and place the target object 5.
[0049] Specifically, the grasping algorithm first uses a YOLOV8 deep learning model to identify the contour information of the target object, then uses the approxPolyDP polygon fitting algorithm to fit the quadrilateral of the docking hole and obtain the information of the four vertices, and finally uses the PnP algorithm to obtain the pose information of the grasping device relative to the docking hole of the target object.
[0050] Specifically, the placement algorithm uses a PID control model and optimizes the model parameters through a neural network algorithm.
Claims
1. A universal experimental platform suitable for vision-based grasping and releasing systems, comprising a reference platform (1), characterized in that: A controller (7) and an execution device (6) are provided on the end face of the reference platform (1); a detection device (3) is connected to the execution device (6) via an adapter (2), and a gripping and releasing device (4) is connected to one side of the adapter (2); the reference platform (1) is provided with millimeter-level XY plane coordinates and radius data markings with the center of the execution device (6) as the origin; The platform is used as follows: ① Obtain the actual position: Place the target object (5) on the reference platform (1), and read the plane coordinates and radius of the target object (5) on the reference platform (1) through the plane coordinates and radius data markers of the reference platform (1) to obtain the actual position of the target object (5); ② Obtain the solution position: The controller (7) runs the grasping and releasing algorithm. When grasping, the controller (7) adjusts the position of the execution device (6) so that the target object (5) enters the working range of the detection device (3). Then, the detection device (3) collects the image of the target object (5) and transmits the image data to the controller (7). The controller (7) then runs the grasping algorithm to solve the pose information of the target object (5). Finally, the controller (7) adjusts the execution device (6) according to the solved pose information of the target object (5), inserts the grasping and releasing device (4) into the target object (5) and rotates it 90°, locks and grasps the target object (5), and obtains the solution position of the target object (5). ③ Correct the grasping algorithm: Obtain the visual detection error through the actual position and the calculated position, and correct the grasping algorithm through the visual detection error until the calculated position obtained by the detection device (3) is the same as the actual position or the visual detection error meets the requirements; ④ Obtain the confirmed position: Use the modified grasping algorithm to lock and grasp the target object (5) again, and obtain the confirmed position of the target object (5); ⑤ Correct the grasping algorithm again: Obtain the control error of the robotic arm by using the actual position and the confirmed position, and correct the grasping algorithm obtained in step ③ by using the control error of the robotic arm until the confirmed position controlled by the execution device (6) is the same as the actual position or the control error of the robotic arm meets the requirements. ⑥ Complete the grasping of the target object (5): When placing, input the coordinates to be placed in the placement algorithm, run the placement algorithm through the controller (7) and control the execution device (6) to transfer the target object (5) to the designated position on the reference platform (1), rotate the grasping and placing device (4) again by 90°, unlock and place the target object (5).
2. The universal experimental platform for vision-based grasping and releasing systems as described in claim 1, characterized in that: The detection device (3) is a single or two vision cameras, the execution device (6) is a six-degree-of-freedom robotic arm, the gripping and releasing device (4) is a mushroom-shaped lifting device, and the target object (5) is a docking base.
3. The universal experimental platform for vision-based grasping and releasing systems as described in claim 1, characterized in that: The adapter frame (2) includes a crossbeam (9), with mounting plates (10) connected to both ends of the crossbeam (9); one side of the crossbeam (9) is connected to the actuator (6) via an adapter flange (8), and the other side is connected to the gripping and releasing device (4) via an installation interface (12).
4. The universal experimental platform for vision-based grasping and releasing systems as described in claim 3, characterized in that: The mounting plate (10) is connected to the crossbeam (9) by locking screws (11), the adapter (2) is installed at the end of the actuator (6) by the adapter flange (8), and the detection device (3) is connected to the crossbeam (9) by the mounting plate (10).
5. The universal experimental platform for vision-based grasping and releasing systems as described in claim 3, characterized in that: The mounting plate (10) is provided with fixing holes and arc-shaped rotating slide grooves, and the locking screw (11) is set in the arc-shaped rotating slide grooves.
6. The universal experimental platform for vision-based grasping and releasing systems as described in claim 1, characterized in that: The actuator (6) is installed at the origin of the coordinate system on the reference platform (1).
7. The universal experimental platform for vision-based grasping and releasing systems as described in claim 1, characterized in that: The crawling algorithm is as follows: Using the YOLOV8 deep learning model, the contour information of the target object (5) is identified. Then, the approxPolyDP polygon fitting algorithm is used to fit the quadrilateral of the docking hole and obtain the information of the four vertices. Finally, the PnP algorithm is used to obtain the pose information of the gripping and releasing device (4) relative to the docking hole of the target object (5).
8. The universal experimental platform for vision-based grasping and releasing systems as described in claim 1, characterized in that: The placement algorithm is as follows: The PID control model is used, and the model parameters are optimized using a neural network algorithm.
9. The universal experimental platform for vision-based grasping and releasing systems as described in claim 2, characterized in that: The mushroom head lifting device has a 5mm guide bevel at its end and the upper end of its docking base.
10. The universal experimental platform for vision-based grasping and releasing systems as described in claim 1, characterized in that: The visual detection error and the robotic arm control error both meet the requirement that the error between the gripping and releasing device (4) and the target object (5) is less than ±5mm.