Robot automatic bag unpacking method and system based on three-dimensional point cloud

Abstract

The application discloses a kind of robot automatic bagging methods and systems based on three-dimensional point cloud, the method includes: the surrounding information of the material stacking disc is obtained by 3D camera;Each bagged material package is segmented by point cloud clustering;Compact bounding box is obtained, and the center point position and its attitude of each bagged material package are solved by compact bounding box;The center points of all bagged material packages are sorted;Modbus Tcp communication protocol is used to connect with robot, and the center point position and attitude of bagged material package are transmitted to robot;Robot end gripping unit sequentially carries out bagged material package gripping, unpacking, pouring, bag removal, reset, so on and so forth, until bagged material package on material stacking disc is all gripped, unpacked and finished;The application can be applied to complex environment under construction site, can satisfy the requirement of intelligent automatic unpacking, avoid the occurrence of professional disease of active lime post in mixing station, also greatly improve construction efficiency.

Description

Technical Field

[0001] This invention belongs to the field of automatic bag opening technology, and more specifically, relates to a robot automatic bag opening method and system based on three-dimensional point cloud. Background Technology

[0002] Building materials such as cement and refractory mortar are typically packaged in bags. During construction, these bags are usually manually handled, unpacked, and poured into the mixer, generating a large amount of dust. This makes the work at the mixing plant a high-risk area for occupational diseases such as lumbar muscle strain and silicosis. In recent years, with the booming development of the manufacturing industry, a large number of robots have replaced human labor in material handling. Compared to traditional manual handling methods, robots have a larger load capacity, higher accuracy, and significantly higher efficiency. Using robots to automatically unpack bagged materials can not only free construction workers from heavy handling work but also effectively prevent occupational diseases.

[0003] Because the irregularly shaped material filling in the bagged material packs is not necessarily uniform, the robot's gripping end needs to hold the pack below the center line to prevent it from falling off during handling. Moreover, the bagged material packs on the stack are tightly packed together, making it impossible for the robot's gripping end to hold them below the center line. In addition, the robot needs to know the position of the target bagged material pack during the unpacking process, using laser sensor technology to identify objects for visual guidance. However, due to the complexity of the construction site environment, the bagged material packs are stacked together, and there are many surrounding interference objects, resulting in low accuracy in classifying and segmenting the target bagged material packs.

[0004] Therefore, there is an urgent need to invent a robotic automatic bag-opening system and method based on three-dimensional point clouds to accurately identify bagged material packages and separate the material packages on both sides of the target material package so that the robot gripping end can hold it below the center line of the bagged material package, thus avoiding the material package from falling off during the process of being unable to be gripped and transported. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a robotic automatic bag-opening method and system based on 3D point clouds. By setting a gripping unit at the robot's end effector, with a bag-gripping component and a bag-dispensing component on the gripping unit, the two components respectively perform the gripping of the target bagged material and the dispensing of the bags on both sides of the target bagged material, thus facilitating the gripping unit to hold the bag below its center line and increasing the stability of the entire robotic automatic bag-opening operation. Each bag is segmented using point cloud preprocessing and point cloud segmentation methods. Then, the center point position and orientation of the bag are calculated using a compact bounding box. After sorting the center points, a connection is established with the robot via the Modbus TCP communication protocol, transmitting the center point position and orientation of the bagged material to the robot, driving the robot's end effector gripping unit to perform sequential bag-gripping operations. This invention is suitable for complex construction site environments, meets the requirements of intelligent automatic bag opening, and significantly improves construction efficiency while avoiding occupational diseases in concrete mixing plant workers.

[0006] To achieve the above objectives, one aspect of the present invention provides a robot-based automatic bag-unpacking method based on three-dimensional point clouds, comprising the following steps:

[0007] S1: Obtain environmental information around the stacking tray using a 3D camera;

[0008] S2: Segment each bagged material package using point cloud clustering;

[0009] S3: Obtain the compact bounding box and use the compact bounding box to solve for the center point position and orientation of each bagged material package;

[0010] S4: Sort the center points of all bagged material packets;

[0011] S5: Establishes a connection with the robot via Modbus TCP communication protocol and transmits the center point position and orientation of the bagged material pack to the robot;

[0012] S6: The robot's end-effector gripping unit cyclically grips, unpacks, pours, removes, and resets the bagged material packages until all the bagged material packages on the stacking tray have been gripped and unpacked.

[0013] Furthermore, obtaining the environmental information around the stacking pan in step S1 includes the following steps:

[0014] S11: Collect raw point cloud data of the environment around the stacking pallet using the lidar in the 3D camera above the stacking pallet, and perform downsampling operation on the raw point cloud data.

[0015] S12: Transfer each point in the point cloud data after the downsampling operation to the robot coordinate system;

[0016] S13: Remove point cloud noise from interfering objects around bagged material packages;

[0017] In step S12, transferring each point in the downsampling point cloud data to the robot coordinate system includes the following steps:

[0018] Hand-eye calibration is performed on the 3D camera and the robot to obtain the transformation matrix from the 3D camera coordinate system to the robot coordinate system;

[0019] The point cloud data after downsampling is subjected to rigid body transformation according to the transformation matrix, and the point cloud data after downsampling is transformed into the robot coordinate system.

[0020] Furthermore, the removal of interfering point cloud noise around the bagged material in step S13 includes the following steps:

[0021] S131: Perform pass-through filtering on the point cloud data after rigid body transformation in step S12;

[0022] S132: Perform radius filtering on the point cloud data after pass-through filtering to remove noise from the point cloud of interfering objects around the bagged material package.

[0023] Further, in step S3, obtaining the compact bounding box and solving for the center point position and orientation of each bagged material using the compact bounding box includes the following steps:

[0024] S31: Obtain the eigenvalues ​​and eigenvectors of the point cloud data of the material package through principal component analysis (PCA), and take the eigenvector corresponding to the largest eigenvalue as the principal axis direction of the OBB bounding box;

[0025] S32: Solve for the minimum and maximum coordinates of the target material bag point cloud in the current coordinate system to determine the four vertices of the bounding box and obtain the OBB bounding box;

[0026] S33: Transform the target material bag point cloud to the origin of the coordinate axis through rigid body transformation, and then rotate it around the Z-axis in the robot coordinate system by multiple angles. Take the angle when the AABB bounding box surface area is the smallest as the compensation value to compensate the OBB bounding box angle and obtain a compact bounding box.

[0027] S34: Calculate the Z-axis angle θ from the compact bounding box. z Convert all to

[0028] S35: For the Z-axis angle at The point takes [θ] z -π], for the Z-axis angle in The point takes [θ] z +π], thereby obtaining the center point position and orientation of each bagged material package.

[0029] Furthermore, in step S4, the center points of all bagged material packages are sorted to facilitate the robot's subsequent sequential grasping from the edge to the center, including the following steps:

[0030] S41: Determine the center point X, Y, Z and orientation θ of all material packages. x θ y θ z Save the data to list L1, and then iterate through list L1 to find the height Z of the center point of the bagged material with the largest Z value. max ; Specify the Z range as [Z max -100, Z max ] Filter out the center point set of the top layer of material bags and add it to list L2;

[0031] S42: Traverse list L2 to find the minimum value X. min Specify X as the range [X min X min +300], obtain the set of center points of the edge column, and add it to list L3;

[0032] S43: Traverse list L3 to find the minimum value of Y. min Add this center point to list L4, and repeat this step until list L3 is empty;

[0033] S44: Repeat steps S42 and S43 until list L2 is empty, thus completing the sorting of the center point of the material bag from the edge to the center.

[0034] Another aspect of the present invention provides a robotic automatic bag-unpacking system based on three-dimensional point clouds, including an automatic bag-unpacking mechanism, a 3D camera, a bag-unpacking hopper, and a stacking tray; wherein,

[0035] The automatic bag-opening mechanism is used to automatically grasp bagged material packages, and includes a robot and a grasping unit mounted on the robot; the grasping unit includes a connecting end, a support, a bag-grabbing assembly, and a bag-dispensing assembly; the connecting end is mounted on the support; the connecting end is used to connect to the end of the robot;

[0036] The bottom of the support is provided with a base plate assembly; the bag gripping assembly and the bag pulling assembly are both provided on the base plate assembly;

[0037] The 3D camera is located at the upper front of the robot and is used to collect point cloud data of the bags of materials to be unpacked and their environment at the work site.

[0038] The bag-opening hopper is located within the side operating radius of the robot, and includes a loading hopper and a saw blade that spans the top middle of the loading hopper. After the gripping unit grips the bagged material package, it performs a pushing and pulling action on the saw blade to achieve automatic bag opening.

[0039] The stacking tray containing the bagged material is positioned within the scanning range of the 3D camera and the grasping range of the robot;

[0040] The robot's end-effector gripping unit adjusts its position and posture in the workspace by coordinating the joints driven by servo motors. Simultaneously, with precise identification and positioning using 3D camera vision technology, the robot sequentially completes the tasks of selecting bagged material packets, emptying them after opening, and recycling the bags.

[0041] Furthermore, the support is an I-shaped frame structure, including a first support rod and a second support rod arranged in parallel intervals, and a first connecting rod and a second connecting rod arranged vertically between the first support rod and the second support rod;

[0042] The base plate assembly includes a first base plate, a second base plate, and a third base plate disposed at the bottom of the first support rod and the second support rod; the second base plate is disposed at the center of the bottom of the first support rod and the second support rod; the first base plate and the third base plate are symmetrically disposed at the bottom ends of the first support rod and the second support rod.

[0043] Furthermore, the bag-grabbing assembly includes a first bag-grabbing unit disposed on the first base plate and a second bag-grabbing unit disposed on the third base plate; the first bag-grabbing unit and the second bag-grabbing unit have the same structure and are arranged symmetrically facing each other; the first bag-grabbing unit includes a bearing seat, a rotating shaft, a hook seat, a bag-grabbing tooth assembly, a torsion bar, and a first cylinder disposed at the bottom of the first base plate;

[0044] Two bearing seats are arranged parallel to each other at intervals on the bottom of the first base plate; the rotating shaft is disposed on the two bearing seats;

[0045] One end of the torsion bar is ring-shaped and sleeved in the middle of the rotating shaft, while the other end is connected to the telescopic rod pin of the first cylinder.

[0046] The first cylinder is fixed to the middle of the upper surface of the first base plate;

[0047] The hook seats are respectively provided at both ends of the rotating shaft;

[0048] The bag-gripping tooth assembly includes a fixing plate disposed on two hook seats and a plurality of bag-gripping teeth evenly arranged in parallel on the fixing plate; the front end of the bag-gripping teeth is hook-shaped.

[0049] Furthermore, the bag-pulling assembly is located at the bottom of the second base plate and includes a base, a scissor mechanism and a second cylinder disposed on the base, and a plurality of bag-pulling teeth disposed on the scissor mechanism.

[0050] The base is provided with a guide groove; several pins are movably arranged in the guide groove.

[0051] The scissor mechanism is mounted on the bottom of the base via several pins;

[0052] The base includes a first seat body and a second seat body arranged in parallel and spaced apart; the guide grooves are respectively provided on the first seat body and the second seat body.

[0053] The second cylinder is located between the first and second seats;

[0054] The cylinder body of the second cylinder is fixed to a pin on the first seat near the second seat; the piston rod end of the second cylinder is fixed to a pin on the second seat near the first seat.

[0055] By driving the second cylinder to change the distance between the two pins connected to it, the scissor mechanism can be unfolded or folded, thereby completing the opening and closing of the bag-pulling teeth.

[0056] Furthermore, the scissor mechanism includes several connected or parallel scissor units; each pin has two scissor units spaced parallel to each other from top to bottom;

[0057] Each scissor lift unit includes two links arranged in a cross configuration;

[0058] The centers of the two connecting rods are connected by the pin.

[0059] Four connecting rods are provided on the same pin;

[0060] The bag-pulling teeth are hinged at the ends of the connecting rods at corresponding positions on the upper and lower scissor units.

[0061] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0062] 1. The automatic bag-unpacking robot system based on three-dimensional point cloud of the present invention, by setting a gripping unit at the end of the robot, the gripping unit is equipped with a bag-gripping component and a bag-dispensing component, which respectively perform the gripping of the target bag and the dispensing of the bags on both sides of the target bag, thereby facilitating the gripping unit to clamp the bag below the center line, increasing the stability of the entire automatic bag-unpacking operation process of the robot; in addition, the front end of the gripping teeth of the gripping component is hook-shaped. During operation, the gripping teeth clamp the bag below the center line of the bag, and when the gripping teeth are lifted, the hook end penetrates the bag without causing it to fall off.

[0063] 2. The automatic bag-opening robot system based on three-dimensional point cloud of the present invention has a saw blade horizontally installed on the top of the loading hopper of the bag-opening hopper. After the gripping unit grips the bag of material, it performs a pushing and pulling action on the saw blade to realize automatic bag opening, which can meet the requirements of intelligent automatic bag opening.

[0064] 3. The automatic bag-unpacking robot system based on 3D point cloud of the present invention uses a six-axis articulated robot. The robot's end effector, the gripping unit, is adjusted in position and posture within the workspace by the cooperation of the joints driven by servo motors. Simultaneously, a high-resolution and high-precision 3D scanner is used to scan the industrial scene. LiDAR is used to collect point cloud data of the bagged materials and the surrounding environment, identify the target material bag and provide visual guidance. Under the precise identification and positioning of the 3D camera's visual technology, the system sequentially completes the selection of the bagged material, the emptying after opening, and the bag recycling. This enables automated bag-unpacking operations.

[0065] 4. The automatic bag-unpacking method for robots based on 3D point clouds of the present invention segments each bag through point cloud preprocessing and point cloud segmentation. Then, the center point position and orientation of the bag are calculated using a compact bounding box. After sorting the center points, a connection is established with the robot via the Modbus TCP communication protocol, and the center point position and orientation of the bag are transmitted to the robot, driving the robot's end-effector to perform sequential bag-grabbing operations. The present invention is suitable for complex construction site environments, can meet the requirements of intelligent automatic bag unpacking, and can greatly improve construction efficiency while avoiding occupational diseases of workers in concrete mixing plants. Attached Figure Description

[0066] Figure 1 This is a schematic diagram of the overall structure of the robot automatic bag-unpacking system based on three-dimensional point clouds according to an embodiment of the present invention;

[0067] Figure 2 This is a schematic diagram of the robotic automatic bag-opening system based on three-dimensional point cloud grasping material bags according to an embodiment of the present invention;

[0068] Figure 3 This is a schematic diagram of the three-dimensional structure of the gripping unit of the robot automatic bag-unpacking system based on three-dimensional point cloud according to an embodiment of the present invention.

[0069] Figure 4 This is a side view of the gripping unit of the robot automatic bag-unpacking system based on three-dimensional point cloud according to an embodiment of the present invention.

[0070] Figure 5 This is a schematic diagram of the bottom structure of the gripping unit of the robot automatic bag-unpacking system based on three-dimensional point cloud according to an embodiment of the present invention;

[0071] Figure 6 This is a schematic diagram of the gripping component of the robot automatic bag-unpacking system based on three-dimensional point clouds according to an embodiment of the present invention;

[0072] Figure 7 This is a three-dimensional structural diagram of the bag-removing component of the robot automatic bag-removing system based on three-dimensional point cloud according to an embodiment of the present invention.

[0073] Figure 8 This is a schematic diagram of the bottom structure of the bag-removing component of the robot automatic bag-removing system based on three-dimensional point cloud according to an embodiment of the present invention;

[0074] Figure 9 This is a schematic diagram of the gripping unit of the robot automatic bag-opening system based on three-dimensional point cloud according to an embodiment of the present invention, which separates the two sides of the bag when gripping the bag;

[0075] Figure 10 This is a flowchart illustrating the automatic bag-unpacking method for robots based on three-dimensional point clouds, according to an embodiment of the present invention.

[0076] Figure 11 This is a schematic diagram of the automatic bag-unpacking mechanism of the robot automatic bag-unpacking method based on three-dimensional point cloud according to an embodiment of the present invention.

[0077] Figure 12 This is a schematic diagram of the original point cloud collected by the robot automatic bag-unpacking method based on three-dimensional point cloud in an embodiment of the present invention.

[0078] Figure 13 This is a schematic diagram of the point cloud after voxel filtering, which is a method for automatic bag unpacking by a robot based on three-dimensional point cloud according to an embodiment of the present invention.

[0079] Figure 14 This is a schematic diagram of the point cloud after pass-through filtering in the robot automatic bag-unpacking method based on three-dimensional point cloud according to an embodiment of the present invention;

[0080] Figure 15 This is a schematic diagram of the point cloud after radius filtering, representing an embodiment of the present invention of a robot automatic bag-unpacking method based on three-dimensional point cloud.

[0081] Figure 16 This is a schematic diagram of the point cloud after region growth and clustering segmentation of the material bag according to the robot automatic bag opening method based on three-dimensional point cloud in an embodiment of the present invention;

[0082] Figure 17 This is a schematic diagram illustrating the solution of the center point of the compact bounding box in the robot automatic bag-unpacking method based on three-dimensional point cloud according to an embodiment of the present invention.

[0083] In all the accompanying drawings, the same reference numerals denote the same technical features, specifically: 1-automatic bag-opening mechanism, 11-robot, 12-gripping unit, 121-connecting end, 122-support, 1221-first support rod, 1222-second support rod, 1223-first connecting rod, 1224-second connecting rod, 1225-first base plate, 1226-second base plate, 1227-third base plate, 123-bag-gripping assembly, 13-shaft seat, 1 4-Rotating shaft, 15-Hook seat, 16-Bag gripping tooth assembly, 161-Fixing plate, 162-Bag gripping tooth, 17-Torsion bar, 18-First cylinder, 124-Bag-pulling assembly, 100-Base, 101-Guide groove, 102-Scissor mechanism, 2-3D camera, 103-Second cylinder, 104-Bag-pulling tooth, 105-Connecting rod, 106-Pin, 3-Bag-removing hopper, 31-Filling hopper, 32-Saw blade, 4-Stacking tray, 5-Bagged material. Detailed Implementation

[0084] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0085] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, when an element is referred to as "fixed to," "set on," or "provided on" another element, it can be directly on or indirectly on the other element. When an element is referred to as "connected to" another element, it can be directly connected to or indirectly connected to the other element. The terms "mounted," "connected," "linked," and "provided with" should be interpreted broadly. For example, it can refer to a fixed connection, a detachable connection, or an integral connection; it can refer to a mechanical connection or an electrical connection; it can refer to a direct connection or an indirect connection through an intermediate medium; it can refer to the internal communication of two elements or the interaction between two elements. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0086] Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0087] like Figures 1-9 As shown, one aspect of the present invention provides a robotic automatic bag-unpacking system based on three-dimensional point clouds, including an automatic bag-unpacking mechanism 1, a 3D camera 2, a bag-unpacking hopper 3, and a stacking tray 4; the automatic bag-unpacking mechanism 1 is used to automatically grasp bagged material packages 5, and includes a robot 11 and a grasping unit 12 disposed on the robot 11; the 3D camera 2 is used to collect point cloud data of objects and the environment at the work site, and is disposed in front of the robot 11; the bag-unpacking hopper 3 is disposed within the side running radius of the robot 11; the stacking tray 4 containing the bagged material packages 5 is disposed within the scanning range of the 3D camera 2 and the grasping range of the robot 11.

[0088] Furthermore, such as Figures 1-9 As shown, the robot 11 is a six-axis articulated robot. The robot 11 includes a bottom mounting base, a rotating base on the bottom mounting base, a large arm on the rotating base, a forearm connected to the large arm, a wrist body connected to the forearm, and a wrist connected to the wrist body. The gripping unit 12 is located at the end of the wrist. The bottom mounting base and rotating base, the rotating base and large arm, the large arm and forearm, the forearm and wrist body, the wrist body and wrist, and the wrist and gripping unit 12 are connected by different joints. Each joint is driven by a servo motor. The position and posture adjustment of the gripping unit 12 at the end of the robot 11 in the work space are achieved by the mutual cooperation of the joints driven by the servo motor. Under the precise identification and positioning of the 3D camera 2 and the efficient control of the automatic bag opening mechanism 1, the robot sequentially completes the selection of bagged material packs 5, the emptying after opening, and the bag recycling.

[0089] Furthermore, such as Figures 1-9 As shown, the gripping unit 12 includes a connecting end 121, a support 122, a bag-gripping assembly 123, and a bag-removing assembly 124; the connecting end 121 is disposed on the support 122; the connecting end 121 is used to connect to the end of the robot 11; the connecting end 121 is connected to the wrist; the support 122 is an I-shaped frame structure, and the connecting end 121 is disposed at the center of the upper surface of the I-shaped frame; a base plate assembly is provided at the bottom of the I-shaped frame of the support 122; the support 122... 22 includes a first support rod 1221 and a second support rod 1222 arranged parallel to each other, and a first connecting rod 1223 and a second connecting rod 1224 perpendicularly disposed between the first support rod 1221 and the second support rod 1222; the first connecting rod 1223 and the second connecting rod 1224 are arranged parallel to each other on both sides of the line connecting the midpoints of the first support rod 1221 and the second support rod 1222; the connecting end 121 is disposed at the top of the first connecting rod 1223 and the second connecting rod 1224;

[0090] Furthermore, such as Figures 1-9As shown, the base plate assembly includes a first base plate 1225, a second base plate 1226, and a third base plate 1227 disposed at the bottom of the first support rod 1221 and the second support rod 1222; the first base plate 1225, the second base plate 1226, and the third base plate 1227 are all vertically disposed between the first support rod 1221 and the second support rod 1222; the second base plate 1226 is disposed at the center of the bottom of the first support rod 1221 and the second support rod 1222; the first base plate 1225 and the third base plate 1227 are symmetrically disposed at the bottom ends of the first support rod 1221 and the second support rod 1222.

[0091] Furthermore, such as Figures 1-9 As shown, the bag-grabbing assembly 123 includes a first bag-grabbing unit disposed on the first base plate 1225 and a second bag-grabbing unit disposed on the third base plate 1227; the first bag-grabbing unit and the second bag-grabbing unit have the same structure and are arranged symmetrically facing each other; the first bag-grabbing unit includes a bearing seat 13, a rotating shaft 14, a hook seat 15, a bag-grabbing tooth assembly 16, a torsion bar 17, and a first cylinder 18 disposed at the bottom of the first base plate 1225; two bearing seats 13 are arranged parallel to each other at intervals at the bottom of the first base plate 1225; the rotating shaft 14 is disposed on the two bearing seats 13, and the longitudinal central axis of the rotating shaft 14 is arranged perpendicular to the first support rod 1221 and the second support rod 1222; one end of the torsion bar 17 is annular and is sleeved on the first support rod 1221 and the second support rod 1222. The rotating shaft 14 is located at the middle, with one end connected to the telescopic rod pin of the first cylinder 18. The first cylinder 18 is fixed to the middle of the upper surface of the first base plate 1225. The hook seats 15 are respectively provided at both ends of the rotating shaft 14. The bag-gripping tooth assembly 16 includes a fixing plate 161 provided on the two hook seats 15 and a plurality of bag-gripping teeth 162 evenly arranged in parallel on the fixing plate 161. The front end of the bag-gripping teeth 162 is hook-shaped. During operation, the bag-gripping teeth 162 clamp below the center line of the bag material. The rotating shaft 14 is driven to rotate by the first cylinder 18, which lifts the bag-gripping teeth 162 to grasp the bag material. When the bag-gripping teeth 162 are lifted, the hook end on the bag-gripping teeth 162 pierces into the bag material, which can prevent the bag material from falling off.

[0092] Furthermore, such as Figures 1-9As shown, the bag-pulling assembly 124 is located at the bottom of the second base plate 1226, and includes a base 100, a scissor mechanism 102 and a second cylinder 103 disposed on the base 100, and a plurality of bag-pulling teeth 104 disposed on the scissor mechanism 102; the base 100 is provided with a guide groove 101; a plurality of pins 106 are movably disposed in the guide groove 101; the scissor mechanism 102 is disposed at the bottom of the base 100 through the plurality of pins 106; the base 100 includes parallel spaced... The first and second seats are provided; the guide groove 101 is respectively provided on the first and second seats; the second cylinder 103 is provided between the first and second seats; the cylinder body of the second cylinder 103 is fixed to the pin on the first seat near the second seat; the piston rod end of the second cylinder 103 is fixed to the pin on the second seat near the first seat; by driving the second cylinder 103 to change the distance between the two pins connected thereto, the scissor mechanism 102 can be unfolded or folded, thereby completing the opening and retraction of the bag-pulling teeth.

[0093] Furthermore, such as Figures 1-9 As shown, the scissor mechanism 102 includes several connected or parallel scissor units; two scissor units are arranged parallel to each other from top to bottom on each pin 106; each scissor unit includes two cross-arranged connecting rods 105; the centers of the two connecting rods 105 are connected by the pin; four connecting rods are provided on the same pin 106; the bag-pulling teeth 104 are hinged to the ends of the connecting rods 105 at corresponding positions on the upper and lower scissor units;

[0094] Furthermore, such as Figures 1-9 As shown, the bag-opening hopper 3 includes a filling hopper 31 and a saw blade 32 spanning the top middle of the filling hopper 31. By installing the saw blade 32 on the filling hopper 31, the robot's gripping unit can perform a pushing and pulling action on the saw blade 32 after gripping the bag of material, thereby achieving automatic bag opening and smoothly pouring the material inside the bag into the filling hopper 31. In an embodiment of the present invention, the filling hopper 31 is the filling hopper of a mixer.

[0095] The automatic bag-unpacking system based on three-dimensional point cloud provided by the present invention achieves the position and posture adjustment of the gripping unit 12 at the end of the robot 11 in the work space by the cooperation of the joints driven by the servo motor. At the same time, under the precise identification and positioning of the vision technology of the 3D camera 2, the system sequentially completes the selection of bagged material packs 5, the emptying after opening, and the recycling of the bags.

[0096] like Figure 10 and Figure 11As shown, another aspect of the present invention provides a robot-based automatic bag-unpacking method based on three-dimensional point clouds, implemented using the aforementioned robot-based automatic bag-unpacking system based on three-dimensional point clouds, comprising the following steps:

[0097] S1: Obtain environmental information around the stacking tray using a 3D camera;

[0098] S2: Segment each bagged material package using point cloud clustering;

[0099] S3: Obtain the compact bounding box and use the compact bounding box to solve for the center point position and orientation of each bagged material package;

[0100] S4: Sort the center points of all bagged material packets;

[0101] S5: Establishes a connection with the robot via Modbus TCP communication protocol, and transmits the center point position and posture of the bagged material pack to the robot, driving the robot's end gripping unit to perform sequential gripping of the bagged material pack;

[0102] S6: The robot's end-effector gripping unit cyclically grips, unpacks, pours, removes, and resets the bagged material packages until all the bagged material packages on the stacking tray have been gripped and unpacked.

[0103] Furthermore, such as Figure 12 and Figure 13 As shown, in an embodiment of the present invention, obtaining the environmental information around the stacking tray in step S1 includes the following steps:

[0104] S11: Collect raw point cloud data of the environment around the stacking pallet using the lidar in the 3D camera above the stacking pallet, and perform downsampling operation on the raw point cloud data.

[0105] S12: Transfer each point in the downsampled point cloud data to the robot coordinate system by rotation and translation;

[0106] S13: Remove point cloud noise from interfering objects around bagged material packages.

[0107] Furthermore, in step S11, the original point cloud data volume is large. Downsampling can significantly reduce the number of points and significantly improve the subsequent processing speed. The downsampling of this invention is achieved through voxel filtering. The principle of voxel filtering is as follows: by specifying the voxel grid size, the original point cloud data is divided into several voxel grids. After solving the centroid of each grid in turn, the k-nearest neighbor algorithm is used to search for the point closest to the centroid in the grid to approximate all points in the entire grid. By reasonably setting the voxel grid size, the density of the original point cloud data can be greatly reduced while retaining the basic features.

[0108] Furthermore, in step S12, since there is a difference between the 3D camera coordinate system and the robot coordinate system, it is necessary to map each point in the point cloud data to the robot coordinate system. Specifically, hand-eye calibration is performed on the 3D camera and the robot to obtain the transformation matrix from the 3D camera coordinate system to the robot coordinate system. Based on the transformation matrix, a rigid body transformation (i.e., rotation and translation operation) is performed on the point cloud data after the downsampling operation to transform the downsampling point cloud data to the robot coordinate system.

[0109] Furthermore, such as Figure 14 and Figure 15 As shown, in step S13, due to the complexity of the on-site environment, there are many interfering objects around the bagged material package. These objects will also be collected by the 3D camera, which will not only slow down the processing speed of point cloud data, but also affect the final recognition accuracy of the material package. Therefore, it is necessary to remove these point cloud noises.

[0110] Step S13, the removal of noise from the point cloud of interfering objects around the bagged material package, includes the following steps:

[0111] S131: Perform pass-through filtering on the point cloud data after rigid body transformation in step S12; specifically, by dividing the maximum and minimum values ​​of the X, Y, and Z directions in the robot coordinate system, points within this threshold are retained, and points outside the threshold are deleted, thus completing the initial filtering operation.

[0112] S132: Radius filtering is applied to the point cloud data after pass-through filtering to effectively remove outliers and thus eliminate noise from the point cloud of interfering objects around the bagged material package.

[0113] Furthermore, such as Figure 16 As shown, in the embodiment of the present invention, in step S2, the point cloud clustering adopts the region growing method. By specifying the search method, limiting the minimum and maximum number of points in the cluster, setting the smoothing threshold, and setting the curvature threshold, the region growing algorithm will start growing from the initial seed point, add the points that meet the requirements among the nearby points to the seed point set, and continue to grow on this basis until no new points are added to the point set.

[0114] Furthermore, such as Figure 17 As shown, in an embodiment of the present invention, in step S3, the center point of the bagged material is solved by fitting the bagged material package with a bounding box. Since the bagged material package is not necessarily parallel to the coordinate axis, the AABB bounding box may cause a large error. Therefore, the present invention uses the OBB bounding box. The OBB bounding box is obtained by the following steps:

[0115] The eigenvalues ​​and eigenvectors of the point cloud data of the material package are obtained by principal component analysis (PCA), and the eigenvector corresponding to the largest eigenvalue is used as the principal axis direction of the OBB bounding box.

[0116] Find the minimum and maximum coordinates of the target material bag point cloud in the current coordinate system to determine the four vertices of the bounding box and obtain the OBB bounding box.

[0117] Furthermore, the pose of the target bag can usually be obtained by solving for the center point position and orientation of the OBB bounding box; however, due to the irregular shape of the bagged material and the randomness of OBB bounding box generation, the construction of the OBB bounding box may not be optimal, and the OBB bounding box is often too large. Therefore, it is necessary to solve for the smallest bounding box that best fits the point cloud data of the bag, that is, the compact bounding box. The specific solution process of the compact bounding box is as follows: the point cloud of the target bag is transformed to the origin of the coordinate axis through rigid body transformation, and then rotated by multiple angles around the Z-axis in the robot coordinate system. The angle at which the surface area of ​​the AABB bounding box is minimized is taken as the compensation value, and the OBB bounding box angle is compensated to obtain the compact bounding box.

[0118] Furthermore, due to the compact bounding box, the Z-axis angle θ is calculated. z Between [-π, π], under certain circumstances, the gripping unit at the end of the robot's sixth axis may need to rotate a large angle to reach a specified posture. Based on the above reasons, this invention uses the robot coordinate system Z-axis angle θ calculated by the compact bounding box. z Convert all to

[0119] For the robot coordinate system Z-axis angle in The point takes [θ] z -π], for the Z-axis angle of the robot coordinate system in The point takes [θ] z +π], thereby obtaining the center point position and orientation of each bagged material package.

[0120] Therefore, step S3 involves obtaining a compact bounding box and using the compact bounding box to determine the center point position and orientation of each bagged material. This process specifically includes the following steps:

[0121] S31: Obtain the eigenvalues ​​and eigenvectors of the point cloud data of the material package through principal component analysis (PCA), and take the eigenvector corresponding to the largest eigenvalue as the principal axis direction of the OBB bounding box;

[0122] S32: Solve for the minimum and maximum coordinates of the target material bag point cloud in the current coordinate system to determine the four vertices of the bounding box and obtain the OBB bounding box;

[0123] S33: Transform the target material bag point cloud to the origin of the coordinate axis through rigid body transformation, and then rotate it around the Z-axis in the robot coordinate system by multiple angles. Take the angle when the AABB bounding box surface area is the smallest as the compensation value to compensate the OBB bounding box angle and obtain a compact bounding box.

[0124] S34: Calculate the Z-axis angle θ from the compact bounding box. z Convert all to

[0125] S35: For the Z-axis angle at The point takes [θ] z -π], for the Z-axis angle in The point takes [θ] z +π], thereby obtaining the center point position and orientation of each bagged material package.

[0126] Furthermore, in an embodiment of the present invention, step S4 involves sorting the center points of all bagged material packages to facilitate subsequent sequential grasping by the robot from the edge to the center. This specifically includes the following steps:

[0127] S41: Determine the center point X, Y, Z and orientation θ of all material packages. x θ y θ z Save the data to list L1, and then iterate through list L1 to find the height Z of the center point of the bagged material with the largest Z value. max ; Specify the Z range as [Z max -100, Z max ] Filter out the center point set of the top layer of material bags and add it to list L2;

[0128] S42: Traverse list L2 to find the minimum value X. min Specify X as the range [X min X min +300], obtain the set of center points of the edge column, and add it to list L3;

[0129] S43: Traverse list L3 to find the minimum value of Y. min Add this center point to list L4, and repeat this step until list L3 is empty;

[0130] S44: Repeat steps S42 and S43 until list L2 is empty, thus completing the sorting of the center point of the material bag from the edge to the center.

[0131] Further, in step S6, when the gripping unit 12 at the end of the robot 11 grips the bagged material bag 5, the gripping unit 12 moves above the target material bag, first driving the bag-dispensing component 124 to dispense the material bags on both sides of the target material bag, and then driving the bag-gripping component 123 to clamp it below the center line of the bagged material bag to achieve precise gripping; after the robot grips the material bag, it turns to the top of the loading hopper 31 and performs a push-pull action on the saw blade 32 to complete the automatic unpacking; after unpacking, the amorphous material inside the material bag is automatically poured out, and at the same time the robot controls the robotic arm to shake and swing to accelerate the automatic pouring of the amorphous material; then, the gripping unit 12 at the end of the robot turns away from the loading hopper 31 to perform the opening and closing operation of the bag-gripping component 123 and the bag-dispensing component 124 to remove the empty bag; finally, it returns to the top of the stacking tray to grip the next target material bag.

[0132] The present invention provides a robotic automatic bag-opening method based on 3D point clouds. This method segments each bag using point cloud preprocessing and segmentation, then calculates the center point position and orientation of the bag using a compact bounding box. After sorting the center points, a connection is established with the robot via the Modbus TCP communication protocol. The center point position and orientation of the bag are transmitted to the robot, driving the robot's end-effector to sequentially grasp the bagged materials. This invention is suitable for complex construction site environments, meets the requirements for intelligent automatic bag opening, and significantly improves construction efficiency while avoiding occupational diseases in concrete mixing plant workers.

[0133] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for automatic bag unpacking by a robot based on 3D point clouds, characterized in that, Includes the following steps: S1: Obtain environmental information around the stacking tray using a 3D camera; S2: Segment each bagged material package using point cloud clustering; S3: Obtain the compact bounding box and use the compact bounding box to solve for the center point position and orientation of each bagged material package; S4: Sort the center points of all bagged material packets; S5: Establishes a connection with the robot via Modbus TCP communication protocol and transmits the center point position and orientation of the bagged material pack to the robot; S6: The robot's end-effector gripping unit cyclically grips, unpacks, pours, removes, and resets the bagged material packages until all the bagged material packages on the stacking tray have been gripped and unpacked. Step S3 involves obtaining a compact bounding box and using it to determine the center point position and orientation of each bagged material. This includes the following steps: S31: Obtain the eigenvalues ​​and eigenvectors of the point cloud data of the material package through PCA principal component analysis, and take the eigenvector corresponding to the largest eigenvalue as the principal axis direction of the OBB bounding box; S32: Solve for the minimum and maximum coordinates of the target material bag point cloud in the current coordinate system to determine the four vertices of the bounding box and obtain the OBB bounding box; S33: Transform the target material bag point cloud to the origin of the coordinate axis through rigid body transformation, and then rotate it around the Z-axis in the robot coordinate system by multiple angles. Take the angle when the AABB bounding box surface area is the smallest as the compensation value to compensate the OBB bounding box angle and obtain a compact bounding box. S34: The Z-axis angle calculated from the compact bounding box Convert all to ; S35: For the Z-axis angle at Point of action For the Z-axis angle at Point of action This allows us to obtain the center point position and orientation of each bagged material.

2. The automatic bag-unpacking method for robots based on three-dimensional point clouds according to claim 1, characterized in that, The acquisition of environmental information around the stacking pan in step S1 includes the following steps: S11: Collect raw point cloud data of the environment around the stacking pallet using the lidar in the 3D camera above the stacking pallet, and perform downsampling operation on the raw point cloud data. S12: Transfer each point in the point cloud data after the downsampling operation to the robot coordinate system; S13: Remove point cloud noise from interfering objects around bagged material packages; In step S12, transferring each point in the downsampling point cloud data to the robot coordinate system includes the following steps: Hand-eye calibration is performed on the 3D camera and the robot to obtain the transformation matrix from the 3D camera coordinate system to the robot coordinate system; The point cloud data after downsampling is subjected to rigid body transformation according to the transformation matrix, and the point cloud data after downsampling is transformed into the robot coordinate system.

3. The automatic bag-unpacking method for robots based on three-dimensional point clouds according to claim 2, characterized in that, Step S13, the removal of noise from the point cloud of interfering objects around the bagged material package, includes the following steps: S131: Perform pass-through filtering on the point cloud data after rigid body transformation in step S12; S132: Perform radius filtering on the point cloud data after pass-through filtering to remove noise from the point cloud of interfering objects around the bagged material package.

4. The automatic bag-unpacking method for robots based on three-dimensional point clouds according to claim 3, characterized in that, In step S4, the center points of all bagged material packages are sorted to facilitate the robot's sequential grasping from the edge to the center. This includes the following steps: S41: Determine the center point X, Y, Z and orientation of all material packages. , , Save to list In the middle, by traversing the list The height of the center point of the bagged material with the largest Z value was obtained. ; Specify the Z range as Filter out the center point set of the top layer of material bags and add it to the list. middle; S42: Traversing the list Find the minimum value of X Specify the range of X as Obtain the set of center points of a column of edges and add it to the list. middle; S43: Traversing the list Find the minimum value of Y Add this center point to the list In the middle, repeat this step repeatedly until the list is reached. Empty; S44: Repeat steps S42 and S43 until the list is complete. If empty, then the sorting of the material package center point from the edge to the center is completed.

5. A robotic automatic bag-opening system based on three-dimensional point clouds, used to implement the robotic automatic bag-opening method based on three-dimensional point clouds as described in any one of claims 1-4, characterized in that, It includes an automatic bag-opening mechanism (1), a 3D camera (2), a bag-opening hopper (3), and a stacking tray (4); among which, The automatic bag-opening mechanism (1) is used to automatically grab bagged material packages (5), which includes a robot (11) and a gripping unit (12) disposed on the robot (11); the gripping unit (12) includes a connecting end (121), a support (122), a bag-grabbing assembly (123), and a bag-pulling assembly (124); the connecting end (121) is disposed on the support (122); the connecting end (121) is used to connect to the end of the robot (11); The bottom of the support (122) is provided with a base plate assembly; the bag gripping assembly (123) and the bag pulling assembly (124) are both provided on the base plate assembly; The 3D camera (2) is located in front of the robot (11) and is used to collect point cloud data of the bagged material to be unpacked and its environment at the work site. The bag-removing hopper (3) is located within the side running radius of the robot (11), including a loading hopper (31) and a saw blade (32) spanning the top middle of the loading hopper (31). After the gripping unit (12) grips the bagged material package, it performs a pushing and pulling action on the saw blade (32) to achieve automatic bag removal. The stacking tray (4) containing the bagged material package (5) is located within the scanning range of the 3D camera (2) and the grasping range of the robot (11); By using servo motors to drive the joints of the robot (11) to cooperate with each other, the position and posture of the gripping unit (12) at the end of the robot (11) can be adjusted in the work space. At the same time, under the precise identification and positioning of the 3D camera (2) visual technology, the selection of bagged material packs (5), the pouring after opening and the recycling of bags are completed in sequence.

6. The robotic automatic bag-unpacking system based on three-dimensional point clouds according to claim 5, characterized in that: The support (122) is an I-shaped frame structure, including a first support rod (1221) and a second support rod (1222) arranged in parallel intervals, and a first connecting rod (1223) and a second connecting rod (1224) arranged vertically between the first support rod (1221) and the second support rod (1222). The base plate assembly includes a first base plate (1225), a second base plate (1226), and a third base plate (1227) disposed at the bottom of the first support rod (1221) and the second support rod (1222); the second base plate (1226) is disposed at the center of the bottom of the first support rod (1221) and the second support rod (1222); the first base plate (1225) and the third base plate (1227) are symmetrically disposed at the bottom ends of the first support rod (1221) and the second support rod (1222).

7. The robotic automatic bag-unpacking system based on three-dimensional point clouds according to claim 6, characterized in that: The bag-gripping assembly (123) includes a first bag-gripping unit disposed on the first base plate (1225) and a second bag-gripping unit disposed on the third base plate (1227); the first bag-gripping unit and the second bag-gripping unit have the same structure and are arranged symmetrically facing each other; the first bag-gripping unit includes a bearing seat (13), a rotating shaft (14), a hook seat (15), a bag-gripping tooth assembly (16), a torsion bar (17), and a first cylinder (18) disposed at the bottom of the first base plate (1225); Two bearing seats (13) are arranged parallel to each other at the bottom of the first base plate (1225); the rotating shaft (14) is disposed on the two bearing seats (13); One end of the torsion bar (17) is a ring and is fitted in the middle of the rotating shaft (14), while the other end is connected to the telescopic rod pin of the first cylinder (18). The first cylinder (18) is fixed to the middle of the upper surface of the first base plate (1225); The hook seat (15) is respectively provided at both ends of the rotating shaft (14); The bag-gripping tooth assembly (16) includes a fixing plate (161) on the two hook seats (15) and a plurality of bag-gripping teeth (162) evenly arranged in parallel on the fixing plate (161); the front end of the bag-gripping teeth (162) is hook-shaped.

8. The robotic automatic bag-unpacking system based on three-dimensional point clouds according to claim 7, characterized in that: The bag-pulling assembly (124) is located at the bottom of the second base plate (1226) and includes a base (100), a scissor mechanism (102) and a second cylinder (103) on the base (100), and a plurality of bag-pulling teeth (104) on the scissor mechanism (102). The base (100) is provided with a guide groove (101); a number of pins (106) are movably arranged in the guide groove (101). The scissor mechanism (102) is located at the bottom of the base (100) via a plurality of pins (106); The base (100) includes a first base body and a second base body arranged in parallel and spaced apart; the guide groove (101) is respectively provided on the first base body and the second base body; The second cylinder (103) is located between the first seat and the second seat; The cylinder body of the second cylinder (103) is fixed to the pin on the first seat near the second seat; the piston rod end of the second cylinder (103) is fixed to the pin on the second seat near the first seat; By driving the second cylinder (103) to change the distance between the two pins connected to it, the scissor mechanism (102) can be unfolded or folded, thereby completing the opening and closing of the bag-pulling teeth.

9. The robotic automatic bag-unpacking system based on three-dimensional point clouds according to claim 8, characterized in that: The scissor mechanism (102) includes several connected or parallel scissor units; each pin (106) has two scissor units arranged parallel to each other from top to bottom; Each scissor unit includes two cross-connecting links (105); The centers of the two connecting rods (105) are connected by the pin (106); Four connecting rods are provided on the same pin (106); The bag-pulling teeth (104) are hinged to the ends of the connecting rods (105) at corresponding positions on the upper and lower scissor units.

Images