A method, device, equipment and storage medium for randomly grasping rod-shaped workpieces

Abstract

The present invention discloses a method, device, equipment and storage medium for disorderly grasping of a rod-shaped workpiece, the method comprising: obtaining 2D images and three-dimensional point cloud data of all workpieces to be grasped, and obtaining the 6D posture of each workpiece to be grasped according to the three-dimensional point cloud data; extracting graspable workpieces from the 2D images to generate a graspable workpiece sequence; traversing the graspable workpieces in the graspable workpiece sequence, and generating a grasping point sequence of the current graspable workpiece according to the 6D posture of the current graspable workpiece; traversing the grasping points in the grasping point sequence, and selecting a grasping point as a target grasping point; generating a grasping path of the robot arm from the current posture to the target grasping point according to the current posture of the robot arm and the target grasping point, and then controlling the robot arm to move to the target grasping point according to the grasping path to grasp the current graspable workpiece. The present invention can realize continuous disorderly grasping of rod-shaped workpieces.

Description

Technical Field

[0001] The present invention relates to the field of intelligent manufacturing production process technology, and in particular to a method, device, equipment and storage medium for randomly grasping rod-shaped workpieces. Background Art

[0002] In today's machining industry, automated production has become an important way to improve production efficiency and reduce costs. With the promotion of Industry 4.0 and intelligent manufacturing, more and more companies are beginning to use automated equipment and robots to complete various tasks on the production line to replace traditional manual operations.

[0003] Among them, round rod-shaped workpieces are a common type of blank material. These workpieces usually need to be sent to machine tools for processing, heating furnaces for heating or casting and other processes. However, the traditional way of loading workpieces often relies on manual operation. In the current industrial production line, a large amount of manpower is required to load the workpieces, which is not only costly but also inefficient. Even if some companies use automatic loaders, the workpieces are often stacked and placed in irregular positions. For these stacked and disorderly workpieces, it is difficult for existing automatic equipment to continuously grasp the workpieces, and it is easy to have problems such as jamming and collision. This will cause long downtime of the production line, greatly affecting production efficiency. In addition, the existing production line is usually based on a fixed cycle time when running, and production cannot be easily interrupted. Summary of the invention

[0004] The present invention provides a method, device, equipment and storage medium for randomly grasping a rod-shaped workpiece, so as to solve the technical problem that the prior art cannot realize continuous and randomly grasping of the rod-shaped workpiece.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a method for randomly grasping a rod-shaped workpiece, comprising:

[0006] Acquire 2D images and 3D point cloud data of all workpieces to be grasped, and obtain the 6D pose of each workpiece to be grasped according to the 3D point cloud data;

[0007] Extracting graspable workpieces from the 2D image to generate a graspable workpiece sequence;

[0008] Traversing the graspable workpieces in the graspable workpiece sequence, and generating a grasping point sequence of the current graspable workpiece according to the 6D pose of the current graspable workpiece;

[0009] Traversing the grasping points in the grasping point sequence, and selecting a grasping point as a target grasping point;

[0010] According to the current posture of the robot arm and the target grasping point, a grasping path of the robot arm from the current posture to the target grasping point is generated, and then the robot arm is controlled to move to the target grasping point according to the grasping path to grasp the currently graspable workpiece.

[0011] As a preferred solution, the step of obtaining the 6D pose of each workpiece to be grasped according to the three-dimensional point cloud data includes:

[0012] Preprocessing the three-dimensional point cloud data; wherein the preprocessing includes: filtering, smoothing and reconstruction;

[0013] The features of each workpiece to be grasped are extracted from the preprocessed 3D point cloud data, and then the extracted features are matched with the preset 3D model of the bar material to obtain the 6D pose of the successfully matched workpiece to be grasped.

[0014] As a preferred solution, the step of extracting graspable workpieces from the 2D image and generating a graspable workpiece sequence includes:

[0015] The 2D image is input into a preset workpiece classification model, so that the workpiece classification model classifies the workpieces to be grasped that are successfully matched in the 2D image into graspable workpieces and unclawable workpieces, and outputs a graspable workpiece sequence.

[0016] As a preferred solution, the generation of the workpiece classification model includes:

[0017] Acquire several 2D images of workpieces;

[0018] Selecting the workpiece contour of each workpiece in the 2D image of the workpiece, and comparing the selected area of ​​each workpiece with a preset workpiece cross section, classifying the workpiece whose selected area is smaller than a preset proportion of the workpiece cross section as an ungraspable workpiece, and classifying the workpiece whose selected area is larger than a preset proportion of the workpiece cross section as a graspable workpiece, and then marking each workpiece according to the classification result;

[0019] A preset deep learning model is trained according to the annotation information to generate the workpiece classification model.

[0020] As a preferred solution, the method of generating a grasping point sequence of the currently graspable workpiece according to the 6D pose of the currently graspable workpiece includes:

[0021] Convert the 6D pose of the current graspable workpiece into a pose matrix;

[0022] A three-dimensional coordinate system is established with the center position of the current graspable workpiece as the origin, and then the preset grasping tool is rotated along the Y axis at a preset angle step to obtain a rotation matrix;

[0023] The rotated grabbing tool is offset by a preset distance along the Z axis to obtain a translation matrix;

[0024] According to the pose matrix, the rotation matrix and the translation matrix, a grasping point sequence of the currently graspable workpiece is obtained.

[0025] As a preferred solution, traversing the grasping points in the grasping point sequence and selecting a grasping point as a target grasping point includes:

[0026] Traverse the grasping points in the grasping point sequence, perform collision detection on the current grasping point, and determine whether the robot arm will collide with the surrounding environment when moving to the current grasping point. If no collision occurs, use the current grasping point as the target grasping point.

[0027] As a preferred solution, the step of generating a grasping path of the robot arm from the current posture of the robot arm to the target grasping point according to the current posture of the robot arm and the target grasping point includes:

[0028] Obtain preset path planning parameters and start path planning timing; wherein the path planning parameters include: sampling range, exploration step length, maximum sampling times and maximum planning time;

[0029] Performing random sampling according to the sampling range and a preset path planning method to generate a first path point, and calculating a next path point according to the first path point and the exploration step size;

[0030] Determine whether the current sampling number exceeds the maximum sampling number. If not, determine whether the current path planning time exceeds the maximum planning time. If not, calculate the position and posture of each part of the robot arm in Cartesian space.

[0031] According to the preset three-dimensional model of each part of the robotic arm and the posture, the collision detection of the robotic arm is performed to determine whether each part of the robotic arm will collide with the surrounding environment during the process of the robotic arm moving to the current path point;

[0032] If no collision occurs, determine whether the distance between the current path point and the target grasping point is less than the exploration step size. If so, use the current path point as the last path point of the grasping path to generate a grasping path from the current position of the robot arm to the target grasping point.

[0033] If it is not less than, then according to the preset greedy search strategy, the next path point is generated along the direction from the current path point to the target grasping point, and then the grasping path of the robot arm from the current posture to the target grasping point is generated.

[0034] Based on the above embodiment, another embodiment of the present invention provides a disordered grasping device for rod-shaped workpieces, comprising: a data acquisition module, a graspable workpiece sequence generation module, a grasping point sequence generation module, a target grasping point selection module, and a grasping path generation module;

[0035] The data acquisition module is used to acquire 2D images and 3D point cloud data of all workpieces to be grasped, and obtain the 6D pose of each workpiece to be grasped according to the 3D point cloud data;

[0036] The graspable workpiece sequence generation module is used to extract graspable workpieces from the 2D image and generate a graspable workpiece sequence;

[0037] The grasping point sequence generation module is used to traverse the graspable workpieces in the graspable workpiece sequence and generate a grasping point sequence of the current graspable workpiece according to the 6D posture of the current graspable workpiece;

[0038] The target grasping point selection module is used to traverse the grasping points in the grasping point sequence and select a grasping point as the target grasping point;

[0039] The grasping path generation module is used to generate a grasping path from the current posture of the robotic arm to the target grasping point according to the current posture of the robotic arm and the target grasping point, and then control the robotic arm to move to the target grasping point according to the grasping path to grasp the currently graspable workpiece.

[0040] Based on the above embodiments, another embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, the method for disorderly grasping rod-shaped workpieces described in the above invention embodiments is implemented.

[0041] Based on the above embodiments, another embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the disordered grasping method for rod-shaped workpieces described in the above invention embodiment.

[0042] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0043] The present invention provides a disordered grasping method for rod-shaped workpieces, by acquiring 2D images and three-dimensional point cloud data of all workpieces to be grasped, and obtaining the 6D posture of each workpiece to be grasped according to the three-dimensional point cloud data; extracting graspable workpieces from the 2D images to generate a graspable workpiece sequence; traversing the graspable workpieces in the graspable workpiece sequence, and generating a grasping point sequence of the current graspable workpiece according to the 6D posture of the current graspable workpiece; traversing the grasping points in the grasping point sequence, and selecting a grasping point as a target grasping point; generating a grasping path of the robotic arm from the current posture to the target grasping point according to the current posture of the robotic arm and the target grasping point, and then controlling the robotic arm to move to the target grasping point according to the grasping path to grasp the current graspable workpiece. The present invention can realize continuous grasping of stacked and disorderly placed workpieces, greatly improving the automation of workpiece loading, reducing labor costs, and improving production efficiency. Moreover, this method can adapt to different workpiece layouts and positions, and plan the grasping path of the robot arm in real time, effectively solving problems such as material jamming and machine collision that are prone to occur in the existing technology, making the production line operation more stable and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic flow chart of a method for randomly grasping a rod-shaped workpiece provided by one embodiment of the present invention;

[0045] Figure 2 It is a general flow chart of the disorderly grasping of the rod-shaped workpiece of the present invention;

[0046] Figure 3 It is a workpiece contour frame selection diagram of each workpiece in the workpiece 2D image;

[0047] Figure 4 It is a schematic diagram of the three-dimensional point cloud data of the workpiece to be grasped;

[0048] Figure 5 It is a schematic diagram of 2D image data of the workpiece to be grasped;

[0049] Figure 6 It is a schematic diagram of the generation of the grasping point sequence of the currently graspable workpiece;

[0050] Figure 7 It is a flowchart of crawling path planning;

[0051] Figure 8 It is a structural schematic diagram of a disordered grasping device for rod-shaped workpieces provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0054] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0055] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0056] In the description of the embodiments of the present application, the term "and / or" is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0057] In the description of the embodiments of the present application, the term "multiple" refers to more than two (including two). Similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).

[0058] In the description of the embodiments of the present application, unless otherwise clearly specified and limited, technical terms such as "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0059] Embodiment 1

[0060] Please refer to Figure 1 , is a flow chart of a method for randomly grasping a rod-shaped workpiece provided by an embodiment of the present invention, comprising the following specific steps:

[0061] S1. Acquire 2D images and 3D point cloud data of all workpieces to be grasped, and obtain the 6D pose of each workpiece to be grasped according to the 3D point cloud data;

[0062] Preferably, the generation of the workpiece classification model includes: acquiring a number of workpiece 2D images; selecting the workpiece contour of each workpiece in the workpiece 2D image, and comparing the selected area of ​​each workpiece with a preset workpiece cross-section, classifying the workpiece whose selected area is smaller than a preset proportion of the workpiece cross-section as an unclawed workpiece, and classifying the workpiece whose selected area is larger than a preset proportion of the workpiece cross-section as a graspable workpiece, and then labeling each workpiece according to the classification result; training a preset deep learning model according to the labeling information to generate the workpiece classification model.

[0063] Preferably, obtaining the 6D pose of each workpiece to be grasped based on the three-dimensional point cloud data includes: preprocessing the three-dimensional point cloud data; wherein the preprocessing includes: filtering, smoothing and reconstruction; extracting features of each workpiece to be grasped from the preprocessed three-dimensional point cloud data, and then matching the extracted features with a preset three-dimensional model of the bar material to obtain the 6D pose of the successfully matched workpiece to be grasped.

[0064] For details, please refer to Figure 2 , is the overall flow chart of the disordered grabbing of the rod-shaped workpiece of the present invention, and the specific implementation steps are as follows:

[0065] Step 1: In the preparation stage of disordered grasping, the preset deep learning model is first trained to obtain a workpiece classification model. The workpiece classification model can classify the workpieces into two categories: graspable workpieces and ungrappable workpieces according to their stacking conditions.

[0066] Please refer to Figure 3, for the workpiece contour frame selection of each workpiece in the workpiece 2D image, in the model training stage, the 3D camera is used to collect images of the workpieces in the material basket, and at least 500 images are collected; then Figure 3 The outline of each workpiece is selected and annotated, and the selected area of ​​each workpiece in the image is compared with the workpiece cross section. If the selected area is less than two-thirds of the workpiece cross section, it is classified as unclawed, otherwise it is classified as prehensile. Finally, these annotated data are used to train a deep learning model to obtain a workpiece classification model for subsequent classification tasks in the actual grasping process.

[0067] Step 2: In the process of random grasping, first use the 3D camera to obtain the 2D image and 3D point cloud data of the workpiece to be grasped, and accurately identify its 6D posture. It is divided into the following three steps:

[0068] Step 1: Please refer to Figure 4 and Figure 5 , respectively, are schematic diagrams of the three-dimensional point cloud data and the 2D image data of the workpiece to be grasped, use a 3D camera to scan the workpiece to be grasped in the material basket, obtain the three-dimensional point cloud data and 2D image data of its surface, and transmit them to the computer system;

[0069] Step 2: Preprocess the 3D point cloud data, including filtering, smoothing, reconstruction and other operations, to improve data quality and accuracy, and extract the features of the workpiece to be grasped from the processed point cloud data for subsequent posture recognition;

[0070] Step 3: Use OpenCV’s template matching technology to match the features extracted from the workpiece point cloud with the pre-set 3D bar model to obtain the 6D pose of the successfully matched workpiece. i Base ). It should be noted that the present invention uses the robot base as the reference coordinate system (R Base ).

[0071] S2, extracting graspable workpieces from the 2D image and generating a graspable workpiece sequence;

[0072] Preferably, extracting graspable workpieces from the 2D image and generating a graspable workpiece sequence includes: inputting the 2D image into a preset workpiece classification model, so that the workpiece classification model classifies the workpieces to be grasped that are successfully matched in the 2D image into graspable workpieces and unclawable workpieces, and outputting a graspable workpiece sequence.

[0073] Step 3: In the process of disordered grasping, the workpiece classification model trained in Step 1 and the 2D image in Step 2 are used to classify the successfully matched workpieces to be grasped into graspable workpieces and unclawed workpieces, and the graspable workpieces are extracted from them to form a sequence of successfully matched graspable workpieces.

[0074] S3, traversing the graspable workpieces in the graspable workpiece sequence, and generating a grasping point sequence of the current graspable workpiece according to the 6D pose of the current graspable workpiece;

[0075] Preferably, generating a grasping point sequence of the currently graspable workpiece according to the 6D posture of the currently graspable workpiece includes: converting the 6D posture of the currently graspable workpiece into a posture matrix; establishing a three-dimensional coordinate system with the center posture of the currently graspable workpiece as the origin, and then rotating a preset grasping tool along the Y axis with a preset angle step to obtain a rotation matrix; offsetting the rotated grasping tool along the Z axis by a preset distance to obtain a translation matrix; and obtaining a grasping point sequence of the currently graspable workpiece according to the posture matrix, the rotation matrix and the translation matrix.

[0076] Step 4: In the process of unordered grasping, first determine whether the graspable workpiece sequence is empty. If it is empty, it means that there is no graspable workpiece or no successfully matched workpiece, and the grasping task is terminated at this time. If the graspable workpiece sequence is not empty, traverse the graspable workpiece sequence and enter Step 5. If all workpieces are traversed and still not grasped, it means that the path planning of all graspable workpieces has failed, and the grasping task is also terminated.

[0077] Step 5: In the process of random grabbing, generate the grab point sequence of the current grabbable workpiece. Figure 6 , is a schematic diagram of generating a grasping point sequence of a currently graspable workpiece, and the steps of generating the grasping point sequence are as follows:

[0078] The gripping point of the rod-shaped workpiece is based on the center position of the workpiece. Based on the reference, the preset gripping tool is rotated along the workpiece y-axis by a fixed angle, and then offset along the negative z-axis by the distance of the workpiece radius. The calculation is as follows:

[0079] First, the 6D pose of the currently graspable workpiece Converted to pose matrix, as shown in formula 1:

[0080]

[0081] Then, rotate along the y-axis from 0 with a fixed angle step θ, with a rotation range of 0 to 2π. The rotation matrix is ​​shown in Equation 2:

[0082]

[0083] Finally, the fixed distance is offset along the z-axis, which is the radius of the cylinder. The translation matrix is ​​shown in formula 3:

[0084]

[0085] In summary, all grasping points are calculated based on the 6D pose of the current graspable workpiece as shown in Formula 4:

[0086] T pickingpoints =T O ·T R ·T G ,θ∈(0,2π) (4)

[0087] S4, traversing the grasping points in the grasping point sequence, and selecting a grasping point as a target grasping point;

[0088] Preferably, traversing the grasping points in the grasping point sequence and selecting a grasping point as a target grasping point includes: traversing the grasping points in the grasping point sequence, performing collision detection on the current grasping point, and determining whether the robot arm will collide with the surrounding environment when moving to the current grasping point. If no collision occurs, the current grasping point is used as the target grasping point.

[0089] Step 6: During the unordered grasping process, the grasping point sequence of the current graspable workpiece is traversed one by one, and then enter Step 7. If the grasping is still not performed after traversing all grasping points, it means that the path planning of all grasping points has failed. At this time, return to Step 4 to analyze the next workpiece to be grasped.

[0090] Step 7: In the process of disordered grasping, in order to avoid the path planning at the end point, that is, the target grasping point, to find that the tool or robot arm collides with the surrounding environment, thereby wasting resources, increasing planning time and affecting grasping efficiency, this step first performs a collision check on the end point. If a collision occurs, it directly returns to Step 6 to analyze the next grasping point. If no collision occurs, the current grasping point is used as the target grasping point.

[0091] S5. Generate a grasping path of the robot arm from the current posture to the target grasping point according to the current posture of the robot arm and the target grasping point, and then control the robot arm to move to the target grasping point according to the grasping path to grasp the currently graspable workpiece.

[0092] Preferably, the method of generating a grasping path of the robotic arm from the current posture of the robotic arm to the target grasping point according to the current posture of the robotic arm and the target grasping point includes: obtaining preset path planning parameters and starting path planning timing; wherein the path planning parameters include: sampling range, exploration step, maximum sampling number and maximum planning time; performing random sampling according to the sampling range and the preset path planning method to generate the first path point, and calculating the next path point according to the first path point and the exploration step; judging whether the current sampling number exceeds the maximum sampling number, if not, judging whether the current path planning time exceeds the maximum planning time, if not, calculating each part of the robotic arm posture in Cartesian space; perform collision detection on the robotic arm according to the preset three-dimensional model of each part of the robotic arm and the posture, and judge whether each part of the robotic arm will collide with the surrounding environment when the robotic arm moves to the current path point; if no collision occurs, judge whether the distance between the current path point and the target grasping point is less than the exploration step size, if so, take the current path point as the last path point of the grasping path, and generate the grasping path of the robotic arm from the current posture to the target grasping point; if not, generate the next path point along the direction from the current path point to the target grasping point according to the preset greedy search strategy, and then generate the grasping path of the robotic arm from the current posture to the target grasping point.

[0093] Step 8: In the process of random grasping, after confirming the collision-free target grasping point, it is necessary to further plan the grasping path of the robot arm from the current posture to the target grasping point. This can ensure that during the entire grasping process, each step of the action will not collide with the surrounding environment, thereby ensuring the safety of grasping and improving the stability of automatic feeding. If the path planning is successful, go to Step 9. If the planning fails, return to Step 6. Please refer to Figure 7 , is a flowchart of crawling path planning. The specific implementation steps are as follows:

[0094] Step 8-1: Configure the optimal path planning parameters according to the beat and grasping complexity: sampling range, exploration step, maximum sampling times, maximum planning time, and then start the path planning timing;

[0095] Step 8-2: According to the set sampling range, use the RRT-Connect path planning method to generate a random sampling point as the first path point, increase the sampling times by 1, and calculate the next path point according to the set step size. In order to avoid blind exploration, the sampling can be set to have a higher probability near the grasping point;

[0096] Step8-3: Determine whether the current sampling times have reached the set maximum sampling times. If so, return to Step6. Then determine whether the current path planning time has exceeded the preset maximum planning time. If so, also return to Step6. This can avoid continuous exploration at the current grasping point and transfer to the next grasping point in time, ensuring that the system efficiently explores the grasping points and improving the efficiency and stability of automated feeding;

[0097] Step 8-4: The current path point is the position of the tool end in Cartesian space. According to the kinematic model of the robot arm, the inverse solution is performed to obtain the six angle values ​​of the robot arm in the joint space. Then, the position of each part of the robot arm in Cartesian space is calculated to facilitate subsequent collision detection.

[0098] Step 8-5: Using the 3D model of each part of the robot and the corresponding position obtained in the previous step, first perform self-collision detection of the robot. Then check whether each part of the robot collides with the surrounding environment. If any collision is found, it will return to Step 8-2 to replan the path. This process ensures that the robot will not collide with itself or the surrounding environment when performing tasks, ensuring work safety and reliability;

[0099] Step 8-6: Check whether the distance between the current path point and the grasping point is less than the step length. If so, it is considered that the grasping point has been reached. At this time, the grasping point is inversely solved and used as the last path point. Then the inverse solutions of the previous path points are sorted out, and a collision-free joint angle path point sequence from the current position of the robot to the grasping point is output, and the path planning is completed;

[0100] Step8-7: If the grasping point is not reached, a greedy search strategy is used, that is, the next path point is generated along the direction from the current path point to the target point, so that a feasible path can be found quickly in simple scenarios without complex tree structure search, which improves computational efficiency. Then return to step 4 for collision detection. If a collision is found, return to Step8-2. In this way, when the greedy search cannot find a feasible path, RRT-Connect is used to supplement it, which improves the robustness and globality of the path planning, so as to ensure that the robot arm can perform grasping tasks safely and efficiently in various complex situations.

[0101] Step 9: Send the generated joint angle sequence to the robot controller, and then control the robot to move to the target grasping point according to the grasping path to grasp the currently graspable workpiece, and then return to Step 2 to start the next grasping task.

[0102] It can be seen that the present invention provides a method for randomly grasping a rod-shaped workpiece, and the present invention can achieve the following beneficial effects:

[0103] 1. Continuous grasping capability: Traditional automatic loaders often have difficulty in handling stacked and disorderly placed workpieces, while the disorderly grasping method of the robotic arm proposed in the present invention can achieve continuous grasping of such workpieces;

[0104] 2. Adaptability and flexibility: The present invention can adapt to the sizes of rod-shaped workpieces of different sizes, perform collision detection in real time, and plan the grasping path of the robot arm;

[0105] 3. Safety and stability: The present invention can effectively solve the problems of material jamming and machine collision that are prone to occur in the prior art, thereby ensuring the safety and stability of the production line. Compared with traditional methods, the improvement of safety and stability can reduce the possibility of production line accidents and ensure the safety of equipment and personnel;

[0106] 4. Cost saving: Compared with traditional methods, the present invention can accurately identify the workpiece posture, realize disordered grasping of rod-shaped workpieces, completely replace manual labor, greatly improve production efficiency, and ensure safe and stable grasping operations.

[0107] Embodiment 2

[0108] Please refer to Figure 7 , is a schematic structural diagram of a disordered grasping device for a rod-shaped workpiece provided by an embodiment of the present invention, the device comprising: a data acquisition module, a graspable workpiece sequence generation module, a grasping point sequence generation module, a target grasping point selection module and a grasping path generation module;

[0109] The data acquisition module is used to acquire 2D images and 3D point cloud data of all workpieces to be grasped, and obtain the 6D pose of each workpiece to be grasped according to the 3D point cloud data;

[0110] The graspable workpiece sequence generation module is used to extract graspable workpieces from the 2D image and generate a graspable workpiece sequence;

[0111] The grasping point sequence generation module is used to traverse the graspable workpieces in the graspable workpiece sequence and generate a grasping point sequence of the current graspable workpiece according to the 6D posture of the current graspable workpiece;

[0112] The target grasping point selection module is used to traverse the grasping points in the grasping point sequence and select a grasping point as the target grasping point;

[0113] The grasping path generation module is used to generate a grasping path from the current posture of the robotic arm to the target grasping point according to the current posture of the robotic arm and the target grasping point, and then control the robotic arm to move to the target grasping point according to the grasping path to grasp the currently graspable workpiece.

[0114] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0115] Those skilled in the art can clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0116] Embodiment 3

[0117] Accordingly, an embodiment of the present invention provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the method for unordered grasping of rod-shaped workpieces described in the above-mentioned embodiment of the invention when executing the computer program.

[0118] The electronic device may be a computing device such as a desktop computer, a notebook, a palm computer, a cloud server, etc. The device may include, but is not limited to, a processor and a memory.

[0119] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the device, and various interfaces and lines are used to connect various parts of the entire device.

[0120] Embodiment 4

[0121] Accordingly, an embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the disordered grasping method for rod-shaped workpieces described in the above-mentioned embodiment of the invention.

[0122] The memory can be used to store the computer program, and the processor realizes various functions of the device by running or executing the computer program stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0123] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the steps of each method embodiment described above can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0124] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for grabbing a rod-shaped workpiece in an unordered manner, characterized in that: include: Acquire 2D images and 3D point cloud data of all workpieces to be grasped, and obtain the 6D pose of each workpiece to be grasped according to the 3D point cloud data; Extracting graspable workpieces from the 2D image to generate a graspable workpiece sequence; Traversing the graspable workpieces in the graspable workpiece sequence, and generating a grasping point sequence of the current graspable workpiece according to the 6D pose of the current graspable workpiece; Traversing the grasping points in the grasping point sequence, and selecting a grasping point as a target grasping point; Obtain preset path planning parameters and start path planning timing; wherein the path planning parameters include: sampling range, exploration step length, maximum sampling times and maximum planning time; Performing random sampling according to the sampling range and a preset path planning method to generate a first path point, and calculating a next path point according to the first path point and the exploration step size; Determine whether the current sampling number exceeds the maximum sampling number. If not, determine whether the current path planning time exceeds the maximum planning time. If not, calculate the position and posture of each part of the robot arm in Cartesian space. According to the preset three-dimensional model of each part of the robotic arm and the posture, the collision detection of the robotic arm is performed to determine whether each part of the robotic arm will collide with the surrounding environment during the process of the robotic arm moving to the current path point; If no collision occurs, determine whether the distance between the current path point and the target grasping point is less than the exploration step size. If so, use the current path point as the last path point of the grasping path to generate a grasping path from the current position of the robot arm to the target grasping point. If it is not less than, then according to the preset greedy search strategy, the next path point is generated along the direction from the current path point to the target grasping point, and then the grasping path of the robot arm from the current posture to the target grasping point is generated, and then the robot arm is controlled to move to the target grasping point according to the grasping path to grasp the currently graspable workpiece.

2. The method for grabbing a rod-shaped workpiece in a disordered manner according to claim 1, characterized in that: The step of obtaining the 6D pose of each workpiece to be grasped according to the three-dimensional point cloud data includes: Preprocessing the three-dimensional point cloud data; wherein the preprocessing includes: filtering, smoothing and reconstruction; The features of each workpiece to be grasped are extracted from the preprocessed 3D point cloud data, and then the extracted features are matched with the preset 3D model of the bar material to obtain the 6D pose of the successfully matched workpiece to be grasped.

3. The method for grabbing a rod-shaped workpiece in a disordered manner as claimed in claim 2, characterized in that: The step of extracting graspable workpieces from the 2D image and generating a graspable workpiece sequence comprises: The 2D image is input into a preset workpiece classification model, so that the workpiece classification model classifies the workpieces to be grasped that are successfully matched in the 2D image into graspable workpieces and unclawable workpieces, and outputs a graspable workpiece sequence.

4. The method for grabbing a rod-shaped workpiece in a disordered manner as claimed in claim 3, characterized in that: The generation of the artifact classification model includes: Acquire several 2D images of workpieces; Selecting the workpiece contour of each workpiece in the 2D image of the workpiece, and comparing the selected area of ​​each workpiece with a preset workpiece cross section, classifying the workpiece whose selected area is smaller than a preset proportion of the workpiece cross section as an ungraspable workpiece, and classifying the workpiece whose selected area is larger than a preset proportion of the workpiece cross section as a graspable workpiece, and then marking each workpiece according to the classification result; A preset deep learning model is trained according to the annotation information to generate the workpiece classification model.

5. The method for grabbing a rod-shaped workpiece in an unordered manner according to claim 1, characterized in that: The step of generating a grasping point sequence of the currently graspable workpiece according to the 6D posture of the currently graspable workpiece comprises: Convert the 6D pose of the current graspable workpiece into a pose matrix; A three-dimensional coordinate system is established with the center position of the currently graspable workpiece as the origin, and then the preset grasping tool is rotated along the Y axis at a preset angle step to obtain a rotation matrix; The rotated grabbing tool is offset by a preset distance along the Z axis to obtain a translation matrix; According to the pose matrix, the rotation matrix and the translation matrix, a grasping point sequence of the currently graspable workpiece is obtained.

6. The method for grabbing a rod-shaped workpiece in a disordered manner according to claim 1, characterized in that: The traversing the grasping points in the grasping point sequence and selecting a grasping point as a target grasping point includes: Traverse the grasping points in the grasping point sequence, perform collision detection on the current grasping point, and determine whether the robot arm will collide with the surrounding environment when moving to the current grasping point. If no collision occurs, use the current grasping point as the target grasping point.

7. A disorderly grasping device for rod-shaped workpieces, characterized in that: include: Data acquisition module, graspable workpiece sequence generation module, grasping point sequence generation module, target grasping point selection module and grasping path generation module; The data acquisition module is used to acquire 2D images and 3D point cloud data of all workpieces to be grasped, and obtain the 6D pose of each workpiece to be grasped according to the 3D point cloud data; The graspable workpiece sequence generation module is used to extract graspable workpieces from the 2D image and generate a graspable workpiece sequence; The grasping point sequence generation module is used to traverse the graspable workpieces in the graspable workpiece sequence and generate a grasping point sequence of the current graspable workpiece according to the 6D posture of the current graspable workpiece; The target grasping point selection module is used to traverse the grasping points in the grasping point sequence and select a grasping point as the target grasping point; The grasping path generation module is used to obtain preset path planning parameters and start path planning timing; wherein the path planning parameters include: sampling range, exploration step, maximum sampling times and maximum planning time; random sampling is performed according to the sampling range and the preset path planning method to generate the first path point, and the next path point is calculated according to the first path point and the exploration step; it is determined whether the current sampling times exceeds the maximum sampling times, if not, it is determined whether the current path planning time exceeds the maximum planning time, if not, the posture of each part of the robot arm in Cartesian space is calculated; according to the preset three-dimensional model of each part of the robot arm and the posture, the robot arm is Collision detection is performed to determine whether each part of the robotic arm will collide with the surrounding environment when the robotic arm moves to the current path point; if no collision occurs, determine whether the distance between the current path point and the target grasping point is less than the exploration step size. If it is less than, the current path point is used as the last path point of the grasping path, and a grasping path for the robotic arm from the current posture to the target grasping point is generated; if it is not less than, the next path point is generated along the direction from the current path point to the target grasping point according to a preset greedy search strategy, and then a grasping path for the robotic arm from the current posture to the target grasping point is generated, and then the robotic arm is controlled to move to the target grasping point according to the grasping path to grasp the currently graspable workpiece.

8. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the disorderly grasping method for rod-shaped workpieces according to any one of claims 1 to 6 when executing the computer program.

9. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is executed, the device where the storage medium is located is controlled to execute the disordered grasping method for rod-shaped workpieces according to any one of claims 1 to 6.

Images