Automatic grabbing and sorting system based on 3D vision

Abstract

The invention discloses an automatic grabbing and sorting system based on 3D vision, which belongs to the production line technology and comprises a 3D vision sensor part, an image processing and identifying part, a robot control system part, a mechanical arm and grabbing device part, a sorting and classified placement part, a human-computer interaction interface and monitoring system part and a software algorithm and integration part. According to the system, three-dimensional image information of an object is obtained through a 3D vision sensor, then an image is processed and analyzed through an image processing algorithm, and therefore information such as the type, the position and the posture of the object is recognized. And then the control system plans the motion trail of the mechanical arm according to the information and controls the mechanical arm to grab and sort the objects, so that accurate recognition, positioning and grabbing of the objects and subsequent sorting and classification are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of production lines, and in particular to an automatic grasping and sorting system based on 3D vision. Background Art

[0002] Production lines are common in the automotive manufacturing industry, logistics industry, food processing industry, etc. Currently, most production lines require a lot of manual assistance during use, such as placing objects on the production line and taking objects off the production line. In view of this, in order to reduce the manual participation of the production line during use, save labor costs, and increase the degree of automation of the production line, an automatic grasping and sorting system based on 3D vision is proposed. Summary of the invention

[0003] The purpose of the present invention is to solve the shortcomings of the prior art mentioned in the background technology and to propose an automatic grasping and sorting system based on 3D vision.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions: The automatic grasping and sorting system based on 3D vision is characterized by comprising the following parts: A 3D vision sensor part for capturing three-dimensional image information of objects in the working area; Image processing and recognition part for preprocessing point cloud data captured by 3D vision sensors; The part of the robot control system used to plan and control the robot's motion based on the object information provided by the image processing and recognition module; The robot arm and the grasping device part are used to move according to the instructions of the control system and realize the grasping and carrying of objects; A sorting and classification placement part for sorting and placing objects according to preset rules or algorithms; The human-computer interaction interface and monitoring system part used to configure, monitor and troubleshoot the system; Software algorithms and integration components responsible for processing and analyzing data, planning and controlling motion, and realizing the overall functions of the system.

[0005] As a further feature of the system of the present invention, the 3D vision sensor portion includes a structured light sensor, a time-of-flight sensor, a laser radar, and a stereo vision camera; These sensors can generate point cloud data, which is a series of three-dimensional coordinate points on the surface of an object, providing the basis for subsequent processing and analysis; The 3D vision sensor part is the "eyes" of the system, which can obtain three-dimensional point cloud data of objects, including shape, size, and position.

[0006] As a further feature of the system of the present invention, the image processing and recognition part includes an image processing module and a recognition module; The image processing module includes image processing algorithms for processing and analyzing point cloud data acquired by 3D vision sensors and extracting useful object information; The image processing algorithm processes and analyzes the images acquired by the 3D vision sensor to extract the feature information of the object, such as edges, corners, surfaces, etc. At the same time, the module can also perform pre-processing operations such as filtering, denoising, and segmentation on the image to improve the accuracy of subsequent recognition; The recognition module includes object recognition and positioning. It uses machine learning or deep learning algorithms to perform object recognition, classification, positioning and posture estimation on the processed images, determine the type of object, and calculate the position and posture of the object in three-dimensional space, providing accurate guidance for the robotic arm's grasping. This step is the key to the system's ability to accurately identify and grasp the target object. Machine learning and deep learning algorithms are used to achieve accurate recognition and positioning of objects.

[0007] As a further feature of the system of the present invention, the robot control system part is the "brain" of the system, including the motion planning of the robot arm, the calculation of the joint angles, and the control of the grasping force.

[0008] As a further feature of the system of the present invention, the robot arm and the grasping portion include a robot arm module and a grasping device module; The robot module is the actuator of the system. The robot uses robot control technology to plan the motion trajectory, joint angle, grasping force control, etc. of the robot according to the position and posture information of the object, which is used to grasp and place the object, ensuring the accuracy and efficiency of the grasping process. The gripping device module is customized according to the shape, size and material of the object to be gripped to ensure stable and reliable gripping; As a further feature of the system of the present invention, in specific operations, a grasping control module is used to control a robotic arm or grasping device to perform precise grasping operations on objects based on the results of identification and positioning. The module can calculate the coordinates of the grasping point and the grasping angle to ensure the stability and accuracy of the grasping process.

[0009] As a further feature of the system of the present invention, the sorting and classification placement part includes object classification, sorting algorithm, path planning and selection of placement location. The sorting and classification placement part grabs the identified objects and performs sorting and classification operations according to preset sorting rules or algorithms.

[0010] As a further human-computer interaction interface and monitoring system part of the system of the present invention, the human-computer interaction interface allows the operator to configure, monitor and troubleshoot the system, and the monitoring system displays the system's working status, capture results and any possible abnormal conditions in real time. These interfaces and systems provide the necessary feedback and control means to ensure the stable operation and efficient performance of the system.

[0011] As a further step in the system of the present invention, the software algorithm and the integrated part are the core of the system, and the integration work of the integrated part ensures seamless collaboration and communication between various modules to achieve the overall performance and function of the system.

[0012] Compared with the prior art, the present invention has the following beneficial effects: The 3D vision automated grasping and sorting system combines knowledge from multiple fields such as advanced 3D vision technology, automated grasping technology, robotics technology, control technology, and software algorithms. Specifically, it obtains the three-dimensional image information of the object through the 3D vision sensor, and then uses the image processing algorithm to process and analyze the image to identify the type, position, posture and other information of the object. Then the control system plans the motion trajectory of the robot arm based on this information, and controls the robot arm to grasp and sort the object, realizing the accurate identification, positioning and grasping of the object, as well as the subsequent sorting and classification.

[0013] The 3D vision automated grasping and sorting system has the following effects and characteristics: High precision and high reliability: 3D vision technology can provide three-dimensional information of objects, thus achieving more accurate grasping and sorting operations. At the same time, advanced image processing algorithms and robotic arm control technology also ensure the high reliability of the system; Flexibility and scalability: The system can adapt to the grasping needs of objects of different shapes, sizes and materials, and can be flexibly adjusted according to actual application scenarios. In addition, by adding extended functions such as sensors and algorithm modules, the performance and application scope of the system can be further improved; Intelligence and automation: The system can realize automatic identification, positioning, grasping and sorting of objects, and can complete complex production tasks without or with little human intervention, which not only improves production efficiency and quality stability, but also reduces labor costs and labor intensity. DETAILED DESCRIPTION

[0014] The content of the present invention can be more easily understood by selecting the following detailed description of the preferred implementation method of the present invention and the embodiments included. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those of ordinary skill in the art to which the present invention belongs. When there is a conflict, the definition in this specification shall prevail.

[0015] As used herein, the term "prepared from" is synonymous with "comprising." As used herein, the terms "comprising," "including," "having," "containing," or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a composition, process, method, article, or apparatus that comprises the listed elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such composition, process, method, article, or apparatus.

[0016] The conjunction "consisting of excludes any unspecified element, step, or component. If used in a claim, this phrase renders the claim closed-ended so that it does not include materials other than those described, except for conventional impurities associated therewith. When the phrase "consisting of" appears in a clause of the body of a claim rather than immediately following the subject matter, it limits only the elements described in that clause; other elements are not excluded from the claim as a whole.

[0017] When amount, concentration or other value or parameter is expressed as range, preferred range or a series of upper preferred value and lower preferred value limit range, this should be understood as specifically disclosing all ranges formed by any pairing of any range upper limit or preferred value and any range lower limit or preferred value, regardless of whether the range is disclosed separately. For example, when disclosing range "1 to 5", described range should be interpreted as including range "1 to 4", "1 to 3", "1 to 2", "1 to 2 and 4 to 5", "1 to 3 and 5" etc. When numerical range is described in this article, unless otherwise stated, the range is intended to include its end value and all integers and fractions within the range.

[0018] Singular forms include plural references unless the context clearly indicates otherwise. "Optional" or "either" means that the subsequently described event or incident may or may not occur, and that the description includes instances where the event occurs and instances where it does not.

[0019] Approximate terms in the specification and claims are used to modify quantities, indicating that the present invention is not limited to the specific quantity, but also includes acceptable and modified parts close to the quantity without causing changes in the relevant basic functions. Accordingly, the use of "about", "approximately", etc. to modify a numerical value means that the present invention is not limited to the exact numerical value. In some examples, the approximate terms may correspond to the accuracy of the instrument for measuring the numerical value. In the specification and claims of this application, range limitations can be combined and / or interchanged, and if not otherwise stated, these ranges include all subranges contained therein.

[0020] In addition, the indefinite articles "a" and "an" before the elements or components of the present invention have no restrictions on the quantity requirements (i.e. the number of occurrences) of the elements or components. Therefore, "a" or "an" should be interpreted as including one or at least one, and the elements or components in the singular form also include the plural form, unless the number is obviously intended to be in the singular form.

[0021] The technical solutions in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0022] Embodiment 1 The automatic grasping and sorting system based on 3D vision includes the following parts: The 3D vision sensor is the "eye" of the system, which is used to capture the three-dimensional image information of objects in the working area, that is, it can obtain the three-dimensional point cloud data of the object, including the shape, size, and position. Common 3D vision sensors include structured light sensors, time-of-flight sensors, lidar, and stereo vision cameras. These sensors can generate point cloud data, that is, a series of three-dimensional coordinate points on the surface of the object, providing a basis for subsequent processing and analysis. The image processing and recognition part is used to pre-process the point cloud data captured by the 3D vision sensor. The image processing and recognition part includes an image processing module and a recognition module. The image processing module includes an image processing algorithm for processing and analyzing the point cloud data obtained by the 3D vision sensor and extracting useful object information. The image processing algorithm processes and analyzes the image obtained by the 3D vision sensor and extracts the feature information of the object, such as edges, corners, surfaces, etc. At the same time, the module can also perform pre-processing operations such as filtering, denoising, and segmentation on the image to improve the accuracy of subsequent recognition. The recognition module includes object recognition and positioning, and uses machine learning or deep learning algorithms to perform object recognition, classification, positioning, and posture estimation on the processed image to determine the type of the object, thereby calculating the position and posture of the object in three-dimensional space, and providing accurate guidance for the grasping of the robotic arm. This step is the key to the system's ability to accurately recognize and grasp the target object. The robot control system is used to plan and control the robot's movement according to the object information provided by the image processing and recognition module. The robot control system is the "brain" of the system, including the movement planning of the robot arm, the calculation of the joint angle, and the control of the grasping force. The control system also needs to be real-time to ensure that the robot can respond quickly and accurately perform the grasping task; The robot arm and the grasping device part are used to move according to the instructions of the control system and realize the grasping and transportation of objects. The robot arm and the grasping part include a robot arm module and a grasping device module; the robot arm module is the actuator of the system. The robot arm plans the movement trajectory, joint angle, grasping force control, etc. of the robot arm according to the position and posture information of the object through the robot arm control technology, which is used to realize the grasping and placement of the object and ensure the accuracy and efficiency of the grasping process; the grasping device module is customized according to factors such as the shape, size and material of the grasped object to ensure stable and reliable grasping; in specific operations, the grasping control module is used to control the robot arm or grasping device to perform precise grasping operations on the object according to the results of identification and positioning. The module can calculate the coordinates and grasping angle of the grasping point to ensure the stability and accuracy of the grasping process. The design of the robot arm and the grasping device needs to take into account multiple factors such as accuracy, speed, and load capacity; The sorting and classification placement part is used to sort and place objects according to preset rules or algorithms. The sorting and classification placement part includes object classification, sorting algorithm, path planning and placement location selection. The sorting and classification placement part grabs the identified objects and performs sorting and classification operations according to the preset sorting rules or algorithms. The sorting and placement module needs to ensure the stability and accuracy of the objects at the target location; The human-machine interface and monitoring system are used to configure, monitor and troubleshoot the system. The human-machine interface allows operators to configure, monitor and troubleshoot the system, and the monitoring system displays the system's working status, capture results and any possible abnormal conditions in real time. These interfaces and systems provide the necessary feedback and control means to ensure the stable operation and efficient performance of the system. The software algorithm and integration part is responsible for processing and analyzing data, planning and controlling movement, and realizing the overall function of the system. The software algorithm in the software algorithm and integration part is the core of the system, and the integration work of the integration part ensures seamless collaboration and communication between various modules to realize the overall performance and function of the system.

[0023] Embodiment 2 The 3D vision automated grasping and sorting method comprises the following steps: S1. Object recognition and positioning 1) Use 3D vision sensors to obtain 3D point cloud data of objects in the working area; 2) Preprocess the point cloud data, including filtering, noise reduction, segmentation, etc., to extract useful object information; 3) Use machine learning or deep learning algorithms to identify objects in the processed images and determine the type of objects; 4) Based on the three-dimensional information of the object, calculate the precise position of the object in the working area.

[0024] S2. Grasping strategy planning 1) Select appropriate grasping points and grasping methods based on the shape, size, material and other factors of the object; 2) Plan the motion trajectory of the robot arm to ensure that the robot arm can accurately reach the grasping point and grasp the object with the appropriate posture.

[0025] 3) Consider factors such as the weight and friction of the object and adjust the grasping force to ensure stable grasping.

[0026] S3. System optimization and adjustment 1) Optimize and adjust system parameters according to actual application scenarios and requirements; 2) Monitor the system's operating status and crawling results, and promptly detect and handle possible abnormal situations; 3) Evaluate and improve the system's performance to improve crawling efficiency and accuracy.

[0027] The 3D vision automated grasping and sorting method uses 3D vision sensors to obtain three-dimensional image information of objects, processes and analyzes the images through image processing algorithms, identifies key information such as the type, position, and posture of the objects, and then plans the motion trajectory of the robotic arm based on this information, and controls the robotic arm to grasp and sort the objects.

[0028] The examples referred to herein are merely illustrative and are used to explain some features of the method of the present invention. The appended claims are intended to claim the widest possible range that can be imagined, and the embodiments presented herein are only illustrations of selected implementations according to the combination of all possible embodiments. Therefore, it is the applicant's intention that the appended claims are not limited by the selection of examples that illustrate the features of the present invention. Some numerical ranges used in the claims also include sub-ranges within them, and changes in these ranges should also be interpreted as being covered by the appended claims where possible.

Claims

1. The automatic grasping and sorting system based on 3D vision is characterized by: Includes the following parts: A 3D vision sensor part for capturing three-dimensional image information of objects in the working area; Image processing and recognition part for preprocessing point cloud data captured by 3D vision sensors; The part of the robot control system used to plan and control the robot's motion based on the object information provided by the image processing and recognition module; The robot arm and the grasping device part are used to move according to the instructions of the control system and realize the grasping and carrying of objects; A sorting and classification placement part for sorting and placing objects according to preset rules or algorithms; The human-computer interaction interface and monitoring system part used to configure, monitor and troubleshoot the system; Software algorithms and integration components responsible for processing and analyzing data, planning and controlling motion, and realizing the overall functions of the system.

2. The 3D vision-based automated grasping and sorting system according to claim 1, characterized in that: The 3D vision sensor part includes a structured light sensor, a time-of-flight sensor, a lidar, and a stereo vision camera. The 3D vision sensor part can obtain three-dimensional point cloud data of objects.

3. The 3D vision-based automated grasping and sorting system according to claim 1, characterized in that: The image processing and recognition part includes an image processing module and a recognition module; The image processing module includes image processing algorithms for processing and analyzing point cloud data acquired by 3D vision sensors and extracting useful object information; The recognition module includes object recognition and positioning. It uses machine learning or deep learning algorithms to perform object recognition, classification, positioning, and posture estimation on the processed images to determine the type of object, thereby calculating the position and posture of the object in three-dimensional space, providing accurate guidance for the robotic arm's grasping.

4. The 3D vision-based automated grasping and sorting system according to claim 1, characterized in that: The robot control system is the "brain" of the system. It includes the motion planning of the robotic arm, calculation of joint angles, and control of grasping force.

5. The 3D vision-based automated grasping and sorting system according to claim 1, characterized in that: The robot arm and the grasping part include a robot arm module and a grasping device module; The robot module is the actuator of the system. The robot uses robot control technology to plan the robot's motion trajectory, joint angle, grasping force control, etc. according to the position and posture information of the object, which is used to grasp and place the object, ensuring the accuracy and efficiency of the grasping process; The gripper modules are customized according to factors such as the shape, size and material of the object to be gripped to ensure stable and reliable gripping.

6. The 3D vision-based automated grasping and sorting system according to claim 5, characterized in that: The robotic arm module and the grasping device module use the grasping control module to control the robotic arm or grasping device to perform precise grasping operations on objects according to the results of recognition and positioning.

7. The 3D vision-based automated grasping and sorting system according to claim 1, characterized in that: The sorting and classification placement part includes object classification, sorting algorithm, path planning and placement location selection. The sorting and classification placement part grabs the identified objects and performs sorting and classification operations according to preset sorting rules or algorithms.

8. The 3D vision-based automated grasping and sorting system according to claim 1, characterized in that: In the human-computer interaction interface and monitoring system part, the human-computer interaction interface allows operators to configure, monitor and troubleshoot the system, while the monitoring system displays the system's working status, capture results and any possible abnormal situations in real time.

9. The 3D vision-based automated grasping and sorting system according to claim 1, characterized in that: The software algorithm is the core of the system, and the integration work of the integration part ensures seamless collaboration and communication between various modules to achieve the overall performance and functionality of the system.