A robot vision automated grasping system

Through calibration, data storage, image preprocessing and grasping force adjustment of robot vision automation grab system, the problems of low accuracy of two-dimensional image recognition and complex grasping route planning in the existing technology are solved, and more efficient grasping is achieved Get the smoothness of the target recognition and grabbing process.

CN118322215BActive Publication Date: 2024-12-06EXCEPT GUARDIAN ENVIRONMENTAL TECH (BEIJING) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410626683.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-20
Publication Date
2024-12-06
Estimated Expiration
2044-05-20

AI Technical Summary

Technical Problem

When the existing robot visual grabbing system handles disorderly stacked items, the two-dimensional image features are not obvious, the recognition accuracy is low, and the grab position and force are not effectively adjusted, resulting in complex and unsmoothing crawling route planning.

Method used

Design a robot vision automation grabbing system, including calibration unit, data storage unit, three-dimensional vision processing unit, robot control unit and target grabbing unit, through camera calibration, hand-eye calibration, data storage, image preprocessing, position adjustment and grabbing Adjust the force to achieve accurate positioning and grasping route planning for the grasping targets.

Benefits of technology

Improve the identification accuracy and crawling success rate of the grab target, reduce the randomness during the push process, ensure the smoothness and adaptability of the robot robot arm during the grab process, and avoid the target object's damage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118322215B_ABST
    Figure CN118322215B_ABST
Patent Text Reader

Abstract

The present invention discloses a robot vision automated grasping system, which relates to the field of robot vision technology, and includes a calibration unit, wherein the calibration unit is used to respectively determine the relationship between the image coordinate system and the camera coordinate system and the lens distortion of the camera, determine the relationship between the camera coordinate system and the robot terminal coordinate system, and determine the relationship between the terminal grasping tool coordinate system and the robot terminal coordinate system. The present invention proposes a robot vision automated grasping system, which is used to determine the relationship between the image coordinate system and the camera coordinate system, the camera coordinate system and the robot terminal coordinate system, and the terminal grasping tool coordinate system and the robot terminal coordinate system through a set calibration unit, so as to prepare for grasping operations, and is used to store data information related to a target to be grasped through a set data storage unit, and is used to collect and pre-process the image of the target to be grasped before grasping operations through a set three-dimensional vision processing unit.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of robot vision, in particular to a robot vision automatic grasping system. Background Art

[0002] With the development of robot technology, robotic arms have been widely used in many occasions. Robots can replace humans to perform complex and repetitive work, and their efficiency and accuracy are much higher than humans. In the Chinese patent with application number 202111337906.9, "a grasping system and grasping method are disclosed, and the grasping method is used to grasp items placed in an item holding device, including the following steps: collecting visual information of the items to be grasped; processing the collected visual information; determining a grasping strategy based on the processed visual information; grasping the items based on the grasping strategy; processing the collected visual information includes: determining the graspable space, and identifying the contours of all items in the graspable space, segmenting different items from the contours, and performing three-dimensional modeling on the segmented items, and confirming the relative spatial relationship between different items. The present disclosure adopts three-dimensional machine vision to accurately identify the contours of items in a disordered stacking state and perform three-dimensional modeling, which solves the problem of unclear two-dimensional image features and inaccurate category recognition."

[0003] This comparative document only solves the problem that in the recognized image, there may be a complete plane connected with the grasped object, which makes it impossible to determine the graspable object, and the two-dimensional features of the object to be grasped are not obvious, but each product has different differences, and the recognition accuracy rate is relatively low relying solely on two-dimensional images. It does not take into account the adjustment of the position of the grasped target by pushing alone, which is too random, and only the type of the grasped target is identified. Other relevant information is not obtained based on the type identification result, and thus the grasping force of the robot arm during the grasping operation cannot be adjusted, and many factors should be considered when planning the grasping route. Summary of the invention

[0004] The purpose of the present invention is to provide a robot vision automated grasping system to solve the problems raised in the above background technology.

[0005] To achieve the above-mentioned object, the present invention provides the following technical solutions: a robot vision automated grasping system, comprising a calibration unit, the calibration unit being used to respectively determine the relationship between an image coordinate system and a camera coordinate system and the lens distortion of the camera, determine the relationship between the camera coordinate system and the robot end coordinate system, and determine the relationship between the end grasping tool coordinate system and the robot end coordinate system;

[0006] A data storage unit, the data storage unit is used to store data information related to the target to be grasped based on the established data storage library related to the target to be grasped, so as to compare and identify the collected image with the image stored in the data storage library during the grasping operation, thereby obtaining specific information of the target to be grasped;

[0007] A three-dimensional visual processing unit, which is used to collect images of the target to be grasped during the grasping operation, and compare and identify the collected images with the images stored in the data storage library after preprocessing, so as to obtain specific information of the target to be grasped, and locate the target to be grasped based on the preprocessed image information;

[0008] A robot control unit, the robot control unit is used to adjust the position of the target to be grasped and the grasping posture of the robot when it is detected that there is not enough space for grasping operation, and after the adjustment is completed, plan and adjust the grasping route of the target to be grasped;

[0009] The target grasping unit is used to select the viewing angle at which the robot starts grasping operations, and after the selection is completed, the grasping force of the grasping tool at the end of the robot is adjusted based on the specific information of the target to be grasped, and the grasping operation is performed after the adjustment is completed.

[0010] Preferably, the calibration unit includes a camera calibration module, which is used to obtain the correspondence between a point in the three-dimensional space and a point in the pixel coordinate system through a traditional camera calibration method based on a calibration plate, and simultaneously obtain the intrinsic parameters, extrinsic parameters and distortion parameter matrix of the binocular camera, thereby determining the relationship between the image coordinate system and the camera coordinate system to achieve calibration of the binocular camera.

[0011] Preferably, the calibration unit also includes a hand-eye calibration module and a tool calibration module. The hand-eye calibration module is used to determine the relationship between the camera coordinate system and the robot end coordinate system. Based on the positional relationship between the binocular camera and the robot manipulator, the hand-eye calibration can be divided into two types: eye on hand and eye outside hand. The specific operation steps are as follows: select the hand-eye calibration mode as eye on hand or eye outside hand, select the calibration plate type and set the calibration plate parameters, the hand-eye calibrator object creates a corresponding calibration plate object for identifying the calibration plate posture, the user controls the robot to move and sends a collection signal to the hand-eye calibrator, and the hand-eye calibrator obtains the target image through the visual sensor after receiving the signal, and then calls the calibration plate object for identification. The hand-eye calibrator then obtains the current coordinate information of the robot from the robot, and stores the current coordinate of the robot, the calibration plate posture and the corresponding target image as a set of data in the calibration data set, and performs hand-eye calibration through the hand-eye calibrator. The tool calibration module is used to determine the relationship between the end grasping tool coordinate system and the robot end coordinate system through the six-point method.

[0012] Preferably, the data storage unit includes a target data storage module and a target data update module, the target data storage module is used to upload data information related to the target to be captured to the data storage repository for storage, and the data information related to the target to be captured includes the size, weight and type of the target to be captured in the image, and the target data update module is used to update the data information in the data storage repository by uploading an image that is compared and identified with the image stored in the data storage repository and the comparison and identification result is the same target to the data storage repository.

[0013] Preferably, the three-dimensional vision processing unit includes a target image acquisition module and an image preprocessing module, wherein the target image acquisition module realizes multi-directional image acquisition of the target to be captured through a binocular camera, and the image preprocessing module is used to perform preprocessing operations on the acquired image to remove irrelevant information in the image and improve the visibility of the image, specifically, the image preprocessing is realized through image graying, image filtering, image binarization and image morphological processing, wherein the image graying operation is realized by a maximum value method, an average value method or a weighted average method, wherein the image filtering operation is realized by a mean filtering method, a median filtering method, a Gaussian filtering method or a bilateral filtering method, wherein the image binarization operation is realized by a maximum inter-class variance method, wherein the image morphological processing is realized by four basic operations of corrosion, expansion, opening operation and closing operation.

[0014] Preferably, the three-dimensional vision processing unit also includes a target recognition module and a target positioning module. The target recognition module is used to compare and identify the preprocessed image in the image preprocessing module with the image stored in the data storage repository, and obtain relevant information of the target to be grasped based on the comparison and recognition result, and upload the preprocessed image to the data storage repository for storage. The target positioning module is used to perform contour recognition on the target to be grasped from four aspects: circularity of the contour, ratio of contour area to circumscribed minimum rectangle area, contour hierarchy relationship and Hu moment based on the preprocessed image information, thereby segmenting different types of targets in the image, and performing three-dimensional modeling on the target to be grasped to locate the target to be grasped.

[0015] Preferably, the robot control unit includes a target position adjustment module, which is used to adjust the position of the target to be grasped by controlling the robot to perform grasping and pushing actions on the target to be grasped at the same time when there is not enough space for grasping operation, and to train the robot to perform actions through the designed grasping reward function and pushing reward function to reduce the influence of randomness on the grasping success rate during the pushing process, and the grasping reward function adopts a sparse reward function, and its specific calculation formula is as follows:

[0016]

[0017] In the formula, the reward R collsion ={0, -0.5}, and the specific calculation formula of the driving reward function is as follows:

[0018] R p =R collsion +w·D(s t ,s t+1 )+c'-c

[0019] In the formula, c'-c represents the change of the number of clusters c and c' before and after the promotion, D(s t ,s t+1 ) represents the point cloud s before pushing t With the pushed point cloud s t+1 The chamfer distance between them, w represents the weight coefficient of the state change before and after pushing.

[0020] Preferably, the robot control unit also includes a grasping posture adjustment module and a grasping route planning module, the grasping posture adjustment module is used to adjust the position of the robot based on the position information of the target to be grasped adjusted by the target position adjustment module, and adjust the lower claw position, grasping angle and grasping posture of the robot mechanical arm after the adjustment is completed, the grasping route planning module is used to derive the initial grasping route of the robot mechanical arm based on the current position of the robot mechanical arm and the position of the target to be grasped, and partially adjust the initial grasping route in consideration of route obstacles and robot mechanical arm freedom limitation, and on this basis, further adjust the adjusted route through the route interpolation algorithm, and its specific calculation formula is as follows:

[0021]

[0022] In the formula, f arm represents the operating frequency of the robot arm, (x p-1 ,y p-1 ,z p-1 ) represents the route node on the initial crawling route, (x' p ,y' p ,z' p ) represents the route node on the initial crawling route after partial adjustment, d c represents the interpolation distance of the initial grasping route after partial adjustment, N c Represents the number of steps of the initial grasping path after some adjustments to improve the smoothness of the robot arm during the grasping process.

[0023] Preferably, the target grasping unit includes a viewing angle selection module, and the viewing angle selection module is used to select a grasping viewing angle of the robot when performing a grasping operation through a viewing angle selector, so that the robot can further identify the position of the target to be grasped in the global image.

[0024] Preferably, the target grasping unit further includes a grasping force adjustment module and a grasping action execution module. The grasping force adjustment module is used to calculate the required grasping force through a grasping force prediction algorithm based on the acquired weight information of the target to be grasped. The specific calculation formula is as follows:

[0025] F=mg-k f mg

[0026] In the formula, F represents the grasping force required by the robot arm, m represents the weight of the target to be grasped, g represents the acceleration of gravity, and k f It represents the friction coefficient between the robot arm and the target to be grasped, and adjusts the grasping force of the robot arm based on the calculation result. The grasping action execution module is used to control the robot arm to perform the grasping operation.

[0027] Compared with the prior art, the beneficial effects of the present invention at least include: the present invention proposes a robot vision automated grasping system, wherein a calibration unit is provided to determine the relationship between an image coordinate system and a camera coordinate system, a camera coordinate system and a robot end coordinate system, and an end grasping tool coordinate system and a robot end coordinate system, so as to prepare for grasping operations; a data storage unit is provided to store data information related to a target to be grasped; a three-dimensional visual processing unit is provided to collect and pre-process an image of a target to be grasped before grasping operations, so as to compare and identify, obtain specific information of the target to be grasped, and locate the target to be grasped; a robot control unit is provided to adjust the position of the target to be grasped by combining a grasping action and a pushing action, so as to reduce the influence of randomness in the pushing process on the grasping success rate; and a grasping route of the target to be grasped is planned and adjusted, so as to improve the smoothness of the robot mechanical arm during the grasping process; and a target grasping unit is provided to adjust the grasping force of the robot mechanical arm based on the data information related to the target to be grasped, so as to avoid damage to the target to be grasped. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the overall process of an embodiment of the present invention;

[0029] Figure 2 This is a schematic diagram of the calibration unit flow in an embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram of a data storage unit process according to an embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of the flow of a three-dimensional visual processing unit according to an embodiment of the present invention;

[0032] Figure 5 This is a schematic diagram of the flow of a robot control unit according to an embodiment of the present invention;

[0033] Figure 6 The figure is a flow chart of a target grabbing unit according to an embodiment of the present invention.

[0034] In the figure: 100, calibration unit; 101, camera calibration module; 102, hand-eye calibration module; 103, tool calibration module; 200, data storage unit; 201, target data storage module; 202, target data update module; 300, three-dimensional vision processing unit; 301, target image acquisition module; 302, image preprocessing module; 303, target recognition module; 304, target positioning module; 400, robot control unit; 401, target position adjustment module; 402, grasping posture adjustment module; 403, grasping route planning module; 500, target grasping unit; 501, viewing angle selection module; 502, grasping force adjustment module; 503, grasping action execution module. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] See also Figure 1-6 , the present invention provides a technical solution: a robot vision automated grasping system, comprising a calibration unit 100, the calibration unit 100 is used to respectively determine the relationship between the image coordinate system and the camera coordinate system and the lens distortion of the camera, determine the relationship between the camera coordinate system and the robot end coordinate system, and determine the relationship between the end grasping tool coordinate system and the robot end coordinate system;

[0037] The data storage unit 200 is used to store data information related to the target to be grasped based on the established data storage library related to the target to be grasped, so as to compare and identify the collected image with the image stored in the data storage library during the grasping operation, thereby obtaining specific information of the target to be grasped;

[0038] The three-dimensional visual processing unit 300 is used to collect images of the target to be grasped during the grasping operation, and after preprocessing the collected images, compare and identify them with the images stored in the data storage library to obtain specific information of the target to be grasped, and locate the target to be grasped based on the preprocessed image information;

[0039] The robot control unit 400 is used to adjust the position of the target to be grasped and the grasping posture of the robot when it is detected that there is not enough space for grasping operation. After the adjustment is completed, the grasping route of the target to be grasped is planned and adjusted;

[0040] The target grasping unit 500 is used to select the viewing angle at which the robot starts grasping operations, and after the selection is completed, based on the specific information of the target to be grasped, adjust the grasping force of the grasping tool at the end of the robot, and perform the grasping operation after the adjustment is completed.

[0041] The calibration unit 100 includes a camera calibration module 101, which is used to obtain the correspondence between a point in the three-dimensional space and a point in the pixel coordinate system through a traditional camera calibration method based on a calibration plate, and simultaneously obtain the intrinsic parameters, extrinsic parameters and distortion parameter matrix of the binocular camera, thereby determining the relationship between the image coordinate system and the camera coordinate system, and realizing the calibration of the binocular camera;

[0042] The calibration unit 100 also includes a hand-eye calibration module 102 and a tool calibration module 103. The hand-eye calibration module 102 is used to determine the relationship between the camera coordinate system and the robot end coordinate system. Based on the positional relationship between the binocular camera and the robot manipulator, the hand-eye calibration can be divided into two types: eyes on the hand and eyes outside the hand. The specific operation steps are as follows: select the hand-eye calibration mode as eyes on the hand or eyes outside the hand, select the calibration plate type and set the calibration plate parameters, the hand-eye calibrator object creates a corresponding calibration plate object for identifying the calibration plate posture, the user controls the robot to move and sends a collection signal to the hand-eye calibrator, the hand-eye calibrator obtains the target image through the visual sensor after receiving the signal, and then calls the calibration plate object for identification, the hand-eye calibrator then obtains the current coordinate information of the robot from the robot, stores the current coordinate of the robot, the calibration plate posture and the corresponding target image as a set of data into the calibration data set, and performs hand-eye calibration through the hand-eye calibrator, the tool calibration module 103 is used to determine the relationship between the end grasping tool coordinate system and the robot end coordinate system through the six-point method;

[0043] The data storage unit 200 includes a target data storage module 201 and a target data update module 202. The target data storage module 201 is used to upload data information related to the target to be captured to the data storage library for storage, and the data information related to the target to be captured includes the size, weight and type of the target to be captured in the image. The target data update module 202 is used to update the data information in the data storage library by uploading an image that is compared and identified with an image stored in the data storage library and the comparison and identification result shows that the image is the same type of target to the data storage library;

[0044] The three-dimensional visual processing unit 300 includes a target image acquisition module 301 and an image preprocessing module 302. The target image acquisition module 301 realizes multi-directional image acquisition of the target to be captured through a binocular camera. The image preprocessing module 302 is used to perform preprocessing operations on the acquired image to remove irrelevant information in the image and improve the visibility of the image. Specifically, the image is preprocessed through image graying, image filtering, image binarization and image morphological processing, wherein the image graying operation is realized by a maximum value method, an average value method or a weighted average method, wherein the image filtering operation is realized by a mean filtering method, a median filtering method, a Gaussian filtering method or a bilateral filtering method, wherein the image binarization operation is realized by a maximum inter-class variance method, wherein the image morphological processing is realized by four basic operations of corrosion, expansion, opening operation and closing operation;

[0045] The three-dimensional visual processing unit 300 also includes a target recognition module 303 and a target positioning module 304. The target recognition module 303 is used to compare and identify the preprocessed image in the image preprocessing module 302 with the image stored in the data storage library, and obtain relevant information of the target to be grasped based on the comparison and recognition result, and upload the preprocessed image to the data storage library for storage. The target positioning module 304 is used to perform contour recognition of the target to be grasped from four aspects: circularity of the contour, ratio of contour area to circumscribed minimum rectangle area, contour hierarchy relationship and Hu moment based on the preprocessed image information, thereby segmenting different types of targets in the image, and performing three-dimensional modeling of the target to be grasped to locate the target to be grasped;

[0046] The robot control unit 400 includes a target position adjustment module 401. The target position adjustment module 401 is used to adjust the position of the target to be grasped by controlling the robot to perform grasping and pushing actions on the target to be grasped at the same time when there is not enough space for grasping operation. The robot is trained to perform actions through the designed grasping reward function and pushing reward function to reduce the influence of randomness on the grasping success rate during the pushing process. The grasping reward function adopts a sparse reward function, and its specific calculation formula is as follows:

[0047]

[0048] In the formula, the reward R collsion ={0, -0.5}, and the specific calculation formula of the driving reward function is as follows:

[0049] R p =R collsion +w·D(s t ,s t+1 )+c'-c

[0050] In the formula, c'-c represents the change of the number of clusters c and c' before and after the promotion, D(s t ,s t+1 ) represents the point cloud s before pushing t With the pushed point cloud s t+1 The chamfer distance between them, w represents the weight coefficient of the state change before and after pushing;

[0051] The robot control unit 400 also includes a grasping posture adjustment module 402 and a grasping route planning module 403. The grasping posture adjustment module 402 is used to adjust the position of the robot based on the position information of the target to be grasped adjusted by the target position adjustment module 401, and adjust the lower claw position, grasping angle and grasping posture of the robot manipulator after the adjustment is completed. The grasping route planning module 403 is used to obtain the initial grasping route of the robot manipulator based on the current position of the robot manipulator and the position of the target to be grasped, and partially adjust the initial grasping route in consideration of route obstacles and the limitation of the degree of freedom of the robot manipulator, and on this basis, further adjust the adjusted route through the route interpolation algorithm, and the specific calculation formula is as follows:

[0052]

[0053] In the formula, f arm represents the operating frequency of the robot arm, (x p-1 ,y p-1 ,z p-1 ) represents the route node on the initial crawling route, (x' p ,y' p ,z' p ) represents the route node on the initial crawling route after partial adjustment, d c represents the interpolation distance of the initial grasping route after partial adjustment, N c represents the number of steps of the initial grasping route after some adjustments to improve the smoothness of the robot arm during the grasping process;

[0054] The target grabbing unit 500 includes a view angle selection module 501, which is used to select a grabbing view angle of the robot when performing a grabbing operation through a view angle selector, so that the robot can further identify the position of the target to be grabbed in the global image;

[0055] The target grasping unit 500 further includes a grasping force adjustment module 502 and a grasping action execution module 503. The grasping force adjustment module 502 is used to calculate the required grasping force through a grasping force prediction algorithm based on the acquired weight information of the target to be grasped. The specific calculation formula is as follows:

[0056] F=mg-kf mg

[0057] In the formula, F represents the grasping force required by the robot arm, m represents the weight of the target to be grasped, g represents the acceleration of gravity, and k f It represents the friction coefficient between the robot arm and the target to be grasped, and adjusts the grasping force of the robot arm based on the calculation result. The grasping action execution module 503 is used to control the robot arm to perform the grasping operation.

[0058] Working principle: The present invention determines the relationship between the image coordinate system and the camera coordinate system through the camera calibration module 101 to realize the calibration of the binocular camera, determines the relationship between the camera coordinate system and the robot terminal coordinate system through the hand-eye calibration module 102, determines the relationship between the terminal grasping tool coordinate system and the robot terminal coordinate system through the tool calibration module 103, uploads the data information related to the target to be grasped to the data storage library for storage through the target data storage module 201, updates the data information in the data storage library through the target data update module 202, acquires multi-directional images of the target to be grasped through the target image acquisition module 301, pre-processes the acquired images through the image pre-processing module 302, and compares the pre-processed images in the image pre-processing module 302 with the images stored in the data storage library through the target recognition module 303. The image is compared and identified, and relevant information of the target to be grasped is obtained based on the comparison and identification result. The target to be grasped is positioned by the target positioning module 304. The target position adjustment module 401 performs grasping and pushing actions on the target to be grasped at the same time to adjust the position of the target to be grasped. The position of the robot is adjusted by the grasping posture adjustment module 402, and after the adjustment, the lower claw position, grasping angle and grasping posture of the robot mechanical arm are adjusted. The adjusted route is further adjusted by the grasping route planning module 403 to improve the smoothness of the robot mechanical arm during the grasping process. The grasping perspective of the robot when performing the grasping operation is selected by the perspective selection module 501. The grasping force adjustment module 502 adjusts the grasping force of the robot mechanical arm. The grasping operation is executed by the grasping action execution module 503.

[0059] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0060] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A robot vision automated grasping system, characterized in that: It comprises a calibration unit (100), wherein the calibration unit (100) is used to respectively determine the relationship between the image coordinate system and the camera coordinate system and the lens distortion of the camera, determine the relationship between the camera coordinate system and the robot end coordinate system, and determine the relationship between the end gripping tool coordinate system and the robot end coordinate system; The calibration unit (100) further comprises a hand-eye calibration module (102) and a tool calibration module (103). The hand-eye calibration module (102) is used to determine the relationship between the camera coordinate system and the robot end coordinate system. Based on the positional relationship between the binocular camera and the robot mechanical arm, the hand-eye calibration can be divided into two types: eyes on the hand and eyes outside the hand. The specific operation steps are as follows: selecting the hand-eye calibration mode as eyes on the hand or eyes outside the hand, selecting the calibration plate type and setting the calibration plate parameters, the hand-eye calibrator object creates a corresponding calibration plate object for identifying the calibration plate. After the user controls the robot to move, the robot sends a collection signal to the hand-eye calibrator. After receiving the signal, the hand-eye calibrator obtains the target image through the visual sensor, and then calls the calibration plate object for identification. The hand-eye calibrator then obtains the current coordinate information of the robot from the robot, stores the current coordinate of the robot, the calibration plate posture and the corresponding target image as a set of data into the calibration data set, and performs hand-eye calibration through the hand-eye calibrator. The tool calibration module (103) is used to determine the relationship between the end grasping tool coordinate system and the robot end coordinate system through the six-point method; A data storage unit (200), the data storage unit (200) being used to store data information related to the target to be grasped based on an established data storage library related to the target to be grasped, so as to compare and identify the collected image with the image stored in the data storage library during the grasping operation, thereby obtaining specific information of the target to be grasped; A three-dimensional visual processing unit (300), the three-dimensional visual processing unit (300) being used to collect images of a target to be grasped during a grasping operation, and to compare and identify the collected images with images stored in a data storage library after preprocessing, so as to obtain specific information of the target to be grasped, and to locate the target to be grasped based on the preprocessed image information; The three-dimensional visual processing unit (300) further comprises a target recognition module (303) and a target positioning module (304), wherein the target recognition module (303) is used to compare and identify the preprocessed image in the image preprocessing module (302) with the image stored in the data storage library, and obtain relevant information of the target to be grasped based on the comparison and recognition result, and upload the preprocessed image to the data storage library for storage, and the target positioning module (304) is used to perform contour recognition of the target to be grasped from four aspects: circularity of the contour, ratio of contour area to circumscribed minimum rectangle area, contour hierarchy relationship and Hu moment based on the preprocessed image information, thereby segmenting different types of targets in the image, and performing three-dimensional modeling on the target to be grasped, so as to locate the target to be grasped; A robot control unit (400), the robot control unit (400) being used to adjust the position of a target to be grasped and the grasping posture of the robot when it is detected that there is insufficient space for grasping operations, and after the adjustments are completed, to plan and adjust the grasping route of the target to be grasped; The robot control unit (400) comprises a target position adjustment module (401), which is used to adjust the position of the target to be grasped by controlling the robot to simultaneously perform a grasping action and a pushing action on the target to be grasped when there is not enough space for grasping operation, and to perform action training on the robot by using a designed grasping reward function and a pushing reward function to reduce the influence of randomness on the grasping success rate during the pushing process, and the grasping reward function adopts a sparse reward function, and its specific calculation formula is as follows: In the formula, the reward R collsion ={0, -0.5}, and the specific calculation formula of the driving reward function is as follows: R p =R collsion +w·D(s t ,s t+1 )+c'-c In the formula, c'-c represents the change of the number of clusters c and c' before and after the promotion, D(s t ,s t+1 ) represents the point cloud s before pushing t With the pushed point cloud s t+1 The chamfer distance between them, w represents the weight coefficient of the state change before and after pushing; The robot control unit (400) further comprises a grasping posture adjustment module (402) and a grasping route planning module (403), wherein the grasping posture adjustment module (402) is used to adjust the position of the robot based on the position information of the target to be grasped adjusted by the target position adjustment module (401), and after the adjustment is completed, the position of the lower claw of the robot arm, the grasping angle and the grasping posture are adjusted, and the grasping route planning module (403) is used to obtain the initial grasping route of the robot arm based on the current position of the robot arm and the position of the target to be grasped, and partially adjust the initial grasping route in consideration of route obstacles and the limitation of the degree of freedom of the robot arm, and on this basis, further adjust the adjusted route through a route interpolation algorithm to improve the smoothness of the robot arm during the grasping process, and the specific calculation formula is: In the formula, f arm represents the operating frequency of the robot arm, (x p-1 ,y p-1 ,z p-1 ) represents the route node on the initial crawling route, (x' p ,y' p ,z' p ) represents the route node on the initial crawling route after partial adjustment, d c represents the interpolation distance of the initial grasping route after partial adjustment, N c represents the number of steps of the initial grasping route after partial adjustment; A target grasping unit (500) is used to select a viewing angle for the robot to start a grasping operation, and after the selection is completed, based on the acquired specific information of the target to be grasped, adjust the grasping force of the grasping tool at the end of the robot, and perform the grasping operation after the adjustment is completed.

2. A robot vision automated grasping system according to claim 1, characterized in that: The calibration unit (100) comprises a camera calibration module (101), which is used to obtain the correspondence between a point in a three-dimensional space and a point in a pixel coordinate system through a traditional camera calibration method based on a calibration plate, and simultaneously obtain the intrinsic parameters, extrinsic parameters and distortion parameter matrix of the binocular camera, thereby determining the relationship between the image coordinate system and the camera coordinate system, thereby achieving calibration of the binocular camera.

3. A robot vision automated grasping system according to claim 1, characterized in that: The data storage unit (200) comprises a target data storage module (201) and a target data update module (202); the target data storage module (201) is used to upload data information related to a target to be captured to a data storage repository for storage, and the data information related to the target to be captured includes the size, weight and type of the target to be captured in the image; the target data update module (202) is used to update the data information in the data storage repository by uploading an image that is compared and identified with an image stored in the data storage repository and the comparison and identification result shows that the image is of the same type of target to the data storage repository.

4. The robot vision automated grasping system according to claim 1, characterized in that: The three-dimensional visual processing unit (300) comprises a target image acquisition module (301) and an image preprocessing module (302), wherein the target image acquisition module (301) realizes multi-directional image acquisition of a target to be captured by using a binocular camera, and the image preprocessing module (302) is used to perform a preprocessing operation on the acquired image to remove irrelevant information in the image and improve the visibility of the image, specifically realizing the preprocessing of the image by image graying, image filtering, image binarization and image morphological processing, wherein the image graying operation is realized by a maximum value method, an average value method or a weighted average method, wherein the image filtering operation is realized by a mean filtering method, a median filtering method, a Gaussian filtering method or a bilateral filtering method, wherein the image binarization operation is realized by a maximum inter-class variance method, and wherein the image morphological processing is realized by four basic operations of corrosion, expansion, opening operation and closing operation.

5. The robot vision automated grasping system according to claim 1, characterized in that: The target grasping unit (500) comprises a view angle selection module (501), and the view angle selection module (501) is used to select a grasping view angle of the robot when performing a grasping operation through a view angle selector, so that the robot can further identify the position of the target to be grasped in the global image.

6. The robot vision automated grasping system according to claim 1, characterized in that: The target grasping unit (500) further comprises a grasping force adjustment module (502) and a grasping action execution module (503), wherein the grasping force adjustment module (502) is used to calculate the required grasping force based on the acquired weight information of the target to be grasped by using a grasping force prediction algorithm, and adjust the grasping force of the robot mechanical arm based on the calculation result, and the grasping action execution module (503) is used to control the robot mechanical arm to perform a grasping operation.