Method and device for grasping an object that is not completely occluded, computer device and medium
By generating point cloud models using a 3D vision camera and performing hand-eye matrix conversion, the accuracy problem of intelligent robots grasping partially occluded objects was solved, achieving higher grasping accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- FOSHAN XIANYANG TECHNOLOGY CO LTD
- Filing Date
- 2023-05-26
- Publication Date
- 2026-06-23
Smart Images
Figure CN116604558B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of intelligent robot technology, and in particular to a method, apparatus, computer equipment and medium for grasping objects that are not completely obscured. Background Technology
[0002] Intelligent robot grasping technology has been widely applied in industrial tasks such as material handling, assembly, picking, and sorting. In material handling and conveying tasks, robots perceive the position, size, and shape of objects along the transport path, and adjust their movement trajectory and speed based on real-time traffic conditions and path planning algorithms, achieving efficient and precise material handling. In product assembly tasks, robots perceive the position and shape of components and perform assembly operations according to preset assembly sequences and quality standards, achieving efficient and precise assembly.
[0003] Existing intelligent robot grasping methods require a large amount of computing resources and time for object modeling and segmentation, which cannot meet the requirements of real-time performance. Furthermore, during the grasping process, there may be occlusions that block the target, making it difficult for existing grasping methods to accurately segment the object when there are many occlusions, thus making it difficult for the robot to perform precise grasping. Summary of the Invention
[0004] The purpose of this application is to provide a method, apparatus, computer device, and medium for grasping objects that are not completely obscured, thereby improving the accuracy of robot grasping of target objects.
[0005] To address the aforementioned technical problems, embodiments of this application provide a method for grasping objects that are not completely obscured, including:
[0006] A target point cloud model is generated by capturing images of the object to be grasped and processing the model construction through a 3D vision camera.
[0007] The point cloud information of the object to be captured in the target point cloud model is matched with the point cloud information of the capturing environment to obtain the spatial information to be captured.
[0008] Based on the spatial information, the target grasping pose is obtained through the conversion of the hand-eye matrix;
[0009] Determine whether there is obstructing point cloud information in the region corresponding to the target grasping pose;
[0010] If the obstructing point cloud information does not exist, the robot is controlled to grasp the object to be grasped based on the target grasping pose.
[0011] To address the aforementioned technical problems, embodiments of this application provide a grasping device for objects that are not completely obscured, comprising:
[0012] The target point cloud model generation unit is used to capture and model the object to be grasped using a 3D vision camera to generate a target point cloud model.
[0013] The spatial information acquisition unit is used to match the point cloud information of the object to be captured in the target point cloud model with the point cloud information of the capture environment in order to obtain the spatial information to be captured.
[0014] The target grasping position generation unit is used to obtain the target grasping pose based on the spatial information through the transformation of the hand-eye matrix;
[0015] The obstruction point cloud information determination unit is used to determine whether there is obstruction point cloud information in the area corresponding to the target grasping pose;
[0016] The object-to-be-grabbed unit is used to control the robot to grasp the object based on the target grasping pose if the obstructing point cloud information does not exist.
[0017] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is to provide a computer device, including one or more processors; and a memory for storing one or more programs, so that the one or more processors implement the grasping method for partially obscured objects as described in any one of the above-mentioned methods.
[0018] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is: a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program, when executed by a processor, implements the grasping method for partially obscured objects as described in any one of the above-mentioned methods.
[0019] This invention provides a method, apparatus, computer device, and medium for grasping objects that are not completely occluded. The invention uses a 3D vision camera to photograph and model the object to be grasped, generating a target point cloud model. The point cloud information of the object to be grasped in the target point cloud model is matched with the point cloud information of the grasping environment to obtain the spatial information to be grasped. Based on the spatial information, a target grasping pose is obtained through hand-eye matrix conversion. It is determined whether there is obstructing point cloud information in the area corresponding to the target grasping pose. If no obstructing point cloud information exists, the robot is controlled to grasp the object based on the target grasping pose. This invention uses a 3D vision camera to photograph and model the object to be grasped, generating a target point cloud model. This allows for the identification of whether there are occluded objects on the object to be grasped based on the target point cloud model, avoiding the impact of occlusion on the accuracy of the robot's grasping, thereby improving the accuracy of grasping the target object. Attached Figure Description
[0020] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 A flowchart illustrating an implementation of a method for grasping objects that are not completely obscured, according to an embodiment of this application;
[0022] Figure 2 This is a flowchart of a sub-process in the grasping method for objects that are not completely obscured, provided in the embodiments of this application;
[0023] Figure 3 This is another implementation flowchart of a sub-process in the grasping method for objects that are not completely obscured provided in the embodiments of this application;
[0024] Figure 4 This is another implementation flowchart of a sub-process in the grasping method for objects that are not completely obscured provided in the embodiments of this application;
[0025] Figure 5 This is another implementation flowchart of a sub-process in the grasping method for objects that are not completely obscured provided in the embodiments of this application;
[0026] Figure 6 This is another implementation flowchart of a sub-process in the grasping method for objects that are not completely obscured provided in the embodiments of this application;
[0027] Figure 7 This is another implementation flowchart of a sub-process in the grasping method for objects that are not completely obscured provided in the embodiments of this application;
[0028] Figure 8 This is a schematic diagram of a gripping device for objects that are not completely obscured, provided in an embodiment of this application.
[0029] Figure 9 This is a schematic diagram of the computer device provided in the embodiments of this application. Detailed Implementation
[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.
[0031] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0032] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.
[0033] It should be noted that the grasping method for partially obscured objects provided in this application embodiment is generally executed by a computer device, and correspondingly, the grasping device for partially obscured objects is generally configured in the computer device.
[0034] Please see Figure 1 , Figure 1 This illustrates one specific implementation of a method for grasping objects that are not completely obscured.
[0035] It should be noted that if substantially the same result is obtained, the method of this invention is not based on... Figure 1 Limited to the order of the processes shown, this method includes the following steps:
[0036] S1. The object to be grasped is photographed and modeled using a 3D vision camera to generate a target point cloud model.
[0037] In this embodiment of the application, before implementing step S1, a 3D vision camera and a robot are connected to a computer device so that the computer device and the 3D vision camera, and the computer device and the robot can communicate respectively.
[0038] Specifically, a 3D vision camera is a high-speed, high-definition 3D vision camera that uses structured light to create point cloud images of the object to be grasped. The 3D vision camera also includes a point cloud generation module, which generates point cloud data from the captured image after the object is captured. The generated point cloud data is then transmitted to a computer for model building processing, thereby generating a target point cloud model of the object to be grasped.
[0039] Please see Figure 2 , Figure 2 A specific implementation of step S1 is shown below:
[0040] S11. The three-dimensional vision camera emits a raster-coded light pattern to the object to be grasped to obtain an image corresponding to the object to be grasped, thus obtaining an initial input image.
[0041] In this embodiment, a set of raster-coded light patterns is emitted towards the object to be grasped using a 3D vision camera. These light patterns are then illuminated on the surface of the object. The 3D vision camera then captures an image of the light patterns illuminating the surface of the object, thus obtaining an initial input image. The initial input image includes the raster-coded light pattern image and a white light image illuminating the object.
[0042] S12. Generate the three-dimensional coordinates of the object to be captured based on the initial input image, and generate point cloud data based on the three-dimensional coordinates.
[0043] Please see Figure 3 , Figure 3 A specific implementation of step S12 is shown below:
[0044] S121. By decoding the raster-coded light pattern image, the encoded value of each pixel is obtained.
[0045] S122. Based on the encoded value, the camera intrinsic parameters of the three-dimensional vision camera, and the position information of the light ripple within the field of view of the three-dimensional vision camera, calculate the depth information of each pixel.
[0046] S123. The depth information of each pixel is converted into the three-dimensional coordinates of the object to be captured, and the point cloud data is generated based on the three-dimensional coordinates.
[0047] Specifically, the point cloud generation module of the 3D vision camera performs point cloud generation processing on the captured image to obtain point cloud data. In this embodiment, by decoding the captured raster-coded light ripple image, the encoded value of each pixel can be obtained; then, using the obtained encoded value and camera intrinsic parameters, combined with the position information of the light ripple within the field of view of the 3D vision camera, the depth information of each pixel can be calculated; finally, by converting the depth information of each pixel into corresponding 3D coordinate information, the 3D coordinates of each point on the surface of the object to be grasped can be obtained. Once the 3D coordinates of all points are generated, point cloud data is obtained.
[0048] S13. The target point cloud model is obtained by filtering and model generation processing of the point cloud data.
[0049] Please see Figure 4 , Figure 4 A specific implementation of step S13 is shown below:
[0050] S131. The point cloud data is filtered and downsampled to obtain initial point cloud data.
[0051] S132. Obtain target point cloud data by selecting a bounding box or performing ROI processing on the initial point cloud data.
[0052] S133. According to the preset model construction method, the target point cloud data is processed to generate a model, thereby obtaining the target point cloud model.
[0053] In this embodiment of the application, in order to generate a more accurate point cloud model and reduce the amount of data used to build the model, the point cloud data is filtered and downsampled to obtain initial point cloud data.
[0054] Further, before implementing step S132, this embodiment first constructs a rectangular bounding box, which is used to select the point cloud data within the range of the object to be grasped. Further, this embodiment can also construct a region of interest (ROI), which can be a rectangle, circle, ellipse, or irregular shape, for ROI processing of the object to be grasped to obtain target point cloud data. It should be noted that the constructed rectangular bounding box or ROI is based on the actual object to be grasped. Model construction methods include Normalized point-to-point ICP, feature point-based model registration, Euclidean distance-based point cloud registration, Randomized Randomized Transform Estimation (RANSAC), and surface reconstruction algorithms for point clouds. This embodiment selects any model construction method to perform model generation processing on the target point cloud data to obtain the target point cloud model.
[0055] S2. Match the point cloud information of the object to be captured in the target point cloud model with the point cloud information of the capturing environment to obtain the spatial information to be captured.
[0056] In this embodiment, the point cloud information of the object to be grasped in the target point cloud model is matched with the point cloud information of the grasping environment to obtain the spatial information to be grasped. That is, the information of the object to be grasped in the grasping scene is determined, facilitating the subsequent construction of the target grasping pose. Furthermore, the point cloud information of the grasping environment can be obtained from the environment corresponding to the object to be grasped in the target point cloud model, or it can be obtained by capturing and constructing point cloud information of the grasping environment using a 3D vision camera. Here, the point cloud information of the object to be grasped refers to the point cloud information corresponding to the object to be grasped.
[0057] S3. Based on the spatial information, the target grasping pose is obtained through the conversion of the hand-eye matrix.
[0058] In this embodiment of the application, the process of forming a mechanical hand-eye matrix requires the use of a basic coordinate system, a robot arm end effector coordinate system, a 3D vision camera coordinate system, and a coordinate system of the object to be grasped. The above coordinate systems are transformed by matrix to obtain the target grasping pose.
[0059] Please see Figure 5 , Figure 5 A specific implementation method prior to step S3 is shown below in detail:
[0060] S31. Obtain the current position of the robot.
[0061] S32. Calculate the target position of the robot based on the current position and the target point cloud model.
[0062] S33. Control the robot to move to the target location.
[0063] In this embodiment, in a scenario involving an object to be grasped, the robot is moved to a suitable position to grasp the object, enabling its robotic arm to grab it. Therefore, in this embodiment, the robot's current position is first obtained. Then, based on the point cloud information of the object to be grasped in the target point cloud model and the current position, the target point the robot needs to move to is calculated. Once the target point is calculated, the robot is controlled to move to that target point for subsequent grasping of the object.
[0064] S4. Determine whether there is obstructing point cloud information in the area corresponding to the target grasping pose.
[0065] In this embodiment, if multiple obstructions exist around the object to be grasped, it will interfere with the robot's grasping of the object. Therefore, this embodiment first determines whether there is obstructing point cloud information in the area corresponding to the target grasping pose. If there is, it means that there are obstructions around the object to be grasped, which may affect the robot's grasping accuracy; if there are no obstructions, it at least means that there are no obstructions on the path that the robot wants to grasp, that is, there are no obstructions affecting the robot's grasping.
[0066] Please see Figure 6 , Figure 6 A specific implementation method following step S4 is shown below:
[0067] S41. If the obstruction point cloud information exists, calculate the distance between the obstruction point cloud information and the object to be captured to obtain the target distance.
[0068] S42. The target distance is compared with the grasping pose threshold to determine the magnitude of the distance and the result.
[0069] In this embodiment, if obstructing point cloud information exists, meaning there is an occlusion on the object to be grasped, this embodiment needs to determine whether the occlusion will affect the robot's grasping of the object. Therefore, the target distance is obtained by calculating the distance between the obstructing point cloud information and the object to be grasped, and then the target distance is compared with the grasping pose threshold.
[0070] Please see Figure 7 , Figure 7 A specific implementation method following step S42 is shown below:
[0071] S421. If the judgment result is that the target distance is less than the grasping pose threshold, then obtain the position of the occluder corresponding to the blocking point cloud information and the direction of the occluder relative to the robot, and obtain the position and direction of the occluder.
[0072] S422. Control the robot to move towards the position and direction of the obstruction to push away the obstruction and grasp the object to be grasped.
[0073] In this embodiment, if the target distance is less than the grasping pose threshold, it indicates that the robot's grasping process will be affected by occlusions, potentially leading to the simultaneous grasping of both the object to be grasped and the occluder. Therefore, the position of the occluder corresponding to the obstructing point cloud information and the direction of the occluder relative to the robot are obtained to determine the occluder's position and direction. The robot is then controlled to move towards the occluder's position and direction to push away the occluder and grasp the object to be grasped. The grasping pose threshold refers to the range within which the robot can grasp the object to be grasped; all objects within this threshold will be grasped by the robot if not pushed aside. Therefore, to avoid simultaneously grasping both the occluder and the object to be grasped, the position and direction of the occluder need to be obtained so that the robot can push it away and prevent it from interfering with the robot's normal grasping process.
[0074] S43. If the judgment result is that the target distance is greater than the grasping pose threshold, then control the robot to directly grasp the object to be grasped.
[0075] In this embodiment of the application, if the determination result is that the target distance is greater than the grasping pose threshold, that is, the occluder is outside the robot's grasping range, the occluder will not affect the robot's grasping of the object to be grasped, so the robot is controlled to directly grasp the object to be grasped.
[0076] S5. If the obstruction point cloud information does not exist, the robot is controlled to grasp the object to be grasped based on the target grasping pose.
[0077] In this embodiment, if there is no obstructing point cloud information, it at least indicates that there are no obstructions on the path the robot wants to grasp, meaning there are no obstructions affecting the robot's grasping ability. Therefore, the robot can directly grasp the object to be grasped. Furthermore, after the robot finally grasps the object, it obtains a preset placement position and then controls the robot to place the object to be grasped at that position.
[0078] This embodiment of the application uses a 3D vision camera to photograph and model the object to be grasped, generating a target point cloud model. The point cloud information of the object to be grasped in the target point cloud model is matched with the point cloud information of the grasping environment to obtain the spatial information to be grasped. Based on the spatial information, a target grasping pose is obtained through hand-eye matrix conversion. It is then determined whether there is obstructing point cloud information in the area corresponding to the target grasping pose. If no obstructing point cloud information exists, the robot is controlled to grasp the object to be grasped based on the target grasping pose. This embodiment of the application uses a 3D vision camera to photograph and model the object to be grasped, generating a target point cloud model. This allows for the identification of whether there are obstructing objects on the object to be grasped based on the target point cloud model, avoiding the impact of obstructing objects on the accuracy of the robot's grasping of the object, thereby improving the accuracy of grasping the target object.
[0079] Please refer to Figure 8 As a response to the above Figure 1 The present application provides an embodiment of a grasping device that does not completely obscure an object, based on the method shown. This embodiment of the device is similar to... Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various computer devices.
[0080] like Figure 8 As shown, the grasping device for objects that are not completely obscured in this embodiment includes: a target point cloud model generation unit 61, a spatial information acquisition unit 62, a target grasping position generation unit 63, an obstruction point cloud information judgment unit 64, and a grasping object grasping unit 65, wherein:
[0081] The target point cloud model generation unit 61 is used to capture and model-build the object to be grasped using a 3D vision camera to generate a target point cloud model.
[0082] The spatial information acquisition unit 62 is used to match the point cloud information of the object to be captured in the target point cloud model with the point cloud information of the capture environment in order to obtain the spatial information to be captured.
[0083] The target grasping position generation unit 63 is used to obtain the target grasping pose based on the spatial information through the conversion of the hand-eye matrix;
[0084] The obstruction point cloud information determination unit 64 is used to determine whether there is obstruction point cloud information in the area corresponding to the target grasping pose.
[0085] The object-to-be-grabbed unit 65 is used to control the robot to grasp the object based on the target grasping pose if the obstructing point cloud information does not exist.
[0086] Furthermore, after the blocking point cloud information judgment unit 64, it also includes:
[0087] The target distance calculation unit is used to calculate the distance between the obstructing point cloud information and the object to be captured if the obstructing point cloud information exists, so as to obtain the target distance;
[0088] The judgment result generation unit is used to compare the target distance with the grasping pose threshold to obtain the judgment result;
[0089] The first grasping unit is used to control the robot to directly grasp the object to be grasped if the judgment result is that the target distance is greater than the grasping pose threshold.
[0090] Furthermore, the judgment result generation unit also includes:
[0091] An occlusion location acquisition unit is used to acquire the location of the occlusion corresponding to the blocking point cloud information and the direction of the occlusion relative to the robot if the judgment result is that the target distance is less than the grasping pose threshold, thereby obtaining the occlusion location and direction.
[0092] The second grasping unit is used to control the robot to move towards the position and direction of the obstruction, so as to push away the obstruction and grasp the object to be grasped.
[0093] Furthermore, the target point cloud model generation unit 61 includes:
[0094] The initial input image acquisition unit is used to emit a raster-coded light pattern to the object to be grasped through the three-dimensional vision camera to obtain an image corresponding to the object to be grasped, thereby obtaining an initial input image;
[0095] A point cloud data generation unit is used to generate three-dimensional coordinates corresponding to the object to be captured based on the initial input image, and to generate point cloud data based on the three-dimensional coordinates.
[0096] The model generation unit is used to obtain the target point cloud model by filtering and generating the point cloud data.
[0097] Furthermore, the point cloud data generation unit includes:
[0098] A raster-coded stencil image decoding unit is used to decode the raster-coded stencil image to obtain the encoded value of each pixel.
[0099] The depth information calculation unit is used to calculate the depth information of each pixel based on the encoded value, the camera intrinsic parameters of the three-dimensional vision camera, and the position information of the light ripple within the field of view of the three-dimensional vision camera.
[0100] The three-dimensional coordinate generation unit is used to convert the depth information of each pixel into the three-dimensional coordinates corresponding to the object to be grasped, and to generate the point cloud data based on the three-dimensional coordinates.
[0101] Furthermore, the three-dimensional coordinate transformation unit includes:
[0102] An initial point cloud data generation unit is used to filter and downsample the point cloud data to obtain initial point cloud data.
[0103] The target point cloud data filtering unit is used to obtain target point cloud data by performing box selection or ROI processing on the initial point cloud data;
[0104] The model building unit is used to perform model generation processing on the target point cloud data according to a preset model building method to obtain the target point cloud model.
[0105] Furthermore, the spatial information acquisition unit 62 also includes:
[0106] Current position acquisition unit, used to acquire the current position of the robot;
[0107] The target point cloud computing unit is used to calculate the target point location of the robot based on the current location and the target point cloud model.
[0108] A robot movement unit is used to control the robot to move to the target point.
[0109] This invention utilizes a 3D vision camera to capture and model an object to be grasped, generating a target point cloud model. The point cloud information of the object to be grasped in the target point cloud model is matched with the point cloud information of the grasping environment to obtain the spatial information to be grasped. Based on this spatial information, a target grasping pose is obtained through hand-eye matrix conversion. It is then determined whether obstructing point cloud information exists in the region corresponding to the target grasping pose. If no obstructing point cloud information exists, the robot is controlled to grasp the object based on the target grasping pose. This embodiment of the invention uses a 3D vision camera to capture and model the object to be grasped, generating a target point cloud model. This allows for the identification of whether there are occluding objects on the object to be grasped based on the target point cloud model, avoiding the impact of occluding objects on the accuracy of the robot's grasping, thereby improving the precision of grasping the target object.
[0110] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 9 , Figure 9 This is a basic structural block diagram of the computer device in this embodiment.
[0111] Computer device 8 includes a memory 81, a processor 82, and a network interface 83 that are interconnected via a system bus. It should be noted that only computer device 8 with three components—memory 81, processor 82, and network interface 83—is shown in the figure. However, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented alternatively.
[0112] The memory 81 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 81 may be an internal storage unit of the computer device 8, such as the hard disk or memory of the computer device 8. In other embodiments, the memory 81 may also be an external storage device of the computer device 8, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 8. Of course, the memory 81 may also include both internal storage units and external storage devices of the computer device 8. In this embodiment, the memory 81 is typically used to store the operating system and various application software installed on the computer device 8, such as program code for a method of grasping objects that are not completely obscured. In addition, the memory 81 can also be used to temporarily store various types of data that have been output or will be output.
[0113] In some embodiments, processor 82 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. This processor 82 is typically used to control the overall operation of the computer device 8. In this embodiment, processor 82 is used to run program code stored in memory 81 or process data, for example, to run the program code of the above-described method for grasping objects that are not completely obscured, to implement various embodiments of the method for grasping objects that are not completely obscured.
[0114] The network interface 83 may include a wireless network interface or a wired network interface, which is typically used to establish a communication connection between the computer device 8 and other electronic devices.
[0115] This application also provides another embodiment, namely, a computer-readable storage medium storing a computer program that can be executed by at least one processor to cause the at least one processor to perform the steps of the grasping method for an object that is not completely obscured as described above.
[0116] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this application.
[0117] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.
Claims
1. A method for grasping an object that is not completely obscured, characterized in that, include: A target point cloud model is generated by capturing images of the object to be grasped and processing the model construction through a 3D vision camera. The point cloud information of the object to be captured in the target point cloud model is matched with the point cloud information of the capturing environment to obtain the spatial information to be captured. Based on the spatial information, the target grasping pose is obtained through the conversion of the hand-eye matrix; Determine whether there is obstructing point cloud information in the region corresponding to the target grasping pose; If the obstructing point cloud information does not exist, the robot is controlled to grasp the object to be grasped based on the target grasping pose. If the obstruction point cloud information exists, the distance between the obstruction point cloud information and the object to be captured is calculated to obtain the target distance; The target distance is compared with the grasping pose threshold to determine the magnitude of the distance and the result. If the determination result is that the target distance is greater than the grasping pose threshold, then control the robot to directly grasp the object to be grasped; If the judgment result is that the target distance is less than the grasping pose threshold, then the position of the occluder corresponding to the blocking point cloud information and the direction of the occluder relative to the robot are obtained to obtain the position and direction of the occluder; The robot is controlled to move towards the location and direction of the obstruction to push it aside and then grasp the object to be grabbed.
2. The method for grasping an object that is not completely obscured according to claim 1, characterized in that, The process of capturing and modeling the object to be grasped using a 3D vision camera to generate a target point cloud model includes: The three-dimensional vision camera emits a raster-coded light pattern to the object to be grasped in order to obtain an image corresponding to the object to be grasped, thus obtaining an initial input image. The three-dimensional coordinates of the object to be captured are generated based on the initial input image, and point cloud data is generated based on the three-dimensional coordinates. The target point cloud model is obtained by filtering and model generation of the point cloud data.
3. The method for grasping an object that is not completely obscured according to claim 2, characterized in that, The initial input image includes a raster-coded light pattern image and a white light image. The process of generating the three-dimensional coordinates of the object to be grasped based on the initial input image, and generating point cloud data based on the three-dimensional coordinates, includes: By decoding the raster-coded light pattern image, the encoded value of each pixel is obtained; Based on the encoded value, the camera intrinsic parameters of the 3D vision camera, and the position information of the light ripple within the field of view of the 3D vision camera, the depth information of each pixel is calculated. The depth information of each pixel is converted into the three-dimensional coordinates of the object to be captured, and the point cloud data is generated based on the three-dimensional coordinates.
4. The method for grasping an object that is not completely obscured according to claim 2, characterized in that, The process of filtering and generating a model from the point cloud data to obtain the target point cloud model includes: The point cloud data is filtered and downsampled to obtain initial point cloud data. Target point cloud data is obtained by selecting boxes or performing ROI processing on the initial point cloud data; According to the preset model construction method, the target point cloud data is processed to generate a model, thereby obtaining the target point cloud model.
5. The method for grasping an object that is not completely obscured according to any one of claims 1 to 4, characterized in that, Before obtaining the target grasping pose based on the spatial information through the hand-eye matrix conversion, the method further includes: Obtain the current position of the robot; The target point of the robot is calculated based on the current position and the target point cloud model; Control the robot to move to the target location.
6. A gripping device for objects that are not completely obscured, characterized in that, include: The target point cloud model generation unit is used to capture and model the object to be grasped using a 3D vision camera to generate a target point cloud model. The spatial information acquisition unit is used to match the point cloud information of the object to be captured in the target point cloud model with the point cloud information of the capture environment in order to obtain the spatial information to be captured. The target grasping position generation unit is used to obtain the target grasping pose based on the spatial information through the transformation of the hand-eye matrix; The obstruction point cloud information determination unit is used to determine whether there is obstruction point cloud information in the area corresponding to the target grasping pose; The object-to-be-grabbed unit is used to control the robot to grasp the object based on the target grasping pose if the obstructing point cloud information does not exist. The target distance calculation unit is used to calculate the distance between the obstructing point cloud information and the object to be captured if the obstructing point cloud information exists, so as to obtain the target distance; The judgment result generation unit is used to compare the target distance with the grasping pose threshold to obtain the judgment result; The first grasping unit is configured to control the robot to directly grasp the object to be grasped if the judgment result is that the target distance is greater than the grasping pose threshold. An occlusion location acquisition unit is used to acquire the location of the occlusion corresponding to the blocking point cloud information and the direction of the occlusion relative to the robot if the judgment result is that the target distance is less than the grasping pose threshold, thereby obtaining the occlusion location and direction. The second grasping unit is used to control the robot to move towards the position and direction of the obstruction, so as to push away the obstruction and grasp the object to be grasped.
7. A computer device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the grasping method for an object that is not completely obscured as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the grasping method for an object that is not completely obscured as described in any one of claims 1 to 5.