Robot-based object grasping method and device, robot, and storage medium
By processing point cloud images and determining attribute information, the robot can accurately grasp three-dimensional objects in complex environments, solving the problems of high computational load and low efficiency in existing technologies, and achieving efficient object recognition and grasping.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INDUSTRIAL AND COMMERCIAL BANK OF CHINA
- Filing Date
- 2023-04-24
- Publication Date
- 2026-06-09
Smart Images
Figure CN116485896B_ABST
Abstract
Description
Technical Field
[0001] This application relates to artificial intelligence technology, and more particularly to a robot-based object grasping method, device, robot, and storage medium. Background Technology
[0002] Robot vision is a rapidly developing branch of artificial intelligence. Simply put, robot vision uses robots to replace human eyes for measurement and judgment. Through robot vision technology, robots can be controlled to automatically grasp objects in their environment.
[0003] With the rapid development of low-cost depth sensors and LiDAR, acquiring 3D point cloud data has become increasingly convenient. How to identify objects in the environment using 3D point cloud data and accurately grasp them has become a key research focus. Summary of the Invention
[0004] This application provides a robot-based object grasping method, device, robot, and storage medium to improve the accuracy of object grasping.
[0005] Firstly, this application provides a robot-based object grasping method, which is applied to a robot equipped with a gripper; the method includes:
[0006] A point cloud image of the scene where the target object is located is obtained, wherein the point cloud image includes point cloud data of the scene where the target object is located; and the target object is automatically selected in the point cloud image to obtain a target bounding box of the target object; wherein the target bounding box contains the point cloud data of the target object;
[0007] Based on the point cloud data in the target box, surface extraction processing is performed on the target object in the target box to obtain a data set to be processed for the target object; wherein, the data set to be processed includes point cloud data of the surface of the target object;
[0008] Based on the data set to be processed for the target object, the attribute information of the target object is determined; wherein, the attribute information is used to represent the size information and position information of the target object;
[0009] Based on the attribute information of the target object, the robot's gripper is controlled to grasp the target object.
[0010] Secondly, this application provides a robot-based object grasping device, which is applied to a robot equipped with a gripper; the device includes:
[0011] An object bounding box module is used to acquire a point cloud image of the scene where a target object is located, wherein the point cloud image includes point cloud data of the scene where the target object is located; and to perform automatic bounding box processing on the target object in the point cloud image to obtain a target bounding box of the target object; wherein the target bounding box contains the point cloud data of the target object;
[0012] The surface extraction module is used to perform surface extraction processing on the target object in the target box based on the point cloud data in the target box, to obtain a data set to be processed of the target object; wherein, the data set to be processed includes point cloud data of the surface of the target object;
[0013] An information determination module is used to determine the attribute information of the target object based on the data set to be processed of the target object; wherein the attribute information is used to represent the size information and position information of the target object;
[0014] The object grasping module is used to control the robot's gripper to grasp the target object based on the target object's attribute information.
[0015] Thirdly, this application provides a robot, including: a processor, and a memory communicatively connected to the processor;
[0016] The memory stores computer-executed instructions;
[0017] The processor executes computer execution instructions stored in the memory to implement the robot-based object grasping method as described in the first aspect of this application.
[0018] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the robot-based object grasping method as described in the first aspect of this application.
[0019] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the robot-based object grasping method described in the first aspect of this application.
[0020] This application provides a robot-based object grasping method, apparatus, robot, and storage medium. It acquires a point cloud image of the scene containing the target object and obtains a bounding box containing the target object from the point cloud image. That is, the target object in the point cloud image is initially identified. Surface extraction is performed on the target object within the bounding box to obtain point cloud data of each surface of the target object, which serves as data to be processed. Based on the data to be processed, the size and position of the target object are determined, thereby controlling the robot's gripper to grasp the target object. This solves the problems of high computational load and low computational efficiency caused by the need for intensive computation to determine the 6D pose of the object in existing technologies. By determining each surface of the target object, the boundary of the target object can be accurately determined, improving object recognition accuracy, and thus improving object grasping accuracy and efficiency. Attached Figure Description
[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0022] Figure 1 A flowchart illustrating a robot-based object grasping method provided in this application embodiment;
[0023] Figure 2 A schematic diagram of gripping provided in an embodiment of this application;
[0024] Figure 3 A flowchart illustrating a robot-based object grasping method provided in this application embodiment;
[0025] Figure 4 A structural block diagram of a robot-based object grasping device provided in an embodiment of this application;
[0026] Figure 5 A structural block diagram of a robot-based object grasping device provided in an embodiment of this application;
[0027] Figure 6 A structural block diagram of an electronic device provided in an embodiment of this application;
[0028] Figure 7 This is a structural block diagram of an electronic device provided in an embodiment of this application.
[0029] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0030] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0031] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.
[0032] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0033] In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.
[0034] It should be noted that, due to space limitations, this application specification does not exhaustively list all possible implementation methods. Those skilled in the art, after reading this application specification, should be able to deduce that, as long as the technical features do not contradict each other, any combination of technical features can constitute an optional implementation method. The following provides a detailed description of each embodiment.
[0035] Robot vision is a rapidly developing branch of artificial intelligence. Simply put, robot vision uses robots to replace human eyes for measurement and judgment, enabling computers to perform human visual functions. Robot vision modules can be used to understand the shape, size, distance from the observation point, texture, and motion characteristics of observed objects. The vision module converts the observed object into an image signal through an image acquisition device, which is then processed by a dedicated image processing system to obtain the object's shape and position information.
[0036] For robots, their vision module perceives the environment and identifies and locates objects through visual sensors. The quality of the vision module directly determines the accuracy of the robot's subsequent grasping and placement. Benefiting from the continuous improvement of computing power and the rapid development of sensor imaging capabilities, the technology for grasping single-target objects on a planar surface based on two-dimensional images in structured or semi-structured environments has matured and yielded rich research results. However, for three-dimensional objects in complex real-world environments, using only two-dimensional information to identify three-dimensional target objects inevitably leads to information loss, thus affecting the robot's grasping operation.
[0037] In robot grasping and placement, information loss is inevitable when dealing with 3D objects in confined and cluttered real-world environments due to factors such as partial occlusion, insufficient lighting, and variations in object shape and size. This inevitably affects the robot's grasping operation. Common robot 3D vision algorithms in the industry use deep learning to directly process point cloud images, determining the object's 6D pose through intensive computation. This method is computationally intensive, resulting in low efficiency and accuracy, thus impacting the efficiency and precision of object grasping.
[0038] This application provides a robot-based object grasping method, device, robot, and storage medium, which aims to solve the above-mentioned technical problems in the prior art.
[0039] It should be noted that the robot-based object grasping method, apparatus, robot, and storage medium disclosed herein can be used in the field of artificial intelligence, or in any field other than artificial intelligence. The application areas of the robot-based object grasping method, apparatus, robot, and storage medium disclosed herein are not limited.
[0040] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.
[0041] Figure 1 This is a flowchart illustrating a robot-based object grasping method according to an embodiment of this application. The method is applied to a robot equipped with a gripper and can be executed by a robot-based object grasping device. Figure 1 As shown, the method includes the following steps:
[0042] S101. Obtain a point cloud image of the scene where the target object is located, wherein the point cloud image includes point cloud data of the scene where the target object is located; and perform automatic bounding selection on the target object in the point cloud image to obtain the target bounding box of the target object; wherein the target bounding box contains the point cloud data of the target object.
[0043] For example, a point cloud image within a preset range can be acquired using a preset data acquisition device, and the preset range contains the object to be captured. The object to be captured is identified as the target object, that is, a point cloud image of the scene where the target object is located is acquired. For example, the scene where the target object is located can be scanned using a LiDAR to obtain a point cloud image of the scene where the target object is located, and the point cloud image includes the point cloud data of the scene where the target object is located.
[0044] When grasping an object, the object to be grasped needs to be determined in advance, for example, a rectangular box. The point cloud image contains the point cloud data of the target object. Since the shape and other information of the target object are already known, the point cloud data corresponding to the target object can be found from the point cloud image. For example, based on a preset detection algorithm, point cloud data clustered into a rectangular shape can be identified from the point cloud image and used as the point cloud data of the target object. After determining the point cloud data of the target object in the point cloud image, automatic bounding box selection is performed on the target object in the point cloud image. For example, a rectangular bounding box can be used to enclose the target object. The bounding box enclosing the target object is defined as the target bounding box of the target object; the target bounding box contains the boundary of the target object, that is, the target bounding box contains the point cloud data of the target object.
[0045] In this embodiment, the detection algorithm for identifying target objects in point cloud images is not specifically limited. Depending on the accuracy of the detection algorithm, the bounding box may include other objects near the target object. That is, the bounding box must at least include the complete target object.
[0046] In this embodiment, the automatic selection of target objects in a point cloud image to obtain the target bounding box of the target object includes: acquiring the appearance information of the target object; and, based on the appearance information of the target object and a preset neural network model, performing selection of target objects in the point cloud image to obtain the target bounding box of the target object; wherein, the neural network model is used to identify the target object in the point cloud image based on the appearance information.
[0047] Specifically, a neural network model is pre-trained and constructed, which can be built using deep learning algorithms. This neural network model can then be used to detect and identify target objects in point cloud images. The neural network model can include multiple network layers, such as convolutional layers, pooling layers, and fully connected layers, which can be used for feature extraction from point cloud images. In this embodiment, the specific structure of the neural network model is not limited.
[0048] The target object to be captured is predetermined, thus its appearance information can be obtained beforehand. Appearance information can include the object's name, size, and shape. For example, if the object's name is determined to be a soccer ball, then its shape can be determined to be spherical. The point cloud image and the target object's appearance information are input into a neural network model, which extracts features from the point cloud image. Regions corresponding to the target object's appearance information are found in the point cloud image; for example, regions where point cloud data clusters into spherical shapes are located. A bounding box is then created around the target object, resulting in a bounding box that encloses it.
[0049] In this embodiment, a neural network model based on a deep learning algorithm is used to detect target objects in space. For a given target object, the neural network model can return the bounding box with the highest probability, i.e., the bounding box most likely to enclose the entire target object. The bounding box can be a rectangular region on a point cloud image defined by (x, y, w, h), where (x, y) are the pixel coordinates of the lower left corner of the bounding box, and (w, h) represent the width and height of the bounding box. Based on the bounding box returned by the neural network model, the position of the target object in the point cloud image can be determined. Depending on the accuracy of the deep learning algorithm, the bounding box can be larger, including other objects near the target object.
[0050] The advantage of this setup is that it uses a neural network model to determine the target bounding box, enabling automatic identification of the target object, reducing manual identification operations, and improving the efficiency of object grasping.
[0051] S102. Based on the point cloud data in the target bounding box, perform surface extraction processing on the target object in the target bounding box to obtain the data set to be processed of the target object; wherein, the data set to be processed includes the point cloud data of the surface of the target object.
[0052] For example, a target object may include one or more surfaces; for instance, a spherical object may have one sphere, and a cuboid object may have six planes. A target object may display one or more of its own surfaces in a point cloud image; for example, a cuboid box may display three planes in a point cloud image.
[0053] The point cloud data within the target bounding box is acquired, and the surface normal attributes of each pixel are determined, i.e., the normals are identified. Surface normal attributes can include information such as the slope of the normal. Based on the surface normal attributes of each pixel, surface extraction is performed on the target object; that is, the various surfaces displayed by the target object within the target bounding box are determined. For example, pixels with consistent normal slopes can be identified as pixels on a single surface, thereby segmenting the target object into multiple surfaces. In this embodiment, a preset surface extraction algorithm can be used to determine the various surfaces of the target object. Different surfaces of the target object can be segmented into independent parts, and bounding boxes can be selected for each independent surface to obtain the bounding box of each surface.
[0054] The point cloud data of each surface is determined and used as the data set to be processed for the surface of the target object. This data set represents the point cloud data of the target object's surface. For example, if the target bounding box shows three planes of a cuboid box, then these three planes can be extracted, and the data set to be processed for these three planes can be determined. By segmenting the surface of the target object, point cloud data that does not belong to the target object can be removed from the target bounding box, improving the accuracy of object grasping.
[0055] S103. Determine the attribute information of the target object based on the data set to be processed for the target object; wherein, the attribute information is used to represent the size information and position information of the target object.
[0056] For example, attribute information can represent 3D attributes such as the size and position of an object. For instance, attribute information can include the object's shape, length, width, and height. The object's shape can be cylindrical, box-shaped, spherical, etc. The attribute information of the target object can be determined based on the dataset to be processed. For example, the shape of a surface can be determined from the dataset, and the various surfaces can be fitted to determine the overall shape of the target object. For example, if the surface shape is a curved surface, the target object can be determined to be spherical; if the surface shape has both curved and planar surfaces, the target object can be determined to be cylindrical, etc.
[0057] Based on the range of the dataset to be processed, specific size information such as the length, width, and height of the target object can also be determined. For example, the size of the surface can be obtained based on the size of the bounding box of the dataset to be processed, and then the size of the target object can be determined. In this embodiment, attribute information such as the centroid of the target object can also be determined; for example, the coordinate position of the centroid can be determined.
[0058] In this embodiment, determining the attribute information of the target object based on the data set to be processed of the target object includes: performing surface fitting processing on the target object based on a preset empirical geometric rule algorithm according to the data set to be processed of the target object to obtain the fitted target object; and analyzing and processing the fitted target object according to a preset principal component analysis algorithm to obtain the attribute information of the target object.
[0059] Specifically, after obtaining the data set of the target object to be processed, the surface of the target object is reassembled into a complete target object, achieving precise grasping of the target object. Empirical geometric rules can be pre-set to determine the shape of the target object. For the data set of the target object to be processed, the empirical geometric rules can be applied to fit the segmented surface of the target object, obtaining a fitted target object and determining the shape closest to the target object.
[0060] After obtaining the complete target object, its spatial information, i.e., 3D attribute information, is acquired. A pre-defined principal component analysis algorithm can be used to process the fitted point cloud data of the target object, calculating its principal axes, secondary axes, and normal axes, thereby obtaining the target object's attribute information.
[0061] The advantage of this setup is that after segmentation, only the point cloud data of the target object remains. 3D attribute information is obtained by processing the individual point clouds of the object. Principal component analysis (PCA) and empirical geometric rules can be used to fit the point cloud of the segmented target object surface into a whole. PCA is used to calculate the principal axis, secondary axis, and normal axis of the target object, obtaining the object's length, width, height, centroid, etc. This achieves accurate identification of the target object and avoids interference from surrounding objects in the grasping process. This embodiment does not require any computationally intensive training phase to calculate the object pose, facilitating real-time execution of the method. Multiple surfaces are created using surface normals and their directions, and then combined using empirical geometric rules to identify shapes, effectively detecting the grasping posture of box-shaped objects. Compared to other shapes such as spheres or cylinders, the recognition of box-shaped objects is considered a more difficult problem; this embodiment effectively improves object recognition accuracy and grasping accuracy.
[0062] S104. Based on the attribute information of the target object, control the robot's gripper to grasp the target object.
[0063] For example, after determining the attribute information of the target object, that is, after determining the position and orientation of the target object, the robot can control its gripper to move to the position of the target object and grasp it. When grasping, the gripper can perform a one-dimensional search perpendicular to or along the principal axis of the target object and apply geometric constraints to the gripper to determine the effective grasping area, thereby transforming the three-dimensional 6D pose grasping into one-dimensional planar grasping and improving grasping efficiency.
[0064] In this embodiment, the gripping method can be different for objects of different shapes. For example, for cylindrical and box-shaped objects, gripping can be performed vertically or along the main axis; for spherical objects, gripping can be performed by rotating a two-finger gripper around the surface normal and continuing to grip along the circumference.
[0065] In this embodiment, the attribute information includes the length, width, height, centroid coordinates, and the transfer matrix information of the target object in the world coordinate system. Controlling the robot's gripper to grasp the target object based on its attribute information includes: determining the movement path of the robot's gripper based on the length, width, height, centroid coordinates, and the transfer matrix information of the target object in the world coordinate system; and controlling the robot's gripper to move to the target object and grasp it based on the movement path.
[0066] Specifically, attribute information can include the object's length, width, height, centroid coordinates, and the transition matrix in the world coordinate system, which can be used to represent the coordinate system of the point cloud image. Using principal component analysis (PCA), the object's length, width, height, centroid coordinates, and the transition matrix in the world coordinate system can be calculated.
[0067] When grasping a target object, the robot's gripper's movement path can be planned first. Based on the target object's length, width, height, centroid coordinates, and the transition matrix in the world coordinate system, the robot's gripper is guided to move in the direction opposite to the normal to the target object's centroid, approaching the target object. Following the movement path, the gripper is controlled to reach the target object, close, and grasp the target object. Figure 2 This is a diagram illustrating the grasping action of the gripper. Figure 2 The gripper moves to the right to grab the target object.
[0068] The advantage of this setup is that, in real-world narrow and cluttered environments, this embodiment transforms the problem of estimating the 6D pose of an object into a one-dimensional search problem through scalar projection, and utilizes the geometric characteristics of a two-finger gripper to improve the efficiency of object grasping.
[0069] This application provides a robot-based object grasping method. It acquires a point cloud image of the scene containing the target object and obtains a bounding box containing the target object from the point cloud image. That is, the target object in the point cloud image is initially identified. Surface extraction is performed on the target object within the bounding box to obtain a set of data to be processed. Based on the set of data to be processed, the size and position of the target object are determined, thereby controlling the robot's gripper to grasp the target object. This solves the problems of high computational load and low efficiency caused by the need for intensive computation to determine the 6D pose of the object in existing technologies. By determining each surface of the target object, the boundary of the target object can be accurately determined, improving object recognition accuracy, and thus improving object grasping accuracy and efficiency.
[0070] Figure 3 This is a flowchart illustrating a robot-based object grasping method provided in this application embodiment. This embodiment is an optional embodiment based on the above embodiment.
[0071] In this embodiment, the surface extraction processing of the target object in the target box is performed based on the point cloud data in the target box to obtain the data set to be processed of the target object. This can be further refined as follows: the surface of the target object in the target box is segmented based on the point cloud data in the target box to obtain the initial data set of the target object; wherein, the initial data set includes the point cloud data of the surface of the target object and the point cloud data of the background of the target object; the initial data set of the target object is filtered based on a preset Gaussian mixture model to obtain the data set to be processed of the target object.
[0072] like Figure 3 The method includes the following steps:
[0073] S301. Obtain a point cloud image of the scene where the target object is located, wherein the point cloud image includes point cloud data of the scene where the target object is located; and perform automatic bounding box selection on the target object in the point cloud image to obtain the target bounding box of the target object; wherein the target bounding box contains the point cloud data of the target object.
[0074] For example, this step can refer to step S101 above, and will not be repeated here.
[0075] S302. Based on the point cloud data in the target box, the surface of the target object in the target box is segmented to obtain the initial data set of the target object; wherein, the initial data set includes the point cloud data of the surface of the target object and the point cloud data of the background of the target object.
[0076] For example, the target bounding box includes at least one surface of the target object. Point cloud data within the target bounding box is determined, and the target object is segmented into individual surfaces. The point cloud data on these segmented surfaces is then used as the initial data set for each surface. This initial data set may contain point cloud data that does not belong to a surface, such as point cloud data of the target object's background. For instance, when segmenting the target object's surfaces, point cloud data from objects other than the target object within the target bounding box may be assigned to the target object's surfaces.
[0077] The surface of the target object can be divided according to its shape. For example, if the target object is a cube, point cloud data clustered into squares can be retrieved from the target bounding box as the initial data set for the target object. A preset image segmentation algorithm can be used to segment the surface; however, this embodiment does not specify a particular image segmentation algorithm.
[0078] In this embodiment, based on the point cloud data in the target box, the surface of the target object in the target box is segmented to obtain the initial data set of the target object, including: determining the coordinate position, width, and height of the target box; based on the coordinate position, width, and height of the target box, the surface of the target object is segmented according to a preset region growing algorithm to obtain the initial data set of the target object.
[0079] Specifically, a region growing algorithm is pre-set. Region growing is an algorithm used for image segmentation. The point cloud data in the target bounding box (x, y, w, h) is used as the input of the algorithm. The region growing algorithm is used to determine the coordinate position, width, and height of the target bounding box, and the point cloud data within the target bounding box is segmented to divide the different surfaces of the target object into independent parts.
[0080] For example, a region growing algorithm can obtain each pixel in the bounding box based on its coordinates, width, and height. The normal vector of each pixel is determined, yielding its surface normal attribute. Surface extraction is then performed based on the surface normal attribute of each pixel, and the bounding box of each surface is determined. For example, for the i-th surface, the bounding box can be represented as (x... i y i w i h i ).
[0081] The advantage of this setup is that, by using a region growing algorithm, the surface of the target object can be quickly segmented, improving the efficiency of object grasping.
[0082] S303. Based on the preset Gaussian mixture model, perform point cloud data filtering on the initial data set of the target object to obtain the data set of the target object to be processed.
[0083] For example, due to over-segmentation or under-segmentation, region growing algorithms may not necessarily yield ideal bounding boxes for the target object's surface. For instance, the initial dataset returned by the region growing algorithm might contain point cloud data from background images. A pre-created Gaussian mixture model can be used to accurately segment the target object from the background. That is, the initial dataset of the target object is filtered using a pre-defined Gaussian mixture model, deleting point cloud data that does not belong to the target object's surface and retaining the point cloud data that does belong to the surface, resulting in the dataset to be processed for the target object. For example, the color similarity between pixels in the initial dataset can be determined, identifying pixels with high similarity as belonging to the target object and pixels with low similarity as belonging to the background.
[0084] In this embodiment, point cloud data filtering processing is performed on the initial dataset of the target object according to a preset Gaussian mixture model to obtain the dataset to be processed for the target object. This includes: determining the color information and curvature information of the point cloud data in the initial dataset; determining the target probability of the point cloud data in the initial dataset based on the color information and curvature information of the point cloud data in the initial dataset and the preset Gaussian mixture model; wherein, the target probability is the probability that the point cloud data is retained in the initial dataset; and filtering processing is performed on the point cloud data in the initial dataset according to the target probability of the point cloud data in the initial dataset to obtain the dataset to be processed for the target object.
[0085] Specifically, a Gaussian mixture model is created based on the color and depth curvature information of the RGBD (Red, Green, Blue, Depth) point cloud in the initial dataset of the target object. This model removes the point cloud from the non-target object portion, thus segmenting the target object from the background.
[0086] RGBD data is converted to HSV (Hue Saturation Value) data, where the hue and saturation components of the HSV color space are used to replace RGB color information. Lightness (V) is discarded to eliminate lighting effects. A Gaussian mixture model is used to calculate probabilities. Based on hue, saturation, and depth curvature, the probability of each point cloud data point in the initial dataset being retained is determined as the target probability. For example, if the color information of a point cloud data point is completely different from that of other point cloud data points in the initial dataset, then the target probability of that point cloud data point can be considered low. The target probability is the posterior probability. Using the Gaussian mixture model, the prior probability and conditional probability can be determined, and the posterior probability can be calculated based on the prior and conditional probabilities.
[0087] Based on the target probability, the initial data set of the target object is filtered by point cloud data. Point cloud data with low target probability can be deleted, while point cloud data with high target probability can be retained, thus obtaining the data set of the target object to be processed.
[0088] The advantage of this setup is that by combining color and depth curvature information to create a Gaussian mixture model, the target object can be segmented from a complex background without requiring intensive training to locate the object's 6D pose. This simplifies the 6D pose detection process, enabling near real-time object pose detection and resolving the problem of incorrect identification of target object boundary regions caused by noisy information, thus improving the efficiency and accuracy of object grasping.
[0089] In this embodiment, based on the color and curvature information of the point cloud data in the initial dataset and a preset Gaussian mixture model, the target probability of the point cloud data in the initial dataset is determined, including: based on the color and curvature information of the point cloud data in the initial dataset and a preset Gaussian mixture model, determining a first probability and a second probability of the point cloud data in the initial dataset; wherein, the first probability is the probability that the point cloud data belongs to the target object; the second probability is the probability that the point cloud data belongs to the background of the target object; and the target probability of the point cloud data in the initial dataset is determined based on the first probability and the second probability of the point cloud data in the initial dataset.
[0090] Specifically, point cloud data can be divided into two categories: those belonging to the target object and those belonging to the background. The Gaussian mixture model can calculate the probability that a point cloud data point belongs to the target object (first probability) and the probability that it belongs to the background (second probability) based on the color and curvature information of each point cloud data point—that is, based on hue, saturation, and depth curvature. In other words, this embodiment uses two mixture models: one to estimate the probability that a pixel belongs to the target object, and the other to estimate the probability that a pixel belongs to the background.
[0091] Based on the first and second probabilities, a decision is made regarding whether to retain the point cloud data. For example, the first and second probabilities can be compared; if the first probability is greater than the second, the point cloud data is retained; if the first probability is less than the second, the point cloud data is deleted. Alternatively, a final target probability can be determined based on the first and second probabilities; for instance, the larger of the first and second probabilities can be used as the target probability. The retention of the point cloud data is then determined based on the magnitude of the target probability. Probability calculations are performed on all pixels in the initial dataset to avoid data omissions and improve the accuracy of object recognition.
[0092] The advantage of this setup is that by calculating the first probability and the second probability, the set of data to be processed can be determined more accurately, avoiding errors caused by only calculating the first probability or the second probability, improving the accuracy of object recognition, and thus improving the accuracy of object grasping.
[0093] In this embodiment, the target probability of the point cloud data in the initial dataset is:
[0094] Where P is the target probability, P1 is the first probability, and P2 is the second probability.
[0095] Specifically, the target probability can be calculated by determining the ratio of the first probability to the sum of the first and second probabilities, where the sum of the first and second probabilities is not necessarily 1.
[0096] In this embodiment, the point cloud data in the initial data set is filtered according to the target probability of the point cloud data in the initial data set to obtain the data set to be processed for the target object. This includes: for each point cloud data in the initial data set, if it is determined that the target probability of the point cloud data is less than a preset probability threshold, the point cloud data is filtered out from the initial data set to obtain the data set to be processed for the target object.
[0097] Specifically, a probability threshold is preset. After obtaining the target probability, the target probability is compared with the probability threshold. For example, the probability threshold can be set to 0.5. If the target probability of a point cloud data in the initial dataset is less than the preset probability threshold, the point cloud data corresponding to that target probability is removed from the initial dataset, that is, it is determined that the point cloud data does not belong to the surface of the corresponding target object. If the target probability of point cloud data in the initial dataset is equal to or greater than the preset probability threshold, the point cloud data corresponding to that target probability is retained, that is, the point cloud data belongs to the target object.
[0098] Identify the point cloud data retained from the initial dataset of the target object. This retained set of point cloud data is then defined as the dataset to be processed for the target object.
[0099] The advantage of this setting is that if the target probability of the point cloud data is equal to or greater than the preset probability threshold, the point cloud data is considered to be part of the target object; otherwise, it is considered to be part of the background. The judgment process is simple and fast, improving the efficiency of point cloud data filtering.
[0100] S304. Based on the data set to be processed for the target object, determine the attribute information of the target object; wherein, the attribute information is used to represent the size information and position information of the target object.
[0101] For example, this step can refer to step S103 above, and will not be repeated here.
[0102] S305. Based on the attribute information of the target object, control the robot's gripper to grasp the target object.
[0103] For example, this step can refer to step S104 above, and will not be repeated here.
[0104] This application provides a robot-based object grasping method. It acquires a point cloud image of the scene containing the target object and obtains a bounding box containing the target object from the point cloud image. That is, the target object in the point cloud image is initially identified. Surface extraction is performed on the target object within the bounding box to obtain a set of data to be processed. Based on the set of data to be processed, the size and position of the target object are determined, thereby controlling the robot's gripper to grasp the target object. This solves the problems of high computational load and low efficiency caused by the need for intensive computation to determine the 6D pose of the object in existing technologies. By determining each surface of the target object, the boundary of the target object can be accurately determined, improving object recognition accuracy, and thus improving object grasping accuracy and efficiency.
[0105] Figure 4 This is a structural block diagram of a robot-based object grasping device provided in an embodiment of this application. The device is applied to a robot equipped with a gripper. For ease of explanation, only the parts relevant to the embodiments of this disclosure are shown. (Refer to...) Figure 4 The device includes: an object selection module 401, a surface extraction module 402, an information determination module 403, and an object grasping module 404.
[0106] The object selection module 401 is used to acquire a point cloud image of the scene where the target object is located, wherein the point cloud image includes point cloud data of the scene where the target object is located; and to perform automatic selection processing on the target object in the point cloud image to obtain a target box of the target object; wherein the target box contains the point cloud data of the target object;
[0107] The surface extraction module 402 is used to perform surface extraction processing on the target object in the target box based on the point cloud data in the target box, to obtain a data set to be processed of the target object; wherein, the data set to be processed includes point cloud data of the surface of the target object;
[0108] The information determination module 403 is used to determine the attribute information of the target object based on the data set to be processed of the target object; wherein the attribute information is used to represent the size information and position information of the target object;
[0109] The object grasping module 404 is used to control the robot's gripper to grasp the target object based on the attribute information of the target object.
[0110] Figure 5 This application provides a structural block diagram of a robot-based object grasping device, in which... Figure 4 Based on the illustrated embodiments, as Figure 5 As shown, the surface extraction module 402 includes a set determination unit 4021 and a filtering unit 4022.
[0111] The set determination unit 4021 is used to segment the surface of the target object in the target box according to the point cloud data in the target box to obtain an initial data set of the target object; wherein, the initial data set includes point cloud data of the surface of the target object and point cloud data of the background of the target object;
[0112] The filtering unit 4022 is used to perform point cloud data filtering processing on the initial data set of the target object according to the preset Gaussian mixture model to obtain the data set to be processed of the target object.
[0113] In one example, set determination unit 4021 is specifically used for:
[0114] Determine the coordinate position, width, and height of the target bounding box;
[0115] Based on the coordinates, width, and height of the target bounding box, a preset region growing algorithm is used to perform surface segmentation of the target object to obtain an initial data set of the target object.
[0116] In one example, filter unit 4022 includes:
[0117] The first determining subunit is used to determine the color information and curvature information of the point cloud data in the initial data set;
[0118] The second determining subunit is used to determine the target probability of the point cloud data in the initial data set based on the color information and curvature information of the point cloud data in the initial data set, and on a preset Gaussian mixture model; wherein, the target probability is the probability of the point cloud data being retained in the initial data set.
[0119] The third determining subunit is used to filter the point cloud data in the initial data set according to the target probability of the point cloud data in the initial data set, so as to obtain the data set to be processed for the target object.
[0120] In one example, the second determined subunit is specifically used for:
[0121] Based on the color and curvature information of the point cloud data in the initial dataset, and based on a preset Gaussian mixture model, a first probability and a second probability of the point cloud data in the initial dataset are determined; wherein, the first probability is the probability that the point cloud data belongs to the target object; and the second probability is the probability that the point cloud data belongs to the background of the target object.
[0122] The target probability of the point cloud data in the initial data set is determined based on the first probability and the second probability of the point cloud data in the initial data set.
[0123] In one example, the target probability of the point cloud data in the initial dataset is:
[0124] Wherein, P is the target probability, P1 is the first probability, and P2 is the second probability.
[0125] In one example, the third determining subunit is specifically used for:
[0126] For each point cloud data in the initial data set, if the target probability of the point cloud data is determined to be less than a preset probability threshold, the point cloud data is removed from the initial data set to obtain the data set to be processed for the target object.
[0127] In one example, the information determination module 403 is specifically used for:
[0128] Based on the data set to be processed of the target object, and using a preset empirical geometric rule algorithm, the surface of the target object is fitted to obtain the fitted target object.
[0129] The fitted target object is analyzed and processed according to a preset principal component analysis algorithm to obtain the attribute information of the target object.
[0130] In one example, the attribute information includes the target object's length, width, height, centroid coordinates, and the target object's transition matrix in the world coordinate system;
[0131] Object grasping module 404 is specifically used for:
[0132] Based on the attribute information of the target object, control the robot's gripper to grasp the target object, including:
[0133] Based on the length, width, height, centroid coordinates, and the transfer matrix of the target object in the world coordinate system, the movement path of the robot's gripper is determined.
[0134] According to the movement path, control the robot's gripper to move to the target object and grasp the target object.
[0135] In one example, object selection module 401 is specifically used for:
[0136] Obtain the appearance information of the target object;
[0137] Based on the appearance information of the target object and a preset neural network model, the target object is selected in the point cloud image to obtain the target bounding box of the target object; wherein, the neural network model is used to identify the target object in the point cloud image based on the appearance information.
[0138] Figure 6 This is a structural block diagram of an electronic device provided in an embodiment of this application. The electronic device can be a robot. Figure 6 As shown, the electronic device includes: a memory 61 and a processor 62; the memory 61 is a memory for storing executable instructions of the processor 62.
[0139] The processor 62 is configured to perform the methods provided in the above embodiments.
[0140] The electronic device also includes a receiver 63 and a transmitter 64. The receiver 63 is used to receive instructions and data sent by other devices, and the transmitter 64 is used to send instructions and data to external devices.
[0141] Figure 7 This is a structural block diagram of an electronic device according to an exemplary embodiment. The device may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, or robot, etc.
[0142] Device 700 may include one or more of the following components: processing component 702, memory 704, power supply component 706, multimedia component 708, audio component 710, input / output (I / O) interface 712, sensor component 714, and communication component 716.
[0143] Processing component 702 typically controls the overall operation of device 700, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 702 may include one or more processors 720 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 702 may include one or more modules to facilitate interaction between processing component 702 and other components. For example, processing component 702 may include a multimedia module to facilitate interaction between multimedia component 708 and processing component 702.
[0144] Memory 704 is configured to store various types of data to support the operation of device 700. Examples of this data include instructions for any application or method operating on device 700, contact data, phonebook data, messages, pictures, videos, etc. Memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0145] Power supply component 706 provides power to various components of device 700. Power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 700.
[0146] Multimedia component 708 includes a screen that provides an output interface between the device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 708 includes a front-facing camera and / or a rear-facing camera. When the device 700 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
[0147] Audio component 710 is configured to output and / or input audio signals. For example, audio component 710 includes a microphone (MIC) configured to receive external audio signals when device 700 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 704 or transmitted via communication component 716. In some embodiments, audio component 710 also includes a speaker for outputting audio signals.
[0148] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.
[0149] Sensor assembly 714 includes one or more sensors for providing state assessments of various aspects of device 700. For example, sensor assembly 714 may detect the on / off state of device 700, the relative positioning of components such as the display and keypad of device 700, changes in the position of device 700 or a component of device 700, the presence or absence of user contact with device 700, the orientation or acceleration / deceleration of device 700, and temperature changes of device 700. Sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 714 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 714 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.
[0150] Communication component 716 is configured to facilitate wired or wireless communication between device 700 and other devices. Device 700 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 716 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 716 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0151] In an exemplary embodiment, device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.
[0152] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, which can be executed by a processor 720 of device 700 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.
[0153] A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of a terminal device, enable the terminal device to perform the aforementioned robot-based object grasping method of the terminal device.
[0154] This application also discloses a computer program product, including a computer program that, when executed by a processor, implements the method described in this embodiment.
[0155] Various embodiments of the systems and technologies described above in this application can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0156] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or electronic device.
[0157] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0158] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0159] The systems and technologies described herein can be implemented in computing systems that include back-end components (e.g., as data electronic devices), or computing systems that include middleware components (e.g., application electronic devices), or computing systems that include front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.
[0160] Computer systems can include client and electronic devices. Clients and electronic devices are generally geographically separated and typically interact via communication networks. The client-electronic device relationship is created by computer programs running on the respective computers and having a client-electronic device relationship with each other. The electronic device can be a cloud electronic device, also known as a cloud computing electronic device or cloud host, a host product within the cloud computing service system, addressing the shortcomings of traditional physical hosts and VPS services ("Virtual Private Server," or simply "VPS") in terms of management difficulty and weak business scalability. The electronic device can also be an electronic device in a distributed system or an electronic device incorporating blockchain technology. It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application is achieved, and this is not limited herein.
[0161] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.
[0162] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A robot-based object grasping method, characterized in that, The method is applied to a robot equipped with a gripper; the method includes: A point cloud image of the scene where the target object is located is obtained, wherein the point cloud image includes point cloud data of the scene where the target object is located; and the target object is automatically selected in the point cloud image to obtain a target bounding box of the target object; wherein the target bounding box contains the point cloud data of the target object; Based on the point cloud data in the target bounding box, the surface of the target object in the target bounding box is segmented to obtain an initial data set of the target object; based on a preset Gaussian mixture model, the initial data set of the target object is filtered to obtain a data set to be processed of the target object; wherein, the initial data set includes point cloud data of the surface of the target object and point cloud data of the background of the target object, and the Gaussian mixture model determines the target probability of each point cloud data being retained in the data set to be processed based on the color information and curvature information of the point cloud data in the initial data set, and filters are performed according to the target probability; Based on the data set to be processed for the target object, a surface fitting process is performed on the target object according to a preset empirical geometric rule algorithm to obtain a fitted target object; the fitted target object is then analyzed and processed according to a preset principal component analysis algorithm to obtain the attribute information of the target object; the attribute information includes the length, width, height, centroid coordinates of the target object, and the transition matrix information of the target object in the world coordinate system; Based on the length, width, height, centroid coordinates, and the transition matrix of the target object in the world coordinate system, the movement path of the robot's gripper is determined; based on the movement path, the robot's gripper is controlled to move to the target object and grasp the target object.
2. The method according to claim 1, characterized in that, Based on the point cloud data in the target bounding box, the surface of the target object in the target bounding box is segmented to obtain an initial data set of the target object, including: Determine the coordinate position, width, and height of the target bounding box; Based on the coordinates, width, and height of the target bounding box, a preset region growing algorithm is used to perform surface segmentation of the target object to obtain an initial data set of the target object.
3. The method according to claim 1, characterized in that, Based on a preset Gaussian mixture model, the initial dataset of the target object is subjected to point cloud data filtering processing to obtain the dataset of the target object to be processed, including: Determine the color and curvature information of the point cloud data in the initial dataset; Based on the color and curvature information of the point cloud data in the initial dataset, and using a preset Gaussian mixture model, the target probability of the point cloud data in the initial dataset is determined; wherein, the target probability is the probability of the point cloud data being retained in the initial dataset. Based on the target probability of the point cloud data in the initial data set, the point cloud data in the initial data set is filtered to obtain the data set to be processed for the target object.
4. The method according to claim 3, characterized in that, Based on the color and curvature information of the point cloud data in the initial dataset, and using a preset Gaussian mixture model, the target probability of the point cloud data in the initial dataset is determined, including: Based on the color and curvature information of the point cloud data in the initial dataset, and based on a preset Gaussian mixture model, a first probability and a second probability of the point cloud data in the initial dataset are determined; wherein, the first probability is the probability that the point cloud data belongs to the target object; and the second probability is the probability that the point cloud data belongs to the background of the target object. The target probability of the point cloud data in the initial data set is determined based on the first probability and the second probability of the point cloud data in the initial data set.
5. The method according to claim 4, characterized in that, The target probability of the point cloud data in the initial dataset is: ; Where P is the target probability. For the first probability, This is the second probability.
6. The method according to claim 3, characterized in that, Based on the target probability of the point cloud data in the initial dataset, the point cloud data in the initial dataset is filtered to obtain the data set to be processed for the target object, including: For each point cloud data in the initial data set, if the target probability of the point cloud data is determined to be less than a preset probability threshold, the point cloud data is removed from the initial data set to obtain the data set to be processed for the target object.
7. The method according to any one of claims 1-6, characterized in that, Automatic bounding box selection is performed on the target object in the point cloud image to obtain the target bounding box of the target object, including: Obtain the appearance information of the target object; Based on the appearance information of the target object and a preset neural network model, the target object is selected in the point cloud image to obtain the target bounding box of the target object; wherein, the neural network model is used to identify the target object in the point cloud image based on the appearance information.
8. A robot-based object grasping device, characterized in that, The device is applied to a robot equipped with a gripper; the device includes: An object bounding box module is used to acquire a point cloud image of the scene where a target object is located, wherein the point cloud image includes point cloud data of the scene where the target object is located; and to perform automatic bounding box processing on the target object in the point cloud image to obtain a target bounding box of the target object; wherein the target bounding box contains the point cloud data of the target object; The surface extraction module is used to segment the surface of the target object in the target box based on the point cloud data in the target box to obtain an initial data set of the target object; and to perform point cloud data filtering on the initial data set of the target object according to a preset Gaussian mixture model to obtain a data set to be processed of the target object; wherein, the initial data set includes point cloud data of the surface of the target object and point cloud data of the background of the target object, and the Gaussian mixture model determines the target probability of each point cloud data being retained in the data set to be processed based on the color information and curvature information of the point cloud data in the initial data set, and filters according to the target probability; The information determination module is used to perform surface fitting processing on the target object based on the target object's data set to be processed, using a preset empirical geometric rule algorithm, to obtain a fitted target object; and to analyze the fitted target object according to a preset principal component analysis algorithm to obtain the target object's attribute information; the attribute information includes the target object's length, width, height, centroid coordinates, and the target object's transition matrix information in the world coordinate system; The object grasping module is used to determine the movement path of the robot's gripper based on the length, width, height, centroid coordinates, and the transition matrix information of the target object in the world coordinate system; and to control the robot's gripper to move to the target object according to the movement path and grasp the target object.
9. A robot, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the robot-based object grasping method as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the robot-based object grasping method as described in any one of claims 1-7.
11. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the robot-based object grasping method according to any one of claims 1-7.