Stacked object grabbing method and device, robot and storage medium

Abstract

The invention relates to the technical field of robots, and discloses a stacked object grabbing method and device, a robot and a storage medium, and the method comprises the steps: obtaining a to-be-recognized image, shot by a shooting device, of a stacked object; the stacked objects comprise at least two objects placed in a stacked mode; extracting three-dimensional point cloud data of the stacked object from the image to be recognized; based on the three-dimensional point cloud data of the stacked objects, identifying a top-layer object on the uppermost layer in the stacked objects; identifying the mass center position of the top-layer object according to the three-dimensional point cloud data of the top-layer object; and the grabbing track of the robot is generated according to the mass center position, a grabbing component is controlled to grab the top layer object according to the grabbing track, the invalid grabbing effect of grabbing objects placed disorderly can be achieved, and the grabbing efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of robotics technology, specifically to methods, devices, robots, and storage media for grasping stacked objects. Background Technology

[0002] In industrial production and logistics, various parts often require grasping operations. Existing technologies rely on the orderly arrangement of objects. An orderly arrangement means that objects need to be arranged according to certain rules. For example, goods in a supermarket are arranged according to the shelf layout. Because orderly arrangements have certain rules, the grasping system can successfully grasp objects by moving according to those rules. However, in some special scenarios, the placement of objects is often random and disorderly. Examples include building materials on a construction site or unsorted waste in a recycling center. In these scenarios, due to the lack of rules in the object placement, existing grasping systems often cannot handle the situation, resulting in poor grasping efficiency. Summary of the Invention

[0003] This application discloses a method, device, robot, and storage medium for grasping stacked objects. It can automatically identify the centroid position of the top object in the stack and grasp the top object based on this, achieving the effect of disordered grasping and improving grasping efficiency.

[0004] This application discloses a method for grasping stacked objects, the method comprising: Acquire images of stacked objects to be identified, captured by an imaging device; the stacked objects include at least two objects placed in a stacked manner; Extract the three-dimensional point cloud data of the stacked objects from the image to be identified; Identify the topmost object in the stacked objects based on the three-dimensional point cloud data of the stacked objects; The centroid position of the top-level object is identified based on the three-dimensional point cloud data of the top-level object; The robot generates a grasping trajectory based on the centroid position and controls the grasping component to grasp the top-level object according to the grasping trajectory.

[0005] As an optional implementation, identifying the topmost object in the stacked objects based on the 3D point cloud data of the stacked objects includes: The depth component is obtained from the three-dimensional point cloud data of the stacked objects; wherein the depth component represents the distance between data points in the three-dimensional point cloud data and the imaging device. Based on depth components, data points within a preset range from the shooting device are identified from the 3D point cloud data of the stacked objects as the top layer object; wherein, the depth components of the data points within the preset range are all less than the depth components of the data points outside the preset range.

[0006] As an optional implementation, identifying the centroid position of the top-level object based on the three-dimensional point cloud data of the top-level object includes: Extract the upper surface edge contour of the top object based on the three-dimensional point cloud data of the top object; The centroid position of the top-level object is calculated based on the data points included in the upper surface edge contour of the top-level object.

[0007] As an optional implementation, calculating the centroid position of the top-layer object based on data points included in the upper surface edge contour of the top-layer object includes: The coordinates of each data point in the upper surface edge contour of the top object are added together according to their corresponding coordinate components; the coordinate components include: horizontal axis component, vertical axis component and depth component; Divide the sum of the corresponding coordinate components by the total number of data points on the upper surface edge contour to obtain the coordinate value corresponding to each coordinate component as the centroid position of the top layer object.

[0008] As an optional implementation, the step of extracting the upper surface edge contour of the top-level object based on the three-dimensional point cloud data of the top-level object includes: Iterate through the points in the 3D point cloud data of the top-level object, and perform the following operations for each point: The currently visited point is used as the sampling point; A local neighborhood point set with radius r is established centered on the sampling point, and a spherical neighborhood of the sampling point is obtained based on the neighborhood point set; r is a positive integer greater than or equal to 1. Using the sphere neighborhood as sample data, the local tangent plane of the sampling point is fitted using the least squares method; The covariance matrix formed by the data points contained in the local tangent plane of the sampling point is subjected to eigenvalue decomposition, and the eigenvector corresponding to the smallest eigenvalue is taken as the normal vector of the sampling point. Based on the difference between the normal vectors of each data point in the 3D point cloud data of the top-level object, data points belonging to the upper surface edge contour of the top-level object are identified.

[0009] As an optional implementation, identifying data points belonging to the upper surface edge contour of the top-level object based on the difference between the normal vectors of each data point in the 3D point cloud data of the top-level object includes: Select a data point as the center point from the three-dimensional point cloud data of the top-level object; Calculate the angle between the normal vectors of the data points within the neighborhood of the center point and the normal vector of the center point; Compare the included angles in each direction with the smoothing threshold; When the directional angle is less than the smoothing threshold, the data points within the corresponding neighborhood range are added to the seed point set; Repeat the above steps until all points have been classified; The top surface edge contour of the top object is obtained from the set of seed points.

[0010] As an optional implementation, the method further includes: During the process of controlling the grasping component to grasp the top-level object according to the grasping trajectory, the image of the stacked object captured by the imaging device is used as a new image to be identified. When the grasping component fails to grasp the top object, a new centroid position of the top object is identified based on the new image to be recognized, and the grasping trajectory of the grasping component is adjusted according to the new centroid position.

[0011] This application discloses a gripping device for stacked objects, comprising: An acquisition module is used to acquire images of stacked objects to be identified captured by the imaging device; the stacked objects include at least two objects placed in a stacked manner; An extraction module is used to extract the three-dimensional point cloud data of the stacked objects from the image to be identified; The identification module is used to identify the topmost object in the stacked objects based on the three-dimensional point cloud data of the stacked objects; and to identify the centroid position of the topmost object based on the three-dimensional point cloud data of the topmost object. A generation module is used to generate the robot's grasping trajectory based on the centroid position; The control module is used to control the gripping component to grip the top-level object according to the gripping trajectory.

[0012] This application discloses a robot, wherein the electronic device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor implements any of the stacked object grasping methods disclosed in this application.

[0013] This application discloses a computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements any of the stacked object grasping methods disclosed in this application.

[0014] Compared with related technologies, the embodiments of this application have the following beneficial effects: In this embodiment, an image of a stacked object to be identified is captured by an imaging device, from which 3D point cloud data of the stacked object is identified. Then, based on the 3D point cloud data of the stacked object, the topmost object in the stack is identified, thereby identifying the centroid position of the top object based on its 3D point cloud data. A robot's grasping trajectory is then generated based on the centroid position, and the grasping component is controlled to grasp the top object according to the trajectory. Therefore, the robot can grasp disordered objects, improving grasping efficiency and flexibility. Simultaneously, it reduces human intervention in disordered grasping scenarios, lowering production costs. Furthermore, the identification of the centroid position allows the robot to grasp objects more stably, improving grasping accuracy and reducing the occurrence of misgrasping and damage to parts. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1A This is an example diagram of a shooting device disclosed in an embodiment of this application.

[0017] Figure 1B This is an example diagram of a robot working in an embodiment of this application.

[0018] Figure 2 This is a flowchart illustrating a method for grasping stacked objects disclosed in an embodiment of this application.

[0019] Figure 3 This is an example diagram of a stacked object disclosed in an embodiment of this application.

[0020] Figure 4 This is a flowchart illustrating a method for identifying the contour of an upper surface edge disclosed in an embodiment of this application.

[0021] Figure 5 This is a schematic diagram of a gripping device for stacked objects disclosed in an embodiment of this application.

[0022] Figure 6 This is a schematic diagram of the structure of a robot disclosed in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0025] This application discloses a method, device, robot, and storage medium for grasping stacked objects. It can automatically identify the centroid position of the top object in the stack and grasp the top object based on this, achieving the effect of disordered grasping and improving grasping efficiency.

[0026] Please see Figure 1A , Figure 1A This is an example diagram of a shooting device disclosed in an embodiment of this application. Figure 1A As shown, the robot is equipped with cameras and other imaging devices, capable of capturing images of stacked objects to be grasped. The field of view of the imaging devices varies depending on the working distance. Figure 1A As shown, the closer the working distance, the smaller the visual range. In this embodiment, the robot and the imaging device can also be separate; the imaging device is not mounted on the robot body but can communicate with it. Furthermore, the robot disclosed in this embodiment may also include a grasping component, which performs the action of grasping objects. This component can be a combination of one or more components such as a gripper, a gripping arm, or a vacuum suction head, and is not specifically limited. For example, the grasping component may include a robotic arm and a grasping device. The robotic arm includes multiple joints, each of which is movable; the grasping device can be installed at the end of the robotic arm for grasping objects on the top layer.

[0027] Please see Figure 1B , Figure 1B This is an example diagram of a robot working in an embodiment of this application. Figure 1B As shown, multiple building material fragments are stacked together, with the fragments placed randomly without any stacking pattern.

[0028] Please see Figure 2 , Figure 2 This is a flowchart illustrating a method for grasping stacked objects disclosed in an embodiment of this application. Figure 2 The method shown can be executed by the robot's processor or other computing modules, or by devices such as personal computers or industrial computers that are connected to the robot for communication; there is no specific limitation. For ease of description, the following description will use the robot's processor as an example. Figure 2 As shown, the method may include the following steps:

[0029] 210. Acquire images of stacked objects to be identified captured by the imaging device.

[0030] In this application embodiment, one example of stacked objects is as follows: Figure 1B As shown. The robot can acquire images captured by the imaging device through any one or more existing methods; some examples are shown below.

[0031] The robot can acquire images directly from the sensor hardware interface of the shooting device, such as USB interface, GigE interface, Camera Link interface, etc.

[0032] The robot can also acquire images through a file system, such as local storage devices (SD cards, silver disks, etc.) or network shared storage. The images stored in the file system are those captured by the imaging device.

[0033] In this embodiment of the application, one optional shooting angle of the shooting device is to shoot the stacked objects from a top-down perspective. It should be noted that the top-down perspective does not need to be strictly aligned with the center axis of the shooting device and the center of the stacked objects, but can be referenced to the angle of shooting from the top surface downwards.

[0034] 220. Extract the 3D point cloud data of the stacked objects from the image to be identified.

[0035] In the embodiments of this application, 3D point cloud data can be identified through the following methods: Iterative Closest Point (ICP) algorithm; establishing point pair correspondences using handcrafted feature descriptors (such as FPFH, SHOT), and estimating transformations using methods such as RANSAC, but not limited to these. Simultaneously, noise and outliers can be removed using filtering methods such as statistical filtering, morphological filtering, or bilateral filtering to improve the accuracy of the 3D point cloud data. Optionally, the imaging device can capture images of the stacked object from two directions at a top-down angle to extract the 3D point cloud data of the stacked object from two or more images to be identified.

[0036] It should be noted that the 3D point cloud data extracted from the image to be identified may include the 3D point cloud data of multiple objects.

[0037] 230. Identify the topmost object in a stacked object based on 3D point cloud data.

[0038] In this embodiment, the 3D point cloud data of stacked objects identified from the image can characterize the position of feature points on each object, including the horizontal axis component (x), the vertical axis component (y), and the depth component (z). When multiple objects are stacked together, the horizontal axis components, vertical axis components, and depth components of different objects exhibit different patterns. Specifically, the same object may contain multiple data points with the same depth component but different horizontal and vertical axis components; different objects may correspond to multiple data points with different horizontal and vertical axis components but the same depth component. Therefore, the robot can identify the top-level object by analyzing the above patterns.

[0039] As an alternative implementation, the robot can invoke or pre-load a trained artificial intelligence model. This model is trained by inputting a large amount of 3D point cloud data of stacked object samples, and can identify the 3D point cloud data belonging to the top-level object from point cloud data mixed with multiple stacked objects. This method offers high flexibility and can be applied to different types of stacked objects.

[0040] As an alternative implementation, the robot can identify the top-level object by analyzing the depth components of the 3D point cloud data within the stacked objects. Data points closer to the imaging device are identified from the 3D point cloud data as the top-level object. These closer data points can be defined as data points within a range whose depth components are all smaller than those outside that range.

[0041] Optionally, the depth components of each 3D point cloud data can be sorted, and the data point corresponding to the depth component closest to the camera device can be selected. This closest data point is then identified as belonging to the top-level object. Further, optionally, the 3D point cloud data can be clustered based on the depth components, identifying the depth components into the same category to obtain multiple distinct subsets of 3D point cloud data. Then, by counting the number of 3D point clouds with the same or small differences, the subset with the smallest corresponding depth component is identified as belonging to the top-level object.

[0042] Alternatively, a depth threshold can be set directly based on the known distance between the object and the object being photographed. For example, if the distance between the top object and the photographing device is known to be 50 centimeters, the depth threshold can be set to 50 centimeters or slightly less than 50 centimeters, such as 49-45 centimeters. Then, data points with depth components less than or equal to the depth threshold are identified as data points belonging to the top object.

[0043] 240. Identify the centroid position of the top object based on the 3D point cloud data of the top object.

[0044] In this embodiment, to improve the success rate of grasping, the robot grasps the centroid of the object, rather than its edge. Therefore, it is necessary to identify the centroid of the top-level object.

[0045] As an optional implementation, all the three-dimensional point cloud data of the top-level object can be used to calculate the centroid position, so that the calculated centroid position can be as close as possible to the true centroid position of the top-level object.

[0046] However, based on different top-level object recognition methods, when determining the 3D point cloud data included in the top-level object in the previous step, some data points that actually belong to the top-level object may be discarded. In other words, the 3D point cloud data of the top-level object determined based on the aforementioned steps may be partially missing. In addition, for robot grasping, the alignment of the centroid in the vertical and horizontal axes has a greater impact on grasping stability, while the extension of the grasping component in the depth direction has a relatively smaller impact on grasping stability.

[0047] Therefore, in some possible embodiments, it may be considered to further extract the upper surface edge contour of the top-level object from the 3D point cloud data of the top-level object, and calculate the centroid position of the top-level object based on the data points included in the upper surface contour. For example, please refer to... Figure 3 , Figure 3 This is an example diagram of a stacked object disclosed in an embodiment of this application. Figure 3 As shown, there are multiple stacked objects 310. The outermost layer of the top object 310a is the outline of its upper surface, which is shown as a darker line in the figure.

[0048] Furthermore, in this embodiment, the centroid position can be calculated with reference to the following formula:

[0049]

[0050] in, The location of the center of mass. The total number of data points used to calculate the centroid location. , , These are the horizontal axis component, vertical axis component, and depth component of the i-th data point used to calculate the centroid position, respectively. That is, if the data points used to calculate the centroid position are all the data points identified in the top-level object, then n is the total number of all such data points; if the data points used to calculate the centroid position only include the data points of the top surface edge contour of the top-level object, and do not include the data points of other parts (such as the sides), then n is the total number of data points of the top surface edge contour of the top-level object.

[0051] Optionally, in some possible embodiments, the centroid location may also refer to the average of the horizontal and vertical axis components, with the depth component replaced by a fixed value. For example, the depth component of the data point closest to the imaging device is known to be... The position of the centroid is represented as follows:

[0052] 250. Generate the robot's grasping trajectory based on the position of the center of mass.

[0053] 260. Control the gripping component to grip the top-level object according to the gripping trajectory.

[0054] In this embodiment, the grasping trajectory refers to the path of the robot's grasping component from its current position to its center of mass. Depending on the robot's configuration, the grasping estimate may include movement in three degrees of freedom (horizontal axis, vertical axis, and depth direction); or, in addition, it may include rotation about the horizontal axis, vertical axis, and depth direction.

[0055] Furthermore, as an optional implementation, during step 260 when the robot moves along the grasping trajectory, it acquires images of the stacked objects captured by the imaging device as new images to be recognized. After the grasping component performs the grasping action, it can detect whether the grasping was successful.

[0056] Optionally, the gripping component can be equipped with contact sensors such as pressure sensors and capacitance sensors. These sensors can detect whether the gripping component has touched the top-level object; if so, the gripping is successful; otherwise, the gripping fails. Alternatively, the image captured by the imaging device after the gripping action is performed can be analyzed to determine whether the position of the top-level object in the image has changed compared to its position before the gripping action; if so, the gripping is successful; otherwise, the gripping fails.

[0057] If the grasping component fails to grasp the object, the new object to be identified can be captured by the aforementioned imaging device during the grasping component's movement. The centroid position of the top-level object can then be re-identified, and the grasping trajectory of the grasping component can be adjusted based on the new execution position. The specific methods for identifying the new centroid position and adjusting the grasping trajectory are detailed in the preceding steps and will not be repeated here. In other words, the robot can monitor the grasping situation in real time, and if deviations or failures occur, adjustments can be made using feedback information to improve the success rate of grasping.

[0058] As can be seen, by implementing the embodiments of this application, the robot can automatically identify the centroid position of the top object based on the recognition of images of stacked objects, and grasp the top object accordingly. It can also grasp objects placed in a disordered manner, which can improve grasping efficiency and flexibility. At the same time, it can reduce human intervention in disordered grasping scenarios and reduce production costs. In addition, the recognition of the centroid position enables the robot to grasp objects more stably, which helps to improve the accuracy of grasping and reduce the occurrence of misgrabbing and damage to parts.

[0059] It should be noted that after the top object in the stacked objects is grasped, the object below the previous top object becomes the new top object. The robot can repeatedly execute the method disclosed in the embodiments of this application and can continuously grasp multiple stacked objects until the grasping task of the stacked objects is completed.

[0060] In one embodiment of this application, step 230 includes identifying the upper surface edge contour of the top-layer object, and then calculating the centroid position based on the data points of the upper surface edge contour. The following describes a step for identifying the upper surface edge contour disclosed in this application. Please refer to... Figure 4 , Figure 4 This is a flowchart illustrating a method for identifying the contour of an upper surface edge disclosed in an embodiment of this application. Figure 4 As shown, the following steps may be included:

[0061] Iterate through the points in the 3D point cloud data of the top-level object, and perform the following operations for each point:

[0062] 2310. Use the currently accessed point as the sampling point; the sampling point can be any point, which can be randomly accessed or a preset fixed point.

[0063] 2320. Establish a local neighborhood point set with radius r centered on the sampling point, and transform the neighborhood point set to obtain the spherical neighborhood of the sampling point; r is a positive integer greater than or equal to 1.

[0064] For example, the set of points included in the sphere's neighborhood can be represented by the following formula:

[0065]

[0066] Where N is the total number of point clouds, For sampling points, It represents three-dimensional coordinates.

[0067] Let sampling points Then the neighborhood of the r-sphere As shown in equation (4):

[0068]

[0069] 2330. Using the neighborhood of the sphere as sample data, fit the local tangent plane of the sampling point using the least squares method.

[0070] 2340. Perform eigenvalue decomposition on the covariance matrix formed by the data points contained in the local tangent plane of the sampling point, and take the eigenvector corresponding to the smallest eigenvalue as the normal vector of the sampling point.

[0071] For example, steps 2330 to 2340 can be described as follows.

[0072] Sphere Neighborhood For the sample data, fit the local optimal tangent plane using the least squares principle. Construct the objective function. The parameters of the target plane are solved as shown in formula (5).

[0073]

[0074] Where v is the normal vector of the target plane; d is the spatial distance from the origin to the target plane.

[0075] The solution process of formula (5) is transformed into eigenvalue decomposition of the covariance matrix C formed by the local neighborhood point set, as shown in formula (6).

[0076]

[0077] : It is the centroid of the neighborhood point set; Let C be the eigenvalues ​​of matrix C. for The eigenvectors of , where m=1,2,3, and ,but( Sampling points The estimated value of the normal vector n.

[0078] By following the steps described above, the normal vector of each data point in the 3D point cloud data of the top-level object can be calculated. Based on this, the following steps can be performed to extract the edge contour of the upper surface of the top-level object.

[0079] 2350. Based on the difference between the normal vectors of each data point in the 3D point cloud data of the top-level object, identify the data points belonging to the upper surface edge contour of the top-level object.

[0080] It is understandable that step 2350 is achieved through the following steps:

[0081] Select a data point as the center point in the 3D point cloud data of the top-level object;

[0082] Calculate the angle between the normal vectors of the data points within the neighborhood of the center point and the normal vector of the center point. The data points within the neighborhood can be a set of local points with radius r centered on the center point.

[0083] The included angles in each direction are compared with the smoothing threshold.

[0084] When the directional angle is less than the smoothing threshold, the data points within the corresponding neighborhood are added to the seed point set. The smoothing threshold is... It can be a preset angle value, compared with the direction angle corresponding to each non-sampling point. With smoothing threshold Non-sampling points refer to data points within the neighborhood of the center point; in the directional angle Less than the smoothing threshold ,Right now When the sampling point is selected, the corresponding non-sampling point is added to the seed point set.

[0085] Repeat the above steps until all points are classified, and obtain the top surface edge contour of the top-level object from the seed point set. The points in the seed point set include at least the points in the top surface edge contour; therefore, the top surface edge contour of the top-level object can be determined from the seed point set.

[0086] The above method can accurately identify the upper surface edge contour of the top object, and more accurately calculate the centroid position of the top object, thus improving the robot's grasping stability. Furthermore, accurate calculation of the upper surface edge contour reduces the amount of data required for the top object's data points, allowing the robot to achieve good grasping results with only a small amount of top-level 3D point cloud data.

[0087] Please see Figure 5 , Figure 5 This is a schematic diagram of a gripping device for stacked objects disclosed in an embodiment of this application. This gripping device for stacked objects can be applied to any of the aforementioned robots or electronic devices such as personal computers or industrial computers. Figure 5 As shown, the device includes:

[0088] The acquisition module 510 is used to acquire an image of a stacked object to be identified captured by the imaging device; the stacked object includes at least two objects placed in a stacked manner;

[0089] Extraction module 520 is used to extract the three-dimensional point cloud data of stacked objects from the image to be identified;

[0090] The recognition module 530 is used to identify the top object in the stacked objects based on the three-dimensional point cloud data of the stacked objects; and to identify the centroid position of the top object based on the three-dimensional point cloud data of the top object.

[0091] The generation module 540 is used to generate the robot's grasping trajectory based on the centroid position;

[0092] The control module 550 is used to control the gripping component to grip the top-level object according to the gripping trajectory.

[0093] As an optional implementation, the identification module 510 is used to extract the upper surface edge contour of the top object based on the three-dimensional point cloud data of the top object; and to calculate the centroid position of the top object based on the data points included in the upper surface edge contour of the top object.

[0094] As an optional implementation, the recognition module 510 is used to add the coordinates of each data point in the upper surface edge contour of the top object according to the corresponding coordinate components; the coordinate components include: horizontal axis component, vertical axis component and depth component; and, it is used to divide the sum of the corresponding coordinate components by the total number of data points in the upper surface edge contour to obtain the coordinate value corresponding to each coordinate component as the centroid position of the top object.

[0095] As an optional implementation, the identification module 510 is used to traverse the points in the 3D point cloud data of the top-level object and perform the following operation for each point: take the currently visited point as the sampling point;

[0096] A local neighborhood point set with radius r is established centered on the sampling point, and a spherical neighborhood of the sampling point is obtained based on the neighborhood point set; r is a positive integer greater than or equal to 1; using the spherical neighborhood as sample data, the local tangent plane of the sampling point is fitted using the least squares method; eigenvalue decomposition is performed on the covariance matrix formed by the data points contained in the local tangent plane of the sampling point, and the eigenvector corresponding to the smallest eigenvalue is used as the normal vector of the sampling point; based on the difference between the normal vectors of each data point in the 3D point cloud data of the top-level object, the data points belonging to the upper surface edge contour of the top-level object are identified.

[0097] As an optional implementation, the recognition module 510 is used to select a data point as the center point in the 3D point cloud data of the top-level object; calculate the angle between the normal vector of the data point in the neighborhood of the center point and the normal vector of the center point; compare each angle with a smoothing threshold; when the angle is less than the smoothing threshold, add the corresponding data point in the neighborhood to the seed point set; traverse the above steps until the classification of all points is completed; and obtain the upper surface edge contour of the top-level object from the seed point set.

[0098] As an optional implementation, the acquisition module 510 is also used to acquire images of stacked objects captured by the imaging device as new images to be identified during the process of the control module 550 controlling the grasping component to grasp the top-level object according to the grasping trajectory.

[0099] The gripping device for stacked objects may also include:

[0100] The monitoring module is used to trigger the extraction module 520 and the recognition module 530 to identify the new centroid position of the top object based on the new image to be recognized when the grasping component fails to grasp the top object, and to trigger the generation module 540 and the control module 550 to adjust the grasping trajectory of the grasping component based on the new centroid position.

[0101] The stacked object grasping device disclosed in the embodiments of this application can automatically identify the centroid position of the top object in the stacked objects and grasp the top object based on this, achieving the effect of disordered grasping and improving grasping efficiency.

[0102] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a robot disclosed in an embodiment of this application. Figure 6 As shown, the robot may include:

[0103] Memory 610 containing computer programs;

[0104] Processor 620 coupled to memory 610;

[0105] When the computer program stored in the memory 610 is executed by the processor 620, the processor 620 implements any of the stacked object grasping methods disclosed in the embodiments of this application.

[0106] It should be noted that, Figure 6 The robot shown may also include components not shown, such as a power supply, input buttons, screen, RF circuit, Wi-Fi module, and Bluetooth module, which will not be described in detail in this embodiment.

[0107] This application discloses a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements any of the stacked object grasping methods disclosed in this application.

[0108] This application discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform any of the stacked object grasping methods disclosed in this application.

[0109] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also recognize that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0110] In the various embodiments of this application, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0111] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they can be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0112] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0113] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-accessible memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several requests to cause a computer device (which can be a personal computer, server, or network device, specifically a processor in the computer device) to execute some or all of the steps of the methods described in the various embodiments of this application.

[0114] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compactdisc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0115] The foregoing has provided a detailed description of a method, apparatus, robot, and storage medium for grasping stacked objects according to embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method of grasping stacked objects, characterized by, The method comprises: acquiring an image of a stacked object to be identified photographed by a photographing device; the stacked object comprises at least two objects placed in a stacked manner; extracting three-dimensional point cloud data of the stacked object from the image to be identified; identifying a top layer object in the uppermost layer of the stacked object based on the three-dimensional point cloud data of the stacked object; identifying a centroid position of the top layer object according to the three-dimensional point cloud data of the top layer object; generating a robot grabbing trajectory according to the centroid position, and controlling a grabbing component to grab the top layer object according to the grabbing trajectory.

2. The method of claim 1, wherein, The method comprises: acquiring a depth component in the three-dimensional point cloud data of the stacked object; wherein the depth component represents a distance between a data point in the three-dimensional point cloud data and the photographing device; identifying data points within a preset range from the photographing device as the top layer object in the uppermost layer from the three-dimensional point cloud data of the stacked object based on the depth component; wherein the depth component of the data points within the preset range is smaller than the depth component of the data points outside the preset range.

3. The method according to claim 1 or 2, characterized in that, The method comprises: extracting an upper surface edge contour of the top layer object according to the three-dimensional point cloud data of the top layer object; calculating the centroid position of the top layer object according to the data points included in the upper surface edge contour of the top layer object.

4. The method of claim 3, wherein, The method comprises: adding coordinates of each data point in the upper surface edge contour of the top layer object according to corresponding coordinate components; the coordinate components comprise a horizontal axis component, a vertical axis component and a depth component; dividing the sum of the corresponding coordinate components by the total number of data points in the upper surface edge contour to obtain a coordinate value corresponding to each coordinate component as the centroid position of the top layer object.

5. The method of claim 3, wherein, The method comprises: iterating through points in the three-dimensional point cloud data of the top layer object, and performing the following operations on each point: taking the currently accessed point as a sampling point; establishing a local neighborhood point set with a radius r centered on the sampling point, and converting the local neighborhood point set to obtain a spherical neighborhood of the sampling point; r is a positive integer greater than or equal to 1; fitting a local tangent plane of the sampling point using the spherical neighborhood as sample data by using the least square method; performing eigenvalue decomposition on a covariance matrix formed by data points included in the local tangent plane of the sampling point, and taking a feature vector corresponding to the smallest eigenvalue as a normal vector of the sampling point; identifying data points belonging to the upper surface edge contour of the top layer object according to the differences between the normal vectors of each data point in the three-dimensional point cloud data of the top layer object.

6. The method of claim 5, wherein, The method comprises: selecting a data point in the three-dimensional point cloud data of the top layer object as a center point; calculating a directional included angle between a normal vector of a data point in a neighborhood range of the center point and a normal vector of the center point; comparing each directional included angle with a smoothing threshold value; when the directional included angle is less than the smoothing threshold value, adding the data point in the neighborhood range corresponding to the directional included angle to a seed point set; iterating the above steps until the classification of all points is completed; obtaining an upper surface edge contour of the top layer object from the seed point set.

7. The method according to claim 1 or 2, characterized in that, The method further comprises: acquiring a stacked object image captured by a shooting device as a new to-be-recognized image during control of the grabbing component to grab the top layer object according to the grabbing trajectory; when it is detected that the grabbing component fails to grab the top layer object, identifying a new center of mass position of the top layer object according to the new to-be-recognized image, and adjusting the grabbing trajectory of the grabbing component according to the new center of mass position.

8. A grasping device for stacked objects, characterized in that comprise: an acquisition module configured to acquire a to-be-recognized image of a stacked object captured by a shooting device; the stacked object comprises at least two objects placed in a stacked manner; an extraction module configured to extract three-dimensional point cloud data of the stacked object from the to-be-recognized image; an identification module configured to identify a top layer object in an uppermost layer of the stacked object based on the three-dimensional point cloud data of the stacked object; and configured to identify a center of mass position of the top layer object according to the three-dimensional point cloud data of the top layer object; a generation module configured to generate a grabbing trajectory of a robot according to the center of mass position; a control module configured to control a grabbing component to grab the top layer object according to the grabbing trajectory.

9. A robot, characterized in that The electronic device comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to enable the processor to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 7.

Images