Object grasping robot

Abstract

Object grasping software (3) for enabling a robotic arm's gripper (18) to grasp various objects. The object grasping software (3) is loadable in, and executable by, electronic processing resources (2) and designed to cause, when executed, the electronic processing resources (2) to become configured to receive an image (5) to be analysed, to detect an object (15) represented in the received image (5) to be analysed, to and compute a representation (C) of the detected object (15). In addition, the object grasping software (3) is designed to cause, when executed, the electronic processing resources (2) to become configured to classify, by means of a geometry classifier (6), the detected object (15) either as a simple geometry object or a complex geometry object. A simple geometry object and a complex geometry object exhibit distinct geometric features. The object grasping software (3) is further designed to cause, when executed, the electronic processing resources (2) to become configured to compute (block 8) one or more grasp poses (9), representing respective target positions and orientations for the robotic arm's gripper (18) to grasp the detected object (15), by executing a geometry-based algorithm (8A) on the representation (C) of the detected object (15) if it is classified as simple geometry object; and by executing a trained machine learning model (8B) on the representation (C) of the detected object (15) if it is classified as complex geometry object. Furthermore, the object grasping software (3) is designed to cause, when executed, the electronic processing resources (2) to become configured to control (block 10) the operation of the robotic arm's gripper (18) based on the computed grasp poses (9) to cause the robotic arm's gripper (18) to grasp the detected object (15).

Description

[0001] “OBJECT GRASPING ROBOT”

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This Patent Applications claims priority from European Patent Application No. 24192552.8 filed on August 2, 2024 and Italian Patent Application No. 102025000000567 filed on January 15, 2025, the entire disclosure of which is incorporated herein by reference.

[0004] TECHNICAL FIELD OF THE INVENTION

[0005] The invention relates in general to the computer vision field. In particular, the present invention relates to a robot configured to grasp various objects by employing computer vision techniques.

[0006] STATE OF THE ART

[0007] As is known, in the field of computer vision and robotics, the development of object grasping robots has seen significant advancements. These robots are increasingly required to operate alongside humans in dynamic and unpredictable environments, necessitating reliable and accurate object grasping and manipulation capabilities. Having an automatic vision-based object grasping robot able to work accurately in real -world situations is essential for various fields, from manufacturing and warehousing to healthcare and home assistance. The primary challenge lies in enabling robots to grasp and manipulate new or previously unseen objects, which requires sensors, such as cameras, and advanced algorithms.

[0008] As is known, most of the advanced used methods for vision-based object grasping rely heavily on deep learning techniques to identify optimal 3D grasp poses for robotic arms, in order to compute a trajectory to be followed by the robot based on said grasp poses. These methods typically employ machine learning, or neural, models like 3D convolutional neural networks (3D CNNs) to extract features from images, generate grasp poses candidates, and evaluate them based on datasets with labelled grasp poses. In detail, these models are then configured to predict and select the best grasp poses during operation in order to control the robotic arm based on said grasp poses. In detail, a grasp pose is defined as the specific location that the robot's gripper must reach, with a precise orientation and position relative to the robot's base. An optimal grasp pose is the location on the object that allows the gripper of the robot to securely hold the object without causing damage.

[0009] The accuracy of these models in estimating grasp poses is contingent upon the availability of extensive and varied training datasets.

[0010] Additionally, there are methods that generate grasp poses based on the object's central coordinate system, such as top-down grasping. SUBJECT-MATTER AND SUMMARY OF THE INVENTION

[0011] The Applicant notes that the known solutions can be improved. In fact, the Applicant observes that the known methods that can be applied for automatic vision -based grasping present several problems that limit their effectiveness in real-world scenarios. Specifically, the Applicant notes that the known, best-performing methods often struggle because they depend solely on machine learning. Notably, the Applicant observes that these methods frequently fail to generate accurate grasp poses even for objects with simple geometries. The Applicant further notes that the grasp orientation is often misaligned with the orientation of the object, leading to potential collisions between the object and the gripper and resulting in unsuccessful grasping. While some grasp failures are acceptable for objects with complex geometrical shapes, failures with simple shapes with uniform structures, such as boxes or cylinders, are not.

[0012] To enhance accuracy, current solutions typically require larger training datasets, which have several disadvantages. If the training data is insufficient or biased, the machine learning model may not perform well with new or unseen objects, resulting in poor performance, especially with objects not well -represented in the training data. Additionally, training and executing these models require significant computational power, which can be costly. The training process can also be time-consuming, delaying practical deployment. Furthermore, these algorithms may struggle when the environment or object characteristics differ from the training data; for example, changes in lighting, occlusions, or sensor noise can impact performance. Furthermore, ensuring robustness across different conditions requires extensive fine-tuning and additional data collection, adding complexity and cost.

[0013] On the other hand, the Applicant notes that methods generating grasp poses based on the object's central coordinate system, such as top-down grasping, have two main drawbacks. First, not all objects are graspable from the top, even if their geometry is simple. For instance, generating a grasp pose on the top central coordinate system of a cylinder might result in an unsuccessful grasp due to the limited surface area for the gripper, increasing the risk of the object slipping or tipping over. Second, these methods fail to grasp most objects with complex geometries (with irregular, non-uniform top surfaces). Additionally, the top face of a complex object may have various contours and protrusions, complicating the identification of an optimal grasping pose.

[0014] Hence, the object of the invention is to provide an object grasping software, in detail a unified object grasping software, that at least partially addresses the issues present in the known prior art. Specifically, the object of the invention is to offer a solution that accurately determines grasp poses for objects with both simple and complex geometric shapes. In detail, the proposed object grasping software is designed to distinguish the geometric type of the objects, whether simple or complex, and to enable a robotic arm’s gripper to automatically grasp them. More in detail, the object grasping software is configured to apply two distinct grasp pose generation strategies based on the geometric types of the objects. The Applicant notes that the proposed solution significantly improves the automatic grasp success rate for simple geometrical object shapes compared to the known solutions, while effectively handling objects with complex geometries.

[0015] According to the invention, there are provided an object grasping software and an object grasping robot as claimed in the appended claims.

[0016] According to the invention, there is provided an object grasping software for enabling a robotic arm’s gripper to grasp various objects. The object grasping software being loadable in, and executable by, electronic processing resources and designed to cause, when executed, the electronic processing resources to become configured to receive an image, in particular a singleview image to be analysed, detect an object represented in the received image to be analysed, and to compute a representation of the detected object. In addition, the object grasping software is designed to cause, when executed, the electronic processing resources to become configured to classify, by means of a geometry classifier, the detected object either as a simple geometry object or a complex geometry object; wherein, a simple geometry object and a complex geometry object exhibit distinct geometric features. Furthermore, the object grasping software is designed to cause, when executed, the electronic processing resources to become configured to compute one or more grasp poses, representing respective target positions and orientations for the robotic arm’s gripper to grasp the detected object, by executing a geometry-based algorithm on the representation of the detected object if it is classified as simple geometry object; and by executing a trained machine learning model on the representation of the detected object if it is classified as complex geometry object. The object grasping software is further designed to cause, when executed, the electronic processing resources to become configured to control operation of the robotic arm’s gripper based on the computed grasp poses to cause the robotic arm’s gripper to grasp the detected object.

[0017] In particular, said object grasping software is designed to cause, when executed, the electronic processing resources to become configured to compute a partial point cloud representation of the detected object by performing segmentation of the detected object on the received image to be analysed, and possibly map the masked information on the Depth image and use camera information. In addition, the object grasping software is designed to cause, when executed, the electronic processing resources to become configured to predict missing, or non- visible, parts of the detected object in the partial point cloud representation by executing a point cloud completion neural network on the partial point cloud representation. Wherein, the point cloud completion neural network is trained on various point cloud representations to predict missing, or non-visible, parts of an object in an input point cloud representation. The object grasping software is further designed to cause, when executed, the electronic processing resources to become configured to compute the representation of the detected object by including said predicted missing, or non-visible, parts in the partial point cloud representation of the detected object.

[0018] In detail, the object grasping software is designed to cause, when executed, the electronic processing resources to become configured to classify the detected object as simple geometry object if the detected object is classifiable as complex geometry object but it is determined to comprise one or more simple geometry objects.

[0019] In particular, said object grasping software is designed to cause, when executed, the electronic processing resources to become configured to determine whether the detected object is classifiable or is not classifiable with one of a plurality of primitive geometry classes . In particular, the primitive geometry classes comprise cylinders, cuboids, spheres, cones, and ellipsoids. The object grasping software is further designed to cause, when executed, the electronic processing resources to become configured to classify said detected object as simple geometry object if it is determined that it is classifiable with a primitive geometry class, and to classify the detected object as complex geometry object otherwise.

[0020] In addition, said object grasping software is possibly designed to cause, when executed, the electronic processing resources to become configured to discard, or prevent the computation of, one or more of the grasp poses based on the primitive geometry class with which the detected object is determined to be classifiable.

[0021] In detail, the geometry-based algorithm is designed to cause, when executed, the electronic processing resources to become configured to compute a three-dimensional bounding box having a plurality of faces and encapsulating the representation of the detected object based on the representation of the detected object. In addition, the electronic processing resources become further configured to determine, for each face of the three-dimensional bounding box, a possible grasp point, in particular the centre, located on said face. Additionally, the electronic processing resources become configured to compute the grasp poses, each relating to a three-dimensional bounding box’s face, based on the determined possible grasp points located on the respective faces of the three-dimensional bounding box.

[0022] In more detail, the three-dimensional bounding box is represented in a three-dimensional local reference system, and the geometry-based algorithm is further designed to cause, when executed, the electronic processing resources to become configured to compute a rotation matrix that represents the orientation of the three-dimensional bounding box relative to a three- dimensional global reference system based on the determined possible grasp points; and to compute the grasp poses based on the computed rotation matrix.

[0023] More in detail, the geometry-based algorithm is further designed to cause, when executed, the electronic processing resources to become configured to compute each of the grasp poses as representing the target position as a possible grasp point located on a face of the three-dimensional bounding box, and the orientation as defined by the computed rotation matrix. In particular, said object grasping software is designed to cause, when executed, the electronic processing resources to become configured to determine the dimension of the robotic arm’s gripper, and the dimension of a face of the three-dimensional bounding box. If the dimension of the robotic arm’s gripper is determined to be lower than the dimension of said face, the electronic processing resources become configured to discard, or prevent the computation of, each of the grasp poses relating to said face.

[0024] According to the invention, there is further provided an object grasping robot comprising a robotic arm, comprising at least one gripper configured to grasp objects. Said object grasping robot further comprises electronic processing resources storing, and configured to execute, the object grasping software in order to control the operation of the robotic arm’s gripper based on the computed grasp poses to cause the robotic arm’s gripper to grasp the detected object.

[0025] BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 schematically shows a block diagram of an object grasping system according to the present invention.

[0027] Figures 2A, 2B, 2C, 2D show examples of objects classifiable as simple geometry or an aggregation of simple geometry objects.

[0028] Figure 3 schematically shows a functional block diagram related to a method for grasping objects executable by means of an object grasping software according to a preferred aspect of the present invention.

[0029] Figure 4 schematically shows a functional block diagram of a geometry-based algorithm according to the preferred aspect of the present invention.

[0030] Figure 5 schematically shows a functional block diagram representing a first example of grasp pose computation according to the preferred aspect and a second example of grasp pose computation performed according to a distinct aspect.

[0031] DESCRIPTION OF PREFERRED EMBODIMENTS OF THE INVENTION

[0032] The invention will now be described in detail with reference to the accompanying figures, so as to allow a person skilled in the art to carry it out and use it. Possible changes to the embodiments described herein will be immediately evident to skilled people and the generic principles described herein can be applied to other embodiments and applications without for this reason going beyond the scope of protection of the invention as it is defined in the appended claims. Therefore, the invention cannot be considered as limited to the embodiments described and shown herein, but it has to be associated with the widest scope of protection possible in accordance with the features described and claimed herein. If not specifically defined otherwise, all technical and scientific terms have the meaning commonly used by people ordinarily skilled in the industry to which the invention belongs. In case of conflict, the description - including the definitions provided therein - is binding. Furthermore, the examples are provided by mere way of explanation and, as such, should not be considered as limiting.

[0033] In particular, the block diagrams included in the accompanying figures and described below should not be interpreted as representation of structural features, namely as constructive limits, but they should be regarded as representation of functional features, i.e. intrinsic properties of the devices defined by the obtained effects, namely as functional limits, which can be implemented in different manners, hence so as to protects the functionality thereof (possibility of functioning).

[0034] In order to make it easier for readers to understand the embodiments described herein, reference is made to some specific embodiments and a specific language will be used to describe them. The terms used in this document are aimed at exclusively describing particular examples and are not suited to limit the scope of protection of the invention.

[0035] Figure 1 schematically shows a block diagram of an object grasping system 1 according to the present invention. In particular, the object grasping system 1 comprises electronic processing resources 2 storing, and configured to execute, an object grasping software 3, or computer product, being designed for enabling an object grasping robot 17, in detail a robotic arm’s gripper 18, to grasp various objects. Said object grasping robot 17 comprises the robotic arm, which comprises said gripper 18. In detail, the gripper 18 comprises gripper fingers 19 designed to be controlled to grasp objects. In addition, the object grasping robot 17 comprises said electronic processing resources 2 storing and configured to execute the object grasping software 3 to become configured to cause the robotic arm’s gripper 18 to grasp an object. In more detail, the object grasping system 1 further comprises the robot 17. Alternatively, the object grasping robot 17 comprises electronic communication resources configured to receive data, specifically instructions for grasping a detected object 15, from the electronic processing resources 2, if the latter are external to the object grasping robot 17.

[0036] In particular, the object grasping software 3 is designed to cause, when executed, said electronic processing resources 2 to become configured to perform one or more operations in order to cause the robotic arm’s gripper 18 to grasp an object 15 detected in a working environment 16. More specifically, the object grasping software 3 is designed to cause, when executed, said electronic processing resources 2 to become configured to perform one or more operations to compute one or more grasp poses 9 in order to cause the robotic arm’s gripper 18 to grasp an object 15 detected in a working environment 16.

[0037] With regard to the object grasping task, it is emphasized that what matters are the operations that must be implemented to realize such functionality and not the hardware and software architectures with which such operations are implemented; these could be implemented through a concentrated architecture, for example a single electronic device, or through a distributed cooperative architecture, comprising several electronic devices in communication and cooperating with each other, according to a proprietary logical architecture that the manufacturer of the object grasping software 3 (and of the object grasping robot 17) will decide to adopt.

[0038] The electronic processing resources 2 are configured to receive an image 5 to be analysed. Specifically, the image 5 to be analysed represents a working environment 16, in which the object grasping robot 17 can operate, comprising one or more objects 15 at the time. Wherein, specifically, the working environment 16 can be of any type and can possess any kind of features. In a way of example, the working environment 16 is a workbench, a floor, or any other surface suitable for the task. In detail, the image 5 to be analysed fails to represent an object, and therefore the working environment 16, in its entirety; more specifically, it represents the working environment 16 from a specific and limited viewpoint or followed path of the object grasping robot 17.

[0039] In particular, the electronic processing resources 2 are configured to receive said image 5 to be analysed from a sensor 4, in detail a camera, designed to capture images of a working environment 16. In detail, the sensor 4 is a camera configured to provide, possibly in real-time, both depth and colour data as output; for example, it is an RGB-D (Red, Green, Blue - Depth) camera. In more detail, the image 5 to be analysed represents both depth and colour data.

[0040] In more detail, the image 5 to be analysed is a single-shot image 5 captured by the camera in a single shot.

[0041] In particular, the electronic processing resources 2 are configured to receive the image 5 to be analysed from a sensory platform comprising at least said sensor 4, and possibly other different sensors 4. In detail, the object grasping robot 17 further comprises the sensory platform. Alternatively, the sensory platform is external to the object grasping robot 17; in particular, the object grasping system 1 comprises said sensory platform. In a way of example, according to the aspect in which the object grasping system 1 comprises said sensory platform, the sensor 4 is fixed in a position of the working environment 16.

[0042] The electronic processing resources 2 are further configured to detect an object 15 represented in the received image 5 to be analysed. It should be noted that, hereinafter, the term detected object 15 will be used, but the operations described for this detected object 15 can be possibly applied to any of the objects detected in the image 5 to be analysed.

[0043] In addition, the electronic processing resources 2 are configured to classify, by means of a geometry classifier 6, the detected object 15 either as a simple geometry object or a complex geometry object.

[0044] A simple geometry object (object with simple geometrical shapes) and a complex geometry object (object with complex geometrical shapes) exhibit distinct geometric features. In detail, a simple geometry object is an object with uniform structures or regular shapes, such as, for example, cylinders, cuboids (boxes), spheres, cones, ellipsoids and others. According to a preferred embodiment of the present invention, an object can be classified as simple geometry object if it can be classified with one of a plurality of primitive geometry classes. In particular, the primitive geometry classes comprise cylinders, cuboids, spheres, cones, ellipsoids; optionally comprise also hyperboloids, parabolic cylinders, and similar shapes. Conveniently, the primitive geometry classes include any type of structures or shapes that are easy to be recognized and easy to be managed from grasping definition point of view. In detail, it can be observed that many everyday objects can be effectively categorized into these shapes. For instance, objects like a can of soda, despite minor deformations like bulges or dents, can still be classified and grasped as cylinders. Similarly, a book with slightly curved edges can be handled as a box (cuboid). Preferably, an object is also a simple geometry object if it is determined to be the aggregation of simple geometry objects (for example an aggregation of cylinders and cuboids). Figures 2A, 2B, 2C, 2D show examples of objects classifiable as simple geometry objects. In detail, Figure 2A and Figure 2C show respectively that a box and a ball are classifiable as simple geometry objects. In addition, Figure 2B shows that a cleaner is identified as a simple geometry object (in detail as a cylinder) due to its simple geometric shape. According to the example of Figure 2D, when the geometric classifier was applied to an image 5 of a drill, it classified said drill as simple geometry object as it can be treated as an aggregation of simple shapes (in detail of cylinders and cuboids).

[0045] On the other hand, unlike simple geometry objects, complex geometry objects exhibit intricate and irregular surfaces, multiple contours, and varying dimensions. According to said embodiment of the present invention, an object can be classified as complex geometry object if it fails to be classified with at least one of the primitive geometry classes.

[0046] In detail, the electronic processing resources 2 are configured to execute, or to run, the geometry classifier 6 to classify the object's geometry type; more in detail, to classify the detected object 15 either as a simple geometry object or a complex geometry object.

[0047] According to a first aspect of the present invention, the geometry classifier 6 is configured to detect one or more objects represented in the received image 5 to be analysed, and to classify the detected object(s) 15. In particular, the geometry classifier 6 is configured to detect at least one object in said image 5 to be analysed and to classify said detected object 15.

[0048] According to said first aspect of the present invention, the electronic processing resources 2 are configured to input the received image 5 to be analysed to the geometry classifier 6 and to receive the geometry classifier 6’s output; which is at least indicative of the class (in detail simple or complex geometry object) associated to the detected object 15. In addition, in particular, the output comprises data indicating the performance of the geometry classifier 6 in processing the image 5 to be analysed.

[0049] According to a second and different aspect of the present invention, the electronic processing resources 2 are configured to detect an object represented in the received image 5 to be analysed by executing an object detection model, and to classify the detected object 15 by executing of the geometry classifier 6. In particular, the electronic processing resources 2 are configured to input the received image 5 to be analysed to the object detection model, or object detection algorithm, and to receive the object detection model’s output; and are configured to input the obj ect detection model ’ s output to the geometry classifier 6 in order to receive its output. In detail, according to said second aspect of the present invention, the object detection model and the geometry classifier 6 are distinct models, unlike the first aspect of the present invention.

[0050] In particular, the object detection model is configured to detect one or more object(s) (possibly of any type) represented in the received image 5 to be analysed; and to compute an output at least indicative of the presence or absence of one or more objects within a working environment 16 represented by the image 5 to be analysed based on said image 5. Conveniently, the object detection model is a machine learning, in detail a neural network model, trained on a dataset (possibly a generic dataset) of representations in order to become configured to detect object(s) based on an input image 5 to be analysed. Alternatively, the object detection model is configured to perform a different object detection technique on the image 5 to be analysed to detect said object.

[0051] More in detail, the object detection model is further configured to locate the objects determined to be represented within said working environment 16; in particular, to determine the position of said objects with respect to the working environment 16. Optionally, the object detection model is configured to perform a segmentation of the object on the image 5 to be analysed in order to obtain an output indicative of said detected object 15. Optionally, the object detection model’s output is a position of the detected object 15 filtering its location with respect to the represented working environment 16; in detail, a representation position the detected object 15. According to the second aspect of the present invention, the geometry classifier 6 is a neural network trained (in detail solely) to classify the detected object 15 based on the object detection model’s output.

[0052] Alternatively, the geometry classifier 6 comprises the object detection model and comprises a classifier model configured to classify an object as simple geometry object or as complex geometry object.

[0053] It should be noted that, unless otherwise specified, when referring to the geometry classifier 6 from here onwards, it pertains, without limitation, to the first aspect of the present invention, wherein the geometry classifier 6 functions as both a detector and a classifier.

[0054] Preferably, the geometry classifier 6 is a machine learning model, or a neural network model, trained on an image dataset comprising several images (or representations) representing different objects (possibly within different working environments 16), in order to become configured to classify object(s) in an input image or representation. In more detail, the image dataset includes several images representing different objects, each labelled with a corresponding tag or label (for example a textual content) and bounding box for detection identifying a class of the object and its location. In particular, an image dataset’s image is labelled with multiple classes and multiple bounding boxes when it represents multiple objects, either aggregated to form a single object or arranged in different positions within the represented working environment 16.

[0055] Conveniently, the geometry classifier 6 is a neural network trained by adjusting its weights through the execution of a training algorithm based on the image dataset, in order to become configured to determine and associate one or more classes and bounding boxes to an input image, and to output said classes and bounding boxes.

[0056] In detail, according to the first aspect of the present invention, the geometry classifier 6 includes at least one Transformer neural network designed to enhance the accuracy of object detection and classification. This architecture significantly enhances the ability to detect and classify objects within images (especially within RGB images) with high precision. In a way of example, the geometry classifier 6 comprises the Grounding DINO model proposed in ‘Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection’ by Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, Lei Zhang. In detail, Grounding DINO extends a closed-set object detection model with a text encoder, enabling open-set object detection. This type of approach allows to assign accurate labels to the detected objects, ensuring robust and reliable identification.

[0057] In detail, the electronic processing resources 2 are configured to train the geometry classifier 6. Alternatively, the electronic processing resources 2 are configured to receive the already trained geometry classifier 6 from external electronic processing resources.

[0058] According to a preferred embodiment of the present invention, the geometry classifier 6 is configured, in detail trained, to classify an input (in detail the detected object 15) into one or more of the primitive geometry classes.

[0059] In particular, each sample of said image dataset is labelled, and associated, with one or more of the primitive geometry classes. In detail, each object represented in each representation of said image dataset is labelled with at least one primitive geometry class.

[0060] Possibly, an image can be associated with one or more primitive geometry classes, or with a class indicating that the image fails to present an object classifiable with a primitive geometry class.

[0061] According to said preferred embodiment of the present invention, the electronic processing resources 2 are configured to determine whether the detected object 15 is classifiable or is not classifiable with one of a plurality of primitive geometry classes; in detail, based on the geometry classifier’s 6 output.

[0062] In addition, the electronic processing resources 2, possibly the geometry classifier 6, are configured to classify said detected object 15 as simple geometry object if it is determined that it is classifiable (in particular associable) with a primitive geometry class, and to classify the detected object 15 as complex geometry object otherwise. More specifically, the electronic processing resources 2 are configured to determine the primitive geometry classes associated with the detected object 15 based on the geometry classifier’s 6 output (for example said output comprising textual contents indicating the primitive geometry classes), and to classify said detected object 15 as simple geometry object if the primitive geometry classes include the classes associated with the detected object 15.

[0063] According to a different embodiment of the present invention, the geometry classifier 6 is trained to directly classify the detected object 15 as having either a simple or complex geometry, without any preceding classification needed. In detail, each representation or image of said image dataset is labelled, and associated, with a simple geometry object class indicating that the object presents a simple geometry, or with a complex geometry object indicating that the object presents a complex geometry.

[0064] Optionally, the image dataset is collected and labelled by one or more operators.

[0065] Preferably, the electronic processing resources 2, possibly the geometry classifier 6, are further configured to classify the detected object 15 as simple geometry object if the detected object 15 is determined to comprise one or more simple geometry objects. In detail, the electronic processing resources 2 are further configured to classify the detected object 15 as simple geometry object if the detected object 15 is classifiable as complex geometry object but it is determined to comprise one or more simple geometry objects. In a way of example, the electronic processing resources 2 are configured to classify the detected object 15 as simple geometry object if the detected object 15 is determined to be, in detail correspond to, an aggregation of simple geometry objects. In detail, the geometry classifier 6 is trained to detect one or more sub-objects within said detected object 15. In more detail, the geometry classifier 6 is trained to detect one or different sub-objects classifiable as simple geometry objects within the detected object 15. In more detail, the geometry classifier 6 is trained to detect different sub-objects that aggregate to form the detected object 15 and to determine the class of each detected sub-object. According to an aspect of the present invention, the electronic processing resources 2 are configured to detect and classify the sub-objects if (in detail only if) the detected object 15 is classified as complex geometry object by means of the geometry classifier 6. More specifically, the electronic processing resources 2 are configured to classify the detected object 15 based on the classes determined, by means of the geometry classifier 6, for the detected sub-objects forming the object. In detail, once a current simple geometry object is detected, the electronic processing resources 2 are configured to remove the corresponding part of the object from the image 5 to be analysed, and eventually to analyse the remaining portion of the image 5 to be analysed. Possibly, the process concludes either when all simple geometry objects have been attempted or when the entire image 5 to be analysed has been analysed. In particular, if one or more of the sub-objects are classified as simple geometry object the electronic processing resources 2 are configured to classify the detected object 15 as a simple geometry object; and are configured to classify the detected object 15 as a complex geometry object otherwise. In a way of example, according to the preferred embodiment of the present invention, if at least one sub-object is classifiable with one of the primitive geometry classes, the electronic processing resources 2 are configured to classify the detected object 15 as a simple geometry object. Optionally, the electronic processing resources 2 are configured to classify the detected object 15 as a simple geometry object if one of its detected sub-objects is classifiable as simple geometry object and if a feasible grasping condition is determined to be satisfied for said sub-object. In particular, the feasible grasping condition is determined to be satisfied if it is determined that there is at least one feasible grasp pose for said sub -object that can be computed and used for grasping the detected object 15.

[0066] The electronic processing resources 2 are further configured to compute (block 11) a representation C of the detected object 15. In detail, for each of the detected objects 15, the electronic processing resources 2 are configured to compute (block 11) a representation C of said detected object 15. Optionally, the electronic processing resources 2 are configured to compute (block 11) said detected object’s 15 representation C by performing (block 11A) a segmentation on the image 5 to be analysed. In detail, the representation C of the detected object 15 isolates the detected object 15 from the working environment 16 and fails to represent any other elements or backgrounds other than the detected object 15. In particular, the electronic processing resources 2 are configured to compute (block 11 A) a segmented image A, and to compute (block 11) a representation C of the detected image based on the segmented image A.

[0067] Figure 3 schematically shows a functional block diagram related to a method for grasping objects executable by means of the object grasping software 3 according to a preferred aspect of the present invention.

[0068] According to the preferred aspect of the present invention, the electronic processing resources 2 are configured to compute (block 1 IB) a partial point cloud representation B (in detail a three-dimensional representation B) of the detected object 15 by performing a segmentation (block 11 A) of the detected object 15 on the received image 5 to be analysed. In detail, the electronic processing resources 2 are configured to execute (block 11A) a segmentation model trained to perform a segmentation on the received image 5 to be analysed to output data representing the detected object 15 (or sub-objects) isolated from the working environment 16. In more detail, said segmentation model is configured to segment (block 11 A) the background information of the image 5 to be analysed by applying mask of the segmented object background removal-based approaches which masks the detected object 15 in the image 5 to be analysed. In detail, the segmentation model being trained to isolate the detected object 15 and apply mask to the depth to extract point cloud. In a way of example, the segmentation model is Grounded SAM (Segment Anything Model), which is configured to isolate objects from their background. In a way of example, for the segmentation on the image, the electronic processing resources 2 are configured to execute grounding dino algorithm to detect the form of the detected object 15 then use the Segment Anything framework to segment the detected form. This approach ensures that the objects are accurately identified and isolated from the background, making it easier to analyse and interact with them. The electronic processing resources 2 are configured to compute (block 1 IB) the partial point cloud representation C of the detected object 15 based on the segmentation model’s output. In detail, the electronic processing resources 2 are configured to convert (block

[0069] 1 IB) the segmentation model’s output into the partial point cloud representation C of the detected object 15 based on the image 5 to be analysed. More in detail, the electronic processing resources

[0070] 2 are configured to transform an RGB-D image into a point cloud by converting depth data into 3D coordinates and mapping corresponding colours from the RGB image. In more detail, by using the camera’s intrinsic parameters, the electronic processing resources 2 are configured to project each pixel in the depth map into 3D space, creating points that represent the scene's geometry; the RGB image adds colour information to these points, resulting in a detailed and visually enriched representation. This process combines spatial structure and visual appearance, producing a point cloud that captures the 3D shape and colour of the environment or object for applications like 3D modelling and object recognition.

[0071] It should be noted that, since the image 5 to be analysed is determined by a potentially single - view sensor 4, the computed partial point cloud representation B may suffer from occlusion. In real-world object grasping scenarios, partial observations of 3D data are often encountered. Achieving a complete 3D scan of an object necessitates a feasible camera path capable of executing a full 360° trajectory around the object. However, such atrajectory demands significant energy and may not be viable for the robot 17, as its motion can be constrained by specific paths or limited viewpoints. Nevertheless, generating an accurate grasp proposal requires a precise estimation of the object's complete 3D geometry.

[0072] This can result in generating unreliable or inaccurate grasp poses 9 due to incomplete data caused by occlusions and limited visibility; particularly because the object grasping system 1 may not have a comprehensive understanding of the object's 15 geometry. As a result, the grasp may be improperly positioned or oriented, leading to unsuccessful or unstable grasp attempts and potentially causing the detected object 15 to slip or be dropped.

[0073] To solve this issue, the electronic processing resources 2 are further configured to predict (block 11C) missing, or non-visible, parts Cl of the detected object 15 in the partial point cloud representation B by executing a point cloud completion neural network on the partial point cloud representation B. The point cloud completion neural network being trained on various point cloud representations to predict (block 11C) missing, or non-visible, parts Cl of an object in an input point cloud representation. In particular, the point cloud completion neural network is configured to receive the partial point cloud representation B in input, and to predict (block 11C) the missing, or non-visible, parts Cl of the detected object 15 based on the partial point cloud representation B. The point cloud completion neural network is conveniently trained on large datasets, allowing it to generalize well to different objects and environments. In particular, the point cloud completion neural network is designed to use hierarchical structures to capture geometric information at multiple scales, providing a robust solution for point cloud completion. Alternatively, the point cloud completion neural network is configured to convert point clouds into grid formats and uses 3D convolutional neural networks (CNNs) to predict the completed structures. In a way of example, the point cloud completion neural network can be a PCN (Point Completion Network), an AtlasNet (Atlas Network) designed to generate surfaces using parameterized elements, a PoinTr (Point Transformer) configured to employ transformers to capture complex dependencies, or a SnowflakeNet (Snowflake Network), which also utilizes transformers for intricate reconstructions. Additionally, the point cloud completion neural network eventually can be a generative model like PointGAN (Point Generative Adversarial Network) or TopNet (Topological Network).

[0074] According to said preferred aspect of the present invention, the electronic processing resources 2 are configured to compute (block 11) the representation C of the detected object 15 by including (block 11C) said predicted missing, or non -visible, parts Cl in the partial point cloud representation B of the detected object 15. In particular, the representation C of the detected object 15 is a completed point cloud representation C including the partial point cloud representation B of the detected object 15 and the predicted missing, or non-visible, parts Cl. In detail, the point cloud completion neural network is configured to compute and output (block 11) a representation C of the detected object 15 by including (block 11C) the predicted missing, or non-visible, parts Cl in the partial point cloud representation B of the detected object 15. Alternatively, in detail, the electronic processing resources 2 are configured to receive the predicted missing, or non- visible, parts Cl from the point cloud completion neural network; and to compute (block 11) said representation C of the detected object 15 by including (block 11C) the predicted missing, or non- visible, parts Cl in the partial point cloud representation B of the detected object 15.

[0075] The electronic processing resources 2 are further configured to compute (block 8) one or more grasp poses 9 by executing a geometry-based algorithm 8A on the representation C of the detected obj ect 15 if it is classified as simple geometry obj ect; and by executing a trained machine learning model 8B on the representation C of the detected object 15 if it is classified as complex geometry object. The grasp poses 9 represent respective target positions and orientations for the robotic arm’s gripper 18 to grasp the detected object 15.

[0076] In detail, the electronic processing resources 2 are configured to select (block 7) between the two grasp pose 9 generation strategies, or algorithms, based on the object geometry class (simple geometry object or complex geometry object) determined by the geometry classifier 6. In more detail, this system enhances the accuracy and reliability of robotic grasping by switching (block 7) between geometric -based (block 8A) and machine learning (block 8B) based grasp pose 9 generation methods. In detail, the electronic processing resources 2 are configured to input into the machine learning model 8B the representation C of the detected object 15 to receive one or more computed grasp poses 9 for the detected object 15. In more detail, the machine learning model 8B is trained on several object representations to become configured to compute (block 8) one or more grasp poses 9 based on an input representation of an object. More specifically, the machine learning model 8B is trained on a dataset of point cloud representations representing different objects. In more detail, the machine learning model 8B is configured to analyze the shape of the detected object 15 in the input representation, in detail the point cloud representation, to find potential grasping points. In particular, the machine learning model 8B is configured to identify one or more features (like curvature, edges, and flat surfaces) on the input representation and to determine candidate grasp poses based on said identified features. In particular, the machine learning model 8B is a convolutional neural network. For example, said machine learning model 8B can be one of Dex-Net (Dexterity Network), GraspNet, GGCNN (Generative Grasping CNN), or VGN (Volumetric Grasping Network). Preferably, the geometry-based algorithm 8A is designed to cause, when executed, the electronic processing resources 2 to become configured to compute (block 12) a three-dimensional bounding box 12A having a plurality of faces 14 and encapsulating, in particular constraining, the representation C of the detected object 15 based on the representation C of the detected object 15. In detail, Figure 4 schematically shows a functional block diagram of a geometry-based algorithm 8A according to the preferred aspect of the present invention. In detail, each face 14 of the three-dimensional bounding box 12A constrain a part of the detected object 15. The three-dimensional bounding box 12A (in detail being an oriented bounding box) is preferably represented in a three- dimensional local reference system; which is in detail a three-dimensional Cartesian coordinate system. The three-dimensional local reference system is conveniently a local 3D coordinate system that defines the three-dimensional bounding box's 12A position, orientation, and size. In detail, it has a centre point (the origin of the three-dimensional bounding box 12A), three perpendicular axes (X, Y, Z) that can be rotated relative to the global coordinate system, and extents along each axis that represent half the three-dimensional bounding box's 12A dimensions. This local frame allows the three-dimensional bounding box 12A to tightly align with the detected object 15 it encloses, even if the object is rotated.

[0077] In particular, three-dimensional bounding box 12A exhibits six different faces 14. Preferably, the electronic processing resources 2 are configured to fit (block 12) the three- dimensional bounding box 12A around the detected object 15 based on the representation C of the detected object 15; in more detail, and to orient the three-dimensional bounding box 12A around the detected object 15. More specifically, the electronic processing resources 2 are further configured to determine data indicative of the extents of the detected object 15 in three spatial dimensions (in detail x, y, and z) of the three-dimensional local reference system in order to compute (block 12) the three-dimensional bounding box 12A. In detail, the determined data indicative of the extents comprises a minimum extent and a maximum extent of the detected objects 15 in the three spatial dimensions. In addition, the electronic processing resources 2 are preferably configured to represent the three-dimensional bounding box 12A in the three- dimensional local reference system by orienting the three-dimensional bounding box 12A to align with the principal axes of the detected object 15.

[0078] In addition, the electronic processing resources 2 are configured to determine (block 13) one or more possible grasp points P on the faces 14 of the three-dimensional bounding box 12A. In detail, the electronic processing resources 2 are configured to determine (block 13), for each face 14 (in detail for each of the six faces 14) of the three-dimensional bounding box 12 A, a possible grasp point P, in particular the centre, located on said face 14. The possible grasp points P are positions (in detail defined by spatial coordinates) located on the faces 14 of the three- dimensional bounding box 12A, and correspond preferably with the centres of the respective faces 14 because they are geometrically significant locations that can provide stable and balanced grasping positions. In addition, the electronic processing resources 2 are configured to compute the dimension of said face 14. In a way of example, the possible grasp points P are defined as follows: is the centre of the bottom face 14 of the three-dimensional bounding box 12A, P2is the centre of the top face 14, P3is the centre of the front face 14, P4is the centre of the back face 14, P5is the centre of the left face 14, and P6is the centre of the right face 14. Alternatively, the electronic processing resources 2 are configured to determine (block 13), for each face 14 of the three-dimensional bounding box 12A, a possible grasp point P, optionally not a centre, located on said face 14 based on one or more determined three-dimensional bounding box 12A features; for example, the dimensions of the faces 14 of the three-dimensional bounding box 12A.

[0079] Additionally, the electronic processing resources 2 are configured to compute (block 8) the grasp poses 9, each relating to a three-dimensional bounding box’s 12A face 14, based on the determined possible grasp points P located on the respective faces 14 of the three-dimensional bounding box 12A. In particular, each of the computed grasp poses 9 include or represent at least one of the possible grasp points P. In more detail, the electronic processing resources 2 are configured to compute (block 8) the grasp poses 9 to include the corresponding possible grasp points P and to include data indicative of an orientation to be assumed by the robotic arm’s gripper 18 to grasp the detected object 15.

[0080] In particular, to compute the data indicative of the orientation to be assumed, the electronic processing resources 2 are configured to compute a rotation matrix (in detail a roto -translation matrix) that represents the orientation and translation of the three-dimensional bounding box 12A relative to a three-dimensional global reference system (or world reference system) based on the possible grasp points P. In particular, the rotation matrix is a spatial transformation model (more in detail a roto-translation matrix) designed to align the three-dimensional local reference system with the three-dimensional global reference system; in particular, to allow the robot 17 to carry out the grasp operations in the working environment based on instructions received.

[0081] In more detail, to compute the rotation matrix, the electronic processing resources 2 are configured to determine vectors (in detail grasping translation vectors) along the axes of the three- dimensional bounding box 12A; and to compute the rotation matrix based on said determined vectors.

[0082] In detail, by executing the geometry-based algorithm 8A, the electronic processing resources 2 are further configured to compute a plurality of vectors along the axes of the three- dimensional local reference system based on pairs of the determined possible grasp points P. In more detail, the electronic processing resources 2 are configured to determine multiple pairs of possible grasp points P by coupling the grasp points P determined to be located on opposing faces 14 of the three-dimensional bounding box 12A. Subsequently, the electronic processing resources 2 compute each of the vectors along the axes based on the determined pairs of possible grasp points P. In detail, the electronic processing resources 2 compute each of the vectors along the axes by performing a mathematical operation, in particular a subtraction, between the possible grasp points P of a determined pair. In detail, the determined vectors along the axes of the bounding box 12A are the following: Ax= P6— P5, Ay= P2— P1, Az= P4— P3. In more detail, these vectors represent the directions from the left face 14 to the right face 14, the bottom face 14 to the top face 14, and the front face 14 to the back face 14, respectively.

[0083] Additionally, the electronic processing resources 2 are possibly configured to normalize (by executing a normalization algorithm) the computed vectors to obtain corresponding unit vectors. In detail, for each of the computed vectors, the electronic processing resources 2 are configured to compute the length, or norm, of the vector and to normalize it based on its length. More specifically, the electronic processing resources 2 are further configured to normalize a computed vector by dividing its components by the computed length. In particular, the normalized vectors are computed by using the following formulas:

[0084] Alternatively, the electronic processing resources 2 may also employ other normalization techniques, such as LI normalization, which involves dividing each component by the sum of the absolute values of the vector's components. The electronic processing resources 2 optionally can employ the Min-Max normalization, which scales the vector's components to a specified range, typically between 0 and 1.

[0085] In addition, the electronic processing resources 2 are configured to compute the rotation matrix based on the computed vectors. In particular, the electronic processing resources 2 are configured to stack the vectors, in detail the normalized vectors, as columns of the rotation matrix. In a way of example, the rotation matrix is defined as:

[0086] Wherein, R is the rotation matrix, and A , A and A are the and normalized vectors.

[0087] In detail, by accurately defining and normalizing the principal axes, the orientation can be robustly determined, ensuring consistent and correct alignment with the bounding box's 12A true geometric orientation.

[0088] The electronic processing resources 2, by executing the geometrical-based algorithm, are configured to compute (block 8) the grasp poses 9 based on the computed rotation matrix. In detail, the electronic processing resources 2 are configured to compute (block 8) each of the grasp poses 9 as representing the target position as a possible grasp point P located on a face 14 of the three-dimensional bounding box 12A, and the orientation as defined by the computed rotation matrix. In more detail, each grasp pose 9 consists of a possible grasp point P coupled with the computed rotation matrix. In particular, the orientation of the robotic arm’s gripper 18 should match the orientation of the bounding box 12A; therefore, the computed rotation matrix above will also define the gripper 18's orientation.

[0089] Alternatively, the electronic processing resources 2 are optionally configured to compute an orientation data based on the computed rotation matrix, and to compute (block 8) each of the grasp poses 9 to comprise a possible grasp point P coupled with said computed orientation data. If the gripper 18 is to grasp the object from a different face 14, such as the front face 14, the grasping point would be P3, and the orientation remains consistent with the rotation matrix. In detail, the alignment of the gripper 18's approach vector is adjusted accordingly.

[0090] To ensure the gripper 18 aligns correctly with the three-dimensional bounding box’s 12A face 14, the electronic processing resources 2 can define the approach direction (radial vector), tangent direction, and axial direction relative to the grasping face 14. For example, for the top face 14 (centered at P2) the radial vector points downwards towards the bottom face 14 ( Ay norm= and the tangent and axial directions are aligned with Az norm= —£-. By setting the approach of gripper 18, tangent, and axial directions to match these vectors, the gripper 18 can grasp the object securely and with the correct orientation. The final grasping position and orientation are the center point of the desired face 14 (for example P2for the top face 14) and the rotation matrix, respectively. In addition, the electronic processing resources 2 are conveniently configured to determine one or more desired faces 14 to grasp the detected object 15 using the robotic arm’s gripper 18 based on the representation C of the detected object 15 and / or based on the class of the detected object 15. For instance, if the gripper 18 is to grasp the top face 14 of the bounding box 12A, the grasping point would be P2.

[0091] In particular, the electronic processing resources 2 are configured to discard (by executing a filtering algorithm) one or more of computed grasp poses 9 in order to grasp the detected object 15 based on the remaining grasp poses 9; optionally, based on the desired faces 14 to grasp the detected object 15. In particular, after generating the grasp poses 9, the electronic processing resources 2 are configured to execute the filtering algorithm to improve the grasp success rate before sending the computed grasp poses 9 for the real robot grasp execution.

[0092] Conveniently, the electronic processing resources 2 are configured, by executing the filtering algorithm, to discard or to prevent the computation (block 8) of grasp poses 9 based on the shape of the detected object 15 and / or the dimension of the faces 14 of the three-dimensional bounding box 12A. In detail, the filtering algorithm is configured to cause, when executed, the electronic processing resources 2 to become configured to discard or to prevent the computation (block 8) of grasp poses 9 based on the shape of the detected object 15 and the dimension of the faces 14 of the three-dimensional bounding box 12A.

[0093] In detail, the electronic processing resources 2, in detail by executing the filtering algorithm, are configured to determine the dimension of the robotic arm’s gripper 18, and the dimension of a face 14, in detail of each face 14, of the three-dimensional bounding box 12A. In particular, the electronic processing resources 2 are configured to receive the robotic arm’s gripper 18 from electronic storing resources storing data indicating the dimension of the robotic arm’s gripper 18. In addition, the electronic processing resources 2 are conveniently configured to compute or receive the dimension of each face 14 of the three-dimensional bounding box 12A based on the latter. In particular, the electronic processing resources 2 are configured to compute the dimension of a face 14 of the three-dimensional bounding box 12A by determining the width (which is preferably the difference between the maximum and minimum x-coordinates of the vertices defining the face 14), and the height of that face 14 (which is preferably the difference between the maximum and minimum y-coordinates of the vertices defining the face). In a way of example, the dimension corresponds to the area of the face 14, and the electronic processing resources 2 are configured to compute the dimension of the face 14 as the product of the width and height.

[0094] Furthermore, the electronic processing resources 2 are configured to discard, or prevent the computation (block 8) of, grasp poses 9 based on the dimension of the faces 14 of the three- dimensional bounding box 12A. In particular, the electronic processing resources 2 are configured to discard, or prevent the computation (block 8) of, grasp poses 9 based on the dimension of the robotic arm’s gripper 18 and based on the dimension of the faces 14 of the three-dimensional bounding box 12A. In detail, if the dimension of the robotic arm’s gripper 18 is determined to be lower than the dimension of said face 14, the electronic processing resources 2 are configured to discard, or prevent the computation (block 8) of, each of the grasp poses 9 relating to said face 14 (in detail computed based on possible grasp points P located on said face 14). In detail, the electronic processing resources 2 are configured to discard the grasp poses 9 relating to said face 14. Optionally, the electronic processing resources 2 are configured to prevent the computation (block 8) of said grasp poses 9. This is because the larger geometry of the object, in detail of the face 14, compared to the gripper 18 may result in collisions between the object and gripper 18, causing the object to fall.

[0095] Optionally, but conveniently, the electronic processing resources 2 are further configured (by executing the filtering algorithm) to discard or to prevent the computation (block 8) of grasp poses 9 based on the shape of the detected object 15. In particular, the electronic processing resources 2 are further configured to determine data (in detail a three-dimensional representation) indicative of the shape of the part of the detected object 15 constrained by (or resting on) a face 14 of the three-dimensional bounding box 12A. In addition, the electronic processing resources 2, for each of the faces 14 of the three-dimensional bounding box 12A, are configured to determine that a grasping condition is satisfied or is not satisfied for said face 14 based on the data indicative of the shape of the part of the detected object 15 constrained by said face 14. In more detail, the electronic processing resources 2 are configured to discard or to prevent the computation (block 8) of grasp poses 9 based on the satisfaction of the grasping condition for the respective faces 14. More specifically, if the grasping condition is determined to be unsatisfied for a face 14 of the three-dimensional bounding box 12A, the electronic processing resources 2 are configured to discard or to prevent the computation (block 8) of grasp poses 9 relating to said face 14. In more detail, if the grasping condition is determined to be satisfied for at least a face 14, the electronic processing resources 2 are configured to determine that the feasible grasping condition is satisfied. Conveniently, the electronic processing resources 2 are configured to determine that the feasible grasping condition is unsatisfied if the grasping condition is determined to be unsatisfied for each face 14.

[0096] According to the preferred embodiment of the present invention, the electronic processing resources 2 are configured, in detail by executing the filtering algorithm, to discard, or prevent the computation (block 8) of, one or more of the grasp poses based on the primitive geometry class with which the detected object 15 is determined to be classifiable; in particular, by means of the geometry classifier 6. For example, the electronic processing resources 2 are configured to determine whether the detected object 15 is classifiable (or classified) or is not classifiable (or classified) with a cylinder by executing the geometry classifier 6.

[0097] In detail, the electronic processing resources 2, in detail by executing the filtering algorithm, are configured to determine (or select) one or more unfeasible grasping faces between the faces 14 of the three-dimensional bounding box 12A based on the primitive geometry class with which the detected object 15 is determined to be classifiable. In particular, one or more (optionally all) of the primitive geometry classes are associated with one or more unfeasible grasping faces which are not graspable by the gripper 18. For example, the unfeasible grasping face of the cylinder is the top face 14 of the three-dimensional bounding box 12A. In addition, for each of said unfeasible grasping faces, the electronic processing resources 2 are configured to discard, or prevent the computation (block 8) of, each of the grasp poses 9 relating to said unfeasible grasping face. In a way of example, the electronic processing resources 2 are configured to discard, or prevent the computation (block 8) of, each of the grasp poses 9 relating to the top face 14 if the detected object 15 is determined to be classifiable with a cylinder or with a cone or with an ellipsoid.

[0098] In detail, the electronic processing resources 2 are configured to discard the grasp poses 9 relating to said unfeasible grasping face. Optionally, the electronic processing resources 2 are configured to prevent the computation (block 8) of said grasp poses 9. This is because grasping a cylinder from the top face 14, or a cone, or other primitive geometries from the top face 14, can be unstable due to the limited surface area for the gripper 18. In more detail, the electronic processing resources 2 are configured to determine at least if a face 14 of the three-dimensional bounding box 12A is a top or a bottom face 14.

[0099] In addition, the electronic processing resources 2 are configured to control (block 10) operation of the robotic arm’s gripper 18 based on the computed grasp poses 9 to cause the robotic arm’s gripper 18 to grasp the detected object 15. In detail, the electronic processing resources 2 are configured to control (block 10) the operation of the robotic arm and to control the operation of the robotic arm’s gripper 18 based on the computed grasp poses 9 (in detail the remaining grasp poses 9).

[0100] In detail, the electronic processing resources 2 are configured to transmit (block 10) the remaining grasp poses 9, for example as a list, to the robot 17 to execute the grasp poses 9; in detail that are determined to be feasible for the robot's 17 trajectory (to be followed by the robot 17). This ensures they account for potential collisions with other objects, adherence to joint limits, and the robot's 17 trajectory constraints. Additionally, the electronic processing resources 2 are conveniently configured to compute (block 8) a collision-free trajectory to be followed by the robot 17; and to control (block 10) the robotic arm’s operation to cause the robotic arm to follow said computed trajectory. In detail, the electronic processing resources 2 are configured to compute (block 8) the collision -free trajectory as the optimal path from the current position of the robotic arm to the grasp pose 9, ensuring smooth and efficient movement. Conveniently, the electronic processing resources 2 are configured to compute (block 8) the collision-free trajectory based on the robot's 17 kinematic and dynamic constraints, as well as the environment's spatial configuration. In particular, computing a collision-free trajectory for a robotic arm involves determining a smooth, efficient path from its current position to a target grasp pose while avoiding obstacles and adhering to kinematic and dynamic constraints. This process may begin with modelling the robot's kinematics and dynamics and representing the environment, including static and dynamic obstacles. The electronic processing resources 2 are possibly configured to implement path planning techniques, such as sampling-based methods (Probabilistic Roadmap Method - PRM, Rapidly-exploring Random Tree - RRT, RRT*), optimization-based approaches (Covariant Hamiltonian Optimization for Motion Planning - CHOMP, Stochastic Trajectory Optimization for Motion Planning - STOMP, Trajectory Optimization - TrajOpt) to compute a geometrical path (or trajectory) in the robot's configuration space (C-space). The electronic processing resources 2 are optionally configured to detect collisions, using tools like the Flexible Collision Library (FCL) or Bullet Physics, ensuring the path is safe by checking for intersections with obstacles. Trajectory optimization refines the path into a smooth, time -parameterized trajectory considering velocity, acceleration, and constraints, often employing spline interpolation or dynamic programming. The electronic processing resources 2 are then configured to cause the collision-free trajectory to be followed; in detail, to be executed by a robotic arm's motion controller with real-time feedback and re-planning capabilities to adjust for dynamic environments or deviations. For example, frameworks like the Open Motion Planning Library (OMPL) integrate these components, while advanced techniques such as machine learning enhance planning efficiency and adaptability in complex, shared spaces.

[0101] Moreover, the electronic processing resources 2 are possibly configured to continuously (and in real time) monitor and adjust (block 10) the gripper 18's position during the grasping operation. This real-time control (block 10) can ensure a secure and stable grasp, adapting to any unexpected changes in the object's position or orientation. Additionally, the electronic processing resources 2 may be configured to adjust (block 10) the gripper 18's force in real-time to accommodate variations in the object's material properties. Furthermore, the electronic processing resources 2 can generate (block 10) corrective actions if the object slips or shifts during the grasping process, maintaining the integrity of the operation.

[0102] In detail, the electronic processing resources 2 can control the robotic gripper 18 (in more detail the grippers fingers 19) to adjust its force in real-time to handle diverse materials. In detail, the electronic processing resources 2 are configured to determine, for example to receive, data indicative of the material of the detected object 15 and to control the robotic gripper’s 18 operation based on said data indicative of the material. In more detail, the electronic processing resources 2 are configured to determine and to transmit commands to control the operation of the robotic gripper 18; wherein, said commands are determined based on the data indicative of the material. In a way of example, the electronic processing resources 2 are configured to receive the commands, to control the operation of the robotic gripper 18, determined to be associated with the data indicative of the material. For fragile items like glass or eggs, the electronic processing resources 2 conveniently can cause the robotic gripper’ s 18 fingers 19 to reduce pressure to prevent breakage ; while for heavier and / or rigid objects like metal blocks, cause the robotic gripper’s 18 fingers 19 to increase force for a secure grip. In detail, soft items like bread or foam necessitate gentle handling to avoid deformation, whereas slippery materials like plastic bottles need sufficient force to prevent slipping without causing damage. For mixed-material objects, like bubble-wrapped boxes, the gripper 18 is conveniently configured to apply varied pressure across surfaces. In addition, elastic materials like balloons demand light, adaptive grips to prevent overstretching or popping. In detail, these adjustments use sensors (in detail force, tactile, vision sensors) and control algorithms to optimize grip strength and stability.

[0103] The present solution described herein offers several advantages.

[0104] In detail, the present solution enables the robot 17 to effectively grasp detected objects 15, either having a simple or complex geometry, within a working environment 16. In more detail, the present solution allows to improve the automatic grasp success rate for simple geometrical object shapes and effectively handles objects with complex geometries. The Applicant notes that, by classifying objects based on their geometrical type and selecting the appropriate grasping method, the object grasping system 1 ensures secure and accurate grasps. In detail, the Applicant notes that this hybrid approach reduces errors, increases efficiency, and is highly versatile. More specifically, the Applicant notes that the proposed object grasping system 1 is a valuable tool for handling a wide range of objects with varying shapes and sizes.

[0105] In addition, the object grasping system 1 can classify objects as simple geometry objects even if they are complex geometry objects consisting of an aggregation of several simple geometry objects. One key advantage of this method is its ability to accurately classify objects with simple geometries, even if they bear resemblance to more complex shapes. The Applicant notes that this feature allows the object grasping system 1 to execute the geometry-based algorithm 8A in this scenario as well, enabling a more precise and accurate generation of grasp poses 9 on each segmented part of the object separately.

[0106] Furthermore, the Applicant notes that the preferred aspect of the present invention allows to compute (block 8) more accurate grasp poses 9. In particular, the implementation of this aspect of the present invention allows to use effectively a single-view camera which is more desirable due to its cost-effectiveness, simplicity, and ease of deployment. In detail, a single-view camera requires less space and computational power, making it ideal for environments with space constraints and limited resources. Additionally, single-view setups avoid the complexity of data fusion and synchronization issues that arise with multiple cameras. In particular, the preferred aspect of the present invention allows to compute a complete point cloud representation C of the detected object 15, by predicting (and including in the representation) missing or non-visible parts Cl of the detected object 15. Figure 5 schematically shows a functional block diagram representing a first example of grasp pose computation (block 8) according to the preferred aspect of the present invention and a second example of grasp pose computation (block 8) performed according to a distinct aspect of the present invention. In detail, Figure 5 illustrates the impact of point cloud completion on the computation (block 8) of grasp poses 9 on a cylinder. In more detail, the represented arrows indicate the positions of the grasp poses 9. Figure 5 compares the grasp poses generated by applying the geometric-based algorithm to a partial point cloud (top of the Figure 5) and to a completed point cloud representation C (bottom of the Figure 5).

[0107] It can be observed in Figure 5 that the grasp poses 9 generated from partial point cloud data are inaccurate. This inaccuracy arises because the partial point cloud does not fully capture the object's geometry, leading to a lower success rate due to collisions with the object. In particular, the grasp poses 9 computed on the side and top are close to the object's edge, and executing these poses would result in collisions between the object and the gripper 18. Conversely, the grasp poses 9 computed from the completed point cloud representation C are more precise due to the accurate estimation of the object's geometry which result in successful grasp execution.

[0108] In fact, for both the partial and completed point clouds, the predefined grasp location is on the right side of the object. However, according to the example represented in Figure 5, due to the missing geometric information in the partial point cloud, the generated grasp pose fails to represent a secure location for picking the object, leading to a collision between the gripper fingers and the object. In contrast, the completed point cloud representation C, obtained by passing the partial input through the point cloud completion neural network, provides an estimation of the object’s full geometry, enabling the generation of a secure and accurate grasp pose.

[0109] In addition, the object grasping system 1 can enhance productivity, streamlines operations, and reduces downtime and manual intervention, in several industrial applications such as manufacturing, warehousing, healthcare, agriculture, and retail. Additionally, the flexibility and accuracy of the object grasping system 1 make it useful in many fields beyond its primary industrial applications. In particular, in manufacturing, the object grasping system 1 can enhance the handling of different objects during loading and unloading, thereby improving transport and logistics efficiency. In space exploration, the object grasping system 1 can enable robots 17 to manage tools and materials in space, assisting with construction and maintenance tasks on space stations or other planets. Furthermore, in disaster response, the object grasping system 1 can be employed in search and rescue operations, allowing robots 17 to move and handle debris and obstacles of various shapes and sizes, thereby making rescue missions safer and more effective. These additional applications demonstrate the wide-ranging impact and utility of the object grasping system 1 and software in various challenging environments.

Claims

CLAIMS1. Object grasping software (3) for enabling a robotic arm’s gripper (18) to grasp various objects; the object grasping software (3) being loadable in, and executable by, electronic processing resources (2) and designed to cause, when executed, the electronic processing resources (2) to become configured to: receive an image (5) to be analysed; detect an object (15) represented in the received image (5) to be analysed; and compute (block 11) a representation (C) of the detected object (15); classify, by means of a geometry classifier (6), the detected object (15) either as a simple geometry object or a complex geometry object; wherein, a simple geometry object and a complex geometry object exhibit distinct geometric features; compute (block 8) one or more grasp poses (9), representing respective target positions and orientations for the robotic arm’s gripper (18) to grasp the detected object (15), by executing a geometry-based algorithm (8A) on the representation (C) of the detected object (15) if it is classified as simple geometry object; and by executing a trained machine learning model (8B) on the representation (C) of the detected object (15) if it is classified as complex geometry object; and control (block 10) operation of the robotic arm’s gripper (18) based on the computed grasp poses (9) to cause the robotic arm’s gripper (18) to grasp the detected object (15).

2. Object grasping software (3) according to claim 1, and designed to cause, when executed, the electronic processing resources (2) to become configured to:- compute (block 1 IB) a partial point cloud representation (C) of the detected object (15) by performing (block 11 A) a segmentation of the detected object (15) on the received image (5) to be analysed;- predict (block 11C) missing, or non- visible, parts (Cl) of the detected object (15) in the partial point cloud representation (B) by executing a point cloud completion neural network on the partial point cloud representation (B); the point cloud completion neural network being trained on various point cloud representations to predict missing, or non-visible, parts (Cl) of an object in an input point cloud representation; and- compute (block 11) the representation (C) of the detected object (15) by including (block 11C) said predicted missing, or non -visible, parts (Cl) in the partial point cloud representation (B) of the detected object (15).

3. Object grasping software (3) according to claim 1 or 2, and further designed to cause, when executed, the electronic processing resources (2) to become configured to classify the detected object (15) as simple geometry object if the detected object (15) is determined to comprise one or more simple geometry objects.

4. Object grasping software (3) according to any one of the preceding claims, wherein the geometry classifier (6) is configured to:- determine whether the detected object (15) is classifiable or is not classifiable with one of a plurality of primitive geometry classes; in particular, the primitive geometry classes comprising cylinders, cuboids, spheres, cones, and ellipsoids; and- classify said detected object (15) as simple geometry object if it is determined that it is classifiable with a primitive geometry class, and to classify the detected object (15) as complex geometry object otherwise.

5. Object grasping software (3) according to claim 4, and designed to cause, when executed, the electronic processing resources (2) to become configured to discard, or prevent the computation (block 8) of, one or more of the grasp poses (9) based on the primitive geometry class with which the detected object (15) is determined to be classifiable.

6. Object grasping software (3) according to any one of the preceding claims, wherein the geometry-based algorithm (8A) is designed to cause, when executed, the electronic processing resources (2) to become configured to:- compute (block 12) a three-dimensional bounding box (12A) having a plurality of faces (14) and encapsulating the representation (C) of the detected object (15) based on the representation (C) of the detected object (15);- determine (block 13), for each face (14) of the three-dimensional bounding box (12A), a possible grasp point (P), in particular the centre, located on said face (14); and- compute (block 8) the grasp poses (9), each relating to a three-dimensional bounding box’s (12A) face (14), based on the determined possible grasp points (P) located on the respective faces (14) of the three-dimensional bounding box (12A).

7. Object grasping software (3) according to claim 6. wherein the three-dimensional bounding box (12A) is represented in a three-dimensional local reference system, and wherein the geometry-based algorithm (8A) is further designed to cause, when executed, the electronic processing resources (2) to become configured to:- compute a rotation matrix that represents the orientation of the three-dimensional bounding box (12A) relative to a three-dimensional global reference system based on the determined possible grasp points (P); and- compute (block 8) the grasp poses (9) based on the computed rotation matrix.

8. Object grasping software (3) according to claim 7, wherein the geometry-based algorithm (8A) is further designed to cause, when executed, the electronic processing resources (2) to become configured to compute (block 8) each of the grasp poses (9) as representing the target position as a possible grasp point (P) located on a face (14) of the three-dimensional bounding box (12A), and the orientation as defined by the computed rotation matrix.

9. Object grasping software (3) according to any one of the claims 6 to 8, and designed to cause, when executed, the electronic processing resources (2) to become configured to:- determine the dimension of the robotic arm’s gripper (18), and the dimension of a face (14) of the three-dimensional bounding box (12A); and- if the dimension of the robotic arm’s gripper (18) is determined to be lower than the dimension of said face (14), discard, or prevent the computation (block 8) of, each of the grasp poses (9) relating to said face (14).

10. Object grasping robot (17) comprising:- a robotic arm comprising at least one gripper (18) configured to grasp objects; and- electronic processing resources (2) storing, and configured to execute, the object grasping software (3) according to any one of the preceding claims.

Images