Processes for automated handling of products within a plant

Abstract

The present invention relates to a plant (100) including a workstation (32a) with a first robotic arm (34) and a plurality of gripping tools (45), and a process for automated handling of products (20) within the plant (100). The process provides for randomly placing products (20) on a service plane (37) of the workstation (32a), acquiring images of the products (20) placed on the service plane (37), and processing the images to obtain geometric characteristics of each of the products (20) placed on the service plane (37). For each of the products (20), the process also provides for identifying a corresponding gripping tool (45) from the plurality of gripping tools (45) based on the respective obtained geometric characteristics, associating the identified corresponding gripping tool with a first robot arm (34), moving the first robot arm (34) to grip the product (20) with the corresponding gripping tool (45), moving the gripped product (20) from the service plane (37) to a work station (39) of the plant (100), and releasing the product (20) at the work station (39) with the corresponding gripping tool (45).

Description

[Technical Field]

[0001] The present invention relates to a plant and a process for the automated handling of products within the plant. [Background technology]

[0002] In the context of the logistics of production companies, automated handling of products between different work stations in a plant is particularly important.

[0003] For example, especially in medium to large enterprises with a medium to high level of automation, products may be moved between the aforementioned work stations by automated vehicles, and / or where products have to be temporarily stored in a storage warehouse in anticipation of their future use, automated warehouses may be provided, with the insertion of products into the storage warehouse and the pick-up of such products from the storage warehouse being controlled by special warehouse management software.

[0004] In many cases, even in the presence of automated systems, the product is still handled by trained operators at several work stations.

[0005] For example, even when automated warehouses exist, delivery of products to storage warehouses is performed manually by operators who pick up products from pick-up stations located near the storage warehouse and insert them into appropriate collection trays intended to be placed in predetermined locations within the storage warehouse.

[0006] The applicant has considered automating some of the tasks that are typically performed manually by an operator, such as, for example, the delivery of products to an automated warehouse, and has provided for performing the aforementioned tasks using a robotic arm.

[0007] However, the applicant recognizes that the aforementioned products typically have widely different shapes and that even identical products may be randomly positioned on the surface from which they must be picked up, thus presenting the products in very different spatial arrangements to the robotic arm that must pick them up.

[0008] In this regard, the Applicant has recognized the need to provide a solution that allows a robotic arm to pick up a product safely and stably, while maintaining sufficient grip stability during its handling. Summary of the Invention

[0009] The present invention therefore relates in a first aspect to a process for the automated handling of products in a plant.

[0010] Preferably, the plant comprises a workstation, the workstation comprising a first robotic arm and a plurality of gripping tools.

[0011] Preferably, the products are randomly placed on the service plane of the workstation.

[0012] Preferably, an image is acquired of the product positioned on the servicing plane.

[0013] Preferably, the image is processed to obtain geometric characteristics of each of the products positioned on the service plane.

[0014] Preferably, for each of the products, a corresponding gripping tool is identified from the plurality of gripping tools based on the geometric characteristics obtained.

[0015] Preferably, for each of said products, an identified corresponding gripping tool is associated with the first robotic arm.

[0016] Preferably, for each of said products, the first robotic arm is moved to grip the product with a corresponding gripping tool and move the gripped product from said service plane to a work station of the plant.

[0017] A work station may be adjacent to the service plane.

[0018] Preferably, for each of said products, the product is released at said work station by a corresponding gripping tool.

[0019] In a second aspect, the invention also relates to a plant.

[0020] Preferably, the plant comprises a workstation.

[0021] Preferably, the workstation comprises a first robotic arm.

[0022] Preferably, the workstation comprises a service plane.

[0023] Preferably, the plant comprises a work station.

[0024] A work station may be adjacent to the service plane.

[0025] Preferably, the workstation includes a plurality of gripping tools positioned near the servicing plane.

[0026] Preferably, the workstation comprises an image capture device configured to capture images of products positioned on the servicing plane.

[0027] Preferably, the workstation comprises a computer configured to process said images to obtain geometric characteristics of each of the products positioned on the service plane.

[0028] Preferably, for each of the products, the computer is configured to identify a corresponding gripping tool from the plurality of gripping tools based on geometric characteristics respectively obtained by processing the image.

[0029] Preferably, for each of said products, said computer is configured to associate a corresponding gripping tool with a first robotic arm.

[0030] Preferably, for each of the products, the computer is configured to move a first robotic arm to grasp the product with a corresponding grasping tool, move the grasped product from the service plane to the work station, and release the product at the work station with the corresponding grasping tool.

[0031] By providing a plurality of gripping tools suitable for picking up products with different geometric characteristics, and by virtue of the fact that each product is handled by a gripping tool associated with the robot arm based on the geometric characteristics that each product has in the acquired image, it is possible to safely and stably handle products with different geometric characteristics by the same robot arm, thus achieving the desired automation.

[0032] The present invention can have at least one of the preferred features described below, either individually or in combination.

[0033] Preferably, the process is computer implemented.

[0034] Preferably, the process is implemented by a suitable algorithm, which is preferably updated by special machine learning and / or statistical algorithms.

[0035] For each of the products placed on the service plane, identifying a respective gripping tool of the plurality of gripping tools includes: accessing a database containing instances of product data for a plurality of predetermined reference products based on the geometric characteristics obtained by processing the image, wherein the product data includes the geometric characteristics of the reference products, and each instance of product data is associated with one of the plurality of gripping tools, each of which is associated with a respective score indicating a probability of successful gripping; retrieving from the database instances of product data having geometric characteristics corresponding to the geometric characteristics obtained by processing the image; identifying the corresponding grasp tool from grasp tools associated with the retrieved instances based on a score associated therewith; Includes:

[0036] Preferably, the geometric characteristics include at least one of shape, size, volume, center of gravity, and spatial location relative to a service plane.

[0037] Preferably, the identification of the corresponding grasp is performed by taking into account the working area available to the first robotic arm in the workstation.

[0038] Preferably, the identification of the corresponding gripping tool is performed by taking into account the spatial arrangement of the product relative to the service plane.

[0039] Preferably, identifying the corresponding gripping tools includes identifying optimal pairs formed by the corresponding gripping tools and the respective corresponding gripping modes.

[0040] Preferably, each instance of product data is associated in the database with one of the plurality of gripping tools, the gripping tool being associated with a plurality of gripping modes, and each pair formed by a gripping tool and one of the plurality of gripping modes being associated with a score indicating the probability of successful gripping for the pair.

[0041] Preferably, the identification of optimal pairs is performed between the grasping tools and their associated grasping modes in instances retrieved from the database based on a score associated with each pair.

[0042] Preferably, the identification of the optimal pair is also performed by taking into account the working area available to the first robotic arm within the workstation.

[0043] Preferably, the identification of the optimal pair is also performed by taking into account the spatial arrangement of the products relative to the service plane.

[0044] Preferably, identifying the optimal pair is performed by selecting from the pair associated with the highest score the one that allows the first robotic arm to grasp and handle the product within the working area without encountering any obstacles.

[0045] Preferably, the gripping mode defines at least one gripping point on the product.

[0046] Preferably, the gripping mode defines operating parameters adapted to drive the gripping tool.

[0047] Preferably, at least one characteristic parameter of a product placed on the service plane is obtained.

[0048] Preferably, in addition to the geometric characteristics, the product data in the database includes at least one characteristic parameter of the product of a predetermined plurality of reference products.

[0049] The characteristic parameters can be selected from the weight, material, color and surface characteristics of the product.

[0050] In a preferred embodiment, if the database lacks an instance of product data for one of the products placed on the service plane having geometric properties corresponding to the geometric properties obtained by processing the image, identification of the corresponding gripping tool from the plurality of gripping tools is performed by real-time processing of the acquired image.

[0051] In a preferred embodiment, if the database lacks an instance of product data for one of the products placed on the service plane that has geometric properties corresponding to those obtained by processing the image, the identification of the best match is performed by real-time processing of the acquired image.

[0052] In a preferred embodiment, the service plane is at least partially elastically deformable.

[0053] In a preferred embodiment, the service plane is operational.

[0054] Preferably, the servicing planes are actuatable by actuators associated with the servicing planes that can be driven independently of each other.

[0055] Preferably, the actuators can be driven independently of each other in terms of stroke and actuation frequency.

[0056] Preferably, prior to carrying out said real-time processing of the acquired images, the servicing plane is operated so as to move at least some of the products placed thereon.

[0057] The captured images are then preferably processed again to obtain updated geometric characteristics of each of the products placed on the service plane.

[0058] Thereafter, it is preferable to access the database again based on the updated geometric characteristics to retrieve instances of product data having geometric characteristics corresponding to the updated geometric characteristics, and identify the corresponding gripping tool (or the best-fit pair) from the gripping tools (and gripping modes) associated with the instances thus retrieved from the database based on the score associated therewith.

[0059] Preferably, said real-time processing of the captured images is performed when the database lacks an instance of product data having geometric properties corresponding to the updated geometric properties. Preferably, the image capture device is located above said servicing plane.

[0060] The image capture device may comprise a 3D vision system.

[0061] The image capture device may comprise a camera.

[0062] For example, the image capture device may use range imaging techniques adapted to provide, for each point in the captured image, information regarding the spatial location of the point.

[0063] The image capture device may be configured to capture a point cloud or a pair of 2D images consisting of a 2D intensity image and a 2D depth image.

[0064] It should be noted that a brightness image is intended to be a 2D image defined by a set of brightness (or intensity) values ​​associated with image points in the X,Y plane of the frame of reference of the image acquisition device, whereas a depth image or depth map is a 2D image or map defined by a set of distance values ​​associated with image points relative to a predetermined viewpoint in said X,Y plane.

[0065] The image capture device may include, for example, a stereo camera, a TOF camera (TOF stands for "time of flight"), or a structured light camera.

[0066] In one embodiment, the products randomly placed on the service plane are identical to one another. Identical products are intended as a single type of product, but may differ in geometric characteristics due to, for example, packaging (e.g., some products may be packaged and others may not).

[0067] Preferably, the products are placed in containers before being randomly placed on the service plane.

[0068] Preferably, randomly placing the products on a service plane includes picking up the containers by an automated transfer device, such as a second robotic arm.

[0069] Preferably, randomly placing the products on a service plane includes tilting or inverting the containers above the service plane by moving the automated transport device.

[0070] Preferably, the containers are arranged in a carrier tray which accommodates a plurality of containers, each container containing a plurality of identical products.

[0071] Preferably, prior to picking up the container by the automated transfer device, the carrier tray is transferred to a first conveyor adjacent the service plane.

[0072] Preferably, the transfer is performed by an autonomous vehicle.

[0073] Preferably, after transferring the transport tray to the first conveyor, and before picking up the container by the automated transfer device, the transport tray is transferred from the first conveyor to a detection station adjacent to the first conveyor.

[0074] Preferably, an image of the transport tray is captured by an image capture device provided at the detection station.

[0075] Preferably, the acquired image of the transport tray is processed to check the correct positioning of the tray and the plurality of containers contained therein against predetermined reference parameters.

[0076] Preferably, the correct spatial orientation of the carrier tray is checked against predetermined reference parameters by processing the acquired image of the carrier tray.

[0077] Preferably, the transport tray is locked in position within the detection station before an image of the transport tray is acquired.

[0078] Preferably, the gripping tool is selected from the group comprising a suction cup, a caliper, a soft caliper, a dosing hopper (funnel), a hook, and any combination thereof.

[0079] Preferably, a collection tray is placed within the work station prior to releasing each of the products at the work station.

[0080] Preferably, releasing each of the products at the workstation includes placing the product in the collection tray.

[0081] Preferably, after placing the product in a collection tray, the collection tray is picked up from the work station.

[0082] Preferably, the collection trays are stored in predetermined locations within the storage warehouse.

[0083] Preferably, the plant or workstation comprises one or more sensors configured to detect characteristic parameters of the products placed on the service plane selected from weight, surface properties, material and color.

[0084] Further features and advantages of the present invention will become more apparent from the following detailed description of preferred embodiments of the invention, given by way of illustrative and non-limiting examples, made with reference to the accompanying drawings, in which: [Brief explanation of the drawings]

[0085] [Figure 1] FIG. 1 shows a simplified schematic layout of a plant in which a process for automated handling of products according to the invention is implemented. [Figure 2] FIG. 1 is a plan view from above of a carrier tray used in the process of the present invention. [Figure 3] FIG. 3 is a side view of the carrier tray of FIG. 2. [Figure 4] FIG. 1 is a plan view from above of a package insert used in the process of the present invention. [Figure 5] FIG. 1 is a plan view from above of a collection tray used in the process of the present invention. [Figure 6] FIG. 6 is a side view of the collection tray of FIG. 5. [Figure 7] 3 shows the frame of the carrier tray of FIG. 2 analyzed by the process management software of the present invention. [Figure 8] FIG. 1 is a perspective view of the entrance area of ​​a storage warehouse used in the process of the present invention. [Figure 9] FIG. 9 is a plan view from above of the entrance area of ​​the storage warehouse of FIG. 8. [Figure 10] FIG. 9 is a perspective view of a portion of the entrance area of ​​the storage warehouse of FIG. 8. [Figure 11] FIG. 10 is a schematic diagram of an example of randomly placed products on a service plane with associated respective gripping points when the gripping tools are suction cups; [Figure 12] FIG. 2 is a schematic diagram of an example of a database that can be used to implement the process of the present invention. [Figure 13] FIG. 1 shows a schematic example block diagram of a possible implementation of the process of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0086] In FIG. 1, reference numeral 100 indicates an area of ​​a plant in which a process for the automated handling of products 20 (shown in FIG. 11) according to the invention is carried out.

[0087] The product 20 may be supplied from an external supplier, or from other areas of the same plant, or from other plants of the same company or group of companies. In a non-limiting example, the product 20 is a component or part of a packing machine, for example a cigarette packing machine.

[0088] The products 20 described above are first processed at the product receiving station 10. At this station, the products 20 are placed into containers 16 (FIG. 2) which are placed into the carrier trays 14.

[0089] In addition to the product receiving station 10, the plant area 100 is provided with a storage warehouse 30 where the aforementioned products 20 are stored for pick-up as required.

[0090] The storage warehouse 30 is preferably an automated warehouse.

[0091] Each product, or each of a plurality of identical products 20, present at product receiving station 10 is accompanied by an accompanying document 18, shown in Figure 4. This document 18 indicates the number and type of products 20 associated with it. This information is also contained in a product identification code 40 printed on the document 18 itself. Preferably, this product identification code 40 is an optical code that can be read by an optical reader, and more preferably, is a bar code.

[0092] The insert 18 may be prepared by an operator present at the product receiving station 10 or may be prepared directly by the supplier of the product 20 and delivered to the product receiving station 10 along with the product 20 itself.

[0093] In some embodiments as shown herein, a first service optical code 47 is also printed on the package insert 18, which identifies the possibility of handling the respective product 20 by the robotic arm 36 located at the robot entry station 32a (FIGS. 8-10) of the storage warehouse 30. However, embodiments are envisioned in which the package insert 18 does not have the aforementioned first service optical code 47.

[0094] The product receiving station 10 includes a plurality of product loading stations 12 where various operators place products 20 to be stored into their respective containers 16 and into the transport trays 14 .

[0095] The carrier trays 14 without containers 16 are transferred to the product loading station 12 for loading with containers 16 at least partially filled with products 20 .

[0096] The transfer of each transport tray 14 to the product loading station 12 can be performed after an operator calls an automated vehicle 50, which departs from a parking area 52 where multiple automated vehicles 50 are present and arrives at the product loading station 12. This call is made by the operator, for example, by activating a call button (not shown) specially provided at the product loading station 12.

[0097] 2 and 3, each carrier tray 14 has a substantially parallelepiped shape and includes a bottom wall 14a, four side walls 14b, and an upper periphery 15 opposite the bottom wall 14a that defines a top opening 14c that allows access to an essentially parallelepiped-shaped compartment 14d.

[0098] In the context of this specification and the appended claims, spatial references such as "upper," "top," "above," etc., and "bottom," "below," etc., should be understood to refer to the operational position of the transport tray 14 as shown in Figures 8-10 attached hereto, in which the transport tray 14 is resting by its bottom wall 14a.

[0099] Compartment 14d accommodates ten containers 16 in the non-limiting example of FIG.

[0100] The container 16 has a substantially parallelepiped shape.

[0101] The containers 16 are arranged side by side within the carrier tray 14. In the non-limiting example of Figure 2, the containers 16 are arranged in two rows of five containers 16 along each long side of the carrier tray 14.

[0102] Preferably, the container 16 is an open-fronted storage box.

[0103] An operator places products 20 into the carrier trays 14 and separates the products according to product type. Thus, each container 16 contains the same type of product 20. Containers 16 in the same carrier tray 14 may contain identical or even different products 20. Some containers 16 may contain a single product if the product is large.

[0104] The products 20 can be placed into their corresponding containers 16 before the containers 16 are placed in the carrier tray 14 or after the containers 16 are placed in the carrier tray 14 .

[0105] The product 20 is identified at the product receiving station 10 by reading the product identification code 40 printed on the package insert 18 .

[0106] After the product 20 is loaded into the container 16, the package insert 18 is also placed within the container 16 along with its associated product 20.

[0107] The product identification code 40 can be read before or after the product 20 is placed in the container 16 and before or after the container 16 is placed in the carrier tray 14 .

[0108] Each carrier tray 14 has a respective tray identification code 42 .

[0109] The tray identification code 42 is preferably located either on the upper periphery 15 of the carrier tray 14 or on the side wall 14b of the carrier tray 14 so that it is visible when the carrier tray 14 is viewed from above.

[0110] Preferably, the tray identification code 42 is an optical code that can be read by an optical reader, and more preferably a bar code.

[0111] Associated adjacent the tray identification code 42 is an alphanumeric code (not shown) that allows visual identification of the carrier tray 14 by an operator.

[0112] In the embodiment shown in FIG. 2, two tray identification codes 42 are provided on the upper periphery 15 of the carrier tray 14. These are located adjacent to one long side of the carrier tray 14, opposite the top opening 14c. This arrangement identifies a row into which the first five containers 16 should be placed, starting from the left (or from below, based on the position of the carrier tray 14 in FIG. 2) and moving to the right (or upward, based on the position of the carrier tray 14 in FIG. 2). The second row of containers 16 is arranged in the same order as the containers 16 in the first row. For example, referring to FIG. 7, ten containers 16 are arranged consecutively at positions numbered I through X, with position I located adjacent to the tray identification code 42 on the left (or below, based on the position of the carrier tray 14 in FIG. 7), positions II through V located alongside each other along the row between the two tray identification codes 42, and positions VI through X located adjacent to positions I through V, respectively, along the row adjacent to the row of positions I through V.

[0113] Before or after filling the carrier tray 14 with containers 16 at least partially filled with products 20 , the carrier tray 14 is identified by an operator by reading one of the tray identification codes 42 .

[0114] 2 and 3, the carrier tray 14 further includes a plurality of container positioning identification codes 46 on its upper periphery 15. Each of these codes 46 is positioned adjacent to a respective zone of the compartment 14d configured to receive a corresponding container 16.

[0115] Preferably, the container positioning identification code 46 is an optical code, more preferably a bar code, that can be read by an optical reader.

[0116] After each container 16 is placed in the transport tray 14, the location of that container 16 relative to the transport tray 14 (and therefore the location and amount of product 20 contained therein) is identified by reading the container positioning identification code 46 adjacent to that container 16.

[0117] 3, the transport tray 14 further comprises a first service optical code 47 that identifies the possibility of being emptied by the robot arm 36 provided at the robot entry station 32a of the storage warehouse 30 (FIGS. 8-10). This code 47 is identical to the first service optical code 47 that may be printed on the attached document 18.

[0118] In the embodiment shown, the carrier tray 14 also includes a second service optical code 47a which identifies that it can also be emptied manually by an operator.

[0119] The first service optical code 47 and the second service optical code 47a are located on the side wall 14b of the carrier tray 14 where the tray identification code 42 is also located.

[0120] Thus, the transport trays 14 shown in the accompanying drawings are suitable for being emptied either by the robotic arm 36 or manually. An operator determines the type of transport tray 14 to be emptied, robotically or manually, by reading either the first service optical code 47 or the second service optical code 47a with an optical reader. All containers 16 housed in the transport tray 14 for which the first service optical code 47 is read are emptied by the robotic arm, just as all containers 16 housed in the transport tray 14 for which the second service optical code 47a is read are emptied manually.

[0121] There may be carrier trays 14 that contain only the first service optical code 47. These trays 14 should be filled only with products 20 with package inserts 18 that also carry the first service optical code 47.

[0122] Additional carrier trays 14 may be provided that contain only the second service optical code 47a. These trays 14 should be filled only with products 20 with package inserts 18 that do not have the first service optical code 47.

[0123] Any discrepancy in reading, such as, for example, reading a first service optical code 47 that is on an insert 18 placed within a container 16 but not on the carrier tray 14 on which the container 16 containing that insert 18 and the corresponding product 20 is placed, or vice versa, or reading a first service optical code 47 on an insert 18 placed within a container 16 and a second service optical code 47a on the carrier tray 14 on which the container 16 containing that insert 18 and the associated product 20 is placed, will generate an alarm signal to alert the operator to make appropriate checks.

[0124] 3, the transport tray 14 also includes an optical tray completion code 48 on the same side wall 14b where the tray identification code 42 is also located. An operator reads this optical code 48 to notify a computer system 60 (shown diagrammatically in FIG. 12) managing the storage warehouse 30 that the loading of the products 20 into their corresponding containers 16, and the containers into the transport tray 14, is complete and therefore the transport tray 14 can be transferred to the storage warehouse 30.

[0125] The computer system 60 that manages the storage warehouse 30 may include one or more local computers equipped with appropriate software and / or firmware configured to implement the processes of the present invention.

[0126] One or more local computers may be networked to each other and, in some cases, to remote servers.

[0127] The transport trays 14 can be transported to the storage warehouse 30 by an automated vehicle 50.

[0128] The transfer of the carrier trays 14 from the product receiving station 10 to the automated vehicle 50 and from there to the storage warehouse 30 is effected by moving corresponding conveyors, for example roller conveyors.

[0129] When the computer system 60 that manages the storage warehouse 30 detects that the storage warehouse 30 is ready to receive the products 20 contained in the transport trays 14, it transmits a request to remove the transport trays 14.

[0130] In accordance with this request, the autonomous vehicle 50 moves from the parking area 52 to the product receiving station 10, loads the transport tray 14, and then moves from the product receiving station 10 to the entrance area 32 of the storage warehouse 30.

[0131] The other autonomous vehicle 50 mentioned above may or may not be the same autonomous vehicle that previously delivered the same empty carrier tray 14 to the product receiving station 10 .

[0132] At the entrance area 32 of the storage warehouse 30, the products 20 placed in the containers 16 contained in the transport trays 14 are sorted according to their type and transferred to collection trays 38 (FIGS. 8-10). The collection trays 38, not necessarily completely filled, are then deposited in predetermined locations within the storage warehouse 30.

[0133] The entrance area 32 may include multiple robotic entrance stations 32a, as shown in Figure 1, or both a robotic entrance station 32a and a non-robotic entrance station 32b, as shown in Figures 8-10. In the latter case, the automated vehicle 50 transfers the transport tray 14 to either the robotic entrance station 32a or the non-robotic entrance station 32b depending on whether the first service optical code 47 or the second service optical code 47a is read.

[0134] Each robot entry station 32a of the storage warehouse 30 includes a robot arm 34 and a robot arm 36 and a supply conveyor 33a. The robot arms 34 and 36 are positioned adjacent to the supply conveyor 33a.

[0135] The carrier trays 14 are transferred from the autonomous vehicle 50 to a supply conveyor 33a and by the supply conveyor to a detection station 35 where the carrier trays 14 are locked in a fixed position by removable locking elements 35a (FIG. 9).

[0136] A vision system comprising a camera 35b and a pair of infrared emitters 35c is provided at detection station 35. The vision system is positioned to illuminate the carrier tray 14 from above and capture an image of the carrier tray 14, as shown in FIG.

[0137] This is done to detect the tray identification code 42 and to check that the container 16 and carrier tray 14 are in the correct position.

[0138] 7, dashed lines indicate areas in the image captured by camera 35b that indicate the correct positioning of transport trays 14 (dashed lines defining dashed rectangle 35d) and containers 16 (dashed lines on short side 35e of each container 16). For example, in the case of correct positioning, the dashed lines may be expected to be green, whereas in the case of incorrect positioning, the dashed lines may be expected to turn red. In the latter case, computer system 60 managing storage warehouse 30 may alert an operator to the need to manually correct the position of transport trays 14 and / or containers 16 until the dashed lines turn green.

[0139] The vision system further allows checking the correct spatial orientation of the carrier tray 14. This is done by identifying the location of a tray identification code 42 located on the upper periphery 15 of the carrier tray 14.

[0140] The above-mentioned checks advantageously make it possible to control that the containers 16 are correctly positioned in the carrier trays 14 at the product receiving station 10, as described with reference to Figure 2. Furthermore, these checks advantageously make it possible to ensure proper operation of the robotic arm 36 that picks up the containers 16 from the carrier trays 14.

[0141] Next to the supply conveyor 33a there is a service plane 37 and a table supporting a number of different types of gripping tools 45 (Fig. 9), which can be suction cups, calipers, soft calipers, or input hoppers (funnels).

[0142] Preferably, the service plane 37 has a continuous, elastically deformable surface. For example, this can be achieved by making the service plane 37 wholly or partly from an elastically deformable material (such as rubber) and / or by providing the service plane 37 with elastically deformable supports (e.g., by providing springs and / or rubber feet supporting the service plane).

[0143] This property of the service plane 37 advantageously makes it possible to avoid damage to the service plane 37 itself or to the gripping tool 45 associated with the robot arm 34 during gripping of the product 20 by the gripping tool 45 .

[0144] Additionally, service plane 37 is preferably actuatable (eg, movable and / or deformable) by special actuators (not shown).

[0145] The actuators can preferably be driven independently of one another, for example, the actuators can be driven independently of one another in terms of stroke and actuation frequency.

[0146] This advantageously allows the service plane 37 to temporarily assume different shapes and configurations as needed.

[0147] In particular, as will be explained in more detail below, the service plane 37 can be subjected to different movements (tilting, rocking, vibrating, local deformation and / or shaking movements) to move the products 20 placed thereon, as needed.

[0148] The computer system 60 that manages the storage warehouse 30 can be configured to establish where and how the identical products 20 contained in each container 16 placed on the transport tray 14 are managed.

[0149] For example, depending on predetermined criteria, the computer system 60 can establish that the products 20 contained in the containers 16 are transported according to a first mode, which provides for them all to be dumped together directly onto one of the collection trays 38, or according to a second mode, which provides for them all to be dumped together onto the service plane 37 and then transported one by one to said collection trays 38.

[0150] When the computer system 60 selects the first transfer mode, the robotic arm 36 picks up the target container 16 stored in the transport tray 14, moves the container onto the collection tray 38, tilts or inverts the container, and deposits the identical product 20 stored in the container 16 into the collection tray 38, for example, using a special funnel supported by the robotic arm 34.

[0151] The criteria for selecting one or the other of the possible transport modes for the products 20 can be based on various parameters, such as the delicacy / robustness of the products 20, their size, and their quantity in the container 16. For example, the computer system 60 can be configured to select a first mode when the products 20 are robust, small, and numerous (but in any case in an amount less than the capacity of the collection tray 38). The computer system 60 can then be configured to select a second mode in the case of an amount of products 20 that exceeds the capacity of the collection tray 38, or in the case of products 20 that are delicate or of a particular shape or relatively large size that may jam together or bunch or be damaged during deposition into the collection tray 38 using the first bulk transport mode.

[0152] When the computer system 60 selects the second transfer mode, the robotic arm 36 picks up the target container 16 stored in the transport tray 14, moves the container onto the service plane 37, tilts or inverts the container, and deposits the identical product 20 stored in the container 16 onto the service plane 37. The robotic arm 36 then returns the container 16, now emptied of the product 20, to its previous position on the transport tray 14.

[0153] Thus, the products 20 are randomly placed on the service plane 37 .

[0154] A 3D vision system 37 a is positioned above the service plane 37 and captures images of the products 20 placed on the service plane 37 .

[0155] From such images, the computer system 60 managing the storage warehouse 30 obtains the geometric characteristics of each product 20 placed on the service plane 37 .

[0156] As will be described in more detail below with reference to Figures 11 to 13, depending on the geometric characteristics obtained for each of the products 20, the aforementioned computer system 60 selects the most suitable gripping tool 45 for picking up each of the products 20.

[0157] The robotic arm 34 couples to a selected gripping tool 45 to grip each of the products 20 placed on the service plane 37 .

[0158] The robotic arm 34 is then moved to move each of the gripped products 20, one at a time, from the service plane 37 to a work station 39 adjacent to the service plane 37 and in which a collection tray 38 is located. The robotic arm 34 then releases each of the products 20, one at a time, into the collection tray 38 according to a second transfer mode. The collection tray 38 located in the work station 39 may be empty or may contain several products 20 of the same type as those to be picked up from the service plane 37.

[0159] 5 and 6, the collection tray 38 has a substantially parallelepiped shape and includes a bottom wall 38a, four side walls 38b, and an upper periphery 41 opposite the bottom wall 38a that defines an upper opening 38c. The upper opening 38c provides access to a substantially parallelepiped-shaped compartment 38d. In such compartment 38d, multiple separators 38e may be positioned, for example, orthogonally to one another to define, for example, multiple (six in the non-limiting example of FIG. 5) parallelepiped-shaped compartments 38f.

[0160] A tray identification code 43, preferably an optical code that can be read by an optical reader, is located on the upper periphery 41.

[0161] After the robotic arm 34 places the products 20 in the collection tray 38 , the collection tray is picked up from the work station 39 and placed in a predetermined location within the storage warehouse 30 .

[0162] The transport tray 14 with the products 20 inserted into the collection tray 38 picked up is transported from the detection station 35 of the storage warehouse 30 to the sorting station 33d adjacent to the detection station 35, and then from the sorting station 33d to the discharge conveyor 33c adjacent to the supply conveyor 33a, and finally moved away from the discharge conveyor 33c by the autonomous vehicle 50.

[0163] In the particular example of Figures 8-10, a non-robot entry station 32b is provided alongside the robot entry station 32a. The non-robot entry station 32b includes an infeed conveyor 33b and shares a sorting station 33d and an output conveyor 33c with the robot entry station 32a.

[0164] 8-10, the supply conveyors 33a and 33b and the discharge conveyor 33c are essentially parallel to one another, with the discharge conveyor 33c being interposed between the supply conveyors 33a and 33b.

[0165] 11-13, an embodiment of how the computer system 60 can select the optimal gripping tool 45, and in particular the optimal gripping tool-gripping point pair for gripping each product 20 placed on the service plane 37, will now be described.

[0166] FIG. 11 shows the product 20 randomly tipped onto a service plane 37 by a robotic arm 36 in the example of an identical product 20 consisting of a parallelepiped-shaped plate 20a with a pin 20b on one of the two large faces of the plate 20a.

[0167] As can be seen in the example of Figure 11, the products 20 have different spatial arrangements relative to the service plane 37. In particular, the products 20 are positioned at different positions on the service plane 37, such as the center, side, bottom, top, right, left, etc., which can be defined by predetermined coordinates in an appropriate reference system. Furthermore, the products 20 are arranged on the service plane 37 at different support positions (i.e., the products are placed on the service plane 37 at different surfaces, edges, or support points). Furthermore, although not shown, the products 20 can completely or partially overlap each other and thus, for example, can be tilted relative to the service plane 37. For example, referring to the position of the service planes 37 in FIG. 11, the product 20 located at the top left is supported on the service plane 37 with the largest free surface (i.e., the surface without pins), the product 20 in the middle is supported on the service plane 37 with the largest pinned surface, the product 20 located at the bottom right is supported on the service plane 37 with the largest free surface (i.e., the surface without pins), and the other two products 20 are supported on the service plane 37 with one of the two smaller surfaces.

[0168] According to the present invention, given a predetermined plurality of N reference products (where N is an integer greater than 1), the computer system 60 is configured to analyze a priori (offline) geometric properties of the N reference products to determine, for each reference product and for the most likely stable spatial configuration that the reference product can assume relative to the service plane 37 after being randomly tipped, a respective gripping point 21 of each gripping tool 45 of said plurality of gripping tools 45. The predetermined plurality of N reference products may be defined, for example, by a list of catalog products that can be stored in the storage warehouse 30.

[0169] The geometric characteristics of the reference product may be provided to the computer system 60, for example, in the form of a three-dimensional digital model that three-dimensionally represents the shape, dimensions, volume and centre of gravity of the reference product.

[0170] The gripping point 21 defines the location of a point or area on the surface of the reference product (defined by predetermined coordinates in an appropriate reference system) at which the reference product is adapted to be gripped by the gripping tool 45 under consideration.

[0171] Preferably, when determining the grip point, the computer system 60 is configured to also take into account other characteristic parameters of the reference product, such as weight, material, color, and possibly surface characteristics (e.g., roughness, porosity, slipperiness, deformability, etc.).

[0172] FIG. 11 shows, by way of example and by way of example, gripping points 21 that can be determined for a gripping tool 45 consisting of a suction cup for different spatial arrangements shown in FIG. 11 in the case of a reference product similar to the product 20 shown in FIG. 11, consisting of a parallelepiped-shaped plate 20a and having a pin 20b on one of the two large faces of the plate 20a.

[0173] 11, the gripping points 21 are illustratively shown as small cylinders that simulate the areas that would be gripped by a suction cup having a diameter of the cylinder. For other gripping tools 45, the gripping points 21 may be represented by different areas or patterns on the surface of the product 20.

[0174] According to the present invention, the computer system 60 is also configured to determine a priori (offline) an index score of the probability of success for each gripping point 21 determined for each gripping tool 45 in relation to each possible stable spatial configuration of each reference product.

[0175] Preferably, in addition to the score, the computer system 60 is also configured to determine a priori (off-line) operating parameters useful for properly driving the gripping tool 45 under consideration. For example, such operating parameters may relate to electrical current, control voltage, suction cup suction pressure, compression pressure, clamping force, caliper opening, etc.

[0176] The results of the above-described analysis are stored in a database 62 of a computer system 60, illustratively shown in FIG.

[0177] Thus, the database 62 is compiled a priori (offline) and the data therein is typically calculated overnight by the engineering department each time a new product is created to meet specific size and weight constraints.

[0178] By way of example, database 62 includes a number of records, illustratively shown as table rows in the schematic diagram of FIG.

[0179] Each record is the reference product identification code (corresponding to product identification code 40 described above); Geometric characteristics of the reference product (e.g., stored in the form of a three-dimensional digital model of the product); Weight of the reference product, Possible stable spatial arrangements for the service plane 37; Gripping tools, A list of determined grasp points with their respective scores Contains data related to.

[0180] In particular, in the example of FIG. 12, for a reference product identified by identification code ID-0001, having predetermined geometric characteristics and a predetermined weight (illustratively indicated by xxx1 and yyyy1, respectively), four possible stable spatial configurations (illustratively indicated by zzz1, zzz2, zzz3, zzz4) relative to the service plane 37 are analyzed for two gripping tools (a suction cup (illustratively indicated by the letter "V") and a caliper (illustratively indicated by the letter "P"). The last column contains the gripping point determined for each record with an associated score. Even if not shown, for each gripping point, the last column may also contain operational parameters useful for driving the gripping tool under consideration.

[0181] According to the present invention, after the products 20 contained in the container 16 are randomly placed on the service plane 37, the computer system 60 is adapted to identify the optimal gripping tool-gripping point pair for gripping each product 20.

[0182] FIG. 13 shows a block diagram illustrating an exemplary process for automated product handling implemented by computer system 60 for each product 20 located on service plane 37.

[0183] At block 70, the product identification code 40 of the product 20 is obtained.

[0184] In block 71, a check is made to see if the obtained product identification code 40 exists in the database 62.

[0185] If the resulting product identification code 40 is present in the database 62, then in block 72, the geometric characteristics of the product 20 (e.g., including at least one of shape, size, volume, center of gravity, and spatial location relative to the service plane 37) are obtained from the image acquired by the 3D vision sensor 37a, and possibly other characteristic parameters of the product 20, such as weight (acquired by a suitable sensor (not shown) that may be located in the plant at a station upstream of or at the robot entry station 32a).

[0186] In block 73, the geometric characteristics of the product 20 (e.g., shape, size, and spatial location relative to the service plane 37), and, if appropriate, other characteristic parameters of the product 20, are checked to be compatible (according to predetermined correspondence criteria) with instances of product data stored in various records in the database 62 for the product identification code 40.

[0187] If there is a match, then in block 74, a record corresponding to the instance of product data having geometric characteristics, and possibly other characteristic parameters corresponding to those obtained in block 72 from the product 20 placed on the service plane 37, is retrieved from the database 62.

[0188] In block 74, the most suitable gripping tool 45-gripping point 21 pair among the retrieved records is identified based on the information contained in the database 62 and based on the spatial location of the product 20 relative to the service plane 37.

[0189] In particular, in block 74, a ranking of the gripping tool 45-gripping point 21 pairs is preferably defined based on the scores associated therewith in the records retrieved from the database 62, and from the ranked pair with the highest score, one is selected that enables the robot arm 34 to safely grip the product 20, taking into account the spatial arrangement of the product 20 relative to the service plane 37 and any obstacles present in the working area within the robot entry station 32a.

[0190] Once the optimal gripping tool-gripping point pair is selected, the motion parameters useful for driving the gripping tool of the selected pair can also be derived from the last column of the database 62 (if present).

[0191] For example, in database 62 shown in FIG. 12, if product identification code 40 of product 20 corresponds to ID-0001 and the spatial location of product 20 corresponds to zzz2, then once a match is identified in block 73, instances of product data present in the second and sixth records (second and sixth rows) of database 62 are searched for in block 74, and a ranking of the gripping tool-gripping point pairs is defined based on the data present in the last column of such records.

[0192] Returning to block 71, if it is verified that the obtained product identification code 40 does not exist in the database 62, then in block 75 the most suitable gripping tool 45-gripping point 21 pair for handling the product 20 is identified by real-time processing of the point cloud provided by the 3D vision system 37a.

[0193] The real-time processing can be performed by algorithms implemented by, for example, artificial intelligence, neural networks or expert systems.

[0194] Preferably, such algorithms are aided by suitable machine learning and / or statistical algorithms.

[0195] Returning to block 73, if there is no match between the geometric characteristics of the product 20 and possibly other characteristic parameters of the product 20 and the instances of product data stored in the various records of the database 62 of product identification codes 40, it is provided to proceed to block 76, where it is checked whether a certain threshold number of iterations of the check has been exceeded.

[0196] If the threshold number of times is exceeded, a move to block 75 is provided.

[0197] If the threshold number of times has not been exceeded, then in block 77, the servicing plane 37 is actuated by the actuators described above to change the spatial arrangement of the products 20 relative to the servicing plane 37. As previously described, the actuators may be driven to tilt, oscillate, vibrate, or otherwise temporarily deform the servicing plane 37. The optimal specific action to be actuated may preferably be selected in response to information from processing of images acquired by the 3D vision system 37a. For example, it may be useful to move some of the products 20 toward the center or edge of the servicing plane 37.

[0198] After execution of block 77, a return to execution of blocks 72 and 73 is provided.

[0199] The provision of real-time processing implemented in block 75 advantageously makes it possible to manage instances of products 20 placed on the service plane 37 that are not included in the predetermined plurality N of reference products. This may occur, for example, in the case of new products that have not yet been entered into the database 62.

[0200] The real-time processing performed in block 75 also makes it possible to manage cases of products 20 appearing on the service plane 37 that are present in the database 20 but have different geometric characteristics than those associated with the corresponding reference products. This may occur, for example, in the case of products 20 that turn out to have a very different appearance than that provided by the three-dimensional digital model stored in the database 62. Consider, for example, the case of packaged products (e.g., screws packed in parallelepiped-shaped casings) or products 20 that have piled or piled up on the service plane 37 due to random tipping, making them difficult to distinguish from one another.

[0201] If overlapping or piled products 20 on the service plane 37 are difficult to distinguish from one another due to random tipping, execution of blocks 76 and 77 advantageously allows for attempts to move the service plane 37 a certain number of times to better separate the products 20 from one another and make them more easily identifiable and distinguishable from one another.

[0202] Although not shown in FIG. 13, if block 75 fails to identify a gripping tool 45-gripping point 21 pair, it may be provided to return to execution of block 77 for further attempts before asking the operator to manage the failure.

[0203] Additionally or alternatively, although not shown in FIG. 13, in the event of a failure at block 74, it may be provided to move to execution of block 77 for further attempts before asking the operator to manage the failure.

[0204] Thus, the present invention makes it possible to select the optimal gripping tool-gripping point pair for safely picking up and handling each product 20 randomly placed on the service plane 37, depending on its geometric characteristics, preferably other characteristic parameters such as the weight of the product 20, and taking into account the working area available to the robot arm 34 in the robot entry station 32a.

[0205] The present invention therefore enables products 20 to be picked up from the service plane 37 and handled within the robot entry station 32a in a safe and stable manner, and to maintain sufficient gripping stability during their handling by the robot arm 34.

[0206] Obviously, those skilled in the art can make many modifications and variations to the invention described above to meet specific contingent needs, all of which still fall within the scope of protection defined by the following claims.

Claims

1. 1. A process for automated handling of products (20) in a plant (100) comprising a workstation (32a) comprising a first robotic arm (34) and a plurality of gripping tools (45), comprising: randomly placing the products (20) on a service plane (37) of the workstation (32a); acquiring an image of the product (20) positioned on the servicing plane (37); processing the images to obtain geometric characteristics of each of the products (20) placed on the service plane (37); For each of said products (20), identifying a corresponding gripping tool (45) from the plurality of gripping tools (45) based on the geometric characteristics obtained for each; Associating the identified corresponding gripping tool with the first robotic arm (34); moving the first robot arm (34) to grip the product (20) with the corresponding gripping tool (45), moving the gripped product (20) from the service plane (37) to a work station (39) of the plant (100), and releasing the product (20) at the work station (39) with the corresponding gripping tool (45); The process includes:

2. Identifying a corresponding gripping tool (45) from the plurality of gripping tools (45) for each of the products (20) placed on the service plane (37) includes: accessing a database (62) containing instances of product data relating to a plurality of predetermined reference products based on the geometric characteristics obtained by processing the image, wherein the product data includes the geometric characteristics of the reference products, and each instance of product data is associated with one of the plurality of gripping tools (45), each of which is associated with a respective score indicating a probability of successful gripping; retrieving from said database (62) instances of product data having geometric properties corresponding to said geometric properties obtained by processing said image; identifying the corresponding grasping tool (45) from the grasping tools (45) associated with the instance retrieved from the database (62) based on a score associated therewith; 2. The process of claim 1, comprising:

3. The process of claim 1 or 2, wherein the geometric characteristics include at least one of shape, size, center of gravity, volume, and spatial orientation relative to the service plane (37).

4. Identifying the corresponding gripping tools (45) includes identifying optimal pairs formed by the corresponding gripping tools (45) and their respective corresponding gripping modes (21). The process according to any one of claims 1 to 3.

5. each instance of product data in the database (62) is associated with one of the plurality of gripping tools (45), the gripping tool (45) being associated with a plurality of gripping modes (21), and each pair formed by the gripping tool (45) and one of the plurality of gripping modes (21) being associated with a score indicative of the probability of successful gripping for the pair; The process of claim 4, wherein an optimal pair between the gripping tool and the associated gripping mode is identified for the instance retrieved from the database (62) based on the score associated with each pair.

6. 6. The process according to claim 4 or 5, wherein the gripping mode defines operating parameters adapted to drive at least one gripping point (21) on the product, preferably the gripping tool (45).

7. The process of any one of claims 1 to 6, comprising obtaining at least one characteristic parameter of the product placed on the service plane (37) selected from weight, material, color, and surface characteristics.

8. 8. The process of claim 7, wherein the product data in the database (62) includes at least one characteristic parameter of the products of the predetermined plurality of reference products selected from weight, material, color, and surface characteristics in addition to geometric characteristics.

9. 9. The process according to claim 2, wherein if, for one of the products placed on the servicing plane (37), the database lacks an instance of product data having geometric properties corresponding to the geometric properties obtained by processing the image, the identification of the corresponding gripping tool (45) from the plurality of gripping tools (45) is performed by real-time processing of the acquired image.

10. Before randomly placing the products on the service plane (37), the products are contained in a container (16), and randomly placing the products on the service plane (37) is performed. picking up said container (16) by an automated transfer device (36); tilting or inverting the container (16) above the service plane (37) by moving the automated transfer device (36); The process according to any one of claims 1 to 9, comprising:

11. 11. The process of claim 10, wherein the containers (16) are placed in a carrier tray (14) that accommodates multiple containers (16), each container (16) containing multiple identical products.

12. Prior to picking up the container (16) by the automated transfer device (36), transferring the carrier tray (14) from the automated vehicle (50) to a first conveyor (33a) adjacent the service plane (37); transferring the carrier tray (14) from the first conveyor (33a) to a detection station (35) adjacent to the first conveyor (33a); acquiring an image of the carrier tray (14) at the detection station (35); 12. The process of claim 11, comprising:

13. 13. The process of claim 12, including processing the acquired image of the transport tray (14) to check correct positioning of the tray (14) and the plurality of containers (16) against predetermined reference parameters.

14. The process of any one of claims 1 to 13, wherein the gripping tool (45) is selected from the group consisting of a suction cup, a caliper, a soft caliper, a feed hopper, a hook, and any combination thereof.

15. A process according to any one of the preceding claims, wherein the service plane (37) is at least partially elastically deformable.

16. The process of any one of claims 1 to 15, wherein the service plane (37) is operable.

17. 17. The process of claim 16, wherein the servicing planes are actuable by actuators associated with the servicing planes that can be driven independently of one another.

18. 17. The process of claim 16, wherein the actuators can be driven independently of each other in terms of stroke and actuation frequency.

19. Prior to performing the real-time processing of the acquired images, the process sequentially comprises: operating the servicing plane (37) to move at least some of the products (20) disposed on the servicing plane (37); re-processing the acquired images to obtain updated geometric characteristics of the product (20) positioned on the servicing plane (37); re-accessing the database (62) based on the updated geometric characteristics to retrieve the instances of product data having geometric characteristics corresponding to the updated geometric characteristics, and identifying the corresponding gripping tool (45) from the gripping tools (45) associated with the instances thus retrieved from the database (62) based on the score associated therewith; Including, The process of any one of claims 9 and 16 to 18, wherein the real-time processing of the acquired image is performed if the database (62) lacks an instance of product data having geometric properties that correspond to the updated geometric properties.

20. A plant (100) comprising a workstation (32a), the workstation (32a) comprising: a first robotic arm (34); a service plane (37); a plurality of gripping tools (45) arranged near the service plane (37); an image capture device (37a) configured to capture images of products (20) arranged on the service plane (37); and a computer (60) configured to process the images to obtain geometric characteristics of each of the products (20) arranged on the service plane (37); The computer (60) calculates, for each of the products (20), identifying a corresponding gripping tool (45) from the plurality of gripping tools (45) based on the geometric characteristics respectively obtained by processing the images; Associating the corresponding gripping tool (45) with the first robotic arm (34); a plant (100) configured to move the first robot arm (34) to grip the product with the corresponding gripping tool (45), move the gripped product from the service plane (37) to a work station (39) of the plant (100), and release the product at the work station (39) with the corresponding gripping tool (45).

Images