Method for automated processing of products in a plant

Abstract

The invention relates to a plant (100) and to a method for automatically handling products (20) within a plant (100), the plant comprising a workstation (32a), in turn comprising a first robot arm (34) and a plurality of gripping tools (45). The method provides for randomly arranging products (20) on a service plane (37) of a workstation (32a); images of the products (20) arranged on the service plane (37) are acquired, and geometric features of each of the products (20) arranged on the service plane (37) are obtained by processing the images. For each of the products (20), the method also provides for identifying a respective gripping tool (45) among the plurality of gripping tools (45) based on the obtained respective geometric features; associating the respective identified gripping tool with the first robot arm (34); and moving the first robot arm (34) to grip the product (20) by means of the respective gripping tool (45), moving the gripped product (20) from the service plane (37) to an operating station (39) of the plant (100), and releasing the product (20) in the operating station (39) by means of the respective gripping tool (45).

Description

[0001] Description

[0002] The present invention relates to a factory and a method for automatically processing products within the factory.

[0003] In the context of the logistics of a production company, the automatic processing of products between different operating stations in a factory is particularly important.

[0004] For example, especially in medium to large companies with a medium to high level of automation, products can be moved between the above-mentioned operating stations by autonomous vehicles, and / or if the products have to be temporarily stored in a storage warehouse for their future use, an automated warehouse can be provided, where the operations of inserting products into the storage warehouse and picking up such products from the storage warehouse are controlled by dedicated warehouse management software.

[0005] Generally, even in the presence of automation systems, in some operating stations, products are still handled by trained operators.

[0006] For example, even in the presence of an automated warehouse, the delivery of products to the storage warehouse is performed manually by an operator who picks up the products from a pick-up station arranged near the storage warehouse and inserts the products into a suitable collection tray, which is intended to be placed in a predetermined position within the storage warehouse.

[0007] The applicant has thought of automating some operations that are usually performed manually by an operator, such as delivering products to an automated warehouse, and has provided a robotic arm to perform the above operations.

[0008] However, the applicant has noticed that the above products usually have very different shapes, and has noticed that in the case of the same product, they can be randomly arranged on the surface from which they have to be picked up, thus presenting them to the robotic arm that has to pick them up in very different spatial arrangements.

[0009] In this context, the applicant has realized the need to provide a solution that allows the robotic arm to pick up products safely and stably and allows the robotic arm to maintain sufficient gripping stability during the handling by the robotic arm.

[0010] Therefore, in its first aspect, the present invention relates to a method for automatically processing products within a factory.

[0011] Preferably, the factory includes a workstation, which in turn includes a first robotic arm and a plurality of gripping tools.

[0012] Preferably, the products are randomly arranged on the service plane of the workstation.

[0013] Preferably, an image of the product arranged on the service plane is acquired.

[0014] Preferably, the image is processed to obtain the geometric features of each of the products arranged on the service plane.

[0015] Preferably, for each of the products, a corresponding gripper tool is identified among the plurality of gripper tools based on the geometric features obtained respectively.

[0016] Preferably, for each of the products, the corresponding identified gripper tool is associated with the first robotic arm.

[0017] Preferably, for each of the products, the first robotic arm is moved to grip the product by the corresponding gripper tool and move the gripped product from the service plane to the operating station of the factory.

[0018] The operating table can be adjacent to the service plane.

[0019] Preferably, for each of the products, the product is released in the operating station by the corresponding gripper tool.

[0020] In a second aspect, the present invention also relates to a factory.

[0021] Preferably, the factory includes a workstation.

[0022] Preferably, the workstation includes a first robotic arm.

[0023] Preferably, the workstation includes a service plane.

[0024] Preferably, the factory includes an operating station.

[0025] The operating table can be adjacent to the service plane.

[0026] Preferably, the workstation includes a plurality of gripper tools arranged near the service plane.

[0027] Preferably, the workstation includes an image acquisition device configured to acquire an image of the product arranged on the service plane.

[0028] Preferably, the workstation includes a computer configured to process the image to obtain the geometric features of each of the products arranged on the service plane.

[0029] Preferably, for each of the products, the computer is configured to identify the corresponding gripper tool among the plurality of gripper tools based on the geometric features obtained by processing the image respectively.

[0030] Preferably, for each product in the products, the computer is configured to associate a corresponding gripping tool with the first robotic arm.

[0031] Preferably, for each product in the products, the computer is configured to move the first robotic arm to grip the product by the corresponding gripping tool, move the gripped product from the service plane to the operating station, and release the product in the operating station by the corresponding gripping tool.

[0032] Since a plurality of gripping tools suitable for picking up products with different geometric features are provided, and since each product is processed by the gripping tool associated with the robotic arm based on the geometric features that each product has in the acquired image, products with different geometric features can be safely and stably processed by the same robotic arm, thereby achieving the required automation.

[0033] The present invention may have at least one of the following preferred features, either individually or in combination.

[0034] Preferably, the method is implemented by a computer.

[0035] Preferably, the method is implemented by an appropriate algorithm. The algorithm is preferably updated by specialized machine learning and / or statistical algorithms.

[0036] Preferably, for each product arranged on the service plane, identifying the corresponding gripping tool among the plurality of gripping tools includes:

[0037] - Accessing a database based on geometric features obtained by processing an image, the database including examples of product data related to a predefined plurality of reference products, wherein the product data includes the geometric features of the reference products and each example of the product data is associated with a gripping tool among the plurality of gripping tools, and the gripping tool is associated with a corresponding score indicating the probability of a successful grip;

[0038] - Retrieving an example of product data having geometric features corresponding to the geometric features obtained by processing the image from the database; and

[0039] - Identifying the corresponding gripping tool among the gripping tools associated with the retrieved example and based on the score associated therewith.

[0040] Preferably, the geometric features include at least one of the following: shape, size, volume, center of gravity, and spatial arrangement relative to the service plane.

[0041] Preferably, the identification of the corresponding grip is performed by considering the working area available to the first robotic arm in the workstation.

[0042] Preferably, the recognition of the corresponding gripping tool is performed by considering the spatial arrangement of the product relative to the service plane.

[0043] Preferably, the recognition of the corresponding gripping tool includes recognizing the best pair formed by the corresponding gripping tool and the corresponding gripping pattern.

[0044] Preferably, each example of product data is associated in the database with a gripping tool among the plurality of gripping tools, the gripping tool being associated with a plurality of gripping patterns, each pair being formed by a gripping tool, and one gripping pattern among the plurality of gripping patterns being associated with a corresponding score indicating the probability of a successful grip for that pair.

[0045] Preferably, in the examples retrieved from the database, the recognition of the best pair is performed between the gripping tool and the gripping pattern associated therewith based on the score associated with each pair.

[0046] Preferably, the recognition of the best pair is also performed by considering the working area available to the first robotic arm in the workstation.

[0047] Preferably, the recognition of the best pair is also performed by considering the spatial arrangement of the product relative to the service plane.

[0048] Preferably, the recognition of the best pair is performed by selecting, from the pairs associated with the highest scores, a pair that allows the product to be gripped and handled by the first robotic arm within the working area without encountering obstacles.

[0049] Preferably, the gripping pattern defines at least one gripping point on the product.

[0050] Preferably, the gripping pattern defines operating parameters adapted to drive the operation of the gripping tool.

[0051] Preferably, at least one characteristic parameter of the product arranged on the service plane is obtained.

[0052] Preferably, in addition to geometric features, the product data in the database includes at least one characteristic parameter of the product of a predefined plurality of reference products.

[0053] The characteristic parameter can be selected from the weight, material, color, and surface features of the product.

[0054] In a preferred embodiment, if for one of the products arranged on the service plane, the database lacks an example of product data whose geometric features correspond to the geometric features obtained by processing the image, then the recognition of the corresponding gripping tool among the plurality of gripping tools is performed by processing the image obtained in real time.

[0055] In a preferred embodiment, if, for one of the products arranged on the service plane, the database lacks an example of product data whose geometric features correspond to the geometric features obtained by processing the image, the recognition of the best pair is performed by processing the acquired image in real time.

[0056] In a preferred embodiment, the service plane is at least partially elastically deformable.

[0057] In a preferred embodiment, the service plane is actuable.

[0058] Preferably, the service plane can be actuated by actuators associated therewith and drivable independently of each other.

[0059] Preferably, the actuators can be driven independently of each other in terms of stroke and actuation frequency.

[0060] Preferably, before performing the real-time processing of the acquired image, the service plane is actuated so that at least some of the products arranged thereon are moved.

[0061] Then, preferably, the acquired image is processed again to obtain updated geometric features of each of the products arranged on the service plane.

[0062] Then, preferably, the database is accessed again based on the updated geometric features to retrieve examples of product data having geometric features corresponding to the updated geometric features, and the corresponding gripping tool (or the best pair) is identified among the gripping tools (and gripping modes) associated with the examples retrieved from the database based on the associated scores.

[0063] Preferably, if the database lacks an example of product data whose geometric features correspond to the updated geometric features, the real-time processing of the acquired image is performed. Preferably, the image acquisition device is arranged above the service plane.

[0064] The image acquisition device may include a 3D vision system.

[0065] The image acquisition device may include a camera.

[0066] For example, the image acquisition device may use range imaging techniques adapted to provide information about the spatial position of each point in the acquired image.

[0067] The image acquisition device may be configured to acquire a point cloud or a 2D image pair consisting of a 2D luminance image and a 2D depth image.

[0068] It should be noted that the luminance image is intended to be a 2D image defined by a set of luminance (or intensity) values associated with the image points in the X plane and the Y plane of the reference frame of the image acquisition device. Further, the depth image or depth map is a 2D image or map defined by a set of distance values associated with the points of the image relative to a predefined viewing point in the X plane and the Y plane.

[0069] The image acquisition device may include, for example, a stereo camera, a TOF camera (where TOF stands for "time of flight"), or a structured light camera.

[0070] In an embodiment, the products randomly arranged on the service plane are identical to each other. The identical products are intended to be of a single type. However, products of a single type may be different in terms of geometric features due to, for example, packaging (e.g., some products may be packaged while others may not be packaged).

[0071] Preferably, the products are accommodated in a container before being randomly arranged on the service plane.

[0072] Preferably, randomly arranging the products on the service plane includes picking up the container by an automatic transfer device (such as a second robotic arm).

[0073] Preferably, randomly arranging the products on the service plane includes tilting or flipping the container above the service plane by moving the automatic transfer device.

[0074] Preferably, the container is arranged in a transport tray containing a plurality of containers, and each container includes a plurality of identical products.

[0075] Preferably, the transport tray is transferred to a first conveyor adjacent to the service plane before the container is picked up by the automatic transfer device.

[0076] Preferably, the transfer is performed by an autonomous vehicle.

[0077] Preferably, after the transport tray is transferred to the first conveyor and before the container is picked up by the automatic transfer device, the transport tray is transferred from the first conveyor to a detection station adjacent to the first conveyor.

[0078] Preferably, an image of the transport tray is acquired by an image acquisition device provided in the detection station.

[0079] Preferably, by processing the acquired image of the transport tray, the correct positioning of the tray and the plurality of containers contained therein relative to predetermined reference parameters is checked.

[0080] Preferably, by processing the acquired image of the transport pallet, the correct spatial orientation of the transport pallet relative to a predetermined reference parameter is checked.

[0081] Preferably, before acquiring the image of the transport pallet, the transport pallet is locked in position at the detection station.

[0082] Preferably, the gripping tool is selected from: suction cups, calipers, soft calipers, tipping hoppers (funnels), hooks, and any combination thereof.

[0083] Preferably, before releasing each product in the product at the operation station, a collection tray is arranged at the operation station.

[0084] Preferably, releasing each product in the product at the workstation includes storing the product in the collection tray.

[0085] Preferably, after storing the product in the collection tray, the collection tray is picked up from the operation station.

[0086] Preferably, the collection tray is stored in a predefined location within the storage warehouse.

[0087] Preferably, the factory or workstation includes one or more sensors configured to detect characteristic parameters of the product arranged on the service plane, and the characteristic parameters are selected from: weight, surface characteristics, material, and color.

[0088] The following detailed description of the preferred embodiments of the present invention provided by reference to the accompanying drawings and by indicative and non-limiting examples will make the additional features and advantages of the present invention clearer. In the drawings:

[0089] - Figure 1 shows a simplified and schematic layout of a factory in which a method for automated processing of a product according to the present invention is performed;

[0090] - Figure 2 shows a top plan view of a transport pallet used in the method of the present invention;

[0091] - Figure 3 shows Figure 2 a side view of the transport pallet;

[0092] - Figure 4 shows a top plan view of an accompanying document used in the method of the present invention;

[0093] - Figure 5 shows a top plan view of a collection tray used in the method of the present invention;

[0094] - Figure 6 showsFigure 5 Side view of the collection tray;

[0095] - Figure 7 Shows the Figure 2 Frame of the transport tray analyzed by the management software of the method of the present invention;

[0096] - Figure 8 Perspective view showing the entrance area of the storage warehouse used in the method of the present invention;

[0097] - Figure 9 Shows Figure 8 Top plan view of the entrance area of the storage warehouse of

[0098] - Figure 10 Shows Figure 8 Perspective view of a part of the entrance area of the storage warehouse of

[0099] - Figure 11 Schematically shows an example of products randomly arranged on the service plane and the associated respective gripping points in the case of a suction cup gripping tool;

[0100] - Figure 12 Schematically shows an example of a database that can be used to implement the method of the present invention;

[0101] - Figure 13 Schematically shows an example block diagram of a possible implementation of the method of the present invention.

[0102] In Figure 1 , the reference numeral 100 indicates the area of the factory where a method for automated processing of the product 20 (shown in Figure 11 ) according to the present invention is carried out.

[0103] The product 20 can come from an external supplier or from other areas of the same factory or from other factories of the same company or group of companies. In a non-limiting example, the product 20 is a component or part of a packaging machine (such as a cigarette packaging machine).

[0104] The above-mentioned product 20 is initially processed at the product receiving station 10. At the receiving station 10, the product 20 is placed in a container 16 ( Figure 2 ), and the container 16 is in turn arranged in a transport tray 14.

[0105] In the area 100 of the factory, in addition to the product receiving station 10, a storage warehouse 30 is provided, and the above-mentioned product 20 will be stored in the storage warehouse 30 for being picked up as needed.

[0106] Preferably, the storage warehouse 30 is an automated warehouse.

[0107] AsFigure 4 As shown in Figure 4 , each product or each plurality of identical products 20 present in the product receiving station 10 is accompanied by an accompanying document 18. The accompanying document 18 shows the quantity and type of the product 20 associated therewith. This information is also included in the product identification code 40 printed on the accompanying document 18 itself. Preferably, the product identification code 40 is an optical code that can be read by an optical reader, and more preferably a barcode.

[0108] The accompanying document 18 can be prepared by an operator participating in the product receiving station 10 or directly by the supplier of the product 20 and delivered to the product receiving station 10 together with the product 20 itself.

[0109] In some embodiments, such as one embodiment illustrated herein, a first service optical code 47 is also printed on the accompanying document 18, and the first service optical code 47 identifies the possibility of processing the corresponding product 20 by the robotic arm 36 ( Figures 8 - 10 ) in the robotic entry station 32a ( Figures 8 - 10 ) provided in the storage warehouse 30. However, embodiments are foreseen in which the accompanying document 18 does not have the above-mentioned first service optical code 47.

[0110] The product receiving station 10 includes a plurality of product loading stations 12, where different operators place the products 20 to be stored in their respective containers 16 and place the containers 16 in the transport trays 14.

[0111] The transport tray 14 without the container 16 is transferred to the product loading station 12 to load the container 16 at least partially filled with the product 20.

[0112] The transfer of each transport tray 14 to the product loading station 12 can be carried out after the operator has called the autonomous vehicle 50, and the autonomous vehicle 50 starts from the parking area 52 where there are a plurality of autonomous vehicles 50 and arrives at the product loading station 12. This call is made by the operator, for example, by actuating a call button (not shown) specifically provided at the product loading station 12.

[0113] Reference Figure 2 and Figure 3 , each transport tray 14 has a generally parallelepiped shape and includes a bottom wall 14a, four side walls 14b, and an upper peripheral edge 15 defining a top opening 14c on the side opposite to the bottom wall 14a. The top opening 14c allows access to the generally parallelepiped-shaped compartment 14d.

[0114] In the context of the present specification and the appended claims, spatial references (such as "upper", "top", "above", etc. and "bottom", "below", etc.) shall be understood to refer to the operating position of the transport pallet 14, as shown in the appended Figures 8 - 10 wherein the transport pallet 14 is rested by its bottom wall 14a.

[0115] The compartment 14d houses the containers 16, which are ten in a non - limiting example of Figure 2 .

[0116] The containers 16 have a generally parallelepiped shape.

[0117] The containers 16 are arranged side - by - side in the transport pallet 14. In a non - limiting example of Figure 2 , the containers 16 are arranged in two rows of five containers 16 side - by - side along the respective long sides of the transport pallet 14.

[0118] Preferably, the containers 16 are open - front storage boxes.

[0119] The operator places the products 20 in the transport pallet 14, spacing them according to the product type. Thus, each container 16 contains the same type of product 20. The containers 16 of the same transport pallet 14 may contain the same or even different products 20. If the products are large, some containers 16 may contain a single product.

[0120] The products 20 can be placed in their respective containers 16 either before or after placing the containers 16 in the transport pallet 14.

[0121] The products 20 are identified at the product receiving station 10 by reading the product identification code 40 printed on the accompanying document 18.

[0122] After loading the products 20 into the containers 16, the accompanying document 18 is also placed in the container 16 together with the associated products 20.

[0123] The product identification code 40 can be read either before or after placing the products 20 in the containers 16 and either before or after placing the containers 16 in the transport pallet 14.

[0124] Each transport pallet 14 has a corresponding pallet identification code 42.

[0125] The pallet identification code 42 is preferably arranged on the upper peripheral edge 15 of the transport pallet 14 (so as to be visible when the transport pallet 14 is viewed from above) or on the side wall 14b of the transport pallet 14.

[0126] Preferably, the pallet identification code 42 is an optical code (more preferably a bar code) readable by an optical reader.

[0127] An alphanumeric code (not shown) allowing an operator to visually identify the transport pallet 14 is associated next to the pallet identification code 42.

[0128] In Figure 2 the illustrated embodiment, two pallet identification codes 42 are provided on the upper peripheral edge 15 of the transport pallet 14. They are arranged opposite the top opening 14c in a position adjacent to one long side of the transport pallet 14. This positioning identifies the row of the first five containers 16 to be placed therein, starting from the left (or from the bottom with reference to the position of the transport pallet 14 in Figure 2 and moving towards the right (or upwards with reference to the position of the transport pallet 14 in Figure 2 ). The second row of containers 16 is positioned in the same positioning order as the first row of containers 16. For example, with reference to Figure 7 , ten containers 16 are arranged successively in positions numbered I-X, where position I is positioned adjacent to the left pallet identification code 42 (or below with reference to the position of the transport pallet 14 in Figure 7 ), positions II-V are arranged side by side along the row between the two pallet identification codes 42, and positions VI-X are arranged respectively along the row adjacent to and close to the row of positions I-V.

[0129] Before or after filling the transport pallet 14 with containers 16 which are in turn at least partially filled with the product 20, the transport pallet 14 is identified by an operator by reading one of the pallet identification codes 42.

[0130] As Figure 2 and Figure 3 shown, the transport pallet 14 further includes a plurality of container positioning identification codes 46 on its upper peripheral edge 15. Each of these codes 46 is arranged adjacent to a corresponding area of the compartment 14d configured to receive the corresponding container 16.

[0131] Preferably, the container positioning identification code 46 is an optical code (more preferably a bar code) readable by an optical reader.

[0132] After each container 16 has been placed in the transport pallet 14, the position of the container 16 relative to the transport pallet 14 (and thus the product 20 contained therein and the quantity of the product 20) is identified by reading the container positioning identification code 46 adjacent to the container 16.

[0133] Again referring to Figure 3, the transport pallet 14 further includes a first service optical code 47 that identifies the possibility of being emptied by the robotic arm 36 provided in the robotic entry station 32a of the storage warehouse 30 ( Figures 8 - 10 ). This code 47 is the same as the first service optical code 47 that can be printed on the accompanying document 18.

[0134] In the illustrated embodiment herein, the transport pallet 14 further includes a second service optical code 47a that identifies that it can also be emptied manually by an operator.

[0135] The first service optical code 47 and the second service optical code 47a are arranged on the side wall 14b of the transport pallet 14, and the pallet identification code 42 is also arranged on the side wall 14b of the transport pallet 14.

[0136] Therefore, the transport pallet 14 shown in the figures is suitable for being emptied by the robotic arm 36 or manually. The operator determines the type of mechanical or manual emptying of the transport pallet 14 by the optical reader reading the first service optical code 47 or the second service optical code 47a. All the containers 16 contained in the transport pallet 14 in which the first service optical code 47 is read will be emptied by the robotic arm, just as all the containers 16 contained in the transport pallet 14 in which the second service optical code 47a is read will be emptied manually.

[0137] Transport pallets 14 that contain only the first service optical code 47 can be provided. These transport pallets 14 should be filled only with products 20 accompanied by the accompanying document 18, which also bears the first service optical code 47.

[0138] Additional transport pallets 14 that contain only the second service optical code 47a can also be provided. These transport pallets 14 should be filled only with products 20 accompanied by the accompanying document 18 that does not have the first service optical code 47.

[0139] Any reading inconsistency, such as reading the first service optical code 47 on the accompanying document 18 placed in the container 16 but not reading the first service optical code 47 on the transport pallet 14 in which the container 16 containing the accompanying document 18 and the corresponding product 20 is placed, or vice versa, or reading the first service optical code 47 on the accompanying document 18 placed in the container 16 and the second service optical code 47a on the transport pallet 14 in which the container 16 containing the accompanying document 18 and the related product 20 is placed, generates an alarm signal that warns the operator to make an appropriate inspection.

[0140] Referring again to Figure 3, the transport pallet 14 further includes an optical pallet completion code 48 on the same side wall 14b where the pallet identification code 42 is also arranged. The operator reads this optical code 48 to signal to the computer system 60 (schematically shown in Figure 12 ) that manages the storage warehouse 30 that the loading operations of the products 20 into their respective containers 16 and the containers 16 into the transport pallet 14 are completed, and thus, the transport pallet 14 is transferred to the storage warehouse 30.

[0141] The computer system 60 that manages the storage warehouse 30 may include one or more local computers, and one or more local computers are provided with appropriate software and / or firmware configured to implement the method of the present invention.

[0142] One or more local computers may be networked together and may be connected to a remote server.

[0143] The transport pallet 14 can be transferred to the storage warehouse 30 by an autonomous vehicle 50.

[0144] The transport pallet 14 is transferred from the product receiving station 10 to the autonomous vehicle 50 and from the autonomous vehicle 50 to the storage warehouse 30 by moving a corresponding conveyor (such as a roller conveyor).

[0145] When the computer system 60 that manages the storage warehouse 30 detects that the storage warehouse 30 is in a state of receiving the products 20 contained in the transport pallet 14, a request to withdraw the transport pallet 14 is sent.

[0146] After this request, the autonomous vehicle 50 moves from the parking area 52 to the product receiving station 10, and after loading the transport pallet 14, moves from the product receiving station 10 to the entrance area 32 of the storage warehouse 30.

[0147] The above-mentioned other autonomous vehicle 50 may or may not be the same autonomous vehicle that previously emptied the same transport pallet 14 to the product receiving station 10.

[0148] In the entrance area 32 of the storage warehouse 30, the products 20 placed in the containers 16 contained in the transport pallet 14 are transferred to collection pallets 38 separated according to their types ( Figures 8 - 10 ). Then, the collection pallets 38 (although not necessarily fully filled) are placed in predefined positions within the storage warehouse 30.

[0149] The entrance area 32 may include a plurality of robotic entrance stations 32a as shown in Figure 1 , or include as shown in Figures 8 - 10The shown robotic entry station 32a and non-robotic entry station 32b. In the latter case, the autonomous vehicle 50 transfers the transport pallet 14 to the robotic entry station 32a or the non-robotic entry station 32b depending on whether the first service optical code 47 or the second service optical code 47a is read.

[0150] Each robotic entry station 32a of the storage warehouse 30 includes robotic arms 34, 36 and a supply conveyor 33a. The robotic arms 34, 36 are placed beside the supply conveyor 33a.

[0151] Transfer the transport pallet 14 from the autonomous vehicle 50 to the supply conveyor 33a and through the supply conveyor 33a to the inspection station 35, where the transport pallet 14 is locked in a fixed position by a removable locking element 35a ( Figure 9 ).

[0152] A vision system including a camera 35b and a pair of infrared illuminators 35c is provided in the inspection station 35. The vision system is arranged such that it illuminates the transport pallet 14 from above and captures an image of the transport pallet 14, as Figure 7 the shown image.

[0153] This is done to detect the pallet identification code 42 and to check whether the container 16 and the transport pallet 14 are in the correct positions.

[0154] In Figure 7 a non-limiting example, the areas identifying the correct positioning of the transport pallet 14 (dashed lines defining the dashed rectangular 35d) and the container 16 (dashed lines at the short sides 35e of each container 16) in the image captured by the camera 35b are indicated by dashed lines. For example, it can be foreseen that in the case of correct positioning, the above-mentioned dashed lines will be green, while in the case of incorrect positioning, the above-mentioned dashed lines will turn red. In the latter case, the computer system 60 managing the storage warehouse 30 warns the operator that manual correction of the position of the transport pallet 14 and / or the container 16 is required until the dashed lines turn green.

[0155] The vision system further allows checking the correct spatial orientation of the transport pallet 14. This is done by identifying the position of the pallet identification code 42 arranged on the upper peripheral edge 15 of the transport pallet 14.

[0156] The above checks allow advantageously controlling that the container 16 has been correctly arranged within the transport pallet 14 at the product receiving station 10, as referenced Figure 2 above. In addition, they advantageously allow ensuring the correct operation of the robotic arm 36 picking up the container 16 from the transport pallet 14.

[0157] Beside the supply conveyor 33a, a service plane 37 and a workbench supporting multiple different types of gripping tools 45 are provided (Figure 9 )。The gripping tool 45 can be a suction cup, calipers, soft calipers, tipping hopper (funnel).

[0158] Preferably, the service plane 37 has a continuous elastically deformable surface. For example, this can be achieved by making the service plane 37 entirely or partially of an elastically deformable material (such as rubber) and / or by providing an elastically deformable support to the service plane 37 (for example, by providing spring feet and / or rubber feet that support the service plane 37).

[0159] During the gripping of the product 20 by the gripping tool 45, this feature of the service plane 37 advantageously allows avoiding damage to the service plane 37 itself or to the gripping tool 45 associated with the robotic arm 34.

[0160] Furthermore, the service plane 37 is preferably actuatable (e.g., movable and / or deformable) by a dedicated actuator (not shown).

[0161] Preferably, the actuators can be driven independently of each other. For example, the actuators can be driven independently of each other in terms of stroke and actuation frequency.

[0162] This advantageously allows the service plane 37 to temporarily assume different shapes and configurations as needed.

[0163] Specifically, as described in more detail below, the service plane 37 can undergo different movements (tilting, oscillating, vibrating, local deformation, and / or shock movements) as needed to move the product 20 disposed thereon.

[0164] The computer system 60 that manages the storage warehouse 30 can be configured to determine where and how to manage the same product 20 contained in each container 16 placed in the transport pallet 14.

[0165] For example, according to a predefined criterion, the computer system 60 can be set up to transfer the product 20 contained in the container 16 according to a first mode or according to a second mode. The first mode provides for directly pouring all of the product 20 together into the collection tray 38, and the second mode provides for transferring the product 20 one by one to the collection tray 38 after pouring all of the product 20 together onto the service plane 37.

[0166] When the computer system 60 selects the first transfer mode, the robotic arm 36 picks up the relevant container 16 contained in the transport pallet 14 and moves it above the collection tray 38, tilting or flipping it so as to pour, for example, the same product 20 contained in the container 16 into the collection tray 38 by means of a dedicated funnel supported by the robotic arm 34.

[0167] The criteria for selecting one transfer mode or another among the possible transfer modes for product 20 can be based on different parameters, such as the delicacy / robustness of product 20, the size and quantity of product 20 in container 16. For example, computer system 60 can be configured to select the first mode when product 20 is robust, small, and large in quantity (but in any case the quantity is less than the capacity of collection tray 38). Further, computer system 60 can be configured to select the second mode when the quantity of product 20 exceeds the capacity of collection tray 38 or when product 20 is delicate or has a specific shape or relatively large size, during which the product may stick together or stick to the funnel or be damaged when pouring product 20 into collection tray 38 using the first batch of transfer modes.

[0168] When computer system 60 selects the second transfer mode, robotic arm 36 picks up the relevant container 16 contained in transport tray 14 and moves it above service plane 37, tilting or flipping it so as to pour the same product 20 contained in container 16 onto service plane 37. Then, robotic arm 36 places the emptied container 16 of product 20 back to its previous position on transport tray 14.

[0169] Thus, product 20 is randomly arranged on service plane 37.

[0170] 3D vision system 37a is arranged above service plane 37, and this 3D vision system acquires an image of product 20 arranged on service plane 37.

[0171] Computer system 60 that manages storage warehouse 30 obtains the geometric features of each product among product 20 arranged on service plane 37 from such images.

[0172] As explained in more detail below Figures 11 - 13 According to the geometric features obtained for each product among product 20, the above computer system 60 selects the most suitable gripping tool 45 for picking up each product among product 20.

[0173] Robotic arm 34 is coupled to the selected gripping tool 45 and grips each product among product 20 arranged on service plane 37.

[0174] Then, robotic arm 34 is moved to move each gripped product among product 20 one by one from service plane 37 to operating station 39 adjacent to service plane 37 and where collection tray 38 is placed. Then, robotic arm 34 releases each product among product 20 in collection tray 38 one by one according to the second transfer mode. Collection tray 38 arranged in operating station 39 can be empty or can contain some products 20 of the same type as the products picked up from service plane 37.

[0175] ReferenceFigure 5 and Figure 6 ,the collection tray 38 has a generally parallelepiped shape and includes a bottom wall 38a, four side walls 38b, and an upper peripheral edge 41 that defines a top opening 38c on a side opposite the bottom wall 38a. The top opening 38c allows access to a generally parallelepiped-shaped compartment 38d. In such a compartment 38d, a plurality of dividers 38e can be positioned, for example, orthogonal to each other to define a plurality of compartments 38f having, for example, a parallelepiped shape. In Figure 5 a non-limiting example, the number of the plurality of compartments 38f is six.

[0176] A tray identification code 43 (preferably an optical code that can be read by an optical reader) is disposed on the upper peripheral edge 41.

[0177] After the robotic arm 34 places the product 20 in the collection tray 38, the collection tray 38 is picked up from the operating station 39 and placed in a predefined position within the storage warehouse 30.

[0178] The transport tray 14 from which the product 20 inserted in the collection tray 38 has been picked up is transferred from the inspection station 35 of the storage warehouse 30 to the sorting station 33d adjacent to the inspection station 35, then from the sorting station 33d to the unloading conveyor 33c adjacent to the feeding conveyor 33a, and finally is moved away from the unloading conveyor 33c by the autonomous vehicle 50.

[0179] In Figures 8 - 10 a specific example, a non-robotic entry station 32b is provided beside the robotic entry station 32a. The non-robotic entry station 32b includes a feeding conveyor 33b and shares the sorting station 33d and the unloading conveyor 33c with the robotic entry station 32a.

[0180] In Figures 8 to 10 a non-limiting example, the feeding conveyors 33a, 33b and the unloading conveyor 33c are generally parallel to each other. The unloading conveyor 33c is located between the supply conveyors 33a, 33b.

[0181] Referring Figures 11 - 13 , an embodiment of how the computer system 60 can select the best gripping tool 45, particularly the best gripping tool-gripping point pair for gripping each product 20 arranged on the service plane 37, will now be described.

[0182] Figure 11 An example is shown in which the product 20 is randomly flipped onto the service plane 37 by the robotic arm 36 in the case of the same product 20 composed of a parallelepiped-shaped plate 20a having a pin 20b on one of the two larger faces of the plate 20a.

[0183] As inFigure 11 As can be seen in the example of Figure 11 , product 20 has a different spatial arrangement relative to service plane 37. Specifically, product 20 is positioned at different locations on service plane 37, such as at a central location, a side location, a bottom location, a top location, a right location, a left location, etc., which can be defined by the predetermined coordinates of an appropriate reference system. In addition, product 20 is arranged on service plane 37 with different support positions (i.e., product 20 is placed on service plane 37 with different surfaces, edges or support points). In addition, even if not shown, products 20 can completely or partially overlap each other and thus be inclined relative to service plane 37, for example. For example, referring to Figure 11 the position of service plane 37 in Figure 11 , the product 20 placed in the upper left is supported on service plane 37 with the largest free surface (i.e., the surface without pins); the product 20 in the center is supported on service plane 37 with the largest surface provided with pins; the product 20 placed in the lower right is supported on service plane 37 with the largest free surface (i.e., the surface without pins); the other two products 20 are supported on service plane 37 by one of the two smaller surfaces.

[0184] According to the present invention, given a predefined plurality of N reference products (where the integer N is greater than 1), computer system 60 is configured to analyze the prior (offline) geometric features of the N reference products to determine the corresponding gripping points 21 of each gripping tool 45 among the plurality of gripping tools 45 for each reference product and for the most likely stable spatial arrangement that the reference product can present relative to service plane 37 after being randomly flipped on the service plane. For example, the predefined plurality of N reference products can be defined by a list of catalog products that can be stored in storage warehouse 30.

[0185] For example, the geometric features of the reference products can be supplied to computer system 60 in the form of a three-dimensional digital model that three-dimensionally represents the shape, size, volume and center of gravity of the reference products.

[0186] The gripping point 21 defines the position of a point or area on the surface of the reference product (defined by the predetermined coordinates of an appropriate reference system), at which the reference product is adapted to be gripped by the considered gripping tool 45.

[0187] Preferably, when determining the gripping points, computer system 60 is configured to also consider other characteristic parameters of the reference products, such as weight, material, color, and possible surface features (such as roughness, porosity, anti-slip property, deformability, etc.).

[0188] Figure 11 The gripping points 21 are indicatively and by way of example shown at the Figure 11In the case of a product 20 similar to a reference product as shown, the grip point 21 can be determined for a gripping tool 45 consisting of suction cups with different spatial arrangements as shown in Figure 11 The product 20 consists of a parallelepiped-shaped plate 20a having pins 20b on one of the two larger faces of the plate 20a.

[0189] In Figure 11 's example, the grip point 21 is indicatively illustrated as a small cylinder simulating the area gripped by a suction cup having the diameter of a cylinder. In the case of other gripping tools 45, the grip point 21 can be represented by different areas or patterns at the surface of the product 20.

[0190] According to the invention, the computer system 60 is configured to also determine a prior (offline) indication score of the success probability of each grip point 21 determined for each gripping tool 45 in combination with each possible stable spatial arrangement of each reference product.

[0191] Preferably, in addition to this score, the computer system 60 is configured to also determine prior (offline) operating parameters for properly driving the considered gripping tool 45. For example, such operating parameters can be related to current, control voltage, suction pressure of the suction cup, compression pressure, gripping force, and the opening degree of the caliper, etc.

[0192] Store the results of the above analysis in Figure 12 the database 62 of the computer system 60 indicated as shown.

[0193] Therefore, the database 62 is compiled a priori (offline), and its data is usually calculated overnight whenever the technical department creates a new product and meets certain dimensional and weight constraints.

[0194] For example, the database 62 includes multiple records, indicatively illustrated as rows of a table in the schematic representation in Figure 12 Each record includes data related to the following:

[0195] - The identification code of the reference product (corresponding to the above product identification code 40);

[0196] - The geometric features of the reference product (stored, for example, in the form of a three-dimensional digital model of the product);

[0197] - The weight of the reference product;

[0198] - The possible stable spatial arrangements relative to the service plane 37;

[0199] - The gripping tool;

[0200] - The grip point 21;

[0201] - A list of defined gripping points with corresponding scores.

[0202] Specifically, in the example of Figure 12 , for a reference product identified by the identification code ID-0001 and having predetermined geometric features (indicatively indicated by xxx1 and yyyy1 respectively) and a predetermined weight, four possible stable spatial arrangements (indicatively indicated by zzz1, zzz2, zzz3, zzz4) relative to the service plane 37 are analyzed for two gripping tools (a suction cup indicatively indicated by the letter "V" and a caliper indicatively indicated by the letter "P"). The last column includes the gripping points determined for each record with the associated score. Even if not shown, for each gripping point, the last column may also include the operating parameters for driving the considered gripping tool.

[0203] According to the present invention, after the products 20 contained in the container 16 are randomly arranged on the service plane 37, the computer system 60 is adapted to identify the best gripping tool - gripping point pairs for gripping each product 20.

[0204] Figure 13 A block diagram is illustrated, which indicatively shows the method for automatically processing products implemented by the computer system 60 for each product 20 arranged on the service plane 37.

[0205] At block 70, the product identification code 40 of the product 20 is obtained.

[0206] At block 71, it is checked whether the obtained product identification code 40 exists in the database 62.

[0207] If the obtained product identification code 40 exists in the database 62, then at block 72, the geometric features of the product 20 (including, for example, at least one of the following: shape, size, volume, center of gravity, spatial arrangement relative to the service plane 37) are obtained from the image acquired by the 3D vision sensor 37a, and possibly other characteristic parameters of the product 20 (such as weight) are obtained (by a suitable sensor (not shown), which may be arranged upstream of the robot entry station 32a in the factory or at the robot entry station 32a).

[0208] At block 73, it is checked whether the geometric features of the product 20 (e.g., shape, size, and spatial arrangement relative to the service plane 37), and, where appropriate, other characteristic parameters of the product 20 correspond to the examples of product data stored in different records in the database 62 for the product identification code 40 (according to predefined correspondence criteria).

[0209] In the case of a match, at block 74, records corresponding to examples with geometric features and possibly product data are retrieved from database 62, and possibly other feature parameters corresponding to those feature parameters obtained from product 20 disposed on service plane 37 at block 72 are retrieved.

[0210] At block 74, among the retrieved records, the most suitable gripper tool 45 - gripper point 21 pair is identified based on the information contained in database 62 and based on the spatial arrangement of product 20 relative to service plane 37.

[0211] Specifically, at block 74, preferably a ranking of the gripper tool 45 - gripper point 21 pairs is defined based on the scores associated with them in the records retrieved from database 62, and a pair that allows the robotic arm 34 to safely grasp product 20 is selected from the ranked pairs with the highest scores taking into account the spatial arrangement of product 20 relative to service plane 37 and the obstacles present in the working area within the robotic entry station 32a.

[0212] Once the best gripper tool - gripper point pair has been selected, the operating parameters for driving the gripper tool of the selected pair can also be derived from the last column (when present) of database 62.

[0213] For example, for Figure 12 the database 62 shown, if the product identification code 40 of product 20 corresponds to ID - 0001 and the spatial arrangement of product 20 corresponds to zzz2, once a match is determined at block 73, at block 74, examples of product data present in the second and sixth records (second and sixth rows) of database 62 are retrieved and a ranking of the gripper tool - gripper point pairs is defined based on the data present in the last column of such records.

[0214] Returning to block 71, if it is verified that the obtained product identification code 40 does not exist in database 62, then at block 75, the most suitable gripper tool 45 - gripper point 21 pair for processing product 20 is identified through real - time processing of the point cloud provided by 3D vision system 37a.

[0215] The real - time processing can be performed by algorithms implemented, for example, by artificial intelligence, neural networks, or expert systems.

[0216] Preferably, such algorithms are assisted by appropriate machine - learning and / or statistical algorithms.

[0217] Return to block 73. If there is no match between the geometric features of product 20 and possibly other characteristic parameters of product 20 and the examples of product data in the different records stored in database 62 for product identification code 40, then set to transfer to block 76, where it is checked whether a certain threshold number of inspection iterations has been exceeded.

[0218] If the threshold number is exceeded, then set to move to block 75.

[0219] If the threshold number is not exceeded, then in block 77, the service plane 37 is actuated by the above actuator so as to change the spatial arrangement of product 20 relative to the service plane 37. As described above, the actuator can be driven to tilt, oscillate, vibrate or shake or temporarily deform the service plane 37. Preferably, the best specific action to be actuated can be selected according to the information resulting from the image processing obtained by the 3D vision system 37a. For example, it may be useful to move some products 20 towards the center or the edge of the service plane 37, etc.

[0220] After executing block 77, set to return to the execution of blocks 72 and 73.

[0221] Performing the real-time processing at block 75 advantageously allows managing the situation of product 20 arranged on the service plane 37, which is not included in a predefined plurality of N reference products. This can occur, for example, when a new product has not yet been entered into database 62.

[0222] The real-time processing performed at block 75 also allows managing the situation of product 20 that exists in database 20 but appears on the service plane 37 with geometric features different from those associated with the corresponding reference product. This can occur, for example, when product 20 becomes very different in appearance from the appearance provided by the three-dimensional digital model stored in database 62. For example, consider the case of a packaged product (e.g., a screw packaged in a parallelepiped-shaped housing) or the case where products 20 stacked or piled on the service plane 37 due to random flipping are difficult to distinguish from each other.

[0223] In the case where products 20 overlapping or piled on the service plane 37 due to random flipping are difficult to distinguish from each other, the execution of blocks 76 and 77 advantageously allows attempting to move the service plane 37 a certain number of times so as to better space products 20 apart from each other and make them more easily identifiable and distinguishable from each other.

[0224] Although not shown in Figure 13 In the case where the gripping tool 45 - gripping point 21 pair cannot be recognized at block 75, it is also possible to set to return to the execution of block 77 for further attempts before calling the operator to manage the failure.

[0225] Additionally or alternatively, although not shown in Figure 13 , in the event of a failure at box 74, it is also possible to set the execution to go to box 77 for further attempts before calling an operator to manage the failure.

[0226] Thus, the present invention allows for the selection of the best gripper-tool - gripper-point pair in order to safely pick up and handle each product 20 randomly arranged on the service plane 37 according to its geometric characteristics, preferably according to other characteristic parameters (such as the weight of the product 20), and taking into account the working area available to the robotic arm 34 within the robotic entry station 32a.

[0227] Thus, the present invention allows for picking up the product 20 from the service plane 37, handling the product 20 within the robotic entry station 32a in a safe and stable manner, and maintaining sufficient gripping stability during the handling of the product 20 by the robotic arm 34.

[0228] Obviously, in order to meet specific and possible needs, those skilled in the art will be able to make many modifications and variations to the invention described above, and moreover, all such modifications and variations fall within the scope of protection defined by the appended claims.

Claims

1. A method for automatically handling products (20) within a factory (100), the factory (100) including workstations (32a), the workstations (32a) in turn including a first robotic arm (34) and a plurality of gripping tools (45), the method comprises: - randomly arranging the products (20) on a service plane (37) of the workstation (32a); - acquiring an image of the products (20) arranged on the service plane (37); - processing the image to obtain geometric features of each of the products (20) arranged on the service plane (37); - for each of the products (20): · identifying a corresponding gripping tool (45) among the plurality of gripping tools (45) based on the geometric features that have been respectively obtained; · associating the identified corresponding gripping tool with the first robotic arm (34); · moving the first robotic arm (34) to grip the product (20) by the corresponding gripping tool (45) and moving the gripped product (20) from the service plane (37) to an operating station (39) of the factory (100), and releasing the product (20) in the operating station (39) by the corresponding gripping tool (45).

2. The method according to any one of the preceding claims, wherein, identifying a corresponding gripping tool (45) for each of the products (20) arranged on the service plane (37) among the plurality of gripping tools (45) includes: - accessing a database (62) based on the geometric features obtained by processing the image, the database (62) including examples of product data related to a predefined plurality of reference products, wherein the product data includes geometric features of the reference products, and each example of the product data is associated with one of the plurality of gripping tools (45), and the gripping tool (45) is associated with a corresponding score indicating the probability of successful gripping; - retrieving from the database (62) an example of product data having geometric features corresponding to the geometric features obtained by processing the image, and - among the gripping tools (45) associated with the example retrieved from the database (62), identifying the corresponding gripping tool (45) based on the score associated with the example retrieved from the database (62).

3. The method according to claim 1 or 2, wherein, the geometric features include at least one of the following: shape, size, center of gravity, volume, and spatial arrangement relative to the service plane (37).

4. The method according to any one of the preceding claims, wherein, identifying the corresponding gripping tool (45) includes identifying an optimal pair formed by the corresponding gripping tool (45) and a corresponding gripping pattern (21).

5. The method according to claim 4, wherein: - Each example of the product data in the database (62) is associated with one of the plurality of gripping tools (45), the gripping tool (45) is associated with a plurality of gripping patterns (21), each pair is formed by the gripping tool (45), and one of the plurality of gripping patterns (21) is associated with a corresponding score indicating the probability of a successful grip for the pair; and - In the examples retrieved from the database (62), based on the scores associated with each pair, the identification of the best pair is performed between the gripping tool and the gripping pattern associated with the gripping tool.

6. The method according to claim 4 or 5, wherein, The gripping pattern defines at least one gripping point (21) on the product and preferably defines operating parameters suitable for driving the gripping tool (45).

7. The method according to any one of the preceding claims, including obtaining at least one characteristic parameter selected from weight, material, color, and surface characteristics of the product arranged on the service plane (37).

8. The method according to claim 7, wherein, The product data in the database (62) includes at least one characteristic parameter selected from weight, material, color, and surface characteristics of the product of the predefined plurality of reference products in addition to geometric characteristics.

9. The method according to any one of claims 2 to 8 preceding, wherein, If for a product arranged on the service plane (37), the database lacks an example of product data, then the identification of the corresponding gripping tool (45) is performed among the plurality of gripping tools (45) by real-time processing of the acquired image, and the geometric characteristics of the product data correspond to the geometric characteristics obtained by processing the image.

10. The method according to any one of the preceding claims, wherein, Before randomly arranging the product on the service plane (37), the product is contained in a container (16), and wherein randomly arranging the product on the service plane (37) includes: - Picking up the container (16) by an automatic transfer device (36); - By moving the automatic transfer device (36), tilting or flipping the container (16) above the service plane (37).

11. The method according to claim 10, wherein, The container (16) is arranged in a transport pallet (14) containing a plurality of containers (16), and each container (16) includes a plurality of identical products.

12. The method according to claim 11, including before picking up the container (16) by the automatic transfer device (36): - Transferring the transport pallet (14) from an autonomous vehicle (50) to a first conveyor (33a) adjacent to the service plane (37); - Transferring the transport pallet (14) from the first conveyor (33a) to a detection station (35) adjacent to the first conveyor (33a); - Obtain an image of the transport pallet (14) at the inspection station (35).

13. The method according to claim 12, comprising checking the correct positioning of the pallet (14) and the plurality of containers (16) relative to predetermined reference parameters by processing the obtained image of the transport pallet (14).

14. The method according to any one of the preceding claims, wherein, the gripping tool (45) is selected from: suction cups, calipers, soft calipers, tipping hoppers, hooks, and any combination thereof.

15. The method according to any one of the preceding claims, wherein, the service plane (37) is at least partially elastically deformable.

16. The method according to any one of the preceding claims, wherein, the service plane (37) is actuable.

17. The method according to claim 16, wherein, the service plane can be actuated by an actuator associated therewith, and the actuators can be driven independently of each other.

18. The method according to claim 16, wherein, in terms of stroke and actuation frequency, the actuators can be driven independently of each other.

19. The method according to claim 9 and any one of the preceding claims 16 to 18, wherein, before performing the real-time processing of the obtained image, the method sequentially includes: - Actuating the service plane (37) to move at least some of the products (20) disposed on the service plane, - Processing the obtained image again to obtain updated geometric features of the products (20) disposed on the service plane (37), - Accessing the database (62) again based on the updated geometric features to retrieve examples of product data having geometric features corresponding to the updated geometric features, and among the gripping tools (45) associated with the examples retrieved from the database (62), identifying the corresponding gripping tool (45) based on a score associated with the examples retrieved from the database, If the database (62) lacks examples of product data, perform the real-time processing of the obtained image, and the geometric features of the product data correspond to the updated geometric features.

20. A factory (100) including a workstation (32a), the workstation (32a) further including a first robotic arm (34), a service plane (37), a plurality of gripping tools (45) disposed near the service plane (37), an image acquisition device (37a) configured to acquire an image of a product (20) disposed on the service plane (37), and a computer (60) configured to process the image to obtain geometric features of each of the products (20) disposed on the service plane (37), and for each of the products (20), the computer (60) is configured to: - Identify the corresponding gripping tool (45) among the plurality of gripping tools (45) based on the geometric features respectively obtained by processing the image; - associate the respective gripping tool (45) with the first robotic arm (34); and - move the first robotic arm (34) to grip the product by the respective gripping tool (45), move the gripped product from the service plane (37) to the operating station (39) of the factory (100), and release the product in the operating station (39) by the respective gripping tool (45).

Images