System for individually handling articles that is capable of dealing with instances of multiple articles being picked up at once, and associated method

Abstract

Disclosed is a method for individually handling articles that is capable of managing the risk of instances of multiple articles being picked up at once, the method comprising the steps of: transporting the articles on a first collective-movement unit (CONV); acquiring an image of an articles pickup zone (PickZ) by a first image sensor (CAM1); analysing the image and locating an article (IT); commanding an individual handling unit (MA) provided with a second image sensor (CAM2) and positioning the second sensor above the located article; acquiring an image (IMG2) of this article; analysing this image (IMG2) and evaluating a risk of the individual handling unit (MA) picking up multiple articles at once; then commanding the individual handling unit (MA) to pick up the article and move it to a destination (INS, RET, REJ) chosen from at least a reject bin (REJ) and a second collective-movement unit (INS).

Description

INDIVIDUAL ARTICLE HANDLING SYSTEM CAPABLE OF PROCESSING MULTIPLE PLUGS, AND ASSOCIATED METHOD TECHNICAL FIELD OF THE INVENTION

[0001] The invention relates to a system for individual handling of articles between two units for collective movement of these articles, for example for the individual placement of these articles on a conveyor belt during a picking operation in a sorting installation. TECHNOLOGICAL BACKGROUND

[0002] Systems for sorting items such as luggage, parcels, postal items use conveyors, such as conveyor belts, capable of transporting a plurality of items simultaneously and / or gripping robots capable of handling each of these items individually.

[0003] Conveyors and gripper robots can be used in combination, for example for a bin picking operation: the articles are originally in bulk and possibly stacked on a first conveyor belt, then are handled individually by a gripper robot for individualized placement, for example on an inserter responsible for introducing each of the articles into a machine dedicated to sorting these articles. The inserter can be formed by a second conveyor belt.

[0004] The gripper robot is equipped with a gripper head, which is responsible for gripping the items to be moved individually from the first conveyor belt to the inserter. However, it sometimes happens that the head accidentally grips more than one item, this is called a "multiple grip". This is the case, for example, when a gripper head uses a plurality of suction cups to grip items, and some of the suction cups attach to a first item and others attach to a second item. Such an event is accidental and preferably avoided. However, when it occurs, it is necessary to detect it and then handle the multiple gripped items in a particular way, generally by sending them onto an alternative sorting path or by preventing all but one of the items placed on the second conveyor belt from continuing its movement on this second belt.In this regard, reference may be made to patent documents US 4,733,226 and FR 2,546,083. Regardless of the specific treatment applied to the multiple items seized, the objective is to avoid the simultaneous insertion of several objects into the sorting machine. The risks associated with such insertion range from the sorting or orientation error of an item to jamming and breakage of a sorting machine.

[0005] Some systems anticipate the risk of multiple picking by sending a set of insufficiently sorted items on the first conveyor belt, very close to each other or even stacked, on an alternative sorting path, and therefore risking causing multiple picking.

[0006] However, the implementation of such methods requires optimization of multiple take detection, limiting false positives in order to reduce undue differential treatment of items perceived as posing a risk of multiple take, without significantly reducing the throughput of items.

[0007] An object of the invention is an individual item handling system capable of managing a risk of simultaneous picking of more than one item by anticipating the occurrence of a multiple pick.

[0008] To achieve this aim, a first aspect of the invention is a method for individual handling of articles capable of managing a risk of simultaneous picking of more than one article, the method comprising the steps of transporting articles on a first collective article movement unit and bringing them to a picking area; acquiring an image of the picking area by means of a first image sensor; analyzing the image of the picking area by means of a computer system so as to locate an article of the picking area; in response to the analysis of the image of the picking area, the computer system controls an individual article handling unit provided with a second image sensor, so as to position the second image sensor above the located article; acquiring an image of the located article by means of the second image sensor positioned above the located article;analyzing the image of the located item by means of the computer system so as to assess a risk of multiple picking in the event of the located item being picked up by the individual item handling unit, and in particular assessing whether the risk of multiple picking is associated with a probability lower or higher than a certain predefined value; in response to the analysis of the image of the located item, controlling the individual item handling unit to pick up the located item and move it to a destination selected from at least one reject bin and a second collective item moving unit.;

[0009] One application of this system is to avoid the simultaneous insertion of more than one item into a sorting machine while limiting the drop in the rate of handling of items and therefore their sorting. In particular, the use of two cameras, one of which is brought directly above an item to be processed, makes it possible to improve the quality of an image serving as a basis for detecting a risk of multiple picking and therefore to improve its reliability. Mechanically, the number of specific treatments aimed at limiting multiple transfers of items from the first collective item movement unit to the second collective item movement unit is reduced, which improves the throughput of the sorting system considered.

[0010] According to additional non-limiting characteristics of the first aspect of the invention, considered individually or in any technically feasible combination:

[0011] - the image of the located item may have a resolution higher than a resolution of the image of the sampling area, these resolutions being able to be expressed in pixels per unit of length;

[0012] - if the probability is greater than the certain predefined value, then the individual item handling unit can grab the located item and place it in the reject bin;

[0013] - if the probability is less than the certain predefined value, then the individual item handling unit can move the located item to a second collective item moving unit;

[0014] - analyzing the image of the located item may include a step of determining whether an address block is located on a visible face of the located item;

[0015] - if the computer system determines that the probability is less than the certain predefined value and the address block is not located on a visible face of the located item, then the individual item handling unit can move the located item to an item turning unit;

[0016] - the article turning unit can receive and turn the article brought to it and then can move the returned article to the second collective article movement unit;

[0017] - if the probability is less than the certain predefined value and if the address block is located on a visible face of the pickable item, then the individual item handling unit can move the located item to a second collective item moving unit;

[0018] - the second collective item movement unit can be a sorting machine inserter;

[0019] - when acquiring the image of the located item, the second image sensor is positioned at a predefined height above the considered located item.

[0020] The invention extends to an individual article handling system capable of managing a risk of simultaneous picking up of more than one article and configured to implement the method according to the invention, the system comprising a first unit for collectively moving articles; a first image sensor configured to acquire images of a picking area of ​​the first unit for collectively moving articles, a second unit for collectively moving articles; an individual article handling unit, provided with a second image sensor and configured to move articles from the first unit for moving articles to the second unit for moving articles;and a computing device configured to control the individual article handling unit, the computing device being operatively connected to the first image sensor and the second image sensor, and configured to control the individual article handling unit in response to data generated by the first sensor and the second sensor.;

[0021] The system according to the invention may further comprise an article turning unit configured to turn an article brought thereto and move the turned article to the second collective article movement unit. BRIEF DESCRIPTION OF THE FIGURES

[0022] Other characteristics and advantages of the invention will emerge from the detailed description of the invention which follows with reference to the appended figures in which:

[0023] It represents an overall view of an installation in which the method according to the invention can be applied;

[0024] Illustrates a system capable of implementing the method according to the invention, using a top view;

[0025] This is a diagram comprising the steps of implementing a method according to the invention; and

[0026] This is a variation of the process according to the diagram of the. DETAILED DESCRIPTION OF THE INVENTION

[0027] Method of implementation

[0028] Figures 1 to 4 illustrate an embodiment of a handling method according to the invention, as well as an associated system for handling articles, applied to the particular case of an installation for sorting these articles.

[0029] The illustrates an installation FAC for sorting articles, comprising a PickZ picking zone in which IT articles to be sorted brought by a truck Trck are brought in bulk by means of a first unit CONV for collective movement of articles. A robotic manipulator arm MA, constituting an individual article handling unit, is configured to pick up the IT articles one by one and place them on a second unit INS for collective movement of articles. The second unit for collective movement of articles is in the present embodiment an inserter INS configured to insert articles into a sorting machine SORT. This sorting machine may for example comprise a circular conveyor belt CCB receiving the articles from the inserter INS and sorting outlets Out (three outlets shown here) towards which articles are directed according to their final destination.The CONV and INS collective item movement units are systems for collective item movement and can independently consist of one or more conveyors such as conveyor belts or rollers. The installation can, for example, be designed to sort postal parcels and divide them into distribution rounds, each corresponding to one of the Out sorting outlets.

[0030] More specifically, it illustrates a SYS system for individual item handling integrated into the FAC sorting installation, with the first CONV unit for collective item movement CONV with its PickZ picking zone, the second INS unit for collective item movement with its DZ depositing zone, and the MA manipulator arm for individual item handling with its GH gripping head.

[0031] The first CONV unit for collective movement of articles moves the IT articles from a bulk unloading area of ​​the truck Trck to the picking area PickZ in a direction Dir1 represented by an arrow in the. The gripping head GH can be equipped with suction cups connected to a pumping system, each suction cup can be controlled individually so as to apply a vacuum to the surface of an article to be picked up in the picking area PickZ.

[0032] A CONT unit for controlling the sorting line, and therefore the SYS system for individual handling of articles, comprises a first digital camera and a second digital camera, such as cameras, respectively as image sensors CAM1 and CAM2, and a computer system INF comprising a computer memory and being functionally connected to the sensors CAM1 and CAM2, and to the manipulator arm MA.

[0033] The first CAM1 sensor is configured to acquire images each representing a general view of the PickZ sampling area. The first CAM1 sensor can be adapted to the acquisition of a pair of digital images capable of forming a depth map of said scene according to known principles of photogrammetry.

[0034] The second CAM2 sensor is mounted on the manipulator arm, preferably on or near the GH gripping head, and configured to acquire images each representing one or more items located under the gripping head.

[0035] In the present embodiment, the articles IT brought into the picking zone PickZ by the first unit CONV for collectively moving articles, which are manipulated by the manipulator arm MA so as to make them pass from a disordered state resulting from the bulk unloading of the truck Trck on the first unit for moving articles CONV to an ordered state on the second unit INS for collectively moving articles. The manipulator arm MA has the function of individually manipulating these articles by grasping them one by one to deposit them one by one in a deposit zone DZ of the second moving unit INS.

[0036] The disordered state is characterized by an irregular distribution of the articles, possibly with articles piled on top of each other or simply placed next to each other. The ordered state is characterized, for example, by placing the articles one by one on a continuously rotating conveyor belt, in such a way that each of the articles is placed in the middle of the conveyor belt, according to a constant pitch P and defined as the distance separating two of the articles placed consecutively on the conveyor belt by the manipulator arm.

[0037] The second article collective movement unit INS is configured to move the articles placed therein to an entrance of the sorting machine SORT, in a direction Dir2 which is defined as going from an area DZ for depositing the articles by the manipulator arm MA on the second article movement unit INS to an entrance of the sorting machine SORT. The articles, entrusted in an orderly manner to the second article movement unit INS, can be correctly processed by the sorting machine SORT.

[0038] As an alternative to the orderly deposit of the articles in the deposit zone DZ of the second movement unit INS, when the conditions for transmitting an article from the first to the second collective article movement unit are not met, the manipulator arm can bring the processed articles to one or other of a reject bin REJ and an article turning unit RET, according to the methods detailed below.

[0039] Illustrates a method 100 for sorting articles using, for example, the FAC sorting installation illustrated by figures 1 and 2.

[0040] In a step S10, IT articles are placed in bulk on the first CONV unit for collective movement of articles, at an article reception end, located opposite the PickZ picking zone. This may, for example, involve the bulk unloading of the contents of the truck Trck.

[0041] In a step S20, the articles placed in bulk are brought collectively to the picking area PickZ by the CONV unit for collective movement of articles. During this movement, a granulation can be carried out, reducing the density of the articles in order to facilitate their individual handling in step S80.

[0042] In a step S30, the sensor CAM1 acquires an image IMG1 of the sampling area. The image IMG1 has a first resolution, for example of approximately 1 pixel per millimeter, which means that each pixel of the image represents approximately 1 mm of the sampling area.

[0043] At a step S40, the computer system INF carries out an analysis Anal1 of the image IMG1 of the sampling area, using conventional segmentation and / or photogrammetry techniques, so as to isolate one of the IT articles visible in the image.

[0044] A processing following the segmentation can calculate the attributes of the articles: position, height, orientation, or even inclination, attributes on the basis of which the computer system can possibly associate with each article a pickability score calculated so as to define an order of priority in the gripping of the articles by the manipulator arm, for example on the basis of criteria such as (i) the distance separating the article in question from the drop-off zone DZ of the second unit INS for collective movement of articles to which it must be taken by the manipulator arm, (ii) the risk of collision with another article on the path to the drop-off zone, (iii) the possibility of gripping the article and / or (iv) the expected quality of gripping in consideration of the type and configuration of the gripping head used and the geometry, dimensions and arrangement of the article in question.The items are ordered in descending order of pickability score: the easiest items to pick, i.e. those that minimize the risks of collision and dropping during movement while optimizing flow, will be chosen first. Reference may be made to French patent application FR2305495.

[0045] The isolated item can advantageously be the one with the highest pickability score among those present in image Img1, which corresponds to a form of optimization in the choice of the order of individual movement of the items present in the picking zone.

[0046] In a step S50, based on the analysis of the image IMG1 of the picking area, the computer system INF controls the individual article handling unit MA by means of a command COM, so as to position its gripping head GH and the second image sensor CAM2 above the located article. Preferably, the second image sensor is always placed at the same predefined height, typically 200 mm, above an upper surface of an article to be picked up, with a view to acquiring an image of this article.

[0047] In a step S60, the second image sensor CAM2 acquires an image IMG2 of the located article by means of the second image sensor CAM2 positioned above the located article, separated from the predefined height of the surface of the article.

[0048] In a step S70, the computer system INF carries out an analysis Anal2 of the image IMG2 of the located article so as to evaluate a probability of multiple taking in the event of seizure of the located article by the individual article handling unit MA. Step S70 comprises a test step S72, consisting of evaluating (action indicated by "MULT?" on the diagram of the) whether a risk of multiple taking of the located article exceeds (event indicated by Y on the diagram) a given threshold or whether, conversely, this probability is lower than this threshold (event indicated by N on the diagram).

[0049] Such an analysis can, for example, be based on the use of a deep neural network, advantageously on a VGG19 model with two output neurons, the activation of one corresponding to the detection of articles leading to so-called "single" grips when the network evaluates that a single article of the image IMG2 will be grasped by the grasping head of the manipulator arm, the activation of the other corresponding to the detection of articles leading to multiple grips when the network evaluates that more than one article of the image IMG2 will be grasped by the grasping head of the manipulator arm. The objective of the neural network, after learning, is to correctly associate the images presented to it with different classes, "multiple grip" or "single grip" here, by activating the correct output neuron.This architecture is relatively simple but presents good results on simple classification problems (two or three output classes) and above all a controlled execution time.

[0050] The computation time of VGG19 models depends almost entirely on the size of the input image. The native images of the CAM2 on-board camera contain a significant portion of background, so it is advantageous to select only a portion of the IMG2 image centered on the item to be sampled.

[0051] The main difficulty in training the neural network for multiple tap detection lies in the fact that it is an asymmetric problem: the rate of multiple taps in a flow of articles generally fluctuates between 1 and 2%, while that of single taps is higher than 98%. Thus, 1% of misclassification of single taps (false positives) represents approximately 1% of the flow while 1% of misclassification of multiple taps (false negatives) represents barely 0.02% of the flow. It is generally very difficult to reduce the number of false positives and false negatives simultaneously. For exploitation of the SYS system in real conditions, it may be advantageous to keep the rate of false positives lower than 1% and to reduce the rate of false negatives as much as possible, which the method and the system according to the invention make it possible to do effectively.

[0052] One way to improve the quality of neural network learning is to increase the proportion of images representing multiple shots in the training databases to approach perfect parity where the number of images representing single shots would be equal to the number of images representing multiple shots. To do this, we can use methods called "Data augmentation", in which data is artificially produced, here to increase the proportion of images representing multiple shots.Data augmentation can consist of transforming images from the training database using random color distortions by modifying RGB (red, green, blue) colors, random contrast adjustments, random horizontal inversions, random brightness adjustments, random saturation adjustments, rotations, or elastic deformations from random displacement fields.

[0053] Furthermore, acquiring IMG2 images acquired from a constant height above an item of interest and relatively close to it (closer than the first CAM1 image sensor) makes it possible to feed a learning base with IMG2 images of a better resolution (for example of the order of 6 to 10 pixels per millimeter) than the IMG1 images, which makes it possible to further improve the quality of the neural network's learning and the relevance of its response when analyzing an IMG2 image by providing images as close as possible to the images obtained under real operating conditions of the SYS system.

[0054] It should be noted that the neural network does not deliver certain answers, but associates an image provided to it as input with a class defined by the user and by the network's training phase. This association can only be guaranteed with a certain level of probability depending on the training and the user's requirements. In other words, at the output of step S70, we can only ensure whether or not the probability for an article to lead exceeds a certain threshold, evaluated according to the performance of the neural network following its training.

[0055] At a step S80, in response to the analysis of the image IMG2, and therefore to a probability of multiple taking of the located article, the computer system INF commands the unit MA for individual article handling to grasp the located article and move it to a destination chosen from a reject bin REJ and a second unit INS for collective movement of articles.

[0056] Thus, if the probability of multiple taking is greater than a certain predefined value, then the individual article handling unit MA picks up the located article and places it in the reject bin REJ. In the case where a neural network as described above is employed to carry out the analysis of the image IMG2, such a probability results in the activation of the neuron corresponding to the detection of articles leading to multiple taking. In a step S90, articles accumulated in the reject bin REJ can be returned to the first collective article movement unit CONV by means of a return action Bck, for a new attempt at transmission to the second collective article movement unit (INS). Step S90 can be carried out manually or by any conventional automatic article handling means.

[0057] Conversely, if the probability of multiple picking is less than a certain predefined value, then the individual article handling unit MA moves the located article to the second collective article moving unit INS, on which it deposits the article at a depositing zone. In the case where a neural network as described above is used to carry out the analysis of the image IMG2, such a probability results in the activation of the neuron corresponding to the detection of articles leading to single pickings. In a step S100, the second collective moving unit INS inserts the article deposited in its depositing zone DZ into the sorting machine SORT.

[0058] Illustrates a variant 100' of the method 100 illustrated by the. An advantage of this variant is that it can be implemented using equipment already present on installations configured to detect the presence of a destination address block of the article on a visible face of the latter, and to turn it over if necessary to make the block the address block visible. For the steps with the same identifier as a step of the method 100, reference can be made to the explanations concerning the latter.

[0059] From an explanatory point of view, the variant 100' differs from the method 100 in that the step S70 of analyzing the image IMG2 is enriched with an additional test step S74 provided to detect the presence of an address block on a visible face of the article, visible in the image IMG2. The enriched step S70 is identified as step S70' in the diagram of the, during which an analysis Anal2' of the image IMG2 is carried out.

[0060] During step S70', in the case where it is estimated, following step S72 of evaluating the risk of multiple taking, that the located article does not present a priori a danger of multiple taking, the computer system INF proceeds to the test step S74 on the basis of the same image IMG2. Reference may be made to the European patent application published under number EP 3 984 922 A1.

[0061] Test step S74 consists of detecting (action indicated by "ADR?" on the diagram) whether an address block is present on a visible face of the located item (event indicated by Y on the diagram) or whether, conversely, no address block is detected on a visible face of the item (event indicated by N on the diagram).

[0062] Steps S72 and S74 illustrate the detection principle, which could be implemented by employing two separate tests as illustrated by steps S72 and S74 of the diagram of the. Step S74 could also be implemented by means of a second neural network according to a principle similar to that detailed for step S72, using as a training basis annotated images indicating the presence or absence of the address block.

[0063] Another method consists of implementing the analysis Anal2' using a neural network such as that mentioned for the implementation of step S72, this time defining three output classes of the network: a first for the articles likely to lead to a multiple take, a second for the articles not likely to lead to a multiple take but not having a visible address block, and a third for the articles not likely to lead to a multiple take and having a visible address block. In this case, the classification of the article located in the first, second or third class constitutes the analysis step S70' of the image IMG2. The training of the neural network is done using a bank of images annotated to indicate their class of belonging among the three classes defined above.

[0064] At a step S80', the computer system INF controls the individual item handling unit (MA) in response to the analysis step S70' of the image IMG2.

[0065] In case the probability of multiple capture is higher than a certain predefined value (i.e. the image is considered by the neural network as belonging to the first class), then the individual item handling unit MA picks up the located item and places it in the reject bin REJ.

[0066] In case the probability is less than the certain predefined value and the address block is not detected on a visible face of the located item (i.e. the image is considered by the neural network as belonging to the second class), then the individual item handling unit MA moves the located item to an item turning unit RET, which turns the item over and transmits it to the second collective item moving unit INS.

[0067] In case the probability is less than the certain predefined value and if the address block is detected on a visible face of the pickable item (i.e. the image is considered by the neural network as belonging to the third class), then the MA unit for individual item handling moves the located item to the second INS unit for collective item movement.

[0068] The figures in this document are not necessarily to scale. Some features and components may be shown exaggerated in relation to other components or in a somewhat schematic form, and some details of conventional items may not be shown in the interest of clarity and conciseness.

[0069] Of course, the invention is not limited to the method of implementation described and variant embodiments can be made without departing from the scope of the invention as defined by the claims.

Claims

Method (100) for individual handling of articles capable of managing a risk of simultaneous picking of more than one article, the method comprising the steps of:- transporting (S20) articles on a first unit (CONV) for collective movement of articles and bringing them to a picking zone (PickZ);- acquiring (S30) an image (IMG1) of the picking zone (PickZ) by means of a first image sensor (CAM1);- analyzing (S40) the image (IMG1) of the picking zone by means of a computer system (INF) so as to locate an article (IT) of the picking zone;- in response to the analysis of the image of the picking zone, the computer system (INF) controls (S50) a unit (MA) for individual handling of articles provided with a second image sensor (CAM2), so as to position the second image sensor above the located article;- acquiring (S60) an image (IMG2) of the located article by means of the second image sensor (CAM2) positioned above the located article;- analyzing (S70, S70') the image (IMG2) of the located article by means of the computer system (INF) so as to evaluate a risk of multiple taking in the event of the located article being seized by the individual article handling unit (MA), and in particular evaluating whether the risk of multiple taking is associated with a probability lower or higher than a certain predefined value;- in response to the analysis of the image of the located article, controlling (S80, S80') the individual article handling unit (MA) to seize the located article and move it to a destination (INS, RET, REJ) chosen from at least one reject bin (REJ) and a second unit (INS) for collective movement of articles.; Method (100) according to claim 1, wherein the image (IMG2) of the located article has a resolution greater than a resolution of the image (IMG1) of the picking zone (PickZ), these resolutions being able to be expressed in pixels per unit of length. Method (100) according to claim 1 or 2, wherein, if the probability is greater than the certain predefined value, then the individual article handling unit (MA) picks up the located article and places it in the reject bin (REJ). Method (100) according to any one of claims 1 to 3, wherein, if the probability is less than the certain predefined value, then the individual article handling unit (MA) moves the located article to a second collective article moving unit (INS). Method (100') according to any one of claims 1 to 3, in which the analysis of the image (IMG2) of the located article comprises a step (S74) of determining whether an address block (AD) is located on a visible face of the located article. Method (100') according to claim 5, wherein, if the computer system determines (S70') that the probability is less than the certain predefined value and that the address block is not located on a visible face of the located article, then the individual article handling unit (MA) moves the located article to an article turning unit (RET). Method (100') according to claim 6, wherein the article turning unit (RET) receives and turns the article brought to it and then moves (S90') the returned article to the second unit (CONV) for collective movement of articles. Method (100') according to claim 4, wherein, if the probability is less than the certain predefined value and if the address block is located on a visible face of the pickable article, then the individual article handling unit (MA) moves the located article to a second collective article moving unit (INS). Method (100, 100') according to claim 4 or 8, wherein the second unit (INS) for collective movement of articles is a sorting machine inserter (SORT). Method (100, 100') according to any one of the preceding claims, wherein, during the acquisition (S60) of the image (IMG2) of the located article, the second image sensor (CAM2) is positioned at a predefined height above the localized article considered. System (SYS) for individual handling of articles capable of managing a risk of simultaneous picking of more than one article and configured to implement the method according to any one of the preceding claims, the system comprising:- a first unit (CONV) for collective movement of articles;- a first image sensor (CAM1) configured to acquire images of a picking zone (PickZ) of the first unit (CONV) for collective movement of articles,- a second unit (INS) for collective movement of articles;- a unit (MA) for individual handling of articles, provided with a second image sensor (CAM2) and configured to move articles (IT) from the first unit (CONV) for moving articles to the second unit (INS) for moving articles;and- a computing device (INF) configured to control the individual article handling unit (MA), the computing device being operatively connected to the first image sensor (CAM1) and to the second image sensor (CAM2), and configured to control the individual article handling unit (MA) in response to data generated by the first sensor (CAM1) and the second sensor (CAM2).; The system of claim 11, further comprising an article turning unit (RET) configured to turn an article brought thereto and move the turned article to the second collective article moving unit (CONV).

Images