Control system and method for machine for preparing and dispensing hot drinks by controlled infusion of precursor substance in particulate or powder form based on identification and classification of dose units inserted in machine

Abstract

A control system and method for a machine (1) for preparing and dispensing hot drinks by controlled infusion of a precursor substance in particulate or powder form based on identification and classification of dosage units (8) inserted in the machine (1). A control system and method are described for a machine (1) for preparing and dispensing hot drinks by controlled infusion of a precursor substance in the form of particles or powder arranged in a dosage unit (8), an automatic image recognition and processing unit (84) of the machine learning type is configured to classify a dose unit (8) received by the machine as one of a plurality of predetermined categories of dose units based on a set of training images of the dose unit taken in the learning step. The training image set of the dose unit comprises at least a plurality of primary images of the dose unit (8) and a plurality of composite images obtained by changing the plurality of primary images, the primary images comprising respective predetermined graphical identification marks (70, 72) on a reference surface of the dose unit (8) indicating a corresponding category of the dose unit.

Description

Technical Field

[0001] The present invention relates to a machine for preparing and dispensing hot beverages by controlled infusion of dosage units containing precursor substances of one or more ingredients, such as coffee.

[0002] More specifically, the invention relates to a control system and a method for a machine for preparing and dispensing hot beverages by controlled infusion of a precursor substance in granular or powdered form arranged in dosage units, according to the preambles of claims 1 and 25, respectively. Background Art

[0003] Machines for preparing and dispensing hot drinks by controlled infusion of dosage units are known, which machines comprise means for automatically recognizing a dosage unit inserted into the machine, so as to classify the dosage unit as one of a plurality of predetermined types or categories of dosage units and thereby control at least one parameter of the infusion or the operating mode of the machine according to the type of dosage unit identified, or to record and / or transmit machine usage data to a remote management system.

[0004] Automatic identification can be achieved using machine learning techniques, such as through pattern recognition methods or artificial neural networks based on predefined classification schemes.

[0005] Utility model IT 274555 of the same applicant describes a machine for preparing beverages, in particular coffee, wherein the pods or capsules used by the machine are selected from a plurality of types of pods or capsules having different characteristics, in particular having the same size characteristics but containing different substances or ingredients. The machine comprises: a detector device adapted to detect at least one predetermined detectable characteristic or code, which identifies the type of pod or capsule inserted into the machine; and a processing and control unit connected to the detector device, the processing and control unit being configured to implement an operating cycle of the machine that varies based on a predetermined method according to the type of pod inserted into the machine and identified by the detector device, and / or the processing and control unit being configured to collect data indicating the method in which the user uses the machine, such as the total amount and / or the individual number of pods of various types used in a given period of time, possible use of "non-original" compatible pods, etc.

[0006] A detectable feature or code is for example a color, or a barcode, or a so-called RFID tag.

[0007] Automatic identification of dosage units can also be achieved using machine learning techniques, for example by pattern recognition methods or artificial neural networks based on predefined classification schemes, as taught by international patent application WO 2019 / 154527.

[0008] WO 2019 / 154527 discloses a machine for preparing and dispensing beverages (such as tea, coffee, hot chocolate, cold chocolate, milk, soup or baby food), the machine comprising a module for identifying a capsule inserted into the machine at an identification position, the module comprising a camera for capturing an image of at least a portion of the capsule and a neural network computing device configured to determine the type of the capsule from a plurality of predefined capsule types based on the image of at least a portion of the capsule captured by the camera.

[0009] Although machines for preparing and dispensing beverages of the above-mentioned type ensure automatic recognition of packets or capsules, these machines are not suitable for accurately recognizing and classifying objects that may change their characteristics over time, such as coffee tablets or other precursor substances in granular or powdered form, which may deteriorate due to, for example, tablet storage conditions or exposure to non-optimal temperature or humidity environmental conditions or handling by the user.

[0010] Furthermore, even in the case of packaged dosage units, such as bags or capsules, whose outer surface is not perishable, the cleanliness state of the image acquisition optics located near the infusion chamber, or the wear of the illumination means of the dosage unit and the means for acquiring the image during the life of the machine (especially if they are low-cost devices), may cause artifacts when acquiring the image of the dosage unit inserted into the machine, on which the automatic recognition process is performed, leading to misclassification.

[0011] Other factors that influence the classification accuracy are: the randomness of the spatial position of the dosage unit inserted into the machine when the image for identification is acquired, in particular if the image is acquired during the transfer path of the dosage unit to the infusion chamber, for example due to the falling of the dosage unit's containing compartment or due to moving the dosage unit's containing compartment, in which the dosage unit is not stationary. Summary of the invention

[0012] The object of the present invention is to provide a satisfactory solution to the above-mentioned problems, avoiding the disadvantages of the prior art.

[0013] More specifically, the object of the present invention is to provide a control system for a machine for preparing and dispensing beverages based on an accurate and robust classification of dosage units of a beverage precursor substance supplied to the machine, wherein the classification is as independent as possible of the conditions of the dosage units and of the machine throughout the operating life of the machine.

[0014] Another object of the invention is to enable the assortment of dosage units to adapt to changes in machine operating conditions or changes in dosage unit types that may occur over time.

[0015] Another object of the present invention is to provide a control system that can be manufactured and integrated in a machine for preparing and dispensing beverages in a cost-effective manner, for example for domestic use where size and cost are limited.

[0016] These objects are achieved according to the invention by a control system for a machine for preparing and dispensing hot beverages having the features set forth in claim 1 .

[0017] The detailed description forms the subject matter of the dependent claims, whose content is to be understood as an integral part of the present description.

[0018] Another subject matter of the invention is a method for controlling a machine for preparing and dispensing hot beverages, the method having the features according to claim 25 .

[0019] In summary, the present invention is based on the following principle: automatic identification of dosage units inserted into a machine by machine learning classification techniques, preferably by neural networks or similar deterministic classification models, wherein the classification is based on a set of dosage unit training images representing a plurality of predetermined types or categories of dosage units in a plurality of conditions for capturing images of dosage units. Specifically, the dosage units include corresponding graphic identification marks indicating the category of the dosage units, and the set of training images of the dosage units includes a plurality of primary (or original) images of dosage unit samples pre-classified in a supervised manner and a plurality of synthetic images obtained by modifying the plurality of primary images by data augmentation techniques to take into account possible changes in the actual samples of dosage units or the devices and conditions for capturing images of dosage units during the operational life cycle of the machine, and to increase the amount of images used for training.

[0020] Advantageously, creating a plurality of composite images makes it possible to simulate images of degraded dosage units, artifacts of intact dosage units, for example due to degradation of the image acquisition device or due to non-optimal alignment of the dosage unit relative to the image acquisition device or images of dosage units that differ from a sample dosage unit; for example in the case of dosage units for different beverages such as espresso or Americano, these dosage units have different sizes, so that the acquired image may be susceptible to alterations due to optical distortions.

[0021] Advantageously, the graphic identification marking indicating the type of dosage unit comprises an inscription identifying the product line and a further denotation marking, thereby allowing a first classification based on the product line of the dosage units and a second classification based on the mixture of precursor materials.

[0022] The term "graphic mark" refers to: surface (two-dimensional) graphic marks obtainable, for example, by printing or other application of an edible additional substance, thermal marking or by application of a localized laser beam, which marks exhibit a color different from the color of the precursor substance of the dosage unit, or contrasting with the color of the precursor substance, or in any case exhibit a luminous intensity or brightness different from the luminous intensity or brightness of the precursor substance in response to irradiation of the dosage unit; or raised or notched (three-dimensional) graphic marks obtained by modifying the morphological properties of the surface of the precursor substance of the dosage unit (for example by applying an edible additional substance or by mechanical and / or thermal imprinting on predetermined areas of the surface of the precursor substance adapted to form a relief or engraving), which marks exhibit changed reflective properties and / or shadows when irradiated.

[0023] In one embodiment, the automatic recognition of the dosage unit can be performed by identifying the graphic identification mark in the acquired image and determining a comparison index, which represents the similarity between the graphic identification mark (or a portion thereof) of the acquired image and the graphic identification mark (or a portion thereof) of the training image of the dosage unit or the set of reference images of the dosage unit class. The similarity is defined by one or more comparison thresholds, which are preferably dynamically modifiable during the life cycle of the machine.

[0024] It is also conceivable that the automatic recognition of the dosage unit is achieved after a correction process of the acquired image, or even based on the time the machine has been running or the number of dispensing actions performed, due to a deterioration in the classification accuracy or similarity in the comparison between the graphic recognition signature of the acquired image and the graphic recognition signature of the training image of the dosage unit or the set of reference images of the dosage unit class, or even based on the time the machine has been running or the number of dispensing actions performed. The image correction process can be achieved by applying a predetermined correction model or by updating the characteristic parameters of the classification model according to instructions received from an external management system, which is temporarily connected to the machine control system via a local wired connection or a remote wireless connection. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Further features and advantages of the invention will appear in more detail in the following detailed description of embodiments of the invention given by way of non-limiting example with reference to the accompanying drawings, in which:

[0026] Figure 1 is a schematic cross-sectional view of a machine for preparing and dispensing hot drinks by infusion according to the invention;

[0027] Figure 2a to Figure 2c is an exemplary illustration of a coffee dosage unit in tablet form with a graphical identifying mark;

[0028] Figure 3 is used for Figure 1The block diagram of the control system of the machine;

[0029] Figure 4 Is Figure 3 A block diagram of the neural network used in the system;

[0030] Figure 5 yes Figure 4 An exemplary illustration of elements of a neural network;

[0031] Figure 6 is an illustration of a plurality of composite images obtained by varying a primary image of a sample of dosage units;

[0032] Figure 7 is a graphical representation of a plurality of additional composite images obtained by processing a primary image of a sample of dosage units; and

[0033] Figure 8a and Figure 8b is a flow chart of a training method and a control method of a machine for preparing and dispensing hot drinks by infusion according to the invention. DETAILED DESCRIPTION

[0034] refer to Figure 1 , a machine for preparing a beverage, in particular coffee, from a compacted tablet comprising one or more ingredients, in particular coffee powder, is indicated as a whole by 1.

[0035] Although the present invention is described with reference to the use of a compacted tablet comprising one or more ingredients for preparing a beverage, it should not be understood as being limited to dosage units in the form of tablets, but it is also applicable to situations where, instead of tablets, a dose of a powder contained in a capsule, a bag or other similar packaging suitable for preparing a beverage by infusion is provided. For the purposes of the present invention, the term "precursor substance in granular or powder form" therefore contemplates products packaged as rigid capsules or flexible bags as well as compacted products in the form of tablets.

[0036] In the embodiment shown by way of example, the machine 1 comprises a machine body 2 having an operating area 4. The machine 1 has a lever 6 for pushing a dosage unit 8 into the area 4 through an introduction opening 10, a closing device 20 operating on the dosage unit 8 to close it in an infusion chamber delimited by two opposing cooperating elements 22, 24 (and possibly piercing the coating membrane of the bag or the cover of the capsule), and a circulation system 26 for circulating a flow of hot water and / or pressurized steam through the dosage unit in the infusion chamber.

[0037] The dosage unit 8 contains a dose or portion of granular or powdered material intended for infusion to produce a hot drink, in the form of a tablet having a self-supporting structure not requiring a casing, or packaged in a flexible, water-permeable wrapper (typically a paper wrapper, which is usually referred to as a "pack"), or in a more or less rigid capsule.

[0038] The operating lever 6 is rotatable about a fulcrum 28 and is adapted to push the dosage unit 8 through the opening 10 towards the operating area 4 .

[0039] In the example shown, the closing device 20 comprises a supporting element 22, a thrust element 24 movable relative to the supporting element 22, and a clamping unit 30 operating on the element 24 to move it in a sliding direction.

[0040] The element 24 comprises a cup-shaped body defining, in a manner known per se, an infusion chamber adapted to receive the dosage unit 8 .

[0041] The circulation system 26 is also of a basically known type and comprises a water tank 40, a boiler 42 and a hydraulic circuit 44 including a circulation pump 46 between the boiler 42 and the inlet duct of the support element 22. A duct 48 connects the outlet manifold of the movable thrust element 24 with a beverage dispensing nozzle 50.

[0042] The machine 1 also comprises a pre-chamber 60 for the dosage units 8 , in transition between the introduction opening 10 and the operating area 4 .

[0043] The machine 1 is intended to be used with dosage units which are preferably provided with one or more predetermined graphic identification marks which identify the category to which each dosage unit belongs, in particular the type of ingredients or substances contained and the type of beverages, or product lines, they are allowed to manufacture. These identification marks are of optically detectable type and are visible on the reference surface of the dosage unit, for example they comprise one or more images with the product denomination and indicator marks.

[0044] Figure 2a to Figure 2c By way of example, some possible solutions are shown according to which such a dosage unit 8 in tablet form can be provided on its outer surface with a graphic identification mark comprising (at least) one inscription and a plurality of indicator marks associated therewith.

[0045] Specifically, Figure 2a , Figure 2b and Figure 2cThe dosage unit 8 is shown with an inscription 70 (e.g. a cartouche identifying the product line) and zero, one or two indicator marks 72 respectively associated therewith, in this example formed by simple symbols (e.g.: circle, square) attached to the corners of the cartouche, identifying the mixture of precursor materials. In the case of coffee powder, the mixture of precursor materials has a brown color of variable intensity depending on the composition of the mixture or its storage conditions, while the inscriptions and indicator marks produced by printing or other application of edible additional substances, thermal marking or by applying a localized laser beam or embossing / engraving have a contrasting color, such as dark brown or black. Naturally, a similar identification effect can be obtained by making the graphic mark of a contrasting color lighter than the background color of the mixture of precursor materials or in any case by making the luminescence intensity or brightness of the graphic mark different from the luminescence intensity or brightness of the precursor substance in response to the irradiation of the dosage unit. In some embodiments, the visibility of the graphic mark may appear in an irradiation wavelength or an observation (image acquisition) wavelength, the spectrum of which is different from the spectrum of wavelengths visible to the human eye, by contrast with the background of the precursor material.

[0046] This type of dosage unit 8 can advantageously be used Figure 1 In a machine of the type shown, a detector device (reader) 62 is positioned along the path of the dosage unit between the introduction opening 10 and the operating area 4 , for example on one side of the pre-chamber 60 .

[0047] The machine 1 comprises an electronic processing and control unit 64 to which a code detector device 62 is connected and which is configured to classify dosage units and implement an operating cycle of the machine for managing dosage units, which operating cycle is variable based on a predetermined pattern according to the category of the dosage unit 8 introduced into the machine, the image of which is captured by the detector device 62; and / or the electronic processing and control unit is configured to capture data indicating the user's usage pattern of the machine 1, such as preferred beverage type, etc.

[0048] The processing and control unit 64 may for example be configured to vary depending on the type of dosage unit and thus the type of beverage, the duration of infusion, and / or the amount and / or temperature and / or pressure of water and / or steam passing through the dosage unit.

[0049] pass Figure 3 The block diagram shows in more detail the Figure 1 The control system of the machine.

[0050] The dosage unit 8 is shown (in the pre-chamber 60) between a pair of devices 80 and 82 adapted to detect the presence of the dosage unit, for example by interrupting an infrared signal transmitted by the transmitter device 80 to the receiver device 82. An image recognition and classification microcontroller unit 84 in communication with the devices 80 and 82 is responsible for detecting the introduction of the dosage unit into the pre-chamber and is coupled to one or more illumination modules 86 adapted to at least temporarily illuminate the pre-chamber to allow acquisition of at least one image of at least one reference surface of the dosage unit. Conveniently, the illumination module 86 may include means for adjusting the intensity and / or color (or more generally, wavelength) of a particular illumination beam.

[0051] Typically, the illumination beam may be in one or more selected bands of the visible spectrum or another spectrum (eg, infrared or ultraviolet), with a predetermined illumination angle or a variable illumination angle.

[0052] Reference numeral 88 denotes an image acquisition module, for example comprising an RGB camera or a CCD sensor with LED ring illumination, for acquiring the aforementioned images of the dosage unit under the control of the microcontroller unit 84; the image acquisition module is configured to trigger the taking of one or more images of the dosage unit, as well as the recognition and classification of these images, during the transfer of the dosage unit inserted into the machine towards the infusion chamber, by means of the control of the illumination module. The microcontroller unit 84 is coupled to the image acquisition module 88 via an adapter module 90 suitable for transmitting image data to the microcontroller unit 84. The microcontroller unit 84 is programmed to classify the detected images according to a procedure to be described below, which is governed by a classifier model, such as a neural network, stored in the form of a computer program in an associated permanent memory module, which memory module is optionally rewritable due to the connection of the microcontroller unit to an update module, for example a wired connection to a local update module or a wireless connection to a remote update module, for example via a local or wide area communication network, in order to update the classifier model to new categories of dosage units or to update them at different times of the operating life of the machine.

[0053] The image acquisition module 88 may operate in one or more selected bands of the visible spectrum or another spectrum (eg, infrared or ultraviolet) at at least one predetermined acquisition angle, which is not necessarily frontal, and is, for example, also variable.

[0054] The microcontroller unit 84 is configured to communicate with separate control devices of the machine (not shown), which are configured to control at least one parameter or operating management mode of the dosage unit, such as: an infusion parameter or operating mode of the machine or a parameter or operating mode related to other steps; for example, these control devices are used to control the compression of the dosage unit during the pre-infusion step and / or the infusion step according to a constant curve or according to a curve defined by a predetermined control curve; control the squeezing of the dosage unit after infusion to remove residual water, control the automatic removal of the dosage unit from the infusion chamber to a collection compartment for spent units after infusion according to the identified dosage unit category and based on relevant data transmitted by the adapter module 90; or these control devices are used to communicate with a remote management system configured to record machine usage data, for example for counting the number of dispensing actions for statistical purposes and predicting possible wear of the image acquisition device.

[0055] Advantageously, the results of the classification process of dosage units gradually inserted into the machine during its operating life can be transmitted over time to external management systems, which are temporarily associated via a local wired connection or a remote wireless connection (for example via a local or wide area communication network), and the microcontroller unit can receive programming instructions from the external management system, for example for updating the characteristic parameters of the classification model.

[0056] In a currently preferred embodiment, there are three images acquired of the dosage unit inserted into the machine and the microcontroller unit 84 selects the best image by a selection algorithm, for example a "voting" algorithm based on a voting strategy, such as majority voting Naive Bayes weighted majority voting.

[0057] In a pre-chamber connected to the immersion chamber, there may be steam coming from the immersion chamber, which steam is represented in the image by a pictogram marked V, and the pre-chamber can therefore preferably be associated with a fan device 92, which is used to clean the detector device 62 integrated in the image acquisition and recognition module 88, for example, for sucking steam or particles of suspended form of precursor material produced by a previous immersion operation and subsequent cleaning of the optical device of the image acquisition device, and the fan device 92 is controlled by a corresponding electronic control module 94.

[0058] Finally, reference numeral 96 denotes a module for managing the power supply to the system components.

[0059] Figure 4A currently preferred embodiment of a microcontroller unit 84 is shown, which operates as an electronic system of the machine learning type for processing and automatically identifying dosage units; the microcontroller unit is configured (i.e. programmed) to receive at least one acquired image I of a dosage unit as input, and based on a set of training images of dosage units or the category of dosage units taken in a learning step, classify the dosage unit into one of a plurality of predetermined categories (or types) of dosage units.

[0060] In a currently preferred embodiment, the microcontroller unit 84 comprises a convolutional neural network configured to receive as input a set of values ​​of a predetermined pixel matrix of a captured image I of the dosage unit, these values ​​representing the grayscale level or intensity of a color channel (for example RGB channels) and more generally indicating the luminous intensity or brightness, that is, the amount of light reflected or emitted (for example by the phosphorescence or fluorescence of an edible additional substance making a graphic mark) at the captured wavelength by the illuminated surface of the dosage unit; the neural network comprises an input layer, a plurality of cascaded hidden layers and an output layer adapted to provide a classification result of the dosage unit, classifying it into one of a plurality of manufacturer categories and into one of a plurality of product categories. More specifically, a first classification is intended to identify the production line of the dosage unit inserted into the machine based on the marking 70, for example by distinguishing between a graphic marking associated with a dispensable production line, a similar marking or a production associated with a non-dispensable production line (depending on the type of dispensing machine model), and a second classification is intended to identify a mixture of precursor materials based on the indicator marking 72.

[0061] In detail, the neural network comprises a first subset FR of hidden feature extraction layers and a second subset CL of hidden classification layers. The subset FE of hidden feature extraction layers having the purpose of extracting salient features from an image that can be used for classification purposes comprises a plurality of hidden layers; each hidden layer comprises a series of convolutional blocks of a residual configuration, the series of convolutional blocks being configured to extract a series of feature maps from an acquired image of a dosage unit by repeatedly applying a plurality of predetermined filters, the values ​​of which are defined in a learning step of the neural network, which will be discussed later in this discussion, while reducing the data size of the image representing the dosage unit. The subset CL of hidden classification layers comprises two parallel processing branches CL1, CL2, each having a plurality of fully connected layers, which output corresponding vectors V1, V2 indicating respective classification results, which are then concatenated into a single output vector VC.

[0062] More specifically, refer to Figure 5 For example, the subset of feature extraction layers FE includes five layers, as shown in Figure 5As represented in the enlarged illustration of , each layer is configured to apply a pair of convolution filters to the input matrix and the ReLU activation function according to the residual configuration, and to apply a pooling operation so that the size of the input matrix at the output is halved. As an alternative to the pooling operation, the convolution filter can be applied with a stride other than 1. Advantageously, this configuration of the feature extraction layer allows to mitigate the problem of gradient vanishing in the supervised training of the network through error back propagation. In addition, according to the depthwise separable convolution method, the application of the convolution filter is performed to accelerate the training convergence.

[0063] The size of the layer can vary depending on the processing resources available on the machine.

[0064] The first layer is adapted to receive an input matrix representing pixel values ​​of a predefined region of the original image, in the exemplary case a central region of size 256×256 pixels, which is convolved with 16 filters of size 3×3 and stride 1 and a ReLU activation function applied in a residual configuration, and is followed by a maximum pooling operation with stride 2, thereby outputting a plurality of 16 feature maps of size 128×128.

[0065] The second layer is adapted to submit the 16 feature maps of size 128×128 pixels calculated at the first layer to convolution with 32 filters of size 3×3 and stride 1, apply a ReLU activation function in the residual configuration, and subsequently submit to a maximum pooling operation with a stride of 2, thereby outputting a plurality of 32 feature maps of size 64×64.

[0066] The third layer is adapted to submit the 32 feature maps of size 64×64 pixels calculated at the second layer to convolution with 32 filters of size 3×3 and stride 1, apply a ReLU activation function in a residual configuration, and subsequently submit to a maximum pooling operation with a stride of 2, thereby outputting a plurality of 32 feature maps of size 32×32.

[0067] The fourth layer is adapted to submit the 32 feature maps of size 32×32 pixels calculated at the third layer to convolution with 64 filters of size 3×3 and stride 1, apply a ReLU activation function in a residual configuration, and subsequently submit to a maximum pooling operation with a stride of 2, thereby outputting a plurality of 64 feature maps of size 16×16.

[0068] Finally, the fifth layer is adapted to submit the 64 feature maps of size 16×16 pixels calculated at the fourth layer to convolution with 64 filters of size 3×3 and stride 1, apply a ReLU activation function in the residual configuration, and subsequently submit to a maximum pooling operation with a stride of 2, thereby outputting a plurality of 64 feature maps of size 8×8.

[0069] Through the linearization operation (flattening), the pixel values ​​of the 64 feature maps of size 8×8 are concatenated into a vector of size 4096, which is provided as input to each branch CL1, CL2 of the subset CL of the classification layer.

[0070] The subset of classification layers includes four layers for each processing branch, each layer being configured to perform a weighted linear combination of input values ​​and apply an associated activation function, preferably a non-linear activation function, such as a ReLU function in the inner layers and a sigmoid or softmax function in the output layer.

[0071] The first layer is adapted to receive an input vector of size 4096 output by a subset of feature extraction layers, performing a non-linear weighted sum to produce an output vector of size 64.

[0072] The second layer is adapted to receive the vector of size 64 computed in the first layer, subjecting it to a non-linear weighted sum, thereby producing an output vector of size 32.

[0073] The third layer is adapted to receive the vector of size 32 computed in the second layer, subjecting it to a non-linear weighted sum, thereby producing an output vector of size 16.

[0074] Finally, the fourth layer is adapted to receive the vector of size 16 calculated in the third layer, subjected to a non-linear weighted sum, thereby producing an output vector of size 3 representing the final result of the classification, preferably normalized using a softmax function, expressed in terms of the probability that the dosage unit belongs to each of the predefined categories.

[0075] The above content is described by way of example only, and it should be understood that different convolutional network configurations (e.g., the number of hidden layers), different filtering and pooling parameters, or different activation functions may be used. In addition, the above content, with reference to an input matrix representing pixel values ​​of a predefined area of ​​the original image, may be applied to a grayscale image in which each pixel has a grayscale value, and may be extended to a color image by appropriately referring to multiple input matrices representing pixel values ​​of a predefined area of ​​the original image in a color channel (e.g., three RGB color channels).

[0076] The above neural network is trained in a supervised manner for determining the values ​​of the convolution filters and the parameters of the activation functions of a subset of the hidden layers starting from a data set comprising a plurality of images representing all the different classifications of dosage units required by the system, acquired by one or more training image acquisition modules similar to the image acquisition modules provided on the machine, each of which has been preliminarily assigned at least one relevant category, in a specific case a category indicating a product line and a category indicating a mixture of precursor materials. Preferably, the preliminarily classified data set is divided into: a first test data set, representing 20% ​​of the data set; a second training data set, representing 80% of the remaining data set; and a third validation data set, representing the remaining 20% ​​of the remaining data set. Validation can be performed, for example, by exchanging validation data with training data in each cycle according to a cross-validation procedure with 5 partitions (k-fold cross-validation, k=5). Advantageously, the data of the test, training and validation data sets can be selected from the data set manually or automatically, taking into account as much as possible the percentage subdivisions mentioned above for each known category.

[0077] Conveniently, the data set may include samples of different colours and different sizes (thicknesses), if any.

[0078] The plurality of dosage unit images constituting the data set, representing all different classifications required by the system, comprises: at least a plurality of primary images of dosage units acquired by one or more training image acquisition modules starting from available samples of dosage units, and a plurality of composite images obtained by modifying the plurality of primary images according to a data augmentation technique. The plurality of composite images comprises the corresponding graphic identification mark modified relative to the predetermined graphic identification mark or relative to a reference surface of the dosage unit with the predetermined graphic identification mark, which is Figure 6 The modification of the graphic identification mark can be obtained by modifying only the graphic identification mark relative to the reference surface of the dosage unit or by modifying the image as a whole. In particular, the modified graphic identification mark has a modified morphology, size, position or optical properties relative to the predetermined graphic identification mark; and more particularly, the modified graphic identification mark includes, for example, at least one predetermined graphic identification mark being translated, rotated, reduced or enlarged, distorted, incomplete, blurred, changed in light intensity or brightness, changed in color or changed in contrast.

[0079] The use of synthetic images with altered graphical identification marks advantageously allows for more extensive training of the neural network and provides greater generalization to the neural network even when images with altered characteristics are acquired relative to primary images acquired starting from an available sample of dosage units. Specifically:

[0080] - the generation of a synthetic image with a vertical or horizontal translation of the graphic identification mark (for example, 0.05% relative to the primary reference image) allows strengthening the neural network so that the classification is invariant with respect to the position of the dosage unit in the pre-chamber 60;

[0081] - the generation of a synthetic image with a rotated graphic identification mark (e.g. at a variable angle between -180° and +180° relative to the primary reference image) allows strengthening the neural network so that the classification is invariant with respect to the rotation of the dosage unit in the pre-chamber 60;

[0082] - the generation of a synthetic image with a reduced or enlarged scale of the graphic identification mark (for example, a variable scale between -0.10% and +0.10% relative to the primary reference image) allows strengthening the neural network so that the classification is invariant with respect to the axial position of the dosage unit in the prechamber 60;

[0083] - the generation of a synthetic image with distorted graphic identification marks (e.g. variable spacing between -0.10 px and +0.10 px relative to the primary reference image) allows strengthening the neural network so that the classification is invariant with respect to the acquisition speed of the images of the dosage units in the pre-chamber 60 (or at the falling rate of the dosage units);

[0084] - the generation of synthetic images with incomplete graphic identification marks (e.g. due to the absence of certain parts relative to the primary reference image within a predetermined incompleteness limit threshold) allows strengthening the neural network, making the classification invariant with respect to the integrity of the graphic mark and therefore more robust in the event of wear and tear on the reference surface of the dosage unit carrying the graphic mark, leading to its partial elimination;

[0085] - the generation of a synthetic image with blurred graphic identification marks relative to the primary reference image allows strengthening the neural network so that the classification is invariant with respect to the focus of the dosage unit in the pre-chamber 60;

[0086] - the generation of synthetic images with a graphic identification mark of varying light intensity or brightness, respectively brighter or darker (e.g. in a brightness range between 0.35% and 1.25% relative to the primary reference image) allows strengthening the neural network so that the classification is invariant with respect to the color and reflectivity of the dosage units in the pre-chamber 60, which may depend on the storage conditions of the tablets, on their state of degradation (e.g. due to exposure to environmental conditions of non-optimal temperature or humidity) or even on the state of degradation and cleanliness of the lighting means of the pre-chamber and of the image acquisition optics;

[0087] - the generation of a synthetic image with a graphical identification mark of varying color (e.g. a graphical identification mark of lighter color than the background, relative to a primary “double” reference image in which the graphical identification mark is darker in color than the background) allows strengthening the neural network so that the classification is invariant with respect to the color and reflectivity of the dosage units in the prechamber 60, which may depend on the manufacturing technology of the graphical identification mark;

[0088] - The generation of a synthetic image with a graphic identification mark of varying contrast, which is lighter or darker, respectively, than a predetermined reference contrast between the graphic identification mark of the dosage unit and the reference surface in the primary reference image, allows strengthening the neural network so that the classification is invariant with respect to the color and reflectivity of the graphic identification mark of the dosage unit, which may depend on the storage conditions of the tablets, their state of degradation or the manufacturing technology of the reference graphic identification mark.

[0089] If the sample of dosage units of the data set does not have a sufficiently large population representing the predetermined class, it is also possible to prepare a plurality of further synthetic images obtained by manipulating the graphic identification marks of a plurality of primary images or by manipulating the primary images themselves, with the aim of generating images that do not belong to the same class as the original images, for example by varying the number of indicator marks 72 associated with the inscription 70, e.g. Figure 7 This can be achieved, for example, by applying pattern recognition techniques to the images of the dosage units in order to discern the inscription 70 and locate the associated indicator marks 72 (rectangular and circular measurement areas shown as superimposed on the left column of images), and by applying image overlay techniques in order to completely or partially remove the local indicator marks 72 (center and right columns).

[0090] Advantageously, it is possible to assume that supervised training also comprises: acquiring images of dosage units without identification marks (having similar features to the dosage units to be identified, i.e. the background reference surface of the dosage units is consistent in color and texture; or having different features, i.e. the background reference surface of the dosage units is inconsistent), or images when no dosage units are present.

[0091] Based on a data set including primary images and synthetic images, supervised training of a neural network is performed, for example by applying a well-known error back-propagation algorithm, such as the Adam optimization algorithm, feeding the training data set in groups of 32 images (batch size) and updating the network parameters for 300 cycles (epochs), where the learning rate is 0.01, the decay factor is 0.5, the patience value is 10 epochs, the minimum learning rate is equal to 0.0000002, and the label smoothing is equal to 0.2.

[0092] Supervised training of a neural network may employ any known algorithm for minimizing an error function, such as a categorical cross entropy error function.

[0093] Preferably, a subset of the feature extraction layer can be provided with at least one embedding layer to optimize classification by switching to a supervised mode, in which, in addition to the above, a "triplet loss" technique is used to supervise the space of features output from the feature extraction subset so as to correct the distances between reference samples in the feature reference space, samples to be identified as "positive" samples (i.e., samples of the same category), and samples to be excluded as "negative" samples (i.e., samples of different categories), and to separate samples that are very similar to the samples to be identified but may cause confusion in unsupervised training.

[0094] Figure 8a A flow chart showing the operation of an initialization process of a machine for preparing and dispensing hot drinks by infusion.

[0095] In particular, in step 100, a primary image of the dosage unit inserted into the machine is acquired, which is intended to collect training data; and in step 120, the acquired image is subjected to a preprocessing adapted to improve the contrast of the image without distorting the pixel values. During the preprocessing, in step 122, the acquired image is subjected to, for example, a histogram analysis and an Otsu thresholding, which thresholding is particularly suitable for bimodal images, i.e. for images with a histogram having a clear separation between two main peaks, for example with reference to the acquisition of an image in the visible wavelength range, which thresholding occurs in the case of brown dosage units; wherein there may be contrasting graphic identification marks, i.e. darker or lighter than the background of the mixture of precursor materials, for example due to the influence of black or white graphic traces on the brown background or due to the influence of concave or convex surface areas, or engravings or reliefs, which are dark or shiny, respectively, relative to the surrounding surface of the dosage unit. Subsequently, an upper limit (cropping) of the pixel values ​​of the image is performed in step 124, followed by equalization of the image histogram with background suppression in step 126, and finally a zero-point flattening operation is performed in step 128 to minimize depth effects due to image lighting and acquisition (except where the graphic markings include relief and / or engraving).

[0096] By separating the application of normalization or equalization in the three channels, the above can be applied indifferently to monochrome images in gray tones or to color images.

[0097] In a next step 140, a transformation of the image is performed in a format and size acceptable to the neural network. In fact, in order to ensure a high performance of the system in classifying the dosage units inserted into the machine, it is important to provide the neural network for feature extraction and classification with images having features of the same size as the images used in the neural network training step. To this end, scaling of the scanned image is preferably performed. The acquired image is, for example, a 640×480 pixel image and, taking into account the balance of opposing computational requirements (i.e. memory saving, computational speed and recognition efficiency needs), is reduced to a 256×256 pixel image by the following operations: a central area of ​​the original image of size 480×480 pixels is selected by a first cropping operation, and the cropped image is transformed into a useful image of size 256×256 by a second bilinear interpolation operation, or in any case to a predetermined size based on the number of pixel values ​​to be processed in the neural network.

[0098] In step 160, data augmentation techniques are applied to obtain a composite image from the primary image; and in step 180, an image normalization or standardization process is applied to the primary image and the composite image, including subtracting its mean from the image and then dividing by its standard deviation. In an alternative embodiment, after step 160, the image preprocessing described in step 120 may also be performed on the composite image.

[0099] Then, in step 200 , it is verified whether the collection of the training data set is completed, and if completed, the training of the neural network described previously is performed, which is generally referred to as step 220 herein.

[0100] After the network is trained, in step 240, quantization of the neural network model is performed by approximating floating point values ​​to integer values ​​(e.g., 8-bit integer values) based on a predetermined zero value and a predetermined scaling factor according to the following formula:

[0101] Real value = (integer value - zero point) * scaling factor

[0102] The quantization operation allows obtaining a more compact representation of the neural network model and allows performing neural network calculations with less memory footprint than calculations with floating point precision, thereby reducing the space occupied by the neural network and the classification time.

[0103] Figure 8b A flow chart showing the operation of a control or management process for a machine for preparing and dispensing hot drinks by infusion.

[0104] Specifically, in step 500, an image of a dosage unit inserted in the machine is acquired; and in step 520, the acquired image is preprocessed, the preprocessing being adapted to improve the contrast of the image without distorting the pixel values. During the preprocessing, the acquired image is subjected to the processing applied to the training image of the neural network, specifically, histogram analysis and Otsu thresholding in step 522, upper limit (cropping) of the pixel values ​​of the image in step 524, equalization of the image histogram with background suppression in step 526, and finally a zero-point flattening operation in step 528.

[0105] In the next step 540, a transformation of the image is performed in a format and size acceptable to the neural network, for example, as described above, the acquired 640×480 pixel image is resized to a 256×256 pixel image by the following operations: a central area of ​​480×480 pixel size is selected by a first cropping operation, and the cropped image is transformed into a useful image of 256×256 size by a second bilinear interpolation operation.

[0106] In step 580, a normalization or standardization procedure of the image is applied, including subtracting its mean from the image and then dividing by its standard deviation.

[0107] Then, in step 640, quantization of the image is performed by approximating a floating point value obtained by normalizing or equalizing the image to an integer value (e.g., an 8-bit integer value) based on a predetermined zero value and a predetermined scaling factor according to the following formula:

[0108] Real value = (integer value - zero point) * scaling factor

[0109] Then, in step 660, the image is classified by means of a neural network and at the end of the classification an inverse operation of the quantization is performed by applying the above formula to obtain the correct distribution of the probabilities of the different classes.

[0110] Advantageously, the aforementioned neural network configuration and the processing of its input data allow performing image recognition and classification processes with reduced memory resources (approximately 1-2 MB), such as are commonly available in machines for preparing and dispensing hot drinks for domestic use.

[0111] Finally, in step 700, based on the knowledge of the type of dosage unit introduced into the machine, at least one of the following operations is performed:

[0112] - controlling at least one parameter or operating mode of the dosage unit, for example: a parameter or operating mode of the infusion of the machine or a parameter or operating mode related to other steps; for example for controlling the compression of the dosage unit during the pre-infusion step and / or the infusion step according to a constant curve or according to a curve defined by a predetermined control curve; controlling the squeezing of the dosage unit after infusion to remove residual water, controlling the automatic removal of the dosage unit from the infusion chamber to a collection compartment for used units after infusion;

[0113] - recording and / or transmitting machine usage data indicative of the category of the identified dosage unit for statistical monitoring purposes, for example for the benefit of a remote telemetry system.

[0114] It should be noted that the embodiments of the invention presented in the foregoing discussion are provided by way of example only and are not intended to limit the invention. A person skilled in the art may easily implement the invention in different embodiments without departing from the general principles outlined herein, and these embodiments are therefore included within the scope of protection of the invention as defined by the appended claims. This applies in particular to the possibility of acquiring moving images, i.e., for example, a sequence of images (films) of dosage units in different spatial positions along a transfer path leading to an infusion chamber, which represents a dimensional extension of the above-described situation; whereby the image recognition unit comprises a neural network which, relative to the aforementioned neural network, is adapted to receive a sequence of images as input and to classify the dosage units based on the sequence of images. In an embodiment, the neural network is configured to act solely on a single image, applying a "voting" procedure to select the most likely classification. In an alternative embodiment, the neural network is configured to act on a captured video file of at least a portion of the transfer movement of the dosage unit, the captured video file comprising, for example, frames in which the dosage unit was captured in its entirety by a wide-angle camera; whereby the neural network comprises a matrix stack equal to the number of video frames in the case of an uncompressed file, or the neural network is adapted to classify a compressed file comprising one or more reference frames (completely describing the image) and a predicted frame comprising corresponding partial information representing only changes relative to the corresponding reference image.

[0115] Of course, without prejudice to the principle of the invention and without departing from the scope of protection of the invention as defined by the appended claims, the embodiments and implementation details may vary widely with respect to those described and illustrated purely by way of non-limiting example.

Claims

1. A control system for a machine (1) for preparing and dispensing hot beverages by controlled infusion of a precursor substance in granular or powdered form arranged in a dosage unit (8), the control system comprising: - Image acquisition device (62; 88), adapted to acquire at least one image (I) of the dosage unit (8) or a selected portion of the dosage unit inserted into the machine (1); - an electronic device (84) of machine learning type for processing and automatically identifying the dosage unit (8), the electronic device being configured to receive as input at least one acquired image (I) of the dosage unit (8) and to classify the dosage unit (8) into one of a plurality of predetermined categories of dosage units based on a set of training images of the dosage unit taken in a learning step; and - a machine control device (64) coupled to the electronic device (84) for processing and automatic identification, the machine control device being configured to control at least one parameter or operating mode for managing the dosage units (8) as a function of the type of the dosage units identified, or the machine control device being configured to record a data item of the use of the machine (1) and transmit the data item to a remote management system, the data item indicating the type of the dosage units (8) identified, Characterized in that the training image set of the dosage unit includes at least a plurality of primary images of the dosage unit (8) and a plurality of synthetic images obtained by changing the plurality of primary images, wherein the primary images include corresponding predetermined graphic identification marks (70, 72) on the reference surface of the dosage unit (8) indicating the corresponding category of the dosage unit.

2. The control system according to claim 1, wherein: The plurality of composite images comprises respective graphic identification marks which vary relative to the predetermined graphic identification marks (70, 72) or relative to the reference surface of the dosage unit (8).

3. The control system according to claim 2, wherein: The modified graphic identification mark exhibits a modified form, size, position or optical property relative to the predetermined graphic identification mark (70, 72).

4. The control system according to claim 3, wherein: The changed graphic identification mark includes at least one of the predetermined graphic identification marks being translated, rotated, reduced or enlarged, distorted, incomplete, blurred, with changed light intensity or brightness, changed color or changed contrast.

5. A control system according to any one of the preceding claims, wherein: The electronic device (84) of machine learning type for processing and automatically identifying the dosage unit (8) comprises a convolutional neural network, which is configured to receive a set of values ​​of a predetermined pixel matrix of the acquired image (I) as input, and the convolutional neural network comprises an input layer, a plurality of cascaded hidden layers and an output layer, wherein the plurality of cascaded hidden layers comprises a first subset (FE) of feature extraction layers and a second subset (CL) of classification layers.

6. The control system according to claim 5, wherein: The subset (FE) of feature extraction layers comprises a plurality of layers, each of the plurality of layers being configured to extract a plurality of feature maps from the image (I) of the dosage unit (8) by repeatedly applying a predetermined filter, the value of the predetermined filter being defined in a learning step of the neural network, and each of the plurality of layers being configured to reduce the data size of the feature map representing the pixel matrix of the acquired image (I).

7. The control system according to claim 6, wherein: The plurality of layers of the subset of hidden feature extraction layers (FE) comprises five layers, each of the five layers comprising a pair of convolutional filters intended to be applied to an input pixel matrix or an input feature map, and a corresponding ReLU activation function according to a residual configuration at an output, the convolutional filters or subsequent pooling stage being configured to halve the size of the input feature map.

8. The control system according to claim 5, wherein: The subset of classification layers (CL) comprises two parallel processing branches (CL1, CL2), each branch having a plurality of fully connected layers, each fully connected layer being adapted to receive as input a data array comprising pixel values ​​of a plurality of feature maps output from the subset of feature extraction layers (FE).

9. The control system according to claim 8, wherein: The plurality of fully connected layers of the subset of classification layers (CL) comprises four layers, each layer being configured to perform a weighted linear combination of input values ​​and to apply a relevant activation function, preferably a non-linear activation function.

10. A control system according to any one of claims 5 to 9, wherein: The electronic means (84) for processing and automatic identification are configured to communicate with a remote management system temporarily connected to the control system via a local or remote connection, which may be a wired or wireless connection, to transmit classification results or receive programming instructions for the parameters of the neural network.

11. A control system according to any one of the preceding claims, comprising: A trigger device (80, 82), the trigger device being coupled to the image acquisition device (62; 88), the trigger device is adapted to detect the insertion or passage of the dosage unit (8) into the associated identification seat (60), and the trigger device is configured to trigger the acquisition of at least one image (I) of the dosage unit (8) or a selected portion of the dosage unit located in the identification seat (60).

12. The control system according to claim 11, further comprising: An illumination device (86) is adapted to direct an illumination beam toward the identification seat (60).

13. The control system according to claim 12, wherein: The illumination light beam is emitted in one or more selected wavelength bands, including at least one wavelength band in the visible spectrum, infrared spectrum, or ultraviolet spectrum.

14. The control system according to claim 13, comprising: Means for adjusting the intensity and / or wavelength of the illumination beam associated with the illumination means (86).

15. The control system according to claim 12, wherein: The illumination device (86) is configured to operate at at least one predetermined variable collection angle.

16. A control system according to any one of the preceding claims, wherein: The electronic device (84) for processing and automatically identifying the dosage unit (8) is configured to implement a process of correcting the acquired image (I) by applying a predetermined correction model for correcting the acquired image according to instructions received from the remote management system, and the remote management system is temporarily connected to the control system via a local or remote connection, which is a wired connection or a wireless connection.

17. A control system according to any one of the preceding claims, wherein: The electronic device (84) for processing and automatically identifying the dosage unit (8) is configured to identify the predetermined graphic identification mark (70, 72) in the at least one acquired image (I), and the electronic device (84) is configured to generate a comparison index representing the similarity between the at least one acquired image and a reference image.

18. The control system according to claim 17, wherein: The similarity is defined by a dynamically modifiable threshold.

19. A control system according to any one of the preceding claims, wherein: The parameters or the operating mode for managing the dosage unit (8) comprise at least one infusion parameter or operating mode, comprising at least one of the following: temperature, pressure and amount of infusion liquid.

20. A control system according to any preceding claim, wherein: The parameters or operating modes for managing the dosage units (8) include at least one parameter or operating mode for controlling the compression of the dosage units (8) during a pre-soaking step and / or a soaking step, squeezing the dosage units (8) after soaking to remove residual water, or automatically removing the dosage units (8) from the soaking chamber (4) to a collection compartment for used units after soaking.

21. A control system according to any one of the preceding claims, wherein: The graphic identification mark (70, 72) includes at least one of a surface graphic mark and a raised or recessed graphic mark, the surface graphic mark exhibiting a color different from the color of the precursor material of the dosage unit.

22. A control system according to any preceding claim, wherein: The image acquisition device (62; 88) is configured to operate in one or more selected wavelength bands, the one or more selected wavelength bands including at least one wavelength band in the visible spectrum, infrared spectrum or ultraviolet spectrum.

23. A control system according to any one of the preceding claims, wherein: The image acquisition device (62; 88) is configured to operate at at least one predetermined variable acquisition angle.

24. A control system according to any one of the preceding claims, wherein: The image acquisition device (62; 88) is configured to acquire a sequence of images (I) of the dosage unit (8) at different spatial positions along a transfer path of the dosage unit (8) to the infusion chamber (4).

25. A control method for a machine (1) for preparing and dispensing hot beverages by controlled infusion of a precursor substance in granular or powdered form arranged in a dosage unit (8), wherein: The machine (1) comprises: - electronic means (84) of the machine learning type for processing and automatically identifying the dosage unit (8), the electronic means being configured to receive as input at least one image (I) of the dosage unit (8) or a selected portion of the dosage unit and to classify the dosage unit (8) into one of a plurality of predetermined categories of dosage units; and - a control device (64) for managing the dosage unit (8), the control device being configured to control at least one parameter or operating mode for managing the dosage unit (8), The method is characterized in that the method comprises the following steps: - in a learning step, the electronic device (84) is configured to process and automatically recognize by means of a set of training images of dosage units, the set of training images comprising at least a plurality of primary images of the dosage units and a plurality of composite images obtained by modifying the plurality of primary images, the primary images comprising respective predetermined graphic identification marks (70, 72) on a reference surface of the dosage units (8) and representing the corresponding category of the dosage units; and -In the operation steps, acquiring at least one image (I) of the dosage unit (8) of the precursor substance inserted into the machine (1) or of a selected portion of the dosage unit; providing at least one acquired image (I) of the dosage unit (8) as input to the electronic device (84) for processing and automatic recognition; Classifying said dosage unit (8) into one of a plurality of predetermined categories of dosage units by said electronic means (84) for processing and automatic identification; and At least one of the following steps: - controlling at least one parameter or operating mode for managing the dosage unit (8) according to the category of the dosage unit; and - recording a usage data item of the machine (1) indicating the category of the dosage unit (8) and / or transmitting the data item to a remote management system for statistical monitoring purposes.

26. A computer program or a group of programs executable by a processing system, comprising one or more code modules for implementing a control method for a machine (1) according to claim 25 for preparing and dispensing hot drinks by controlled infusion of a precursor substance in granular or powdered form.

Images