A control system and method for a machine that prepares and dispenses hot beverages by controlling the injection of a granular or powdered precursor substance based on the recognition and classification of a dose unit inserted into the machine

Abstract

A control system and method for a machine (1) for preparing and dispensing hot beverages by means of a controlled injection of a granular or powdered precursor substance arranged in a dosing unit (8) are described. A machine learning type automatic image recognition and processing unit (84) is configured, in a learning step, to classify a dosing unit (8) received by the machine into one of a plurality of predetermined classifications of the dosing unit based on a set of training images of the dosing unit. The set of training images of the dosing unit comprises at least a main image of a plurality of dosing units (8) comprising predetermined graphic recognition markings (70, 72) on a reference plane of the dosing unit (8), and a plurality of composite images obtained by changing the plurality of main images.

Description

Technical Field

[0001] The present invention relates to a machine for preparing and dispensing a warm beverage by a controlled injection of a dose unit of a precursor substance containing one or more components, such as coffee for example.

[0002] More specifically, the present invention relates to a control system and method for a machine for preparing and dispensing a warm beverage by a controlled injection of a granular or powdered precursor substance arranged in a dose unit, according to the preambles of each of claims 1 and 25.

[0003] It is known that a machine for preparing and dispensing a warm beverage by a controlled injection of a dose unit comprises means for automatically identifying a dose unit inserted into the machine in order to classify the dose unit into one of a plurality of predetermined types or categories and, as a result, control at least one parameter or execution mode of the insertion of the machine or record and / or transmit usage data of the machine to a remote management system according to the recognized type of dose unit.

[0004] The automatic identification may be carried out using machine learning techniques, for example, through a pattern recognition method or an artificial neural network based on a predefined classification scheme.

Background Art

[0005] Utility model IT274555 of the same applicant describes a machine for preparing beverages, particularly coffee, using pods or capsules selected from a plurality of types of pods or capsules having different characteristics, particularly characteristics of the same dimensions but containing different substances or components. The machine is equipped with a detection device adapted to detect at least one predetermined detectable feature or code for identifying the type of pod or capsule inserted into the machine, and a processing and control unit to which the detection device is connected. The processing and control unit is configured to execute an operating cycle of the machine that varies according to a predetermined method corresponding to the type of pod inserted into the machine and detected by the detection device, and / or to acquire data indicating how the user uses the machine, such as the total quantity and / or individual quantities of the various types of pods used during a specific period, and / or the usability of "non-original" compatible pods.

[0006] The detectable feature or code is, for example, a color or barcode, or a so-called RFID tag.

[0007] As disclosed in international patent application WO2019 / 154527, the automatic identification of the dosing unit may be performed using machine learning techniques, for example, through a pattern recognition method or an artificial neural network based on a predefined classification scheme.

[0008] WO2019 / 154527 discloses a machine for preparing and dispensing beverages, such as tea, coffee, warm cocoa, cold cocoa, milk, soup, or baby food. The machine is equipped with a module for recognizing a capsule inserted into the machine at a recognition position. The module includes a camera for taking at least a partial image of the capsule, and a neural network computing device configured to determine the capsule type from among a plurality of predefined capsule types based on at least a partial image of the capsule taken by the camera.

[0009] Machines for preparing and dispensing beverages of the type described above can ensure automatic recognition of pods or capsules, but tablets, due to storage conditions or exposure to inappropriate environmental conditions of temperature or humidity, or handling by the user, may deteriorate over time, for example, coffee tablets or other precursors in granular or powder form, and are not suitable for accurately recognizing and classifying objects whose characteristics may change over time.

[0010] Furthermore, even in the case of a dosing unit packaged in a package with a non-perishable outer surface like a pod or capsule, if the imaging optics located near the injection chamber are in a clean state, or if the lighting means of the dosing unit and the means for acquiring images, especially if they are low-cost devices that wear out during the life of the machine, artifacts may be caused in the acquisition of images of the dosing unit inserted into the machine where the automatic recognition process is carried out, leading to misclassification.

[0011] A further factor affecting classification accuracy lies in the randomness of the spatial position of the dosing unit inserted into the machine during image acquisition for recognition, especially when this image is acquired in the transfer path of the dosing unit to the injection chamber, for example, as a result of dropping, or when acquired by the movement of the housing compartment of the dosing unit that does not remain stationary.

[0012] The object of the present invention is to provide a satisfactory solution to the above problems and avoid the drawbacks of the prior art.

[0013] More specifically, the object of the present invention is to provide a control system for a machine for preparing and dispensing beverages based on accurate and robust classification of dosing units of beverage precursors supplied to the machine, and the classification is, as much as possible, independent of the conditions of the dosing unit and the machine throughout the entire operating life of the machine.

[0014] A further object of the invention is to enable the classification of the dosing unit to be adapted when the operating conditions of the machine change or when the type of dosing unit that may be generated over time changes.

[0015] Another object of the invention is to provide a control system that can be manufactured and integrated into a machine for preparing and dispensing beverages in a cost-effective manner, for example, for domestic machines where size and cost are restricted.

[0016] According to the invention, these objects can be achieved by a control system for a machine for preparing and dispensing hot beverages, having the features described in claim 1.

[0017] Specific embodiments form the subject matter of the dependent claims, and the content of the subject matter should be understood as an essential part of this description.

[0018] A further subject matter of the invention is a method for controlling a machine for preparing and dispensing hot beverages, having the features described in claim 25.

SUMMARY OF THE INVENTION

[0019] To summarize, the present invention is based on the principle of performing automatic recognition of a dose unit inserted into a machine through machine learning classification technology, preferably through a neural network or a similar defined classification model. The classification is based on a set of training dose unit images indicating a plurality of predetermined types or classifications of the dose unit under a plurality of conditions for acquiring an image of the dose unit. Specifically, the dose unit is provided with each graphic recognition marking indicating a dose unit classification, and the training image set of the dose unit includes a plurality of main images (or original images) of dose unit samples pre-classified in a super-vice manner, and considering possible changes in the actual samples of the dose unit or the means and conditions for acquiring images of the dose unit during the operating life cycle of the machine, and in order to increase the amount of images for training, a plurality of synthetic images obtained by changing a plurality of main images according to data augmentation technology.

[0020] Advantageously, the creation of a plurality of synthetic images enables the simulation of images of dose units different from the sample dose unit, such as degraded images of the dose unit, for example, artifact images of an intact dose unit due to degradation of the image acquisition means or non-optimal alignment of the dose unit with respect to the image acquisition means, or images of dose units with different dimensions in the case of dose units for different beverages such as espresso or American coffee, where the acquired images are susceptible to changes due to optical distortion.

[0021] Advantageously, the graphic recognition marking indicating the type of the dose unit includes an engraving for identifying the product line and additional display markings that enable a first classification based on the product line of the dose unit and a second classification based on the mixture of precursor materials.

[0022] The term "graphic marking" refers to, for example, a (2D) graphic marking on a surface obtained by printing or other application of an edible additive, hot marking or application of a local laser beam, where the marking has a color different from the color of the precursor of the dose unit or a color symmetrical to the color of the precursor, or in any case, a marking showing a light intensity or luminance different from the light intensity or luminance of the precursor, depending on the illumination of the dose unit, or a relief or engraving that forms a marking showing reflection characteristics and / or shadows that change when illuminated, by applying mechanical and / or thermal imprints on a predetermined area of the surface of the precursor adapted to form such a relief or engraving, or a (3D) graphic marking with protrusions or notches obtained by changing the morphological characteristics of the surface of the precursor of the dose unit.

[0023] In one embodiment, the automatic recognition of the dose unit can be carried out through the identification of the graphic recognition marking in the acquired image and the determination of a comparison metric indicating the similarity between the graphic recognition marking (or a part of the graphic recognition marking) of the acquired image and the graphic recognition marking (or a part of the graphic recognition marking) of the training image of the dose unit or the reference image set of the classification of the dose unit. The similarity is defined by one or more comparison thresholds, which are preferably dynamically changeable during the life cycle of the machine.

[0024] Automatic recognition of the dosing unit can also be envisaged as being carried out after a process of modifying the acquired image as a result of a decrease in classification accuracy, or a decrease in similarity in a comparison between the graphic recognition markings of the acquired image and the graphic recognition markings of the training images of the dosing unit or the set of reference images for dosing unit classification, or based on the elapsed operating time of the machine or the number of dispensing operations performed. The image modification process can be carried out by applying a predetermined modification model or by updating the characteristic parameters of the classification model according to instructions received from an external management system temporarily connected to the machine control system via a local wired connection or a remote connection.

[0025] Further features and advantages of the present invention will be described in detail, by way of non-limiting example, with reference to the accompanying drawings, from the detailed description of the embodiments of the invention.

Brief Description of the Drawings

[0026]

Figure 1

Figure 2a

Figure 2b

Figure 2c

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8a

Figure 8b

DETAILED DESCRIPTION OF THE INVENTION

[0027] As shown in FIG. 1, a machine for preparing a beverage, particularly coffee, from a compressed tablet containing one or more ingredients, particularly coffee powder, is shown generally as 1.

[0028] The invention is described with respect to the use of compressed tablets containing ingredients or ingredients for the preparation of beverages, but is not limited to dose units in the form of tablets. Instead of tablets, it is also applicable when provided with a dose of powder contained in a capsule, pod or other similar package suitable for the preparation of beverages by injection. Thus, for the purposes of the present invention, "granular or powdery precursor" is assumed to include both products packaged in hard capsules or flexible pods and compressed products in the form of tablets.

[0029] In the embodiment shown by way of example, the machine 1 comprises a machine body 2 having an operating area 4. The machine 1 has a lever 6 for pushing the dosing unit 8 into the area 4 through an introduction opening 10, and a closing device 20 which operates on the dosing unit 8 and encloses the dosing unit 8 (and, optionally, for piercing the coding film of the pod or the closure lid of the capsule) in an injection chamber delimited by two opposing cooperating elements 22, 24, and a circulation system 26 for circulating a flow of hot water and / or pressurized steam through the dosing unit in the injection chamber.

[0030] The dosing unit 8 contains a dose or portion of granular or powdered material of coffee or other substances, in the form of tablets having a self-supporting structure that does not require an outer case, or packaged in a flexible water-permeable wrapping, usually called a "pod", which is usually a paper wrapping, or in a more or less rigid capsule, intended to be injected to make a hot beverage.

[0031] The lever 6 is rotatable about a fulcrum 28 and is adapted to push the dosing unit 8 through the opening 10 towards the operating area 4.

[0032] By way of example, the closing device 20 includes an abutment element 22, a thrust element 24 movable relative to the abutment element 22, and a clamping unit 30 which operates on the element 24 to move it along the sliding direction.

[0033] The element 24 comprises a cup-shaped body defining an injection chamber adapted to receive the dosing unit 8 in a manner known per se.

[0034] The circulation system 26 is of a substantially known type and comprises a water tank 40, a boiler 42, and a hydraulic circuit 44 including a circulation pump 46 between the boiler 42 and the suction duct of the abutment element 22. A duct 48 connects the outlet manifold of the movable thrust element 24 and the beverage dispenser nozzle 50.

[0035] The machine 1 also comprises a prechamber 60 for the dosing unit 8 that moves between the introduction opening 10 and the operating area 4.

[0036] The machine 1 is preferably intended for use with dosing units, preferably provided with one or more predetermined graphic recognition markings that identify the classification to which each dosing unit belongs, in particular the type of ingredients or substances included for making beverages and the type of beverages or product lines that can be manufactured. These recognition markings are of an optically detectable type visible on the reference surface of the dosing unit and are, for example, of the type comprising one or more images bearing product names and display markings.

[0037] Figures 2a to 2c show, by way of example, some possible solutions by means of a dosing unit 8 in the form of a tablet that can be provided with a graphic recognition marking including (at least) one inscription on the outer face of the graphic recognition marking and an associated display marking.

[0038] In particular, FIGS. 2a, 2b, and 2c show a dosing unit 8 to which instructions 70 are attached, such as, for example, a cartouche identifying a product line and zero, one, or two display markings 72 associated with the cartouche, and in particular, as an example, show the case where it is formed by a simple symbol (e.g., a circle, a square) affixed to a corner of the cartouche identifying a mixture of precursor materials. In the case of coffee powder, the mixture of precursor materials has a brown color of different intensities depending on the composition of the mixture or the storage conditions of the mixture, while the instructions and display markings produced by printing or the application of other edible additives, thermal marking, or the application of a local laser beam or embossing / engraving are, for example, contrasting colors such as dark brown or black. Of course, a similar recognition effect can be obtained by creating graphic markings in a contrasting color lighter than the color of the background of the mixture of precursor materials, or, in any case, by having an illuminance or luminance different from that of the precursor substances depending on the illuminance of the dosing unit. In some embodiments, the visible range of the graphic markings due to the contrast with the background of the precursor materials may indicate an illumination wavelength or an observation (image acquisition) wavelength different from the spectrum of wavelengths visible to the human eye.

[0039] This type of dosing unit 4 can be advantageously used in a machine of the type shown in FIG. 1. The detection device (reader) 62 is located, for example, on one side of the prechamber 60 along the path of the dosing unit between the introduction opening 10 and the opening region 4.

[0040] The machine 1 is equipped with an electrical processing and control unit 64 to which the code detection device 62 is connected and which executes the operating cycle of the machine for classifying and managing the dosing unit. The electrical processing and control unit 64 is configured to be changeable according to a predetermined model according to the classification of the dosing unit 8 introduced into the machine, and the image is obtained by using the detection device 62 and / or, preferably, by acquiring data indicating the mode of use of the machine 1 by the user, such as the type of beverage.

[0041] The processing and control unit 62 may be configured to change the injection time and / or the amount and / or temperature and / or pressure of water and / or steam passing through the dosing unit, for example, according to the type of the dosing unit and the type of the beverage dosing unit.

[0042] The control system for the machine of FIG. 1 is shown in more detail by the block diagram of FIG. 3.

[0043] The dosing unit 8 is shown (in the pre-chamber 60) between a pair of devices 80, 82 adapted to detect the presence of the dosing unit 8 by interrupting an infrared signal transmitted from the emitter device 80 to the receiving device 82, for example. In communication with the devices 80 and 82, the micro-control unit 84 for image recognition and classification coordinates the detection of the introduction of the dosing unit into the pre-chamber and is coupled to one or more lighting modules 86 adapted to at least temporarily illuminate the pre-chamber to enable the acquisition of at least one image of at least one reference surface of the dosing unit. Conveniently, the lighting module 86 may include means for adjusting the intensity and / or color (or, more generally, wavelength) of each lighting beam.

[0044] Generally, the lighting beams may be in the visible spectrum or in other spectra such as, for example, infrared or ultraviolet, in one or more selected wavelength bands, and may have variable lighting angles.

[0045] Reference numeral 88 denotes an image acquisition module including, for example, an RGB camera or a CCD sensor with LED ring illumination. The image acquisition module is used for acquiring the aforementioned images of the dose unit under the control of the microcontrol unit 84. The image acquisition module, under the control of the illumination module, triggers the taking of one or more images of the dose unit inserted into the machine while moving towards the injection chamber, and is configured to perform recognition and classification of the images. The microcontrol unit 84 is coupled to the image acquisition module 88 via an adapter module 90 adapted to transmit image data to the microcontrol unit 84. The microcontrol unit 84 is programmed to classify the detected images according to the procedures described below. The detected images are determined by a classification model such as a neural network in the form of a computer program stored in the associated permanent memory module. The associated permanent memory module can be optionally rewritten by a connection of the microcontrol unit with an update module, for example, a wired connection with a local update module or a wireless connection with a remote update module, for example, a wireless connection via a local or wide area communication network, thereby enabling the update of the classification model to a new classification of the dose unit or an update at different times during the operating life of the machine.

[0046] The image acquisition module 88 can operate at at least one predetermined acquisition angle in one or more selected wavelength bands, in the visible spectrum or, for example, in other spectra such as infrared or ultraviolet. The at least one predetermined acquisition angle is not necessarily frontal and can be variable, for example.

[0047] The microcontroller 84 is configured to communicate with a separate control means (not shown) of the machine, and the separate control means of the machine is configured to control at least one parameter or execution management mode of the dosing unit, such as, for example, the injection parameters of the machine or the execution mode or parameters or execution modes related to other steps. The separate control means of the machine is, for example, to control the compression of the dosing unit during the pre-injection step and / or during the injection step, a control according to a certain profile, or, according to the dosing unit classification according to the relevant data moved by the adapter module 90, for the collection compartment for the units used from the injection chamber, for the control defined according to a predetermined control curve of the squeezing of the post-injection dosing unit to remove the residual water of the automatic removal of the post-injection dosing unit, or, for example, for recording the usage data of the machine, such as for counting the number of dispensing operations for statistical purposes and for predicting the wear possibility of the image acquisition means, etc., for communicating with the remote management system, is configured to control at least one parameter or operation management mode of the dosing unit.

[0048] Advantageously, the result of the classification process of the dosing units gradually inserted into the machine during the machine operation is communicated over time to an external management system temporarily associated via a wired connection or a remote wireless connection, such as via a local or wide area communication network, for example, and the microcontroller may receive programming instructions, for example, to update the characteristic parameters of the classification model.

[0049] In the current preferred embodiment, there are three images acquired for the dosing unit inserted into the machine, and the microcontroller 84 selects the best image through a selection algorithm, such as a "voting" algorithm based on a voting strategy, such as stacked majority voting Naive Bayes.

[0050] In a pre-chamber communicating with the injection chamber, in the image, there may be vapor from the injection chamber indicated by the pictogram denoted by V. For example, for the suction of vapor generated by a previous injection operation or for the suction of particles of the precursor in the suspension, and for cleaning the optical system of the image acquisition device where the fan means 92 is controlled by each electrical control module 92, the pre-chamber is preferably associated with the fan means 92 for cleaning the detection device 62 integrated in the image acquisition and recognition module 88.

[0051] Finally, the reference numeral 92 denotes a module for managing the power supply to the system components.

[0052] FIG. 4 shows a current preferred embodiment of the microcontrol unit 84, which operates as an electrical system for the processing and automatic recognition of a machine learning type dosing unit. The electrical system is configured to receive at least one acquired image I of the dosing unit as an input and to classify the dosing unit into one of a plurality of predetermined classifications (or types) of the dosing unit based on the classification of the dosing unit used in a set of training images or learning steps of the dosing unit.

[0053] In a presently preferred embodiment, the microcontroller unit 84 includes a convolutional neural network configured to receive, as input, a set of values of a predetermined matrix of pixels of the acquired image I of the dosing unit, the set of values of the predetermined matrix of pixels indicating the gray levels or intensities of color channels, such as, for example, RGB channels, and more generally indicating the amount of light reflected or emitted by the illuminated surface of the dosing unit at the acquired wavelength, (e.g., by phosphorescence or fluorescence of an edible additive substance on which a graphic marking is created), the neural network comprising an input layer, a plurality of cascaded hidden layers, and an output layer adapted to provide a classification of the dosing unit into one of a plurality of manufacturer classifications and one of a plurality of product classifications. More specifically, the first classification is intended to recognize, based on the marking 70, the product line of the dosing unit inserted into the machine by distinguishing, for example, between graphic markings associated with distributable product lines, similar markings, or markings associated with non-distributable product lines (depending on the type of dispensing machine model), and the second classification is intended to recognize the mixture of precursor materials based on the display marking 72.

[0054] Specifically, the neural network includes a first subset FR of hidden feature extraction layers and a second subset CL of hidden classification layers. A subset FE of hidden feature extraction layers, which has the purpose of extracting prominent features from images useful for classification purposes, includes a plurality of hidden layers, each of which includes a series of convolutional blocks in a residual configuration. The series of convolutional blocks is configured to extract a series of feature maps from the acquired images of the dose unit by repeating the application of a predetermined filter whose value is defined in the learning step of the neural network described later in this description, along with the reduction of the size of the data indicating the image of the dose unit. The subset CL of hidden classification layers includes two parallel processing legs CL1 and CL2. Each of the two parallel processing legs CL1 and CL2 outputs corresponding vectors V1 and V2 indicating each classification result. Thereafter, each classification result has a plurality of fully connected layers that are concatenated into a single output vector VC.

[0055] More specifically, referring to the example of FIG. 5, the subset FE of the feature hidden layer includes five layers, and each of the five layers is configured to apply a pair of convolutional filters to the input matrix and the ReLU activation function according to the residual configuration, as shown in the enlarged inset of FIG. 5, and to apply a pooling operation that halves the size of the input matrix at the output. Instead of the pooling operation, the convolutional filter may be applied with a stride other than 1. Advantageously, this configuration of the feature hidden layer enables the problem of vanishing gradients to be mitigated by error backpropagation in the supervised training of the network. Further, the application of the convolutional filter is performed according to a depthwise separable convolutional approach so as to accelerate the convergence of training.

[0056] The size of the layers may vary according to the processing resources available to the machine.

[0057] The first layer is adapted to receive an input matrix representing pixel values of a predefined region of the original image. As an example for the case of a central region of size 256×256 pixels, it undergoes convolution with 16 filters of size 3×3 and a stride of 1 and application of the ReLU activation function in a residual configuration, and then performs a max-pooling operation with a stride of 2, resulting in an output of a plurality of 16 feature maps of size 128×128.

[0058] The second layer sends the 16 feature maps of size 128×128 pixels calculated in the first layer to convolution with 32 filters of size 3×3 and a stride of 1 and application of the ReLU activation function in a residual configuration, and then performs a max-pooling operation with a stride of 2, resulting in an output of a plurality of 32 feature maps of size 64×64.

[0059] The third layer sends the 32 feature maps of size 64×64 pixels calculated in the second layer to convolution with 32 filters of size 3×3 and a stride of 1 and application of the ReLU activation function in a residual configuration, and then performs a max-pooling operation with a stride of 2, resulting in an output of a plurality of 32 feature maps of size 32×32.

[0060] The fourth layer sends the 32 feature maps of size 32×32 pixels calculated in the third layer to convolution with 64 filters of size 3×3 and a stride of 1 and application of the ReLU activation function in a residual configuration, and then performs a max-pooling operation with a stride of 2, resulting in an output of a plurality of 64 feature maps of size 16×16.

[0061] Finally, the fifth layer is adapted to send the 64 feature maps of size 16×16 pixels, calculated in the fourth layer, to the convolution of 64 filters of size 3×3 with a stride of 1 and the application of the ReLU activation function in the residual configuration, and then perform a max pooling operation with a stride of 2, resulting in outputting a plurality of 64 feature maps of size 8×8.

[0062] Leading to the linearization (flattening) operation, the pixel values of the 64 feature maps of size 8×8 are concatenated into a vector of size 4069, and the vector of size 4069 is supplied as an input to each leg CL1, CL2 of the subset CL of the classification layer.

[0063] The subset of the classification layer includes four layers for each process leg, and each of the four layers is configured to perform a multilinear combination of the input values and apply the associated activation function, preferably a non-linear activation function, for example, the ReLU function in the input layer and the sigmoid or softmax function in the output layer.

[0064] The first layer is adapted to receive the input vector of size 4069 output by the subset of the feature extraction layer, and the input vector of size 4069 undergoes a non-linear weighted sum to become an output vector of size 64.

[0065] The second layer is adapted to receive the vector calculated in the first layer of size 64, and the vector undergoes a non-linear weighted sum to become an output vector of size 32.

[0066] The third layer is adapted to receive the vector calculated in the second layer of size 32, and the vector undergoes a non-linear weighted sum to become an output vector of size 16.

[0067] Finally, the fourth layer is adapted to receive the vectors computed in the third layer of size 16, the vectors undergo a non-linear weighted sum and result in an output vector of size 3, the output vector of size 3 indicates the final result of the classification, and the classification is preferably normalized using a softmax function that indicates, in probability, the dose units belonging to each of the predefined classifications.

[0068] The above is shown as merely an example, and it is understood that different convolutional network configurations, such as, for example, the number of hidden layers, different filters and pooling parameters, or different activation functions, may be used. Further, what has been described above can be extended to a color image by appropriately referring to a plurality of input matrices indicating pixel values of a predefined area of the original image in color channels, for example, three RGB color channels, with reference to an input matrix indicating pixel values of a predefined area of the original image where each pixel has a grayscale level value.

[0069] The neural network described starts in a supervised fashion to determine the values of the convolutional filter of a subset of hidden layers and the values of the parameters of the activation function, starting from a data collection comprising a plurality of images of dose units that represent all the different classifications required by the system. The plurality of images are acquired by one or more training image acquisition modules similar to the image acquisition module provided to the mounted machine, and each of the plurality of images is pre-classified with at least one relative classification, in certain cases, a classification indicating the product line and a classification indicating the mixture of precursor materials. Preferably, the pre-classified data set is classified into a first test data set comprising 20% of the data set, a second training data set comprising 80% of the remaining data set, and a third validation data set comprising the remaining 20% of the remaining data set. Validation is performed in each cycle, for example, by following a cross-validation procedure with five partitions (k-fold cross-validation, k = 5) and exchanging the validation data with the training data. Advantageously, the data of the test, training, and validation data sets can be selected manually or automatically from the data collection, while respecting as much as possible the percentage breakdown shown above for each known classification.

[0070] Conveniently, the data collection may, if any, include samples of different colors and different sizes (thicknesses).

[0071] Images of a plurality of dose units indicating all the different classifications required for a system constituting data collection are obtained by at least a plurality of main images of dose units obtained by one or more training image acquisition modules from samples of available dose units, and a plurality of synthetic images obtained by changing the plurality of main images according to data augmentation techniques. The plurality of synthetic images include, as shown in FIG. 6 as an example, each graphic recognition marking changed with respect to a predetermined graphic recognition marking or with respect to a reference plane of a dose unit to which the predetermined graphic recognition marking is attached. The change of the graphic recognition marking can be obtained by changing only the graphic marking with respect to the reference plane of the dose unit or by overall changing the image. In particular, the changed graphic recognition marking has morphological, dimensional, positional or optical characteristics changed with respect to the predetermined graphic recognition marking. More specifically, the changed graphic recognition marking includes, for example, at least one predetermined graphic recognition marking with changed movement, rotation, reduction or enlargement, distortion, imperfection, blur, change in light intensity or brightness, change in color, or change in contrast.

[0072] The use of synthetic images with changed graphic recognition markings has the advantage that even in the case of acquired images with changed features with respect to the main images acquired starting from available samples of dose units, extensive training of the neural network can be carried out and the generalization of the neural network can be enhanced. In particular, - For example, the generation of a synthetic image having a graphic recognition marking moved 0.05% vertically or parallel to the main reference image can strengthen the neural network and make it classification-invariant with respect to the position of the dose unit in the prechamber 60. - For example, the generation of a synthetic image having a graphic recognition marking rotated at an angle changeable between -180° and +180° with respect to the main reference image can strengthen the neural network and make it classification-invariant with respect to the rotation of the dose unit in the prechamber 60. - For example, generating a synthetic image with a graphic recognition marking that is reduced or enlarged with a variable scale between -0.10% and +0.10% with respect to the main reference image can enhance the neural network and make it classification-invariant with respect to the axial position of the dose unit in the pre-chamber 60, - For example, generating a synthetic image with a distorted graphic recognition marking with a pitch that can be changed between -0.10 px and +0.10 px with respect to the main reference image can enhance the neural network and make it classification-invariant with respect to the acquisition speed of the image of the dose unit (or the falling speed of the dose unit) in the pre-chamber 60, - For example, generating a synthetic image with an incomplete graphic recognition marking due to defects of several parts within a predetermined incompleteness limit threshold with respect to the main reference image can enhance the neural network and be classification-invariant with respect to the completeness of the graphic marking, and be more robust in the case of wear of the reference plane of the dose unit that causes partial cancellation of the reference plane of the dose unit with a graphic marking attached, - Generating a synthetic image with a blurred graphic recognition marking with respect to the main reference image can enhance the neural network and make it classification-invariant with respect to the focus of the dose unit in the pre-chamber 60, - For example, generating a synthetic image with a graphic recognition marking having a light intensity or brightness that is changed to be brighter or darker, respectively, within a brightness range between 0.35% and 1.25% with respect to the main reference image can enhance the neural network and make it classification-invariant with respect to the color and reflectivity of the dose unit in the pre-chamber 60, and the color and reflectivity of the dose unit may depend even on the storage conditions of the tablet, (e.g., due to exposure to unsuitable temperature or humidity environmental conditions), the deterioration state of the tablet or the deterioration and cleanliness of the lighting means of the pre-chamber and the image acquisition optical system, - For example, for a main "dual" reference image where the color of the graphic marking is darker than the background color, generating a composite image with a graphic recognition marking of a color where the color of the graphic marking is changed to be lighter than the background color can enhance the neural network. In the pre-chamber 60, it is made classification-invariant with respect to the color and reflectivity of the dose unit that depends on the manufacturing technology of the graphic recognition marking. - Generating a composite image with a graphic recognition marking having a changed contrast that is brighter or darker than a predetermined reference contrast between the graphic marking and the reference surface of the dose unit in the main reference image can enhance the neural network and make it classification-invariant with respect to the storage conditions, deterioration state of the tablet, or the color and reflectivity of the graphic marking of the dose unit that depends on the manufacturing technology of the reference graphic marking.

[0073] When the samples of the dose units for data collection do not have a sufficiently large population showing a predetermined classification, as shown in FIG. 7, for example, by changing the number of display markings 72 associated with the engraving 70, for the purpose of generating an image that does not belong to the same classification as the original image, it is also possible to prepare a plurality of additional composite images or the main images themselves obtained by manufacturing the graphic recognition markings of a plurality of main images. This can be done, for example, by applying pattern recognition technology to the image of the dose unit to identify the engraving 70 and construct the associated display markings 72 (in the rectangular and circular investigation areas displayed over the images in the left column), and applying image overwriting technology to completely or partially delete the local display markings 72 (in the middle and right columns).

[0074] Advantageously, the self-supervised training also provides for obtaining an image of a dose unit without recognition markings (having features similar to the recognized dose unit, i.e., having the background reference plane of said dose unit that matches in color and texture, i.e., having the reference plane of said dose unit that is inconsistent) or an image without a dose unit.

[0075] The self-supervised training of the neural network is performed by applying a well-known error backpropagation algorithm, such as the Adam optimization algorithm with 300 cycles (epochs), for providing a training data set and updating network parameters in each batch of 32 images (batch size) with a learning rate of 0.01, a decay factor of 0.5, a patience of 10 epochs, a minimum learning rate of 0.0000002, and label smoothing equal to 0.2, based on a data collection comprising a main image and a synthetic image.

[0076] The self-supervised training of the neural network may employ a known algorithm for minimizing an error function, such as a categorical cross-entropy error function.

[0077] Preferably, a subset of the feature extraction layers is provided in at least one embedding layer to optimize classification by switching to the self-supervised mode in addition to the above, and the "triplet loss" technique is used for self-vision of the feature output space from the feature extraction subset, thereby modifying the distance in the reference space of the features between a reference sample that is recognized as a "positive" of the same classification and a sample that is rejected as a "negative" of a different classification, and separating samples that are very similar to the recognized samples and that may be mixed with non-self-supervised training.

[0078] FIG. 8a shows a flowchart of the execution of an initial process of a machine for preparing and dispensing a warm beverage by infusion.

[0079] Specifically, in step 100, the acquisition of the main image of the dose unit intended for the collection of training data and inserted into the machine is executed. In step 120, preprocessing of the acquired image is executed to improve the contrast of the acquired image without distorting the pixel values. During preprocessing, the acquired image is, for example, subjected to histogram and Otsu threshold analysis in step 122. The analysis is suitable for a bimodal image, for example, an image having a histogram that is clearly separated between two main peaks. The two main peaks are - for example, in the visible wavelength range, related to image acquisition - and occur in the case of a brown dose unit. For example, due to the influence of black or white graphic traces on a brown background or the influence of recessed or convex surface areas or matte or shiny engravings or reliefs on the surface around the dose unit, there may be graphic recognition markings that are darker or brighter than the background of the matrix of the precursor material in terms of contrast. Subsequently, the upper limit of the pixel values of the image (clipping) is executed in step 124. Next, equalization of the image histogram for background suppression is performed in step 126. Finally, zero filtering is executed in step 128 to minimize the influence of image illumination and acquisition (except when graphic markings include reliefs and / or engravings) on the induced depth.

[0080] The above may be applied regardless of whether it is a monochrome image or a color image in grayscale by dividing the application of normalization or equalization in three channels.

[0081] In the next step 140, the conversion of the image is performed in a format and dimensions acceptable by the neural network. In practice, in order to ensure the high performance of the system in the classification of the dose unit inserted into the machine, it is important to supply the neural network for feature extraction and classification of an image having the same size characteristics as the images used in the neural network training step. For this purpose, the scale of the scanned image is preferably performed. The acquired image is, for example, a 640×480 pixel image, and is reduced to a 256×256 pixel image through a first cropping operation for selecting the central region of the 480×480 pixel size of the original image and a second bilinear interpolation operation for changing the cropped image within the image to be used with a size of 256×256, or in any case of a predetermined size based on the amount of pixel values intended to be processed in the neural network considering the balance of opposing calculation needs, i.e., memory savings and calculation speed, and recognition efficiency needs.

[0082] In step 160, the data augmentation technique is adapted to obtain a synthetic image from the main image, and in step 180, the image normalization or standardization procedure is adapted to the main image and the synthetic image comprising subtraction from the image of the average value and subsequent division by the standard deviation. Instead of the embodiment, the image preprocessing described in step 120 may also be performed on the synthetic image, following step 160.

[0083] Subsequently, in step 200, it is demonstrated that the acquisition of the training data collection is completed, and when the training of the neural network described above is performed, it is shown as step 220 herein.

[0084] Following the training of the neural network, the acquisition of the neural network model is performed in step 240 by approximating floating-point values to integer values, such as 8-bit integer values, based on a predetermined zero value and a predetermined scale factor according to the following formula.

[0085]

Number

[0086] The quantization operation enables a more compact representation of the neural network model and allows the calculations of the neural network to be performed with less memory than the memory usage required for floating-point precision calculations that reduce the space occupied by the neural network and the classification time.

[0087] FIG. 8b shows a flowchart of the execution of a control or management process of a machine for preparing and dispensing a warm beverage by injection.

[0088] Specifically, in step 500, an image of the dose unit inserted into the machine is acquired, and in step 520, preprocessing of the acquired image is performed and adapted to improve the contrast of the acquired image without distorting the pixel values. During preprocessing, the acquired image is subjected to processing adapted to the training image of the neural network, particularly in the analysis of the histogram and Otsu's threshold in step 522, the upper limit of the pixel values of the image (clipping) in step 524, the quantization of the image histogram with background suppression in step 526, and finally the zero filtering operation in step 528.

[0089] In the next step 540, the image modification is performed, for example, as described above, by selecting a central region of 480×480 pixel size for the first cropping operation and a second bilinear interpolation operation for changing the cropped image to a usable image of 256×256 size, and resizing the image acquired at 640×480 pixels to a 256×256 pixel image, so as to be executed in an acceptable format and dimension by the neural network.

[0090] In step 580, the image normalization or standardization procedure is adapted to comprise subtraction from the mean value image and addition by the standard deviation.

[0091] Subsequently, in step 640, the quantization of the image is performed by approximating the floating-point values obtained by normalizing or equalizing the image, for example, 8-bit floating-point values, to integer values based on a predetermined zero value and a predetermined scale factor according to the following formula.

[0092]

Equation

[0093] Subsequently, in step 660, the classification of the image by the neural network is performed, and at the end of the classification, the inverse operation of quantization is performed by applying the above formula, and the correct distribution of probabilities of various classifications is obtained.

[0094] Advantageously, the foregoing neural network configuration and the processing of the neural network input data can execute the image recognition and classification process with reduced memory resources (on the order of 1-2 megabytes), such as those typically installed in a machine for preparing and serving warm beverages for home use.

[0095] Finally, in step 700, based on the classification knowledge of the dose unit introduced into the machine, at least one of the following operations is performed: - For example, in the pre-injection step and / or the injection step, control of the compression of the dose unit, such as control of at least one parameter or execution management mode of the dose unit, for example, the injection parameters or execution mode of the machine or parameters or execution modes related to other steps, the control being from the injection chamber to the collection compartment for the unit used, according to a predetermined profile, or control according to a predetermined control curve of the squeezing of the dose unit after injection for removing residual water from the automatic removal of the dose unit after injection; - Recording and / or transmitting machine usage data indicating the classification of the dose unit recognized for the purpose of statistical monitoring, for example, for a remote measurement system, etc.

[0096] It should be noted that the embodiments of the present invention proposed above are provided merely as examples and do not limit the present invention. Those skilled in the art can easily implement the present invention in different embodiments within the protection scope of the invention defined by the appended claims without departing from the general principles outlined in this specification. This applies in particular with regard to the possibility of acquiring a video, i.e., a sequence of images (films) of the dose unit, at different spatial positions along the transfer path to the injection chamber, which represents an extension of the dimensions in the case described above, whereby the image recognition unit receives, as input, the sequence of said images for one of the above and is adapted to perform the classification of the dose unit based on the sequence of images and includes a neural network. In one embodiment, the neural network is configured to perform individually on each of the individual images to which the "voting" procedure is applied to select the most likely classification. Instead of the embodiment, the neural network is configured to be performed on a video file of at least one transfer operation of the dose unit. For example, the video file includes frames taken as a whole of the dose unit by a wide-angle camera, whereby the neural network, in the case of an uncompressed file, includes a matrix stack equal to the number of video frames, or is applied to classify a compressed film including one or more reference frames (of the fully described images) and prediction frames each including partial information indicating only the changes relative to the corresponding reference images.

[0097] Of course, without impairing the principles of the present invention, the embodiments and details of implementation may be significantly changed with respect to those described and illustrated merely as non-limiting examples without departing from the protection scope of the present invention defined by the appended claims.

Claims

1. A control system for a machine (1) for preparing and dispensing hot beverages by controlled injection of granular or powdered precursors arranged in a dose unit (8), Image acquisition means (62, 88) adapted to acquire at least one image (I) of the dose unit (8) inserted into the machine (1) or at least one image (I) of a selected portion of the dose unit (8) inserted into the machine (1), An electrical means (84) for processing and automatically recognizing the machine learning type dose unit (8), wherein the electrical means (84) is configured to receive at least one acquired image (I) of the dose unit (8) as input, and to classify the dose unit (8) into one of a plurality of predetermined classifications of dose units based on a set of training images of the dose unit used in the learning step, A machine control means (64) connected to the electrical means (84) for processing and automatic recognition, configured to control at least one parameter or execution mode for managing the dose unit (8) according to the recognition classification of the dose unit, or to record a data item of machine (1) usage indicating the recognition classification of the dose unit (8) and transmit it to a remote management system, Equipped with, The set of training images for the dose unit is characterized by comprising at least a plurality of main images of the dose unit (8), each including predetermined graphic recognition markings (70, 72) on the reference surface of the dose unit (8) that indicate the corresponding classification of the dose unit, and a plurality of composite images obtained by changing the plurality of main images. Control system.

2. The plurality of composite images include the modified graphic recognition markings with respect to the predetermined graphic recognition markings (70, 72) or the reference surface of the dose unit (8), The control system according to claim 1.

3. The modified graphic recognition marking exhibits morphological, dimensional, positional, or optical characteristics that have been altered with respect to the predetermined graphic recognition markings (70, 72). The control system according to claim 2.

4. The modified graphic recognition marking includes at least one of the predetermined graphic recognition markings that has been moved, rotated, reduced or enlarged, distorted, incomplete, blurred, altered in light intensity or brightness, altered in color or contrast. The control system according to claim 3.

5. The electrical means (84) for processing or automatic recognition of the machine learning type dose unit (8) includes a convolutional neural network configured to receive a predetermined matrix and set of pixel values ​​of the at least one acquired image (I) as input. The aforementioned convolutional neural network comprises an input layer, a plurality of cascaded hidden layers, and an output layer. The plurality of cascade hidden layers comprise a first subset (FE) of feature extraction layers and a second subset (CL) of classification layers. The control system according to claim 1.

6. The feature extraction layer subset (FE) comprises multiple layers, Each of the aforementioned layers is configured to extract a plurality of feature maps from the image (I) of the dose unit (8) by repeatedly applying a predetermined filter. The predetermined filter value is defined in the learning step of the neural network. Each of the aforementioned layers is configured to reduce the data size of the feature map that shows the matrix of pixels in the acquired image (I). The control system according to claim 5.

7. The plurality of layers of the feature extraction layer subset (FE) includes five layers, Each layer includes a set of convolutional filters intended to be applied to the input pixel matrix or input feature map, and a ReLU activation function depending on the residual configuration in the output. The convolutional filter or subsequent pooling stage is configured to halve the data size of the input feature map. The control system according to claim 6.

8. The subset of the classification layer (CL) comprises two parallel processing legs (CL1, CL2), each of which has a plurality of fully connected layers and is adapted to receive as input to a data array containing the pixel values ​​of the plurality of feature maps output from the subset of the feature extraction layer (FE). The control system according to claim 6.

9. The plurality of fully connected layers of the subset (CL) of the classification layer comprises four layers, each of which is configured to perform a weighted linear combination of input values ​​and apply an associated activation function, preferably a nonlinear activation function. The control system according to claim 8.

10. The electrical means (84) for processing and automatic recognition are configured to communicate with a remote management system that is temporarily connected to the control system via a local or remote connection. The aforementioned connection is either wired or wireless. The electrical means (84) transmits the classification result or receives programming instructions for the parameters of the neural network. The control system according to claim 5.

11. The system comprises trigger means (80, 82) coupled to image acquisition means (62, 88) which are adapted to detect the insertion or passage of a dose unit (8) within an association recognition sheet (60) and configured to trigger the acquisition of at least one image (I) of the dose unit (8) or a selected portion of the dose unit (8) located within the association recognition sheet (60), The control system according to claim 1.

12. The system further comprises an illumination means (86) adapted to direct illumination rays toward the associated recognition sheet (60), The control system according to claim 11.

13. The illumination light is emitted in one or more selected wavelength bands, which include at least one wavelength band in the visible light, infrared, or ultraviolet spectrum. The control system according to claim 12.

14. With respect to the illumination means (86), means for adjusting the intensity and / or wavelength of the illumination light, The control system according to claim 13.

15. The illumination means (86) is configured to operate at at least one predetermined variable acquisition angle. The control system according to claim 12.

16. The electrical means (84) for processing or automatic recognition of the dose unit (8) is configured to perform a process of modifying the acquired image (I) by applying a predetermined modification model to modify the acquired image in accordance with instructions received from the remote management system, the remote management system being temporarily connected to the control system via a local or remote connection, the connection being wired or wireless. The control system according to claim 1.

17. The electrical means (84) for processing and automatic recognition of the dose unit (8) is configured to identify the predetermined graphic recognition markings (70, 72) in the at least one acquired image (I) and to generate a comparison index indicating the similarity between the at least one acquired image and a reference image. The control system according to claim 1.

18. The aforementioned similarity is defined by a dynamically changeable threshold. The control system according to claim 17.

19. The parameters or execution mode for managing the dose unit (8) include at least one injection parameter or execution mode comprising at least one of the temperature, pressure, or amount of the injected liquid. The control system according to claim 1.

20. The parameters or execution modes for managing the dose unit (8) include at least one parameter or execution mode for controlling the compression of the dose unit (8) during a pre-injection step and / or injection step to compress the dose unit (8) after injection to remove residual water, or the automatic removal of the dose unit (8) after injection from the injection chamber (4) into a collection compartment for used units. The control system according to claim 1.

21. The graphic recognition markings (70, 72) include at least one surface graphic marking that shows a color different from the color of the precursor material of the dose unit, and a graphic marking that has a raised or cut surface. The control system according to claim 1.

22. The image acquisition means (62, 88) is configured to operate in one or more selected wavelength bands, which include at least one wavelength band in the visible light, infrared, or ultraviolet spectrum. The control system according to claim 1.

23. The image acquisition means (62, 88) is configured to operate at at least one predetermined variable acquisition angle. The control system according to claim 1.

24. The image acquisition means (62, 88) is configured to acquire a sequence of images (I) of the dose unit (8) at different spatial positions along the transport path of the dose unit (8) to the injection chamber (4). A control system according to any one of claims 1 to 23.

25. A control method for a machine (1) for preparing and dispensing hot beverages by controlled injection of granular or powdered precursors arranged in a dose unit (8), The aforementioned machine (1) is Electrical means (84) for processing and automatic recognition of machine learning type dose units (8), wherein the electrical means (84) is configured to receive at least one image (I) of the dose unit (8) or a selected portion of the dose unit (8) as input, and to classify the dose unit (8) into one of a plurality of predetermined classifications of dose units, A control means (64) for managing a dose unit (8), the control means (64) configured to control at least one parameter or execution mode for managing the dose unit (8), Equipped with, The method is, The learning step involves configuring the electrical means (84) to process and automatically recognize a set of training images of a dose unit, each including predetermined graphic recognition markings (70, 72) on a reference surface of the dose unit (8), and comprising at least a plurality of main images of the dose unit indicating the corresponding classification of the dose unit, and a plurality of composite images obtained by changing the main images. In the execution step, The steps include: obtaining an image (I) of at least one of the dose unit (8) of the precursor inserted into the machine (1) or of a selected portion of the dose unit (8); The steps include providing at least one acquired image (I) of the dose unit (8) as input to the electrical means (84) for processing and automatic recognition, The steps include classifying the dose unit (8) into one of a plurality of predetermined classifications of dose units using the electrical means (84) for processing and automatic recognition, A step comprising at least one of the following: controlling at least one parameter or execution mode for managing the dose unit (8) according to the classification of the dose unit; and recording and / or transmitting data items of machine (1) usage indicating the classification of the dose unit (8) to a remote management system for the purpose of statistical monitoring. A feature comprising: Control method.

26. The invention comprises one or more code modules for performing a control method for a machine (1) for preparing and dispensing hot beverages by controlled injection of a granular or powdered precursor as described in claim 25, Computer programs and groups of programs that can be executed by a processing system.

Images