Camera-based monitoring of beverage dispensing

Abstract

The beverage dispenser includes a nozzle for dispensing the beverage. The beverage dispenser further includes a camera that captures an image of the beverage as it is dispensed from the nozzle. The camera has a field of view that includes the beverage. The beverage dispenser further includes a light source that illuminates the field of view of the camera. The beverage dispenser further includes a computer. The computer analyzes the image of the beverage and determines characteristics of the beverage.

Description

Technical Field

[0001] The embodiments described herein generally relate to beverage dispensing. Specifically, the embodiments described herein relate to monitoring beverage dispensing by a camera.

Background Art

[0002] A beverage dispenser can dispense beverages made within the beverage dispenser by mixing two or more components such as water, additive components (e.g., syrup), and optionally carbonation. The components can be stored in canisters, and a single dispenser can include multiple canisters, each associated with a different beverage or flavor. Improper mixing of two or more components can result in poor taste and customer dissatisfaction.

Summary of the Invention

[0003] Some embodiments described herein relate to a beverage dispenser including a nozzle configured to dispense a beverage and a camera configured to capture an image of the beverage as it is dispensed from the nozzle, the camera having a field of view that includes the beverage. The beverage dispenser also includes a light source configured to illuminate the field of view of the camera.

[0004] In any of the various embodiments discussed herein, the nozzle is configured to dispense a plurality of beverages, and the beverage is one of the plurality of beverages.

[0005] In any of the various embodiments discussed herein, the image includes optical data.

[0006] In any of the various embodiments discussed herein, the camera is an RGB camera.

[0007] In any of the various embodiments discussed herein, the field of view of the camera In the direction in which the beverage is dispensed. extends vertically with respect thereto.

[0008] In any of the various embodiments discussed herein, the beverage dispenser is Dispensing area The camera includes, Dispensing area It is mounted on the outside of the nozzle, at an angle both above and to the side.

[0009] In any of the various embodiments discussed herein, the light source is opposite the camera's field of view.

[0010] Some embodiments described herein are methods for monitoring a beverage being dispensed from a beverage dispenser, comprising dispensing the beverage from the nozzle of the beverage dispenser and, during the dispensing of the beverage, using a camera on the beverage dispenser to capture an image of the beverage as it is dispensed from the nozzle. This invention relates to a method that includes analyzing an image to determine the characteristics of a beverage.

[0011] In any of the various embodiments discussed herein, the characteristic is an additive component characteristic, which includes the concentration or ratio of the additive component of the beverage to other components of the beverage.

[0012] In any of the various embodiments discussed herein, the additive component comprises at least one flavoring agent, enhancer, sweetener, and coloring agent.

[0013] In any of the various embodiments discussed herein, the other components include at least one of the basic components and auxiliary basic components.

[0014] In any of the various embodiments discussed herein, the basic component comprises a consumable liquid, and the auxiliary basic component comprises a consumable gas.

[0015] In any of the various embodiments discussed herein, the characteristic is the presence of carbonic acid.

[0016] In any of the various embodiments discussed herein, the characteristic is the presence of water.

[0017] In any of the various embodiments discussed herein, the method includes determining an error in dispensing a beverage based on the characteristics of the beverage.

[0018] Some embodiments described herein relate to a beverage dispenser comprising: a nozzle configured to dispense a beverage; a camera configured to capture an image of the beverage as it is dispensed from the nozzle; a computer; and a non-temporary computer-readable medium operably connected to the computer, the non-temporary computer-readable medium comprising, when executed by the computer, instructions causing the computer to analyze an image of the beverage and determine the characteristics of the beverage.

[0019] In any of the various embodiments discussed herein, when an instruction is executed by a computer, the computer causes the computer to determine an error in dispensing a beverage based on the characteristics of the beverage.

[0020] In any of the various embodiments discussed herein, the characteristic is an additive component characteristic, which includes the concentration or ratio of the additive component of the beverage to other components of the beverage.

[0021] In any of the various embodiments discussed herein, the analysis of the images includes linear unmixing analysis. [Brief explanation of the drawing]

[0022] The accompanying drawings, incorporated herein and forming part thereof, illustrate the disclosure and, together with the description, further illustrate the principles of the disclosure and enable those skilled in the art to construct and use the disclosure. [Figure 1] An exemplary beverage dispenser according to one embodiment is shown in a perspective view. [Figure 2] A schematic diagram of an exemplary beverage dispenser according to one embodiment is shown. [Figure 3A] Shows the field of view of the camera of the beverage dispenser of FIG. 1. [Figure 3B] Shows the field of view of the camera of the beverage dispenser of FIG. 1. [Figure 4] Shows an exemplary computer of a beverage dispenser according to one embodiment. [Figure 5] Shows an exemplary process for monitoring a dispensed beverage according to one embodiment. [Figure 6A] Shows a heat map of the estimated concentration percentage of the additive component of the first dispensed beverage using calibrated optical data for the first beverage according to one embodiment. [Figure 6B] Shows a heat map of the error associated with the estimated concentration of FIG. 6A. [Figure 7A] Shows a heat map of the estimated concentration percentage of the additive component of the first dispensed beverage using calibrated optical data for the second beverage according to one embodiment. [Figure 7B] Shows a heat map of the error associated with the estimated concentration of FIG. 7A. [Figure 8] Shows an exemplary process for determining whether there is a dispensing error using a combination of linear separation analysis and morphological analysis according to one embodiment. [Figure 9A] Shows images of a carbonated beverage stream and a flat beverage stream according to one embodiment. [Figure 9B] Shows an edge detection analysis performed on the image of FIG. 9A.

DETAILED DESCRIPTION OF THE INVENTION

[0023] Hereinafter, reference will be made in detail to representative embodiments illustrated in the accompanying drawings. It should be understood that the following description is not intended to limit the plurality of embodiments to one preferred embodiment. On the contrary, the present invention is intended to include alternatives, modifications, and equivalents that may be included within the spirit and scope of the embodiments defined by the claims.

[0024] A beverage dispensing system can mix two or more components, such as a base component and an additive component, to form a beverage. For example, many beverage dispensers have a water line for supplying water as the base component and one or more containers for holding flavorings as the additive component. The components can be dispensed from a single nozzle that can dispense multiple different beverages. Alternatively, multiple nozzles can be used, each dispensing a single beverage.

[0025] Ensuring that ingredients are dispensed according to the prescribed beverage recipe is crucial for beverage quality. For example, if too little of the additive ingredients are used, the dispensed beverage may be diluted and fail to achieve the desired flavor profile. Similarly, in the case of carbonated beverages, too much or too little gas can alter the carbonation of the dispensed beverage.

[0026] Beverage dispenser operators ensure that ingredients are dispensed according to the prescribed beverage recipe. Beverage manufacturers may have limited control over operators, especially when operators are managed by third-party employees, such as in restaurants, cinemas, theaters, or other places where food and beverages are sold to the public. Dispensing problems can negatively impact consumer perception of the dispensed beverage. Negative consumer experiences with beverage dispensers can negatively affect future sales. Therefore, to ensure consistent quality of dispensed beverages, it is necessary to automate the monitoring of dispensed beverages at a low cost.

[0027] Some embodiments described herein relate to beverage dispensers that may include a nozzle configured to dispense beverages. The beverage dispenser may also include a camera configured to capture an image of the beverage as it is dispensed from the nozzle. The camera may have a field of view that includes the nozzle. The beverage dispenser may also include a light source configured to illuminate the camera's field of view.

[0028] In one embodiment, the beverage dispenser may include a computer and a non-temporary computer-readable medium operablely connected to the computer. The non-temporary computer-readable medium may, when executed by the computer, include instructions that cause the computer to analyze an image of the beverage and determine its characteristics.

[0029] Some embodiments described herein relate to methods for monitoring beverages dispensed from a beverage dispenser. These methods may include dispensing beverages from the nozzle of the beverage dispenser. While dispensing, these methods may include using a camera on the beverage dispenser to capture images of the dispensed beverage as it is dispensed from the nozzle. These methods may also include analyzing the images to determine the characteristics of the dispensed beverage.

[0030] Embodiments of the present disclosure address the need for efficient automation of monitoring dispensed beverages and improve the consistent quality of dispensed beverages. For example, embodiments of the present disclosure can detect inappropriate ingredient ratios or quantities in dispensed beverages.

[0031] Figure 1 shows a perspective view of an exemplary beverage dispenser 100 according to an aspect of the present disclosure. Figure 2 shows a schematic diagram of the beverage dispenser 100. In an embodiment, the beverage dispenser 100 includes a nozzle 102 for dispensing a beverage 104. As used herein, the term “beverage” may refer to the basic component 106, either alone or in combination with one or more additive components 110, with or without an auxiliary basic component 108. The beverage 104 may be dispensed into a cup 112.

[0032] As used herein, the term “basic ingredients” may, in particular, refer to any freely flowing, consumable liquid such as water, or a dairy beverage such as milk.

[0033] As used herein, the term “auxiliary basic component” may refer to any consumable gas, such as carbon dioxide, pressurized nitrogen, or other suitable gas, that can be injected into or dissolved in basic component 106. As used herein, references to “carbonate” or “carbonation” include injecting or dissolving carbon dioxide or any other auxiliary basic component 108 into basic component 106.

[0034] As used herein, the term “additive ingredients” may, in particular, refer to certain flavorings such as cola, grape, orange, lemon-lime, cherry, or vanilla, or to enhancers (e.g., multivitamin complexes, minerals, and energy boosters), sweeteners, or colorants, whether in the form of liquids, syrups, concentrates, or other forms.

[0035] In embodiments, the beverage dispenser 100 may include one or more nozzles 102. Each of the nozzles 102 may dispense a single dedicated beverage 104. As used herein, a reference to “each” encompasses embodiments of the beverage dispenser 100 that include any number of named structures. For example, a reference to each nozzle 102 encompasses embodiments of the beverage dispenser 100 that include only a single nozzle 102, or any number of nozzles 102.

[0036] In this embodiment, each nozzle 102 may dispense multiple different beverages 104. That is, a single nozzle 102 may be configured to dispense multiple different beverages 104.

[0037] In an alternative embodiment, the beverage dispenser 100 may include only one nozzle 102. The single nozzle 102 may be configured to dispense multiple different beverages.

[0038] In embodiments, the beverage dispenser 100 may include one or more basic component supply units 114 that supply one or more basic components 106 to nozzles 102. Each basic component supply unit 114 can supply at least one basic component 106 (e.g., water, milk, etc., as described above) to each nozzle 102. The basic component supply unit 114 may be a water line connected to a local water supply unit. The water line may include a filter. The basic component supply unit 114 may be one or more containers that contain the basic components 106. The basic component supply unit 114 may include a pump for pumping the basic components 106 through a component path 120, as will be discussed further herein.

[0039] In one embodiment, the basic component supply unit 114 can supply a single basic component 106 to each nozzle 102. In an alternative embodiment, the basic component supply unit 114 can supply two or more basic components 106 to each nozzle 102.

[0040] In some embodiments having two or more nozzles 102, the basic component supply unit 114 can supply one basic component 106 to one or more nozzles 102, and another different basic component 106 to one or more different nozzles 102. In some embodiments, the basic component 106 can be supplied to the nozzles 102 with or without an auxiliary basic component 108.

[0041] In one embodiment, the beverage dispenser 100 may include one or more auxiliary basic component supply units 116 that supply one or more auxiliary basic components 108 for injection or dissolution with the basic component 106. The auxiliary basic component supply unit 116 can supply at least one auxiliary basic component 108 (for example, the aforementioned carbon dioxide) to each basic component 106. The auxiliary basic component supply unit 116 may be a container for holding the auxiliary basic components 108.

[0042] In one embodiment, the auxiliary basic component supply unit 116 can store the auxiliary basic component 108 at a pressure higher than atmospheric pressure.

[0043] In this embodiment, the auxiliary basic component supply unit 116 can store the auxiliary basic component 108 at a pressure sufficient to maintain it in the liquid phase within the container. The auxiliary basic component 108 can transition from the liquid phase to the gas phase after leaving the auxiliary basic component supply unit 116.

[0044] In this embodiment, the auxiliary basic component supply unit 116 may include a pump for pressurizing the auxiliary basic component 108 through the component path 120. The auxiliary basic component 108 may be injected together with the basic component 106 at any point in the component path 120, or may be dissolved in the basic component 106.

[0045] In some embodiments, the beverage dispenser 100 does not need to include the auxiliary basic component supply unit 116. For example, the auxiliary basic component 108 may be injected together with the basic component 106, or dissolved in the basic component 106, and supplied together with the basic component 106 from the basic component supply unit 114.

[0046] In the embodiment, the beverage dispenser 100 may include one or more additive component supply units 118 that supply one or more additive components 110 to the nozzles 102. The additive component supply unit 118 can supply at least one additive component 110 (e.g., flavorings, enhancers, sweeteners, colorants, etc., as described above) to each nozzle 102. The additive component supply unit 118 may be one or more containers containing the additive components 110. The additive component supply unit 118 may include a pump for pumping the additive components 110 through the component pathway 120.

[0047] In this embodiment, the additive component supply unit 118 can supply a single additive component 110 to each nozzle 102.

[0048] In an alternative embodiment, the additive component supply unit 118 can supply two or more additive components 110 to each nozzle 102 so that each nozzle 102 can dispense two or more beverages 104.

[0049] In some embodiments having two or more nozzles 102, the additive component supply unit 118 can supply one additive component 110 to one or more nozzles 102 and another different additive component 110 to one or more different nozzles 102, so that each nozzle 102 can supply its own beverage 104.

[0050] In the embodiment, the basic component 106 can be supplied to the nozzle 102 with or without the additive component 110. The additive component 110 can be injected together with the basic component 106 at any point in the component pathway 120, or dissolved in the basic component 106.

[0051] As described above, in embodiments, the beverage dispenser 100 may include component paths 120 that connect basic component, auxiliary basic component, or additive component supply units 114, 116, 118 to the nozzles 102. The component paths 120 can fluidly connect the basic component, auxiliary basic component, or additive component supply units 114, 116, 118 to one or more nozzles 102 of the beverage dispenser 100. The component paths 120 may include hoses, pipes, connectors, etc. The component paths 120 may include valves for controlling the flow of the basic component 106, auxiliary basic component 108, or additive component 110. The valves may be controlled mechanically or electromechanically.

[0052] In one embodiment, the beverage dispenser 100 may include a camera 122 capable of capturing images of the dispensed beverage 104 in real time during dispensing. The camera 122 can capture images of the beverage 104 while it is being dispensed from the nozzle 102. The images may include optical data. The camera 122 may include an imaging system configured to receive different color ranges separately.

[0053] In the embodiment, the camera 122 may separately receive multiple color ranges, for example, including red, green, and blue ranges. The camera 122 may be an affordable, commercially available camera 122.

[0054] In this embodiment, camera 122 may be an RGB camera.

[0055] In this embodiment, the camera 122 may be a rolling shutter camera.

[0056] In this embodiment, the camera 122 may be a global shutter camera.

[0057] In this embodiment, the camera 122 may capture images at a standard Super Extended Graphics Array resolution, i.e., a resolution of 1280 × 1024 pixels.

[0058] The beverage dispenser 100 may include one or more cameras 122. In one embodiment, a single camera 122 may be provided for each nozzle 102.

[0059] In this embodiment, multiple cameras 122 (for example, two cameras) mounted at different positions may be provided on each nozzle 102.

[0060] The camera 122 can be mounted on the nozzle 102 such that the nozzle 102, the dispensed beverage 104, and / or the cup 112 are within the camera 122's field of view.

[0061] In one embodiment, the camera 122 may be mounted such that its field of view includes the respective portions of the nozzle 102, the dispensed beverage 104, and the cup 112.

[0062] As shown in Figure 1, in this embodiment, the beverage dispenser 100 may include a first camera 122a and a second camera 122b. The first camera 122a may be mounted on the side (for example, vertically) of the nozzle 102 and the cup 112. Figure 3A shows the field of view of the first camera 122a in Figure 1, including the nozzle 102, the cup 112, and the dispensed beverage 104. The field of view of the first camera 122a is Beverage 104 can be dispensed. It can extend perpendicular to the direction.

[0063] In this embodiment, the first camera 122a has a field of view in which part of it is Beverage 104 can be dispensed. The lens may include a suitable lens that can be effectively perpendicular to the direction while allowing different parts of the field of view to face downwards.

[0064] The second camera 122b may be mounted at an angle above and to the side of the nozzle 102 so that the nozzle 102, the dispensed beverage 104, and the cup 112 are each within its field of view. Figure 3B shows the field of view of the second camera 122b in Figure 1, including the nozzle 102, the cup 112, and the dispensed beverage 104.

[0065] In this embodiment, the camera 122 is located near the beverage dispenser 100. Dispensing area It may be mounted on the outside of 124. For example, the second camera 122b is positioned above and to the side of the nozzle 102. Dispensing area It may be mounted at an angle to the outside of 124.

[0066] In one embodiment, camera 122 is Dispensing area It may be mounted inside 124. For example, the first camera 122a is Dispensing area It may be mounted inside 124.

[0067] Figure 1 includes first and second cameras 122a and 122b, but in an embodiment, the beverage dispenser 100 may include only one camera 122 (for example, only the first camera 122a).

[0068] In this embodiment, the camera 122 may be mounted using a general-purpose bracket that can accommodate multiple different types of cameras.

[0069] According to the embodiment, the field of view of the camera 122 can provide a view of the dispensed beverage 104, while also allowing a sufficient view into the cup 112, for example, to determine the filling level of the cup 112.

[0070] In one embodiment, the beverage dispenser 100 may include a light source 126 capable of illuminating the field of view of the camera 122, including while the beverage 104 is being dispensed from the nozzle 102. The light source 126 may provide illumination for the beverage dispenser 100, or it may be separate from a separate light source 127 of the beverage dispenser 100. The light source 126 may be fluorescent, incandescent, LED, OLED, or any other suitable light source.

[0071] In this embodiment, the light source 126 may selectively illuminate when the camera 122 is in use or when a user is near the beverage dispenser 100. The light source 126 may also illuminate while the beverage 104 is being dispensed from the nozzle 102.

[0072] As shown in Figure 1, the beverage dispenser 100 may include a first light source 126a and a second light source 126b.

[0073] In the embodiment, the light source 126 (e.g., the first light source 126a) can be positioned opposite the camera 122 (e.g., the first camera 122a) to illuminate the field of view.

[0074] The light source 126 may selectively illuminate with wavelengths in the range of 300 nm to 1100 nm. In such embodiments, image analysis, such as linear separation analysis, which will be discussed in more detail herein, can be improved by increasing the spectral information content.

[0075] Alternatively, the light source 126 may illuminate within a specific wavelength range, such as the wavelength range corresponding to white light. Such embodiments enable analysis of a single image (e.g., spectral decomposition) and real-time analysis of the dispensed beverage 104.

[0076] In some embodiments, the beverage dispenser 100 may include a computer 600 for analyzing an image of the beverage 104 and determining its characteristics. Figure 4 shows an exemplary computer 600 in which an embodiment or part thereof may be implemented as computer-readable code. For example, aspects of process 500 or process 800 may be implemented via the computer 600, as will be discussed further herein.

[0077] When programmable logic is used, such logic can be executed on a commercially available processing platform or a device for a particular purpose. Those skilled in the art will understand that embodiments of the disclosed subject matter can be put into practice in a variety of computer configurations, including multicore multiprocessor systems, minicomputers, and mainframe computers, distributed and linked or clustered computers, and popular or small computers that can be virtually embedded in any device.

[0078] For example, at least one processor device and memory may be used to implement the above embodiment. The processor device may be a single processor, multiple processors, or a combination thereof. The processor device may have one or more processor "cores".

[0079] Various embodiments of the present invention may be implemented with respect to this exemplary computer 600. After reading this description, methods for implementing one or more of the present invention using other computers or computer architectures will become apparent to those skilled in the art. While operations may be described as sequential processes, some operations may actually be performed by program code stored locally or remotely for access by single or multiprocessor machines, in parallel, simultaneously, or in a distributed environment. In addition, in some embodiments, the order of operations may be rearranged without departing from the spirit of the disclosed subject matter.

[0080] The processor 604 may be a dedicated or general-purpose processor device. As will be understood by those skilled in the art, the processor 604 may also be a single processor in a multicore / multiprocessor system, such a system operating independently or in a cluster of computing devices operating in a cluster or server farm. The processor 604 is connected to a communication infrastructure 606, such as a bus, message queue, network, or multicore message passing scheme.

[0081] The computer 600 includes main memory 608, for example, random access memory (RAM), and may also include secondary memory 610. Examples of secondary memory 610 include a hard disk drive 612 or a removable storage drive 614. Examples of removable storage drives 614 include floppy disk drives, magnetic tape drives, optical disk drives, flash memory, and Universal Serial Bus (USB) drives. The removable storage drive 614 reads from and writes to a removable storage unit 618 in a well-known manner. Examples of removable storage units 618 include floppy disks, magnetic tapes, optical disks, etc., which are read from and written to by the removable storage drive 614. As will be understood by those skilled in the art, a removable storage unit 618 may be a computer-usable storage medium storing computer software or data.

[0082] The computer 600 may include a display interface 602 (which may include input and output devices such as a keyboard and mouse) for transferring graphics, text, and other data from a communication infrastructure 606 (or from a frame buffer, not shown) for display on the display unit 630.

[0083] In an implementation, the secondary memory 610 may include other similar means for enabling computer programs or other instructions to be loaded into the computer 600. Such means may include, for example, a removable storage unit 622 and an interface 620. Examples of such means include a program cartridge and cartridge interface (such as those found in video game devices), a removable memory chip (such as an EPROM or PROM) and associated socket, and other removable storage units 622 and interfaces 620 that can transmit software and data from the removable storage unit 622 to the computer 600.

[0084] The computer system 600 may also include a communication interface 624. The communication interface 624 enables the transmission of software and data between the computer 600 and external devices such as the camera 122. The communication interface 624 may include a modem, a network interface (such as an Ethernet card), a communication port, a PCMCIA slot and card, etc. The software and data transmitted through the communication interface 624 may be in the form of signals, which may be electronic signals, electromagnetic signals, optical signals, or other signals that can be received by the communication interface 624. These signals may be provided to the communication interface 624 via a communication path 626. The communication path 626 carries the signals and may be implemented using wires or cables, optical fibers, telephone lines, cell phone links, RF links, or other communication channels.

[0085] In this specification, the terms “non-temporary computer-readable medium,” “computer program medium,” and “computer-usable medium” may refer to media such as the removable storage unit 618, the removable storage unit 622, and the hard disk installed in the hard disk drive 612. The computer program medium and computer-usable medium may also refer to memory such as the main memory 608 and the secondary memory 610, which may be memory semiconductors (e.g., DRAM).

[0086] A computer program (also called computer control logic) or database is stored in main memory 608 or secondary memory 610. The computer program may also be received via the communication interface 624. When such a computer program is executed, it enables the computer system 600 to implement the embodiments described herein. Specifically, when this computer program is executed, it enables the processor device 604 to implement the processes of the embodiments described herein. Thus, such a computer program represents a controller of the computer system 600. If the embodiments are implemented using software, the software is stored in the computer program product and may be loaded into the computer 600 using a removable storage drive 614, interface 620, and hard disk drive 612, or the communication interface 624.

[0087] Embodiments of the present invention may also cover computer program products that include software stored on any computer-readable medium. When such software is executed on one or more data processing devices, it causes the data processing devices to operate as described herein. Embodiments of the present invention may utilize any computer-readable or readable medium. Examples of computer-readable media include, but are not limited to, primary storage devices (e.g., any type of random-access memory) and secondary storage devices (e.g., hard drives, floppy disks, CD-ROMs, ZIP disks, tapes, magnetic storage devices, and optical storage devices, MEMS, nanotechnology storage devices, etc.).

[0088] Monitoring of dispensed beverages The following describes monitoring by a computer 600 that can be performed based on one or more images of the dispensed beverage 104. In embodiments, the computer 600 may implement instructions stored in a non-temporary computer-readable medium, which cause the computer 600 to perform a process 500 of monitoring the beverage 104 dispensed from the beverage dispenser 100. The beverage dispenser 100 may be any of the embodiments of the beverage dispenser 100 described above.

[0089] Process 500 may include dispensing a beverage 104 from a beverage dispenser 100 in step 501. For example, a customer or operator may approach the beverage dispenser 100 and select a specific beverage 104 (hereinafter, the "selected" beverage 104) to be dispensed.

[0090] In one embodiment, the computer 600 can control the beverage dispenser 100 to dispense the selected beverage 104 according to the recipe of the selected beverage 104. The computer 600 may store the recipes of each beverage 104 configured for the beverage dispenser 100 to dispense. For example, the computer 600 can dispense the selected beverage 104 by controlling the flow of ingredients from the ingredient supply unit through the ingredient path 120 and out of the nozzle 102. The computer 600 may, for example, use a motion detector (not shown) that activates a camera 122 to automatically detect the presence of the cup 112 and control the beverage dispenser 100. Dispensing area Dispensing of the beverage 104 can be started without any physical user interaction other than placing the cup 112 on 124. Process 500 allows the user to place the cup 112 on the beverage dispenser 100. Dispensing area Dispensing can be automatically started by the computer 600 based on the automatic detection of cup 112, which may include placing it on 124.

[0091] In the embodiment, one or more aspects of the beverage dispenser 100 may be mechanically controlled, for example, by a user pressing a cup 112 against a lever (not shown) to start dispensing, or by a user pressing a button (not shown) to start dispensing.

[0092] In step 502, process 500 may include capturing one or more images of the dispensed beverage 104. The computer 600 may control the camera 122 to capture images. The computer 600 may automatically control the camera 122 to capture images in response to the start of dispensing. The computer 600 may automatically control the camera 122 to capture one or more images before, during, or after dispensing.

[0093] In step 503, process 500 may include analyzing one or more captured images to determine the properties of the dispensed beverage 104. The properties of the dispensed beverage 104 may be additive component properties, auxiliary basic component properties, or basic component properties, either individually or in combination.

[0094] In embodiments, the properties may be additive component properties. The computer 600 may determine the additive component properties in several ways, including analyzing an image of the beverage 104 and evaluating the color or entropy of the image of the dispensed beverage 104, as will be discussed further herein.

[0095] In the embodiment, the characteristics may be auxiliary basic component characteristics, such as the presence or absence of auxiliary basic component 108. For example, the computer 600 may determine the presence of carbonation, which includes determining both the presence or absence of carbonation.

[0096] In the embodiment, the characteristics may be basic component characteristics, such as the presence or absence of basic component 106. For example, the computer 600 may determine the presence of water, including both situations in which water is present and situations in which water is not present.

[0097] In some embodiments, the characteristic may be the quality of the beverage flow. The quality can be represented by the form of the beverage, such as turbulence.

[0098] In step 504, the process may include determining, based on the characteristics of the beverage 104 determined in step 503, whether there is a detectable error in the dispensing of the beverage 104 in step 501, i.e., a dispensing error.

[0099] In an embodiment, the dispensing error may be an error in the dispensing of additive components. For example, an error in the dispensing of additive components may be a determination that the dispensed beverage 104 contains one or more additive components 110 at an inappropriate concentration. In an embodiment, the computer 600 may store the target concentration of each beverage 104 configured to be dispensed by the beverage dispenser 100. The estimated concentration of the dispensed beverage 104 may be compared to the target concentration of a particular beverage 104. If the computer 600 is outside a predetermined range (e.g., + / - 5%) of the target concentration of the beverage 104, it may determine that the dispensed beverage 104 contains additive components 110 at an inappropriate concentration.

[0100] Additionally or alternatively, a dispensing error may be a determination that the wrong additive component 110 is being dispensed. For example, in an embodiment of a beverage dispenser 100 that can dispense different beverages from the same nozzle 102, the user may select one of several beverages 104 for dispensing. In step 501, the computer 600 can determine the dispensing error of the additive component when the beverage dispenser 100 dispenses a beverage 104 other than the selected beverage 104.

[0101] In this embodiment, the computer 600 can determine dispensing errors of basic or auxiliary basic components, for example, significant errors in the amount of auxiliary basic component 108 being dispensed. An example of a detectable significant error in auxiliary basic component 108 is a failure to carbonate the beverage 104 that requires carbonation.

[0102] If the computer 600 determines in step 504 that there is a dispensing error, the process 500 can proceed to step 505, where the dispensing error can be addressed.

[0103] In some embodiments, addressing dispensing errors may include issuing a warning that a dispensing error has occurred.

[0104] In one embodiment, the warning can be displayed on the display of the beverage dispenser 100.

[0105] In one embodiment, the warning may be transmitted over the network to an operator who may be designated for further diagnosis or repair.

[0106] In some embodiments, the warning may include an automatic reordering of the product associated with the dispensing error.

[0107] If the computer 600 determines that there is no dispensing error detected in step 504, the process 500 may terminate or repeat step 501 with a new dispenser.

[0108] linear separation analysis In embodiments, steps 503 and 504 may involve linear separation analysis. Linear separation analysis may be particularly accurate for beverages 104 containing an additive component 110 (hereinafter referred to as the "high-contrast additive component") having a color substantially different from that of the base component 106. For example, linear separation analysis may be particularly accurate for monitoring dispensed cola beverages 104 containing an additive component 110 syrup having a color substantially different from that of the base component 106, i.e., carbonated water.

[0109] Linear separation analysis can quantitatively solve for the estimated concentration c of additive component 110 relative to the base component 106, along with the associated error E of linear separation analysis, using measured optical data y of dispensed beverage 104 and calibrated optical data M of selected beverage 104. Linear separation analysis may involve using calibrated optical data M collected from the components of dispensed beverage 104 in the model to reproduce the measured optical data y of dispensed beverage 104 while minimizing the associated error E of the model. Linear separation analysis can be performed pixel by pixel.

[0110] In this embodiment, the measured optical data y may be optical data decomposed from an image of the dispensed beverage 104. The computer 600 can decompose the optical data from the image of the dispensed beverage 104 into pixels. The decomposed optical data may include RGB values ​​contained within one or more pixels of the dispensed beverage image. For example,

[0111]

number

[0112] The basic component 106 used in relation to linear separation analysis may include, with or without an auxiliary basic component 108 (e.g., carbonation), depending on the recipe of a particular dispensed beverage 104. In embodiments where the recipe includes a basic component 106 with an auxiliary basic component 108 such as carbonation, the calibrated optical data M includes the auxiliary basic component 108 such as carbonation mixed with the basic component 106, but does not include the additive component 110.

[0113] A reference to additive components 110, at least as used in relation to linear separation analysis, may include one or more additive components 110, depending on the recipe for a particular dispensed beverage 104. In embodiments where the recipe includes two or more additive components 110, the calibrated optical data M includes information on the base components 106 in addition to all the additive components required for the beverage recipe dispensed together.

[0114] Calibrated optical data M can be collected for each beverage 104 configured to be dispensed by the beverage dispenser 100. The calibrated optical data M can be stored by the computer 600 for use in linear separation analysis. For example, in one embodiment, the computer 600 can store calibrated optical data M1 for a first beverage 104a and calibrated optical data M2 for a second beverage 104b. The calibrated optical data M1 for the first beverage 104a and the calibrated optical data M2 for the second beverage 104b may be different.

[0115] In the embodiment, collecting calibrated optical data M may include independently dispensing the additive components 110 and the base components 106 of the beverage 104. In the embodiment, calibrated optical data M can be collected only once for each beverage 104.

[0116] In this embodiment, the calibrated optical data M can be collected intermittently, for example, after scheduled maintenance of the beverage dispenser 100, or after the ingredient supply has been changed or replenished.

[0117] The calibrated optical data M may include color values ​​such as red, green, and blue (RGB) values ​​of the additive component 110 and the base component 106. For example, the computer 600 can determine the average color values ​​contained within the pixels of an image of an independent dispensing of the additive component 110 and the average color values ​​contained within the pixels of an image of an independent dispensing of the base component 106. The calibrated optical data M may be a matrix of average color values ​​of the additive component 110 and the base component 106 for the beverage 104. For example,

[0118]

number

[0119]

number

[0120]

number

[0121] The calibrated optical data M may be specific to each beverage 104 dispensed by the beverage dispenser 100. In this embodiment, the calibrated optical data M1 for the first beverage 104a and the calibrated optical data M2 for the second beverage 104b may be matrices of average RGB color values. The average RGB color values ​​for the calibrated optical data M1 of the first beverage 104a may differ from the average RGB color values ​​for the calibrated optical data M2 of the second beverage 104b.

[0122] In the embodiment, the linear separation analysis may include the computer 600 quantitatively solving for the estimated concentration c of the additive component 110 relative to the base component 106 while minimizing the associated error E using equation [1]. [1] y=Mc+E The estimated concentration c may be a characteristic of the additive component 110 determined in step 503 of process 500.

[0123] Figures 6A and 6B show heatmaps of the solution to equation [1] using images of the beverage 104 selected in step 501, i.e., the dispensed beverage 104 (e.g., the first beverage 104a) corresponding to the selected beverage 104. Figure 6A shows the estimated concentration c per pixel of the selected beverage 104. Figure 6B shows the associated error E of the linear separation analysis per pixel of the selected beverage 104.

[0124] Once the selected beverage 104 (e.g., the first beverage 104a) is dispensed, the computer 600 can use the optical data M1 of the first beverage 104a to solve equation [1] for the estimated concentration c of the dispensed beverage 104 with an acceptable association error E (e.g., within + / - 5% of the measured optical data y, as shown in Figure 6B). In such embodiments, the computer 600 may determine in step 504 that no dispensing error occurred. Figures 6A and 6B demonstrate that linear separation analysis can quantitatively estimate the concentration c of the dispensed beverage 104 with a low association error using the optical data M of the selected beverage 104.

[0125] Figures 7A and 7B show heatmaps of the solution to equation [1] using images of dispensed beverages 104 (e.g., second beverage 104b) that do not match the beverage 104 selected in step 501, i.e., are unintended beverages 104. Figure 7A shows the pixel-by-pixel estimated concentration c of the unintended beverage 104. Figure 7B shows the associated error E of the pixel-by-pixel linear separation analysis for the unintended beverage 104.

[0126] For an unintended beverage 104, the computer 600 can use optical data M1 for a selected beverage 104 (e.g., a first beverage 104a) to solve equation [1] for the estimated concentration c of the dispensed beverage 104. Nevertheless, since the optical data M1 for the selected beverage (e.g., a first beverage 104a) does not correspond to the dispensed unintended beverage 104 (e.g., a second beverage 104b), the associated error E required for the solution is unacceptable (e.g., outside the range of ±5% of the measured optical data y, as shown in Figure 7B). In such an embodiment, the computer 600 may determine that a dispensing error occurred in step 504 because the associated error is unacceptable. Figures 7A and 7B demonstrate that linear separation analysis can be used to detect the dispensing of an unintended beverage 104.

[0127] Morphological analysis In embodiments, steps 503 and 504 may include morphological analysis. Morphological analysis may be particularly useful for beverages 104 having an additive component 110 (hereinafter, "low-contrast additive component") having a color substantially similar to that of the base component 106. For example, linear separation analysis may be particularly useful for monitoring a dispensed lemon-flavored soft drink containing an additive component 110 (i.e., syrup) having a color substantially similar to that of the base component 106 (i.e., carbonated water).

[0128] In the embodiment, morphological analysis may be qualitative. For example, morphological analysis can determine the presence or absence of additive components 110 in the dispensed beverage 104.

[0129] In embodiments, morphological analysis may include measuring the entropy (e.g., optical entropy) of the dispensed beverage 104. The measured entropy may be the additive component characteristics determined in step 503. The entropy of the dispensed beverage 104 can be compared to the expected entropy. The expected entropy may be experimentally determined from the correct dispensing of each beverage 104 and may be stored by the computer 600 for use in morphological analysis. If the measured entropy is outside the acceptable range (e.g., + / - 5% of the expected entropy), the computer 600 may determine in step 504 that there is a dispensing error. The expected entropy may be unique to each beverage 104.

[0130] A combination of linear separation and morphological analysis Figure 8 shows process 800, which uses a combination of linear separation analysis and morphological analysis to determine if there is a dispensing error. Process 800 may be a subprocess of process 500. For example, process 800 may be a subprocess that determines if there is a dispensing error in step 504. Alternatively, process 800 may be performed independently of process 500.

[0131] Process 800 may be particularly useful in embodiments of a beverage dispenser 100 configured to dispense two or more beverages 104, including at least one beverage 104 having a high-contrast additive component and at least one beverage 104 having a low-contrast additive component. In such embodiments, linear separation analysis alone may be insufficient to distinguish certain types of dispensing errors. For example, if a selected beverage 104 (e.g., cola) contains a high-contrast additive component and an unintended beverage 104 (e.g., a lemon-flavored soft drink) having a low-contrast additive component is dispensed, linear separation analysis can solve equation [1] with a low associated error E for a low estimated concentration c of the additive component 110 in the selected beverage 104 (e.g., cola). In such a situation, linear separation analysis can determine that there is a dispensing error because the estimated concentration c is lower than the expected concentration of the selected beverage 104. Nevertheless, linear separation analysis alone may not be able to determine that the dispensing error was actually a result of dispensing an unintended beverage 104 (e.g., a lemon-flavored soft drink) having a low-contrast additive component. In such situations, process 800 can improve the diagnostic capabilities of beverage dispenser 100 by determining dispensing errors with greater specificity.

[0132] Process 800 may include, in step 801, performing a linear separation analysis of images of the dispensed beverages 104 to determine the estimated concentration c and associated error E, as described above.

[0133] In step 802, process 800 may include determining whether the associated error E of the linear separation analysis performed in step 801 is acceptable (e.g., within + / - 5% of the measured optical data y). If the computer 600 determines that the associated error E required for the solution is unacceptable (e.g., outside the + / - 5% range of the measured optical data y), the process may proceed to step 803 and determine that an unintended beverage dispensing error occurred, i.e., that a beverage 104 other than the selected beverage 104 was dispensed from the beverage dispenser 100. In embodiments where process 800 is a subprocess of process 500, process 500 may proceed from steps 504 to 505 of process 500 to address the dispensing error.

[0134] If the computer 600 determines that the associated error E is acceptable (for example, within + / - 5% of the measured optical data y), the process 800 can proceed to step 804.

[0135] Step 804 may include process 800 evaluating whether the selected beverage 104 contains low-contrast additive components. The contrast classification of additive components for each beverage 104 configured to be dispensed by the beverage dispenser 100 may be stored in the computer 600. The contrast classification may classify each beverage 104 as either a low-contrast additive component or a high-contrast additive component. Step 804 may include the computer 600 determining, based on the classification, whether the selected beverage 104 contains low-contrast additive components. If it is determined that the selected beverage 104 contains low-contrast additive components, process 800 may proceed to step 807. If it is determined that the selected beverage 104 does not contain low-contrast components, process 800 may proceed to step 805.

[0136] In step 805, process 800 can evaluate whether the estimated concentration c is acceptable. The computer 600 may store the range of acceptable concentrations of the additive component 110 for each dispensed beverage 104. The computer 600 may compare the estimated concentration c with the range of acceptable concentrations of the additive component 110 for the selected beverage 104. If the computer 600 determines that the estimated concentration c is within the range of acceptable concentrations of the additive component 110 for the selected beverage 104, process 800 may proceed to step 810, where the computer 600 may determine that no dispensing error has been detected. In embodiments where process 800 is a subprocess of process 500, process 500 may proceed from step 810 to step 501, which restarts process 500.

[0137] If the computer 600 determines that the estimated concentration c is outside the acceptable range of concentrations for the additive component 110 in the selected beverage 104, the process 800 may proceed to step 806, where the computer 600 may determine that there is a dispensing error in the additive component. In embodiments, the dispensing error in the additive component may be a determination that the concentration of the additive component 110 is too low. In embodiments where the process 800 is a subprocess of the process 500, the process 500 may proceed from step 806 to step 505 of the process 500 to address the dispensing error.

[0138] Process 800 may, in step 807, include performing morphological analysis of an image of the dispensed beverage 104 in order to determine the entropy of the dispensed beverage 104, as described above.

[0139] In step 808, process 800 can evaluate whether the entropy of the dispensed beverage 104 determined in step 807 is within the acceptable entropy range for the selected beverage 104. The acceptable range may be + / - 5% of the expected entropy of the dispensed beverage 104. The acceptable entropy range for each beverage 104 may be stored in computer 600. If computer 600 determines that the entropy is within the acceptable entropy range for the selected beverage 104, process 800 may proceed to step 810.

[0140] If the computer 600 determines that the entropy of the dispensed beverage 104 is outside the range of acceptable entropy values ​​for the selected beverage 104, the process 800 may proceed to step 809, where the computer 600 may determine that there is a dispensing error of the additive component. In embodiments, the dispensing error of the additive component may be a determination that too little of the additive component 110 has been supplied, for example, that the beverage dispenser 100 is unable to supply the additive component 110. In embodiments where the process 800 is a subprocess of the process 500, the process 500 may proceed from step 809 to step 505 of the process 500 to address the dispensing error.

[0141] Auxiliary basic component characteristic analysis In this embodiment, the computer 600 can analyze an image of the dispensed beverage 104 to determine the auxiliary basic component characteristics. As described above, the auxiliary basic component characteristics may be those determined in step 503 of process 500. For example, the computer 600 can analyze an image of the dispensed beverage 104 to determine whether or not carbonation is present in the dispensed beverage 104.

[0142] In one embodiment, the computer 600 can analyze an image of the dispensed beverage 104 to detect its edges. Based on the detected edges of the dispensed beverage 104, the computer 600 can determine whether or not carbonation is present in the dispensed beverage 104. For example, the computer may analyze the distance between the detected edges of the dispensed beverage 104, the number of detected edges of the dispensed beverage 104, the shape of the detected edges of the dispensed beverage 104, or other characteristic qualities of the detected edges of the dispensed beverage 104.

[0143] In this embodiment, the computer 600 can analyze an image of the dispensed beverage 104 to detect its color. Based on the color (for example, how bright or dark the dispensed beverage 104 is), the computer 600 can determine whether or not carbonation is present in the dispensed beverage 104.

[0144] In this embodiment, the computer 600 can analyze an image of the dispensed beverage 104 to detect the presence or absence of droplets on the outside of the dispensed beverage container. Depending on whether or not droplets are detected on the outside of the dispensed beverage container 104, the computer 600 can determine whether or not carbonation is present in the dispensed beverage 104.

[0145] Figure 9A shows images of two dispensed beverages 104: a carbonated dispensed beverage 104c and a flat dispensed beverage 104d, i.e., a non-carbonated dispensed beverage 104. Figure 9B shows images of the edges 128 of the carbonated dispensed beverage 104c and the edges 130 of the flat dispensed beverage 104d, as detected by the computer 600 from the images shown in Figure 9A. The detected edges can be compared to the expected edges for the selected beverage 104. The expected edges may be specific to the beverage 104 and may be stored in the computer 600. If the detected edges deviate significantly from the expected edges for the selected beverage 104, the computer 600 may determine that, in a dispensing error that occurred in step 504 of process 500, for example, the beverage dispenser 100 failed to supply carbonation to the dispensed beverage 104. If the detected edge of the dispensed beverage 104 does not deviate significantly from the expected edge, the computer 600 may determine that no carbonation dispensing error has been detected (for example, in step 504 of process 500).

[0146] Filling level analysis In one embodiment, the computer 600 may analyze images captured by the camera 122 of the beverage dispenser 100 to determine the filling level of the dispensed beverage 104 in the cup 112. The computer 600 may control the camera 122 to capture a series of images as the beverage 104 is dispensed into the cup 112. Each image in the series may be timestamped. The computer 600 may analyze the images in real time and may detect the top edge of the beverage 104 in each image.

[0147] In an embodiment of the beverage dispenser 100 that includes a camera 122 (e.g., a second camera 122b) mounted with at least a partial top-down view into the cup 112, the computer 600 may detect the upper rim of the beverage 104 relative to the inside of the cup 112.

[0148] In an embodiment of the beverage dispenser 100 having a camera 122 (e.g., a first camera 122a) with a side view of the cup 112, the computer 600 may detect the upper edge of the beverage 104 passing through the cup 112 based on the change in the color of the light passing through the cup 112.

[0149] For each image in a series of images, the computer 600 can determine the fill percentage of cup 112 based on the measured values ​​of the top edge of beverage 104 and the top edge of cup 112. Using the timestamp for each image and the determined fill percentage of cup 112 for each image, the computer 600 can estimate the fill percentage of cup 112. Based on the fill percentage of cup 112 and the determined fill percentage of cup 112, the computer 600 can predict when cup 112 will be filled and can control the beverage dispenser 100 to terminate dispensing of beverage 104 to prevent overfilling of cup 112.

[0150] It should be understood that the section "Modes for Carrying Out the Invention," rather than the sections "Summary of the Invention" and "Abstract," is intended to be used to interpret the claims. The sections "Summary of the Invention" and "Abstract" may describe one or more but not all exemplary embodiments of the Invention as conceived by the inventors, and are not intended to limit the Invention and the appended claims in any way.

[0151] The present invention has been described above with the help of functional building blocks illustrating the implementation forms of specific functions and their relationships. The boundaries of these functional building blocks are arbitrarily defined herein for the sake of explanation. Alternative boundaries can be defined, as long as the specific functions and their relationships are appropriately represented.

[0152] The above-mentioned descriptions of specific embodiments fully reveal the general nature of the present invention, and others, by applying the knowledge of those skilled in the art, can easily modify and / or adapt such specific embodiments to various uses without excessive trial and error and without departing from the general concept of the present invention. Therefore, such adaptations and modifications are intended to be within the meaning and scope of the equivalents of the disclosed embodiments based on the teachings and guidance presented herein. It should be understood that the expressions and terminology herein are for illustrative purposes only and not limiting, and therefore, the expressions and terminology herein should be interpreted by those skilled in the art in terms of the teachings and guidance herein.

[0153] The breadth and scope of the present invention should not be limited by any of the exemplary embodiments described above, but should be defined solely in accordance with the following claims and their equivalents.

Claims

1. It is a beverage dispenser, A nozzle configured to dispense beverages, A camera configured to capture an image of the beverage when it is dispensed from the nozzle, Computers and, The computer comprises a non-temporary computer-readable medium operably connected to the computer, The non-temporary computer-readable medium includes a beverage dispenser, which, when executed by the computer, includes instructions causing the computer to analyze the image of the beverage using linear separation analysis and to determine additive component characteristics, including the concentration or ratio of the additive components of the beverage to other components of the beverage.

2. The nozzle is configured to dispense multiple beverages, The beverage dispenser according to claim 1, wherein the beverage is one of the plurality of beverages.

3. The aforementioned image includes optical data, and is a beverage dispenser according to claim 1.

4. The beverage dispenser according to claim 1, wherein the camera is an RGB camera.

5. The beverage dispenser according to claim 1, wherein the field of view of the camera extends perpendicularly to the direction in which the beverage is dispensed.

6. Further equipped with dispensing compartments, The beverage dispenser according to claim 1, wherein the camera is mounted outside the dispensing compartment at an angle above and to the side of the nozzle.

7. The beverage dispenser according to claim 1, further comprising a light source configured to illuminate the field of view of the camera opposite to the field of view of the camera.

8. A method for monitoring beverages dispensed from a beverage dispenser, The beverage dispenser, Dispensing the beverage from the nozzle of the beverage dispenser, During the dispensing of the beverage, the camera of the beverage dispenser is used to capture an image of the beverage as it is dispensed from the nozzle. A method for monitoring a beverage dispensed from a beverage dispenser, comprising: performing linear separation analysis to analyze the image and determining the additive component characteristics, including the concentration or ratio of the additive component of the beverage to other components of the beverage.

9. A method for monitoring a beverage dispensed from a beverage dispenser according to claim 8, wherein the additive component comprises at least one of a flavoring agent, an enhancer, a sweetener, and a coloring agent.

10. A method for monitoring a beverage dispensed from a beverage dispenser according to claim 8, wherein the other component comprises at least one of a basic component and an auxiliary basic component.

11. A method for monitoring a beverage dispensed from a beverage dispenser according to claim 10, wherein the basic component comprises a consumable liquid and the auxiliary basic component comprises a consumable gas.

12. A method for monitoring the characteristics of a beverage dispensed from a beverage dispenser according to claim 8, comprising analyzing the image to determine the presence of carbonation.

13. A method for monitoring a beverage dispensed from a beverage dispenser according to claim 8, comprising analyzing the image to determine the presence of water.

14. A method for monitoring a beverage dispensed from a beverage dispenser according to claim 8, further comprising determining an error in the dispensing of the beverage based on the characteristics of the beverage.

15. The beverage dispenser according to claim 1, wherein when the instruction is executed by the computer, the computer determines the error in dispensing the beverage based on the characteristics of the beverage.

Images