Managing the dispensing of fluids into receptacles

Abstract

Disclosed herein are embodiments of systems, methods, and computer program products for managing the dispensing of fluid into a receptacle using imaging data. Imaging data captured by a camera configured with the fluid dispenser may be used to control an automatic receptacle fill operation of the fluid dispenser. An embodiment operates by receiving first imaging data indicative of a receptacle. Classification information indicative of an empty or full state of the receptacle may be determined, for example, by a predictive model. An embodiment may initiate dispensing of fluid into the receptacle based on an image of the first imaging data and the classification information indicative of an empty state of the receptacle. An embodiment may receive second imaging data and stop dispensing of fluid into the receptacle based on an image of the second imaging data and the classification information indicative of a full state of the receptacle.

Description

[Technical field]

[0001] Fluid (e.g., beverage and / or ice) dispensers often require user interaction (e.g., direct or indirect contact with the dispenser), such as pressing a cup against an activation lever and / or the like, to initiate and / or terminate dispensing. User interaction with a beverage dispenser to initiate beverage dispensing can cause unsafe / unsanitary conditions due to transfer of contaminants between the user's hand and / or cup and the activation lever. Contaminants transferred to the activation lever can transfer to the nozzle opening of the beverage dispenser and grow, thereby contaminating the beverage (and / or ice) to be consumed by an unsuspecting user. Fluid (e.g., beverage and / or ice) dispensers implementing conventional auto-filling technology, for example, fluid dispensers having a virtual activation lever that initiates and terminates dispensing when a virtual plane is disrupted by the cup, often experience unstable operation due to faulty and / or inaccurate sensor information. Inconsistent and / or inaccurate sensor information is often due to sensor failure and / or errors caused by ambient temperature changes and / or other environmental issues. Fluid dispensers implementing conventional auto-filling technologies, such as fluid dispensers with ultrasonic-based auto-filling technologies, often experience unstable operation due to faulty and / or inaccurate sensor information caused by ultrasonic signals bouncing off adjacent cups, spilled ice, or beverages, and / or the like. Fluid dispensers implementing conventional auto-filling technologies, such as virtual activation levers (and / or the like) and ultrasonic-based auto-filling technologies, operate by detecting objects (e.g., cups and hands) indiscriminately, resulting in overfilling or underfilling the cup with beverage (and / or ice). Overfilling a cup with beverage (and / or ice) is often wasteful and cumbersome. Additionally, filling a cup insufficiently with beverage (and / or ice) can be time consuming and can result in a poor user experience. [Brief description of the drawings]

[0002] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate the disclosure and, together with the specification, further serve to explain the principles of the disclosure and to enable those skilled in the art to make and use the disclosure. [Figure 1] FIG. 1 illustrates an example system that uses imaging data to manage the dispensing of fluid into a receptacle, according to some embodiments. [Diagram 2] FIG. 1 illustrates an example system for training an imaging module that may be used to manage the dispensing of fluid into a receptacle, according to some embodiments. [Diagram 3] 1 is a flow diagram of an example training method for generating a machine learning classifier for classifying imaging data used to manage the dispensing of fluid into a receptacle, according to some embodiments. [Figure 4] 1 is a flow diagram of an example method for managing the dispensing of fluid into a receptacle using imaging data according to some embodiments. [Diagram 5] 11 is a flow diagram of another example method of managing the dispensing of fluid into a receptacle using imaging data according to some embodiments. [Figure 6] 1 is a schematic block diagram of an exemplary computer system on which the described aspects may be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0003] Provided herein are exemplary system, apparatus, device, method, computer program product embodiments, and / or combinations and subcombinations thereof for managing the dispensing of fluid into a receptacle using imaging data. According to some aspects, an imaging device (e.g., a camera, etc.) can be positioned to capture imaging data (e.g., video, still images, etc.) of an area associated with a fluid dispenser (e.g., a beverage dispenser, a water dispenser, a fountain drink machine, etc.). For example, the field of view of the imaging device can capture the imaging data from the perspective of a dispensing nozzle of the fluid dispenser. When it is determined from the imaging data that a cup (or similar receptacle) is in the vicinity of (e.g., below, etc.) the dispensing nozzle, a predictive model can classify the cup as an empty cup (e.g., containing no fluid, etc.) or as a full cup (e.g., containing a set amount of fluid, etc.). The imaging data can then be used to auto-fill the cup with fluid from the fluid dispenser.

[0004] Embodiments herein provide various technical improvements over conventional systems by using imaging data to manage the dispensing of fluid into a cup (or similar receptacle). For example, conventionally, when operating a fluid dispenser (e.g., a beverage dispenser, water dispenser, fountain drink machine, etc.), a consumer may be required to physically contact the device to dispense and / or retrieve a fluid, beverage, product, and / or the like. However, fluid dispensers and / or the like may be subject to contamination from multiple consumers touching the device. A consumer may choose not to use a fluid dispenser and / or the like if the consumer feels that the fluid dispenser and / or the like is not clean, unhygienic, and / or that they may come into contact with contamination and become ill. Fluid (e.g., beverage and / or ice) dispensers implementing conventional auto-fill technology, such as those having a virtual activation lever that starts and stops dispensing when a virtual plane is broken by a cup, often experience unstable operation due to faulty and / or inaccurate sensor information. Inconsistent and / or inaccurate sensor information is often caused by sensors failing and / or generating errors due to ambient temperature changes and / or other environmental issues. Fluid dispensers implementing conventional auto-fill technology, such as those with ultrasonic-based auto-fill technology, often experience unstable operation due to faulty and / or inaccurate sensor information caused by ultrasonic signals bouncing off adjacent cups, spilled ice or beverages, and / or the like. Fluid dispensers implementing conventional auto-fill technologies, such as virtual activation levers (and / or the like) and ultrasonic-based auto-fill technologies, detect and operate on objects (e.g., cups and hands) indiscriminately, resulting in over-filling or under-filling of the cup with beverage (and / or ice). Over-filling a cup with beverage (and / or ice) is often wasteful and cumbersome. Additionally, under-filling a cup with beverage (and / or ice) can be time consuming and detract from the user experience.

[0005] Embodiments herein solve these technical problems by using imaging data to manage the dispensing of fluid into a cup (or similar receptacle) and enable non-contact withdrawal of fluid from a fluid dispenser, thereby reducing and / or preventing the transfer of contaminants and / or the like while also limiting overfill or underfill situations. These and other technical advantages are described herein.

[0006] 1 illustrates a block diagram of an example system 100 for managing the dispensing of fluid into a receptacle using imaging data, according to some embodiments. The system 100 can include a fluid dispenser 101, a computing device 103, and a receptacle 109.

[0007] The fluid dispenser 101 may incorporate and / or be configured with any number of components, devices, and / or equivalents that are not shown for ease of illustration and are conventionally incorporated and / or configured in fluid dispensers (e.g., beverage dispensers, water dispensers, fountain drink machines, etc.). For example, the fluid dispenser 101 may include one or more supplies of concentrated beverage syrup attached to a syrup pump via tubing through a cooling system (e.g., chiller, water bath, cold plate, etc.) to the dispensing unit 102. The dispensing unit 102 may meter the flow rate of the syrup at the time it is delivered to the beverage dispensing nozzle 106 after mixing. The fluid dispenser 101 may include a water supply line (e.g., connected to a water source) that supplies water to a carbonator. Carbonated water from the carbonator may be sent via tubing through the cooling system to the dispensing unit 102. The dispensing unit 102 may include syrup and water flow rate controllers that operate to meter the flow rates of syrup and water such that a selected ratio of water to syrup is delivered to the beverage dispensing nozzle 106 .

[0008] The computing device 103 may be in communication with the fluid dispenser 101. The communication between the computing device 103 and the fluid dispenser 101 may include any wired communication (e.g., fiber optics, Ethernet, coaxial cable, twisted pair, circuitry, etc.) and / or wireless communication technology (e.g., infrared technology, BLUETOOTH, near field communication, Internet, cellular, satellite, etc.). According to some aspects, the computing device 103 may be configured with and / or in close proximity to the fluid dispenser 101. According to some aspects, the computing device 103 may be configured separately and / or remotely from the fluid dispenser 101. The computing device 103 may transmit one or more signals (e.g., transmissions, requests, data, etc.) that control the operation of the fluid dispenser 101, such as one or more signals that control when the dispensing unit 102 should dispense fluid from the beverage dispensing nozzle 106.

[0009] To facilitate controlling the timing of when the dispensing unit 102 dispenses fluid from the beverage dispensing nozzle 106, the computing device 103 may include an imaging module 104. The imaging module 104 may include and / or communicate with one or more image capture devices, such as a camera 105 that captures imaging data (e.g., video, still images, etc.). The imaging module 104 may receive imaging data that provides a real-time and / or real-world representation of the receptacle 109. The imaging module 104 may receive imaging data that depicts objects within the field of view of the camera 105 that provides a real-time and / or real-world representation of the receptacle 109. For example, the imaging module 104 may receive imaging data that indicates when the receptacle 109 is positioned and / or placed under the beverage dispensing nozzle 106.

[0010] According to some aspects, the imaging module 104 may be configured to process the imaging data from the camera 105. The imaging module 104 may use artificial intelligence and / or machine learning, such as image recognition and / or object recognition, to identify objects depicted by one or more images of a video frame, still image, and / or the like, included in the imaging data. According to some aspects, the imaging module 104 may use one or more object identification and / or classification algorithms to determine / detect a state of the receptacle 109, such as whether the receptacle 109 contains fluid. According to some aspects, the imaging module 104 may use one or more object identification and / or tracking algorithms to determine / detect a location of a landmark in the imaging data, such as, for example, the fill line 108 of the receptacle 109 (e.g., an indication of available fluid volume, etc.) and / or the amount and / or location of fluid 107 dispensed into the receptacle 109 by the beverage dispensing nozzle 106.

[0011] FIG. 2 is an example system 200 for training the imaging module 104 to manage the dispensing of fluid into a receptacle 109 according to some embodiments. FIG. 2 is described with reference to FIG. 1. According to some embodiments, the imaging module 104 may be trained to determine an empty fluid state of a receptacle (such as, for example, receptacle 109) or a full fluid state of a receptacle. For example, the imaging module 104 may classify a receptacle (such as, for example, receptacle 109) as an empty cup or a full cup. According to some embodiments, the imaging module 104 may be trained to determine a fill line (such as, for example, fill line 108 of FIG. 1) of a receptacle (such as, for example, receptacle 109). According to some embodiments, the imaging module 104 may be trained to determine, for example, an amount and / or location of a fluid (e.g., fluid 107 of FIG. 1 ) dispensed into a receptacle by a beverage dispensing nozzle (e.g., beverage dispensing nozzle 106 of FIG. 1 ). The system 200 may use machine learning techniques to train at least one machine learning based classifier 230 (e.g., a software model, a neural network classification layer, etc.) configured to classify features extracted from imaging data, such as imaging data received from camera 105 of FIG. 1 , based on analysis of one or more training data sets 210A-210N by the imaging module 104 of FIG. 1 . The machine learning based classifier 230 may classify features extracted from the imaging data to identify a receptacle and determine information about the receptacle, such as an empty status of the receptacle, a full status of the receptacle, etc. According to some embodiments, the machine learning based classifier 230 can classify features extracted from the imaging data to identify the receptacle and determine a fill volume threshold and / or the amount of fluid in the receptacle (e.g., whether the amount of fluid in the receptacle meets a fill level threshold).

[0012] One or more of the training data sets 210A-210N may include labeled reference data such as labeled receptacle types (e.g., various shaped cups, bottles, cans, bowls, boxes, etc.), labeled receptacle conditions (e.g., receptacles with ice, receptacles without ice, empty receptacles, full receptacles, receptacles containing various amounts of fluid, receptacles containing straws and / or other objects, etc.), labeled receptacle capacities (e.g., receptacle fill thresholds, indications of amounts of fluid various receptacles can hold, etc.), labeled fluid types (e.g., beverage types, water, juice, etc.), labeled fluid behaviors (e.g., carbonation indications, viscosity indications, etc.), etc. The labeled reference data may include any number of feature sets (e.g., labeled data identifying features extracted from imaging data).

[0013] The labeled reference data may be stored in one or more databases. Data for managing the automatic receptacle fill detection and operation of the fluid dispenser (e.g., imaging data, etc.) may be randomly assigned to a training data set or a test data set. According to some aspects, the assignment of data to a training data set or a test data set may not be completely random. In this case, one or more criteria may be used during the assignment to ensure that similar receptacle types, similar receptacle conditions, similar receptacle capacities, similar fluid types, similar fluid behaviors, different receptacle types, different receptacle conditions, different receptacle capacities, different fluid types, different fluid behaviors, and / or the like are used in the training data set and the test data set, respectively. In general, any suitable method may be used to assign data to a training data set or a test data set.

[0014] The imaging module 104 can train the machine learning based classifier 230 by extracting feature sets from the labeled reference data according to one or more feature selection techniques. According to some aspects, the imaging module 104 can further define feature sets derived from the labeled reference data by applying one or more feature selection techniques to the labeled reference data in the one or more training data sets 210A-210N. The imaging module 104 can extract feature sets from the training data sets 210A-210N in various ways. The imaging module 104 can perform feature extraction multiple times, each time using a different feature extraction technique. In some cases, feature sets generated using different techniques can be used to generate different machine learning based classification models 240. According to some aspects, the feature sets with the highest quality metrics can be selected for use in training. The imaging module 104 can use the feature set(s) to build one or more machine learning based classification models 240A-240N configured to determine and / or predict receptacle type, receptacle status, receptacle capacity, fluid type, fluid behavior, and / or the like.

[0015] According to some aspects, the training data sets 210A-210N and / or the labeled reference data may be analyzed to determine any dependencies, associations, and / or correlations between receptacle types, receptacle conditions, receptacle capacities, fluid types, fluid behaviors, and / or the like in the training data sets 210A-210N and / or the labeled reference data. As used herein, the term "feature" may refer to any characteristic related to an item of data that may be used to determine whether the item of data falls within one or more particular categories. For example, the features described herein may include receptacle types, receptacle conditions, receptacle capacities, fluid types, fluid behaviors, and / or other features.

[0016] According to some embodiments, the feature selection technique may include one or more feature selection rules. The one or more feature selection rules may include determining which features in the labeled reference data occur more than a threshold number of times in the labeled reference data and identifying those features that meet the threshold as candidate features. For example, all features that occur more than twice in the labeled reference data may be considered as candidate features. Features that occur less than twice may be excluded from consideration as features. According to some embodiments, only one feature selection rule may be applied in feature selection, or multiple feature selection rules may be applied in feature selection. According to some embodiments, feature selection rules may be applied in a cascading manner, and further, feature selection rules are applied in a particular order and applied to the results of previous rules. For example, feature selection rules may be applied to the labeled reference data to generate information that may be used in a receptacle auto-fill operation of a fluid dispenser (e.g., an indication of receptacle type, an indication of receptacle status, an indication of receptacle capacity, an indication of fluid type, an indication of fluid behavior, etc.) The final list of candidate features may be analyzed according to additional features.

[0017] According to some aspects, the imaging module 104 may generate information (e.g., an indication of receptacle type, an indication of receptacle status, an indication of receptacle volume, an indication of fluid type, an indication of fluid behavior, etc.) that may be used for receptacle auto-filling operations of the fluid dispenser, which may be based on a wrapper method. The wrapper method may be configured to use a subset of features to train a machine learning model with the subset of features. Features may be added and / or removed from the subset based on inferences drawn from previous models. Wrapper methods include, for example, forward feature selection, backward feature elimination, recursive feature elimination, combinations thereof, etc. According to some aspects, forward feature selection may be used to identify one or more candidate receptacle types, receptacle status, receptacle volume, fluid type, fluid behavior, and / or the like. Forward feature selection is an iterative method that starts with no features in the machine learning model. In each iteration, the feature that most improves the model is added until adding a new variable no longer improves the performance of the machine learning model. According to some aspects, backward elimination can be used to identify one or more candidate receptacle types, receptacle conditions, receptacle volumes, fluid types, fluid behaviors, and / or the like. Backward elimination is an iterative method that starts with all features in the machine learning model. At each iteration, the least important features are removed until removing features does not result in any improvement. According to some aspects, recursive feature elimination can be used to identify one or more candidate receptacle types, receptacle conditions, receptacle volumes, fluid types, fluid behaviors, and / or the like. Recursive feature elimination is a greedy optimization algorithm that aims to find the best performing feature subset. Recursive feature elimination iteratively builds a model, leaving the best or worst features at each iteration. Recursive feature elimination builds the next model with the remaining features until all features are exhausted. Recursive feature elimination then ranks the features based on the order of elimination.

[0018] According to some aspects, one or more candidate receptacle types, receptacle conditions, receptacle volumes, fluid types, fluid behaviors, and / or the like may be determined according to an embedded methodology that combines attributes of filter and wrapper methods. Embedded methods include, for example, least absolute shrinkage selection operator (LASSO) and ridge regression, which implement a penalty function to reduce overfitting. For example, LASSO regression performs L1 regularization, which applies a penalty equivalent to the absolute value of the coefficient magnitude, and ridge regression performs L2 regularization, which applies a penalty equivalent to the square of the coefficient magnitude.

[0019] After the imaging module 104 generates the feature sets, the imaging module 104 can generate a machine learning based predictive model 240 based on the feature sets. A machine learning based predictive model may refer to a complex mathematical model for data classification generated by machine learning techniques. For example, the machine learning based classifier may include a map of support vectors representing boundary features. By way of example, the boundary features may be selected from the feature set and / or may represent the highest ranked features in the feature set.

[0020] According to some aspects, the imaging module 104 can use feature sets extracted from the training data sets 210A-210N and / or the labeled reference data to construct machine learning based classification models 240A-240N for determining and / or predicting receptacle type, receptacle status, receptacle capacity, fluid type, fluid behavior, and / or the like. According to some aspects, the machine learning based classification models 240A-240N may be combined into a single machine learning based classification model 240. Similarly, the machine learning based classifier 230 may represent a single classifier including a single or multiple machine learning based classification models 240 and / or multiple classifiers including a single or multiple machine learning based classification models 240. According to some aspects, the machine learning based classifier 230 may further include each of the training data sets 210A-210N and / or each feature set extracted from the training data sets 210A-210N and / or the labeled reference data. Although shown separately, the imaging module 104 may include a machine learning based classifier 230 .

[0021] Features extracted from the imaging data can be combined in classification models trained using the following machine learning approaches: discriminant analysis, decision trees, nearest neighbor (NN) algorithms (e.g., k-NN models, replicator NN models), statistical algorithms (e.g., Bayesian networks, etc.), clustering algorithms (k-means, mean shift, etc.), neural networks (e.g., reservoir networks, artificial neural networks, etc.), support vector machines (SVMs), logistic regression algorithms, linear regression algorithms, Markov models or chains, principal component analysis (PCA) (e.g., for linear models), multilayer perceptron (MLP) ANNs (e.g., for nonlinear models), replicated reservoir networks (e.g., for nonlinear models, typically for time series), random forest classification, combinations thereof, and / or the like. The resulting machine learning based classifier 230 may include decision rules or mappings that use the imaging data to determine and / or predict receptacle type, receptacle condition, receptacle capacity, fluid type, fluid behavior, and / or the like.

[0022] The imaging data and the machine learning based classifier 230 may be used to determine and / or predict receptacle type, receptacle status, receptacle capacity, fluid type, fluid behavior, and / or the like for the test samples in the test dataset. For example, the results of each test sample may include a confidence level corresponding to the likelihood or probability that the corresponding test sample will correctly determine and / or predict the receptacle type, receptacle status, receptacle capacity, fluid type, fluid behavior, and / or the like. The confidence level may be a value between 0 and 1 that represents the likelihood that the determined / predicted receptacle type, receptacle status, receptacle capacity, fluid type, fluid behavior, and / or the like matches the calculated value. Multiple confidence levels may be provided for each test sample and each candidate (approximated) receptacle type, receptacle status, receptacle capacity, fluid type, fluid behavior, and / or the like. The best performing candidate receptacle type, receptacle status, receptacle capacity, fluid type, fluid behavior, and / or the like may be determined by comparing the results obtained for each test sample with the calculated receptacle type, receptacle status, receptacle capacity, fluid type, fluid behavior, and / or the like for each test sample. In general, the best performing candidate receptacle type, receptacle status, receptacle capacity, fluid type, fluid behavior, and / or the like will have the closest results to the calculated receptacle type, receptacle status, receptacle capacity, fluid type, fluid behavior, and / or the like. The best performing candidate receptacle type, receptacle status, receptacle capacity, fluid type, fluid behavior, and / or the like may be used to manage automatic receptacle fill detection and operation of the fluid dispenser.

[0023] FIG. 3 is a flow diagram illustrating an example training method 300 for generating a machine learning classifier 230 using the imaging module 104 according to some embodiments. The imaging module 104 can implement supervised, unsupervised, and / or semi-supervised (e.g., reinforcement-based) machine learning-based classification models 240. The method 300 shown in FIG. 3 is an example of a supervised learning method, and variations of this example training method are described below, but other training methods can be implemented by analogy to train unsupervised and / or semi-supervised machine learning (predictive) models. The method 300 can be performed by processing logic, which may include hardware (e.g., circuits, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It should be understood that not all steps may be required to practice the disclosure provided herein. Furthermore, as will be appreciated by one of ordinary skill in the art, some of the steps can be performed simultaneously or in a different order than that shown in FIG. 3.

[0024] The method 300 will be described with reference to Figures 1 and 2. However, the method 300 is not limited to the aspects of these figures.

[0025] At 310, the imaging module 104 determines (e.g., accesses, receives, acquires, etc.) imaging data. The imaging data can include one or more data sets, each data set relating to a receptacle type, a receptacle condition, a receptacle capacity, a fluid type, a fluid behavior, and / or the like.

[0026] At 320, the imaging module 104 generates a training data set and a test data set. According to some aspects, the training data set and the test data set may be generated by indicating receptacle type, receptacle status, receptacle volume, fluid type, fluid behavior, and / or the like. According to some aspects, the training data set and the test data set may be generated by randomly assigning receptacle type, receptacle status, receptacle volume, fluid type, fluid behavior, and / or the like to either the training data set or the test data set. According to some aspects, the assignment of imaging data as training samples or test samples may not be completely random. According to some aspects, only labeled reference data for a particular feature extracted from a particular imaging data (e.g., a depiction of a clear cup with ice) may be used to generate the training data set and the test data set. According to some aspects, a majority of the labeled reference data extracted from the imaging data may be used to generate the training data set. For example, 75% of the labeled reference data for determining receptacle type, receptacle condition, receptacle capacity, fluid type, fluid behavior, and / or the like extracted from the imaging data may be used to generate a training data set and 25% may be used to generate a test data set. Any method or technique may be used to create the training and test data sets.

[0027] At 330, the imaging module 104 determines (e.g., extracts, selects, etc.) one or more features that can be used, for example, by a classifier (e.g., a software model, a classification layer of a neural network, etc.) to label features extracted from the various imaging data. The one or more features may include an indication of receptacle type, receptacle condition, receptacle volume, fluid type, fluid behavior, and / or the like. According to some aspects, the imaging module 104 can determine a set of training reference features from a training data set. The features of the imaging data can be determined in any manner.

[0028] At 340, the imaging module 104 trains one or more machine learning models, for example, using the one or more features. According to some aspects, the machine learning models can be trained with supervised learning. According to some aspects, other machine learning techniques can also be employed, including unsupervised learning and semi-supervised learning. The machine learning models trained at 340 can be selected based on different criteria (e.g., how close the predicted receptacle type, receptacle condition, receptacle capacity, fluid type, fluid behavior, and / or the like is to the actual receptacle type, receptacle condition, receptacle capacity, fluid type, fluid behavior, and / or the like) and / or data available in the training dataset. For example, machine learning classifiers may suffer from bias to varying degrees. According to some aspects, multiple machine learning models can be trained.

[0029] At 350, the imaging module 104 optimizes, improves, and / or cross-validates the trained machine learning model. For example, the data in the training data set and / or the test data set may be updated and / or modified to include more labeled data indicative of various receptacle types, receptacle conditions, receptacle volumes, fluid types, fluid behaviors, and / or the like.

[0030] At 360, the imaging module 104 selects one or more machine learning models to build a predictive model (e.g., a machine learning classifier, a prediction engine, etc.). The predictive model can be evaluated using a test dataset.

[0031] At 370, the imaging module 104 executes the predictive model to analyze the test data set and generate classification values ​​and / or prediction values.

[0032] At 380, the imaging module 104 evaluates the classification values ​​and / or prediction values ​​output by the predictive model to determine whether such values ​​achieved a desired level of accuracy. The performance of a predictive model can be evaluated in a number of ways based on the number of true positive, false positive, true negative, and / or false negative classifications of the plurality of data points dictated by the predictive model. For example, false positives of a predictive model may refer to the number of times the predictive model incorrectly predicted and / or determined a receptacle type, receptacle condition, receptacle capacity, fluid type, fluid behavior, and / or the like. On the other hand, a false negative of a predictive model may refer to the number of times that the machine learning model incorrectly predicts and / or determines a receptacle type, receptacle status, receptacle capacity, fluid type, fluid behavior, and / or the like, when in fact the predicted and / or determined receptacle type, receptacle status, receptacle capacity, fluid type, fluid behavior, and / or the like matches the actual receptacle type, receptacle status, receptacle capacity, fluid type, fluid behavior, and / or the like. True negatives and true positives may refer to the number of times that the predictive model correctly predicts and / or determines a receptacle type, receptacle status, receptacle capacity, fluid type, fluid behavior, and / or the like. Related to these measurements are the concepts of recall and precision. In general, recall refers to the ratio of true positives to the sum of true positives and false negatives, and quantifies the sensitivity of a predictive model. Similarly, accuracy refers to the proportion of true positives as the sum of true positives and false positives.

[0033] At 390, the imaging module 104 outputs the predictive model (and / or an output of the predictive model). For example, the imaging module 104 may output the predictive model upon reaching such a desired level of accuracy. The output of the predictive model may terminate the training phase.

[0034] According to some embodiments, if the desired level of accuracy is not reached, at 390, the imaging module 104 may perform a next iteration of the training method 300 starting at 310, including variations such as considering a larger collection of imaging data.

[0035] Returning to FIG. 1 , the output of the imaging module 104, e.g., a determination of receptacle type, receptacle status, receptacle capacity, fluid type, fluid behavior, and / or the like, may be provided to a fluid control module 110 of the computing device 103. For example, the fluid control module 110 may receive an indication from the imaging module 104 that the receptacle 109 is under the beverage dispensing nozzle 106. According to some aspects, the fluid dispenser 101 may be in an inactive state when idle or may transition to an inactive state, e.g., to conserve energy and / or resources. Once the fluid control module 110 receives an indication from the imaging module 104 that the receptacle 109 is under the beverage dispensing nozzle 106, the fluid control module 110 may send a signal to the fluid dispenser 101 to transition the fluid dispenser 101 from an inactive state to an active state. For example, in an active state, the fluid dispenser 101 may begin rapidly cooling and / or heating the fluid within the fluid dispenser 101 in preparation for consumption.

[0036] According to some aspects, the imaging module 104 can determine the state of the receptacle 109. The imaging module 104 may determine from the imaging data that an image (e.g., a frame of video, etc.) indicates that the receptacle 109 is in an empty fluid state (e.g., contains no fluid, contains only ice, etc.). The imaging module 104 can send an indication that the receptacle 109 is in an empty fluid state to the fluid control module 110. The fluid control module 110 can send one or more signals (e.g., transmissions, requests, data, etc.) that control the operation of the dispensing unit 102 to cause the beverage dispensing nozzle 106 to dispense fluid. The imaging module 104 may continue to monitor the imaging data (from the camera 105) for an indication that the receptacle 109 is in a full fluid state. Once the imaging module 104 has determined that the receptacle 109 is in a full fluid state (or reaches a full fluid state within a certain time frame, etc.), the imaging module 104 can send an indication that the receptacle 109 is in a full fluid state to the fluid control module 110. The fluid control module 110 can send one or more signals (e.g., transmissions, requests, data, etc.) that control the operation of the dispensing unit 102 to stop dispensing fluid from the beverage dispensing nozzle 106.

[0037] According to some embodiments, the imaging module 104 may determine, for example, a fill level threshold for the fill line 108 of the receptacle 109 (e.g., an indication of available fluid volume, etc.). From the imaging data, the imaging module 104 may determine an image (e.g., a frame of video, etc.) that indicates that the amount of fluid in the receptacle 109 does not meet the fill level threshold. The imaging module 104 may transmit an indication that the amount of fluid in the receptacle 109 does not meet the fill level threshold to the fluid control module 110. The fluid control module 110 may transmit one or more signals (e.g., transmissions, requests, data, etc.) that control the operation of the dispensing unit 102 to cause the beverage dispensing nozzle 106 to dispense fluid. The imaging module 104 may continue to monitor the imaging data (from the camera 105) for an indication that the amount of fluid in the receptacle 109 meets the fill level threshold. Once the imaging module 104 has determined that the amount of fluid in the receptacle 109 meets the fill level threshold, the imaging module 104 may send an indication to the fluid control module 110 that the amount of fluid in the receptacle 109 meets (or is about to meet) the fill level threshold. The fluid control module 110 may send one or more signals (e.g., transmissions, requests, data, etc.) that control the operation of the dispensing unit 102 to stop dispensing fluid from the beverage dispensing nozzle 106.

[0038] 4 illustrates a flow diagram of an example method 400 describing dispensing a fluid into a receptacle, according to some embodiments. Method 400 can be performed by processing logic, which may include hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It should be understood that not all steps may be required to practice the disclosure provided herein. Furthermore, one of ordinary skill in the art will appreciate that some of the steps may be performed simultaneously or in a different order than that shown in FIG. 4.

[0039] The method 400 will be described with reference to Figures 1-2. However, the method 400 is not limited to the embodiments shown in these figures.

[0040] A computer-based system (such as, for example, system 100) can assist in the automated dispensing of fluid into a receptacle based on imaging data collected by a camera positioned near a beverage dispensing nozzle of a fluid dispenser.

[0041] At 410, the system 100 (e.g., computing device 103, etc.) receives first imaging data. The system 100 may receive the first imaging data from a camera and / or the like located / positioned proximate to a nozzle of a fluid dispenser (e.g., a beverage dispenser, a water dispenser, a fountain drink machine, etc.). The first imaging data may include video and / or still images. The first imaging data may indicate a receptacle. For example, the first imaging data may include an image of a receptacle (e.g., a cup, a bottle, a can, a bowl, a box, etc.) located / positioned below the nozzle of the fluid dispenser.

[0042] At 420, system 100 determines classification information for the receptacle. For example, a predictive model (and / or prediction engine) of the computer-based system may be configured to determine the classification information for the receptacle. For example, determining the classification information for the receptacle may be based on image recognition and / or object recognition applied to the first imaging data. Image recognition and / or object recognition may be used to determine an empty state of the receptacle (e.g., an empty fluid state, etc.) or a full state of the receptacle (e.g., a full fluid state, etc.).

[0043] At 430, the system 100 initiates dispensing of the fluid into the receptacle. The computer-based system may initiate dispensing of the fluid into the receptacle based on the image of the first imaging data and the classification information indicating that the receptacle is empty. According to some aspects, a predictive model may be configured to determine that the image of the first imaging data indicates that the receptacle is empty. For example, the system 100 may input the first imaging data into the predictive model. The system 100 may execute the predictive model based on the first imaging data. The system 100 may receive the classification information of the receptacle and / or an indication that the receptacle is empty based on the execution of the predictive model. Initiating dispensing of the fluid into the receptacle may include, for example, sending a request to the dispensing device to initiate dispensing of the fluid into the receptacle. The dispensing device may be configured to dispense a plurality of fluids.

[0044] At 440, the system 100 receives second imaging data. The system 100 can receive the second imaging data from a camera located / positioned proximate to the nozzle of the fluid dispenser, and / or the like. The second imaging data can be indicative of the receptacle. The first imaging data and the second imaging data can be part of a video stream captured by a camera located / positioned proximate to the nozzle of the fluid dispenser, and / or the like.

[0045] At 450, the system 100 stops dispensing fluid into the receptacle. The system 100 may stop dispensing fluid into the receptacle based on the image of the second imaging data and the classification information indicating that the receptacle is in a full state. According to some aspects, the predictive model may be configured to determine that the image of the second imaging data indicates that the receptacle is in a full state. For example, the system 100 may input the imaging data into the predictive model. The system 100 may execute the predictive model based on the imaging data. The system 100 may receive the classification information of the receptacle and / or an indication that the receptacle is in a full state based on the execution of the predictive model. Stopping the dispensing of fluid into the receptacle may include, for example, sending a request to the dispensing device to stop dispensing fluid into the receptacle. According to some aspects, stopping the dispensing of fluid into the receptacle may transition the dispensing device to an inactive state.

[0046] 5 illustrates a flow diagram of an example method 500 describing dispensing a fluid into a receptacle, according to some embodiments. Method 500 can be performed by processing logic, which may include hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. It should be understood that not all steps may be required to practice the disclosure provided herein. Furthermore, one of ordinary skill in the art will appreciate that some of the steps may be performed simultaneously or in a different order than that shown in FIG. 5.

[0047] The method 500 will be described with reference to Figures 1-2. However, the method 500 is not limited to the embodiments shown in these figures.

[0048] A computer-based system (such as, for example, system 100) can assist in the automated dispensing of fluid into a receptacle based on imaging data collected by a camera positioned near a beverage dispensing nozzle of a fluid dispenser.

[0049] At 510, the system 100 (e.g., computing device 103, etc.) receives imaging data. The system 100 may receive imaging data from a camera and / or the like located / positioned proximate to a nozzle of a fluid dispenser (e.g., a beverage dispenser, a water dispenser, a fountain drink machine, etc.). The imaging data may include video and / or still images. The imaging data may indicate a receptacle. For example, the imaging data may include an image of a receptacle (e.g., a cup, a bottle, a can, a bowl, a box, etc.) located / positioned below the nozzle of the fluid dispenser.

[0050] At 420, system 100 determines a fill level threshold for the receptacle. For example, a predictive model (and / or a predictive engine) of the computer-based system may be configured to determine the fill level threshold for the receptacle. For example, determining the fill level threshold for the receptacle may include determining a type of the receptacle based on object recognition. Based on the type of the receptacle, fill level threshold classification information may be determined. Based on the fill level threshold classification information, the fill level threshold for the receptacle may be determined.

[0051] At 530, the system 100 initiates dispensing of the fluid into the receptacle. The computer-based system may initiate dispensing of the fluid into the receptacle based on a first image of the imaging data indicating that the amount of fluid in the receptacle does not meet the fill level threshold. According to some aspects, the predictive model may be configured to determine that the first image indicates that the amount of fluid in the receptacle does not meet (e.g., is below) the fill level threshold. For example, the system 100 may input the imaging data into the predictive model. The system 100 may execute the predictive model based on the imaging data. The system 100 may receive an indication that the first image indicates that the amount of fluid in the receptacle is below the fill level threshold based on the execution of the predictive model. Initiating dispensing of the fluid into the receptacle may include, for example, sending a request to the dispensing device to initiate dispensing of the fluid into the receptacle. The dispensing device may be configured to dispense a plurality of fluids.

[0052] At 540, the system 100 stops dispensing fluid into the receptacle. The computer-based system may stop dispensing fluid into the receptacle based on a second image of the imaging data indicating that the amount of fluid in the receptacle meets the fill level threshold. According to some aspects, the predictive model may be configured to determine that the second image indicates that the amount of fluid in the receptacle meets (e.g., equals, exceeds, etc.) the fill level threshold. For example, the system 100 may input the imaging data into the predictive model. The system 100 may execute the predictive model based on the imaging data. The system 100 may receive an indication that the second image indicates that the amount of fluid in the receptacle equals the fill level threshold based on the execution of the predictive model. Stopping the dispensing of fluid into the receptacle may include, for example, sending a request to the dispensing device to stop dispensing fluid into the receptacle. According to some aspects, stopping the dispensing of fluid into the receptacle may transition the dispensing device to an inactive state.

[0053] FIG. 6 is an exemplary computer system useful for implementing various embodiments. Various embodiments can be implemented using one or more known computer systems, such as, for example, computer system 600 shown in FIG. 6. One or more computer systems 600 can be used to implement, for example, any of the embodiments discussed herein, as well as combinations and subcombinations thereof. According to some aspects, computing device 103 of FIG. 1 (and / or other devices / components described herein) can be implemented using computer system 600. According to some aspects, computer system 600 can be used to implement methods 300, 400, and 500.

[0054] Computer system 600 may include one or more processors (also referred to as central processing units, or CPUs), such as processor 604. Processor 604 may be connected to a communications infrastructure, or bus 606.

[0055] The computer system 600 also includes user input / output device(s) 602 , such as a monitor, keyboard, pointing device, etc., through which these devices can communicate with a communications infrastructure, or bus 606 .

[0056] One or more of the processors 604 may be a graphics processing unit (GPU). In one embodiment, a GPU may be a processor that is a special-purpose electronic circuit designed to handle mathematically intensive applications. GPUs may have a parallel structure that is efficient for parallel processing of large blocks of data, such as the mathematically intensive data common in computer graphics applications, images, videos, and the like.

[0057] Additionally, computer system 600 may also include a main, or primary, memory 608, such as a random access memory (RAM). The main memory 608 may include one or more levels of cache. The main memory 608 may store therein control logic (i.e., computer software) and / or data.

[0058] Additionally, computer system 600 may also include one or more secondary storage devices, or memories, 610. Secondary memory 610 may include, for example, a hard disk drive 612 and / or a removable storage device, or drive 614. Removable storage drive 614 may be a floppy disk drive, a magnetic tape drive, a compact disk drive, an optical storage device, a tape backup device, and / or other storage devices / drives.

[0059] The removable storage drive 614 can interact with a removable storage device 618. The removable storage device 618 can include a computer usable or readable storage device on which computer software (control logic) and / or data is stored. The removable storage device 618 can be a floppy disk, magnetic tape, compact disk, DVD, optical storage disk, other computer data storage device, and the like. The removable storage drive 614 can read from and / or write to the removable storage device 618.

[0060] Secondary memory 610 may include other means, devices, components, equipment, and / or other approaches by which computer programs and / or other instructions and / or data may be accessed by computer system 600. Such means, devices, components, equipment, and / or other approaches may include, for example, removable storage 622 and interfaces 620. Examples of removable storage 622 and interfaces 620 may include program cartridges and cartridge interfaces (such as those found in video game devices), removable memory chips (such as EPROMs or PROMs) and associated sockets, memory sticks and USB ports, memory cards and associated memory card slots, and / or other removable storage devices and associated interfaces.

[0061] Additionally, computer system 600 may include a communications or network interface 624. Communications interface 624 may enable computer system 600 to communicate and interact with any combination of external devices, external networks, external entities, etc. (individually and collectively referred to by reference numeral 628). For example, communications interface 624 may enable computer system 600 to communicate with external or remote devices 628 via communications path 626, which may be wired and / or wireless (or a combination thereof), and may further include any combination of a LAN, a WAN, the Internet, etc. Control logic and / or data may be transmitted to and from computer system 600 via communications path 626.

[0062] Further, computer system 600 may be any one or any combination of a personal digital assistant (PDA), a desktop workstation, a laptop or notebook computer, a netbook, a tablet, a smartphone, a smart watch or other wearable, an appliance, part of the Internet of Things, and / or an embedded system, to name a few non-limiting examples.

[0063] The computer system 600 may be a client or a server that accesses or hosts any application and / or data through any delivery method, including, but not limited to, remote or distributed cloud computing solutions. The service may be provided via local or on-premise software (an "on-premise" cloud-based solution), an "as-a-service" model (e.g., content as a service (CaaS), digital content as a service (DCaaS), software as a service (SaaS), managed software as a service (MSaaS), platform as a service (PaaS), desktop as a service (DaaS), framework as a service (FaaS), backend as a service (BaaS), mobile backend as a service (MBaaS), infrastructure as a service (IaaS), etc.), and / or a hybrid model including any combination of the foregoing examples or other service or delivery paradigms.

[0064] Data structures, file formats, and schemas applicable in computer system 600 may be derived from standards, including, but not limited to, JavaScript Object Notation (JSON), Extensible Markup Language (XML), Yet Another Markup Language (YAML), Extensible Hypertext Markup Language (XHTML), Wireless Markup Language (WML), MessagePack, XML User Interface Language (XUL), or other functionally similar representations, alone or in combination, or proprietary data structures, formats, and / or schemas may be used, alone or in combination with known or open standards.

[0065] In some embodiments, a tangible, transitory apparatus or article of manufacture comprising a tangible, non-transitory computer usable or readable medium having control logic (software) stored thereon may also be referred to herein as a computer program product or program storage device, including, but not limited to, a tangible article of manufacture embodying computer system 600, main memory 608, secondary memory 610, and removable storage devices 618, 622, as well as any combination of the foregoing. Such control logic, when executed by one or more data processing devices (such as computer system 600), can cause such data processing devices to operate as described herein.

[0066] Based on the teachings contained in this disclosure, it will be apparent to one of ordinary skill in the art how to make and use embodiments of the present disclosure using data processing devices, computer systems, and / or computer architectures other than those shown in Figure 6. In particular, embodiments may operate using software, hardware, and / or operating system implementations other than those described herein.

[0067] It is understood that the "Description of Embodiments" section, and any other sections, are not intended to be used to interpret the scope of the claims. The other sections may describe one or more, but not all, example embodiments contemplated by the inventors, and are not intended to limit the scope of the disclosure and the appended claims in any way.

[0068] Additionally and / or alternatively, the present disclosure describes exemplary embodiments for exemplary fields and applications, but it should be understood that the present disclosure is not limited thereto. Other embodiments and variations thereof are contemplated and are within the scope and spirit of the present disclosure. For example, without limiting the generality of this paragraph, the embodiments are not limited to the software, hardware, firmware, and / or entities illustrated in the drawings and / or described herein. Moreover, the embodiments (whether or not explicitly described herein) have significant utility in fields and applications beyond the examples described herein.

[0069] One or more portions of the above implementations may include software. Software is a general term connoting specific functions and relationships thereof. The boundaries of these functional building blocks have been arbitrarily defined herein for convenience of description. Alternative boundaries may be defined to the extent that the specific functions and relationships (or their equivalents) are appropriately performed. Alternative embodiments may also implement functional blocks, steps, operations, methods, etc. using orders different from those described herein.

[0070] References such as "one embodiment," "an embodiment," "an exemplary embodiment," and the like indicate that the described embodiment may include a particular feature, structure, or characteristic, but that not all embodiments necessarily include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular property, structure, or characteristic is described in connection with one embodiment, it is within the knowledge of one skilled in the art to incorporate such feature, structure, or characteristic in other embodiments, regardless of whether it is explicitly mentioned or described herein. In addition, some embodiments may be described using the expressions "coupled" and "connected," along with derivatives thereof. These terms are not necessarily intended as synonyms for each other. For example, some embodiments may be described using the terms "connected" and / or "coupled" to indicate that two or more elements are in direct physical or electrical contact with each other. However, the term "coupled" may also mean that two or more elements are not in direct contact with each other but still cooperate or interact with each other.

[0071] The breadth and scope of the present disclosure should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

Claims

1. 1. A computer-implemented method for dispensing a fluid into a receptacle, the method comprising: receiving first imaging data, the first imaging data indicating a receptacle; determining classification information for the receptacle based on the first imaging data, the classification information classifying an empty state of the receptacle and a full state of the receptacle; initiating dispensing of fluid into the receptacle based on an image of the first imaging data and the classification information indicating that the receptacle is in the empty state; and receiving second imaging data, the second imaging data indicating the receptacle; and and ceasing dispensing of the fluid into the receptacle based on an image of the second imaging data and the classification information indicating that the receptacle is in the full state.

2. The method of claim 1 , wherein determining the classification information of the receptacle is based on at least one of object recognition or image recognition.

3. initiating the dispensing of the fluid into the receptacle, said step further comprising: The method of claim 1 , comprising sending a request to a dispensing device to initiate dispensing of the fluid into the receptacle, the dispensing device being configured to dispense multiple fluids.

4. ceasing the delivery of the fluid to the receptacle, said step further comprising: The method of claim 1 , comprising sending a request to a dispensing device to stop dispensing the fluid into the receptacle, the dispensing device being configured to dispense multiple fluids.

5. Furthermore, The method of claim 4 , comprising transitioning the ejection device to an inactive state in response to ceasing ejection of the fluid into the receptacle.

6. Furthermore, inputting the first imaging data into a predictive model; executing the predictive model based on the first imaging data; and and receiving an indication that the receptacle is in the empty state based on the classification information of the receptacle.

7. Furthermore, inputting the second imaging data into a predictive model; and executing the predictive model based on the second imaging data; and and receiving an indication that the receptacle is in the full state based on the classification information of the receptacle.

8. A non-transitory computer-readable medium storing instructions that, when executed by at least one computing device, cause the at least one computing device to perform an operation, the operation comprising: receiving first imaging data, the first imaging data indicating a receptacle; determining classification information for the receptacle based on the first imaging data, the classification information classifying an empty state of the receptacle and a full state of the receptacle; initiating dispensing of fluid into the receptacle based on an image of the first imaging data and the classification information indicating that the receptacle is in the empty state; and receiving second imaging data, the second imaging data indicating the receptacle; and and ceasing dispensing of the fluid into the receptacle based on an image of the second imaging data and the classification information indicating that the receptacle is in the full state.

9. 10. The non-transitory computer-readable medium of claim 8, wherein determining the classification information of the receptacle is based on at least one of object recognition or image recognition.

10. initiating the dispensing of the fluid into the receptacle, said step further comprising:

10. The non-transitory computer readable medium of claim 8, comprising sending a request to a dispensing device to initiate dispensing of the fluid into the receptacle, the dispensing device configured to dispense multiple fluids.

11. ceasing the delivery of the fluid to the receptacle, said step further comprising:

10. The non-transitory computer-readable medium of claim 8, further comprising: sending a request to a dispensing device to stop dispensing the fluid into the receptacle, the dispensing device being configured to dispense multiple fluids.

12. Furthermore, The non-transitory computer readable medium of claim 11 , comprising transitioning the ejection device to an inactive state in response to ceasing ejection of the fluid into the receptacle.

13. Furthermore, inputting the first imaging data into a predictive model; executing the predictive model based on the first imaging data; and and receiving an indication that the receptacle is in the empty state based on the classification information of the receptacle.

14. Furthermore, inputting the second imaging data into a predictive model; and executing the predictive model based on the second imaging data; and and receiving an indication that the receptacle is in the full state based on the classification information of the receptacle.

15. 1. A system for dispensing a fluid into a receptacle, comprising: Memory, at least one processor coupled to the memory, the at least one processor comprising: receiving first imaging data, the first imaging data indicating the receptacle; determining classification information for the receptacle based on the first imaging data, the classification information classifying an empty state of the receptacle and a full state of the receptacle; initiating dispensing of fluid into the receptacle based on an image of the first imaging data and the classification information indicating that the receptacle is in the empty state; and receiving second imaging data, the second imaging data indicating the receptacle; and and stopping the dispensing of the fluid into the receptacle based on an image of the second imaging data and the classification information indicating that the receptacle is in the full state.

16. The system of claim 15 , wherein to determine the classification information of the receptacle, the at least one processor is further configured to determine the classification information of the receptacle based on object recognition.

17. To initiate dispensing of the fluid into the receptacle, the at least one processor further comprises: The system of claim 15 , configured to perform the following: sending a request to a dispensing device to initiate dispensing of the fluid into the receptacle, the dispensing device configured to dispense multiple fluids.

18. To stop dispensing the fluid into the receptacle, the at least one processor further comprises: The system of claim 15 , further configured to: send a request to a dispensing device to stop dispensing the fluid into the receptacle, the dispensing device being configured to dispense multiple fluids.

19. The at least one processor further comprises: inputting the first imaging data into a predictive model; executing the predictive model based on the first imaging data; and The system of claim 15 , further configured to: receive an indication that the receptacle is in the empty state based on the classification information of the receptacle.

20. The at least one processor further comprises: inputting the second imaging data into a predictive model; and executing the predictive model based on the second imaging data; and The system of claim 15 , further configured to perform receiving an indication that the receptacle is in the full state based on classification information of the receptacle.

Images