Interactive vending machine

Abstract

To provide a method for presenting merchandise information in an automatic vending machine.SOLUTION: A method of presenting product information at a vending machine may include detecting voice information from a consumer and converting the voice information into a text string. The method may include identifying a keyword in the text string and determining a product associated with the keyword from a product database. The method may include returning a list of products corresponding to the keyword.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] FIELD OF THE INVENTION The embodiments described herein relate generally to interactive vending machines, and more particularly to vending machines that can guide a consumer through available products by recognizing the consumer's spoken words and recommending products to the consumer. [Background technology]

[0002] Vending machines typically require the consumer to pay, select an available item, and wait for the item to be dispensed. Some vending machines allow the consumer to select an item by pressing a button that displays an image of the desired item. Some vending machines include a storage compartment visible from the exterior of the machine so that the consumer can view the available items. The consumer may then enter a code into a keypad to dispense the desired item. Such vending machines may not assist the consumer in selecting an item and may provide limited or no information about the available items. Summary of the Invention

[0003] Some embodiments described herein relate to a method of presenting product information in a vending machine, the method including detecting voice information from a consumer, converting the voice information into a text string, identifying keywords in the text string, determining one or more products associated with the keywords from a product database, and returning a list of one or more products associated with the keywords.

[0004] In any of the various embodiments described herein, the method of presenting product information may further include detecting second audio information from the consumer, converting the second audio information into a second text string, identifying a second keyword in the second text string, determining one or more products from the list of one or more products that correspond to the second keyword, and returning a revised list of one or more products that correspond to both the keyword and the second keyword.

[0005] In any of the various embodiments described herein, detecting audio information from the consumer may be performed by a microphone in the vending machine.

[0006] In any of the various embodiments described herein, the method of presenting product information may further include transmitting the audio information to a remote computer prior to converting the audio information to a text string, the converting the audio information to the text string occurring at the remote computer.

[0007] In any of the various embodiments described herein, the keyword may be a brand name.

[0008] In any of the various embodiments described herein, the keyword may be flavor.

[0009] In any of the various embodiments described herein, a keyword may be a component.

[0010] In any of the various embodiments described herein, the method of presenting product information may further include playing a response by the vending machine in response to the audio information. In some embodiments, playing a response may include playing a response randomly from a list of pre-recorded responses.

[0011] In any of the various embodiments described herein, the method may further include identifying a command in the text string and performing an action by the vending machine based on the command. In some embodiments, the command may include adding or removing an item from an electronic shopping cart.

[0012] Some embodiments described herein relate to a method of making product recommendations to a consumer by a vending machine, the method including receiving location information at the vending machine, receiving user information at the vending machine, determining one or more tags corresponding to the location information and the user information, identifying products in a product database associated with the one or more tags, and making the product recommendations based on the one or more tags.

[0013] In any of the various embodiments described herein, receiving user information may include receiving biometric information from the user's portable electronic device.

[0014] In any of the various embodiments described herein, the location information may include the time and temperature at the location of the vending machine.

[0015] In any of the various embodiments described herein, the user information may include the user's emotions as determined by a camera in the vending machine.

[0016] In any of the various embodiments described herein, the user information may include subject demographic information determined by a camera in the vending machine.

[0017] Some embodiments described herein relate to a method for tracking consumer attraction by a vending machine having a camera, the method including detecting a consumer within a field of view of the vending machine camera, determining the orientation of the consumer, attracting the consumer when the consumer is facing the vending machine, detecting speech of the consumer attracted to the vending machine, and receiving a product selection from the consumer by detecting the speech of the consumer.

[0018] In any of the various embodiments described herein, determining the orientation of the consumer may include detecting the eyes of the consumer.

[0019] In any of the various embodiments described herein, the method may further include detecting a second consumer within the field of view of the camera and applying noise cancellation to the speech of the second consumer when the consumer is attracted to the vending machine.

[0020] In any of the various embodiments described herein, detecting the consumer's speech may include tracking the consumer's lips. [Brief explanation of the drawings]

[0021] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate the present disclosure and, together with the description, serve to further explain the principles of the present disclosure and to enable those skilled in the art to make and use the present disclosure. [Figure 1] 1 illustrates a perspective view of a vending machine, according to one embodiment. [Figure 2] 1 illustrates a diagram of a vending machine showing internal components according to one embodiment. [Figure 3] 1 illustrates a graphical user interface for a vending machine according to one embodiment. [Figure 4] 1 illustrates a schematic diagram of components of a vending machine according to one embodiment. [Figure 5] 1 illustrates an exemplary method of operating a vending machine according to one embodiment. [Figure 6A] 1 illustrates an exemplary product selection method according to one embodiment. [Figure 6B] 1 illustrates another exemplary product selection method according to one embodiment. [Figure 7] 1 illustrates an exemplary method for responding to a consumer, according to one embodiment. [Figure 8] 1 illustrates an exemplary embodiment for controlling sales operations, according to one embodiment. [Figure 9] 1 shows a diagram of a vending machine recommending products to consumers. [Figure 10] 1 illustrates a method for determining product recommendations for consumers, according to one embodiment. [Figure 11] 1 illustrates a method for improving product recommendations, according to one embodiment. [Figure 12] 1 illustrates a top view of a vending machine showing consumer decisions made by the vending machine, according to one embodiment. [Figure 13] FIG. 1 illustrates a method for determining consumer attraction, according to one embodiment. [Figure 14] 1 illustrates a method for improving consumer speech detection, according to one embodiment. [Figure 15] 1 shows a schematic block diagram of an exemplary computer system in which embodiments may be implemented. DETAILED DESCRIPTION OF THE INVENTION

[0022] Reference will now be made in detail to representative embodiments, as illustrated in the accompanying drawings. It should be understood that the following description is not intended to limit the embodiments to a single preferred embodiment. On the contrary, the invention is intended to cover alternatives, modifications, and equivalents, which may be included within the spirit and scope of the embodiments as defined by the appended claims.

[0023] Vending machines generally provide little assistance to consumers in making product selections. To make a product selection, consumers must briefly review various products and make a decision. Consumers are not informed about the products available for purchase or able to filter products based on desired characteristics such as flavor, calorie content, or brand, among other criteria. This is particularly problematic when consumers are not familiar with one or more of the available products, which may result in consumers being discouraged from purchasing new products. Consumers may not thoroughly review all products or may prefer to select products quickly. As a result, consumers may not notice desirable products or may be dissatisfied with their selections. If consumers are dissatisfied with their experience, they may be unlikely to make additional purchases or return for future purchases. Therefore, a vending machine that allows consumers to view and narrow down available products based on consumer-defined criteria is desirable.

[0024] While some vending machines communicate with consumers, they may simply make the same recommendations to every consumer. Alternatively, vending machines may prompt users to make a series of selections to identify products to purchase. Making a series of selections can be time-consuming and tedious, and may not lead to accurate product recommendations if the selections are not related to the consumer's preferences. Therefore, a vending machine that makes product recommendations specific to the consumer and their preferences is desirable.

[0025] Furthermore, some vending machines may require a user to manually operate a touch panel or other input device to make product selections. Manually navigating a list of available products or selecting products by typing product codes can be inconvenient for consumers. This also increases the effort required for consumers to purchase products and can introduce the risk of user error when operating the vending machine. Consumers may prefer to view and select available products without touching the vending machine to simplify transactions and prevent the spread of pathogens. Therefore, a vending machine that allows consumers to conduct transactions by speaking naturally without touching the machine is desirable.

[0026] 1, vending machine 100 may include a housing 110. Housing 110 may be shaped as a cube, a rectangular prism, or a cylinder, among other shapes. In some embodiments, vending machine 100 may be configured to dispense packaged beverages, such as bottled or canned beverages. However, vending machine 100 may be used to dispense any of a variety of products, such as snacks, office supplies, medical supplies, and other items.

[0027] The vending machine 100 may include a user interface 120 for interacting with a consumer, such as providing instructions for operating the vending machine 100 and product information, among other information. The user interface 120 may include a display 122. The display 122 may be a liquid crystal display (LCD), a light emitting diode (LED) display, or an organic light emitting diode (OLED) display, among other things. The display 122 may be located on the front surface 102 of the housing 110.

[0028] In some embodiments, display 122 may display available items, product information, and selected items. In some embodiments, display 122 may be used to display images or videos, such as advertisements, or may display the time and / or weather to attract and entertain consumers. As a result, housing 110 may be opaque so that items stored within housing 110 are not visible to the consumer but are instead visible on display 122. In some embodiments, display 122 may be a touchscreen display, allowing consumers to provide user input by touching a portion of the touchscreen display. For example, a consumer may touch a portion of display 122 where items are displayed to select an item for purchase. In some embodiments, user interface 120 may include one or more actuators 124 (see, e.g., FIG. 4 ), such as buttons, levers, dials, switches, or the like, for navigating available items and making product selections. While a consumer can operate vending machine 100 through spoken language, in some embodiments, a consumer may instead select an item by touching a portion of display 122 or by operating actuator 124. The consumer can then touch the display 122 or operate the actuator 124 to complete the transaction and remove the item.

[0029] Vending machine 100 may include a microphone 117 for receiving audio information from a consumer and a speaker 113 for playing audio responses. Vending machine 100 may receive and analyze the consumer's speech via microphone 117 and may play responses via speaker 113 so that the vending experience becomes a conversational experience. In this manner, the consumer may perform at least a portion of the product selection and retrieval activity solely through speech, without having to manually provide user input to navigate, select, and pay for products.

[0030] The vending machine 100 may include a camera 115 for detecting a consumer 300 in the vicinity of the vending machine 100. In some embodiments, the camera 115 may further detect the eyes 310 of the consumer 300 to determine consumer engagement, and may additionally or alternatively detect the mouth 320 of the consumer 300 to assist with speech recognition, as described in further detail below.

[0031] In some embodiments, a payment processing unit 160 may be disposed on the housing 110 to receive a payment source from a consumer. The payment processing unit 160 may include a slot for receiving coins or bills, a card reader for reading the magnetic stripe or electronic chip of a payment card, such as a debit card, credit card, or gift card, a wireless antenna for accepting contactless payments, a scanner for scanning a code displayed on a mobile electronic device, and a transceiver for communicating with a mobile electronic device to accept mobile payments, such as Apple Pay or Google Pay, among others. In some embodiments, the vending machine 100 may not include a payment processing unit 160 and may instead identify the consumer via biometric information, such as facial recognition, thereby charging a corresponding account linked to the consumer's personal information. In this manner, the consumer need not provide a payment source at the time of the transaction, as described in further detail below.

[0032] As shown in FIG. 2 , the vending machine 100 may include a storage compartment 170 disposed within a housing 110 for storing one or more products 400 available for purchase. The storage compartment 170 may be maintained at ambient temperature or may be a temperature-controlled storage compartment 170. In some embodiments, the storage compartment 170 may be refrigerated. In such embodiments, the vending machine 100 may include a refrigeration unit 190 for maintaining the storage compartment 170 at a predetermined temperature. The refrigeration unit 190 may be a vapor compression refrigeration unit, a thermoelectric refrigeration unit, or a cold plate, among others. The storage compartment 170 may be insulated to maintain the storage compartment 170 at a particular temperature or temperature range. The housing 110 may have opaque walls to prevent consumers from seeing the storage compartment 170 and the products therein. The storage compartment 170 may be completely enclosed to prevent consumers from accessing the storage compartment 170. As a result, the temperature of the storage compartment 170 may be precisely maintained.

[0033] The vending machine 100 may include a dispensing mechanism 155 for dispensing the products 400 from the storage compartment 170 to a dispensing outlet 150 located on the housing 110. In some embodiments, the storage compartment 170 may be in communication with the dispensing outlet 150 via a chute 172. The dispensing mechanism 155 may include a screw drive, an articulating arm, a gravity-fed dispenser with a movable gate, or an automated movable basket, among others.

[0034] In some embodiments, the display 122 may be configured to display a graphic user interface, as shown in FIG. 3 . The graphic user interface 200 may display available products 210, such as by displaying product images 212, and may display product information 240, such as brand, flavor, size (weight / volume), cost, and nutritional information (e.g., calories or ingredients), among other product information. In some embodiments, the graphic user interface 200 may display a transcript of the conversation 220 between the vending machine 100 and the consumer, which may provide the consumer with confirmation that the vending machine 100 accurately interpreted the consumer's spoken words. The graphic user interface 200 may display an electronic shopping cart 230 showing the products the consumer has selected for purchase. The electronic shopping cart 230 may include, among other information, one or more of the name or image of each selected product 231, the quantity of each selected product, the cost of each selected product 232, and the total cost 233 of the products in the electronic shopping cart 230.

[0035] In some embodiments, the vending machine 100 may include a control unit 140 configured to control the operation of the vending machine 100, as shown in FIG. 4. The control unit 140 may be in communication with the vending machine's 100 camera 115 and microphone 117 to detect and receive audio information from the consumer, respectively. In some embodiments, the control unit 140 may receive user input via the actuator 124 and / or the display 122. The control unit 140 may also be in communication with the user interface 120 and may update the information displayed on the display 122 (e.g., displaying available items, selected items, and item information, among other information) based on the audio information received from the consumer; the control unit 140 may also be in communication with a speaker 113 to play responses. The control unit 140 may also be in communication with a transceiver 119 or other communication device for communicating with a computer, server, or cloud remote from the vending machine 100, referred to herein simply as a “remote computer.” The remote computer 450 may analyze and interpret audio information from the consumer received by the microphone 117. The control unit 140 may be in communication with a dispensing mechanism 155 for dispensing items selected by the consumer. Additionally, in some embodiments, the control unit 140 may be in communication with a payment processing unit 160 for accepting user payments and with a refrigeration unit 190 for maintaining the vending machine 100's storage compartment at a predetermined temperature.

[0036] An exemplary method of operating a vending machine 500 is shown in FIG. 5. The vending machine may identify a consumer through facial recognition 510. A camera on the vending machine may capture an image or video of the consumer, which may be analyzed by facial recognition software to determine the consumer's personal information based on known images or video of the consumer. Once the consumer is identified, a consumer account linked to the consumer's personal information may be accessed (520). The consumer account may contain consumer information, including, among other information, biographical information, purchase history, a list of favorite products, and payment information. The payment information may include a linked credit card or bank account, an electronic payment account (e.g., PayPal), or the consumer account may be a prepaid account. However, if the consumer does not already have an account or the vending machine is unable to identify the consumer, the consumer may be prompted to create an account (530). The consumer may create a consumer account by providing consumer information, as described above, including payment information, via the vending machine or a mobile electronic device. Once the consumer's account is accessed or created, the consumer may select a product for purchase (540). Once the consumer has selected all desired items for purchase, the consumer may complete the transaction and remove the items 550. The cost of the removed items may be charged to the consumer's account 560. In this manner, the consumer is not required to provide a form of payment by inserting coins or bills, swiping a credit card, scanning a code, etc. when the vending machine is used; instead, the cost of the removed items may be automatically charged to the consumer's account.

[0037] Some embodiments described herein relate to a method for selling products that includes natural language search. In this manner, a consumer can navigate through a sales operation by researching specific products and filtering available items. During operation, the consumer can navigate available products and make product selections by simply speaking to the vending machine. This can be particularly beneficial for visually impaired consumers who otherwise have difficulty seeing products in cabinets, entering payment information, and / or entering product codes on a keypad. Furthermore, because natural language can be used, the consumer does not need to learn and use specific command phrases to operate the vending machine. The vending machine 100 may include a microphone 117 for receiving voice information from the consumer. As used herein, the term "voice information" may refer to any spoken consumer language, such as a statement or question.

[0038] In some embodiments, the vending machine 100 may store or access a database of product information. The database of product information may include a list of products and one or more keywords associated with each product. The database may be stored locally on the vending machine 100 or on a remote computer. The keywords associated with each product may include, for example, brand (e.g., Pepsi), flavor (e.g., cherry, cola, lime, etc.), beverage type (e.g., carbonated, still, sparkling, soda, sports drink, etc.), ingredient information (e.g., sugar-free, caffeinated, gluten-free, vegan, organic), nutritional information (e.g., diet, low-calorie, etc.), or price, among other words a consumer may use to identify a particular product. In some embodiments, newly released products or products newly added to the vending machine 100 may be associated with the keyword "new." In one example, a bottle of Diet Pepsi may be associated with keywords including, but not limited to, Pepsi, cola, soda, carbonated, caffeinated, diet, and low-calorie. Thus, a search for products that are "diet" products will return Diet Pepsi among other products related to the keyword "diet."

[0039] As shown in FIG. 6A, a method for selecting products via natural language search 600 may include receiving audio information from a consumer 602. The consumer's spoken words may be recorded into an audio file, which may be analyzed and used to generate a text string based on the audio information 604. Software or programming for analyzing audio files and generating the text string is known in the art, such as dictation software. After generating the text string, the text string may be analyzed for the presence of keywords 606. A database of product information may be searched for products related to the keywords identified in the text string 608. A list of products related to the keywords in the text string may be returned 610. The list of products may be displayed to the consumer on a vending machine display, read by the vending machine to the consumer, or both.

[0040] While method 600 may be performed locally on vending machine 100, in some embodiments, one or more of the steps may be performed remotely on a remote computer, server, or cloud. In some embodiments, steps 604, 606, and 608 may be performed on a remote computer, server, or cloud in communication with vending machine 100, such as via wireless transceiver 119, such that remote computer 450 generates the text string, analyzes the text string, searches a product database, and transmits a list of products related to the keyword to vending machine 100. In this manner, vending machine 100 need not have extensive computing power. Vending machine 100 may receive voice information from a consumer, transmit the information to a remote computer, and receive a list of products from the remote computer.

[0041] In some embodiments, method 600 may further include narrowing the list of products, as shown in FIG. 6B. The vending machine may detect second audible information from the consumer 612. The second audible information may be converted to text 614 and analyzed for a second keyword 616, as described above. A database may be searched for products related to the second keyword 618. The list of products may be modified to include products having the first keyword and the second keyword. The narrowed list may be returned (620). Additional refinements may similarly be performed as desired by the consumer to continue narrowing the list of products.

[0042] For example, a consumer may say, "Show me products that contain caffeine." The audible query may be received by a microphone in the vending machine and converted into a text string, which may be analyzed for keywords such as "caffeine." A database of products may then be searched for products with the keyword "caffeine," such as carbonated soft drinks, energy drinks, and coffee-based beverages. A list of caffeinated beverages may be returned to the consumer. The consumer may make a selection based on the list of beverages. Alternatively, the consumer may wish to further narrow the list before making a selection. The consumer may also say, "I want a low-calorie beverage." The query may refine the list to show beverages that are associated with being caffeinated and low-calorie.

[0043] In some embodiments, the audio information may include multiple keywords. In analyzing the consumer's speech, the vending machine may further identify logical operators such as "and," "or," "no or not." For example, if a consumer requests to see diet and caffeinated beverages, the vending machine may analyze the resulting text string and search a product database for diet-related and caffeine-related products. Products may receive a score based on the number of tags, and the vending machine may return the product with the highest score (e.g., the product associated with the greatest number of tags). For example, a diet caffeine-free beverage may have a score of 1 for "diet," while a diet caffeine-free beverage may have a score of 2 for "diet" and "caffeine." The product with a score of 2 is returned to the consumer. In another example, if a consumer specifies three keywords but no products match all three keywords, rather than returning no results, the product with the highest score, such as a product that matches two of the keywords, is returned. However, in some embodiments, the vending machine may indicate that no matching products exist.

[0044] In some embodiments, products may be scored based on the percentage of product-associated tags that match. For example, if a user specifies Cherry Pepsi, a database is searched for the keywords "cherry" and "pepsi." The database may include Cherry Pepsi, which has the keywords "cherry" and "pepsi," and Diet Cherry Pepsi, which includes the keywords "cherry," "pepsi," and "diet." Based on the score of matched tags, both results would receive a score of 2. However, based on the percentage of matched tags, Cherry Pepsi would receive a score of 100% because both of its tags match, while Diet Cherry Pepsi would receive a score of 67% because two of its three tags match.

[0045] The vending machine may determine that the consumer wanted caffeine OR a diet product, in which case the vending machine may also return any diet product and any caffeinated product. The vending machine may also recognize "no caffeine," so that if the consumer says, "I want a product without caffeine," the vending machine may return products that do not have the keyword caffeine. Alternatively, rather than searching for products that do not contain the keyword "caffeine," "no caffeine" may be a keyword, and products may be associated with the keyword "no caffeine" or "caffeine-free."

[0046] In some embodiments, the vending machine 100 may be configured to respond to voice information provided by the consumer. In this manner, the vending machine 100 may provide a conversational experience with the consumer. An exemplary method 700 for a vending machine to interact with a consumer is shown in FIG. 7. The vending machine may receive voice information 710 from the consumer, such as via a microphone. The voice information may be analyzed (720). As described above, the voice information from the consumer may be converted into a text string. The text string may be analyzed, and the vending machine may search a response database containing pre-recorded responses (730). The vending machine may select a response from the database based on the voice information (740). The response may be played to the consumer (750), such as via a speaker.

[0047] For example, if a consumer asks, "Show me the lime flavored item," the text string may be analyzed to recognize the request "show me" and the keywords "lime" or "lime flavored." To reiterate and confirm the consumer's request, a database of responses containing the keyword "lime" may be searched, and the vending machine may respond by stating, for example, "Here's the lime flavored item."

[0048] In some embodiments, when a vending machine returns multiple items for a consumer's request, the vending machine may play prompts to allow the consumer to narrow the results or make a selection. For example, when a consumer asks to see "sugar-free" items, the vending machine may display a list of several sugar-free items as described above, or may play a response that includes a prompt to select a particular item, such as "Which sugar-free item would you like?"

[0049] In some embodiments, a response database may store one or more responses to be provided by the vending machine. The response database may store responses to common questions or utterances. For example, a consumer may frequently ask, "Show me the item that has...." The response played may be based on a command in the text string corresponding to the voice information and / or may be based on keywords in the text string. A computer may analyze the voice information to determine that the consumer has made a request (e.g., "Show me..." or "Which item has...?"). In response, the vending machine may consult a database of responses to requests, which may be identified by keywords in the text string.

[0050] In some embodiments, the response database may store multiple possible responses to a particular query. For example, in response to a consumer request to "show me a Pepsi," the database may include several possible responses to the command "show me," such as "sure," "yes," "let me help you," or "here's the one," among others. Responses may be specific to the keyword "Pepsi" and may include responses such as "here's the Pepsi" or "which Pepsi would you like?" If the database includes multiple possible responses to a particular utterance or query, the responses may be selected randomly to prevent the vending machine from repeating the same phrase multiple times and to better mimic natural conversation. Alternatively, the vending machine may play the possible responses in the order in which they are stored in the database, cycling through the available responses.

[0051] The response database may also store one or more greetings. A greeting may be played when the consumer is first detected by the camera. Alternatively, a greeting may be played when the consumer is identified by facial recognition. The vending machine may greet the consumer and may address the consumer by name if the consumer's name is known based on the consumer's account.

[0052] The response database may also store one or more responses to voice information that cannot be interpreted. If the volume of the voice information provided by the consumer is too low or there is too much background noise, the vending machine may not be able to interpret the voice information. Or, if the computer cannot recognize what the consumer is saying. For example, the response may say that the consumer did not hear and request clarification, such as "Sorry, could you repeat that?", "Can you say that again?", or "Sorry, I don't understand." A redirect response may be played to direct the consumer to a purchase. For example, "I didn't understand. Would you like to buy something?" or "I didn't hear you. What products would you like to see?"

[0053] A vending machine may execute commands 800 from a consumer, as shown in FIG. 8. The vending machine may detect commands in a text string 802 based on the audio information. The commands may include, among others, check out or buy (804), add an item (806), remove an item (808), and display a price or cost (810). When a command is identified in the text string, the vending machine may perform a corresponding action. If the command "check out" or "buy" (804) is identified in the text string, the vending machine may retrieve items in an electronic shopping cart and charge the consumer's account or payment method for the retrieved items (812). If the command "add" or "remove" is identified in the text string, the vending machine may add the selected items to the shopping cart (814) or remove the selected items from the shopping cart (816), respectively. The text string may also be analyzed for the presence of numbers, such as a request to add a certain number of items or a request to remove a certain number of items from the cart. For example, a consumer may say, "Add two Diet Pepsis to my shopping cart." The text string generated from the speech information may be analyzed to identify the command "add" and may also be analyzed to identify that "two" items should be added to the electronic shopping cart. If the command "price" or "cost" is detected, the price or cost of the item may be read or displayed (818). The vending machine may recognize other common commands and provide an appropriate response. In some embodiments, the vending machine may also play a response corresponding to the command. For example, if a consumer says, "Add item to cart," the vending machine may add the item to the electronic shopping cart and further play a response such as, "Item added to cart."

[0054] In some embodiments, vending machine 100 may be configured to provide other information to the consumer. Vending machine 100 may understand a request to display the time, weather, or temperature and may respond by displaying or reading the time, weather, or temperature, respectively. Vending machine 100 may also understand a request to display nutritional information for a product and may respond by displaying or audibly reading the nutritional information.

[0055] Some embodiments described herein relate to vending machines configured to provide product recommendations to consumers. To assist consumers in making purchases, the vending machine 100 may be configured to recommend products for purchase. This encourages the consumer to make a purchase and may save the consumer time that would otherwise be spent reviewing available products for purchase.

[0056] In some embodiments, the vending machine 100 may be configured to make product recommendations 920 based on received input information 910, as shown in FIG. 9 . The input information 910 may be analyzed using artificial intelligence to make the product recommendations 920. The input information 910 may include location information 911, such as the physical location of the vending machine. The location information 911 may be used to ascertain the date and time at the location 912, and the weather at the location 913. The product recommendations may additionally or alternatively be based at least in part on user information. The user information may include, among other information, the consumer's emotions 914, biometric or physiological information 915, demographic information such as the consumer's age 916 or gender 917, the consumer's favorite products 918, or purchase history 919. The input information 910 provided to the vending machine 100 is transmitted to a remote computer 450, which analyzes the input information and determines product recommendations, which may be transmitted back to the vending machine 100 and output by the vending machine 100 to the consumer. In this way, the vending machine 100 can utilize the remote computer 450 to determine product recommendations without having to have extensive computing resources.

[0057] In some embodiments, the vending machine 100 may include a geolocation unit 130, such as a Global Positioning System (GPS) unit, to determine the location of the vending machine 100. From the location of the vending machine 100, the date and time at that location may be readily ascertained. Additionally, the weather at that location may be ascertained. In some embodiments, the location, date, time, weather, or a combination thereof may be used in part to provide product recommendations.

[0058] The vending machine may store or have access to a product database that includes a list of products and one or more tags associated with each product. The tags may be used to categorize products. For example, a "kids" tag may be applied to products that are popular among children, a "morning" tag may be applied to products that are commonly consumed in the morning, and a "hydration" tag may be applied to hydration products. Tags may be generated and applied to products using machine learning, as described in more detail below. Alternatively, or additionally, tags may be entered by an operator of the vending machine 100. The product database may be stored locally on the vending machine 100 or on a remote computer 450.

[0059] The geolocation unit 130 may be used to determine a local time or a time period, such as morning, afternoon, or evening. In some embodiments, "morning" may be defined as a particular time period, such as 6:00 AM to 12:00 PM, and product recommendations may recommend products associated with the "morning" tag. For example, coffee-based beverages and juice-based beverages, such as orange juice, that are commonly consumed in the morning may be associated with the "morning" tag. If the local temperature is above a particular threshold, e.g., 80°F, a vending machine may recommend products with a "hot" tag, such as carbonated water or sports drinks, to provide a refreshing beverage.

[0060] In some embodiments, the vending machine 100 may include one or more cameras 115 configured to capture images or videos of the consumer. Computer vision techniques may be used to perform facial recognition of the consumer on the images or videos. Computer vision may also be used to identify demographic information about the consumer, such as the consumer's approximate age and / or gender. In some embodiments, the consumer's age and / or gender may be used to make product recommendations. For example, if the vending machine 100 determines that the consumer is a child, it may recommend products in the database that have a "child" or "kids" tag, such as chocolate milk or fruit juice.

[0061] In some embodiments, the vending machine 100 may use machine learning to analyze the purchasing patterns of consumers of different demographics. As the vending machine 100 is used over time, the vending machine 100 can track each purchase, including each consumer's approximate age and gender, as well as the type of product the consumer purchased. From this data, the vending machine 100 may determine which products are popular with consumers of different demographics. For example, children may purchase a variety of products, but if children frequently purchase chocolate milk over a period of time, the vending machine may tag the chocolate milk as "children" and become more likely to recommend chocolate milk to subsequent children who use the vending machine. In another example, over time, the vending machine 100 may determine that adult females frequently purchase sparkling water, which may be tagged as "female" and recommended more frequently to women. Machine learning can help recognize patterns in purchasing behavior.

[0062] In some embodiments, the vending machine 100 may determine the consumer's emotions. The vending machine 100 may include a camera 115 for capturing images or video that can be analyzed to perform facial recognition and / or gesture analysis. The vending machine 100 may detect whether the consumer is happy, sad, angry, or tired, among other emotions. For example, the vending machine 100 may detect whether the consumer is smiling or frowning, whether the consumer appears tired, such as if the consumer frequently makes gestures that include closing or rubbing their eyes, or whether the consumer is angry, such as if the consumer is frowning, among other emotions. The product database may include a list of products with tags that correspond to emotions. For example, a caffeinated beverage, such as an energy drink or coffee-based beverage, may have a "fatigue" tag to provide a caffeinated beverage to a tired consumer.

[0063] In some embodiments, machine learning may be used to track consumer emotions and what products the consumer purchases. For example, if a vending machine determines that consumers detected as smiling frequently purchase product A over a period of time and across multiple transactions, product A may be recommended to future consumers who are smiling. The product database may be updated to add a "happy" tag to product A. In another example, if consumers detected as frowning frequently purchase product B, product B may be recommended to future consumers who are frowning, and the product database may be updated accordingly.

[0064] In some embodiments, the vending machine 100 may receive biometric or physiological information from the consumer. The physiological information may be stored in the consumer's account. In some embodiments, to receive the biometric information, the vending machine 100 may be configured to communicate with the consumer's portable electronic device, such as a smartphone, smartwatch, tablet, etc. The biometric information from the portable electronic device may be transmitted to the vending machine 100. The biometric information may include information regarding the user's hydration level, sleep level, among other information.

[0065] For example, the portable electronic device may track a user's hydration level throughout the day, and if the consumer needs to consume more water to reach their hydration goal, the vending machine 100 may recommend products tagged with a "hydration" tag, such as water or sports drinks.

[0066] The vending machine 100 may also receive additional consumer information, such as from a consumer account, which may include age, gender, dietary restrictions, allergies, purchase history, or a list of favorite products. The consumer's account may include favorite products entered by the consumer and may include the consumer's purchasing history. The vending machine 100 may utilize the consumer's favorite products or purchase history when making product recommendations. The vending machine 100 may also ascertain the consumer's purchasing behavior and, therefore, utilize other information when making product recommendations. For example, if the purchase history indicates that the consumer frequently purchases a particular product on weekend mornings, the vending machine may be more likely to recommend that product when the consumer uses the vending machine on weekend mornings. Furthermore, if the consumer's purchase history indicates that the consumer frequently purchases cherry-flavored beverages, the vending machine may recommend a new cherry-flavored beverage. In this manner, purchase history and favorite products may suggest product recommendations, and they are not necessarily used solely to recommend products the consumer has previously purchased.

[0067] The vending machine 100 may provide product recommendations based on one or more of location, time, weather, demographic information (e.g., age and gender), emotions, purchase history preferences, and biometric information. Multiple factors may be used to generate product recommendations. In one example, the vending machine may determine that it is 8:00 AM and 90°F. The vending machine 100 may detect that the consumer is an adult male. The consumer may not provide biometric information. Thus, based on the collected information, the vending machine may search a product database for tags such as "morning," "hot," "male," and "adult." The database may have multiple products tagged with "morning," or the vending machine may search for products tagged with "male," and the list may be further narrowed by products tagged with "adult." In this manner, the vending machine may generate product recommendations that are particularly popular with adult males in the morning. Some factors may be given greater weight when making recommendations; thus, in some embodiments, "male" may be given a higher weight in recommendations than "morning." Additionally, if available, the consumer's purchase history may inform product recommendations.

[0068] A method for determining product recommendations based on input information is illustrated by method 1000 of FIG. 10 . The vending machine may identify location information 1010. The location information may be determined by a geolocation unit of the vending machine, and the location information may include weather information such as local time and temperature. The vending machine may sense consumer demographic information 1020, such as the consumer's approximate age and gender, such as via a camera on the vending machine. The vending machine may also receive biometric information from a portable electronic device 1030, such as a smartphone, smartwatch, or fitness tracker. The vending machine may also receive the consumer's purchase history and favorite products 1040, such as by accessing the consumer's account. A product database may be searched for products having tags associated with the collected input information 1050. Relative weights may be assigned to each factor, resulting in more weighting of some information in product recommendations. For example, if biometric information is provided by the user's portable electronic device, such information may be given a relatively higher weight for accuracy and personalized information. The vending machine may then make product recommendations based on the collected input information and the product database 1060.

[0069] In some embodiments, a vending machine may use machine learning to adjust product recommendations based on the acceptance rate of the product recommendations. As shown in FIG. 11 , a method 1100 of revising product recommendations may include receiving user information 1110 and making product recommendations based on the user information 1120, as described above with respect to FIG. 10 and method 1000. The vending machine may track which products are recommended and whether the consumer purchases the recommended products (1130). If the recommended products are not purchased, the vending machine may determine which products were purchased by the consumer (1140). The vending machine may update its recommendations (1150) based on whether the consumer accepted the product recommendation or whether the consumer selected another product. If the consumer purchased the recommended product, the recommended product may receive more weight in recommendation decisions. If the consumer did not accept the product recommendation, the vending machine may revise its product recommendations for future consumers. In this way, the product recommendations and the likelihood that the consumer will accept the product recommendation may improve over time.

[0070] Some embodiments described herein relate to determining consumer attraction at a vending machine, as shown in Figure 12. The vending machine 100 may include one or more cameras 115 having a field of view 1200 around at least a portion of the vending machine 100. In some embodiments, the one or more cameras 115 may be oriented facing a forward area of ​​the vending machine 100. In some embodiments, the camera 115 may be configured to detect the consumer's position relative to the vending machine 100, the consumer's orientation relative to the vending machine 100 (i.e., the direction the consumer is facing), and / or the consumer's path as they move within the field of view 1200. The vending machine 100 may take different actions depending on the consumer's position, orientation, and path.

[0071] An exemplary method 1300 for attracting a consumer is shown, for example, in FIG. 13 . The vending machine may detect the presence of a consumer near the vending machine 1310. The vending machine may further detect the orientation of the consumer 1320 relative to the vending machine. The vending machine may detect whether the consumer is facing the vending machine. If the consumer is not facing the vending machine, the vending machine may take no action (1340) and may not attempt to interact with the consumer. If the consumer is facing the vending machine, the vending machine may further determine the consumer's location (1330). If the consumer is outside a predetermined distance, the vending machine may attempt to attract the consumer (1360). For example, the vending machine may play a message inviting the consumer to approach the vending machine. If the consumer is within a predetermined distance, the vending machine may greet the consumer (1350) and attempt to initiate a transaction.

[0072] In some embodiments, the vending machine 100 may determine the orientation of a consumer by detecting the consumer's eyes, as shown in FIG. 12 . If one or both of the consumer's eyes are looking at the vending machine 100 as determined by the camera 115, the vending machine 100 may determine that the consumer is facing the vending machine 100, and the vending machine 100 may attempt to attract the consumer or initiate a transaction. For example, a first consumer 330 may be approaching the vending machine 100, and the camera 115 may detect the consumer's eyes 332 and determine that the first consumer 330 is facing the vending machine 100. If the vending machine 100 cannot detect the consumer's eyes 342, for example, if a second consumer 340 is standing facing away from the vending machine 100, the vending machine 100 may not attempt to attract the second consumer 340. A third consumer 350 may be far away from the vending machine 100, and camera 115 may detect the consumer's eyes 352 and attempt to attract the third consumer 350 to the vending machine 100. If a fourth consumer 360 walks by the vending machine 100 such that only a portion of the consumer's eyes 362 are visible, the vending machine 100 may determine that the consumer is oriented away from the vending machine 100, and the vending machine 100 may not again attempt to attract the consumer. The ability to determine the consumer's orientation may be useful in crowded locations, such as train platforms, airports, etc., where many consumers may be near the vending machine 100. This helps prevent the vending machine from attempting to interact with every consumer present and ensures that the vending machine only interacts with potential consumers who are interested in the vending machine.

[0073] In some embodiments, the camera 115 of the vending machine 100 may determine whether the consumer is moving and may determine a path P of the consumer 360. The path P may be used to determine if the consumer is approaching the vending machine 100 or moving in a different direction. The vending machine 100 may attempt to interact with the consumer, such as by saying hello, if the consumer's path is directed toward or approaching the vending machine 100. However, if the consumer's path is not moving toward the vending machine 100, for example, if the consumer is walking by the vending machine (e.g., the fourth consumer 360), the vending machine 100 may not attempt to interact with the consumer.

[0074] In some embodiments, the vending machine 100 may be configured to perform noise cancellation to remove noise from consumers who are not interacting with the vending machine 100. This may help improve the accuracy of the vending machine's detection of consumer speech. This may also help prevent the vending machine from responding to other consumers in the nearby area rather than the consumer conducting the transaction.

[0075] In some embodiments, the vending machine 100 may detect the consumer's lips. When a consumer is at the vending machine 100 and engaging with the vending machine 100 for a transaction, the vending machine 100 may employ noise cancellation to remove background noise from other consumers in the area who are not interacting with the vending machine. The vending machine 100 may detect the consumer's presence and may detect the consumer's eyes and lips. If the consumer is attracted to the vending machine and looking at it, the vending machine may detect voice. If the consumer is not looking at the vending machine but is speaking, the vending machine may not detect voice. For example, if the consumer is having a conversation with a companion who is near the vending machine, the vending machine may not detect the conversation. Additionally, the vending machine may record voice when the consumer's lips are moving. In this manner, voice detected when the consumer's lips are not moving may not be analyzed.

[0076] In some embodiments, computer vision may be employed to read the consumer's lips. Reading the consumer's lips can help improve the accuracy of spoken word recognition. For example, if the consumer is in an area with a lot of ambient noise, it may be difficult to convert speech into a text string, and using lip reading can help improve the accuracy of spoken word recognition.

[0077] An exemplary method 1400 of interacting with a consumer is shown in Figure 14. The vending machine may determine whether a consumer is interacting with the vending machine 1410. If the consumer is not interacting, the vending machine may apply noise cancellation to the consumer's speech (1430). If the consumer is interacting with the vending machine, the vending machine may further detect the consumer's eyes (1420). If the consumer is looking at the vending machine, the vending machine may detect the consumer's speech (1440). The vending machine may also track the consumer's lips to attempt to "read" the consumer's lips to improve speech recognition (1460). If the consumer is not looking, the vending machine may not detect the consumer's speech (1450).

[0078] 15 illustrates an exemplary computer system 1500 in which embodiments or portions thereof may be implemented as computer readable code. The control unit 140 described herein may be a computer system having all or some of the components of computer system 1500 for performing the processes described herein.

[0079] Where programmable logic is used, such logic may be executed on a commercially available processing platform or a special purpose device. Those skilled in the art will appreciate that embodiments of the disclosed subject matter may be practiced with a variety of computer system configurations, including multi-core multiprocessor systems, minicomputers and mainframe computers, computers linked or clustered with distributed functionality, and pervasive or small computers that may be embedded in virtually any device.

[0080] For example, at least one processor device and memory may be used to implement the above embodiments. The processor device may be a single processor, multiple processors, or a combination thereof. The processor device may have one or more processor "cores."

[0081] Various embodiments may be implemented in terms of this example computer system 1500. After reading this specification, it will become apparent to one skilled in the art how to implement one or more of the present inventions using other computer systems and / or computer architectures. While operations may be described as sequential processes, some of the operations may in fact be performed in parallel, concurrently, and / or in a distributed environment, and may be performed by program code stored locally or remotely for access by single or multi-processor machines. Additionally, in some embodiments, the order of operations may be rearranged without departing from the spirit of the disclosed subject matter.

[0082] The processor device 1504 may be a dedicated or general-purpose processor device. As will be appreciated by those skilled in the art, the processor device 1504 may also be a single processor in a multi-core / multi-processor system, operating singly or in a cluster of computing devices operating in a cluster or server farm. The processor device 1504 is connected to a communications infrastructure 1506, such as a bus, message queue, network, or multi-core message passing scheme.

[0083] The computer system 1500 also includes a main memory 1508, e.g., random access memory (RAM), and may also include a secondary memory 1510. The secondary memory 1510 may include, for example, a hard disk drive 1512 or a removable storage drive 1514. The removable storage drive 1514 may include a floppy disk drive, a magnetic tape drive, an optical disk drive, a flash memory, or the like. The removable storage drive 1514 reads from and / or writes to a removable storage unit 1518 in a well-known manner. The removable storage unit 1518 may include a floppy disk, magnetic tape, optical disk, universal serial bus (USB) drive, or the like, which is read from and written to by the removable storage drive 1514. As will be appreciated by those skilled in the art, the removable storage unit 1518 includes a computer-usable storage medium having stored thereon computer software and / or data.

[0084] Computer system 1500 (optionally) includes a display interface 1502 (which may include input and output devices such as a keyboard, mouse, etc.) that transfers graphics, text, and other data to be displayed on display 1540 from a communications infrastructure 1506 (or from a frame buffer, not shown).

[0085] In alternative implementations, secondary memory 1510 may include other similar means for allowing computer programs or other instructions to be loaded into computer system 1500. Such means may include, for example, a removable storage unit 1522 and interface 1520. Examples of such means may include program cartridges and cartridge interfaces (such as those found in video game devices), removable memory chips (such as EPROMs or PROMs) and associated sockets, and other removable storage units 1522 and interfaces 1520 that can transfer software and data from the removable storage unit 1522 to computer system 1500.

[0086] Computer system 1500 may also include a communications interface 1524. Communications interface 1524 allows software and data to be transferred between computer system 1500 and external devices. Communications interface 1524 may include a modem, a network interface (such as an Ethernet card), a communications port, a PCMCIA slot and card, or the like. The software and data transferred via communications interface 1524 may be in the form of signals, which may be electronic, electromagnetic, optical, or other signals capable of being received by communications interface 1524. These signals may be provided to communications interface 1524 via communications path 1526. Communications path 1526 carries signals and may be implemented using wire or cable, fiber optics, a phone line, a cellular phone link, an RF link, or other communications channel.

[0087] As used herein, the terms "computer program medium" and "computer usable medium" are used generally to refer to media such as removable storage unit 1518, removable storage unit 1522, and a hard disk installed in hard disk drive 1512. Computer program medium and computer usable medium may also refer to memory, such as main memory 1508 and secondary memory 1510, which may be memory semiconductors (e.g., DRAM, etc.).

[0088] Computer programs (also called computer control logic) are stored in main memory 1508 and / or secondary memory 1510. Computer programs may also be received via communications interface 1524. Such computer programs, when executed, enable computer system 1500 to implement the embodiments described herein. Specifically, the computer programs, when executed, enable processor device 1504 to perform the processes of the embodiments described herein. Thus, such computer programs represent controllers of computer system 1500. When an embodiment is implemented using software, the software may be stored in a computer program product and loaded into computer system 1500 using removable storage drive 1514, interface 1520, hard disk drive 1512, or communications interface 1524.

[0089] Embodiments of the present invention may also be directed to computer program products including software stored on any computer-usable medium. Such software, when executed on one or more data processing devices, causes the data processing devices to operate as described herein. Embodiments of the present invention may employ computer-usable or readable media. Examples of computer-usable media include, but are not limited to, primary storage devices (e.g., any type of random access memory), secondary storage devices (e.g., hard drives, floppy disks, CD ROMs, ZIP disks, tapes, magnetic and optical storage devices, MEMS, nanotechnology storage devices, etc.).

[0090] It is understood that the "Detailed Description" section, and not the "Summary" and "Abstract" sections, is intended to be used to interpret the claims. The Summary and Abstract sections may set forth one or more, but not all, exemplary embodiments of the invention as contemplated by the inventors, and thus are not intended to limit the scope of the invention and the appended claims in any way.

[0091] The present invention has been described above with the aid of functional building blocks that illustrate implementations of certain functions and their relationships. The boundaries of these functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternative boundaries can be defined so long as the certain functions and their relationships are appropriately performed.

[0092] The foregoing description of specific embodiments makes the general nature of the present invention fully apparent, and others, by applying the knowledge of those skilled in the art, may readily modify and / or adapt such specific embodiments for various uses without undue experimentation and without departing from the general concept of the present invention. Such adaptations and modifications are therefore intended to be within the meaning and range of equivalents of the disclosed embodiments, based on the teaching and guidance presented herein. It is to be understood that the phraseology or terminology used herein is for the purpose of description and not of limitation, and therefore should be interpreted by those skilled in the art in light of the teaching and guidance provided herein.

Claims

1. A method for presenting product information to a consumer by a vending machine, comprising: Detecting voice information from a consumer, converting the voice information into a text string, and identifying keywords within the text string; determining one or more products related to the keyword from a product database; and returning a list of one or more products related to the keyword.

2. Detecting second audio information from the consumer, converting the second audio information into a second text string, and identifying a second keyword within the second text string; determining one or more products from the list of one or more products that correspond to the second keyword; The method of claim 1 , further comprising: returning a revised list of the one or more products that correspond to both the keyword and the second keyword.

3. The method of claim 1 , wherein detecting audio information from the consumer is performed by a microphone on the vending machine.

4. 10. The method of claim 1, further comprising: transmitting the audio information to the remote computer prior to converting the audio information to a text string, the converting occurring at the remote computer.

5. The method of claim 1 , wherein the keyword is a brand name.

6. The method of claim 1 , wherein the keyword is flavor.

7. The method of claim 1 , wherein the keyword is an ingredient.

8. The method of claim 1 , further comprising playing a response by the vending machine in response to the audio information.

9. 9. The method of claim 8, wherein playing a response comprises playing a response randomly from a list of pre-recorded responses.

10. The method of claim 1 , further comprising identifying a command within the text string and performing an action by the vending machine based on the command.

11. The method of claim 10 , wherein the command includes adding or removing an item from an electronic shopping cart.

12. 1. A method for making product recommendations to a consumer via a vending machine, comprising: receiving location information of the vending machine and receiving user information of the vending machine; determining one or more tags corresponding to the location information and the user information; identifying a product in a product database associated with the one or more tags; and and making product recommendations based on the one or more tags.

13. The method of claim 12 , wherein receiving the user information includes receiving biometric information from the user's portable electronic device.

14. The method of claim 12 , wherein the location information includes time and temperature at the location of the vending machine.

15. The method of claim 12 , wherein the user information includes a sentiment of the consumer as determined by a camera at the vending machine.

16. The method of claim 12 , wherein the user information includes subject demographic information determined by a camera at the vending machine.

17. 1. A method for tracking consumer engagement by a vending machine having a camera, comprising: Detecting a consumer within a field of view of the camera of the vending machine and determining an orientation of the consumer; attracting the consumer when the consumer faces the vending machine; and detecting speech from the consumer attracted to the vending machine. receiving a product selection from the consumer by detecting the spoken words of the consumer.

18. 20. The method of claim 17, wherein determining the orientation of the consumer comprises detecting the eyes of the consumer.

19. 18. The method of claim 17, further comprising: detecting a second consumer within the field of view of the camera; and applying noise cancellation to the speech of the second consumer when the consumer is attracted to the vending machine.

20. 20. The method of claim 17, wherein detecting the consumer's speech comprises lip tracking the consumer.

Images