Real-time order fulfillment monitoring

Abstract

The present invention provides a method and apparatus for automated ordering and performance monitoring. [Solution] An order is received from a Point of Sale (PoS) device. One or more items in the order are identified. In response to receiving confirmation of the order, the status of each item in the order is monitored. Graphic elements rendered on the display interface are updated, and the graphic elements display the changes in the status of each item in the order.

Description

Technical Field

[0001] Embodiments of the present invention relate to real-time order fulfillment monitoring.

Background Art

[0002]

[0001] In quick-service restaurants that offer drive-through services, several major problems can occur. One occurs at the drive-through kiosk where orders are made mainly verbally rather than through self-selection on a screen. Verbal interaction is more likely to cause misunderstandings or errors compared to direct and visual order selection methods, so this reliance on verbal communication significantly increases the difficulty of accurately verifying the ordered food. The second problem relates to tracking the fulfillment of these orders to ensure that the cooked and packaged food exactly matches the order list. Such order tracking becomes more important when multiple orders are prepared simultaneously to confirm that each food item is correctly included in its respective order before handoff. Failure to accurately fulfill drive-through orders can degrade the user experience and reduce the overall efficiency of the restaurant.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The problem to be solved by the present invention is to provide a method and an apparatus for automatic order placement and fulfillment monitoring.

Means for Solving the Problems

[0004] The method of the embodiment includes receiving an order from a PoS (Point of Sale) device, identifying one or more products included in the order, monitoring the status of each of the one or more products, and displaying changes in the status of each product included in the order on a display interface.

Brief Description of the Drawings

[0005] [Figure 1] Figure 1 shows an exemplary drive-through ordering station according to several embodiments of the present disclosure. [Figure 2] Figure 2 shows an exemplary order tracking system, according to some embodiments of the present disclosure, in which a camera monitors the preparation of goods for different orders. [Figure 3] Figure 3 shows an exemplary drive-through checkout station with an interface for displaying real-time order status, according to some embodiments of the present disclosure. [Figure 4] Figure 4 shows an exemplary method for transcribing voice-to-text drive-through orders and tracking order status to generate real-time updates and notifications, according to some embodiments of the present disclosure. [Figure 5] Figure 5 is a flowchart illustrating exemplary methods for automated order posting and real-time performance monitoring according to several embodiments of the present disclosure. [Figure 6] Figure 6 shows an exemplary computing device configured to perform various aspects of the Disclosure, according to several embodiments of the Disclosure. [Modes for carrying out the invention]

[0006]

[0008] In at least one instance, this disclosure relates to automated order posting and real-time order fulfillment monitoring. When processing drive-thru orders, restaurants may face several challenges. Because drive-thru orders are typically made verbally rather than by self-selection, employees may have difficulty understanding the customer's voice message, which can lead to misunderstandings and errors when taking orders, such as selecting the wrong product, size, or requirement (e.g., the customer is requesting oat milk but entering whole milk). Additionally, there is currently no system in place to monitor the preparation and fulfillment of orders in real time after they have been placed. This lack of tracking can result in not knowing whether an order has been completed and whether the packaged food items match the order list, leading to items from one order being mistakenly placed in another order's bag or incomplete orders being handed off to customers.

[0007]

[0009] This disclosure provides a method and system for automatically transcribing an order's verbal message into text and generating an order list for customer confirmation, potentially on a drive-thru kiosk screen or the user's personal device (if available). Once an order is confirmed, the system may continue to track the order's fulfillment in real time. In some embodiments, the system may use cameras installed in the restaurant to monitor the preparation process. In some embodiments, based on video recordings captured by the cameras, the system may use computer vision technology to track the preparation and packaging of the food. For example, the system may identify an employee assigned to prepare an order, track their actions to determine the status of each item (preparing or completed), and determine whether items within the same order are packaged in their respective bags. The tracked order details may then be transmitted to a display interface accessible to the employee. In some embodiments, the interface may provide real-time updates on the status of each order, allowing employees at the checkout station to monitor progress and address any issues quickly. Once it is determined that all items are correctly packaged and the order is ready for handoff, the system may update the display interface and / or send a notification to the customer's device (if available). The disclosed system streamlines the ordering process through accurate voice-to-text transcription and real-time monitoring of order preparation, thereby significantly improving the overall accuracy and efficiency of drive-thru services.

[0008]

[0010] Figure 1 shows an exemplary drive-through ordering station 105 according to some embodiments of the present disclosure.

[0009]

[0011] In Figure 1, an exemplary drive-through ordering station 105 is located outside the restaurant (or retail store) and comprises three components: a speaker 110, a microphone 115, and a screen 120. These three components work together to facilitate streamlined communication and interaction between restaurant staff and customers when placing an order. In some embodiments, the screen may be configured to display the restaurant's menu. As a customer (or driver) approaches the drive-through station 105, they can view the menu displayed on the screen and consider the available options. The customer can then place an order verbally using the microphone 115. During the process, restaurant staff can communicate with the customer through the speaker 110, for example, to clarify any details of the order or to address any special requests from the customer. The integrated speaker and microphone system facilitates clear, two-way communication that enables staff and customers to effectively communicate orders.

[0010]

[0012] As shown in the figure, the drive-through station 105 is further connected to the computing system 125 via a network 150. In some embodiments, the network 150 may be a local area network (LAN) (e.g., a Wi-Fi system) and / or a wide area network (WAN) (e.g., the Internet). In some embodiments, the drive-through station 105 may record the conversation for ordering between the restaurant staff and the customer. Once completed, in some embodiments, the drive-through station 105 may send the recorded conversation to a speech-to-text module 140 in the computing system 125. The speech-to-text module 140 may convert the spoken words in the conversation into written text. Using the text, the order generation module 145 may generate a detailed list of the order. In some embodiments, the list may include all ordered items, specifying the size, price, any special requirements (such as ingredient substitution or preparation instructions) for each item, and the total price. The order list may provide a comprehensive overview of the customer's selection. The order list is then sent back to the drive-through station 105 (via the network 150) and may be displayed on the screen 120 for customer review and confirmation. In some embodiments, the customer may confirm the order by simply saying “I've confirmed” or “Yes” into the microphone 115. If adjustments are needed, the customer may reiterate their requirements through the microphone 115. The voice data may be automatically transmitted and processed by the speech-to-text module 140 and / or the order generation module 145 for updates.

[0011]

[0013] In some embodiments, the speech-to-text module 140 may be a software application (or a set of software applications) that runs on the computing system 125 and converts spoken language into written text. In some embodiments, the order generation module 145 may be a software application (or a set of software applications) that runs on the computing system 125 and creates a detailed order list based on the converted text.

[0012]

[0014] The computing system 125 can be a single computing device (e.g., a server) or multiple interconnected computing devices (e.g., a data center or a cloud computing environment). The computing system 125 includes processors 130 representing any number of processing elements, each of which may contain any number of processing cores. The computing system 125 also includes memory 135 for storing a speech-to-text module 140 and an order generation module 145. The memory 135 may include volatile memory elements, non-volatile memory elements, and combinations thereof.

[0013]

[0015] A computing system 125 comprising a speech-to-text conversion module 140 and an order generation module 145 is described for the sake of clarity. In some embodiments, orders may be placed using other AI-related technologies (e.g., natural language processing algorithms) or manually by employees using conventional methods such as paper and pen.

[0014]

[0016] Figure 2 shows an exemplary order tracking system 200 in which a camera 205 monitors the preparation of goods for different orders, according to some embodiments of the present disclosure.

[0015]

[0017] An exemplary order tracking system includes one or more cameras 205 and a computing system 225, both connected via a network 250. In some embodiments, the network 250 may be a local area network (LAN) (e.g., a Wi-Fi system) and / or a wide area network (WAN) (e.g., the Internet). The cameras 205 may be installed within the restaurant in a location (such as the kitchen area) where the food preparation process is to be captured. As illustrated, the cameras 205 record multiple staff members 210 working simultaneously on different orders. For example, staff member 210-1 is preparing a fried chicken hamburger for order "0001", and staff member 210-2 is preparing French fries for order "0002". When finished, staff member 210-1 places the hamburger in bag 220-1, and staff member 210-2 places the fries in bag 220-2.

[0016]

[0018] The video footage captured by camera 205 may then be transmitted in real time to a visual tracking module 240 for further processing and analysis. In some embodiments, the visual tracking module 240 may use computing vision technology to analyze the footage, identify each food item being prepared, and / or attribute the items to the correct order based on the staff's actions and the bags in which they are placed. For example, module 240 may identify that staff member 210-1 prepared a fried chicken burger and placed it in bag 220-1 for order "0001". Based on the identification, module 240 may determine that staff member 210-1 is working on order "0001" and that the fried chicken burger has been packed. In some embodiments, the association of bag 220 with an order number may be determined by a receipt printed on or attached to the bag. The receipt may be captured by camera 205, and the order number contained therein may be recognized by the visual tracking module 240.

[0017]

[0019] In embodiments where an order includes multiple items, such as order "0001" consisting of a fried chicken burger and a large Diet Coke, the visual tracking module 240 may track the status of each item (e.g., the burger and the Diet Coke) and classify them as not yet prepared, prepared, and packaged in bags. The module 240 may then assess the completeness of the order after all items in the same order have been properly packaged.

[0018]

[0020] In an embodiment where multiple staff members 210 prepare goods for the same order (e.g., order "0001"), the visual tracking module 240 may focus on interacting with the package bag for that order (e.g., 220-1). For example, module 240 may monitor which goods are being placed in the bag and update the status of the goods (e.g., not yet prepared, prepared, and packaged) and / or the order (e.g., in progress, completed) accordingly.

[0019]

[0021] In some embodiments, the visual tracking module 240 can verify the accuracy of order fulfillment. For example, if staff member 210-1 mistakenly places French fries in bag 220-1, which does not belong to order "0001", module 240 can detect the error and send a warning. If staff member 210-1 inadvertently moves bag 220-1 to the ready area before the order is completed, for example, if order "0001" includes both a burger and a Diet Coke, but the bag only includes a burger, module 240 can track the status in real time each hour and immediately detect the discrepancy and issue a warning. The real-time order fulfillment monitoring mechanism allows for the rapid correction of errors to ensure that the final order handed to the customer is accurate and complete.

[0020]

[0022] In some embodiments, the visual tracking module 240 may be a software application (or a set of software applications) running on the computing system 225 that tracks the preparation and fulfillment of each order in the restaurant.

[0021]

[0023] The computing system 225 can be a single computing device (e.g., a server) or multiple interconnected computing devices (e.g., a data center or a cloud computing environment). The computing system 225 includes a processor 230 representing any number of processing elements that can each include any number of processing cores. The computing system 225 also includes a memory 235 that stores a visual tracking module 240. The memory 235 can include volatile memory elements, non-volatile memory elements, and combinations thereof.

[0022]

[0024] In some embodiments, the computing system 225 can correspond to the computing system 125 shown in FIG. 1, in which the speech-to-text conversion module 140, the order generation module 145, and the visual tracking module 240 are integrated within a single computing device. In some embodiments, the computing system 225 can be a separate entity from the computing system 125 and can be configured to process motion tracking data. The computing system 225 can be connected to the computing system 125 via a network 150 (or 250) to receive confirmed order information.

[0023]

[0025] For clarity of concept, an exemplary order tracking system 200 is provided that includes one camera 205. However, in some embodiments, any number of cameras can be installed in the restaurant to monitor the order preparation and fulfillment process. In some embodiments, the camera 205 utilized within the system can be an edge camera configured with built-in processing capabilities. In such a configuration, the camera can directly receive confirmed order data from the computing system 125 as shown in FIG. 1 and / or can use its built-in processing capabilities to identify and monitor the status of each order from preparation to packing.

[0024]

[0026] Figure 3 shows an exemplary drive-through checkout station 300 with an interface 305 for displaying real-time order status, according to some embodiments of the present disclosure.

[0025]

[0027] An exemplary drive-through checkout station 300 has an open window facing the drive-through path, allowing restaurant staff 310 to easily hand over prepared orders to customers and / or complete payment transactions. As shown, the drive-through checkout station 300 also includes a display interface 305 and a payment terminal 315. The display interface 305 is connected to a computing system 225 (as shown in Figure 2) via a network 250. Such a connection enables the transmission of visual and status data between the computing system 225 and the interface 305.

[0026]

[0028] As shown in the diagram, interface 305 is configured to display the order status (tracked by the visual tracking module 240 in the computing system 225). Interface 305 organizes the order status information into a clear and structured list 320 containing three columns: order number, order status (labeled "Completed" if all items are packaged or "In Progress" if any items remain unpackaged), and a list of items for each order. To enhance clarity regarding the status of individual items, small checkboxes may be placed next to each item name. Each checkbox may serve as a visual indicator of the progress of the item's preparation and packaging. For example, if an item (e.g., fried chicken burger) has been prepared and placed in a bag (e.g., 220-1), the corresponding checkbox will be checked to indicate its completion. If an item is still being prepared or has not yet been started, the checkbox next to its name will remain unchecked, indicating that it is still in progress. In some embodiments, the interface 305 may use a color-coding scheme to provide a more direct visual representation of the product preparation status, such as using red for products not yet prepared, yellow for products currently prepared, and green for products that are fully packaged. Through checkboxes and / or the color-coding scheme, staff 310 can quickly assess the status of each product within different orders.

[0027]

[0029] In some embodiments, interface 305 may display the package bags prepared for each order, and order status information is organized in text boxes adjacent to each bag. These text boxes may display details such as the order number, order status (labeled "In Progress" or "Completed"), and the items included in each order. In some embodiments, the corresponding text box for each package bag may not only identify the order but also highlight the completion status of the items within it. For example, a checkbox next to each listed item in the text box may serve as an indicator of whether the item is prepared and included in the package. In some embodiments, a checkmark may indicate completion, and an unchecked box may indicate that the item is still being prepared.

[0028]

[0030] In some embodiments, once the order is complete, the computing system 225 may prompt an on-screen notification on the display interface 305. In embodiments where customer contact information is available, the computing system 225 may send a notification directly to the customer's device (e.g., text message, email, phone call, or push notification). The notification informs the customer and / or staff that the order is ready for pickup.

[0029]

[0031] As illustrated, when an error is detected, such as goods placed in the wrong bag or an incomplete order mistakenly placed in the ready-to-receive area, the affected order in List 320 may be highlighted in red or flagged with a warning sign. The warning can serve as an immediate visual reminder for staff 310 to correct the mistake before handing the order to the customer.

[0030]

[0032] The display interface 305 functions as a central information hub, providing real-time updates on the tracked order status. Through interface 305, staff can quickly verify the completion of each drive-thru order and correct errors, such as incorrectly assigned items or incomplete orders, before handing the order to the customer.

[0031]

[0033] Figure 4 illustrates an exemplary method 400 for transcribing drive-through orders from voice to text and tracking order status to generate real-time updates and notifications, according to some embodiments of the present disclosure. In some embodiments, method 400 may be performed by one or more computing systems or devices, such as computing system 125 shown in Figure 1, computing system 225 shown in Figure 2, and / or computing device 600 shown in Figure 6.

[0032]

[0034] Method 400 begins in block 405, where a computing system (e.g., 125 in Figure 1) receives voice messages from a connected drive-thru station (e.g., 105 in Figure 1). In some embodiments, the drive-thru station may include a screen (e.g., 120 in Figure 1), a speaker (e.g., 110 in Figure 1), and a microphone (e.g., 115 in Figure 1). As a customer approaches the station, they may view the menu displayed on the screen and speak into the microphone to place an order. The voice data may be transmitted in real time to a restaurant staff member's headset (or speaker system), enabling the staff member to place the order on behalf of the customer. If the staff member has any questions about the order or requires further clarification, they may communicate with the customer through the speaker built into the drive-thru station. The entire conversation for ordering may be recorded as a voice message and transmitted from the station to the computing system for further processing.

[0033]

[0035] In block 410, the computing system transcribes a speech message into text. The transcription process may include analyzing the speech message and using speech recognition techniques to identify words and phrases. After recognition, the system may then convert these spoken words and phrases into written text. In some embodiments, the system may further perform contextual analysis to detect and correct common speech recognition errors. Contextual analysis may include resolving ambiguities such as homonyms (words that sound similar but have different meanings) or revealing unclear pronunciations.

[0034]

[0036] In block 415, the computing system generates an order list based on transcribed text. The system can parse the text to identify and extract information about the order, such as the product name, quantity, size, and any special requests mentioned by the customer (e.g., material substitution or preparation instructions). Using the parsed information, the computing system can generate a detailed order list. In some embodiments, the order list may include all ordered goods, along with the specified size, price of each item, any special requirements attached to each item, and the total price of the order.

[0035]

[0037] In block 420, the computing system checks whether the order has been confirmed by the customer. In some embodiments, the generated order list may be displayed on a screen at the drive-through station (e.g., 120 in Figure 1) for the customer to review and confirm. If the order is confirmed, such that the customer says “I have confirmed” or “yes” into the station's microphone (e.g., 115 in Figure 1), method 400 proceeds to block 425. If the customer has identified an error or wishes to add additional items, the customer may speak directly into the microphone to communicate these changes. In such a configuration, method 400 returns to block 410, where the voice message is transcribed into text for any modifications or additions made to the order.

[0036]

[0038] In block 425, the computing system monitors the preparation of orders in the kitchen (or other processing area). In some embodiments, the computing system may receive live video of the kitchen area from one or more cameras (e.g., 205 in Figure 2). The system may use computer vision techniques to analyze the captured video and track order status. For example, using an image recognition machine learning (ML) model, the system may identify food items being prepared by different staff members (e.g., one staff member is preparing fried chicken burgers and another is preparing fries) and determine which packaging bags are assigned to different orders (e.g., by recognizing order numbers printed or attached to each bag). After the items and bags are identified, the system may attribute each item to the correct order based on a continuous tracking of staff actions. For example, if the system detects that a staff member has placed a fried chicken burger into a packaging bag labeled order "0001", it may determine that the staff member is working on order "0001" and the burger is being packaged. The system can then load other information about order "0001" and continue to track the actions of staff members as they prepare other items in the order.

[0037]

[0039] In some embodiments, the computing system may classify the status of each item into three main stages: not yet prepared, prepared, and packaged. By aggregating the status information of individual items, the computing system may determine the overall status of an order, whether it is still in progress or completed.

[0038]

[0040] In embodiments where multiple staff members prepare goods for the same order (e.g., order "0001"), the computing system may focus on tracking interactions with the package bags for that order. When goods prepared by any staff member are added to the bag, the system may update the status of the goods (e.g., not yet prepared, prepared, and packaged) and the overall order status (e.g., in progress, completed) accordingly. In some embodiments, if a single staff member (or a defined set of staff members) is assigned to prepare goods for a given order, the computing system may evaluate the movements or actions of the defined staff members to track the order status.

[0039]

[0041] In some embodiments, the computing system may identify errors in order packaging through its image recognition and motion tracking capabilities. For example, based on tracked product status and overall order status, the computing system may identify whether a product has been mistakenly placed in the wrong packaging bag or whether an incomplete order has been mistakenly moved to the ready-to-receive area. Upon detecting such discrepancies, the computing system may generate a warning or notification prompting staff to take immediate corrective action.

[0040]

[0042] In block 430, with order preparation being tracked, the computing system updates a display interface (e.g., 305 in Figure 3) to display the order status in real time. In some embodiments, the display interface may include a screen installed at the checkout station, preparation area, or other relevant location. The screen may provide staff with the latest order details. This configuration allows staff to monitor the progress of each order in real time and prepare for the final stage of order delivery. In some embodiments, the display interface may extend to a screen in the drive-through station (e.g., 120 in Figure 1), through which customers can see the progress of their orders as they are being prepared.

[0041]

[0043] In block 435, the computing system checks whether all the goods in the order are ready and ready to be delivered to the customer. If the order is ready, method 400 proceeds to block 440. If the order is not ready, method 400 returns to block 425, and the computing system continues monitoring until completion.

[0042]

[0044] In block 440, the computing system sends a notification to the display interface to inform the staff at the checkout station that the order is ready to be handed over to the customer. In some embodiments, the computing system may send a notification to the customer's personal device (via text message, email, phone, or push notification) indicating that the order is ready to be picked up.

[0043]

[0045] Figure 5 is a flowchart illustrating exemplary methods for automated order posting and real-time performance monitoring according to several embodiments of the present disclosure.

[0044]

[0046] In block 505, a computing system (e.g., 125 in Figure 1) receives orders from a point-of-sale (PoS) device (e.g., a drive-through station 105 in Figure 1). In some embodiments, the PoS device may comprise at least one of a drive-through checkout station or a self-checkout station.

[0045]

[0047] In block 510, the computing system identifies one or more items of the order.

[0046]

[0048] In block 515, in response to receiving confirmation of an order, the computing system monitors the status of each of the one or more items in the order. In some embodiments, the process of monitoring the status of each item may include accessing one or more images captured by one or more cameras (e.g., 205 in Figure 2) and processing one or more images using one or more object recognition machine learning models.

[0047]

[0049] In some embodiments, the status of each product may include at least one of the following indications: preparation has not been started, preparation is in progress, or preparation is complete.

[0048]

[0050] In block 520, the computing system updates the graphical elements rendered on the display interface (e.g., 305 in Figure 3), which display the changes in the status of each item in the order.

[0049]

[0051] In some embodiments, the computing system may further generate an order list based on multiple products and display the order list on the screen of a PoS device (e.g., 120 in Figure 1) for order confirmation.

[0050]

[0052] In some embodiments, the display interface may show a package bag for an order and a text indicator adjacent to the package bag, the text indicator including at least one of an order number, an order list, and an indicator of order completion. In some embodiments, the indicator of completion may include at least one of a color scheme or one or more checkboxes to indicate completion of each item in the order or completion of the entire order.

[0051]

[0053] In some embodiments, once it determines that an order has been completed, the computing system may send a notification to one or more user devices indicating that the order is ready for receipt.

[0052]

[0054] Figure 6 shows exemplary computing devices 600 configured to perform various aspects of the Disclosure according to several embodiments of the Disclosure. Although shown as a physical device, in some embodiments the computing device 600 may be implemented using virtual devices and / or across several devices (e.g., in a cloud environment). The computing device 600 can be embodied as any computing device or system, such as computing system 125 shown in Figure 1, or computing system 225 shown in Figure 2.

[0053]

[0055] As illustrated, the computing device 600 includes a CPU 605, memory 610, storage device 615, one or more network interfaces 625, and one or more I / O interfaces 620. In the illustrated embodiment, the CPU 605 retrieves and executes programming instructions stored in memory 610, and stores and retrieves application data residing in storage device 615. The CPU 605 generally represents a single CPU and / or GPU, multiple CPUs and / or GPUs, a single CPU and / or GPU with multiple processing cores, etc. Memory 610 is generally considered to represent random access memory. Storage device 615 can be any combination of disk drives, flash-based storage devices, etc., and may include fixed and / or removable storage devices such as fixed disk drives, removable memory cards, caches, optical storage devices, network-attached storage (NAS), or storage area networks (SAN).

[0054]

[0056] In some embodiments, I / O devices 635 (e.g., keyboard, monitor, etc.) are connected via I / O interface 620. Furthermore, via network interface 625, computing device 600 can be communicatively coupled to one or more other devices and components (via a network, which may include, for example, the Internet, a local network, etc.). As shown in the figure, the CPU 605, memory 610, storage device 615, network interface 625, and I / O interface 620 are communicatively coupled by one or more buses 630.

[0055]

[0057] In the illustrated embodiment, memory 610 includes a speech-to-text module 650, an order generation module 655, and a vision tracking module 660. Although shown as separate modules for clarity, in some embodiments, the operation of the shown modules (and other modules not shown) may be combined or distributed across any number of modules. Furthermore, although shown as software residing in memory 710, in some embodiments, the operation of the shown modules (and other modules not shown) may be implemented using hardware, software, or a combination of hardware and software.

[0056]

[0058] In the illustrated embodiment, the speech-to-text module 650 is configured to convert spoken language into written text. When a customer places an order verbally through a drive-thru station, the speech-to-text module 650 captures the voice message and transcribes it into text using a speech recognition algorithm. Following the transcription of the customer's order into text, the order generation module 655 processes the text to extract information about the order, such as product names, quantities, sizes, and any special requests. Based on the extracted information, module 655 generates a detailed order list outlining all components of the customer's order in a clear and organized manner. The visual tracking module 660 uses computer vision technology to monitor order fulfillment, utilizing detailed order information and live video footage from cameras installed within the restaurant. In some embodiments, the visual tracking module 660 may identify the preparation times of food items and associate them with the correct order based on tracking staff actions (such as placing items into packaging bags). Utilizing product recognition and real-time motion tracking, module 660 may track the status of each item (e.g., not yet prepared, in progress, or being packaged). By aggregating the status of individual items, module 660 can then determine the overall order status.

[0057]

[0059] The storage device 615 may contain various data for the efficient operation of the computing device. In the illustrated embodiment, the storage device 615 includes transcribed text data 670 (generated by the speech-to-text module 650), order details 675 (organized by the order generation module 655), order and product status updates 680 (captured by the visual tracking module 660), and video footage 685 (captured by cameras installed in the restaurant for order tracking). In some embodiments, the aforementioned data may be stored in a remote database connected to the computing device 600 via a network (e.g., Wi-Fi or the Internet).

[0058]

[0060] The descriptions of the various embodiments of this disclosure are presented for illustrative purposes only and are not intended to be exhaustive or limitful to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the embodiments described. The terms used herein have been chosen to best describe the principles of the embodiments, their practical applications or technical improvements to the technology found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

[0059]

[0061] The embodiments presented herein are referenced below. However, the scope of this disclosure is not limited to the embodiments described herein. Instead, any combination of the following features and elements is intended to implement and practice the assumed embodiments, whether relating to different embodiments or not. Furthermore, while the embodiments disclosed herein may achieve advantages over other possible solutions or the prior art, whether advantages are achieved by a given embodiment does not limit the scope of this disclosure. Accordingly, the following aspects, features, embodiments, and advantages are merely illustrative and should not be considered elements or limitations of the appended claims unless expressly enumerated in the claims. Similarly, references to “this disclosure” should not be interpreted as generalizations of the subject matter of any invention disclosed herein and should not be considered elements or limitations of the appended claims unless expressly enumerated in the claims.

[0060]

[0062] The embodiments of this disclosure may take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, microcode, etc.), or embodiments that combine software and hardware embodiments which may be commonly referred to herein as “circuits,” “modules,” or “systems.”

[0061]

[0063] This disclosure may also be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium(s) having computer-readable program instructions for causing a processor to perform an aspect of this disclosure.

[0062]

[0064] A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction-executing device. A computer-readable storage medium may, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the above. A non-exhaustive list of more examples of computer-readable storage media includes portable computer diskettes, hard disks, random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random-access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital multipurpose disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punched cards or grooved raised structures on which instructions are recorded, and any suitable combination of the above. As used herein, a computer-readable storage medium should not be construed as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses passing through optical fiber cables), or electrical signals transmitted through wires.

[0063]

[0065] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface of each computing / processing device receives computer-readable program instructions from the network and transfers them for storage in a computer-readable storage medium within the respective computing / processing device.

[0064]

[0066] The computer-readable program instructions for performing the operations of the Disclosure may be either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk and C++, and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may run entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it may be connected to an external computer (for example, via the Internet using an Internet service provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) may execute computer-readable program instructions by personalizing the electronic circuit using state information of computer-readable program instructions in order to perform an aspect of the present disclosure.

[0065]

[0067] Aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It will be understood that each block in a flowchart and / or block diagram, as well as combinations of blocks in a flowchart and / or block diagram, can be implemented by computer-readable program instructions.

[0066]

[0068] These computer-readable program instructions may be provided to a general-purpose computer, a dedicated computer, or a processor of another programmable data processing device for manufacturing a machine, such that instructions executed via the processor of a computer or other programmable data processing device create means for performing functions / operations specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can be instructed to function such that the computer-readable storage medium having the stored instructions contains a product containing instructions that perform a mode of function / operation specified in one or more blocks of a flowchart and / or block diagram.

[0067]

[0069] Computer-readable program instructions may also be loaded onto a computer, other programmable device, or other device to generate a computer implementation process by causing the computer, other programmable device, or other device to execute a series of operational steps on the computer, other programmable device, or other device so that the instructions executed on the computer, other programmable device, or other device perform the functions / operations specified in one or more blocks of a flowchart and / or block diagram.

[0068]

[0070] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of the systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in the flowchart or block diagram may represent a module, segment, or part of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions described in a block may be performed in a different order than shown in the figure. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or blocks may sometimes be executed in reverse order depending on the functions they relate to. It should also be noted that each block in the block diagram and / or flowchart diagram, as well as any combination of blocks in the block diagram and / or flowchart diagram, can be implemented by a dedicated hardware-based system that performs a specified function or operation, or a combination of dedicated hardware and computer instructions.

[0069]

[0071] Embodiments of this disclosure may be provided to end users through a cloud computing infrastructure. Cloud computing generally refers to the provision of scalable computing resources as a service over a network. More formally, cloud computing can be defined as computing capacity that provides an abstraction between computing resources and their underlying technological architecture (e.g., servers, storage, networks), enabling convenient on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and deployed with minimal administrative effort or service provider interaction. Thus, cloud computing enables users to access virtual computing resources in the “cloud” (e.g., storage, data, applications, and even full virtualized computing systems) regardless of the underlying physical systems (or the location of those systems) used to provide the computing resources.

[0070]

[0072] Typically, cloud computing resources are provided to users on a pay-per-use basis, and users are charged for the computing resources they actually use (e.g., the amount of storage space consumed by the user, or the number of virtualization systems instantiated by the user). Users can access any of the resources residing in the cloud from anywhere on the internet at any time. In the context of this disclosure, users can access applications available in the cloud (e.g., order fulfillment tracking applications) or related data. For example, an order fulfillment tracking application may perform data processing, generate corresponding instructions through the cloud computing infrastructure, and store the related results in storage locations in the cloud. In this way, users can access this information from any computing system attached to a network connected to the cloud (e.g., the internet).

[0071]

[0073] While the above applies to embodiments of the present disclosure, other and further embodiments of the present disclosure may be devised without departing from its basic scope, the scope of which is determined by the following claims. [Explanation of Symbols]

[0072] 105 Drive-Thru Station 110 speakers 115 Microphone 120 screens 125 Computing Systems 130 processors 135 memory 140 Speech-to-Text Modules 145 Order Generation Module 150 Networks 200 Order Tracking System 205 Camera 210-1, 210-2 Staff 220-1, 220-2 Bags 225 Computer Systems 230 processors 235 memory 240 Visual Tracking Modules 250 Networks 300 drive-through checkout stations 305 Display Interface 310 Restaurant Staff 315 Payment terminal 320 List 600 Computing Systems 605 CPU 610 memory 615 Storage device 620 I / O interfaces 625 Network Interfaces 630 Interconnection (bus) 635 I / O devices 650 Speech-to-Text Module 655 Order Generation Module 660 Visual Tracking Module 670 Transcribed text data 675 Order Details 680 Order and product status update 685 Video footage

Claims

1. Receiving orders from a PoS (Point of Sale) device, Identifying one or more items included in the aforementioned order, Monitoring the status of each of the one or more of the aforementioned products, A method comprising displaying the changes in the status of each product included in the order on a display interface.

2. To generate an order list showing one or more of the aforementioned products, To confirm the order, the order list is displayed on the screen of the POS device. The method according to claim 1, further comprising:

3. Monitoring the aforementioned status Accessing one or more images captured by one or more cameras, The method according to claim 1, comprising processing one or more images using one or more object recognition machine learning models.

4. The method according to claim 1, wherein the status of each product includes at least one of the following indications: preparation has not been started, preparation is in progress, or preparation is complete.

5. The method according to claim 1, wherein the PoS device comprises at least one drive-through checkout station or self-checkout station.

6. The method according to claim 1, wherein the display interface indicates a package bag for the order and a text indicator adjacent to the package bag, the text indicator including at least one of an order number, an order list, and an indication of the completion of the order.

7. The method according to claim 6, wherein the indication of completion includes a color-coding scheme or at least one of one or more checkboxes to indicate the completion of each item in the order or the completion of the entire order.

8. The method according to claim 1, further comprising determining that the order has been completed, sending a notification to one or more user devices indicating that the order is ready for receipt.

9. It is a system, One or more processors, When executed on any combination of the one or more processors, one or more memories store a program that performs an operation, wherein the operation is Receiving orders from a PoS (Point of Sale) device, Identifying one or more items included in the aforementioned order, Monitoring the status of each of the one or more of the aforementioned products, A memory and a display interface that displays the changes in the status of each product included in the order, A system that includes these features.

10. When the program is executed on any combination of the one or more processors, To generate an order list showing one or more of the aforementioned products, To confirm the order, the order list is displayed on the screen of the PoS device, The system according to claim 9, further comprising performing the operations described above.

11. In order to monitor the status of each product, the program is executed on any combination of one or more processors, Accessing one or more images captured by one or more cameras, The system according to claim 9, which performs the operation comprising processing one or more images using one or more object recognition machine learning models.

12. The system according to claim 9, wherein the status of each product includes at least one of the following indications: preparation has not been started, preparation is in progress, or preparation is complete.

13. The system according to claim 9, wherein the PoS device comprises at least one of a drive-through checkout station or a self-checkout station.

14. The system according to claim 9, wherein the display interface indicates a package bag for the order and a text indicator adjacent to the package bag, the text indicator including at least one of an order number, an order list, and an indication of the completion of the order.

15. The system according to claim 14, wherein the indication of completion includes at least one color-coding scheme or one or more checkboxes for indicating the completion of each item in the order or the completion of the entire order.

16. The system according to claim 9, wherein the program, when executed on any combination of the one or more processors, performs the operation, further comprising determining that the order is complete and sending a notification to one or more user devices indicating that they are ready to receive the order.

17. When executed by the operation of a computer system, Receiving orders from a PoS (Point of Sale) device, Identifying one or more items included in the aforementioned order, Monitoring the status of each of the one or more of the aforementioned products, One or more non-temporary computer-readable media containing any combination of computer program code that performs an operation, including displaying the changes in the status of each product included in the aforementioned order on a display interface.

18. When executed by the operation of a computer system, To generate an order list showing one or more of the aforementioned products, One or more non-temporary computer-readable media according to claim 17, which is the computer program code that performs the operation, further comprising displaying the order list on the screen of the PoS device for order confirmation.

19. To monitor the status of each product, the computer system will perform the following actions: Accessing one or more images captured by one or more cameras, One or more non-temporary computer-readable media according to claim 17, which is the computer program code that performs the operation, which includes processing one or more images using one or more object recognition machine learning models.

20. One or more non-temporary computer-readable media according to claim 17, which is the computer program code that performs the operation, which, when performed by the operation of a computer system, further includes sending a notification to one or more user devices indicating that the order is ready to be received when it is determined that the order has been completed.

Images