Ultra-large artificial intelligence-based unmanned drive-through robot cafe service system

Abstract

The present invention relates to an ultra-large artificial intelligence-based unmanned drive-through robot cafe service system. The ultra-large artificial intelligence-based unmanned drive-through robot cafe service system enables a café service to be provided by receiving orders via interactive voice without human intervention in a drive-through environment, by comprising: a vision sensor unit that collects information on a vehicle entering and moving through a drive-through for beverage ordering in an unmanned drive-through environment; a voice prompt device that receives a beverage order via voice interaction with a user in the vehicle on the basis of the information collected by the vision sensor unit by means of voice prompt technology capable of human-level conversation; a robot cafe device that enables a beverage, which is ordered via the voice prompt technology without human intervention via the voice prompt device, to be prepared by the robot performing a series of operations of a preset beverage recipe, and enables the user in the vehicle to make a payment and receive the beverage from the robot; and a robot service platform that controls overall operational actions of the vision sensor unit, the voice prompt device, and the robot cafe device.

Description

A super-large AI-based unmanned drive-thru robot cafe service system

[0001] The present invention relates to an unmanned drive-through robot cafe service system based on ultra-large artificial intelligence, and more specifically, to an unmanned drive-through robot cafe service system based on ultra-large artificial intelligence, which allows an unmanned robot to prepare a drink ordered by a driver in a vehicle and provide it to the driver in the vehicle, and which can provide cafe service by taking orders through conversational voice without human intervention in a drive-through environment.

[0002] The content described in this section merely provides background information for one embodiment of the present invention and does not constitute prior art.

[0003]

[0004] Drive-thru services, where customers order drinks while in their vehicles and receive them prepared and delivered to their vehicles at a cafe or other establishment, are on the rise. These drive-thrus operate by having employees take orders via voice command, preparing the drinks while the vehicle is en route to the pickup location, and then delivering them directly to the employees at the pickup location.

[0005]

[0006] The taste of drinks made in a cafe is determined by the skill of both unskilled and professional baristas, which directly affects the store's sales. When hiring employees, cafes must receive beverage preparation training, and if an employee who has been trained quits, they must hire a new employee and train them from scratch, making it difficult to manage the workforce. Furthermore, rising labor costs can lead to decreased operating profits. Drive-thru cafes require employees to take orders, prepare drinks, and deliver drinks. Since employees reach out to drivers in their vehicles to deliver drinks, the cafes also require a significant amount of labor.

[0007]

[0008] Currently, the restaurant industry is facing a serious talent shortage amid rising labor costs amid the pandemic. This is due to low wages for simple tasks and a desire to avoid long hours and intense work. Furthermore, consumers experiencing business restrictions and hygiene issues during the COVID-19 pandemic are experiencing a surge in demand for contactless technologies, such as unmanned stores, which were previously viewed with resistance. This growth in demand for contactless, unmanned systems is driven by advancements in IT, the increased demand for contactless services due to the COVID-19 pandemic, and environmental, social, and governance (ESG) issues. The food tech market is expanding, and the sector is developing into a key sector with long-term growth potential supported by factors such as future population growth and an aging population.

[0009]

[0010] Recently, restaurant operators have been facing challenges such as rising labor costs, labor shortages, economic downturns, and rising prices. To overcome these challenges, some are offering services utilizing food tech robots. However, these robots face limitations in fully unmanned operation, and they are limited to replacing some employees. Republic of Korea Patent No. 10-2187090 is disclosed as a prior art document.

[0011]

[0012] The background technology described above is technical information that the inventor possessed for the purpose of deriving the present invention or acquired in the process of deriving the present invention, and cannot necessarily be said to be publicly known technology disclosed to the general public prior to the application for the present invention.

[0013] The present invention has been proposed to solve the above-mentioned problems of the existing proposed methods, and the purpose of the present invention is to provide an unmanned drive-thru robot cafe service system based on ultra-large-scale artificial intelligence, by including a vision sensor unit that collects information on a vehicle entering and moving through a drive-thru for ordering a beverage in an unmanned drive-thru environment, a voice prompt device that receives a beverage order through voice conversation with a user riding in a vehicle collected from the vision sensor unit using voice prompt technology capable of human-level conversation, a robot cafe device that manufactures a beverage ordered through the voice prompt technology without human intervention by a robot that performs a series of operation sequences of a preset beverage recipe while the user is riding in the vehicle, and a robot service platform that controls the overall operation of the vision sensor unit, the voice prompt device, and the robot cafe device, so that an unmanned robot manufactures a beverage ordered by a driver riding in the vehicle and provides it to the driver riding in the vehicle, and provides a cafe service by receiving an order through conversational voice without human intervention in a drive-thru environment.

[0014]

[0015] In addition, the present invention provides an unmanned drive-thru robot cafe service system based on large-scale artificial intelligence, which enables a driver in a vehicle to order a drink using conversational voice in a drive-thru environment without human intervention, and which manufactures the ordered drink according to the work sequence using a multi-joint robot and provides the drink to the driver in the vehicle after payment, thereby reducing the intensity of labor by replacing customer service work and simple repetitive work with artificial intelligence and robots from the worker's perspective, and not only resolving the manpower shortage from the business owner's perspective, but also enabling improved productivity, reduced costs, and increased employment efficiency.

[0016]

[0017] However, the technical problems to be solved by the present invention are not limited to the technical problems described above, and other technical problems may exist.

[0018] In order to achieve the above-mentioned purpose, the ultra-large artificial intelligence-based unmanned drive-through robot cafe service system according to the features of the present invention is:

[0019] As a fully unmanned drive-thru robot cafe service system based on ultra-large artificial intelligence,

[0020] A vision sensor unit that collects information on vehicles entering and moving through a drive-thru to order beverages in an unmanned drive-thru environment;

[0021] A voice prompt device that receives a beverage order through voice conversation with a user riding in a vehicle, which is collected from the vision sensor unit, using voice prompt technology capable of human-level conversation;

[0022] A robot cafe device that allows a robot to produce a beverage ordered through voice prompt technology without human intervention through the voice prompt device and perform a series of operation sequences of a preset beverage recipe, and allows a user to make payment and receive a beverage from the robot while on board a vehicle; and

[0023] Its configuration features include a robot service platform that controls the overall operation of the vision sensor unit, voice prompt device, and robot cafe device.

[0024]

[0025] Preferably, the vision sensor unit,

[0026] A first vision sensor that collects information on vehicles entering and moving through a drive-thru for ordering beverages in an unmanned drive-thru environment, and collects vehicle identification information including vehicle number and vehicle type information by recognizing a vehicle moving through the entrance of the drive-thru; and

[0027] In an unmanned drive-thru environment, information on vehicles entering and moving through a drive-thru for ordering drinks is collected, and a second vision sensor can be configured to recognize the moving vehicle by vision before the vehicle arrives at the robot cafe device after ordering drinks from the voice prompt device of the drive-thru and collecting vehicle identification and vehicle occupant spatial information (location).

[0028]

[0029] More preferably, the vision sensor unit,

[0030] In an unmanned drive-thru environment, a first vision sensor and a second vision sensor are included to collect information on a vehicle entering and moving through a drive-thru for a beverage order, and vehicle identification information including a vehicle number and vehicle type information collected through the first vision sensor and the second vision sensor, and vehicle passenger spatial information (location) can be collected and transmitted to the robot service platform.

[0031]

[0032] Preferably, the voice prompt device,

[0033] By utilizing a super-large language model, it is possible to extract standardized customer order information through human-level general conversation and provide voice response to customers' drink orders.

[0034]

[0035] More preferably, the voice prompt device,

[0036] A microphone that inputs the voice of a user ordering a drink while in the vehicle;

[0037] A noise removal and speaker separation unit that removes noise from the voice of a user ordering a drink in a vehicle input through the microphone and separates and extracts the speaker;

[0038] A voice-to-text conversion unit that converts the drink order voice from which noise has been removed and the speaker has been separated from the noise removal and speaker separation unit into drink order text and outputs the same;

[0039] An order model that receives a beverage order text from the above-mentioned voice text conversion unit, transmits an order request corresponding to the beverage order content to a robot cafe device through the above-mentioned robot service platform, and generates a response text of the order situation;

[0040] A text-to-speech conversion unit that converts the response text of the order situation generated from the above order model into voice and outputs it; and

[0041] It can be configured to include a speaker that outputs a response voice of the order situation converted from the above text-to-speech conversion unit to the user.

[0042]

[0043] Even more preferably, the order model is:

[0044] As a super-large AI language model with voice prompt technology capable of human-level conversation, it can function to provide a fine tuning service through reinforcement learning to a pre-trained general-purpose super-large AI language model.

[0045]

[0046] Even more preferably, the robot service platform,

[0047] The vehicle identification information and vehicle passenger spatial information (location) collected from the above vision sensor unit can be stored and managed, beverage order reception information received from the voice prompt device can be stored and managed, and beverage information prepared from the robot cafe device, payment information, and beverage provision information can be stored and managed.

[0048] According to the ultra-large artificial intelligence-based unmanned drive-thru robot cafe service system proposed in the present invention, the system comprises a vision sensor unit that collects information on vehicles entering and moving through a drive-thru to order drinks in an unmanned drive-thru environment, a voice prompt device that receives drink orders through voice conversations with users in the vehicle collected from the vision sensor unit using voice prompt technology capable of human-level conversation, a robot cafe device that manufactures drinks ordered through the voice prompt technology without human intervention by a robot that performs a series of operation sequences of a preset drink recipe and allows users to make payments and receive drinks from the robot while in the vehicle, and a robot service platform that controls the overall operation of the vision sensor unit, the voice prompt device, and the robot cafe device, so that the unmanned robot manufactures drinks ordered by drivers in the vehicle and provides them to drivers in the vehicle, and cafe service can be provided by receiving orders through interactive voice conversations without human intervention in a drive-thru environment.

[0049]

[0050] In addition, according to the ultra-large artificial intelligence-based unmanned drive-thru robot cafe service system of the present invention, a driver in a vehicle can order a drink using conversational voice without human intervention in a drive-thru environment, and the ordered drink can be manufactured according to the work sequence using a multi-joint robot and provided to the driver in the vehicle after payment. Therefore, from the worker's perspective, it is possible to create quality jobs by reducing the intensity of labor through replacement of customer service work, simple repetitive work, etc. with artificial intelligence and robots, and from the business owner's perspective, it is possible to not only resolve the manpower shortage, but also improve productivity, reduce costs, and increase employment efficiency.

[0051]

[0052] In addition, the various advantageous advantages and effects of the present invention are not limited to the above-described contents, and will be more easily understood in the process of explaining specific embodiments of the present invention.

[0053] FIG. 1 is a diagram illustrating the configuration of an ultra-large artificial intelligence-based unmanned drive-through robot cafe service system according to one embodiment of the present invention as a functional block.

[0054] FIG. 2 is a diagram illustrating the configuration of a vision sensor unit of an ultra-large artificial intelligence-based unmanned drive-thru robot cafe service system according to an embodiment of the present invention as a functional block.

[0055] FIG. 3 is a diagram illustrating a service concept configuration of an ultra-large artificial intelligence-based unmanned drive-through robot cafe service system according to one embodiment of the present invention.

[0056] FIG. 4 is a diagram illustrating an example configuration of a voice prompt device and a robot cafe device in an ultra-large artificial intelligence-based unmanned drive-through robot cafe service system according to one embodiment of the present invention.

[0057] FIG. 5 is a drawing illustrating a beverage manufacturing process of an unmanned robot in a robot cafe device of an ultra-large artificial intelligence-based unmanned drive-through robot cafe service system according to one embodiment of the present invention.

[0058] FIG. 6 is a diagram illustrating a process of processing a voice conversation order in a voice prompt device of an ultra-large artificial intelligence-based unmanned drive-thru robot cafe service system according to one embodiment of the present invention.

[0059] FIG. 7 is a diagram illustrating an example of a response from an ultra-large language model of an ultra-large artificial intelligence-based unmanned drive-thru robot cafe service system according to one embodiment of the present invention.

[0060] FIG. 8 is a diagram illustrating an example of the necessity of fine tuning of an ultra-large language model of an ultra-large artificial intelligence-based unmanned drive-thru robot cafe service system according to an embodiment of the present invention.

[0061] FIG. 9 is a diagram illustrating an example of a flow of fine tuning for dialogue customization of a language model of an ultra-large artificial intelligence-based unmanned drive-thru robot cafe service system according to an embodiment of the present invention.

[0062] FIG. 10 is a diagram illustrating an example of an interactive order automation service process utilizing a large-scale artificial intelligence of an unmanned drive-thru robot cafe service system based on a large-scale artificial intelligence according to one embodiment of the present invention.

[0063] FIG. 11 is a diagram illustrating an example of a process for generating a drive-through order language model of an ultra-large artificial intelligence-based unmanned drive-through robot cafe service system according to an embodiment of the present invention.

[0064] FIG. 12 is a diagram illustrating an example of a design of a conversational language model adaptive to various order situations of an ultra-large artificial intelligence-based unmanned drive-thru robot cafe service system according to one embodiment of the present invention.

[0065] <Explanation of symbols>

[0066] 100: A completely unmanned drive-through robot cafe service system according to one embodiment of the present invention.

[0067] 110: Vision sensor section

[0068] 111: First vision sensor

[0069] 112: Second Vision Sensor

[0070] 120: Voice prompt device

[0071] 121: Mike

[0072] 122: Noise removal and speaker separation

[0073] 123: Voice-to-Text Converter

[0074] 124: Order Model

[0075] 125: Text-to-speech conversion unit

[0076] 126: Speaker

[0077] 130: Robot Cafe Device

[0078] 140: Robot Service Platform

[0079] Below, with reference to the attached drawings, embodiments of the present invention are described in detail so that those skilled in the art can easily implement them. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, irrelevant parts have been omitted for clarity of description, and similar reference numerals have been used throughout the specification to indicate similar parts.

[0080]

[0081] Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected" but also the case where it is "indirectly connected" with another element in between. Furthermore, when a part is said to "include" a component, this should be understood to mean that, unless specifically stated to the contrary, it may include other components rather than excluding them, and does not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0082]

[0083] The following examples are provided as detailed explanations to aid understanding of the present invention and do not limit the scope of the invention. Therefore, inventions with the same scope and function as the present invention are also within the scope of the present invention.

[0084]

[0085] In addition, each configuration, process, procedure or method included in each embodiment of the present invention may be shared within a scope that is not technically inconsistent with each other.

[0086]

[0087] FIG. 1 is a diagram illustrating the configuration of an ultra-large artificial intelligence-based unmanned drive-through robot cafe service system according to an embodiment of the present invention as a functional block, and FIG. 2 is a diagram illustrating the configuration of a vision sensor unit of an ultra-large artificial intelligence-based unmanned drive-through robot cafe service system according to an embodiment of the present invention as a functional block. As illustrated in FIGS. 1 and 2, an ultra-large artificial intelligence-based unmanned drive-thru robot cafe service system (100) according to an embodiment of the present invention may include a vision sensor unit (110) that collects information on a vehicle entering and moving through a drive-thru to order a drink in an unmanned drive-thru environment, a voice prompt device (120) that receives a drink order through a voice conversation with a user riding in a vehicle collected from the vision sensor unit (110) using a voice prompt technology capable of human-level conversation, a robot cafe device (130) that produces a drink ordered through the voice prompt technology without human intervention through the voice prompt device (120) by a robot that performs a series of operation sequences of a preset drink recipe, and allows the user to make a payment and receive a drink from the robot while riding in the vehicle, and a robot service platform (140) that controls the overall operation of the vision sensor unit (110), the voice prompt device (120), and the robot cafe device (130). Hereinafter, with reference to the attached drawings, the specific configuration of an ultra-large artificial intelligence-based fully unmanned drive-through robot cafe service system according to one embodiment of the present invention will be described in detail.

[0088]

[0089] FIG. 3 is a diagram illustrating a service concept configuration of an unmanned drive-through robot cafe service system based on an ultra-large-scale artificial intelligence according to an embodiment of the present invention, FIG. 4 is a diagram illustrating an example configuration of a voice prompt device and a robot cafe device in an unmanned drive-through robot cafe service system based on an ultra-large-scale artificial intelligence according to an embodiment of the present invention, FIG. 5 is a diagram illustrating a beverage manufacturing process of an unmanned robot in a robot cafe device in an unmanned drive-through robot cafe service system based on an ultra-large-scale artificial intelligence according to an embodiment of the present invention, and FIG. 6 is a diagram illustrating a voice conversation order processing process in a voice prompt device in an unmanned drive-through robot cafe service system based on an ultra-large-scale artificial intelligence according to an embodiment of the present invention.

[0090]

[0091] The vision sensor unit (110) is configured to collect information on vehicles entering and moving through the drive-thru for ordering beverages in an unmanned drive-thru environment. As illustrated in FIGS. 2 and 3, the vision sensor unit (110) may include a first vision sensor (111) that collects information on vehicles entering and moving through the drive-thru for ordering beverages in an unmanned drive-thru environment, and that recognizes vehicles passing through the entrance of the drive-thru through vision to collect vehicle identification information including vehicle number and vehicle type information, and a second vision sensor (112) that collects information on vehicles entering and moving through the drive-thru for ordering beverages in an unmanned drive-thru environment, and that recognizes vehicles moving before arriving at the robot cafe device (130) of a vehicle that has ordered a beverage from the voice prompt device (120) of the drive-thru and passed through, and that collects vehicle identification and vehicle passenger spatial information (location). Here, the vision sensor unit (110) can be implemented with AI-based machine vision technology to collect information on vehicles entering and moving through a drive-thru to order beverages in an unmanned drive-thru environment.

[0092]

[0093] In addition, the vision sensor unit (110) includes a first vision sensor (111) and a second vision sensor (112) for collecting information on vehicles entering and moving through the drive-thru to order beverages in an unmanned drive-thru environment, and vehicle identification information including vehicle number and vehicle type information collected through the first vision sensor (111) and the second vision sensor (112) and vehicle passenger spatial information (location) can be collected and transmitted to the robot service platform (140) to be described later. Here, the vehicle passenger spatial information collected by the second vision sensor (112) may be vehicle window position recognition information. That is, since the height of the driver's seat is different for each vehicle when picking up a manufactured beverage from a vehicle, the collected vehicle passenger spatial information can be utilized as information to provide a beverage according to the height of the driver's seat of the vehicle.

[0094]

[0095] In addition, the vision sensor unit (110) is composed of a first vision sensor (111) and a second vision sensor (112) for collecting information on vehicles entering and moving through a drive-thru to order beverages in an unmanned drive-thru environment, but is not limited thereto and may be increased or decreased to collect information on vehicles in an unmanned drive-thru environment.

[0096]

[0097] The voice prompt device (120) is configured to receive a beverage order through a voice conversation with a user riding in a vehicle, which is collected from the vision sensor unit (110) using a voice prompt technology capable of human-level conversation. This voice prompt device (120) can extract standardized customer order information through human-level general conversation using a super-large language model, and can function to enable a voice response corresponding to the customer's beverage order. Here, the voice prompt device (120) provides an order screen for a user riding in a vehicle to order a beverage, and can be implemented using voice processing, voice recognition, natural language processing, and super-large artificial intelligence (e.g., ChatGPT) technology of the user riding in the vehicle.

[0098]

[0099] In addition, as shown in FIG. 6, the voice prompt device (120) includes a microphone (121) into which a voice for ordering a drink of a user on board a vehicle is input, a noise removal and speaker separation unit (122) for removing noise from the voice for ordering a drink of a user on board a vehicle input through the microphone (121) and extracting speakers separately, a voice-to-text conversion unit (123) for converting the voice for ordering a drink from which noise has been removed and speakers have been separated from the noise removal and speaker separation unit (122) into a drink order text and outputting it, an order model (124) for receiving a drink order text from the voice-to-text conversion unit (123) and transmitting an order request corresponding to the drink order content to the robot cafe device (130) through the robot service platform (140), and generating a response text of the order situation, a text-to-speech conversion unit (125) for converting the response text of the order situation generated by the order model (124) into voice and outputting it, and a response voice of the order situation converted from the text-to-speech conversion unit (125) for outputting a voice to the user. It can be configured to include a speaker (126). Here, the order model (124) is a super-large artificial intelligence language model to which a voice prompt technology capable of human-level conversation is applied, and can function to provide a fine tuning service through reinforcement learning to a pre-trained general-purpose super-large artificial intelligence language model.

[0100]

[0101] The robot cafe device (130) is configured to manufacture a beverage ordered through voice prompt technology without human intervention via the voice prompt device (120) by a robot that performs a series of work sequences of a preset beverage recipe, and to enable a user to make payment and receive a beverage from the robot while in a vehicle. This robot cafe device (130) may be configured in the form of a booth in a drive-thru and may include a robot that manufactures a beverage by performing a series of work sequences of a preset beverage recipe for an ordered beverage, a payment means for performing payment for an ordered beverage while a driver is in the vehicle, and a beverage providing means for providing a beverage manufactured by the robot while in the vehicle based on spatial information of a vehicle passenger. Here, the beverage providing means may be configured with a plurality of delivery devices of different heights so that the beverage can be provided according to the collected vehicle passenger spatial information, i.e., the height of the driver's seat of the vehicle, since the height of the driver's seat varies for each vehicle when providing a manufactured beverage and picking it up from the vehicle.

[0102]

[0103] In addition, the robot cafe device (130) may be equipped with various beverage production equipment such as a multi-joint robot, a coffee machine, an ice maker, a syrup supply device, and a carbonated water supply device according to the recipe of the ordered beverage, and may perform payment and beverage provision services through the robot while in the vehicle using the spatial information of the vehicle occupant. It can be understood that such a robot cafe device (130) applies various food tech robots for manufacturing beverages, paying for costs, and removing manufactured beverages.

[0104]

[0105] The robot service platform (140) is a configuration that controls the overall operation of the vision sensor unit (110), the voice prompt device (120), and the robot cafe device (130). This robot service platform (140) stores and manages vehicle identification information and vehicle passenger spatial information (location) collected from the vision sensor unit (110), receives and stores and manages beverage order reception information from the voice prompt device (120), and stores and manages beverage information, payment information, and beverage provision information prepared from the robot cafe device (130). Here, the robot service platform (140) may be configured as a server of Hyper-scale AI (CHatGPT4.O) for overall operation management of an ultra-large artificial intelligence-based unmanned drive-through robot cafe service system, and Raas (Robot as a Service) platform HappyBones2.0 including Computer vision OpenCV. In other words, it can function as a service model that provides services to consumers in the form of subscriptions by including a robot hardware platform that includes hardware that interacts with robots, and a robot software platform that includes intelligent Internet of Things technology that automatically controls hardware infrastructure for service purposes.

[0106]

[0107] Hereinafter, an example of the implementation of a large-scale language model implemented in a voice prompt device (120) of an unmanned drive-through robot cafe service system based on large-scale artificial intelligence according to one embodiment of the present invention will be described.

[0108]

[0109] FIG. 7 is a diagram illustrating an example of a response from a super-large language model of an unmanned drive-through robot cafe service system based on a super-large AI according to an embodiment of the present invention, FIG. 8 is a diagram illustrating an example of the necessity of fine-tuning a super-large language model of an unmanned drive-through robot cafe service system based on a super-large AI according to an embodiment of the present invention, and FIG. 9 is a diagram illustrating an example of a flow of fine-tuning for dialogue customization of a language model of an unmanned drive-through robot cafe service system based on a super-large AI according to an embodiment of the present invention. As illustrated in FIGS. 7 to 9, existing conversational language AI has a limitation in that it provides human language dialogue within a controlled environment (a dialogue scenario designed by researchers). For example, Samsung's virtual assistant Bixby is integrated into Samsung's Galaxy smartphones, tablets, and other devices, and can only understand a limited number of topics for using Samsung product functions such as setting alarms, controlling smart home devices, and making calls. On the other hand, the order model (124) applied to the voice prompt device (120) of the present invention is a super-large AI language model, so that in a drive-thru (DT) that is operated completely unmanned without human intervention, the conversational language AI can understand the meaning of the context of any question and provide an appropriate answer to the customer. Here, the order model (124) of the voice prompt device (120) utilizes a commercial super-large language model AI (such as Microsoft's ChatGPT, Google's Bard, or Naver's HyperCLOVA X), but fine tuning is applied because it does not provide the answer required for the robot cafe service model of the present invention. For example, fine tuning is required to suit the purpose of the service model so that it can respond to a customer's request, "Please give me a cup of hot Americano." with "What size drink would you like?"

[0110]

[0111] Furthermore, existing conversational language AI (Samsung Bixby, Apple Siri) has limitations in that it can only provide responses within controlled environments (conversation scenarios designed by researchers), and is therefore being utilized as a way to implement services for specific companies. Recently, large AI language models (e.g., Microsoft's ChatGPT, Google's Bard) that provide the ability to provide human-level conversational responses have been able to overcome the limitations of existing conversational language AI. In particular, reinforcement learning on pre-trained general-purpose large AI language models (LLMs) has enabled the utilization of large AI language models in various environments. For example, commercial large AI language models (LARs) can provide fine-tuning services to implement human-level AI waiters capable of providing conversations related to food and beverage ordering in any situation.

[0112]

[0113] FIG. 10 is a diagram illustrating an example of a conversational order automation service process utilizing a super-large artificial intelligence of an unmanned drive-through robot cafe service system based on a super-large artificial intelligence according to an embodiment of the present invention, FIG. 11 is a diagram illustrating an example of a drive-through order language model generation process of an unmanned drive-through robot cafe service system based on a super-large artificial intelligence according to an embodiment of the present invention, and FIG. 12 is a diagram illustrating an example of a design of a conversational language model adaptive to various order situations of an unmanned drive-through robot cafe service system based on a super-large artificial intelligence according to an embodiment of the present invention. In an unmanned drive-thru (DT) environment, the order automation service can be implemented as shown in Fig. 10 using noise filtering H / W and S / W technology, speech recognition (STT; Speech-to-Text) that recognizes the voice of the orderer and converts it into text, a drive-thru conversational ordering model based on a super-large AI (the machine understands a person's natural language conversation script, infers the context of the conversation script, and provides an appropriate answer to the orderer or executes the orderer's food and beverage request), and technology that converts the response text generated by a super-large AI into voice (TTS; Text-to-Speech).

[0114]

[0115] Also, as shown in Fig. 11, the process of creating a ChatGPT-based drive-thru (DT) ordering language model uses the Ada, Babbage, Curie, and Davinci models of OpenAI (ChatGPT), and processes the drive-thru automatic ordering script format data in JSON (JavaScript Object Notation) format for OpenAI fine tuning. Here, fine tuning refers to a method of transforming the architecture based on an existing trained AI model to a new purpose and training by finely adjusting the weights of an already trained model. In addition, the drive-thru automatic ordering script can be configured in the form of a prompt (user ordering command) and completion (order confirmation and order confirmation).

[0116]

[0117] Furthermore, as illustrated in Figure 12, in the conversational language model adapting to various order situations, GPT learning generates various conversational scripts that take into account the language generation model that completes sentences rather than questions and answers, considering the drive-thru order situation. In other words, by applying the company's general script, rule-based script, and actual order situation analysis script, natural order scripts are generated in the drive-thru environment, drive-thru model training is performed, and the conversational language model can be designed by analyzing and systematizing exceptional situations through user feedback verification. In other words, in order to train a natural conversation generation model in the drive-thru environment, a training script can be created (a training dataset for fine tuning ChatGPT or bard's ultra-large language model) based on three automatic order forms. Based on these three types of scripts, which are 1) generating scripts based on a universal app-based ordering protocol, 2) analyzing possible scenarios that may occur in offline stores to generate various scripts suitable for ordering situations, and 3) monitoring actual ordering situations and analyzing situations to generate scripts by considering exceptional situations that may occur in ordering situations, a natural ordering script that can occur in a drive-thru situation can be generated. In this way, the completed natural ordering script can be used as learning data to design a GPT drive-thru ordering conversation model, and the completed ordering conversation model can be verified through a feedback process with users, and through the feedback, exceptional situations of the more natural script can be supplemented, the ordering conversation generation model can be advanced, and by analyzing and systematizing unexpected exceptional situations, it can be designed as a real-time conversation system capable of dealing with conversational ordering services in complex situations.

[0118]

[0119] As described above, the ultra-large artificial intelligence-based unmanned drive-thru robot cafe service system according to one embodiment of the present invention comprises a vision sensor unit that collects information on a vehicle entering and moving through a drive-thru to order a drink in an unmanned drive-thru environment, a voice prompt device that receives a drink order through a voice conversation with a user riding in a vehicle collected from the vision sensor unit using a voice prompt technology capable of human-level conversation, a robot cafe device that manufactures a drink ordered through the voice prompt technology without human intervention by a robot that performs a series of operation sequences of a preset drink recipe and allows the user to make a payment and receive a drink from the robot while riding in the vehicle, and a robot service platform that controls the overall operation of the vision sensor unit, the voice prompt device, and the robot cafe device, so that the unmanned robot manufactures a drink ordered by a driver riding in the vehicle and provides it to the driver riding in the vehicle, and the cafe service can be provided by receiving an order through a conversational voice without human intervention in a drive-thru environment, and in particular, the driver riding in the vehicle orders a drink through a conversational voice without human intervention in a drive-thru environment, and the ordered drink is served by a multi-joint robot. By manufacturing according to the work order using AI and providing drinks to drivers who board the vehicle after payment, it is possible to create quality jobs by reducing the intensity of labor by replacing customer service and simple repetitive tasks with AI and robots from the worker's perspective, and from the business owner's perspective, it is possible to not only resolve the labor shortage but also improve productivity, reduce costs, and increase employment efficiency.

[0120]

[0121] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single entity may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined manner.

[0122]

[0123] The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

Claims

1. As a super-large artificial intelligence-based fully unmanned drive-through robot cafe service system (100), A vision sensor unit (110) that collects information on vehicles entering and moving through a drive-thru to order beverages in an unmanned drive-thru environment; A voice prompt device (120) that receives a beverage order through a voice conversation with a user riding in a vehicle, which is collected from the vision sensor unit (110), using voice prompt technology capable of human-level conversation; A robot cafe device (130) that allows a user to receive payment and a drink from a robot while in a vehicle, and in which a drink ordered through voice prompt technology without human intervention through the voice prompt device (120) is manufactured by a robot that performs a series of operation sequences of a preset drink recipe; and Including a robot service platform (140) that controls the overall operation of the vision sensor unit (110), voice prompt device (120), and robot cafe device (130), The above vision sensor unit (110) A system for collecting information on vehicles entering and moving through a drive-thru for ordering beverages in an unmanned drive-thru environment, comprising: a first vision sensor (111) for collecting vehicle identification information including vehicle number and vehicle type information by recognizing a vehicle moving toward the entrance of the drive-thru with vision; and a second vision sensor (112) for collecting information on vehicles entering and moving through a drive-thru for ordering beverages in an unmanned drive-thru environment, comprising: a second vision sensor (112) for collecting vehicle identification and vehicle occupant spatial information (location) by recognizing a vehicle moving before arriving at a robot cafe device (130) of a vehicle that has ordered a beverage from a voice prompt device (120) of the drive-thru and has passed through; The above voice prompt device (120) is It uses a super-large language model to extract standardized customer order information through human-level general conversation and enables voice response to customers' beverage orders. The above voice prompt device (120) is A system characterized by comprising: a microphone (121) for inputting a voice for ordering a drink from a user on board a vehicle; a noise removal and speaker separation unit (122) for removing noise from the voice for ordering a drink from a user on board a vehicle input through the microphone (121) and extracting speakers; a voice-to-text conversion unit (123) for converting the voice for ordering a drink from which noise has been removed and speakers have been separated by the noise removal and speaker separation unit (122) into a drink order text and outputting it; an order model (124) for receiving the drink order text from the voice-to-text conversion unit (123) and transmitting an order request corresponding to the drink order content to the robot cafe device (130) through the robot service platform (140) and generating a response text of the order situation; a text-to-speech conversion unit (125) for converting the response text of the order situation generated by the order model (124) into voice and outputting it; and a speaker (126) for outputting the response voice of the order situation converted by the text-to-speech conversion unit (125) as a voice to the user. A super-large artificial intelligence-based, fully unmanned drive-thru robot cafe service system.

2. In paragraph 1, the vision sensor unit (110) A fully unmanned drive-thru robot cafe service system based on an ultra-large artificial intelligence, comprising a first vision sensor (111) and a second vision sensor (112) for collecting information on a vehicle entering and moving through a drive-thru for ordering a beverage in an unmanned drive-thru environment, wherein vehicle identification information including a vehicle number and vehicle type information collected through the first vision sensor (111) and the second vision sensor (112) and vehicle passenger spatial information (location) are collected and transmitted to the robot service platform (140).

3. In the first paragraph, the order model (124) is A fully unmanned drive-through robot cafe service system based on a large-scale artificial intelligence, characterized by the ability to provide a fine tuning service through reinforcement learning to a pre-trained general-purpose large-scale artificial intelligence language model, which is a large-scale artificial intelligence language model that is capable of human-level conversation.

4. In the first paragraph, the robot service platform (140) A super-large artificial intelligence-based unmanned drive-through robot cafe service system characterized by storing and managing vehicle identification information and vehicle passenger spatial information (location) collected from the vision sensor unit (110), receiving and storing beverage order reception information from the voice prompt device (120), and storing and managing beverage information, payment information, and beverage provision information prepared from the robot cafe device (130).

Images