Order support device, order support method, and program

Abstract

Traditionally, ordering products in stores required a cumbersome and unappealing input process using tablets or smartphones. [Solution] An order support device 1 comprises an avatar storage unit 111 that stores avatar information, an avatar output unit 141 that outputs an avatar constructed using the avatar information, an order receiving unit 121 that receives voice order information indicating an order from the user, a voice response unit 132 that acquires response information indicating a response in accordance with the order information received by the order receiving unit 121 and outputs the response information as voice, and an order processing unit 133 that performs order processing, which is the process for proceeding with the order using the order information received by the order receiving unit 121. This allows for voice orders to be accepted while enjoying a conversation with an avatar.

Description

Technical Field

[0001] The present invention relates to an order support device and the like that support a user's order.

Background Art

[0002] Conventionally, there has been a tablet terminal used as a food and beverage order terminal (see Patent Document 1).

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art, when ordering goods at a store, it was necessary to perform a cumbersome and uninteresting input operation using a tablet terminal or the like.

Means for Solving the Problems

[0005] The order support device of the first invention of the present invention includes an avatar storage unit that stores avatar information, an avatar output unit that outputs an avatar configured using the avatar information, an order reception unit that receives voice order information indicating an order from a user, a voice response unit that acquires response information indicating a response corresponding to the order information received by the order reception unit and outputs the response information as voice, and an order processing unit that performs order processing, which is processing for advancing the order, using the order information received by the order reception unit.

[0006] With such a configuration, it is possible to receive a voice order while enjoying a conversation with the avatar.

[0007] Furthermore, in contrast to the first invention, the order support device of this second invention is an order support device in which the order processing unit performs order processing and transmits a user order based on order information to a kitchen terminal or employee terminal, which is a terminal in the kitchen that prepares the products corresponding to the order.

[0008] This configuration allows avatars to handle order taking, thereby reducing the workload of store employees.

[0009] Furthermore, the order support device of this third invention differs from the first invention in that the order receiving unit sequentially receives order information for two or more products, and the order processing unit performs order processing by outputting a user order, which is a compilation of the order information for two or more products, to the store staff via voice output.

[0010] This configuration allows avatars to handle order taking, thereby reducing the workload of store employees.

[0011] Furthermore, the order support device of this fourth invention, compared to the third invention, is an order support device in which the order processing unit performs a request output process to output request information for calling a waiter to the user's seat, and after detecting that the waiter has arrived at the seat, performs a voice output process to output the user's order to the waiter by voice.

[0012] This configuration allows avatars to handle order taking, reducing the workload for store staff while users are deciding which items to order.

[0013] Furthermore, the order support device of the fifth invention further comprises a determination unit that determines whether or not the confirmation condition, which is a condition for identifying that the receipt of order information from a user has been completed, is met with respect to any one of the first to fourth inventions; the voice response unit outputs an order confirmation voice, which is a voice that confirms the completion of the order, when the determination unit determines that the confirmation condition has been met; and the completion reception unit further comprises a completion reception unit that receives order completion information in the form of a voice indicating the completion of the order, in response to the output of the order confirmation voice; and the order processing unit is an order support device that processes the order when the completion reception unit has received the order completion information.

[0014] This configuration allows for order processing at the appropriate time.

[0015] Furthermore, the order support device of the sixth invention, compared to the fifth invention, is an order support device comprising: a voice response unit which performs voice recognition processing on order information for each order of one or more products and obtains an order string; a response acquisition unit which passes the order string to a generation AI and obtains a response from the generation AI; and a voice output unit which outputs response information based on the response as voice.

[0016] With this configuration, it is possible to use a generating AI to accept voice orders for each ordered product, while customers enjoy interacting with an avatar.

[0017] Furthermore, the order support device of the seventh invention, compared to the fifth invention, is an order support device comprising: a voice response unit comprising a voice recognition means that performs voice recognition processing on order information and obtains an order string; a response acquisition means that passes the order string to a generation AI and obtains a response from the generation AI; and a voice output means that, when the completion reception unit receives order completion information, outputs response information based on the response as voice.

[0018] With this configuration, a generating AI can be used to output the voice of an avatar based on the confirmed order after the order has been completed.

[0019] Furthermore, the order support device of the eighth invention is an order support device that, in addition to any one of the first to seventh inventions, further comprises an order output unit that outputs a user order based on the order information each time the order receiving unit receives order information indicating an order for one or more products.

[0020] This configuration allows users to check the items they have ordered for each product.

[0021] Furthermore, the order support device of this ninth invention, compared to the eighth invention, is an order support device in which the order output unit displays the user order or outputs the user order as voice.

[0022] With such a configuration, the user can check the ordered products for each product.

[0023] Moreover, the order support device of the tenth invention, compared with the first invention, further includes: an order reception unit that receives voice-based order information including product-specific information that does not include a product identifier, performs voice recognition processing on the order information, and obtains an order string; an answer acquisition unit that passes the order string to a generation AI and obtains an answer having one or more product identifiers from the generation AI; and a proposal unit that outputs one or more product identifiers included in the answer. The order processing unit is an order support device that performs order processing using one or more product identifiers output by the proposal unit.

[0024] With such a configuration, it is possible to receive product-specific information that does not include a product identifier and propose products based on the product-specific information.

[0025] Moreover, the order support device of the eleventh invention, compared with the tenth invention, further includes a permission reception unit that receives permission for one or more product identifiers output by the proposal unit. The order processing unit is an order support device that performs order processing using one or more product identifiers output by the proposal unit when the permission reception unit receives permission.

[0026] With such a configuration, order processing can be performed only when permission from the user for the product proposal is received.

[0027] Moreover, the order support device of the twelfth invention, compared with the tenth or eleventh invention, further includes a menu management unit that stores menu information which is a set of two or more products to be ordered. The answer acquisition unit is an order support device that passes the order string and the menu information to a generation AI and obtains an answer from the generation AI.

[0028] With such a configuration, it is possible to receive product-specific information that does not include a product identifier and propose products based on the product-specific information.

[0029] Furthermore, the order support device of the thirteenth invention is an order support device in which, with respect to any one of the tenth to twelfth inventions, the product identification information includes one or more types of information from among product attribute information, user attribute information, or recommendation information that means recommendations for recommended products.

[0030] With this configuration, the system can receive product identification information that includes one or more types of information, such as product attribute information, user attribute information, or recommendation information that means recommendations for products, and can propose products based on said product identification information. [Effects of the Invention]

[0031] According to the order support device of the present invention, it is possible to accept orders by voice while enjoying a conversation with an avatar. [Brief explanation of the drawing]

[0032] [Figure 1] Conceptual diagram of information system A in Embodiment 1 [Figure 2] Block diagram of information system A [Figure 3] Flowchart illustrating an example of the operation of the order support device 1. [Figure 4] A flowchart illustrating an example of the confirmation and judgment process. [Figure 5] A flowchart illustrating an example of the order processing. [Figure 6] A flowchart illustrating an example of the automated order processing process. [Figure 7] A diagram showing an example of the same avatar information. [Figure 8] Figure showing an example of the same output. [Figure 9] A diagram showing an example of a response from the AI. [Figure 10] This diagram shows an example of the same default order template. [Figure 11] A diagram showing an example of the same menu information. [Figure 12] This diagram shows an example of the automated order prompt. [Figure 13]A diagram showing an example of a response from the AI. [Figure 14] This figure shows an example of the output of the proposed section 143. [Figure 15] Block diagram of the computer system [Modes for carrying out the invention]

[0033] The following describes embodiments of the order support device and the like with reference to the drawings. Note that components denoted by the same reference numerals in the embodiments perform similar operations, and therefore, further explanation may be omitted.

[0034] (Embodiment 1) This embodiment describes an order support device that receives voice orders from users, has an avatar respond to the received orders via voice, and outputs information about the ordered products. The output of information about the ordered products is, for example, transmitted to a store employee terminal or a kitchen terminal.

[0035] In this embodiment, we will describe an order support device that outputs instructions to call a waiter to the customer's table once the user has finished placing their order.

[0036] In this embodiment, we will describe an order support device that confirms the completion of an order with the user when the confirmation conditions are met, and processes the order after the completion of the order has been confirmed.

[0037] In this embodiment, an order support device that outputs order details for each product ordered by the user will be described.

[0038] In this specification, information X being associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is irrelevant. Information X and information Y may be linked, may reside in the same buffer, may information X be contained in information Y, or information Y may be contained in information X, and so on.

[0039] Furthermore, in this specification, selecting or determining information Z means obtaining information Z, obtaining a pointer to information Z, obtaining the ID of information Z, setting a flag on information Z, etc., and it is sufficient to be able to access information Z.

[0040] Figure 1 is a conceptual diagram of information system A in this embodiment.

[0041] Information system A comprises one or more order support devices 1, one or more generation AI devices 2, one or more kitchen terminals 3, and one or more staff terminals 4.

[0042] It is preferable that the order support device 1 and the generation AI device 2, the order support device 1 and the kitchen terminal 3, and the order support device 1 and the employee terminal can communicate with each other via a network such as the Internet.

[0043] Order support device 1 is a device that assists users with placing orders. Order support device 1 is usually a terminal, but it can also be a server.

[0044] If the order support device 1 is a terminal, it may be a personal computer, tablet, smartphone, etc., but the type is not limited. If the order support device 1 is a terminal, it is usually installed in the store. For example, the order support device 1 may be installed at one or more seats in the store. The order support device 1 may also be a terminal owned by a user in the store. Note that a store is a place that sells goods. A store may be a bar, restaurant, etc., but the type is not limited.

[0045] If the order support device 1 is a server, it may be, for example, a cloud server, an ASP server, etc., but the type is not specified. If the order support device 1 is a server, it receives instructions and information from terminal devices (not shown) and transmits information to terminal devices (not shown). In such a case, terminal devices are installed, for example, at one or more seats in the store. Terminal devices may also be, for example, terminals owned by users in the store.

[0046] The Generating AI Device 2 is a device that has the functionality of a Generating AI. In this context, Generating AI Device 2 typically has the functionality of a text generation AI. Generating AI Device 2 may also be a device that receives voice information. Generating AI Device 2 may also be a device that outputs voice information. The Generating AI may be, for example, ChatGPT or Google Bard, but is not limited to that. Note that Google is a registered trademark. Generating AI Device 2 may be, for example, a cloud server or an ASP server, but is not limited to that type. Generating AI Device 2 will be referred to as Generating AI as appropriate.

[0047] Kitchen terminal 3 is a terminal used by those preparing goods (e.g., food). Kitchen terminal 3 is, for example, a terminal installed in the kitchen. Staff terminal 4 is a terminal used by staff members. Kitchen terminal 3 and staff terminal 4 can be, for example, so-called personal computers, tablet devices, smartphones, etc., but the type is not limited.

[0048] Figure 2 is a block diagram of information system A in this embodiment. The order support device 1, which constitutes information system A, includes a storage unit 11, a reception unit 12, a processing unit 13, and an output unit 14.

[0049] The storage unit 11 includes an avatar storage unit 111, a prompt template management unit 112, and a menu management unit 113. The reception unit 12 includes an order reception unit 121, a completion reception unit 122, and a permission reception unit 123. The processing unit 13 includes a judgment unit 131, a voice response unit 132, and an order processing unit 133. The voice response unit 132 includes a voice recognition means 1321, a response acquisition means 1322, and a voice output means 1323. The output unit 14 includes an avatar output unit 141, an order output unit 142, and a suggestion unit 143.

[0050] The storage unit 11, which constitutes the order support device 1, stores various types of information. These types of information include, for example, avatar information (described later), prompt templates (described later), and various conditions. It goes without saying that these conditions may also be embedded in the program. These conditions include, for example, confirmation conditions (described later).

[0051] The avatar storage unit 111 stores avatar information. The avatar information includes information for constructing the output avatar. The avatar information is associated with, for example, an avatar identifier. Associating with an avatar identifier may also mean having an avatar identifier. The avatar identifier may be, for example, the avatar's ID or the avatar's name.

[0052] Avatar information may include, for example, model information. Avatar information may also include still images or videos of the avatar. Preferably, avatar information includes avatar motion information, which is information about the avatar's motion.

[0053] Model information is the information used to construct the display of an avatar. Model information may include, for example, mesh information, bone information, and material information. Model information may have a data structure such as glTF (GL Transmission Format). However, the data structure of the model information may also be VRM, OBJ, FBX, STL, GLB, COLLADA, etc., and is not limited to any particular type. In addition to model information, avatar information may also include audio information and one or more attribute values.

[0054] Avatar motion information is, for example, information that identifies the movements an avatar makes when it breathes. Avatar motion information is information about the fixed motions that an avatar possesses.

[0055] The prompt template management unit 112 stores one or more prompt templates.

[0056] A prompt template is a template for a prompt given to the generating AI. Prompt templates usually have one or more variables, but prompts without variables are also acceptable. If a prompt template has variables, information is assigned to those variables to complete the prompt. An example of a prompt template is, "You are a bar employee. Please attend to the customer." and "The following was ordered: <Speech Recognition Result>". Note that "You are a bar employee. Please attend to the customer." is a prompt without variables. <Speech Recognition Result> is the variable to which the speech recognition result is assigned.

[0057] A prompt template is, for example, an automated order template. An automated order template is a prompt template used when performing the automated order processing described later. An automated order template has, for example, a variable to which menu information is assigned. An automated order template has, for example, a variable to which the order string is assigned.

[0058] The menu management unit 113 stores menu information. Menu information is a set of information for two or more items that are ordered. Item information is information about a product. Item information usually has an item identifier. Item information may also have a pair of item identifiers and item attribute information. Item attribute information is information that indicates the attributes of a product. Examples of item attribute information include "healthy," "rich," "calorie count," "item type," and "item price." Item identifier is information that identifies a product. In this case, the item identifier is usually the product name, but it may also be the item ID.

[0059] The reception unit 12 receives various instructions and information. These instructions and information include, for example, order information and order completion information, which will be described later.

[0060] The means of inputting various instructions and information is preferably a microphone. Other input methods such as a touch panel, keyboard, mouse, or menu screen are also acceptable.

[0061] The reception unit 12 may be implemented using a device driver for an input means such as a microphone.

[0062] The order receiving unit 121 receives order information. It is preferable for the order receiving unit 121 to sequentially receive order information for two or more products.

[0063] Order information refers to voice information relating to an order from a user. Order information may include, for example, a product identifier and quantity. Order information may also include, for example, product identification information. Product identification information is information that does not include a product identifier but is able to identify a product. Product identification information may include, for example, one or more types of information such as product attribute information, user attribute information, or recommendation information. User attribute information is information that indicates the attributes of the user. User attribute information may include, for example, the number of people, occupation, user preferences (e.g., likes rich flavors, likes light flavors), and user inclinations (e.g., budget-conscious, luxury-conscious). Recommendation information is information that means a recommendation of a recommended product. Recommendation information may be, for example, "What do you recommend today?" The order receiving unit 121 may also accept voice information from the user other than product orders. Furthermore, "product" can be considered to include transferable items and services. Transferable items are usually items sold for a fee, but free transfers do not need to be excluded.

[0064] The completion reception unit 122 receives order completion information. The completion reception unit 122 receives order completion information, for example, in response to the output of an order confirmation voice message. Order completion information is information indicating that the order has been completed. Order completion information is, for example, voice information based on the user's voice. Order completion information is, for example, "That's all," "Yes, that's all," or "I'm closing the order."

[0065] The order completion reception unit 122 may receive order completion information by receiving the order completion button, etc.

[0066] The permission receiving unit 123 accepts permission for one or more product identifiers output by the proposal unit 143. Permission is "information indicating that it is acceptable to order one or more product identifiers output by the proposal unit 143." The permission receiving unit 123 usually accepts permission via voice from the user, but may also accept permission by pressing an "Permission" button, or by inputting information indicating permission.

[0067] The processing unit 13 performs various processes. These processes include, for example, those performed by the decision unit 131, the voice response unit 132, or the order processing unit 133.

[0068] The determination unit 131 determines whether or not the confirmation conditions are met. Normally, the determination unit 131 determines whether or not the order information received by the order receiving unit 121 meets the confirmation conditions.

[0069] Confirmation conditions are those that identify when the receipt of order information from a user has been completed. Confirmation conditions are those that allow for the confirmation of a user's order. Confirmation conditions are conditions based on the content of the information received by the order receiving unit 121, and conditions related to the time it takes for the order receiving unit 121 to receive the information. For example, confirmation conditions include the receipt of order completion information, or the absence of order information for a period of time equal to or longer than the threshold time.

[0070] The voice response unit 132 acquires response information corresponding to the order information received by the order acceptance unit 121. Next, the voice response unit 132 outputs the response information as voice.

[0071] Response information refers to information that responds to order information. Response information is output as audio. Typically, response information is output as the avatar's voice.

[0072] The voice response unit 132 obtains response information corresponding to the order information, for example, using a generation AI. The voice response unit 132 also obtains response information corresponding to the order information, for example, using a so-called chatbot. Since chatbots are publicly known technology, a detailed explanation is omitted.

[0073] It is preferable for the voice response unit 132 to output an order confirmation voice message when the determination unit 131 determines that the confirmation conditions have been met.

[0074] An order confirmation voice message is an audio message that confirms the completion of an order. For example, an order confirmation voice message might say, "That's all. I'll confirm your order..."

[0075] The voice response unit 132 outputs response information as voice, for example, using voice recognition means 1321, response acquisition means 1322, and voice output means 1323.

[0076] The voice response unit 132, for example, passes the order information, which is voice information received by the order receiving unit 121, to a generating AI capable of receiving voice information, and obtains a response from the generating AI. Next, the voice response unit 132, for example, uses the response to obtain and output response information, which is voice information.

[0077] The voice response unit 132, for example, passes the user order, which is a string based on the result of voice recognition of the order information received by the order receiving unit 121, to the generation AI, and obtains response information, which is a string, from the generation AI.

[0078] The voice response unit 132, for example, passes the order information, which is voice information received by the order receiving unit 121, to a generating AI capable of receiving voice information, obtains response information, which is voice information, from the generating AI, and outputs it.

[0079] The speech recognition means 1321, which constitutes the voice response unit 132, performs speech recognition processing on the order information received by the order receiving unit 121 and obtains an order string. It is preferable for the speech recognition means 1321 to perform speech recognition processing on the order information for each order of one or more products and obtain an order string for each order of one or more products. The order string usually includes a product identifier, but it may also be a string that does not include a product identifier. Examples of strings that do not include a product identifier include "the number of users and their mood" and "product attribute information of products that the user likes".

[0080] The response acquisition means 1322, for example, passes the order string acquired by the speech recognition means 1321 to the generation AI, and obtains a response from the generation AI.

[0081] The response acquisition means 1322, for example, for each order of one or more products, passes the order string acquired by the voice response unit 132 to the generating AI and obtains a response from the generating AI. Such an order string contains information about an order for one product.

[0082] The response acquisition means 1322, for example, passes an order string, which is the result of speech recognition processing for a series of order information from the start to the end of the order, to a generating AI, and obtains a response from the generating AI. Such an order string contains information about an order for one or more products.

[0083] The audio output means 1323 outputs response information based on the response obtained by the response acquisition means 1322. The response information is usually the audio information of the response. The response information may also be the audio information of a part of the response.

[0084] The voice output means 1323 outputs response information based on the answer when, for example, the completion reception unit 122 receives order completion information.

[0085] The order processing unit 133 processes the order using the order information received by the order receiving unit 121. It is preferable for the order processing unit 133 to process the order when the completion receiving unit 122 receives order completion information. When the completion receiving unit 122 receives order completion information, it usually means that the user's order has been confirmed.

[0086] Order processing refers to the process of moving an order forward. Examples of order processing include order submission, payment processing, voice output processing, and request output processing.

[0087] The order processing unit 133 performs order transmission processing. This processing includes, for example, transmitting a user order based on order information to a kitchen terminal 3, which is a terminal in the kitchen that prepares the products corresponding to the order. This processing also includes, for example, transmitting a user order based on order information to a staff terminal 4.

[0088] A user order is information that identifies a user's order. A user order is usually a string. Preferably, a user order is information containing a product identifier and quantity. Preferably, a user order is structured information containing a product identifier and quantity. A user order may be the same information as the order string, but preferably it is information derived from the order string.

[0089] The order processing unit 133 performs payment processing. This processing involves using order information to charge the user. The content of the payment processing is not specified.

[0090] The order processing unit 133 performs a request output process. This process outputs request information, which is information for calling a waiter to the user's seat. This waiter is a person who hears the user's order via voice from the avatar and transmits the user's order information to the kitchen terminal 3 or relays the order to the kitchen using the waiter terminal 4.

[0091] The order processing unit 133 performs voice output processing. This processing involves voice outputting a user order, which combines the order information for two or more items, to the waiter who has come to the customer's table. It is preferable that the order processing unit 133 performs voice output processing to voice output the user order to the waiter after detecting that the waiter has arrived at the customer's table.

[0092] The output unit 14 outputs various types of information. These types of information include, for example, avatars and user orders.

[0093] Here, output refers to display on a screen or audio output, but it may also be a concept that includes projection using a projector, printing with a printer, transmission to an external device, storage on a recording medium, and transfer of processing results to other processing devices or other programs.

[0094] The avatar output unit 141 outputs an avatar configured using the avatar information. For example, when the order support device 1 detects a user, the avatar output unit 141 outputs an avatar configured using the avatar information in the avatar storage unit 111.

[0095] The order output unit 142 outputs user orders based on the order information received by the order receiving unit 121. It is preferable that the order output unit 142 outputs user orders based on the order information each time the order receiving unit 121 receives order information indicating orders for one or more products.

[0096] Here, the order output unit 142 displays the user order based on the order information, or outputs the user order based on the order information as voice.

[0097] The suggestion unit 143 outputs one or more product identifiers included in the response obtained by the response acquisition means 1322. The suggestion unit 143 may also output the quantity associated with each of the one or more product identifiers. The suggestion unit 143 may, for example, output one or more product identifiers to a display. The suggestion unit 143 may, for example, output one or more product identifiers as audio.

[0098] The storage unit 11, avatar storage unit 111, prompt template management unit 112, and menu management unit 113 are preferably made of non-volatile recording media, but can also be made of volatile recording media.

[0099] The process by which information is stored in the storage unit 11, etc. is not relevant. For example, information may be stored in the storage unit 11, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 11, etc., or information input via an input device may be stored in the storage unit 11, etc.

[0100] The reception unit 12, order reception unit 121, completion reception unit 122, and authorization reception unit 123 can be implemented using device drivers for input means such as microphones, touch panels, and keyboards, or control software for menu screens.

[0101] The processing unit 13, the decision unit 131, the voice response unit 132, the order processing unit 133, the voice recognition means 1321, and the response acquisition means 1322 can typically be implemented using a processor, memory, etc. The processing procedures of the processing unit 13, etc., are typically implemented in software, and this software is recorded on a recording medium such as ROM. However, it may also be implemented in hardware (dedicated circuitry). The processor can be a CPU, MPU, GPU, etc., and the type is not limited.

[0102] The output unit 14, including the avatar output unit 141, the order output unit 142, the audio output means 1323, and the suggestion unit 143, may or may not include output devices such as displays and speakers. The output unit 14 can be implemented using driver software for an output device, or driver software for an output device and an output device.

[0103] Next, an example of the operation of the order support device 1 will be explained using the flowchart in Figure 3.

[0104] (Step S301) The processing unit 13 determines whether or not a user has been detected. If a user is detected, the process proceeds to step S302; otherwise, it returns to step S301.

[0105] The processing unit 13 does not have a specific method for detecting the user. The processing unit 13 may detect the user in an image captured by the order support device 1, detect a touch from the user to the order support device 1, detect the user's voice information collected by a microphone, or detect the user using a sensor installed at the seat (e.g., an infrared sensor, a laser).

[0106] (Step S302) The avatar output unit 141 acquires the avatar information from the avatar storage unit 111. The avatar output unit 141 uses the avatar information to construct an avatar and outputs the avatar. Note that the process of constructing an avatar using the avatar information may also be the process of reading the avatar information, which is an image.

[0107] (Step S303) The voice response unit 132 obtains the initial response from the storage unit 11. The voice response unit 132 outputs the initial response as audio.

[0108] The initial response is the voice message spoken when a customer is seated. An example of an initial response might be, "Welcome! Thank you for choosing our restaurant today. Would you like to order something to drink? We can also recommend some dishes and snacks, so please feel free to ask."

[0109] Furthermore, the output unit 14 outputs an initial response as audio, making it appear as if the avatar is speaking. Additionally, it is preferable for the avatar output unit 141 to output an avatar that moves during speech.

[0110] (Step S304) The order receiving unit 121 determines whether or not it has received voice order information from the user. If it has received the order information, it proceeds to step S305; otherwise, it proceeds to step S322.

[0111] (Step S305) The processing unit 13 obtains a user order, which is a string, from the voice order information. The user order includes a product identifier. Preferably, the user order includes a quantity.

[0112] (Step S306) The processing unit 13 determines whether or not it was able to obtain a user order in step S305. If it was able to obtain a user order, it proceeds to step S307; if it was not able to obtain a user order, it proceeds to step S311.

[0113] (Step S307) The order output unit 142 outputs the user order. The order output unit 142 outputs the user order on the screen, for example.

[0114] (Step S308) The processing unit 13 determines whether or not to respond to the receipt of order information. If it responds, it proceeds to step S309; ​​if it does not respond, it proceeds to step S311. Note that the processing unit 13 has predetermined decisions on whether or not to respond.

[0115] (Step S309) The voice response unit 132 provides the voice recognition result of the order information or the user order to the generation AI. The voice response unit 132 obtains a response from the generation AI. The voice response unit 132 obtains response information from the response. The voice response unit 132 outputs the response information as voice. The response information may be the same as the response, or it may be part of the response or information that has been processed from the response.

[0116] (Step S310) The avatar output unit 141 outputs an avatar to operate according to the response information.

[0117] (Step S311) The determination unit 131 determines whether or not the confirmation conditions are met. An example of such confirmation determination processing will be explained using the flowchart in Figure 4.

[0118] (Step S312) If it is determined in step S311 that the confirmation conditions are met, proceed to step S313; otherwise, return to step S304.

[0119] (Step S313) The output unit 14 acquires and outputs order confirmation information. Order confirmation information is information for confirming an order with the user. The order confirmation information may have, for example, one or more user orders. The order confirmation information may be, for example, voice information such as "We have received the following order. One or more <user orders>". <user orders> is a variable to which the user orders are assigned.

[0120] (Step S314) The processing unit 13 decides whether or not to check with the store clerk. If it decides to check with the store clerk, it goes to step S315; if it decides not to check with the store clerk, it goes to step S320. Note that whether or not to check with the store clerk is predetermined, for example.

[0121] (Step S315) The output unit 14 obtains the request information from the storage unit 11 and outputs the request information.

[0122] (Step S316) The determination unit 131 determines whether or not it is time to output a voice message to the staff member. If it is time to output a voice message, the unit proceeds to step S317; otherwise, it returns to step S316. The timing for outputting a voice message is, for example, the timing when it is detected that the staff member has come to the table. The method for detecting that the staff member has come to the table is the same as the method for detecting that the user has taken a seat; it does not matter.

[0123] (Step S317) The output unit 14 outputs one or more user orders by voice. The output unit 14 usually outputs user orders by voice to the store staff. By informing the store staff of a group order at once, the staff's time can be used effectively while the user is thinking about what to order.

[0124] (Step S318) The processing unit 13 determines whether or not to process the order. If it decides to process the order, it proceeds to step S319; otherwise, it returns to step S304. Note that whether or not to process the order automatically is predetermined, for example.

[0125] (Step S319) The order processing unit 133 performs order processing. An example of order processing will be explained using the flowchart in Figure 5.

[0126] (Step S320) The output unit 14 outputs an order confirmation voice message. The order confirmation voice message is voice information for confirming the order. For example, the order confirmation voice message is voice information for order confirmation.

[0127] (Step S321) The completion reception unit 122 determines whether or not it has received the order completion information. If it has received the order completion information, it proceeds to step S319; otherwise, it returns to step S321.

[0128] (Step S322) The processing unit 13 determines whether or not to terminate the process. If the process is to be terminated, it proceeds to step S323; if the process is not to be terminated, it proceeds to step S324. The process is terminated in cases such as when the user presses the "Pay" button on the order support device 1, when voice information indicating termination (e.g., "Pay," "Check, please") is received, or when it is detected that the user has left their seat, and the method of determination is not specified.

[0129] (Step S323) The processing unit 13 performs termination processing. The process returns to step S301. The termination processing is, for example, settlement processing.

[0130] (Step S324) The processing unit 13 performs a screen display change process. The process returns to step S311. The screen display change process may include, for example, making the avatar move or displaying a recommended menu.

[0131] In the flowchart in Figure 3, processing is terminated by power-off or processing termination interrupts.

[0132] Next, an example of the confirmation and judgment process in step S311 will be explained using the flowchart in Figure 4.

[0133] (Step S401) The determination unit 131 assigns 1 to counter i.

[0134] (Step S402) The determination unit 131 determines whether the i-th confirmation condition exists. If the i-th confirmation condition exists, the unit proceeds to step S403; otherwise, the unit proceeds to step S408.

[0135] (Step S403) The determination unit 131 obtains the i-th confirmation condition.

[0136] (Step S404) The determination unit 131 acquires information to be used to determine the i-th confirmation condition. Such information is, for example, information received from the user and the time when the order acceptance unit 121 did not receive any information.

[0137] (Step S405) The determination unit 131 determines whether the i-th confirmation condition is met. If the i-th confirmation condition is met, the process proceeds to step S406; otherwise, the process proceeds to step S407.

[0138] (Step S406) The determination unit 131 assigns "Confirmation condition met (e.g., "1")" to the determination result. It returns to the higher-level process.

[0139] (Step S407) The determination unit 131 increments the counter i by 1. The process returns to step S402.

[0140] (Step S408) The determination unit 131 assigns "Confirmation condition not met (e.g., "0")" to the determination result. It returns to the higher-level process.

[0141] Next, an example of order processing in step S319 will be explained using the flowchart in Figure 5.

[0142] (Step S501) The order processing unit 133 determines whether or not to perform order transmission processing. If order transmission processing is performed, the unit proceeds to step S502; otherwise, the unit proceeds to step S503.

[0143] (Step S502) The order processing unit 133 obtains one or more user orders and transmits the one or more user orders. The destination is one or more terminals from among one or more kitchen terminals 3 and one or more staff terminals 4.

[0144] (Step S503) The order processing unit 133 determines whether or not to perform payment processing. If payment processing is to be performed, the process proceeds to step S504; otherwise, it returns to the higher-level processing unit.

[0145] (Step S504) The order processing unit 133 processes the payment using one or more user orders. It then returns to the higher-level processing unit.

[0146] Next, an example of the operation when the order support device 1 performs automated order processing will be explained using the flowchart in Figure 6. Automated order processing is the process when the order receiving unit 121 receives order information that includes product identification information. Automated order processing is the process when the order information does not have a product identifier. In Figure 6, after step S304 in Figure 3, it is determined whether the order information includes product identification information or not. If the order information includes product identification information, the process goes to step S601; otherwise, (if the order information includes a product identifier), the process goes to step S305.

[0147] (Step S601) The response acquisition means 1322 obtains an automated order template from the prompt template management unit 112.

[0148] (Step S602) The response acquisition means 1322 acquires menu information from the menu management unit 113.

[0149] (Step S603) The response acquisition means 1322 places the menu information acquired in step S602 into the variable area of ​​the default order template acquired in step S601. This variable is a variable to which the menu information is assigned.

[0150] (Step S604) The speech recognition means 1321 recognizes the order information by voice and obtains the order string. If the order information has already been recognized by voice, this step is unnecessary.

[0151] (Step S605) The response acquisition means 1322 places the order string into the variable area of ​​the automated order template obtained in step S601. This variable is the variable to which the order string is assigned. Through the above process, the automated order prompt is constructed.

[0152] (Step S606) The response acquisition means 1322 provides the automated order prompt to the generating AI.

[0153] (Step S607) The response acquisition means 1322 determines whether or not it has acquired a response from the generating AI. If a response has been acquired, it proceeds to step S608; otherwise, it returns to step S607.

[0154] (Step S608) The proposal unit 143 obtains one or more product identifiers, etc. from the response. One or more product identifiers, etc. may be, for example, one or more product identifiers only, or one or more sets of product identifiers and quantities.

[0155] (Step S609) The suggestion unit 143 outputs one or more product identifiers, etc.

[0156] (Step S610) The authorization reception unit 123 determines whether or not it has received information from the user. If it has received the information, it proceeds to step S611; otherwise, it proceeds to step S610.

[0157] (Step S611) The permission reception unit 123 determines whether the received information is "permission". If the information is "permission", proceed to step S612; otherwise, proceed to step S304 in Figure 3.

[0158] (Step S612) The order processing unit 133 processes the order using one or more product identifiers, etc. Proceed to step S304 in Figure 3. The order processing was explained using the flowchart in Figure 5.

[0159] The following describes specific examples of the operation of information system A in this embodiment. Two specific examples are described here. Specific example 1 is the case where an order containing one or more product identifiers is placed. Specific example 2 is the case of a chef's choice order. The establishment in this example is assumed to be an izakaya (Japanese pub). The order support device 1 is assumed to be a terminal installed at one or more tables in the izakaya.

[0160] Assume that the avatar storage unit 111 of the order support device 1 stores the avatar information shown in Figure 7. In this case, the avatar information is an image of the avatar.

[0161] (Specific example 1) Assume that the prompt template management unit 112 of the order support device 1 stores the prompt "You are a bar employee. Please handle customer service." Then, the processing unit 13 provides this prompt to the generating AI, and from the generating AI, it obtains a string response (first string) "Welcome! Thank you for using our restaurant again today. Would you like to order something to drink? We can also recommend some menu items and snacks, so please feel free to ask."

[0162] Furthermore, the storage unit 11 of the order support device 1 stores the confirmation conditions "order completion information has been received" and "no order information has been received for more than one minute." Here, order completion information is defined as "End," "Finished," or "End." Also, "order completion information has been received" can be considered to include "information containing order completion information has been received."

[0163] Next, the processing unit 13 constructs the first voice information from the first string and stores the first voice information in the storage unit 11 in pairs with the first string. Furthermore, the second voice information, constructed from the second string "I will place the order with the staff, so please wait patiently," is stored in the storage unit 11 in association with the second string. The first and second voice information constitute the initial response.

[0164] Then, the processing unit 13 of the order support device 1 detects that the user has entered the izakaya (Japanese pub) and taken a seat.

[0165] Next, the avatar output unit 141 acquires the avatar information from the avatar storage unit 111. The avatar output unit 141 uses this avatar information to construct an avatar and outputs the avatar.

[0166] The voice response unit 132 acquires the first voice information, which is the initial response, and the second voice information from the storage unit 11. The voice response unit 132 outputs the first voice information as audio, and then outputs the second voice information as audio. The output unit 14, in synchronization with the output of each voice information, displays the first string around the avatar (in this case, below the avatar), and then displays the second string around the avatar. Figure 8 shows an image of how the avatar and the second string are displayed. In Figure 8, the avatar is 801 and the second string is 802.

[0167] Next, let's assume the user says to the avatar via voice, "I'd like three medium draft beers, please."

[0168] Next, the order receiving unit 121 of the order support device 1 receives the voice information order information "Please give me 3 medium draft beers." Next, the voice recognition means 1321 performs voice recognition processing on the order information and obtains the order string "Please give me 3 medium draft beers." Next, the processing unit 13 obtains the user order "<product name> Medium draft beer <quantity> 3" from the order string "Please give me 3 medium draft beers." The processing unit 13 also obtains a unique order identifier "111." Then, the processing unit 13 outputs the user order "<No.> 111 <Product Name> Draft Beer (Medium) <Quantity> 3" is formed. Next, the order output unit 142 outputs the user order. An example of such output is 803 in Figure 8.

[0169] Next, the response acquisition means 1322 provides the string "Please give me three medium draft beers" to the generating AI. The response acquisition means 1322 then obtains the response "Certainly! Three medium draft beers, right? Please wait a moment. I will bring them to you shortly." from the generating AI. Next, the voice output means 1323 outputs "Certainly! Three medium draft beers, right? Please wait a moment. I will bring them to you shortly." as voice.

[0170] Next, the judgment unit 131 determines that the order string "Please give me 3 medium draft beers" does not meet the confirmation conditions.

[0171] Next, let's assume the user says "One edamame" to the avatar using voice.

[0172] Next, the order receiving unit 121 of the order support device 1 receives order information "1 edamame" from the user. Next, the voice recognition means 1321 performs voice recognition processing on the voice order information and obtains the order string "1 edamame". Next, the processing unit 13 obtains the user order "<product name> Edamame <quantity> 1" from the order string "1 edamame". The processing unit 13 also obtains a unique order identifier "222". Then, the processing unit 13 outputs the user order "<No.> 222 <Product Name> Edamame <Quantity> 1" is formed. Next, the order output unit 142 outputs the user order. An example of such output is 804 in Figure 8.

[0173] Next, the response acquisition means 1322 provides the order string "one edamame" to the generating AI. The response acquisition means 1322 then obtains the response "Thank you! One edamame, right? We will bring it to you right away along with your draft beer, so please wait a moment." from the generating AI. Next, the voice output means 1323 outputs "Thank you! One edamame, right? We will bring it to you right away along with your draft beer, so please wait a moment." as voice output.

[0174] Next, the determination unit 131 determines that the order string "1 edamame" does not meet the confirmation conditions.

[0175] Next, let's assume the user, after much deliberation, says to the avatar via voice, "One Margherita pizza, please."

[0176] Next, the order receiving unit 121 of the order support device 1 receives order information "1 Margherita" from the user. Next, the voice recognition means 1321 performs voice recognition processing on the voice order information and obtains the order string "1 Margherita". Next, the processing unit 13 obtains the user order "<product name> Margherita <quantity> 1" from the order string "1 Margherita". The processing unit 13 also obtains a unique order identifier "333". Then, the processing unit 13 outputs the user order "<No.> 333 <Product Name> Margherita <Quantity> 1" is formed. Next, the order output unit 142 outputs the user order. An example of such output is 805 in Figure 8.

[0177] Next, the response acquisition means 1322 provides the order string "One Margherita" to the generating AI. The response acquisition means 1322 then obtains the response "Certainly! One Margherita, correct? We will prepare it for you with edamame and a draft beer, so please wait a moment." from the generating AI. Next, the voice output means 1323 outputs "Certainly! One Margherita, correct? We will prepare it for you with edamame and a draft beer, so please wait a moment." as voice output.

[0178] Next, the determination unit 131 determines that the order string "1 Margherita" does not meet the confirmation conditions.

[0179] Next, let's assume the user says to the avatar via voice, "One rolled omelet."

[0180] Next, the order receiving unit 121 of the order support device 1 receives order information "1 rolled omelet" from the user. Next, the voice recognition means 1321 performs voice recognition processing on the voice order information and obtains the order string "1 rolled omelet". Next, the processing unit 13 obtains the user order "<product name> rolled omelet <quantity> 1" from the order string "1 rolled omelet". The processing unit 13 also obtains a unique order identifier "444". Then, the processing unit 13 outputs the user order "<No.> 444 <Product Name>Dashimaki Tamago <Quantity> 1" is formed. Next, the order output unit 142 outputs the user order. An example of such output is 806 in Figure 8.

[0181] Next, the response acquisition means 1322 provides the order string "one dashi-maki tamago" to the generating AI. The response acquisition means 1322 then obtains the response "Thank you! We have received your order for one additional dashi-maki tamago. We will bring it to you along with the Margherita pizza, edamame, and draft beer in due course, so please wait a moment." from the generating AI. Next, the voice output means 1323 outputs the voice message "Thank you! We have received your order for one additional dashi-maki tamago. We will bring it to you along with the Margherita pizza, edamame, and draft beer in due course, so please wait a moment."

[0182] Next, the determination unit 131 determines that the order string "1 rolled omelet" does not meet the confirmation conditions.

[0183] Next, let's assume the user says "One piece of fried chicken" to the avatar using voice.

[0184] Next, the order receiving unit 121 of the order support device 1 receives order information "1 piece of fried chicken" from the user. Next, the voice recognition means 1321 performs voice recognition processing on the voice order information and obtains the order string "1 piece of fried chicken". Next, the processing unit 13 obtains the user order "<product name> Fried Chicken <quantity> 1" from the order string "1 piece of fried chicken". The processing unit 13 also obtains a unique order identifier "444". Then, the processing unit 13 outputs the user order "<No.> 555 <Product Name> Fried Chicken <Quantity> 1" is formed. Next, the order output unit 142 outputs the user order. An example of such output is 807 in Figure 8.

[0185] Next, the response acquisition means 1322 provides the order string "1 piece of fried chicken" to the generating AI. The response acquisition means 1322 then obtains the response from the generating AI: "Understood! We have received your order for one additional piece of fried chicken. Your current order is 3 medium draft beers, edamame, Margherita pizza, rolled omelet, and one piece of fried chicken. We will bring everything to you as soon as it is ready, so please wait a moment." Next, the voice output means 1323 outputs the voice message: "Understood! We have received your order for one additional piece of fried chicken. Your current order is 3 medium draft beers, edamame, Margherita pizza, rolled omelet, and one piece of fried chicken. We will bring everything to you as soon as it is ready, so please wait a moment."

[0186] Next, the determination unit 131 determines that the order string "1 piece of fried chicken" does not meet the confirmation conditions.

[0187] Next, let's assume the user says "That's all" to the avatar via voice.

[0188] Next, the order receiving unit 121 of the order support device 1 receives the order information "That's all." from the user. Next, the voice recognition means 1321 performs voice recognition processing on the voice order information and obtains the string "That's all." Next, the processing unit 13 was unable to obtain the user order from the string "That's all."

[0189] Next, since the character string "That's all." includes the order completion information "That's all", the determination unit 131 determines that the confirmation condition is satisfied.

[0190] Next, the response acquisition means 1322 gives the character string "That's all." to the generation AI. Then, the response acquisition means 1322 acquires the response shown in FIG. 9 from the generation AI. Next, the voice output means 1323 outputs the character string in FIG. 9 as voice.

[0191] Next, it is assumed that the user spoke to the avatar in voice, "Yes. Thank you."

[0192] Next, the reception unit 12 receives the voice information. Then, the voice recognition means 1321 performs voice recognition processing on the voice information and acquires the character string "Yes. Thank you."

[0193] Next, since the completion reception unit 122 includes "Yes", which is one of the order completion information, it determines that the order completion information has been received.

[0194] Next, the order processing unit 133 performs order processing as follows. That is, the order processing unit 133 acquires the seat identifier "A1" of the storage unit 11. Also, the order processing unit 133 acquires the order content "<seat identifier> A1 <product> <product name> draft beer (medium) <quantity> 3 <product> <product name> edamame <quantity> 1 <product> <product name> Margherita <quantity> 1 <product> <product name> rolled omelette <quantity> 1 <product> <product name> tempura <quantity> 1" based on the user order acquired from the order information of two or more each product.

[0195] Next, the order processing unit 133 transmits the order content to the store clerk terminal 4 and the kitchen terminal 3. As a result, the store clerk and the cook in the kitchen can confirm the order content of the user at the seat with the seat identifier "A1" and proceed with the corresponding action (for example, cooking) for the user.

[0196] (Specific Example 2) It is assumed that the order prompt template management unit 112 of the order support device 1 stores the order-by-request template shown in FIG. 10.

[0197] Furthermore, the menu management unit 113 stores the menu information shown in Figure 11.

[0198] Furthermore, the storage unit 11 stores the confirmation condition "that permission has been granted for the output of one or more product identifiers in the automated order."

[0199] Furthermore, the storage unit 11 stores the initial voice response, "Welcome. I will take your request for the product you wish to purchase."

[0200] Then, the processing unit 13 of the order support device 1 detects that the user has entered the izakaya (Japanese pub) and taken a seat.

[0201] Next, the avatar output unit 141 acquires the avatar information from the avatar storage unit 111. The avatar output unit 141 uses this avatar information to construct an avatar and outputs the avatar.

[0202] The voice response unit 132 acquires the initial response from the storage unit 11 and outputs the voice message "Welcome. I will take your request for the product you would like to purchase."

[0203] Next, let's assume the user voice-inputs to the avatar, "There are three of us. Please give us something healthy."

[0204] Next, the order receiving unit 121 of the order support device 1 receives the order information, which is voice information from the user: "There are 3 of us. We would like something healthy."

[0205] Next, the voice recognition means 1321 processes the order information, which is voice information, using voice recognition to obtain the order string "There are 3 of us. Please give us something healthy." Then, the decision unit 131 determines that the string does not contain a product identifier. In other words, the decision unit 131 decides to perform the automated order processing (Figure 6).

[0206] Next, the response acquisition means 1322 acquires the automated order template (Figure 10) from the prompt template management unit 112. Then, the response acquisition means 1322 acquires the menu information (Figure 11) from the menu management unit 113. The response acquisition means 1322 places the acquired menu information into the variable {food menu} area of ​​the acquired automated order template.

[0207] Next, the response acquisition means 1322 obtains the order string "There are 3 of us. Please give us something healthy." Then, the response acquisition means 1322 places the order string into the variable {order} area of ​​the automated order template. As a result of the above processing, the response acquisition means 1322 obtains the automated order prompt (Figure 12).

[0208] Next, the response acquisition means 1322 provides the automated order prompt to the generating AI. Then, the response acquisition means 1322 obtains the response shown in Figure 13 from the generating AI.

[0209] Next, the suggestion unit 143 obtains three pairs of product identifiers and quantities from the response. Then, the suggestion unit 143 displays the three pairs of product identifiers and quantities (Figure 14). The suggestion unit 143 may also output the three pairs of product identifiers and quantities as audio.

[0210] Next, suppose the user says, "Yes, please." The permission receiving unit 123 then receives the voice information from the user. Next, the voice recognition means 1321 performs voice recognition processing on the voice information and obtains the string "Yes, please." Then, the permission receiving unit 123 analyzes "Yes, please." and determines that the user's voice information is "permission."

[0211] Next, the order processing unit 133 processes the order using the three sets of product identifiers and quantity information.

[0212] In summary, according to this embodiment, it is possible to accept voice orders while enjoying conversation with an avatar. In particular, according to this embodiment, using a generative AI, it is possible to accept voice orders for each ordered product while enjoying conversation with an avatar.

[0213] Furthermore, according to this embodiment, the workload of store employees can be reduced by having the avatar take orders on their behalf.

[0214] Furthermore, according to this embodiment, by having the avatar take orders on behalf of the customer, the workload of store staff can be reduced while the customer is deciding which items to order.

[0215] Furthermore, according to this embodiment, order processing can be performed at an appropriate time after the user's order has been completed.

[0216] Furthermore, according to this embodiment, the generation AI can be used to output the voice of an avatar based on the confirmed order after the order has been completed.

[0217] Furthermore, according to this embodiment, the user can confirm the ordered items for each product.

[0218] Furthermore, according to this embodiment, product identification information that does not include product identifiers can be received, and products can be suggested based on said product identification information.

[0219] Furthermore, according to this embodiment, order processing can only be performed after receiving permission from the user for the proposed product.

[0220] The processing in this embodiment may be implemented by software. This software may be distributed by software download or the like. Alternatively, this software may be recorded on a recording medium such as a CD-ROM and distributed. This also applies to other embodiments in this specification. The software that implements the order support device 1 in this embodiment is the following program. In other words, this program is a program that causes a computer that can access an avatar storage unit where avatar information is stored to function as an order processing unit that performs order processing, which is the process of proceeding with an order, using the order information received by the order receiving unit. The computer is configured to function as an avatar output unit that outputs an avatar configured using the avatar information, an order receiving unit that receives voice order information indicating an order from a user, a voice response unit that acquires response information indicating a response corresponding to the order information received by the order receiving unit and outputs the response information as voice, and a computer that performs order processing, which is the process of proceeding with an order, using the order information received by the order receiving unit.

[0221] Figure 15 is a block diagram of a computer system 300 that executes the program described herein to realize the order support device 1 and other devices of the various embodiments described above.

[0222] In Figure 15, the computer system 300 includes a computer 301 with a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0223] In Figure 15, the computer 301 includes, in addition to the CD-ROM drive 3012, an MPU 3013, a bus 3014 connected to the CD-ROM drive 3012, a ROM 3015 for storing programs such as boot-up programs, a RAM 3016 connected to the MPU 3013 for temporarily storing application program instructions and providing temporary storage space, and a hard disk 3017 for storing application programs, system programs, and data. Although not shown here, the computer 301 may further include a network card for providing connectivity to a LAN.

[0224] The program that causes the computer system 300 to execute the functions of the order support device 1, etc., as described above, may be stored on the CD-ROM 3101, inserted into the CD-ROM drive 3012, and then transferred to the hard disk 3017. Alternatively, the program may be transmitted to the computer 301 via a network (not shown) and stored on the hard disk 3017. The program is loaded into the RAM 3016 during execution. The program may also be loaded directly from the CD-ROM 3101 or the network.

[0225] The program does not necessarily have to include an operating system (OS) or third-party program that causes the computer 301 to execute functions such as the order support device 1 of the above-described embodiment. The program only needs to include the instruction portion that calls appropriate functions (modules) in a controlled manner and obtains the desired result. How the computer system 300 operates is well known, so a detailed explanation is omitted.

[0226] In the above program, steps such as sending information and receiving information do not include hardware-based processing, such as processing performed by a modem or interface card in the transmission step (processing that can only be performed by hardware).

[0227] Furthermore, the computer running the above program may be a single computer or multiple computers. In other words, it may perform centralized processing or distributed processing.

[0228] Furthermore, it goes without saying that in each of the above embodiments, two or more communication means present in a single device may be physically implemented in a single medium.

[0229] Furthermore, in each of the above embodiments, each process may be implemented by centralized processing by a single device, or by distributed processing by multiple devices.

[0230] It goes without saying that the present invention is not limited to the embodiments described above, and various modifications are possible, all of which are also included within the scope of the present invention. [Industrial applicability]

[0231] As described above, the order support device 1 according to the present invention has the effect of allowing customers to enjoy interacting with an avatar while receiving orders by voice, and is useful, for example, as an order terminal in a store. [Explanation of Symbols]

[0232] A Information Systems 1. Order support device 2 Generation AI device 3. Kitchen terminal 4. Staff terminal 11 Storage Unit 12 Reception Department 13 Processing Unit 14 Output section 111 Avatar Storage Unit 112 Prompt Template Management Department 113 Menu Management Department 121 Order Reception Department 122 Completion reception department 123 Permit Reception Department 131 Judgment Department 132 Voice Response Unit 133 Order Processing Section 141 Avatar Output Unit 142 Order Output Section 143 Proposal Department 1321 Speech recognition means 1322 Answer acquisition means 1323 Audio output means

Claims

1. An avatar storage unit where avatar information is stored, An avatar output unit that outputs an avatar constructed using the aforementioned avatar information, An order reception unit that receives voice order information from users regarding their orders, A voice response unit acquires response information indicating a response corresponding to the order information received by the order receiving unit, and outputs the response information as voice. An order support device comprising an order processing unit that performs order processing, which is the process for proceeding with an order, using the order information received by the order receiving unit.

2. The aforementioned order processing unit, The order support device according to claim 1, which performs order processing to transmit a user order based on the order information to a kitchen terminal or employee terminal, which is a terminal in the kitchen that prepares the products corresponding to the order.

3. The aforementioned order receiving department is Order information for two or more items will be accepted sequentially. The aforementioned order processing unit, The order support device according to claim 1, which performs an order processing that outputs a user order, which is a compilation of the order information for the two or more products, to a store employee via voice output.

4. The aforementioned order processing unit, The order support device according to claim 3, further comprising: performing a request output process to output request information for calling the employee to the user's seat; and, after detecting that the employee has arrived at the seat, performing a voice output process to output the user's order to the employee via voice.

5. The system further comprises a determination unit that determines whether or not the confirmation conditions, which are conditions for identifying that the receipt of the order information from the user has been completed, The aforementioned voice response unit is If the determination unit determines that the confirmation conditions are met, it outputs an order confirmation voice message, which is an audio message confirming the completion of the order. The system further comprises a completion reception unit that receives order completion information via voice, indicating the completion of the order, in response to the output of the aforementioned order confirmation voice. The aforementioned order processing unit, The order support device according to any one of claims 1 to 4, wherein the completion reception unit performs the order completion processing when it receives the order completion information.

6. The aforementioned voice response unit is A speech recognition means that performs speech recognition processing on the order information for each order of one or more products and obtains an order string, The means for obtaining an answer includes passing the aforementioned order string to a generating AI and obtaining an answer from the generating AI, The order support device according to claim 5, further comprising: an audio output means for outputting the response information based on the aforementioned answer as an audio output.

7. The aforementioned voice response unit is A speech recognition means that performs speech recognition processing on the aforementioned order information to obtain the order string, The means for obtaining an answer includes passing the aforementioned order string to a generating AI and obtaining an answer from the generating AI, The order support device according to claim 5, further comprising: an audio output means for outputting the response information based on the answer when the completion reception unit receives the order completion information.

8. The order support device according to any one of claims 1 to 7, further comprising an order output unit that outputs a user order based on the order information each time the order receiving unit receives the order information indicating an order for one or more products.

9. The aforementioned order output unit is: The order support device according to claim 8, which displays the user order or outputs the user order as an audio signal.

10. The aforementioned order receiving department is We accept voice orders that include product identification information but do not include product identifiers. A speech recognition means that performs speech recognition processing on the aforementioned order information to obtain the order string, A means for obtaining an answer that passes the aforementioned order string to a generating AI and obtains an answer from the generating AI that has one or more product identifiers, The system further comprises a proposal unit that outputs the one or more product identifiers included in the aforementioned response, The aforementioned order processing unit, The order support device according to claim 1, which performs the order processing using the one or more product identifiers output by the proposal unit.

11. The system further comprises a permission receiving unit that receives permission for the one or more product identifiers output by the proposal unit, The aforementioned order processing unit, The order support device according to claim 10, which, when the permission receiving unit receives the permission, performs the order processing using the one or more product identifiers output by the proposal unit.

12. The system further comprises a menu management unit that stores menu information, which is a set of two or more items to be ordered. The means for obtaining the response is, The order support device according to claim 10 or claim 11, which passes the order string and the menu information to a generating AI and obtains the response from the generating AI.

13. The order support device according to any one of claims 10 to 12, wherein the product identification information includes one or more types of information from among product attribute information, user attribute information, or recommendation information meaning a recommendation of a recommended product.

14. An order support method implemented by an avatar storage unit for storing avatar information, an avatar output unit, an order acceptance unit, a voice response unit, and an order processing unit, The avatar output unit performs an avatar output step in which it outputs an avatar configured using the avatar information, The aforementioned order acceptance includes an order acceptance step in which the user provides voice order information regarding the order, The voice response unit acquires response information indicating a response corresponding to the order information received by the order acceptance unit, and outputs the response information as voice in a voice response step, An order support method comprising: an order processing step in which the order processing unit performs an order processing, which is a process for proceeding with an order, using the order information received by the order receiving unit.

15. A computer that can access the avatar storage unit where avatar information is stored, An avatar output unit that outputs an avatar constructed using the aforementioned avatar information, An order reception unit that receives voice order information from users regarding their orders, A voice response unit acquires response information indicating a response corresponding to the order information received by the order receiving unit, and outputs the response information as voice. A program to function as an order processing unit that performs order processing, which is the process of proceeding with an order, using the order information received by the order receiving unit.

Images