Method, device, recording medium, and program

Abstract

At least one embodiment provides an ordering method that includes a step in which a device acquires voice from a user, a step in which the device compares language information that represents the voice and menu information that is a menu for a restaurant and requests that an AI specify at least one piece of menu information that is similar to the language information, and a step in which the device transmits the specified menu information to a cook terminal.

Description

Method, apparatus, recording medium, program

[0001] The present invention relates to a method, an apparatus, a recording medium, and a program.

[0002] The statements in this section only provide background information regarding the present disclosure and do not necessarily constitute prior art.

[0003] Patent Document 1 discloses a passenger management POS system characterized by comprising monitoring means such as a monitoring camera for monitoring a passenger seat, displaying a video signal from the monitoring means for monitoring the passenger seat on a display unit, and being able to confirm the situation of the passenger seat.

[0004] Japanese Patent Application Laid-Open No. Hei 8-161403

[0005] However, the inventor recognized that at least the above-described embodiment has a disadvantage in that there is no method for taking an order for food and drink without a person.

[0006] At least one aspect of the present disclosure is an ordering method, comprising steps in which an apparatus obtains a user's voice, requests an AI to collate language information indicated by the voice with menu information that is a menu of a restaurant and identify one or more menu information items similar to the language information, and transmits the identified menu information to a cook terminal.

[0007] This configuration has at least the usefulness of being able to take an order for food and drink without a person.

[0008] These and other aspects, features, and advantages of the present disclosure will become apparent from the following detailed written description of the preferred embodiments and aspects taken in conjunction with the following drawings, but modifications and variations thereof can be implemented without departing from the spirit and scope of the novel concept of the present disclosure. Aspects in certain embodiments of the present disclosure can be combined with, or replaced by, one or more of the aspects in other disclosed embodiments, as long as they are not contradictory.

[0009] Block diagrams according to one or more embodiments Block diagrams according to one or more embodiments Block diagrams according to one or more embodiments

[0010] The following disclosure provides many different embodiments and examples for carrying out different features of the presented subject matter. For the sake of simplicity, specific examples of components and arrangements are disclosed below. Of course, these are merely examples and are not intended to be limiting. For example, a structure in which a first feature is covered by or in contact with a second feature subsequently disclosed may include embodiments in which an additional feature is formed between the first and second features so that they do not come into direct contact, as well as embodiments in which the first and second features are formed so that they do not come into direct contact. Furthermore, the disclosure may repeat reference numbers and / or letters in various examples. This repetition is for the sake of brevity and clarity and does not require in itself to be related to the various embodiments and / or configurations described. Furthermore, when a first element is described as being “connected” or “joined” with a second element, such description includes embodiments in which the first and second elements are directly connected or joined to each other, as well as embodiments in which the first and second elements are indirectly connected or joined to each other by having one or more other elements interposed between them. As used herein, the phrase “at least one of” encompasses all exemplary variations. For example, the phrase “comprises at least one of A, B, or C” is synonymous with “consisting of A, B, C and combinations thereof,” and encompasses all conceivable variations of A, B, C, A+B, A+C, B+C, and A+B+C.

[0011] In this disclosure, disclosures using an electronic operator or computer may include embodiments of methods, recording media, apparatus, or programs. Any statement used herein that “A is B” may be replaced with “A includes B” unless otherwise stated herein, to the extent that it does not conflict with or otherwise state otherwise herein.

[0012] The following embodiments can be used for the operating method in at least one embodiment. The following will be explained by reference to the description in JP6456303, which describes at least one embodiment in detail (beginning of reference).

[0013] As used herein, the term “computer” schematically includes, as known in the art, a processor, memory, at least one information storage / retrieval device such as a hard drive, disk drive or flash drive or memory stick, or other non-temporary computer-readable medium or non-temporary storage device, at least one input device such as a keyboard, mouse, point and touch device, touchscreen or microphone, and a display structure such as a well-known computer screen. In addition, a computer may include one or more network connections, such as wired or wireless connections. As known in the art, such a computer or computer system may more or less include, but is not limited to, for example, tablet computers or smart devices, other electronic media and electronic devices.

[0014] As used herein, the terms “cloud” or “cloud computing” refer to centralized and virtualized computing facilities where all computing resources are shared. For application systems and subsystems, since they all reside within the “cloud,” it is no longer possible to refer to specific machines.

[0015] As used herein, the term “Distributed Internet Services System” refers to a distributed internet services platform that transforms internet applications for execution in various computing environments. A DIS system delivers internet applications, including content, data, and logic, to whatever extent applicable, and along a network, to any number and type of devices, via Component Distribution Servers / Asset Distribution Servers. Through DIS, internet applications can be hosted and centrally managed as services tailored to each user's needs, and can be locally cached and executed on the user's device or nearby locations while maintaining their integrity. Web-enabled computing devices can be upgraded with DIS software to become DIS-enabled, capable of enjoying and running distributed internet services. The distributed internet service system is fully described in any one of the patent families U.S. Patents 7,136,857, 7150015, 7181731, 7209921, 7430610, 7685183, 7685577, 7752214, 8326883, 8386525, 8443035, 8458142, 8458222, 8473468, 8527545, and 8650226, and U.S. Patent Publications 20120005205 ​​and 20130091252, all of which, like the present invention, are jointly owned by OP40, Holdings, Inc., and all of which are incorporated by reference. (End of quote)

[0016] Regarding the operating method used in at least one embodiment, the following embodiments can be taken for conventional internet systems that do not use a distributed internet. The following will be explained by reference to the description in JP7113047, which describes at least one embodiment in detail (beginning of reference).

[0017] Embodiments including those specifically disclosed herein can provide an automated response system based on artificial intelligence that is implemented in a manner that mimics actual human conversation, thereby enabling more natural communication with users while quickly and conveniently handling inquiries, reservations, delivery orders, and more.

[0018] The multiple electronic devices 110, 120, 130, and 140 may be fixed terminals or mobile terminals implemented by a computer system. Examples of the multiple electronic devices 110, 120, 130, and 140 include AI speakers, smartphones, mobile phones, navigation systems, PCs (personal computers), notebook PCs, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablets, game consoles, wearable devices, IoT (Internet of Things) devices, VR (virtual reality) devices, and AR (augmented reality) devices. As an example, Figure 1 shows an AI speaker as the electronic device 110, but in embodiments of the present invention, the electronic device 110 may mean one of a variety of physical computer systems that can communicate with other electronic devices 120, 130, 140 and / or servers 150, 160 via the network 170 using substantially wireless or wired communication methods.

[0019] The communication method is not limited, and the network 170 may include not only communication methods that utilize communication networks that can be included (for example, mobile communication networks, wired internet, wireless internet, broadcasting networks, satellite networks, etc.), but also short-range wireless communication between devices. For example, the network 170 may include one or more arbitrary networks such as PAN (personal area network), LAN (local area network), CAN (campus area network), MAN (metropolitan area network), WAN (wide area network), BBN (broadband network), and the Internet. Furthermore, the network 170 may include, but is not limited to, one or more network topologies, including bus networks, star networks, ring networks, mesh networks, starbus networks, tree or hierarchical networks.

[0020] Servers 150 and 160 may each be implemented by one or more computer devices that communicate with multiple electronic devices 110, 120, 130, and 140 via a network 170 to provide instructions, code, files, content, services, etc. For example, server 150 may be a system that provides a first service to multiple electronic devices 110, 120, 130, and 140 connected via the network 170, and server 160 may be a system that provides a second service to multiple electronic devices 110, 120, 130, and 140 connected via the network 170. As a more specific example, server 150 may provide the multiple electronic devices 110, 120, 130, and 140 as a first service through an application, which is a computer program installed and executed on the multiple electronic devices 110, 120, 130, and 140, with the service targeted by that application (for example, an automated response service). As another example, server 160 may provide a second service that distributes files for installing and running the aforementioned application to multiple electronic devices 110, 120, 130, and 140.

[0021] Figure 2 is a block diagram illustrating the internal configuration of an electronic device and a server in one embodiment of the present invention. In Figure 2, the internal configuration of electronic device 110 and the internal configuration of server 150 are described as examples of electronic devices. Other electronic devices 120, 130, 140 and server 160 may also have the same or similar internal configuration as the electronic device 110 or server 150 described above.

[0022] The electronic devices 110 and server 150 may include memories 211, 221, processors 212, 222, communication modules 213, 223, and input / output interfaces 214, 224. The memories 211, 221 may be non-temporary computer-readable recording media and may include non-temporary mass storage devices such as RAM (random access memory), ROM (read-only memory), disk drives, SSDs (solid state drives), and flash memory. Here, non-temporary mass storage devices such as ROM, SSDs, flash memory, and disk drives may be included in the electronic devices 110 and server 150 as separate non-temporary storage devices distinct from the memories 211, 221. Furthermore, the memories 211 and 221 may contain an operating system and at least one program code (for example, code for a browser installed and run on the electronic device 110, or code for an application installed on the electronic device 110 to provide a specific service). Such software components may be loaded from a computer-readable recording medium other than the memories 211 and 221. Such a separate computer-readable recording medium may include computer-readable recording media such as floppy® drives, disks, tapes, DVD / CD-ROM drives, and memory cards. In other embodiments, the software components may be loaded into the memories 211 and 221 through communication modules 213 and 223 that are not computer-readable recording media. For example, at least one program may be loaded into the memories 211 and 221 based on a computer program (for example, the application described above) that is installed by a file provided via the network 170 by a developer or a file distribution system (for example, the server 160 described above) that distributes installation files for an application.

[0023] Processors 212 and 222 may be configured to process computer program instructions by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to processors 212 and 222 by memory 211 and 221 or by communication modules 213 and 223. For example, processors 212 and 222 may be configured to execute instructions received according to program code recorded in a recording device such as memory 211 and 221.

[0024] Communication modules 213 and 223 may provide functions for the electronic device 110 and the server 150 to communicate with each other via the network 170, or they may provide functions for the electronic device 110 and / or the server 150 to communicate with other electronic devices (for example, electronic device 120) or other servers (for example, server 160). For example, a request generated by the processor 212 of the electronic device 110 according to program code recorded in a recording device such as memory 211 may be transmitted to the server 150 via the network 170 under the control of the communication module 213. Conversely, control signals, instructions, content, files, etc., provided under the control of the processor 222 of the server 150 may be received by the electronic device 110 via the communication module 223 and the network 170 through the communication module 213 of the electronic device 110. For example, control signals, commands, content, and files from the server 150 received through the communication module 213 may be transmitted to the processor 212 and memory 211, and the content and files may be recorded on a recording medium (the non-temporary recording device described above) that the electronic device 110 may further include.

[0025] The input / output interface 214 may be a means for interface with an input / output device 215. For example, the input device may include devices such as a keyboard, mouse, microphone, or camera, and the output device may include devices such as a display, speaker, or haptic feedback device. As another example, the input / output interface 214 may be a means for interface with a device that integrates input and output functions into one, such as a touchscreen. The input / output device 215 may consist of the electronic device 110 and one other device. The input / output interface 224 of the server 150 may be a means for interface with an input or output device (not shown) that connects to or can be included in the server 150. As a more specific example, when the processor 212 of the electronic device 110 processes instructions for a computer program loaded into memory 211, a service screen or content configured using data provided by the server 150 or the electronic device 120 may be displayed on the display via the input / output interface 214.

[0026] Furthermore, in other embodiments, the electronic device 110 and the server 150 may include more components than those shown in Figure 2. However, it is not necessary to explicitly show most of the conventional components in the figure. For example, the electronic device 110 may be implemented to include at least some of the input / output devices 215 described above, and may further include other components such as transceivers, cameras, various sensors, and databases. As a more specific example, if the electronic device 110 is an AI speaker, the electronic device 110 may be implemented to further include a variety of components that are generally included in an AI speaker, such as various sensors, camera modules, various physical buttons, buttons using a touch panel, input / output ports, and vibrators for vibration. (End of quote)

[0027] The machine will now be described. According to at least one embodiment, the user terminal consists of a control unit, RAM, storage unit, graphics processing unit, communication interface, and interface unit, each connected by an internal bus.

[0028] According to at least one embodiment, the control unit consists of a CPU and ROM. The control unit executes programs stored in the storage unit and controls the user terminal. RAM is the work area of ​​the control unit. The storage unit is a memory area for saving programs and data. The control unit reads programs and data from RAM and processes them. By processing the programs and data loaded into RAM, the control unit outputs drawing commands to the graphics processing unit.

[0029] According to at least one embodiment, the graphics processing unit is connected to the display unit. The display unit has a display screen. When the control unit outputs a drawing command to the graphics processing unit, the graphics processing unit outputs a video signal for displaying an image on the display screen. Here, the display unit may be a touch panel equipped with a touch sensor. The touch panel of this display unit functions as an input unit.

[0030] According to at least one embodiment, the communication interface can be connected to a communication network wirelessly or via a wired connection, and can send and receive data with a server device via the communication network. The data received via the communication interface is loaded into RAM and processed by the control unit. External memory (e.g., an SD card) is connected to the interface unit.

[0031] According to at least one embodiment, the user terminal is not particularly limited as long as it is a computer device having a display screen and an input unit. Examples of user terminals include conventional mobile phones, tablet devices, smartphones, and desktop or notebook personal computers. It may also consist of a screen (or two display panels, one for each eye) attached to a frame (or headset) that is fixed or attached to the head with a strap, i.e., a VR goggle. The user terminal has an audio output unit.

[0032] According to at least one embodiment, a user terminal can communicate with a server device via a communication network. It can transmit or receive information by establishing a communication connection via the communication network.

[0033] According to at least one embodiment, the server device comprises at least a control unit, RAM, a storage unit, and a communication interface, each connected by an internal bus.

[0034] According to at least one embodiment, the control unit consists of a CPU and ROM, executes programs stored in the storage unit, and controls the server device. The control unit also has an internal timer for timing. RAM is the work area of ​​the control unit. The storage unit is a memory area for saving programs and data. The control unit reads programs and data from RAM and performs program execution processing based on information received from the user terminal, etc.

[0035] This explains AI. According to at least one embodiment, artificial intelligence includes machine learning, deep learning, generative AI, large-scale language models (LLMs), foundational models, and generative AI. Generative AI uses transformers and employs numerous attention mechanisms. It uses self-supervised learning and Extract Prediction. In this case, the AI ​​can predict the next word. Given a sentence, it predicts the next word from the sentence up to that point. It generates a large number of supervised learning problems. These allow for an AI that can predict the next word. Generative AI can predict grammatical structure, topic connections, and what kind of sentences people with a certain writing style are likely to write. Furthermore, by simply predicting the next sentence, generative AI can learn the underlying structure, causal relationships, and knowledge. Generative AI scales quickly, and its accuracy improves as the number of parameters increases. Ordinary statistics and machine learning overfit if the model parameters are too large compared to the data sample size. LLMs become more accurate as the number of parameters increases. One generative AI has 175 billion parameters. Generative AI is overlaid with supervised learning to facilitate smooth conversation. They are taught not to say anything strange. They write essays and act as call center operators.

[0036] According to at least one embodiment, Large Language Models (LLMs) are, in a non-inclusive sense, machine learning natural language processing models built using large datasets and deep learning techniques. Generally, they are adapted to various natural language processing (NLP) tasks such as text classification and generation, sentiment analysis, text summarization, and question answering using a technique called "fine-tuning," which involves training them on specific tasks. According to at least one embodiment, self-supervised learning is close to intrinsic human intelligence. When humans act, they are always predicting what will happen next and predicting the next input. In the process, they can learn the structure of the external world. Predicting the next word is considered intrinsic intelligence and is close to what is done in the cerebral cortex. According to at least one embodiment, Large Language Models memorize all the input information but generalize it to the extent necessary to predict the next word. They do not try to generalize all the information from the beginning. Large Language Models require capacity to remember information, and therefore require parameters. According to at least one embodiment, the large-scale language model incorporates eight models, each with 175 billion parameters or 220 billion parameters.

[0037] According to at least one embodiment, videos and images are represented as a collection of visual patches, which are small data units similar to text tokens in LLMs. Patches effectively represent models of visual data and are used as a highly scalable and effective representation for training generative models on various types of videos and images. The video is converted into patches by first compressing the video into a low-dimensional latent space and then decomposing the representation into spatiotemporal patches.

[0038] According to at least one embodiment, a video compression network is a network that reduces the dimensionality of visual data, taking raw video as input and outputting a temporally and spatially compressed latent representation. An AI is trained in this compressed latent space and then generates video within this compressed latent space.

[0039] According to at least one embodiment, Spacetime Latent Patches extract a series of spatiotemporal patches that function as transformer tokens when given a compressed input video. The patch-based representation allows Sora to be trained on videos and images of varying resolutions, lengths, and aspect ratios, and controls the size of the resulting video by arranging randomly initialized patches into a grid of appropriate size during inference.

[0040] According to at least one embodiment, the AI ​​is a diffusion model that, given a noisy patch (and conditioning information such as text prompts) as input, is trained to predict the original "clean" patch. The AI ​​is a diffusion transformer, which exhibits remarkable scaling properties in various domains, including language modeling, computer vision, and image generation. Diffusion transformers are also effective as video generation models. The AI's sample quality improves significantly as the training computation increases.

[0041] According to at least one embodiment, the AI ​​applies caption regeneration technology to train a highly descriptive caption model, which is then used to generate text captions for all videos in the training set. Training highly descriptive captions improves not only the overall quality of the generated videos but also the fidelity of the text. GPT is used to convert short user prompts into long, detailed captions, which are then sent to the model. This allows the AI ​​to generate high-quality videos that precisely follow the user prompts.

[0042] According to at least one embodiment, the AI ​​can perform vectorization in natural language processing according to the following flow. First, as a preprocessing step, it performs a cleaning process on the given text. In the cleaning process, unnecessary words such as JavaScript code and HTML tags contained in the text are removed. These codes are used to display on the internet and are therefore not generally used in natural language processing. Next, the text is divided into words using morphological analysis. Morphological analysis is the classification of a natural language sentence written in characters into the smallest meaningful linguistic unit. "MeCab," "JUMAN," and "JANOME" can be used as morphological analysis tools. In normalization, words with the same meaning, such as variations in spelling, are unified into a single word. Stop words are words that are excluded from processing for reasons such as not being usable in natural language processing. Examples of stop words include particles and auxiliary verbs, which do not have meaning on their own. When calculating vectors, these may be removed, and only meaningful words may be targeted. Vectorization may also be performed without removing these stop words. Vectorization is the process of converting strings of words into vectors. Vectorization transforms word data into numerical data. When converting words to vectors, methods such as Bag of Words and distributed representations are used. Bag of Words is a method of vectorizing a given text using the number of occurrences of each word. It focuses on how often each word appears in the text, and does not consider the order of words or sentences. Distributed representation is a method of vectorizing by focusing on the meaning of words. By vectorizing the meaning of words, it is possible to assign similar vectors to words with similar meanings or usages, and the relationships between words can also be represented by vectors. With vector representation, it is possible to add and subtract the meanings of words. In applied processing, natural language converted into numerical data can be used as input for machine learning. Specifically, vectorized natural language is fed into a classifier to perform text classification.Tools used here include TensorFlow, scikit-learn, and PyTorch.

[0043] The steps for acquiring user voice are disclosed. In at least one implementation, the device can be one or more machines or computers as described in this disclosure. Fixed terminals and mobile terminals acquire voice from the user. As an example, a fixed terminal is a store terminal installed in a store and includes machines, computers, and tablet terminals as described in this disclosure. Store terminals are installed in the dining area of ​​a restaurant or in close proximity to seats or tables where users (synonymous in this disclosure with customers) eat and drink. The fixed terminal requests the user to place an order for food and drink by voice. The request may be displayed in text on the fixed terminal's display or by the fixed terminal emitting a voice message to that effect. The user places an order for food and drink by voice to the fixed terminal. The fixed terminal acquires the user's voice information. In another embodiment, as an example, a mobile terminal includes a user terminal, a smartphone or tablet terminal owned by the user, and such terminals with the application installed. The user sits down at a seat to eat and drinks and launches the application on the mobile terminal. The application is pre-installed on the mobile terminal. Alternatively, a display showing how to install the application to be installed is placed near the seat. The user launches the application. Based on the application's instructions, the mobile terminal prompts the user to voice their food and drink order. This request may be displayed as text on the mobile terminal's screen or as an audible message from the mobile terminal. The user voices their food and drink order to the mobile terminal. The mobile terminal acquires the user's voice information.

[0044] Disclosed is a step of requesting an AI to collate language information indicated by voice with menu information that is a menu of a food and beverage store, and identify one or more menu information items similar to the language information. In at least one embodiment, an operator of a food and beverage store causes a device to store one or more menu information items. The menu information includes information such as language information, images, and videos related to the menu of food and beverages. As an example, language information such as hamburger, cheeseburger, teriyaki burger, tomato, lettuce, etc., and images and videos corresponding to the language information. Association information that associates related information or concepts can be stored in each piece of information. As an example, information that associates related or concepts such as beef, pork, and bread with the information of hamburger is stored. As will be described later, these pieces of information may be used for AI analysis.

[0045] The device requests the AI ​​to identify one or more menu items similar to the linguistic information. In at least one embodiment, the operation of the AI ​​can take any of the embodiments described above. For example, the AI ​​resides in one or more of the following: a distributed internet service system, a fixed terminal, a mobile terminal, an application, or a server. The device provides the AI ​​with linguistic information acquired by the terminal. The AI ​​acquires the stored menu information. The AI ​​performs vectorization on the linguistic information. Bags of Words can be used for the linguistic information. On the other hand, distributed representations may be more suitable for analyzing user utterances. The AI ​​also performs vectorization on the menu information. The AI ​​compares the vectors of the linguistic information with the vectors of the menu information, identifies menu items with vectors similar to the linguistic information vectors, and determines their order of similarity. In another embodiment, the AI ​​performs various natural language processing (NLP) tasks on both the linguistic information and the menu information, such as text classification, sentiment analysis, and text summarization. Through these tasks, the AI ​​compares the classified text of the linguistic information with the classified text of the menu information and identifies menu items with text that matches the classified text of the linguistic information. Alternatively, even if the text doesn't match, the AI ​​can identify menu information with a high degree of similarity and determine the order of those matches. If the AI ​​cannot identify menu information similar to the linguistic information, it can generate status information indicating that no information was found. For example, if a user voice-inputs "I want to eat beef and bread," the AI ​​will identify the menu information "hamburger," and will also identify menu information such as "cheeseburger" and "teriyaki burger."

[0046] The present invention discloses a step of transmitting identified menu information to a chef's terminal. In at least one embodiment, a computer or terminal serving as the chef's terminal is provided in the location where the chef of the restaurant is located, or in a location close to the kitchen. The terminal may be a fixed terminal or a mobile terminal. In the case of a mobile terminal, it can be a terminal that the chef can wear. The device transmits the menu information identified by the AI, or the top-ranking menu information in order of similarity, to the chef's terminal. The chef's terminal conveys the received menu information to the chef. The method of conveying the information is to display the information on the chef's terminal's display or to output the menu information by voice. After outputting the menu information, information regarding the process of making that menu or the recipe may be output along with it.

[0047] This document discloses the steps for obtaining confirmed menu information from the identified menu information to finalize an order. The device transmits the menu information identified by the AI, or multiple menu items in order of similarity, to the user terminal. The user terminal presents this information to the user and receives instructions to order these menus. Methods of presentation include displaying the menu information on the user terminal's display or outputting the menu information audibly. Methods of receiving instructions to order include displaying a button to finalize the order on the user terminal's display and determining that an order has been placed when this button is pressed. Alternatively, the user terminal obtains voice information from the user. The device requests the AI ​​to process the voice information using natural language processing. The AI ​​vectorizes the voice information and confirms its similarity to vectors indicating an order or vector information representing intent or emotion, thereby determining that it is an order instruction. The device obtains the menu information for which an order instruction has been received as confirmed menu information. Obtaining this information may involve generating new information as confirmed menu information or adding a confirmed status to the menu information. For example, the user terminal displays menu information such as "hamburger," "cheeseburger," and "teriyaki burger," and the device acquires the user's selected menu information as confirmed menu information.

[0048] Disclose the step of transmitting the confirmed menu information to the cook terminal. In at least one embodiment, the device transmits the confirmed menu information to the cook terminal. The cook terminal conveys the received confirmed menu information to the cook. The conveying method is to display the information on the display of the cook terminal or output the confirmed menu information by voice. After outputting the confirmed menu information, information regarding the process of making the menu or the recipe can be output concomitantly.

[0049] Disclose the step of requiring AI to collate the language information indicated by the voice with the arrangement information corresponding to the menu information and identify one or more arrangement information similar to the language information. In at least one embodiment, the operator of the food and beverage store causes the device to store one or more arrangement information. The arrangement information includes information such as language information, images, videos, etc. that accompany the menu information. As an example, the arrangement information is language information such as the amount of lemon sauce, the presence or absence of tomatoes, the presence or absence of lettuce, and one selected baking method out of multiple baking methods of the patty for the menu information of hamburger. Also, the arrangement information includes images, videos, etc. corresponding to those language information. Associative information that associates related information or concepts can be stored in each piece of information. As an example, information that associates sour with the information of lemon sauce, healthy with the information of tomato, and related or concepts such as vitamins, etc. is stored. As described later, these pieces of information may be used for AI analysis.

[0050] The device requests the AI ​​to identify one or more arrangement information items similar to the linguistic information. In at least one embodiment, the device provides the AI ​​with linguistic information acquired by the terminal. The AI ​​retrieves the stored arrangement information. The AI ​​vectorizes the linguistic information. Bags of Words can be used for the linguistic information. On the other hand, distributed representations may be more suitable for analyzing user utterances. The AI ​​also vectorizes the arrangement information. The AI ​​compares the vectors of the linguistic information with the vectors of the arrangement information, identifies arrangement information with vectors similar to the linguistic information vectors, and determines the order of their similarity. In another embodiment, the AI ​​performs various natural language processing (NLP) tasks on both the linguistic information and the arrangement information, such as text classification, sentiment analysis, and text summarization. Through these tasks, the AI ​​compares the classified text of the linguistic information with the classified text of the arrangement information and identifies menu information with text that matches the classified text of the linguistic information. Alternatively, even if the texts do not match, it identifies arrangement information with texts that have a high degree of similarity and determines the order of their similarity. If the AI ​​cannot identify any arrangement information similar to the linguistic information, it can generate status information indicating that no similar information was found. For example, if a user voice-inputs "I prefer sour food. I want a healthy meal," the AI ​​will identify arrangement information such as "extra lemon sauce" and "tomato."

[0051] The following describes a step of transmitting identified menu information and arrangement information to a chef's terminal. In at least one embodiment, the device transmits the identified menu information and arrangement information to the chef's terminal. The chef's terminal conveys the received identified menu information and arrangement information to the chef. This is done by displaying the information on the chef's terminal's display or by outputting the confirmed menu information by voice. After outputting the arrangement information, information regarding the process and recipe for making that menu can be output along with it.

[0052] This document discloses the steps for obtaining confirmed arrangement information from the identified arrangement information to finalize an order. The device transmits the arrangement information identified by the AI, or multiple arrangement information in order of similarity, to the user terminal. The user terminal presents this information to the user and receives instructions to order these arrangements. Methods of presentation include displaying the arrangement information on the user terminal's display or outputting the arrangement information by voice. Methods of receiving instructions to order include displaying a button to finalize the order on the user terminal's display and determining that an order has been placed when this button is pressed. Alternatively, the user terminal obtains voice information from the user. The device requests the AI ​​to process the voice information using natural language processing. The AI ​​vectorizes the voice information and confirms its similarity to vector information indicating an order or vector information representing intent or emotion, thereby determining that it is an order instruction. The device obtains the arrangement information for which an order instruction has been received as confirmed arrangement information. Obtaining this information may involve generating new information as confirmed arrangement information or adding the status information "confirmed" to the arrangement information. For example, the user terminal might present customization options such as "extra lemon sauce" and "tomato," and the device would then acquire the user's selected customization options as confirmed customization information.

[0053] The following describes the step of transmitting confirmed menu information and confirmed arrangement information to a chef's terminal. In at least one embodiment, the device transmits the confirmed menu information and confirmed arrangement information to the chef's terminal. The chef's terminal transmits the received confirmed menu information and confirmed arrangement information to the chef. The method of transmission is to display the information on the chef's terminal's display or to output the confirmed menu information and confirmed arrangement information by voice. After outputting the arrangement information, information regarding the process and recipe for making that menu can be output along with it.

[0054] The present invention discloses a step of instructing a self-propelled cart to move to a predetermined location by the approximate time elapsed for the required time corresponding to the menu information or arrangement information, or the confirmed menu information or confirmed arrangement information. In at least one embodiment, the restaurant operator stores the required time corresponding to the menu information or arrangement information in the device. The required time can be arbitrarily determined by the operator. For example, for the menu information of hamburger, it is 20 minutes. Examples of required times for arrangement information include 5 minutes for well-done patty, 3 minutes for medium, 1 minute for adding tomatoes, etc. The required time corresponding to confirmed menu information or confirmed arrangement information is the required time corresponding to each piece of menu information or arrangement information.

[0055] The device generates order time information based on the time at which menu information or arrangement information, or confirmed menu information or confirmed arrangement information, was obtained. The device then adds the corresponding required time to the order time information to generate estimated completion time information.

[0056] This disclosure concerns a self-propelled cart. In at least one embodiment, the self-propelled cart is an unmanned, self-propelled vehicle or machine. The self-propelled cart has one or more spaces for carrying food. The self-propelled cart may have a computer or terminal as defined in this disclosure. That is, the self-propelled cart may communicate wirelessly with the device. In at least one embodiment, if an object is observed to be an unmanned, self-propelled vehicle or machine, it is considered a "self-propelled cart" as defined in this disclosure. The device instructs the self-propelled cart to move to a predetermined location at approximately the estimated completion time. As a method of instructing self-propulsion, the device wirelessly transmits a program or instruction to move the self-propelled cart to the predetermined location at approximately the estimated completion time. The self-propelled cart receives the program or instruction and moves to the predetermined location at approximately the estimated completion time. Examples of predetermined locations include the location of the cook, the kitchen, and the location where the finished food is handed over.

[0057] The present invention discloses a step of summing up numerical values ​​corresponding to menu information or arrangement information, or confirmed menu information or confirmed arrangement information. In at least one embodiment, the operator of a restaurant stores numerical values ​​corresponding to menu information or arrangement information in the device. The numerical values ​​represent at least the value of these menus or arrangements. For example, a hamburger is 1000, and extra lemon sauce is 50. That is, a hamburger costs 1000 yen and lemon sauce costs 50 yen. The device sums up the numerical values ​​corresponding to menu information or arrangement information, or confirmed menu information or confirmed arrangement information. For example, if a hamburger is ordered with extra lemon sauce, the sum will be 1050.

[0058] The following describes the step of presenting the aggregated figures to the user. In at least one embodiment, the device transmits the aggregated figures to the user terminal. In another embodiment, the device transmits the aggregated figures to an accounting device for the user to pay. Examples of accounting devices include accounting devices in restaurants, cash registers, user accounting applications, user electronic payment applications, and user terminals (including smartphones).

[0059] In at least one embodiment, any one of the embodiments described above has industrial applicability and advantages in the following cases: namely, restaurants, drive-throughs, etc. In the case of a drive-through, the user terminal in the step of acquiring the user's voice may correspond to a machine installed outdoors for the drive-through. In other embodiments, in the case of a drive-through, the user terminal may correspond to a device such as a smartphone or computer owned by the user, or these devices with a dedicated application installed.

[0060] The following is an overview of the embodiments described above.

[0061] A method for ordering, comprising the steps of: a device acquiring the user's voice; a device matching the language information indicated by the voice with menu information which is a restaurant's menu, and requesting an AI to identify one or more menu items similar to the language information; and a device transmitting the identified menu items to a chef's terminal.

[0062] According to this disclosure, at the very least, users can enjoy the convenience of ordering food and beverages by voice. Furthermore, restaurants can take orders based on user voice commands without human intervention, offering industrial applicability and advantages.

[0063] A method for placing an order, comprising: a step of acquiring the user's voice; a step of comparing the language information indicated by the voice with menu information which is a restaurant's menu and requesting the AI ​​to identify one or more menu items similar to the language information; a step of acquiring confirmed menu information from the identified menu items to confirm the order; and a step of transmitting the confirmed menu information to the chef's terminal.

[0064] According to this disclosure, at the very least, users can enjoy the convenience of ordering food and beverages by voice. Furthermore, restaurants can take orders based on user voice commands without human intervention, offering industrial applicability and advantages.

[0065] A method for ordering, comprising: a step of acquiring the user's voice; a step of comparing the language information indicated by the voice with menu information which is a restaurant's menu and requesting the AI ​​to identify one or more menu items similar to the language information; a step of comparing the language information indicated by the voice with arrangement information corresponding to the menu information and requesting the AI ​​to identify one or more arrangement items similar to the language information; and a step of transmitting the identified menu information and arrangement information to the chef's terminal.

[0066] According to this disclosure, at the very least, users can enjoy the convenience of ordering food and beverages by voice. Furthermore, restaurants can take orders based on user voice commands without human intervention, offering industrial applicability and advantages.

[0067] A method for placing an order, comprising: a step of acquiring the user's voice; a step of comparing the language information indicated by the voice with menu information which is a restaurant's menu and requesting the AI ​​to identify one or more menu items similar to the language information; a step of comparing the language information indicated by the voice with arrangement information corresponding to the menu information and requesting the AI ​​to identify one or more arrangement items similar to the language information; a step of acquiring confirmed menu information to finalize the order from the identified menu information; a step of acquiring confirmed arrangement information to finalize the order from the identified arrangement information; and a step of transmitting the confirmed menu information and confirmed arrangement information to the chef's terminal.

[0068] According to this disclosure, at the very least, users can enjoy the convenience of ordering food and beverages by voice. Furthermore, restaurants can take orders based on user voices without human intervention, offering industrial applicability and advantages.

[0069] A method of ordering any of the above, further comprising the step of instructing a self-propelled trolley to move to a designated location approximately by the time the required time corresponding to the menu information or arrangement information has elapsed.

[0070] According to this disclosure, at a minimum, food and beverages ordered by the user can be quickly transported by a self-propelled cart. This has industrial applicability and advantages, as it enables the delivery of freshly prepared food to the user unmanned.

[0071] A method of ordering according to any of the above, further comprising the steps of: summing up numerical values ​​corresponding to menu information or arrangement information; and presenting the summed numerical values ​​to the user.

[0072] According to this disclosure, there are industrial applications and advantages, at least in that it is possible to process payments for food and beverages ordered by users without human intervention.

[0073] In at least one embodiment, the "step of obtaining confirmed menu information for confirming an order from the identified menu information" includes the following form of the "method by which the device receives an instruction to order." The restaurant operator defines action information relating to one or more physical actions in the device and stores it as specific action information. The action information includes information such as language information, images, and videos that accompany the physical action. For example, for the physical action of "tapping the table twice with fingers," it includes the language information and information such as images or videos showing examples of the action. Physical actions include all gestures such as raising a hand or nodding. Specific action information is information that the restaurant operator registers in the device from among the action information. The user terminal obtains the user's action information. One method of acquisition is to obtain a video or video of the user's actions using the camera on the user terminal.

[0074] The device requests the AI ​​to determine if the motion information is similar to specific motion information. The AI ​​represents the acquired motion information as a collection of visual patches, which are small data units similar to LLM text tokens, in the form of videos or images. The AI ​​takes raw video as input and outputs a temporally and spatially compressed latent representation in a network that reduces the dimensionality of the visual data about the motion information. The AI ​​also outputs a latent representation of specific motion information in the same manner as described above. The AI ​​compares the latent representation of the motion information with the latent representation of the specific motion information to determine if they are similar. For this determination, the restaurant operator can have the device store a similarity threshold for determining similarity. If the AI ​​determines that the motion information is similar to the specific motion information, it can create data or status information indicating that it is similar.

[0075] The device acquires the menu information to be ordered as confirmed menu information, provided that data or status information indicating similarity has been generated. Acquisition can be done by generating new information as confirmed menu information or by adding the status information of "confirmed" to the menu information. For example, the device presents "hamburger" as menu information to the user. The user taps the table twice with their finger. The device determines that the user's action information is specific action information and acquires "hamburger" as confirmed menu information.

[0076] In at least one embodiment, in the step of "acquiring confirmed arrangement information to confirm an order from among the identified arrangement information," the method by which the device "receives an instruction to place an order" can be the embodiment described above. That is, the operator of a restaurant defines one or more pieces of action information relating to physical actions in the device and stores them as specific action information. The user terminal acquires the user's action information. The AI ​​determines whether the action information is similar to confirmed action information. If data or status information indicating similarity is created, the device acquires the arrangement information for which an order instruction has been received as confirmed arrangement information. Acquisition can be done by generating new information as confirmed arrangement information, or by adding status information indicating confirmation to the arrangement information. As an example, the device presents "extra lemon sauce" to the user as arrangement information. The user nods. The device determines that the user's action information is specific action information and acquires "extra lemon sauce" as confirmed arrangement information.

[0077] In at least one embodiment, a device or terminal for detecting user activity information may be provided separately from the user terminal, or in combination with the user terminal, as described in the above embodiment.

[0078] In at least one embodiment, instead of operation information and specific operation information, language information and specific language information may be used as the information to be determined in the above embodiment. That is, the restaurant operator sets language information equivalent to one or more passwords in the device and stores it as specific language information. The user terminal acquires the user's language information. The acquisition method may include using the microphone of the device or terminal. The AI ​​determines whether the language information is similar to the specific operation information using natural language processing. The device requests the AI ​​to make that determination. As already described, any one or more of the methods described herein can be used for natural language processing. If data or status information indicating that the language information is similar to the specific language information is created by the AI ​​or the device, the device acquires the menu information or arrangement information that has been instructed to be ordered as confirmed menu information or confirmed arrangement information. Acquisition may include generating new information as confirmed arrangement information or adding the status information of "confirmed" to the arrangement information. As an example, the device presents "extra lemon sauce" to the user as arrangement information. The user utters a password set by the restaurant operator (examples include the restaurant's trademark, emblem, brand name, nickname, or special catchphrase). The device determines that the user's language information is specific language information and acquires "extra lemon sauce" as confirmed customization information.

[0079] The following is an overview of the embodiments described above.

[0080] The ordering method described above, in which the step of obtaining confirmed menu information to confirm the order further includes the step of the device storing specific action information relating to one or more physical actions, and the step of requesting the AI ​​to determine whether the action information is similar to the specific action information.

[0081] The ordering method described above, in which the step of obtaining confirmed menu information to finalize the order, further includes the step of making the menu information confirmed menu information when the AI ​​determines that the action information is similar to specific action information.

[0082] According to this disclosure, at the very least, users can confirm their orders by gesture, making it easier to confirm orders than by touching a touch panel as in conventional technologies. This has industrial applicability and advantages, as it improves the convenience of using the device or system.

[0083] The invention disclosed herein only needs to achieve at least one of the effects described above.

[0084] An ordering method in which a device stores, retrieves, or loads associative information that is different from the restaurant's menu information and evokes related information or concepts related to menu information, obtains the user's language information, and presents the menu information identified based on the related or associative information and the language information to the user's terminal.

[0085] A method of ordering, wherein the device stores, retrieves, or loads associative information that is different from the menu information of a restaurant and evokes related information or concepts related to the menu information, obtains the user's language information, and transmits the menu information identified based on the related or associative information and the language information.

[0086] A method of ordering, wherein the device acquires the user's language information, stores, retrieves, or loads associative information that is different from the restaurant's menu information and evokes related information or concepts related to the menu information, and matches the language information with the related information or associative information to identify the menu.

[0087] The above method involves requesting the AI ​​to identify a menu item.

[0088] A method of ordering, wherein the device predicts menu information from related information or associative information that evokes concepts related to menu information, which is different from the menu information of the restaurant, obtains the user's language information, and transmits the menu information identified based on the related information or associative information and the language information.

[0089] The above method, but with the exception of using AI for prediction.

[0090] The above method, further comprising a method for memorizing related or associative information.

[0091] The above method, wherein, when transmitting information, two or more specified menu items are presented to the user's terminal.

[0092] An ordering method, wherein the device stores, retrieves, or loads associative information that is different from the menu information of a restaurant and evokes related information or concepts related to menu information; obtains the user's language information; and matches the language information with the stored, retrieved, or loaded related information or associative information.

[0093] The above method, wherein the linguistic information includes audio or text.

[0094] The above method, wherein the language information includes information entered via keyboard, mouse, or touchscreen.

[0095] An ordering device that stores, retrieves, or loads associative information that is different from the menu information of a restaurant and evokes related information or concepts related to the menu information, obtains the user's language information, and presents the menu information identified based on the related or associative information and the language information to the user's terminal.

[0096] An ordering device that stores, retrieves, or loads associative information that is different from the menu information of a restaurant and evokes related information or concepts related to menu information, obtains the user's language information, and transmits menu information identified based on the related or associative information and the language information.

[0097] An ordering device that acquires the user's language information, stores, retrieves, or loads associative information that is different from the restaurant's menu information and evokes related information or concepts related to the menu information, matches the language information with the related information or associative information, and identifies the menu.

[0098] An ordering device that predicts menu information from related information or associative information that evokes concepts related to menu information, which is different from the menu information of a restaurant, acquires the user's language information, and transmits the menu information identified based on the related information or associative information and the language information.

[0099] An ordering device that stores, retrieves, or loads associative information that is different from the menu information of a restaurant and evokes related information or concepts related to the menu information; acquires the user's language information; and compares the language information with the stored, retrieved, or loaded related information or associative information.

[0100] An ordering method in which a device predicts menu information from related information or associative information that evokes concepts related to menu information, which is different from the menu information of a restaurant, obtains the user's language information, and presents the menu information identified based on the related information or associative information and the language information to the user's terminal.

[0101] A method of ordering, wherein the device predicts menu information from related information or associative information that evokes concepts related to menu information, which is different from the menu information of the restaurant, obtains the user's language information, and transmits the menu information identified based on the related information or associative information and the language information.

[0102] A method of ordering, wherein the device obtains the user's language information, predicts menu information from related information or associative information that is different from the restaurant's menu information and evokes related information or concepts associated with the menu information, and identifies the menu by matching the language information with the related information or associative information.

[0103] The above method involves requesting the AI ​​to identify a menu item.

[0104] A method of ordering, wherein the device predicts menu information from related information or associative information that evokes concepts related to menu information, which is different from the menu information of the restaurant, obtains the user's language information, and transmits the menu information identified based on the related information or associative information and the language information.

[0105] The above method, but with the exception of using AI for prediction.

[0106] The above method, further comprising a method for memorizing related or associative information.

[0107] The above method, wherein, when transmitting information, two or more specified menu items are presented to the user's terminal.

[0108] An ordering method, wherein the device predicts menu information from associative information that evokes related information or concepts associated with menu information, which is different from the menu information of the restaurant; obtains the user's language information; and matches the language information with the stored, retrieved, or loaded related information or associative information.

[0109] An ordering device that predicts menu information from related information or associative information that evokes concepts related to menu information, which is different from the menu information of a restaurant, acquires the user's language information, and presents the menu information identified based on the related information or associative information and the language information to the user's terminal.

[0110] An ordering device that predicts menu information from related information or associative information that evokes concepts related to menu information, which is different from the menu information of a restaurant, acquires the user's language information, and transmits the menu information identified based on the related information or associative information and the language information.

[0111] An ordering device that acquires the user's language information, predicts menu information from related information or associative information that evokes concepts related to menu information but is different from the restaurant's menu information, matches the language information with the related information or associative information, and identifies the menu.

[0112] An ordering device that predicts menu information from related information or associative information that evokes concepts related to menu information, which is different from the menu information of a restaurant, acquires the user's language information, and transmits the menu information identified based on the related information or associative information and the language information.

[0113] An ordering device that predicts menu information from associative information that is different from the menu information of a restaurant and evokes related information or concepts related to the menu information, acquires the user's language information, and compares the language information with the stored, retrieved, or loaded related information or associative information.

[0114] A computer-readable recording medium in a device equipped with a processor, which records a program that causes the processor to execute the above method.

[0115] A program consisting of instructions that cause the processor to execute the above method.

[0116] The invention disclosed herein only needs to achieve at least one of the effects described above.

Claims

1. An ordering method in which a device stores, retrieves, or loads associative information that is different from the menu information of a restaurant and evokes related information or concepts related to menu information, obtains the user's language information, and presents the menu information identified based on the related or associative information and the language information to the user's terminal.

2. An ordering method in which a device stores, retrieves, or loads associative information that is different from the menu information of a restaurant and evokes related information or concepts related to menu information, obtains the user's language information, and transmits the menu information identified based on the related information or associative information and the language information.

3. An ordering method, wherein the device acquires the user's language information, stores, retrieves, or loads associative information that is different from the restaurant's menu information and evokes related information or concepts related to the menu information, and matches the language information with the related information or associative information to identify the menu.

4. The method according to claim 3, wherein when identifying a menu item, the method involves requesting the AI ​​to identify it.

5. An ordering method in which a device predicts menu information from related information or associative information that is different from the menu information of a restaurant and evokes related information or concepts related to menu information, obtains the user's language information, and transmits the menu information identified based on the related information or associative information and the language information.

6. The method according to claim 5, wherein the prediction is made using AI.

7. The method according to claim 5, further comprising a method for storing related information or associative information.

8. A method according to claim 5, wherein, when transmitting, two or more specified menu items are presented to the user terminal.

9. An ordering method, wherein the device stores, retrieves, or loads associative information that is different from the menu information of a restaurant and evokes related information or concepts related to menu information; obtains the user's language information; and matches the language information with the stored, retrieved, or loaded related information or associative information.

10. A method according to any one of claims 1, 2, 3, 5, or 9, wherein the language information includes speech or text.

11. A method according to any one of claims 1, 2, 3, 5, or 9, wherein the language information includes information entered via a keyboard, mouse, or touchscreen.

12. An ordering device that stores, retrieves, or loads associative information that is different from the menu information of a restaurant and evokes related information or concepts related to menu information, acquires the user's language information, and presents the menu information identified based on the related or associative information and the language information to the user's terminal.

13. An ordering device that stores, retrieves, or loads associative information that is different from the menu information of a restaurant and evokes related information or concepts related to menu information, acquires the user's language information, and transmits menu information identified based on the related information or associative information and the language information.

14. An ordering device that acquires the user's language information, stores, retrieves, or loads associative information that is different from the restaurant's menu information and evokes related information or concepts related to the menu information, and identifies the menu by matching the language information with the related information or associative information.

15. An ordering device that predicts menu information from related information or associative information that evokes concepts related to menu information, which is different from the menu information of a restaurant, acquires the user's language information, and transmits the menu information identified based on the related information or associative information and the language information.

16. An ordering device that stores, retrieves, or loads associative information that is different from the menu information of a restaurant and evokes related information or concepts related to menu information; acquires the user's language information; and compares the language information with the stored, retrieved, or loaded related information or associative information.

17. An ordering method in which a device predicts menu information from related information or associative information that is different from the menu information of a restaurant and evokes related information or concepts related to menu information, obtains the user's language information, and presents the menu information identified based on the related information or associative information and the language information to the user's terminal.

18. An ordering method in which a device predicts menu information from related information or associative information that is different from the menu information of a restaurant and evokes related information or concepts related to menu information, obtains the user's language information, and transmits the menu information identified based on the related information or associative information and the language information.

19. An ordering method comprising: a device that acquires the user's language information; predicts menu information from related information or associative information that is different from the restaurant's menu information and evokes related information or concepts related to menu information; and identifies a menu by matching the language information with the related information or associative information.

20. The method according to claim 19, wherein when identifying a menu item, the method involves requesting the AI ​​to identify it.

21. An ordering method in which a device predicts menu information from related information or associative information that is different from the menu information of a restaurant and evokes related information or concepts related to menu information, obtains the user's language information, and transmits the menu information identified based on the related information or associative information and the language information.

22. The method according to claim 21, wherein the prediction is made using AI.

23. The method according to claim 21, further comprising a method for storing related information or associative information.

24. A method according to claim 21, wherein, when transmitting, two or more specified menu items are presented to the user terminal.

25. An ordering method, wherein the device predicts menu information from related information or associative information that is different from the menu information of a restaurant and evokes related information or concepts related to menu information, obtains the user's language information, and matches the language information with the related information or associative information that has been stored, retrieved, or loaded.

26. An ordering device that predicts menu information from related information or associative information that evokes concepts related to menu information, which is different from the menu information of a restaurant, acquires the user's language information, and presents the menu information identified based on the related information or associative information and the language information to the user's terminal.

27. An ordering device that predicts menu information from related information or associative information that evokes concepts related to menu information, which is different from the menu information of a restaurant, acquires the user's language information, and transmits the menu information identified based on the related information or associative information and the language information.

28. An ordering device that acquires the user's language information, predicts menu information from related information or associative information that is different from the restaurant's menu information and evokes related information or concepts related to menu information, matches the language information with the related information or associative information, and identifies the menu.

29. An ordering device that predicts menu information from related information or associative information that evokes concepts related to menu information, which is different from the menu information of a restaurant, acquires the user's language information, and transmits the menu information identified based on the related information or associative information and the language information.

30. An ordering device that predicts menu information from related information or associative information that is different from the menu information of a restaurant and evokes related information or concepts related to menu information, acquires the user's language information, and compares the language information with stored, retrieved, or loaded related information or associative information.

31. A computer-readable recording medium in a device equipped with a processor, which stores a program causing the processor to execute the method according to any one of claims 1, 2, 3, 5, 9, 17, 18, 19, 21, or 25.

32. A program comprising instructions for causing a processor to execute the method according to any one of claims 1, 2, 3, 5, 9, 17, 18, 19, 21, or 25.

Images