System

Abstract

To provide a system for reducing an operation burden and improving customer experience.SOLUTION: A system comprises: means for generating a commodity description from a photograph of a commodity taken by a smartphone; means for inputting an instruction to translate the commodity description into a generated AI model to generate a translated result of the commodity description; means for inputting a question at the time of a user's menu review into the generated AI model to generate an answer to the question; and means for analyzing the user's input to generate recommendation information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Operating a restaurant requires a great deal of time and effort for creating menus, product descriptions, and translation work. Customers also face stressful experiences, such as waiting in line for wait staff or cashiers, and having questions about the menu. In particular, services for foreign tourists are often not adequately provided due to language barriers. [Means for solving the problem]

[0005] This invention provides a means for creating menus using smartphones, an AI means for generating product descriptions from smartphone photos, and a means for AI to generate translation work. This reduces the operational burden on restaurants and improves the customer experience. In addition, the AI ​​can provide explanations for questions when considering menus and make recommendations based on preferences, providing a more personalized service. Furthermore, the multilingual translation AI will enhance services for foreign tourists. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] In this embodiment, a restaurant manager creates a menu using a smartphone. Specifically, the manager uploads photos of the products taken with the smartphone camera, and the AI ​​processes the photos.

[0029] The system automatically recognizes the characteristics and features of products from images, and generates a product description based on the recognition results. This product description is then displayed to customers on the restaurant's website or application.

[0030] "Example 2"

[0031] Furthermore, in this embodiment of the present invention, AI generates translation tasks. Specifically, it translates the generated product descriptions into multiple languages. This makes the product descriptions easier to understand for foreign tourists. For example, it is possible to translate into major tourist languages ​​such as English, Chinese, and Korean.

[0032] "Example 3"

[0033] This embodiment of the present invention also improves the customer experience. Specifically, when a customer is considering a menu, the AI ​​will explain their question and recommend products based on their preferences. For example, if a customer enters information such as "I like spicy food," the AI ​​will recommend spicy dishes. Also, if a customer asks a question such as "What are the ingredients in this dish?", the AI ​​will provide an answer to that question.

[0034] The processing flow of each embodiment will be described below.

[0035] "Example 1"

[0036] Step 1: The restaurant manager takes a photo of the product using a smartphone camera.

[0037] Step 2: Upload the photos you have taken to the system.

[0038] Step 3: The AI ​​in the system automatically recognizes the product's characteristics and features from the photo.

[0039] Step 4: The AI ​​generates a product description based on the recognition results.

[0040] Step 5: The generated product description is displayed to customers on the restaurant's website or application.

[0041] "Example 2"

[0042] Step 1: Get the AI-generated product description.

[0043] Step 2: AI translates the product description into multiple languages.

[0044] Step 3: The translated product description will be displayed to foreign tourists on the restaurant's website or app.

[0045] "Example 3"

[0046] Step 1: As customers consider the menu, they input their questions into the AI.

[0047] Step 2: The AI ​​generates an answer to that question.

[0048] Step 3: AI recommends products based on customer preferences.

[0049] Step 4: The AI-generated answers and recommendations are displayed to the customer.

[0050] Example 1

[0051] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0052] When restaurant operators create menus, it is time-consuming to take photos of the products, recognize the characteristics and features of the products from the photos, and generate product descriptions. Furthermore, there is a lack of multilingual support for foreign tourists and ways to improve the customer experience. This has led to a demand for more efficient menu creation and improved customer satisfaction.

[0053] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0054] In this invention, the server includes a menu creation means using a smartphone, an artificial intelligence means for generating product descriptions from smartphone photos, a means for generating translations by artificial intelligence, a means for improving the customer experience by eliminating the need to wait in line for waiters or cashiers, a means for providing artificial intelligence to answer questions when considering menu items, a means for making recommendations based on preferences, a means for uploading product photos taken with a smartphone camera and having the artificial intelligence automatically recognize the characteristics and features of the products from the photos, and a means for generating product descriptions based on the recognition results, thereby making it possible to improve the efficiency of menu creation and customer satisfaction.

[0055] A "smartphone" is a multi-functional mobile device that can connect to the Internet and use applications in addition to the functions of a mobile phone.

[0056] "Menu creation means" refers to the methods and tools used by restaurant operators to create menus, particularly those using smartphones.

[0057] "Artificial intelligence means" refers to technologies that use machine learning and data analysis to automatically perform specific tasks.

[0058] "Means for generating translation work using artificial intelligence" refers to methods and technologies for translating text into multiple languages ​​using artificial intelligence.

[0059] "Measures to improve customer experience" refers to methods and tools for improving the convenience and satisfaction of customers when using services.

[0060] "Means for AI to explain questions when considering a menu" refers to methods and technologies in which AI automatically provides answers to questions that arise when customers are considering a menu.

[0061] "Preference-based recommendation methods" refers to methods and technologies that recommend appropriate products and services based on a customer's preferences and past choices.

[0062] "Means for automatically recognizing product characteristics and features" refers to methods and technologies that use artificial intelligence to automatically extract product characteristics and features from photographs and data.

[0063] "Means for generating product descriptions" refers to methods or techniques for creating product descriptions in natural language based on recognized product characteristics and features.

[0064] This invention is a system that allows restaurant operators to create menus using smartphones. Specifically, a photo of a product taken with the smartphone camera is uploaded, and artificial intelligence (AI) automatically recognizes the product's characteristics and features from the photo and generates a product description based on the recognition results. This product description is displayed to customers on the restaurant's website or application.

[0065] Hardware and software used

[0066] Smartphone: A device for taking and uploading photos.

[0067] Server: Processes data and runs AI models.

[0068] Artificial intelligence models: Image recognition models and natural language generation models based on TENSORFLOW® and PyTorch (e.g., GPT-3®, BERT).

[0069] Data processing and calculation

[0070] 1. The user takes a photo of the product with their smartphone

[0071] The user opens the camera app on their smartphone and takes a photo of a product, for example, a new dessert called "chocolate cake."

[0072] 2. The user uploads a photo to the system

[0073] The user opens the dedicated application, selects the photos they have taken, and presses the upload button, which sends the photos to the server via the Internet.

[0074] 3. The server receives the photo and inputs it into the AI ​​model

[0075] The server receives the uploaded photos and feeds them into an AI model for image processing, which is based on TensorFlow and PyTorch.

[0076] 4. The server uses an AI model to recognize the product's characteristics and features.

[0077] The server uses AI models to recognize product characteristics and features from the photo, such as the type of dessert, key ingredients, and visual features.

[0078] 5. The server generates a product description based on the recognition results.

[0079] The server generates a product description based on the AI ​​recognition results. This product description is created using natural language generation technology, using software such as GPT-3 and BERT.

[0080] 6. The server displays the product description on the website or application.

[0081] The server then displays the generated product description on the restaurant's website or application, where customers can view it.

[0082] Specific examples

[0083] A user takes a photo of a new dessert, "Chocolate Cake," and uploads it to the system. The server receives the photo and uses an AI model to recognize features such as "Chocolate Cake," "Cream Topping," and "Berry Decoration." The server then uses GPT-3 to generate a product description: "This chocolate cake is characterized by its rich chocolate and creamy topping. The berry decoration makes it a visually stunning dish." This is then displayed on the website.

[0084] Prompt Sentence Examples

[0085] Upload a photo of your new dessert and our AI will recognize its properties and characteristics and generate a product description.

[0086] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0087] Step 1:

[0088] A user takes a photo of an item with their smartphone.

[0089] Input: Product (e.g. Chocolate Cake)

[0090] Output: Product photo

[0091] Specific action: The user opens the camera app on their smartphone and takes a photo of a product, for example, a new dessert called "Chocolate Cake."

[0092] Step 2:

[0093] A user uploads a photo to the system.

[0094] Input: Product photo taken

[0095] Output: Photo data sent to the server

[0096] How it works: The user opens the application, selects the photo they have taken, and presses the upload button, which sends the photo to the server via the Internet.

[0097] Step 3:

[0098] The server receives the photos and inputs them into the AI ​​model.

[0099] Input: Photo data sent to the server

[0100] Output: Photo data fed into the AI ​​model

[0101] Specific operation: The server receives an HTTP request, temporarily stores the photo data, and inputs it into a TensorFlow or PyTorch model.

[0102] Step 4:

[0103] The server uses an AI model to recognize the product's characteristics and features.

[0104] Input: Photo data fed into the AI ​​model

[0105] Output: Recognized product characteristics or features (e.g., chocolate cake, cream topping, berry decoration)

[0106] What it does: The server runs an AI model to extract product characteristics and features from the photo, such as the type of dessert, its main ingredients, and visual features.

[0107] Step 5:

[0108] The server generates a product description based on the recognition results.

[0109] Input: Recognized product characteristics or features

[0110] Output: Generated product description (e.g., "This chocolate cake features rich chocolate and a creamy topping. The berry decoration makes it visually appealing.")

[0111] Specific operation: The server uses GPT-3 or BERT to generate a product description using the recognition results as input.

[0112] Step 6:

[0113] The server displays the product description on the website or application.

[0114] Input: Generated product description

[0115] Output: Product description displayed on your website or application

[0116] What happens: The server saves the product description in the website database and sends it to the front-end for display, where customers can view it.

[0117] (Application example 1)

[0118] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0119] Traditionally, creating menus in restaurants was often done manually, which was time-consuming and labor-intensive, and made it difficult to fully convey the appeal of the products. Other issues included a lack of multilingual support for foreign tourists and a lack of recommendation functions tailored to customer preferences. Furthermore, there were limited ways to improve the in-store customer experience, with customers having to wait in line for wait staff or cashiers, and there was a need to improve customer satisfaction.

[0120] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0121] In this invention, the server includes a menu creation means using a smartphone, an AI means for generating product descriptions from smartphone photos, an AI means for generating translations, a means for improving the customer experience by eliminating the need to wait in line for waiters or cashiers, an AI means for explaining questions when considering a menu, a means for making recommendations based on preferences, a means for recognizing product characteristics and features using an image recognition model, and a means for generating product descriptions based on the recognized features using a natural language generation model. This enables more efficient menu creation and the automatic generation of attractive product descriptions, enabling multilingual support, an improved customer experience, and recommendations based on individual preferences.

[0122] A "menu creation means using a smartphone" is a means for efficiently creating a restaurant menu using a smartphone's camera and applications.

[0123] "AI means for generating product descriptions from smartphone photos" refers to an artificial intelligence means for analyzing product photos taken with a smartphone, recognizing their characteristics and features, and automatically generating product descriptions.

[0124] "Means for AI-generated translation work" refers to a method for using artificial intelligence to translate product descriptions and menu contents into multiple languages.

[0125] "Measures to improve the customer experience by eliminating the need to wait in line for waiters or cashiers" refers to measures that allow customers to receive service smoothly without having to wait in line for waiters or cashiers.

[0126] "Means for AI to explain questions when considering a menu" refers to a means by which artificial intelligence automatically provides answers to questions that arise when customers are considering a menu.

[0127] "Means for recommendation based on preferences" refers to means for recommending appropriate products and menus based on a customer's past choices and preferences.

[0128] "Means for recognizing the characteristics and features of a product using an image recognition model" refers to means for automatically recognizing the characteristics and features of a product from a photograph of the product using image recognition technology.

[0129] "Means for generating product descriptions based on features recognized using a natural language generation model" refers to means for automatically generating attractive product descriptions using natural language generation technology based on recognized product characteristics and features.

[0130] A system for implementing this invention includes a menu creation means using a smartphone, an AI means for generating product descriptions from smartphone photos, a means for generating translation work using AI, a means for improving the customer experience by eliminating the need to wait in line for waiters or cashiers, a means for AI to explain questions that arise when considering a menu, a means for making recommendations based on preferences, a means for recognizing product characteristics and features using an image recognition model, and a means for generating product descriptions based on the features recognized using a natural language generation model.

[0131] System Program

[0132] The server implements the system using the following hardware and software.

[0133] Hardware:

[0134] Smartphone (with camera)

[0135] Server (for hosting AI models)

[0136] software:

[0137] TensorFlow: A library for image recognition

[0138] OpenAI® GPT-3: An API for Natural Language Generation

[0139] Processing Description

[0140] Image Recognition:

[0141] Users take photos of dishes with their smartphone camera and upload them to the server through the application. The server then uses TensorFlow's ResNet50 model to recognize the characteristics and features of the dishes from the images. For example, features such as "chicken curry, spicy, tomato-based" can be recognized.

[0142] Natural language generation:

[0143] The server converts the recognized features into text and sends it to OpenAI GPT-3 as a prompt. An example of a prompt is "Generate an appealing product description for a dish with the following characteristics: chicken curry, spicy, tomato-based." GPT-3 generates an appealing product description based on this prompt. For example, it generates a product description such as "Spicy chicken curry features juicy chicken simmered in a tomato-based sauce. The fragrant spices will whet your appetite."

[0144] Translation work:

[0145] The generated product description can be translated into multiple languages ​​as needed. The server uses AI to translate into multiple languages, such as English and Chinese.

[0146] Improved customer experience:

[0147] Customers can use their smartphones to select menu items and place orders in the store, reducing waiting time for waiters and cashiers and enabling smoother service.

[0148] Question explanation and recommendations:

[0149] AI automatically answers questions that arise as customers consider menu items, and recommends appropriate products and menu items based on customers' past choices and preferences.

[0150] This makes it possible to create menus more efficiently and automatically generate compelling product descriptions, enabling multilingual support, improved customer experience, and personalized recommendations.

[0151] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0152] Step 1:

[0153] A user takes a photo of a dish with the camera on their smartphone and uploads it to the server through the application. The input is the photo of the dish, and the output is the image data sent to the server.

[0154] Step 2:

[0155] The server uses TensorFlow's ResNet50 model to recognize the characteristics and features of food from uploaded image data. The input is the image data, and the output is a list of recognized characteristics and features. Specifically, the server preprocesses the image data, inputs it into the model, and obtains the prediction results.

[0156] Step 3:

[0157] The server converts the recognized traits and features into a string and sends it to OpenAI GPT-3 as a prompt. The input is a list of traits and features, and the output is a prompt. The specific operation is to format the traits and features and generate a prompt.

[0158] Step 4:

[0159] The server receives the response from GPT-3 and generates an attractive product description. The input is the prompt sentence, and the output is the generated product description. Specifically, it calls the GPT-3 API, parses the response, and obtains the product description.

[0160] Step 5:

[0161] The server translates the generated product description into multiple languages ​​as needed. The input is the product description, and the output is the translated product description. Specifically, it calls the translation API to translate into multiple languages.

[0162] Step 6:

[0163] Users use their smartphones to select menu items and place orders in the store. The input is the generated product description, and the output is the user's order information. Specifically, the application displays the product description and the user selects the order.

[0164] Step 7:

[0165] The server uses AI to automatically provide answers to questions that arise when the user is considering the menu. The input is the user's question, and the output is the answer provided by the AI. Specifically, the server analyzes the question and generates an appropriate answer.

[0166] Step 8:

[0167] The server recommends appropriate products and menus based on the user's past selections and preferences. The input is the user's past selection data, and the output is the recommended products and menus. Specifically, the server analyzes the past selection data and uses a recommendation algorithm to recommend products.

[0168] Example 2

[0169] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0170] In the previous system, translation of product descriptions into multiple languages ​​was often done manually, which was a time-consuming and labor-intensive process. There was also the issue of insufficient service for foreign tourists, making product descriptions difficult to understand. Furthermore, there was a risk of customer satisfaction declining because there was insufficient effort to improve the customer experience or resolve questions when considering menu items.

[0171] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0172] In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for generating translations using the artificial intelligence, a means for improving the customer experience, a means for the artificial intelligence to explain questions that arise when considering a menu, a means for making recommendations based on preferences, a means for translating product descriptions into multiple languages, a means for generating prompt sentences using a generative AI model, a means for inputting the prompt sentences and product descriptions into the generative AI model and obtaining the translation results, a means for saving the translation results in a database, a means for transmitting the translation results to a terminal, and a means for the terminal to display the translation results. This automates the translation of product descriptions into multiple languages, improves service for foreign tourists, and enables an improved customer experience and resolution of questions that arise when considering a menu.

[0173] A "smart device" is a portable electronic device such as a smartphone or tablet that has advanced computing power and communication functions.

[0174] "Menu creation means" refers to functions or applications for creating menus for restaurants or retail stores using smart devices.

[0175] "Artificial intelligence means for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with a smart device and automatically generates product descriptions based on those images.

[0176] "Means for generating translation work using artificial intelligence" refers to a function that uses artificial intelligence to translate text data into other languages.

[0177] "Means to improve the customer experience" are functions and methods for improving the convenience and satisfaction of customers when using products and services.

[0178] "A means for AI to explain questions when considering a menu" is a function in which AI automatically provides answers to questions or queries that arise when customers are considering a menu.

[0179] "Preference-based recommendations" is a feature that recommends appropriate products and services based on a customer's past choices and preferences.

[0180] "Means for translating product descriptions into multiple languages" is a function for translating product descriptions into multiple languages.

[0181] A "generative AI model" is an artificial intelligence model that has been trained to perform tasks such as text generation or translation.

[0182] A "prompt" is an instruction entered into a generative AI model to make it perform a specific task.

[0183] "Means for saving translation results in a database" refers to a function for saving translation results generated by a generative AI model in a database.

[0184] The "means for transmitting the translation result to the terminal" is a function for transmitting the translation result stored in the database to the user's terminal.

[0185] The "means for the terminal to display the translation result" is a function that displays the translation result received by the user's terminal on the screen.

[0186] This invention is a system including a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images, a means for generating translation work using artificial intelligence, a means for improving customer experience, a means for artificial intelligence to explain questions when considering menus, a means for making recommendations based on preferences, a means for translating product descriptions into multiple languages, a means for generating prompt sentences using a generative AI model, a means for inputting the prompt sentences and product descriptions into the generative AI model and obtaining the translation results, a means for saving the translation results in a database, a means for transmitting the translation results to a terminal, and a means for the terminal to display the translation results.

[0187] Hardware and software used

[0188] Hardware:

[0189] Smart devices (smartphones, tablets, etc.)

[0190] Server (a computer with high-performance computing capabilities)

[0191] software:

[0192] Generative AI models (e.g., OpenAI's GPT-4 (registered trademark))

[0193] Database Management Systems

[0194] Communication protocol (e.g. HTTP / HTTPS)

[0195] Data processing and calculation

[0196] Generate product description:

[0197] A user takes a picture of a product using a smart device and sends the image to a server, which then uses artificial intelligence means to generate a product description from the image. Specifically, the server uses image analysis algorithms to recognize the product's characteristics and features and generates a product description based on the image.

[0198] Generate prompt statement:

[0199] The server uses the generative AI model to generate a prompt sentence, which specifies the language to be translated (e.g., English, Chinese, or Korean).

[0200] Execution of translation work:

[0201] The server inputs the generated prompt and product description into the generative AI model and obtains the translation results. The generative AI model performs multilingual translation based on the input text.

[0202] Saving and sending translation results:

[0203] The server stores the translation results in a database, which is then sent to the user's device, where they are displayed.

[0204] Specific examples

[0205] Examples:

[0206] A user enters a product description in Japanese: "This product is made with high-quality materials."

[0207] The server generates the prompt "Translate the following product description into English: This product is made from high-quality materials."

[0208] The server inputs the prompt and product description into the generated AI model and obtains the English translation result, "This product uses high-quality materials."

[0209] The server stores the translation results in a database and sends them to the terminal.

[0210] The device displays the translation results for the user to confirm.

[0211] In this way, the system enables the server, terminals, and users to work together to translate product descriptions into multiple languages.

[0212] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0213] Step 1:

[0214] The user enters a product description.

[0215] The user inputs a product description in text format using a smart device, and the input product description is sent from the smart device to the server.

[0216] Input: Product description text

[0217] Output: Product description text sent to the server

[0218] Step 2:

[0219] The server receives the product description.

[0220] The server receives the product description text sent by the user and temporarily stores it in a memory.

[0221] Input: Product description text submitted by the user

[0222] Output: Product description text stored in memory

[0223] Step 3:

[0224] The server generates a prompt.

[0225] The server generates a prompt sentence by specifying the language to be translated. For example, if translation into English, Chinese, and Korean is required, the server generates a prompt sentence corresponding to each language.

[0226] Input: Product description text, target language

[0227] Output: Generated prompt statement

[0228] Step 4:

[0229] The server inputs a prompt and product description into the generative AI model.

[0230] The server inputs the generated prompt and product description text into the generative AI model, which then translates the input text.

[0231] Input: prompt text, product description text

[0232] Output: The data fed into the generative AI model

[0233] Step 5:

[0234] The server retrieves the translation results.

[0235] The server obtains translation results from the generative AI model, which are then separated into separate languages.

[0236] Input: Data fed into a generative AI model

[0237] Output: Translation result text

[0238] Step 6:

[0239] The server stores the translation results in a database.

[0240] The server saves the translation results in a database, associating the original Japanese product description with the translation results.

[0241] Input: Translation result text, original product description text

[0242] Output: Translation results stored in a database

[0243] Step 7:

[0244] The server sends the translation results to the terminal.

[0245] The server then transmits the saved translation results to the user's device, providing the data in a format that the user can access.

[0246] Input: Translation results stored in the database

[0247] Output: Translation results sent to the device

[0248] Step 8:

[0249] The device will display the translation results.

[0250] The terminal receives the translation results sent from the server and displays them to the user, who can then check the displayed translation results and make corrections or retranslations as necessary.

[0251] Input: Translation results sent from the server

[0252] Output: Translation results displayed on the device

[0253] (Application example 2)

[0254] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0255] In modern brick-and-mortar stores, foreign tourists face language barriers when purchasing products, making it difficult for them to understand product descriptions. Visually impaired people and those with limited literacy skills also face limited options for understanding product descriptions. This can lead to a poor customer experience and potentially impact sales.

[0256] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a menu creation means using a smartphone, an artificial intelligence means for generating product descriptions from photos on the smartphone, a means for generating translation work by artificial intelligence, a means for improving the customer experience, a means for explaining questions when considering menus by artificial intelligence, a means for making recommendations according to preferences, and a means for translating product descriptions into multiple languages ​​and displaying and playing aloud the product descriptions in the user's selected language. This makes it possible to make product descriptions easier to understand for foreign tourists, visually impaired people, and people with difficulty reading and writing.

[0257] "Menu creation means using a smartphone" refers to a means for creating a menu using a smartphone.

[0258] "Artificial intelligence means for generating product descriptions from photos taken with a smartphone" refers to a means that uses artificial intelligence technology to automatically generate product descriptions from photos taken with a smartphone.

[0259] "Means for generating translation work using artificial intelligence" refers to means for translating text into multiple languages ​​using artificial intelligence.

[0260] "Measures to improve customer experience" are measures to improve the experience customers have when purchasing products.

[0261] "Means for AI to explain questions when considering a menu" refers to a means for AI to answer questions that arise when customers are considering a menu.

[0262] "Means for recommendation based on preferences" refers to means for recommending products and services based on customer preferences.

[0263] "Means for translating product descriptions into multiple languages ​​and displaying and playing audio in the language set by the user" means means for translating product descriptions into multiple languages ​​and displaying and playing audio in the language set by the user.

[0264] To implement this invention, it is necessary to build a system that combines a smartphone, artificial intelligence technology, a translation API, and audio playback functionality.

[0265] First, smartphones have a camera function that allows users to take pictures of products. When a user takes a picture of a product, the image data is sent to a server. The server then generates a product description using image recognition technology. This image recognition technology may use, for example, Google (registered trademark) Cloud Vision API.

[0266] The generated product description is then translated into multiple languages ​​by artificial intelligence on the server using translation services such as Google Cloud Translation API, and the translated text is sent to and displayed on the user's smartphone based on their language preference.

[0267] Furthermore, the translated product descriptions can be played back aloud using the smartphone's voice playback function, making it accessible to people with visual impairments and those with limited literacy.

[0268] As a concrete example, consider a tourist looking to purchase a leather wallet at a brick-and-mortar store in Japan. When the tourist scans the tag on the leather wallet with their smartphone camera, the app automatically recognizes the product description and translates it into the user's language (e.g., English). The translated description is displayed and played aloud: "This is a high-quality leather wallet."

[0269] An example of a prompt sentence might be:

[0270] I am developing an application that translates product descriptions into multiple languages. Please translate the following text into the specified languages:

[0271] Text: "This is a high-quality leather wallet."

[0272] Target language: Japanese

[0273] In this way, foreign tourists, visually impaired people, and people with limited literacy will be able to understand product descriptions without experiencing language barriers when purchasing products in physical stores.

[0274] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0275] Step 1:

[0276] A user takes a photo of a product using the smartphone camera. The input is the product image taken with the smartphone camera. The output is the captured product image data.

[0277] Step 2:

[0278] The terminal sends the captured product image data to the server. The input is the product image data. The output is the product image data sent to the server.

[0279] Step 3:

[0280] The server generates a product description using image recognition technology. The input is product image data. The server extracts text information from the image using Google Cloud Vision API or similar and generates a product description. The output is the generated product description text.

[0281] Step 4:

[0282] The server translates the generated product description text into multiple languages. The input is the product description text. The server translates it into the specified language using the Google Cloud Translation API. The output is the translated product description text.

[0283] Step 5:

[0284] The server sends the translated product description text to the terminal. The input is the translated product description text. The output is the translated product description text sent to the terminal.

[0285] Step 6:

[0286] The device displays the translated product description text. The input is the translated product description text. The output is the translated product description displayed on the smartphone screen.

[0287] Step 7:

[0288] The device plays the translated product description text aloud. The input is the translated product description text. The device converts the text into speech using the smartphone's voice playback function and plays it back. The output is the translated product description played aloud.

[0289] Example 3

[0290] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0291] Conventional menu creation systems and customer experience improvement systems do not adequately answer questions users have when considering menus, nor recommend products that meet individual preferences. They also lack the ability to reduce waiting times or provide multilingual support. This can lead to lower customer satisfaction and negatively impact store sales.

[0292] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0293] In this invention, the server includes a means for a user to input information, a means for transmitting the input information to the server, a means for the server to analyze the input information using a generative AI model, a means for generating recommendations and answers based on the analysis results, a means for transmitting the generated recommendations and answers to the terminal, and a means for the terminal to display the recommendations and answers to the user. This enables appropriate answers to questions when a user is considering a menu and product recommendations based on individual preferences. It also reduces waiting times and provides multilingual services, which is expected to improve customer satisfaction and increase store sales.

[0294] A "smart device" is an electronic device that allows a user to input information and communicate with a server, and includes smartphones, tablets, and the like.

[0295] "Artificial intelligence means" refers to algorithms or models that analyze input data and generate appropriate answers or recommendations.

[0296] A "generative AI model" refers to a machine learning model that analyzes user input information and generates appropriate answers and recommendations.

[0297] "Measures to improve customer experience" refers to measures to reduce waiting times and enable customers to use services comfortably.

[0298] "Means for AI to explain questions when considering a menu" refers to means for AI to provide appropriate answers to questions that arise when a user is considering a menu.

[0299] The term "means for recommending products according to preferences" refers to a means for recommending appropriate products based on the user's preferences.

[0300] "Means for user input of information" refers to an interface through which a user inputs information in the form of text, voice, or the like.

[0301] "Means for transmitting input information to a server" refers to a communication means for transmitting information input by a user to a server.

[0302] "Means by which the server analyzes input information using a generative AI model" refers to means by which the server uses a generative AI model to analyze information received from a user.

[0303] "Means for generating recommendations and answers based on the analysis results" refers to means for generating recommendations and answers for users based on the analysis results of the generative AI model.

[0304] "Means for transmitting generated recommendations and answers to a terminal" refers to a communication means for transmitting recommendations and answers generated by a server to a user's terminal.

[0305] "Means for the terminal to display recommendations and answers to the user" refers to means for the user's terminal to display recommendations and answers received from the server to the user.

[0306] This invention is a system in which a user inputs information using a smart device, a server analyzes the information using a generative AI model, and generates and provides appropriate recommendations and answers to the user. Specific embodiments of this system are described below.

[0307] First, a user inputs information into the system using a smart device such as a smartphone or tablet. For example, the user inputs "I like spicy food." This information is input through the interface of the smart device.

[0308] Next, the smart device transmits the input information to the server, converting the input data into an appropriate format (e.g., JSON format) and transmitting it to the server via a network.

[0309] The server analyzes the received user input using a generative AI model (e.g., OpenAI's GPT-4). Specifically, it analyzes the information "I like spicy food" and searches a database of spicy food.

[0310] The server then recommends products that match the user's preferences based on the analysis results. For example, it generates a list of spicy dishes and presents it to the user. If the user asks, "What are the ingredients in this dish?", the server generates an answer to that question.

[0311] The generated recommendations and answers are sent from the server to the smart device, where the server converts the data into an appropriate format and sends it to the smart device over the network.

[0312] The smart device displays the recommendations and answers received from the server to the user. For example, it displays "Recommended spicy dishes are mapo tofu, kimchi jjigae, and Sichuan-style spicy hotpot."

[0313] As a concrete example, the following prompt sentence is shown.

[0314] "I like spicy food. Can you recommend any dishes?"

[0315] "What are the ingredients in this dish?"

[0316] In this way, a system is realized in which the server, smart device, and user work together to improve the customer experience. This system allows users to receive appropriate answers to questions when considering menu items and product recommendations tailored to their individual preferences. It also reduces waiting times and provides multilingual services, which is expected to improve customer satisfaction and increase store sales. The flow of the identification process in Example 3 will be described with reference to FIG. 15.

[0317] Step 1:

[0318] The user enters information using the terminal.

[0319] A user opens an application on a smartphone or tablet, enters "I like spicy food" in the text box, and presses the send button. The input data is in text format and is information that indicates the user's preferences.

[0320] Step 2:

[0321] The terminal transmits the input information to the server.

[0322] The terminal converts the user's input information, such as "I like spicy food," into JSON format and sends it to the server using the HTTPS protocol. The input data is the user's preference information in text format, and the output data is the request data in JSON format.

[0323] Step 3:

[0324] The server analyzes the input information using a generative AI model.

[0325] The server analyzes the received JSON data and inputs the information "I like spicy food" into a generative AI model. The generative AI model (e.g., GPT-4) searches a database of spicy food and generates a list of related dishes. The input data is the request data in JSON format, and the output data is a list of dishes as the analysis result.

[0326] Step 4:

[0327] The server generates recommendations and answers based on the analysis results.

[0328] The server generates a list of spicy dishes based on the analysis results obtained from the generative AI model. For example, it lists dish names such as "Mapo Tofu, Kimchi Jjigae, Sichuan-style Spicy Hot Pot." The input data is the list of dishes as a result of the analysis, and the output data is a list of recommendations to present to the user.

[0329] Step 5:

[0330] The server sends the generated recommendations and answers to the device.

[0331] The server converts the generated list of spicy dishes into JSON format and sends it to the terminal using the HTTPS protocol. The input data is the recommendation list, and the output data is the JSON format response data.

[0332] Step 6:

[0333] The device displays recommendations and answers to the user.

[0334] The terminal parses the received JSON data and displays to the user, "Recommended spicy dishes are mapo tofu, kimchi jjigae, and Sichuan-style spicy hotpot." The input data is response data in JSON format, and the output data is text information displayed to the user.

[0335] (Application example 3)

[0336] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0337] Conventional food delivery systems have the drawback of making it difficult for customers to obtain detailed information when selecting a menu item, and lacking recommendations tailored to individual preferences. Furthermore, there is no way to provide real-time information about ingredients and nutritional information for dishes, which reduces customer satisfaction.

[0338] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for generating translation work by artificial intelligence, a means for improving the customer experience by shortening waiting time, a means for explaining questions when considering a menu by artificial intelligence, a means for making recommendations based on preferences, a means for recommending dishes based on the customer's preferences, and a means for providing information on ingredients for dishes. This allows customers to obtain detailed menu information in real time and receive recommendations based on their individual preferences.

[0339] A "smart device" is a portable electronic device with internet connectivity, such as a smartphone or tablet.

[0340] The "menu creation means" is a function for generating a list of dishes and drinks that customers can choose from.

[0341] "Artificial intelligence means for generating product descriptions from images" is a function that analyzes images taken with a smart device and automatically generates product descriptions based on the images' contents.

[0342] "Means for artificial intelligence-generated translation work" refers to the ability to automatically translate text between different languages.

[0343] "Measures to improve customer experience by reducing waiting time" refers to a function that reduces the waiting time when customers use a service.

[0344] "A means for AI to explain questions when considering a menu" is a function in which AI provides appropriate answers to questions that arise when customers are choosing a menu.

[0345] "Means for recommendation based on preferences" is a function that recommends appropriate products and services based on the customer's preferences and past choices.

[0346] "Means for recommending dishes based on customer preferences" is a function that recommends appropriate dishes based on the preference information entered by the customer.

[0347] The "means for providing information on ingredients of a dish" is a function that provides information on ingredients and components contained in a specific dish.

[0348] A system for implementing this invention includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for generating translation work using artificial intelligence, a means for improving customer experience by reducing waiting time, a means for using artificial intelligence to explain questions when considering a menu, a means for making recommendations based on preferences, a means for recommending dishes based on the customer's preferences, and a means for providing information on ingredients for dishes.

[0349] System Program

[0350] The system is implemented using Python and the OpenAI API. The server receives customer input and uses a generative AI model to provide appropriate recommendations and information.

[0351] Processing Description

[0352] The server receives customer preferences and questions sent from the smart device and generates prompts based on them. The generated prompts are input into a generative AI model via the OpenAI API to generate appropriate answers and recommendations. The generated information is then sent back to the smart device and provided to the customer.

[0353] The hardware used is smart devices such as smartphones and tablets, and the software used is Python and the OpenAI API.

[0354] Specific examples

[0355] For example, if a customer enters "I like spicy food," the server generates the following prompt:

[0356] What dishes would you recommend to people who like spicy food?

[0357] When this prompt is input into the OpenAI API, the generative AI model will recommend spicy dishes such as "mapo tofu" and "kimchi hotpot."

[0358] Also, if a customer asks "What are the ingredients in mapo tofu?", the server generates the following prompt:

[0359] "What are the ingredients in mapo tofu?"

[0360] When this prompt is input into the OpenAI API, the generative AI model provides ingredient information such as "tofu, minced pork, green onion, garlic, ginger, chili bean paste, sweet bean paste, soy sauce, sake, sugar, and chicken stock."

[0361] This allows customers to get detailed menu information in real time and receive recommendations based on their individual preferences.

[0362] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0363] Step 1:

[0364] A user launches the application using a smart device and inputs their preferences and questions. The input information is sent to the server in text format. Examples of input include "I like spicy food" and "What are the ingredients in mapo tofu?"

[0365] Step 2:

[0366] The server analyzes the input information received from the user and generates an appropriate prompt. For example, for the input "I like spicy food," it generates the prompt "Please tell me what dishes you would recommend for people who like spicy food." This prompt is then input into the generative AI model.

[0367] Step 3:

[0368] The server sends the generated prompt to the OpenAI API and obtains appropriate answers or recommendations based on the generative AI model. For example, in response to the prompt, "What dishes would you recommend for people who like spicy food?", answers such as "Mapo tofu" and "Kimchi hotpot" are generated.

[0369] Step 4:

[0370] The server analyzes the answers and recommendations obtained from the generative AI model and converts them into a format for presentation to the user. For example, it formats answers such as "mapo tofu" and "kimchi hotpot" into a list format.

[0371] Step 5:

[0372] The server sends formatted answers and recommendations to the smart device, where the user can view the information on the screen of the smart device.

[0373] Step 6:

[0374] The user can make menu choices based on the information provided or ask more detailed questions, for example, re-entering the question "What are the ingredients in mapo tofu?"

[0375] Step 7:

[0376] The server receives input from the user again and repeats the same process: it generates a prompt again, inputs it into the generative AI model, obtains the answer, formats it, and provides it to the user.

[0377] This allows users to obtain detailed menu information in real time and receive recommendations tailored to their individual preferences.

[0378] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0379] "Example 1"

[0380] As one embodiment of the present invention, a system incorporating an emotion engine is provided. This system recognizes a user's emotions and recommends products based on those emotions. Specifically, when a user selects a product, the system recognizes the user's emotions from their facial expressions and tone of voice, and recommends products that match those emotions. For example, if the user shows a happy expression, the system recommends products that the user is likely to find joyful.

[0381] "Example 2"

[0382] The emotion engine also generates a product description based on the user's emotions. Specifically, it recognizes the user's emotions and generates a product description by selecting words that match those emotions. For example, if the user is feeling down, it generates a product description by selecting words that will cheer up the user. "Example 3"

[0383] Furthermore, the emotion engine recognizes the user's emotions and provides services according to those emotions. Specifically, it recognizes the user's emotions and provides services that match those emotions. For example, if the user shows an angry expression, it provides a service such as an apology to the user.

[0384] The processing flow of each embodiment will be described below.

[0385] "Example 1"

[0386] Step 1: As the user selects a product, the system captures the user's facial expressions and tone of voice.

[0387] Step 2: The emotion engine recognizes the user's emotion from the captured information.

[0388] Step 3: The system recommends products that match the recognized emotion.

[0389] "Example 2"

[0390] Step 1: As the user selects a product, the system captures the user's facial expressions and tone of voice.

[0391] Step 2: The emotion engine recognizes the user's emotion from the captured information.

[0392] Step 3: The system generates a product description by selecting words that match the recognized emotions.

[0393] "Example 3"

[0394] Step 1: As the user selects a product, the system captures the user's facial expressions and tone of voice.

[0395] Step 2: The emotion engine recognizes the user's emotion from the captured information.

[0396] Step 3: The system provides services that match the recognized emotions.

[0397] Example 1

[0398] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0399] Creating restaurant menus and improving customer experiences has traditionally required a lot of time and effort. In particular, generating product descriptions from product photos and making recommendations based on customer preferences consumes a large amount of human resources. Furthermore, multilingual service for foreign tourists is insufficient, and improving customer satisfaction is required. Furthermore, the inability to make product recommendations based on customer emotions can sometimes result in a decline in the quality of the customer experience. An efficient system to solve these issues is needed.

[0400] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0401] In this invention, the server includes a menu creation means using a smartphone, an artificial intelligence means for generating product descriptions from photos on the smartphone, a means for generating translation work using artificial intelligence, a means for improving the customer experience by eliminating the need to wait in line for waiters or cashiers, a means for providing explanations using artificial intelligence for questions when considering menu items, a means for making recommendations based on preferences, and a means for recognizing user emotions and recommending products based on those emotions. This makes it possible to create menus more efficiently, recommend products based on customer preferences, improve service for foreign tourists by supporting multiple languages, and recommend products based on customer emotions.

[0402] A "menu creation means using a smartphone" is a means for creating a restaurant menu using a smartphone's camera and applications.

[0403] "Artificial intelligence means for generating product descriptions from smartphone photos" refers to a means that uses artificial intelligence technology to analyze product photos taken with a smartphone, recognize their characteristics and features, and automatically generate product descriptions.

[0404] "Means for generating translation work using artificial intelligence" refers to a means for translating product descriptions and menu contents into multiple languages ​​using artificial intelligence technology.

[0405] "Measures to improve the customer experience by eliminating the need to wait in line for waiters or cashiers" refers to measures that allow customers to receive service smoothly without having to wait in line for waiters or cashiers.

[0406] "Means for AI to explain questions when considering a menu" refers to a means by which AI automatically provides answers to questions that arise when customers are considering a menu.

[0407] "Means for recommendation based on preferences" refers to means for recommending appropriate products based on a customer's past choices and preferences.

[0408] "Means for recognizing a user's emotions and recommending products based on those emotions" refers to a means for recognizing emotions by analyzing a user's facial expressions and tone of voice, and recommending products that match those emotions.

[0409] The present invention provides a system that allows restaurant operators to create menus using smartphones and improve customer experience. Specific embodiments of this system are described below.

[0410] First, a user takes a photo of a product using the smartphone camera. For example, if they want to take a photo of a new dessert, they open the smartphone's camera app, frame the dessert, and press the shutter button.

[0411] Next, the device uploads the captured photo to the server. Specifically, the smartphone application sends the photo data to the server via an HTTP request. At this time, the photo data is sent in the appropriate format (e.g., JPEG, PNG).

[0412] The server analyzes the received photo using image recognition software (e.g., Google Cloud Vision API). The server recognizes the product characteristics (e.g., chocolate cake, fruit parfait) from the photo. For example, the Google Cloud Vision API returns the label "chocolate cake."

[0413] The server then generates a product description based on the recognized characteristics, using a generative AI model (e.g., OpenAI GPT-3) to generate a product description such as "A rich chocolate cake with a moderate sweetness and a moist texture."

[0414] The generated product description is then displayed on the restaurant's website or application by the server. Specifically, the product description is sent to the front end in HTML or JSON format and displayed in the user interface.

[0415] Additionally, as users select products, the device uses a camera and microphone to capture their facial expressions and tone of voice. For example, a smartphone camera captures the user's smile, and a microphone records the user's tone of voice.

[0416] The server analyzes the captured data using an emotion engine (e.g., Microsoft® Azure® Emotion API) to recognize the user's emotions. Based on the recognized emotions, the server recommends products that suit the user. For example, if the user shows a happy expression, the server recommends a "fruit parfait."

[0417] As a concrete example, consider the case where a user takes a photo of a new dessert with their smartphone and uploads it to the system. The server analyzes the photo using the Google Cloud Vision API and recognizes its characteristics as "chocolate cake." The AI ​​then generates a product description: "A rich chocolate cake. It is not too sweet and has a moist texture."

[0418] If a user shows a happy expression while browsing the menu using their smartphone camera, the Microsoft Azure Emotion API will recognize that expression and the server will recommend the "fruit parfait" that the user is likely to enjoy.

[0419] Example prompt sentence:

[0420] "Upload a photo of a product taken with your smartphone. AI automatically recognizes the product's characteristics and generates a product description. It also uses the camera and microphone to recommend products based on your emotions."

[0421] In this way, a system is realized in which the server, terminals, and users work together to efficiently create restaurant menus and recommend products to customers.

[0422] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0423] Step 1:

[0424] The user takes a photo of the product.

[0425] A user uses the smartphone camera to take a photo of a product. For example, to take a photo of a new dessert, the user launches the smartphone's camera app, frames the dessert, and presses the shutter button. The input is the captured photo, and the output is image data stored in the smartphone.

[0426] Step 2:

[0427] The device uploads the photo to the server.

[0428] The device uploads the captured photo to the server. Specifically, the smartphone application sends the photo data to the server via an HTTP request. At this time, the photo data is sent in an appropriate format (e.g., JPEG, PNG). The input is the image data stored in the smartphone, and the output is the image data sent to the server.

[0429] Step 3:

[0430] The server analyzes the photo and recognizes the product's characteristics.

[0431] The server analyzes the received photo using image recognition software (e.g., image recognition API). The server recognizes the product characteristics (e.g., chocolate cake, fruit parfait) from the photo. For example, the image recognition API returns the label "chocolate cake." The input is the image data sent to the server, and the output is the recognized product characteristics information.

[0432] Step 4:

[0433] The server generates a product description.

[0434] The server generates a product description based on the recognized characteristics. Using a generative AI model (e.g., a generative AI model), it generates a product description such as "A rich chocolate cake with a moderate sweetness and a moist texture." The input is the product characteristic information, and the output is the generated product description.

[0435] Step 5:

[0436] The server displays the product description on the website or application.

[0437] The server displays the generated product description on the restaurant's website or application. Specifically, it sends the product description in HTML or JSON format to the front end and displays it in the user interface. The input is the generated product description, and the output is the product description displayed on the website or application.

[0438] Step 6:

[0439] When a user selects a product, the device captures the user's emotions.

[0440] When a user selects a product, the device uses a camera and microphone to capture the user's facial expressions and tone of voice. For example, a smartphone camera captures the user's smile, and the microphone records the user's tone of voice. The input is the user's facial expressions and tone of voice, and the output is the captured emotional data.

[0441] Step 7:

[0442] The server analyzes emotions and recommends appropriate products.

[0443] The server analyzes the captured data using an emotion engine (e.g., emotion analysis API) to recognize the user's emotions. The server then recommends products that suit the user based on the recognized emotions. For example, if the user shows a happy expression, the server recommends a "fruit parfait." The input is the captured emotion data, and the output is recommended product information.

[0444] (Application example 1)

[0445] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0446] Creating menus and improving customer experience in traditional restaurants has been a time-consuming and labor-intensive process. In particular, there are multiple challenges, such as creating menus, generating product descriptions, making recommendations based on customer preferences, and even multilingual support. Furthermore, it is difficult to provide services that respond to customer emotions, making it difficult to improve customer satisfaction.

[0447] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an AI means for generating product descriptions from photos on the smart device, an AI means for generating translation work, a means for improving the customer experience, an AI means for explaining questions when considering a menu, a means for making recommendations based on preferences, and a means for recognizing customer emotions and recommending products based on those emotions. This enables efficient menu creation, automatic generation of product descriptions, multilingual support, improved customer experience, and emotion-based recommendations.

[0448] "Smart devices" are portable electronic devices with advanced functions, such as smartphones and smart glasses.

[0449] "Menu creation means" refers to a function or method for creating a restaurant menu using a smart device.

[0450] "AI means for generating product descriptions" refers to artificial intelligence technology that automatically recognizes the characteristics and features of products from photos taken on a smart device and generates product descriptions based on that information.

[0451] "Means for AI to generate translation work" refers to a function that uses artificial intelligence to automatically perform translations that support multiple languages.

[0452] "Customer experience improvement measures" are methods and functions that improve the customer experience, such as eliminating the need to wait in line for waiters or cashiers.

[0453] "A means for AI to explain questions when considering a menu" is a function that automatically uses artificial intelligence to explain questions that arise when customers are considering a menu.

[0454] "Means for recommending based on preferences" is a function that recommends appropriate products based on the customer's preferences.

[0455] "Means of recognizing customer emotions and recommending products based on those emotions" refers to a function that recognizes emotions from the customer's facial expressions, tone of voice, etc., and recommends products that match those emotions.

[0456] A system for implementing this invention includes a menu creation means using a smart device, an AI means for generating product descriptions from photos on a smart device, an AI means for generating translation work, a means for improving the customer experience, a means for AI to explain questions that arise when considering a menu, a means for making recommendations based on preferences, and a means for recognizing customer emotions and recommending products based on those emotions.

[0457] Program processing explanation

[0458] Hardware

[0459] The server is equipped with a high-performance GPU, and the terminals are smart devices such as smartphones and smart glasses.

[0460] software

[0461] The server uses the following software:

[0462] OpenCV: Image processing library

[0463] Keras: A deep learning library

[0464] Transformers: Generative AI models (e.g., GPT-3)

[0465] Data processing and calculation

[0466] 1. Menu creation method:

[0467] Take a photo of the food using the device's camera and upload it to the server.

[0468] The server preprocesses the images using OpenCV and uses Keras to recognize the product's characteristics and features from the images.

[0469] Based on the recognition results, a product description is generated using Transformers.

[0470] 2. How AI generates translation work:

[0471] The server uses the generative AI model to translate the generated product description into multiple languages.

[0472] Multilingual support will be provided to enhance services, especially for foreign tourists.

[0473] 3. Measures to improve customer experience:

[0474] The terminal allows customers to complete orders and payments on their smart devices, eliminating the need to wait for waiters or in line at the cash register.

[0475] 4. AI-powered explanations for menu selection questions:

[0476] The device sends any questions that arise as customers consider the menu to the server, which then uses AI to automatically generate explanations.

[0477] 5. How to recommend based on your preferences:

[0478] The server recommends appropriate products based on the customer's past order history and preferences.

[0479] 6. A way to recognize customer emotions and recommend products based on those emotions:

[0480] The customer's facial expressions and tone of voice are analyzed through the device's camera, and the server uses Keras to recognize emotions.

[0481] Based on the recognized sentiment, Transformers are used to recommend appropriate products.

[0482] Examples and prompts

[0483] Examples:

[0484] When a customer wears the smart glasses and looks at the menu, the glasses analyze the customer's facial expressions and display a message saying, "You look happy, so I recommend dessert!"

[0485] Example prompt for a generative AI model:

[0486] "This dish is a delicious looking pasta dish. Please generate a detailed product description."

[0487] In this way, the mode for implementing the invention combines smart devices and AI technology to create restaurant menus and improve customer experience.

[0488] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0489] Step 1:

[0490] The user takes a photo of the food using a smart device (smartphone or smart glasses), and the photo is saved on the device.

[0491] Step 2:

[0492] The device uploads the captured photos to the server, which then uses OpenCV to preprocess the images, specifically resizing and removing noise.

[0493] Step 3:

[0494] The server analyzes the preprocessed image using Keras to recognize the product's characteristics and features. The input is the preprocessed image, and the output is data that indicates the product's characteristics and features.

[0495] Step 4:

[0496] The server generates a product description based on the recognized characteristics and features using Transformers. The input is data that describes the product's characteristics and features, and the output is the generated product description.

[0497] Step 5:

[0498] The server uses a generative AI model to translate the generated product description into multiple languages. The input is the generated product description, and the output is the translated product description into multiple languages.

[0499] Step 6:

[0500] The terminal allows customers to complete orders and payments on their smart devices, eliminating the need to wait in line for a waiter or cashier. The input is the customer's order information, and the output is a notification that the order has been completed.

[0501] Step 7:

[0502] The terminal sends questions that arise as customers consider the menu to the server, which then uses AI to automatically generate explanations. The input is the customer's question, and the output is the generated explanation.

[0503] Step 8:

[0504] The server recommends appropriate products based on the customer's past order history and preferences. The input is the customer's order history and preference data, and the output is the recommended products.

[0505] Step 9:

[0506] The server analyzes the customer's facial expression and tone of voice through the device's camera and uses Keras to recognize emotions. The input is the customer's facial expression and tone of voice data, and the output is the recognized emotion.

[0507] Step 10:

[0508] The server recommends appropriate products based on the recognized emotions using Transformers. The input is the recognized emotion and the output is the recommended product.

[0509] Example 2

[0510] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0511] Today's consumers expect to be able to quickly and accurately understand product descriptions, and they also want information to be provided in multiple languages. They also want personalized product descriptions that reflect the user's emotions. However, conventional systems have difficulty meeting these demands, and improving service for foreign tourists in particular is a challenge.

[0512] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0513] In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for generating translations using artificial intelligence, a means for improving the customer experience, a means for providing explanations using artificial intelligence for questions raised when considering a menu, a means for making recommendations based on preferences, a means for recognizing a user's emotions and generating product descriptions based on those emotions, and a means for translating the generated product descriptions into multiple languages. This allows users to quickly and accurately understand product descriptions, and information can be provided in multiple languages. Furthermore, providing personalized product descriptions based on the user's emotions can improve customer satisfaction.

[0514] A "smart device" is a portable electronic device that has Internet connectivity and can run applications.

[0515] The "menu creation means" is a function for creating menus for restaurants and service businesses using a smart device.

[0516] "Artificial intelligence means for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with a smart device and automatically generates product descriptions based on those images.

[0517] "Means for generating translation work using artificial intelligence" refers to artificial intelligence technology for translating generated product descriptions into multiple languages.

[0518] "Means to improve the customer experience" are functions that improve the convenience and satisfaction of customers when using a service.

[0519] "Means for AI to explain questions when considering a menu" is a function that enables AI to provide appropriate answers to questions that arise when customers are considering a menu.

[0520] "Preference-based recommendation means" is a function for recommending appropriate products and services based on a customer's past choices and preferences.

[0521] "Means for recognizing a user's emotions and generating a product description that corresponds to those emotions" refers to artificial intelligence technology that analyzes emotions from a user's input and behavior and generates a product description that is appropriate for those emotions.

[0522] The "means for translating the generated product description into multiple languages" is a function for translating the generated product description into multiple languages.

[0523] MODE FOR CARRYING OUT THE INVENTION

[0524] This invention is a system that includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images, a means for generating translation work using artificial intelligence, a means for improving the customer experience, a means for explaining questions that arise when considering a menu using artificial intelligence, a means for making recommendations based on preferences, a means for recognizing a user's emotions and generating product descriptions based on those emotions, and a means for translating the generated product descriptions into multiple languages.

[0525] Hardware and software used

[0526] The server generates a product description using a generative AI model (e.g., OpenAI's GPT-4). The generated product description is translated into multiple languages ​​using a translation engine (e.g., Google Translate API). To recognize the user's emotions, an emotion engine (e.g., IBM Watson (registered trademark) emotion analysis API) is used. The smart device is a portable electronic device that has Internet connectivity and can run applications.

[0527] Explanation of program processing

[0528] The server receives text entered by the user through the device and generates a product description using a generative AI model. For example, if the user enters "This product is of high quality and will last a long time," the server generates a product description based on this text.

[0529] The generated product description is then translated into multiple languages ​​using a translation engine, for example, into major tourist languages ​​such as English, Chinese, and Korean. Specifically, the description "This product is of high quality and will last a long time" is translated.

[0530] When a user inputs text or voice data related to their emotions through the device, the server uses an emotion engine to recognize the user's emotions. For example, if a user inputs "I'm feeling depressed today," the server analyzes this text and recognizes the user's emotion as "depressed."

[0531] The server uses a generative AI model to generate a product description based on the user's emotions. For example, if the user is feeling down, the server generates a product description using encouraging words.

[0532] Original description: "This item is high quality and will last a long time."

[0533] Emotional description: "This product is high quality and will brighten your day. It's long-lasting so you can use it forever!"

[0534] Examples of concrete examples and prompts

[0535] As a concrete example, consider the following scenario.

[0536] Scenario 1: Multilingual Translation

[0537] If a user types "This product is of high quality and will last a long time" in Japanese, the server will translate this product description into English, Chinese, and Korean.

[0538] Scenario 2: Emotional product descriptions

[0539] If a user types, "I'm feeling down today," the server uses an emotion engine to recognize the user's emotions and generate an uplifting product description.

[0540] Original description: "This item is high quality and will last a long time."

[0541] Emotional description: "This product is high quality and will brighten your day. It's long-lasting so you can use it forever!"

[0542] Prompt Sentence Examples

[0543] An example of a prompt for a generative AI model is as follows:

[0544] "Please translate this product description in Japanese into English, Chinese, and Korean: 'This product is of high quality and will last a long time.'"

[0545] "Generate a product description that will cheer up users when they're feeling down. The original description is 'This product is high quality and will last a long time.'"

[0546] The above is an embodiment of the present invention.

[0547] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0548] Step 1:

[0549] The user uses the terminal to input text that will be the basis for the product description. For example, the user might input "This product is of high quality and will last a long time." The input text is sent to the server.

[0550] Step 2:

[0551] The server uses a generative AI model to generate a product description based on the text entered by the user. Specifically, the server sends the following prompt to the generative AI model: "Please generate a product description based on the user's input: 'This product is high quality and will last a long time.'" The generative AI model generates a product description based on this prompt and returns it to the server. The output is the generated product description.

[0552] Step 3:

[0553] The server sends the generated product description to the translation engine for translation into multiple languages. Specifically, the server sends the following prompt to the translation engine: "Please translate this product description into English, Chinese, and Korean: 'This product is of high quality and will last a long time.'" The translation engine translates based on this prompt and returns it to the server. The output is a product description translated into multiple languages.

[0554] Step 4:

[0555] The user inputs text or voice data related to their emotions through the terminal. For example, the user inputs "I'm feeling depressed today." The input text or voice data is sent to the server.

[0556] Step 5:

[0557] The server uses the emotion engine to recognize the user's emotion. Specifically, the server sends the following prompt to the emotion engine: "Please analyze the user's input, 'I'm feeling depressed today,' and recognize the emotion." The emotion engine analyzes the emotion based on this prompt and returns it to the server. The output is the recognized emotion.

[0558] Step 6:

[0559] The server uses a generative AI model to generate an emotion-based product description based on the recognized user emotion. Specifically, the server sends the following prompt to the generative AI model: "Generate an uplifting product description for the user when they are feeling down. The original description is 'This product is of high quality and will last a long time.'" The generative AI model generates an emotion-based product description based on this prompt and returns it to the server. The output is an emotion-based product description.

[0560] Step 7:

[0561] The server sends a product description corresponding to the generated emotion to the user's device. The user can check the uplifting product description through the device. For example, if the user inputs "I'm feeling down today," the server will provide the user with a product description such as "This product is high quality and will brighten your days. It lasts a long time, so you can use it for a long time!"

[0562] The above is the specific flow of the program processing of this system.

[0563] (Application example 2)

[0564] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0565] On conventional online shopping sites, product descriptions are often provided in a single language, making them difficult for foreign tourists and multilingual users to understand. Furthermore, product descriptions are not tailored to the user's emotions, making it difficult to stimulate their purchasing motivation. Furthermore, users often spend a lot of time trying to understand the product descriptions, resulting in a poor purchasing experience.

[0566] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0567] In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, an artificial intelligence means for generating translations, a means for improving the customer experience, an artificial intelligence means for explaining questions when considering menus, a means for making recommendations based on preferences, a means for recognizing a user's emotions and generating product descriptions based on those emotions, and a means for translating the generated product descriptions into multiple languages. This makes it possible to provide product descriptions in multiple languages ​​that are based on the user's emotions and are easy to understand for foreign tourists and users who speak multiple languages. It also increases the user's purchasing motivation and improves the purchasing experience.

[0568] A "smart device" is a portable electronic device with internet connectivity, such as a smartphone, tablet, or smart glasses.

[0569] "Menu creation means" refers to functions or software that allow a user to create a menu of products or services using a smart device.

[0570] "Artificial intelligence means for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with a smart device and automatically generates product descriptions based on those images.

[0571] "Means for generating translation work using artificial intelligence" refers to artificial intelligence technology for translating generated product descriptions into multiple languages.

[0572] "Customer experience improvement measures" are functions and software that improve the experience customers have when using products and services.

[0573] "Means for AI to explain questions when considering a menu" is a function in which AI provides appropriate explanations for questions that arise when a user is considering a menu.

[0574] "Means for recommending based on preferences" is a function that recommends appropriate products and services based on the user's preferences and past behavior.

[0575] "Means for recognizing a user's emotions and generating a product description that corresponds to those emotions" refers to artificial intelligence technology that analyzes a user's emotions and generates a product description that is appropriate for those emotions.

[0576] The "means for translating the generated product description into multiple languages" refers to a function or software for translating the generated product description into multiple languages.

[0577] The following describes an embodiment of the present invention. This system includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images, a means for generating translations using artificial intelligence, a means for improving the customer experience, a means for providing explanations using artificial intelligence for questions raised when considering a menu, a means for making recommendations based on preferences, a means for recognizing a user's emotions and generating product descriptions based on those emotions, and a means for translating the generated product descriptions into multiple languages.

[0578] System configuration

[0579] Hardware

[0580] Smart Device: A portable electronic device with internet connectivity, such as a smartphone, tablet, or smart glasses.

[0581] Server: A high-performance computer responsible for data processing and storage.

[0582] software

[0583] Artificial intelligence model: Generates product descriptions using OpenAI's API.

[0584] Translation software: Multilingual translation using Google Translate API.

[0585] Emotion Recognition Software: Recognize user emotions using EmotionRecognizer.

[0586] Data processing and calculation

[0587] 1. Image analysis: A product image taken with a smart device is sent to the server, which uses image analysis algorithms to extract the product's characteristics and features and generate a product description.

[0588] 2. Emotion Recognition: Using the camera and microphone of the smart device to analyze the user's facial expressions and tone of voice, the EmotionRecognizer software recognizes the user's emotions and sends the results to the server.

[0589] 3. Product description generation: The server generates prompt sentences according to the user's emotions and generates product descriptions appropriate to the emotions using OpenAI's API.

[0590] 4. Multilingual Translation: The generated product description is translated into multiple languages ​​using Google Translate API. The translation results are available in major languages ​​such as English, Chinese, and Korean.

[0591] Specific examples

[0592] For example, if a user is feeling depressed, the smart device camera captures the user's facial expression and the EmotionRecognizer recognizes it as "depressed." The server generates the following prompt sentence:

[0593] "Generate a product description that cheers up users when they're feeling down: This product is made of high-quality materials and will last a long time."

[0594] This prompt is sent to OpenAI's API to generate an emotionally appropriate product description, which looks like this:

[0595] "This product was created to brighten your life a little. It's made from high-quality materials and is built to last."

[0596] Next, this product description is translated into multiple languages ​​using the Google Translate API.

[0597] In this way, it becomes possible to provide product descriptions in multiple languages ​​that reflect the user's emotions.

[0598] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0599] Step 1:

[0600] The user takes a photo of the product using a smart device.

[0601] Input: Product image

[0602] Output: Captured image data

[0603] How it works: A user uses a smart device such as a smartphone or tablet to take a picture of a product they are considering purchasing. This image data is then stored on the smart device.

[0604] Step 2:

[0605] The terminal transmits the captured image to the server.

[0606] Input: Captured image data

[0607] Output: Image data sent to the server

[0608] Specific operation: The smart device sends the captured image data to a server via the Internet, using protocols such as HTTP.

[0609] Step 3:

[0610] The server performs image analysis and extracts the product's characteristics and features.

[0611] Input: Submitted image data

[0612] Output: Extracted product characteristics and features

[0613] How it works: The server uses image analysis algorithms to automatically extract product characteristics and features from image data, such as product color, shape, and brand logo.

[0614] Step 4:

[0615] The server generates a product description based on the extracted properties and characteristics.

[0616] Input: Extracted product characteristics and features

[0617] Output: Generated product description

[0618] What it does: The server uses a generative AI model to generate a product description based on the extracted properties and characteristics, such as "This product is made from high-quality materials and is built to last."

[0619] Step 5:

[0620] The device recognizes the user's emotions.

[0621] Input: User facial expressions and tone of voice

[0622] Output: Perceived user emotion

[0623] How it works: Using the smart device's camera and microphone, the EmotionRecognizer software captures the user's facial expressions and tone of voice, and then recognizes their emotions, such as "depressed" or "happy."

[0624] Step 6:

[0625] The server generates a prompt sentence according to the user's emotion.

[0626] Input: Recognized user sentiment, generated product description

[0627] Output: Emotion-based prompt

[0628] Specific operation: Based on the recognized user emotion, the server generates a prompt sentence to send to the generative AI model. For example, a prompt sentence such as "Please generate a product description that will cheer up the user when they are feeling down: This product is made of high-quality materials and will last a long time."

[0629] Step 7:

[0630] The server uses a generative AI model to generate emotionally appropriate product descriptions.

[0631] Input: Emotion-specific prompt

[0632] Output: Emotionally relevant product description

[0633] What it does: The server uses a generative AI model (e.g., OpenAI API) to generate an emotionally appropriate product description based on the prompt. For example, it might generate a description like, "This product was made to make your life a little brighter. It's made from high-quality materials and is built to last."

[0634] Step 8:

[0635] The server translates the generated product description into multiple languages.

[0636] Input: Generated product description

[0637] Output: Product description translated into multiple languages

[0638] What it does: The server uses the Google Translate API to translate the generated product description into multiple languages, such as English, Chinese, and Korean. For example, the translation results might be something like, "This product is made to brighten your life a little. It is made of high-quality materials and lasts long."

[0639] Step 9:

[0640] The server transmits product descriptions translated into multiple languages ​​to the terminal.

[0641] Input: Product description translated into multiple languages

[0642] Output: Translation results sent to the device

[0643] Specific operation: The server sends the product description translated into multiple languages ​​to the smart device via the Internet, using protocols such as HTTP.

[0644] Step 10:

[0645] The device displays product descriptions translated into multiple languages ​​to the user.

[0646] Input: Translation result sent to the device

[0647] Output: Multilingual product description displayed to the user

[0648] Specific operation: The smart device receives the translated product description and displays it to the user, allowing the user to understand the product description in multiple languages ​​according to their emotions.

[0649] Example 3

[0650] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0651] Conventional customer experience improvement systems are inefficient because they often require manual tasks such as creating menus, explaining products, and translating them. Furthermore, they lack the ability to provide answers to questions customers have when considering menus, or to recommend products based on their preferences, which can lead to a decline in customer satisfaction. Furthermore, they lack the ability to recognize customer emotions and provide services in response, which hinders the quality of the customer experience.

[0652] The identification process by the identification processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, an artificial intelligence means for generating translation work, a customer experience improvement means for reducing waiting time, an artificial intelligence means for explaining questions when considering a menu, a means for recommending products according to preferences, and a means for recognizing user emotions and providing services according to those emotions. This makes it possible to quickly answer questions when customers are considering a menu and recommend products according to preferences, and further to significantly improve customer satisfaction by providing services according to customer emotions.

[0653] A "smart device" is a portable electronic device with internet connectivity, such as a smartphone or tablet.

[0654] The "menu creation means" is a function for creating menus for restaurants and service businesses using a smart device.

[0655] "Artificial intelligence means for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with a smart device and automatically generates product descriptions based on those images.

[0656] "Means for generating translation work using artificial intelligence" refers to a function that uses artificial intelligence to automatically translate text or speech.

[0657] "Customer experience improvement measures to reduce waiting time" are functions that shorten waiting times when customers use services and provide a smoother experience.

[0658] "A means for AI to explain questions when considering a menu" is a function in which AI provides appropriate answers to questions that arise when customers are considering a menu.

[0659] "Means for recommending products based on preferences" is a function that uses artificial intelligence to recommend appropriate products based on a customer's preferences and past choices.

[0660] "Means for recognizing the user's emotions and providing services according to those emotions" refers to a function that recognizes the user's emotions from their facial expressions and voice and provides appropriate services according to those emotions.

[0661] This invention is a system for improving customer experience, and includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images, a means for generating translation work using artificial intelligence, a customer experience improvement means for reducing waiting time, a means for explaining questions when considering a menu using artificial intelligence, a means for recommending products according to preferences, and a means for recognizing a user's emotions and providing services according to those emotions.

[0662] Hardware and software used

[0663] Hardware

[0664] Smart Device: A portable electronic device with internet connectivity, such as a smartphone or tablet.

[0665] Server: A high-performance server (e.g., a server equipped with an NVIDIA GPU).

[0666] software

[0667] Generative AI models: Using GPT-4 as an example.

[0668] Emotion Recognition Engine: Using Affectiva SDK as an example.

[0669] Program processing explanation

[0670] The server receives the information sent from the smart device and analyzes it using the generative AI model. For example, if a user inputs "I like spicy food," the generative AI model generates a list of spicy dishes and the server sends that list to the smart device. The smart device then displays the received list to the user.

[0671] When a user asks, "What are the ingredients in this dish?", the server uses the generative AI model to generate ingredient information and sends it to the smart device, which then displays the received ingredient information to the user.

[0672] Furthermore, the smart device uses its built-in camera and microphone to capture the user's facial expressions and voice, and uses an emotion recognition engine to recognize the user's emotions. For example, if the user shows anger, the smart device sends that information to the server. The server then uses a generative AI model to generate an apology and sends it to the smart device. The smart device then displays the received apology to the user.

[0673] Examples of concrete examples and prompts

[0674] Example 1: User enters "I like spicy food"

[0675] A user opens an app on their smartphone and types in "I like spicy food."

[0676] The server generates a list of spicy dishes using a generative AI model.

[0677] The server sends the generated list to the smart device.

[0678] A smart device displays a list of spicy dishes to the user.

[0679] Example prompt: "List dishes you would recommend for a customer who likes spicy food."

[0680] Example 2: A user asks, "What are the ingredients in this dish?"

[0681] A user opens the app on their tablet and asks, "What are the ingredients in this dish?"

[0682] The server generates ingredient information using the generative AI model.

[0683] The ingredient information generated by the server is sent to the smart device.

[0684] The smart device displays ingredient information to the user.

[0685] Example prompt: "What are the ingredients in this dish?"

[0686] Example 3: User shows angry facial expression

[0687] The user shows an angry expression while using the smartphone.

[0688] The smart device uses a camera to capture the user's facial expressions and analyzes them using an emotion recognition engine.

[0689] The smart device transmits the anger emotion information it recognizes to a server.

[0690] The server generates an apology using a generative AI model.

[0691] The server generates an apology and sends it to the smart device.

[0692] The smart device displays an apology to the user.

[0693] Example prompt: "Generate an apology for when the user is angry."

[0694] In this way, the server, smart device, and user work together to operate the system, improving the customer experience. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[0695] Step 1:

[0696] The user inputs information through a device. Using a device such as a smartphone or tablet, the user inputs questions and preferences into the system. For example, the user might input "I like spicy food." The input data is saved in text format on the device.

[0697] Step 2:

[0698] The terminal sends the input information to the server. The terminal sends the information entered by the user to the server via the Internet. The input data is sent to the server in text format.

[0699] Step 3:

[0700] The server analyzes the input information using a generative AI model. The server passes the received input information to a generative AI model (e.g., GPT-4) for analysis. The generative AI model generates appropriate answers or recommendations based on the input information. For example, for the information "I like spicy food," it generates a list of spicy dishes. The input data is in text format, and the output data is in list format.

[0701] Step 4:

[0702] The server generates appropriate answers and recommendations based on the analysis results. The server generates answers and recommendations for the user based on the analysis results obtained from the generative AI model. For example, it generates a list of spicy dishes. The output data is in list format.

[0703] Step 5:

[0704] The server sends the generated answers and recommendations to the terminal. The server then sends the generated answers and recommendations to the terminal via the Internet. The output data is sent to the terminal in list format.

[0705] Step 6:

[0706] The terminal displays answers and recommendations to the user. The terminal displays answers and recommendations received from the server to the user. The user can check the displayed information. The output data is displayed in list format.

[0707] Step 7:

[0708] The device recognizes the user's emotions using an emotion engine. The device uses the built-in camera and microphone to capture the user's facial expressions and voice, and then uses an emotion engine (e.g., Affectiva SDK) to recognize the user's emotions. The input data is in the form of images and audio, and the output data is in the form of emotion information.

[0709] Step 8:

[0710] The terminal transmits the recognized emotion information to the server. The terminal transmits the recognized emotion information to the server via the Internet. The input data is in emotion information format, and the output data is in emotion information format.

[0711] Step 9:

[0712] The server generates an appropriate service based on the emotional information. The server uses a generative AI model to generate an appropriate service based on the received emotional information. For example, if the user shows an angry expression, it generates an apology. The input data is in the form of emotional information, and the output data is in text format.

[0713] Step 10:

[0714] The server transmits the generated service to the terminal. The server transmits the generated service to the terminal via the Internet. The output data is transmitted to the terminal in text format.

[0715] Step 11:

[0716] The terminal provides the service to the user. The terminal provides the service received from the server to the user. For example, it displays an apology. The output data is displayed in text format.

[0717] (Application example 3)

[0718] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0719] Conventional food delivery services have faced challenges, such as users having to spend a lot of time selecting menu items and difficulty in providing services that respond to users' preferences and emotions. Furthermore, multilingual support for foreign tourists was insufficient, leaving a need for an improved user experience. There is a need to solve these challenges and provide a more comfortable and personalized service.

[0720] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0721] In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, an artificial intelligence means for generating translation tasks, a customer experience improvement means for reducing waiting time, an artificial intelligence means for explaining questions when considering menu items, a means for recommending products based on preferences, and an emotion recognition means for recognizing user emotions and providing services based on those emotions. This allows users to efficiently select menus and receive personalized services based on their individual preferences and emotions. Multilingual support also enhances services for foreign tourists.

[0722] A "smart device" is a portable electronic device with advanced computing capabilities, such as a smartphone or tablet.

[0723] A "menu generator" is a method or device for generating a list of products or services from which a user can choose.

[0724] "Artificial intelligence means for generating product descriptions from images" refers to an artificial intelligence system that uses image recognition technology to automatically analyze the characteristics and features of a product and generates a product description based on that analysis.

[0725] "Artificial intelligence-generated means of translation work" refers to an artificial intelligence system that automatically translates text or speech between different languages.

[0726] "Customer experience improvement means" refers to a method or device for improving the convenience and satisfaction of users when using a service.

[0727] "A means for AI to explain questions when considering a menu" is a system in which AI provides appropriate answers to questions that arise when a user is choosing a menu.

[0728] A "means for recommending products based on preferences" is a system that recommends appropriate products and services based on the user's past choices and input information.

[0729] "Emotion recognition means" is a system that analyzes the user's emotions from facial expressions, tone of voice, etc., and responds accordingly.

[0730] A system for implementing this invention includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images, an artificial intelligence means for generating translation work, a customer experience improvement means for reducing waiting time, an artificial intelligence means for explaining questions when considering a menu, a means for recommending products according to preferences, and an emotion recognition means for recognizing a user's emotions and providing services according to those emotions.

[0731] System Program

[0732] The program of this system performs the following processing.

[0733] Hardware and Software

[0734] Hardware:

[0735] Smart devices (smartphones, tablets, etc.)

[0736] Camera (to capture the user's facial expressions)

[0737] Computer (processing device for executing programs)

[0738] software:

[0739] OpenCV (image processing library)

[0740] Keras (deep learning library)

[0741] Transformers (a library of generative AI models)

[0742] Data processing and calculation

[0743] 1. Menu creation method:

[0744] The server generates a list of products and services that the user can select using the smart device.

[0745] 2. Artificial intelligence means for generating product descriptions from images:

[0746] The server analyzes images taken by the smart device's camera and automatically recognizes the product's characteristics and features.

[0747] A product description is generated based on the recognized information.

[0748] 3. AI-generated translation tasks:

[0749] The server automatically translates text and audio between different languages.

[0750] The service will be multilingual, particularly to enhance services for foreign tourists.

[0751] 4. Customer experience improvement measures:

[0752] The server reduces the waiting time when the user uses the service, improving convenience.

[0753] 5. How AI can explain questions you might have when considering a menu:

[0754] The server uses generative AI models to provide appropriate answers to questions that arise when users choose from the menu.

[0755] 6. How to recommend products based on your preferences:

[0756] The server recommends appropriate products and services based on the user's past choices and input information.

[0757] 7. Emotion recognition means:

[0758] The server analyzes the user's facial expressions and tone of voice captured by the camera to recognize their emotions.

[0759] Respond appropriately depending on the emotion recognized.

[0760] Specific examples

[0761] When a user inputs "I like spicy food," the server uses a generative AI model to recommend spicy dishes.

[0762] When a user asks, "What are the ingredients in this dish?", the server uses a generative AI model to provide ingredient information.

[0763] If the user shows an angry expression, the server uses emotion recognition means to apologize, saying, "Sorry. Is there a problem?"

[0764] Prompt Sentence Examples

[0765] "I like spicy food. What would you recommend?"

[0766] "What are the ingredients in this dish?"

[0767] "How do you respond when a user displays anger?"

[0768] In this way, users can efficiently select menus and receive personalized services that reflect their individual preferences and feelings. Multilingual support also enhances services for foreign tourists.

[0769] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0770] Step 1:

[0771] The user opens a menu using a smart device.

[0772] Input: A request to display a menu via user operation.

[0773] Data processing: The server retrieves menu information from the database and sends it to the smart device.

[0774] Output: A menu will be displayed on the smart device.

[0775] Step 2:

[0776] The user takes a photo of the food using the camera on their smart device.

[0777] Input: A user-taken image of a dish.

[0778] Data processing: The server receives the images and analyzes them using an image recognition algorithm (OpenCV).

[0779] Output: As a result of the analysis, the characteristics and features of the food are extracted.

[0780] Step 3:

[0781] The server generates a product description from the image.

[0782] Input: Data on food characteristics and properties.

[0783] Data processing: The server uses a generative AI model to generate a product description based on the product's characteristics and features.

[0784] Output: Product description text is generated and sent to the smart device.

[0785] Step 4:

[0786] Users input questions when reviewing the menu.

[0787] Input: The question typed by the user (e.g., "What are the ingredients in this dish?").

[0788] Data processing: The server generates answers to questions using a generative AI model.

[0789] Output: Answer text is generated and displayed on the smart device.

[0790] Step 5:

[0791] The server recommends products based on the user's preferences.

[0792] Input: Previous choices or inputs made by the user (e.g., "I like spicy food").

[0793] Data processing: The server uses a generative AI model to recommend products based on the user's preferences.

[0794] Output: A list of recommended products will be displayed on the smart device.

[0795] Step 6:

[0796] The user captures their facial expression using the camera on their smart device.

[0797] Input: An image of the user's facial expression.

[0798] Data processing: The server analyzes emotions using a facial expression recognition algorithm (Keras).

[0799] Output: The analysis results in the generation of user emotion data.

[0800] Step 7:

[0801] The server provides a service according to the user's emotions.

[0802] Input: User emotion data (e.g., anger emotion).

[0803] Data processing: The server generates an appropriate response based on the emotion data (e.g., an apology message).

[0804] Output: The corresponding message will be displayed on the smart device.

[0805] Step 8:

[0806] The server performs the translation work.

[0807] Input: Text or voice data entered by the user.

[0808] Data processing: The server uses a translation algorithm to translate the input data into multiple languages.

[0809] Output: The translated text and audio data will be displayed on your smart device.

[0810] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0811] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0812] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0813] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0814] [Second embodiment]

[0815] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0816] As shown in FIG. 3, the data processing system 210 includes the data processing device 12 and the smart eye.

[0817] The data processing device 12 includes a mirror 214. An example of the data processing device 12 is a server.

[0818] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0819] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0820] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0821] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0822] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0823] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0824] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0825] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0826] In the smart glasses 214, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is performed by the processor 46 executing the reception output program 60 on the RAM 48.

[0827] This is realized by operating as the control unit 46A in accordance with the force program 60.

[0828] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0829] "Example 1"

[0830] In one embodiment of the present invention, restaurant operators create menus using smartphones. Specifically, they upload photos of products taken with the smartphone camera, and AI automatically recognizes the product's characteristics and features from the photos. Then, it generates product descriptions based on the recognition results. These product descriptions are displayed to customers on the restaurant's website or application.

[0831] "Example 2"

[0832] Furthermore, in this embodiment of the present invention, AI generates translation tasks. Specifically, it translates the generated product descriptions into multiple languages. This makes the product descriptions easier to understand for foreign tourists. For example, it is possible to translate into major tourist languages ​​such as English, Chinese, and Korean.

[0833] "Example 3"

[0834] This embodiment of the present invention also improves the customer experience. Specifically, when a customer is considering a menu, the AI ​​will explain their question and recommend products based on their preferences. For example, if a customer enters information such as "I like spicy food," the AI ​​will recommend spicy dishes. Also, if a customer asks a question such as "What are the ingredients in this dish?", the AI ​​will provide an answer to that question.

[0835] The processing flow of each embodiment will be described below.

[0836] "Example 1"

[0837] Step 1: The restaurant manager takes a photo of the product using a smartphone camera.

[0838] Step 2: Upload the photos you have taken to the system.

[0839] Step 3: The AI ​​in the system automatically recognizes the product's characteristics and features from the photo.

[0840] Step 4: The AI ​​generates a product description based on the recognition results.

[0841] Step 5: The generated product description is displayed to customers on the restaurant's website or application.

[0842] "Example 2"

[0843] Step 1: Get the AI-generated product description.

[0844] Step 2: AI translates the product description into multiple languages.

[0845] Step 3: The translated product description will be displayed to foreign tourists on the restaurant's website or app.

[0846] "Example 3"

[0847] Step 1: As customers consider the menu, they input their questions into the AI.

[0848] Step 2: The AI ​​generates an answer to that question.

[0849] Step 3: AI recommends products based on customer preferences.

[0850] Step 4: The AI-generated answers and recommendations are displayed to the customer.

[0851] Example 1

[0852] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0853] When restaurant operators create menus, it is time-consuming to take photos of the products, recognize the characteristics and features of the products from the photos, and generate product descriptions. Furthermore, there is a lack of multilingual support for foreign tourists and ways to improve the customer experience. This has led to a demand for more efficient menu creation and improved customer satisfaction.

[0854] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0855] In this invention, the server includes a menu creation means using a smartphone, an artificial intelligence means for generating product descriptions from smartphone photos, a means for generating translations by artificial intelligence, a means for improving the customer experience by eliminating the need to wait in line for waiters or cashiers, a means for providing artificial intelligence to answer questions when considering menu items, a means for making recommendations based on preferences, a means for uploading product photos taken with a smartphone camera and having the artificial intelligence automatically recognize the characteristics and features of the products from the photos, and a means for generating product descriptions based on the recognition results, thereby making it possible to improve the efficiency of menu creation and customer satisfaction.

[0856] A "smartphone" is a multi-functional mobile device that can connect to the Internet and use applications in addition to the functions of a mobile phone.

[0857] "Menu creation means" refers to the methods and tools used by restaurant operators to create menus, particularly those using smartphones.

[0858] "Artificial intelligence means" refers to technologies that use machine learning and data analysis to automatically perform specific tasks.

[0859] "Means for generating translation work using artificial intelligence" refers to methods and technologies for translating text into multiple languages ​​using artificial intelligence.

[0860] "Measures to improve customer experience" refers to methods and tools for improving the convenience and satisfaction of customers when using services.

[0861] "Means for AI to explain questions when considering a menu" refers to methods and technologies in which AI automatically provides answers to questions that arise when customers are considering a menu.

[0862] "Preference-based recommendation methods" refers to methods and technologies that recommend appropriate products and services based on a customer's preferences and past choices.

[0863] "Means for automatically recognizing product characteristics and features" refers to methods and technologies that use artificial intelligence to automatically extract product characteristics and features from photographs and data.

[0864] "Means for generating product descriptions" refers to methods or techniques for creating product descriptions in natural language based on recognized product characteristics and features.

[0865] This invention is a system that allows restaurant operators to create menus using smartphones. Specifically, a photo of a product taken with the smartphone camera is uploaded, and artificial intelligence (AI) automatically recognizes the product's characteristics and features from the photo and generates a product description based on the recognition results. This product description is displayed to customers on the restaurant's website or application.

[0866] Hardware and software used

[0867] Smartphone: A device for taking and uploading photos.

[0868] Server: Processes data and runs AI models.

[0869] Artificial intelligence models: Image recognition and natural language generation models based on TensorFlow and PyTorch (e.g., GPT-3, BERT).

[0870] Data processing and calculation

[0871] 1. The user takes a photo of the product with their smartphone

[0872] The user opens the camera app on their smartphone and takes a photo of a product, for example, a new dessert called "chocolate cake."

[0873] 2. The user uploads a photo to the system

[0874] The user opens the dedicated application, selects the photos they have taken, and presses the upload button, which sends the photos to the server via the Internet.

[0875] 3. The server receives the photo and inputs it into the AI ​​model

[0876] The server receives the uploaded photos and feeds them into an AI model for image processing, which is based on TensorFlow and PyTorch.

[0877] 4. The server uses an AI model to recognize the product's characteristics and features.

[0878] The server uses AI models to recognize product characteristics and features from the photo, such as the type of dessert, key ingredients, and visual features.

[0879] 5. The server generates a product description based on the recognition results.

[0880] The server generates a product description based on the AI ​​recognition results. This product description is created using natural language generation technology, using software such as GPT-3 and BERT.

[0881] 6. The server displays the product description on the website or application.

[0882] The server then displays the generated product description on the restaurant's website or application, where customers can view it.

[0883] Specific examples

[0884] A user takes a photo of a new dessert, "Chocolate Cake," and uploads it to the system. The server receives the photo and uses an AI model to recognize features such as "Chocolate Cake," "Cream Topping," and "Berry Decoration." The server then uses GPT-3 to generate a product description: "This chocolate cake is characterized by its rich chocolate and creamy topping. The berry decoration makes it a visually stunning dish." This is then displayed on the website.

[0885] Prompt Sentence Examples

[0886] Upload a photo of your new dessert and our AI will recognize its properties and characteristics and generate a product description.

[0887] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0888] Step 1:

[0889] A user takes a photo of an item with their smartphone.

[0890] Input: Product (e.g. Chocolate Cake)

[0891] Output: Product photo

[0892] Specific action: The user opens the camera app on their smartphone and takes a photo of a product, for example, a new dessert called "Chocolate Cake."

[0893] Step 2:

[0894] A user uploads a photo to the system.

[0895] Input: Product photo taken

[0896] Output: Photo data sent to the server

[0897] How it works: The user opens the application, selects the photo they have taken, and presses the upload button, which sends the photo to the server via the Internet.

[0898] Step 3:

[0899] The server receives the photos and inputs them into the AI ​​model.

[0900] Input: Photo data sent to the server

[0901] Output: Photo data fed into the AI ​​model

[0902] Specific operation: The server receives an HTTP request, temporarily stores the photo data, and inputs it into a TensorFlow or PyTorch model.

[0903] Step 4:

[0904] The server uses an AI model to recognize the product's characteristics and features.

[0905] Input: Photo data fed into the AI ​​model

[0906] Output: Recognized product characteristics or features (e.g., chocolate cake, cream topping, berry decoration)

[0907] What it does: The server runs an AI model to extract product characteristics and features from the photo, such as the type of dessert, its main ingredients, and visual features.

[0908] Step 5:

[0909] The server generates a product description based on the recognition results.

[0910] Input: Recognized product characteristics or features

[0911] Output: Generated product description (e.g., "This chocolate cake features rich chocolate and a creamy topping. The berry decoration makes it visually appealing.")

[0912] Specific operation: The server uses GPT-3 or BERT to generate a product description using the recognition results as input.

[0913] Step 6:

[0914] The server displays the product description on the website or application.

[0915] Input: Generated product description

[0916] Output: Product description displayed on your website or application

[0917] What happens: The server saves the product description in the website database and sends it to the front-end for display, where customers can view it.

[0918] (Application example 1)

[0919] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0920] Traditionally, creating menus in restaurants was often done manually, which was time-consuming and labor-intensive, and made it difficult to fully convey the appeal of the products. Other issues included a lack of multilingual support for foreign tourists and a lack of recommendation functions tailored to customer preferences. Furthermore, there were limited ways to improve the in-store customer experience, with customers having to wait in line for wait staff or cashiers, and there was a need to improve customer satisfaction.

[0921] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0922] In this invention, the server includes a menu creation means using a smartphone, an AI means for generating product descriptions from smartphone photos, an AI means for generating translations, a means for improving the customer experience by eliminating the need to wait in line for waiters or cashiers, an AI means for explaining questions when considering a menu, a means for making recommendations based on preferences, a means for recognizing product characteristics and features using an image recognition model, and a means for generating product descriptions based on the recognized features using a natural language generation model. This enables more efficient menu creation and the automatic generation of attractive product descriptions, enabling multilingual support, an improved customer experience, and recommendations based on individual preferences.

[0923] A "menu creation means using a smartphone" is a means for efficiently creating a restaurant menu using a smartphone's camera and applications.

[0924] "AI means for generating product descriptions from smartphone photos" refers to an artificial intelligence means for analyzing product photos taken with a smartphone, recognizing their characteristics and features, and automatically generating product descriptions.

[0925] "Means for AI-generated translation work" refers to a method for using artificial intelligence to translate product descriptions and menu contents into multiple languages.

[0926] "Measures to improve the customer experience by eliminating the need to wait in line for waiters or cashiers" refers to measures that allow customers to receive service smoothly without having to wait in line for waiters or cashiers.

[0927] "Means for AI to explain questions when considering a menu" refers to a means by which artificial intelligence automatically provides answers to questions that arise when customers are considering a menu.

[0928] "Means for recommendation based on preferences" refers to means for recommending appropriate products and menus based on a customer's past choices and preferences.

[0929] "Means for recognizing the characteristics and features of a product using an image recognition model" refers to means for automatically recognizing the characteristics and features of a product from a photograph of the product using image recognition technology.

[0930] "Means for generating product descriptions based on features recognized using a natural language generation model" refers to means for automatically generating attractive product descriptions using natural language generation technology based on recognized product characteristics and features.

[0931] A system for implementing this invention includes a menu creation means using a smartphone, an AI means for generating product descriptions from smartphone photos, a means for generating translation work using AI, a means for improving the customer experience by eliminating the need to wait in line for waiters or cashiers, a means for AI to explain questions that arise when considering a menu, a means for making recommendations based on preferences, a means for recognizing product characteristics and features using an image recognition model, and a means for generating product descriptions based on the features recognized using a natural language generation model.

[0932] System Program

[0933] The server implements the system using the following hardware and software.

[0934] Hardware:

[0935] Smartphone (with camera)

[0936] Server (for hosting AI models)

[0937] software:

[0938] TensorFlow: A library for image recognition

[0939] OpenAI GPT-3: An API for Natural Language Generation

[0940] Processing Description

[0941] Image Recognition:

[0942] Users take photos of dishes with their smartphone camera and upload them to the server through the application. The server then uses TensorFlow's ResNet50 model to recognize the characteristics and features of the dishes from the images. For example, features such as "chicken curry, spicy, tomato-based" can be recognized.

[0943] Natural language generation:

[0944] The server converts the recognized features into text and sends it to OpenAI GPT-3 as a prompt. An example of a prompt is "Generate an appealing product description for a dish with the following characteristics: chicken curry, spicy, tomato-based." GPT-3 generates an appealing product description based on this prompt. For example, it generates a product description such as "Spicy chicken curry features juicy chicken simmered in a tomato-based sauce. The fragrant spices will whet your appetite."

[0945] Translation work:

[0946] The generated product description can be translated into multiple languages ​​as needed. The server uses AI to translate into multiple languages, such as English and Chinese.

[0947] Improved customer experience:

[0948] Customers can use their smartphones to select menu items and place orders in the store, reducing waiting time for waiters and cashiers and enabling smoother service.

[0949] Question explanation and recommendations:

[0950] AI automatically answers questions that arise as customers consider menu items, and recommends appropriate products and menu items based on customers' past choices and preferences.

[0951] This makes it possible to create menus more efficiently and automatically generate compelling product descriptions, enabling multilingual support, improved customer experience, and personalized recommendations.

[0952] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0953] Step 1:

[0954] A user takes a photo of a dish with the camera on their smartphone and uploads it to the server through the application. The input is the photo of the dish, and the output is the image data sent to the server.

[0955] Step 2:

[0956] The server uses TensorFlow's ResNet50 model to recognize the characteristics and features of food from uploaded image data. The input is the image data, and the output is a list of recognized characteristics and features. Specifically, the server preprocesses the image data, inputs it into the model, and obtains the prediction results.

[0957] Step 3:

[0958] The server converts the recognized traits and features into a string and sends it to OpenAI GPT-3 as a prompt. The input is a list of traits and features, and the output is a prompt. The specific operation is to format the traits and features and generate a prompt.

[0959] Step 4:

[0960] The server receives the response from GPT-3 and generates an attractive product description. The input is the prompt sentence, and the output is the generated product description. Specifically, it calls the GPT-3 API, parses the response, and obtains the product description.

[0961] Step 5:

[0962] The server translates the generated product description into multiple languages ​​as needed. The input is the product description, and the output is the translated product description. Specifically, it calls the translation API to translate into multiple languages.

[0963] Step 6:

[0964] Users use their smartphones to select menu items and place orders in the store. The input is the generated product description, and the output is the user's order information. Specifically, the application displays the product description and the user selects the order.

[0965] Step 7:

[0966] The server uses AI to automatically provide answers to questions that arise when the user is considering the menu. The input is the user's question, and the output is the answer provided by the AI. Specifically, the server analyzes the question and generates an appropriate answer.

[0967] Step 8:

[0968] The server recommends appropriate products and menus based on the user's past selections and preferences. The input is the user's past selection data, and the output is the recommended products and menus. Specifically, the server analyzes the past selection data and uses a recommendation algorithm to recommend products.

[0969] Example 2

[0970] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0971] In the previous system, translation of product descriptions into multiple languages ​​was often done manually, which was a time-consuming and labor-intensive process. There was also the issue of insufficient service for foreign tourists, making product descriptions difficult to understand. Furthermore, there was a risk of customer satisfaction declining because there was insufficient effort to improve the customer experience or resolve questions when considering menu items.

[0972] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0973] In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for generating translations using the artificial intelligence, a means for improving the customer experience, a means for the artificial intelligence to explain questions that arise when considering a menu, a means for making recommendations based on preferences, a means for translating product descriptions into multiple languages, a means for generating prompt sentences using a generative AI model, a means for inputting the prompt sentences and product descriptions into the generative AI model and obtaining the translation results, a means for saving the translation results in a database, a means for transmitting the translation results to a terminal, and a means for the terminal to display the translation results. This automates the translation of product descriptions into multiple languages, improves service for foreign tourists, and enables an improved customer experience and resolution of questions that arise when considering a menu.

[0974] A "smart device" is a portable electronic device such as a smartphone or tablet that has advanced computing power and communication functions.

[0975] "Menu creation means" refers to functions or applications for creating menus for restaurants or retail stores using smart devices.

[0976] "Artificial intelligence means for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with a smart device and automatically generates product descriptions based on those images.

[0977] "Means for generating translation work using artificial intelligence" refers to a function that uses artificial intelligence to translate text data into other languages.

[0978] "Means to improve the customer experience" are functions and methods for improving the convenience and satisfaction of customers when using products and services.

[0979] "A means for AI to explain questions when considering a menu" is a function in which AI automatically provides answers to questions or queries that arise when customers are considering a menu.

[0980] "Preference-based recommendations" is a feature that recommends appropriate products and services based on a customer's past choices and preferences.

[0981] "Means for translating product descriptions into multiple languages" is a function for translating product descriptions into multiple languages.

[0982] A "generative AI model" is an artificial intelligence model that has been trained to perform tasks such as text generation or translation.

[0983] A "prompt" is an instruction entered into a generative AI model to make it perform a specific task.

[0984] "Means for saving translation results in a database" refers to a function for saving translation results generated by a generative AI model in a database.

[0985] The "means for transmitting the translation result to the terminal" is a function for transmitting the translation result stored in the database to the user's terminal.

[0986] The "means for the terminal to display the translation result" is a function that displays the translation result received by the user's terminal on the screen.

[0987] This invention is a system including a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images, a means for generating translation work using artificial intelligence, a means for improving customer experience, a means for artificial intelligence to explain questions when considering menus, a means for making recommendations based on preferences, a means for translating product descriptions into multiple languages, a means for generating prompt sentences using a generative AI model, a means for inputting the prompt sentences and product descriptions into the generative AI model and obtaining the translation results, a means for saving the translation results in a database, a means for transmitting the translation results to a terminal, and a means for the terminal to display the translation results.

[0988] Hardware and software used

[0989] Hardware:

[0990] Smart devices (smartphones, tablets, etc.)

[0991] Server (a computer with high-performance computing capabilities)

[0992] software:

[0993] Generative AI models (e.g., OpenAI's GPT-4)

[0994] Database Management Systems

[0995] Communication protocol (e.g. HTTP / HTTPS)

[0996] Data processing and calculation

[0997] Generate product description:

[0998] A user takes a picture of a product using a smart device and sends the image to a server, which then uses artificial intelligence means to generate a product description from the image. Specifically, the server uses image analysis algorithms to recognize the product's characteristics and features and generates a product description based on the image.

[0999] Generate prompt statement:

[1000] The server uses the generative AI model to generate a prompt sentence, which specifies the language to be translated (e.g., English, Chinese, or Korean).

[1001] Execution of translation work:

[1002] The server inputs the generated prompt and product description into the generative AI model and obtains the translation results. The generative AI model performs multilingual translation based on the input text.

[1003] Saving and sending translation results:

[1004] The server stores the translation results in a database, which is then sent to the user's device, where they are displayed.

[1005] Specific examples

[1006] Examples:

[1007] A user enters a product description in Japanese: "This product is made with high-quality materials."

[1008] The server generates the prompt "Translate the following product description into English: This product is made from high-quality materials."

[1009] The server inputs the prompt and product description into the generated AI model and obtains the English translation result, "This product uses high-quality materials."

[1010] The server stores the translation results in a database and sends them to the terminal.

[1011] The device displays the translation results for the user to confirm.

[1012] In this way, the system enables the server, terminals, and users to work together to translate product descriptions into multiple languages.

[1013] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1014] Step 1:

[1015] The user enters a product description.

[1016] The user inputs a product description in text format using a smart device, and the input product description is sent from the smart device to the server.

[1017] Input: Product description text

[1018] Output: Product description text sent to the server

[1019] Step 2:

[1020] The server receives the product description.

[1021] The server receives the product description text sent by the user and temporarily stores it in a memory.

[1022] Input: Product description text submitted by the user

[1023] Output: Product description text stored in memory

[1024] Step 3:

[1025] The server generates a prompt.

[1026] The server generates a prompt sentence by specifying the language to be translated. For example, if translation into English, Chinese, and Korean is required, the server generates a prompt sentence corresponding to each language.

[1027] Input: Product description text, target language

[1028] Output: Generated prompt statement

[1029] Step 4:

[1030] The server inputs a prompt and product description into the generative AI model.

[1031] The server inputs the generated prompt and product description text into the generative AI model, which then translates the input text.

[1032] Input: prompt text, product description text

[1033] Output: The data fed into the generative AI model

[1034] Step 5:

[1035] The server retrieves the translation results.

[1036] The server obtains translation results from the generative AI model, which are then separated into separate languages.

[1037] Input: Data fed into a generative AI model

[1038] Output: Translation result text

[1039] Step 6:

[1040] The server stores the translation results in a database.

[1041] The server saves the translation results in a database, associating the original Japanese product description with the translation results.

[1042] Input: Translation result text, original product description text

[1043] Output: Translation results stored in a database

[1044] Step 7:

[1045] The server sends the translation results to the terminal.

[1046] The server then transmits the saved translation results to the user's device, providing the data in a format that the user can access.

[1047] Input: Translation results stored in the database

[1048] Output: Translation results sent to the device

[1049] Step 8:

[1050] The device will display the translation results.

[1051] The terminal receives the translation results sent from the server and displays them to the user, who can then check the displayed translation results and make corrections or retranslations as necessary.

[1052] Input: Translation results sent from the server

[1053] Output: Translation results displayed on the device

[1054] (Application example 2)

[1055] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1056] In modern brick-and-mortar stores, foreign tourists face language barriers when purchasing products, making it difficult for them to understand product descriptions. Visually impaired people and those with limited literacy skills also face limited options for understanding product descriptions. This can lead to a poor customer experience and potentially impact sales.

[1057] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a menu creation means using a smartphone, an artificial intelligence means for generating product descriptions from photos on the smartphone, a means for generating translation work by artificial intelligence, a means for improving the customer experience, a means for explaining questions when considering menus by artificial intelligence, a means for making recommendations according to preferences, and a means for translating product descriptions into multiple languages ​​and displaying and playing aloud the product descriptions in the user's selected language. This makes it possible to make product descriptions easier to understand for foreign tourists, visually impaired people, and people with difficulty reading and writing.

[1058] "Menu creation means using a smartphone" refers to a means for creating a menu using a smartphone.

[1059] "Artificial intelligence means for generating product descriptions from photos taken with a smartphone" refers to a means that uses artificial intelligence technology to automatically generate product descriptions from photos taken with a smartphone.

[1060] "Means for generating translation work using artificial intelligence" refers to means for translating text into multiple languages ​​using artificial intelligence.

[1061] "Measures to improve customer experience" are measures to improve the experience customers have when purchasing products.

[1062] "Means for AI to explain questions when considering a menu" refers to a means for AI to answer questions that arise when customers are considering a menu.

[1063] "Means for recommendation based on preferences" refers to means for recommending products and services based on customer preferences.

[1064] "Means for translating product descriptions into multiple languages ​​and displaying and playing audio in the language set by the user" means means for translating product descriptions into multiple languages ​​and displaying and playing audio in the language set by the user.

[1065] To implement this invention, it is necessary to build a system that combines a smartphone, artificial intelligence technology, a translation API, and audio playback functionality.

[1066] First, smartphones have a camera function that allows users to take pictures of products. When a user takes a picture of a product, the image data is sent to a server. The server then generates a product description using image recognition technology. This image recognition technology can use, for example, Google Cloud Vision API.

[1067] The generated product description is then translated into multiple languages ​​by artificial intelligence on the server using translation services such as Google Cloud Translation API, and the translated text is sent to and displayed on the user's smartphone based on their language preference.

[1068] Furthermore, the translated product descriptions can be played back aloud using the smartphone's voice playback function, making it accessible to people with visual impairments and those with limited literacy.

[1069] As a concrete example, consider a tourist looking to purchase a leather wallet at a brick-and-mortar store in Japan. When the tourist scans the tag on the leather wallet with their smartphone camera, the app automatically recognizes the product description and translates it into the user's language (e.g., English). The translated description is displayed and played aloud: "This is a high-quality leather wallet."

[1070] An example of a prompt sentence might be:

[1071] I am developing an application that translates product descriptions into multiple languages. Please translate the following text into the specified languages:

[1072] Text: "This is a high-quality leather wallet."

[1073] Target language: Japanese

[1074] In this way, foreign tourists, visually impaired people, and people with limited literacy will be able to understand product descriptions without experiencing language barriers when purchasing products in physical stores.

[1075] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1076] Step 1:

[1077] A user takes a photo of a product using the smartphone camera. The input is the product image taken with the smartphone camera. The output is the captured product image data.

[1078] Step 2:

[1079] The terminal sends the captured product image data to the server. The input is the product image data. The output is the product image data sent to the server.

[1080] Step 3:

[1081] The server generates a product description using image recognition technology. The input is product image data. The server extracts text information from the image using Google Cloud Vision API or similar and generates a product description. The output is the generated product description text.

[1082] Step 4:

[1083] The server translates the generated product description text into multiple languages. The input is the product description text. The server translates it into the specified language using the Google Cloud Translation API. The output is the translated product description text.

[1084] Step 5:

[1085] The server sends the translated product description text to the terminal. The input is the translated product description text. The output is the translated product description text sent to the terminal.

[1086] Step 6:

[1087] The device displays the translated product description text. The input is the translated product description text. The output is the translated product description displayed on the smartphone screen.

[1088] Step 7:

[1089] The device plays the translated product description text aloud. The input is the translated product description text. The device converts the text into speech using the smartphone's voice playback function and plays it back. The output is the translated product description played aloud.

[1090] Example 3

[1091] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1092] Conventional menu creation systems and customer experience improvement systems do not adequately answer questions users have when considering menus, nor recommend products that meet individual preferences. They also lack the ability to reduce waiting times or provide multilingual support. This can lead to lower customer satisfaction and negatively impact store sales.

[1093] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1094] In this invention, the server includes a means for a user to input information, a means for transmitting the input information to the server, a means for the server to analyze the input information using a generative AI model, a means for generating recommendations and answers based on the analysis results, a means for transmitting the generated recommendations and answers to the terminal, and a means for the terminal to display the recommendations and answers to the user. This enables appropriate answers to questions when a user is considering a menu and product recommendations based on individual preferences. It also reduces waiting times and provides multilingual services, which is expected to improve customer satisfaction and increase store sales.

[1095] A "smart device" is an electronic device that allows a user to input information and communicate with a server, and includes smartphones, tablets, and the like.

[1096] "Artificial intelligence means" refers to algorithms or models that analyze input data and generate appropriate answers or recommendations.

[1097] A "generative AI model" refers to a machine learning model that analyzes user input information and generates appropriate answers and recommendations.

[1098] "Measures to improve customer experience" refers to measures to reduce waiting times and enable customers to use services comfortably.

[1099] "Means for AI to explain questions when considering a menu" refers to means for AI to provide appropriate answers to questions that arise when a user is considering a menu.

[1100] The term "means for recommending products according to preferences" refers to a means for recommending appropriate products based on the user's preferences.

[1101] "Means for user input of information" refers to an interface through which a user inputs information in the form of text, voice, or the like.

[1102] "Means for transmitting input information to a server" refers to a communication means for transmitting information input by a user to a server.

[1103] "Means by which the server analyzes input information using a generative AI model" refers to means by which the server uses a generative AI model to analyze information received from a user.

[1104] "Means for generating recommendations and answers based on the analysis results" refers to means for generating recommendations and answers for users based on the analysis results of the generative AI model.

[1105] "Means for transmitting generated recommendations and answers to a terminal" refers to a communication means for transmitting recommendations and answers generated by a server to a user's terminal.

[1106] "Means for the terminal to display recommendations and answers to the user" refers to means for the user's terminal to display recommendations and answers received from the server to the user.

[1107] This invention is a system in which a user inputs information using a smart device, a server analyzes the information using a generative AI model, and generates and provides appropriate recommendations and answers to the user. Specific embodiments of this system are described below.

[1108] First, a user inputs information into the system using a smart device such as a smartphone or tablet. For example, the user inputs "I like spicy food." This information is input through the interface of the smart device.

[1109] Next, the smart device transmits the input information to the server, converting the input data into an appropriate format (e.g., JSON format) and transmitting it to the server via a network.

[1110] The server analyzes the received user input using a generative AI model (e.g., OpenAI's GPT-4). Specifically, it analyzes the information "I like spicy food" and searches a database of spicy food.

[1111] The server then recommends products that match the user's preferences based on the analysis results. For example, it generates a list of spicy dishes and presents it to the user. If the user asks, "What are the ingredients in this dish?", the server generates an answer to that question.

[1112] The generated recommendations and answers are sent from the server to the smart device, where the server converts the data into an appropriate format and sends it to the smart device over the network.

[1113] The smart device displays the recommendations and answers received from the server to the user. For example, it displays "Recommended spicy dishes are mapo tofu, kimchi jjigae, and Sichuan-style spicy hotpot."

[1114] As a concrete example, the following prompt sentence is shown.

[1115] "I like spicy food. Can you recommend any dishes?"

[1116] "What are the ingredients in this dish?"

[1117] In this way, a system is realized in which the server, smart device, and user work together to improve the customer experience. This system allows users to receive appropriate answers to questions when considering menu items and product recommendations tailored to their individual preferences. It also reduces waiting times and provides multilingual services, which is expected to improve customer satisfaction and increase store sales. The flow of the identification process in Example 3 will be described with reference to FIG. 15.

[1118] Step 1:

[1119] The user enters information using the terminal.

[1120] A user opens an application on a smartphone or tablet, enters "I like spicy food" in the text box, and presses the send button. The input data is in text format and is information that indicates the user's preferences.

[1121] Step 2:

[1122] The terminal transmits the input information to the server.

[1123] The terminal converts the user's input information, such as "I like spicy food," into JSON format and sends it to the server using the HTTPS protocol. The input data is the user's preference information in text format, and the output data is the request data in JSON format.

[1124] Step 3:

[1125] The server analyzes the input information using a generative AI model.

[1126] The server analyzes the received JSON data and inputs the information "I like spicy food" into a generative AI model. The generative AI model (e.g., GPT-4) searches a database of spicy food and generates a list of related dishes. The input data is the request data in JSON format, and the output data is a list of dishes as the analysis result.

[1127] Step 4:

[1128] The server generates recommendations and answers based on the analysis results.

[1129] The server generates a list of spicy dishes based on the analysis results obtained from the generative AI model. For example, it lists dish names such as "Mapo Tofu, Kimchi Jjigae, Sichuan-style Spicy Hot Pot." The input data is the list of dishes as a result of the analysis, and the output data is a list of recommendations to present to the user.

[1130] Step 5:

[1131] The server sends the generated recommendations and answers to the device.

[1132] The server converts the generated list of spicy dishes into JSON format and sends it to the terminal using the HTTPS protocol. The input data is the recommendation list, and the output data is the JSON format response data.

[1133] Step 6:

[1134] The device displays recommendations and answers to the user.

[1135] The terminal parses the received JSON data and displays to the user, "Recommended spicy dishes are mapo tofu, kimchi jjigae, and Sichuan-style spicy hotpot." The input data is response data in JSON format, and the output data is text information displayed to the user.

[1136] (Application example 3)

[1137] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1138] Conventional food delivery systems have the drawback of making it difficult for customers to obtain detailed information when selecting a menu item, and lacking recommendations tailored to individual preferences. Furthermore, there is no way to provide real-time information about ingredients and nutritional information for dishes, which reduces customer satisfaction.

[1139] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for generating translation work by artificial intelligence, a means for improving the customer experience by shortening waiting time, a means for explaining questions when considering a menu by artificial intelligence, a means for making recommendations based on preferences, a means for recommending dishes based on the customer's preferences, and a means for providing information on ingredients for dishes. This allows customers to obtain detailed menu information in real time and receive recommendations based on their individual preferences.

[1140] A "smart device" is a portable electronic device with internet connectivity, such as a smartphone or tablet.

[1141] The "menu creation means" is a function for generating a list of dishes and drinks that customers can choose from.

[1142] "Artificial intelligence means for generating product descriptions from images" is a function that analyzes images taken with a smart device and automatically generates product descriptions based on the images' contents.

[1143] "Means for artificial intelligence-generated translation work" refers to the ability to automatically translate text between different languages.

[1144] "Measures to improve customer experience by reducing waiting time" refers to a function that reduces the waiting time when customers use a service.

[1145] "A means for AI to explain questions when considering a menu" is a function in which AI provides appropriate answers to questions that arise when customers are choosing a menu.

[1146] "Means for recommendation based on preferences" is a function that recommends appropriate products and services based on the customer's preferences and past choices.

[1147] "Means for recommending dishes based on customer preferences" is a function that recommends appropriate dishes based on the preference information entered by the customer.

[1148] The "means for providing information on ingredients of a dish" is a function that provides information on ingredients and components contained in a specific dish.

[1149] A system for implementing this invention includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for generating translation work using artificial intelligence, a means for improving customer experience by reducing waiting time, a means for using artificial intelligence to explain questions when considering a menu, a means for making recommendations based on preferences, a means for recommending dishes based on the customer's preferences, and a means for providing information on ingredients for dishes.

[1150] System Program

[1151] The system is implemented using Python and the OpenAI API. The server receives customer input and uses a generative AI model to provide appropriate recommendations and information.

[1152] Processing Description

[1153] The server receives customer preferences and questions sent from the smart device and generates prompts based on them. The generated prompts are input into a generative AI model via the OpenAI API to generate appropriate answers and recommendations. The generated information is then sent back to the smart device and provided to the customer.

[1154] The hardware used is smart devices such as smartphones and tablets, and the software used is Python and the OpenAI API.

[1155] Specific examples

[1156] For example, if a customer enters "I like spicy food," the server generates the following prompt:

[1157] What dishes would you recommend to people who like spicy food?

[1158] When this prompt is input into the OpenAI API, the generative AI model will recommend spicy dishes such as "mapo tofu" and "kimchi hotpot."

[1159] Also, if a customer asks "What are the ingredients in mapo tofu?", the server generates the following prompt:

[1160] "What are the ingredients in mapo tofu?"

[1161] When this prompt is input into the OpenAI API, the generative AI model provides ingredient information such as "tofu, minced pork, green onion, garlic, ginger, chili bean paste, sweet bean paste, soy sauce, sake, sugar, and chicken stock."

[1162] This allows customers to get detailed menu information in real time and receive recommendations based on their individual preferences.

[1163] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1164] Step 1:

[1165] A user launches the application using a smart device and inputs their preferences and questions. The input information is sent to the server in text format. Examples of input include "I like spicy food" and "What are the ingredients in mapo tofu?"

[1166] Step 2:

[1167] The server analyzes the input information received from the user and generates an appropriate prompt. For example, for the input "I like spicy food," it generates the prompt "Please tell me what dishes you would recommend for people who like spicy food." This prompt is then input into the generative AI model.

[1168] Step 3:

[1169] The server sends the generated prompt to the OpenAI API and obtains appropriate answers or recommendations based on the generative AI model. For example, in response to the prompt, "What dishes would you recommend for people who like spicy food?", answers such as "Mapo tofu" and "Kimchi hotpot" are generated.

[1170] Step 4:

[1171] The server analyzes the answers and recommendations obtained from the generative AI model and converts them into a format for presentation to the user. For example, it formats answers such as "mapo tofu" and "kimchi hotpot" into a list format.

[1172] Step 5:

[1173] The server sends formatted answers and recommendations to the smart device, where the user can view the information on the screen of the smart device.

[1174] Step 6:

[1175] The user can make menu choices based on the information provided or ask more detailed questions, for example, re-entering the question "What are the ingredients in mapo tofu?"

[1176] Step 7:

[1177] The server receives input from the user again and repeats the same process: it generates a prompt again, inputs it into the generative AI model, obtains the answer, formats it, and provides it to the user.

[1178] This allows users to obtain detailed menu information in real time and receive recommendations tailored to their individual preferences.

[1179] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1180] "Example 1"

[1181] As one embodiment of the present invention, a system incorporating an emotion engine is provided. This system recognizes a user's emotions and recommends products based on those emotions. Specifically, when a user selects a product, the system recognizes the user's emotions from their facial expressions and tone of voice, and recommends products that reflect those emotions.

[1182] For example, if a user has a happy expression, the system will recommend products that the user will find pleasing.

[1183] "Example 2"

[1184] The emotion engine also generates a product description based on the user's emotions. Specifically, it recognizes the user's emotions and generates a product description by selecting words that match those emotions. For example, if the user is feeling down, it generates a product description by selecting words that will cheer up the user. "Example 3"

[1185] Furthermore, the emotion engine recognizes the user's emotions and provides services according to those emotions. Specifically, it recognizes the user's emotions and provides services that match those emotions. For example, if the user shows an angry expression, it provides a service such as an apology to the user.

[1186] The processing flow of each embodiment will be described below.

[1187] "Example 1"

[1188] Step 1: As the user selects a product, the system captures the user's facial expressions and tone of voice.

[1189] Step 2: The emotion engine recognizes the user's emotion from the captured information.

[1190] Step 3: The system recommends products that match the recognized emotion.

[1191] "Example 2"

[1192] Step 1: As the user selects a product, the system captures the user's facial expressions and tone of voice.

[1193] Step 2: The emotion engine recognizes the user's emotion from the captured information.

[1194] Step 3: The system generates a product description by selecting words that match the recognized emotions.

[1195] "Example 3"

[1196] Step 1: As the user selects a product, the system captures the user's facial expressions and tone of voice.

[1197] Step 2: The emotion engine recognizes the user's emotion from the captured information.

[1198] Step 3: The system provides services that match the recognized emotions.

[1199] Example 1

[1200] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1201] Creating restaurant menus and improving customer experiences has traditionally required a lot of time and effort. In particular, generating product descriptions from product photos and making recommendations based on customer preferences consumes a large amount of human resources. Furthermore, multilingual service for foreign tourists is insufficient, and improving customer satisfaction is required. Furthermore, the inability to make product recommendations based on customer emotions can sometimes result in a decline in the quality of the customer experience. An efficient system to solve these issues is needed.

[1202] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1203] In this invention, the server includes a menu creation means using a smartphone, an artificial intelligence means for generating product descriptions from photos on the smartphone, a means for generating translation work using artificial intelligence, a means for improving the customer experience by eliminating the need to wait in line for waiters or cashiers, a means for providing explanations using artificial intelligence for questions when considering menu items, a means for making recommendations based on preferences, and a means for recognizing user emotions and recommending products based on those emotions. This makes it possible to create menus more efficiently, recommend products based on customer preferences, improve service for foreign tourists by supporting multiple languages, and recommend products based on customer emotions.

[1204] A "menu creation means using a smartphone" is a means for creating a restaurant menu using a smartphone's camera and applications.

[1205] "Artificial intelligence means for generating product descriptions from smartphone photos" refers to a means that uses artificial intelligence technology to analyze product photos taken with a smartphone, recognize their characteristics and features, and automatically generate product descriptions.

[1206] "Means for generating translation work using artificial intelligence" refers to a means for translating product descriptions and menu contents into multiple languages ​​using artificial intelligence technology.

[1207] "Measures to improve the customer experience by eliminating the need to wait in line for waiters or cashiers" refers to measures that allow customers to receive service smoothly without having to wait in line for waiters or cashiers.

[1208] "Means for AI to explain questions when considering a menu" refers to a means by which AI automatically provides answers to questions that arise when customers are considering a menu.

[1209] "Means for recommendation based on preferences" refers to means for recommending appropriate products based on a customer's past choices and preferences.

[1210] "Means for recognizing a user's emotions and recommending products based on those emotions" refers to a means for recognizing emotions by analyzing a user's facial expressions and tone of voice, and recommending products that match those emotions.

[1211] The present invention provides a system that allows restaurant operators to create menus using smartphones and improve customer experience. Specific embodiments of this system are described below.

[1212] First, a user takes a photo of a product using the smartphone camera. For example, if they want to take a photo of a new dessert, they open the smartphone's camera app, frame the dessert, and press the shutter button.

[1213] Next, the device uploads the captured photo to the server. Specifically, the smartphone application sends the photo data to the server via an HTTP request. At this time, the photo data is sent in the appropriate format (e.g., JPEG, PNG).

[1214] The server analyzes the received photo using image recognition software (e.g., Google Cloud Vision API). The server recognizes the product characteristics (e.g., chocolate cake, fruit parfait) from the photo. For example, the Google Cloud Vision API returns the label "chocolate cake."

[1215] The server then generates a product description based on the recognized characteristics, using a generative AI model (e.g., OpenAI GPT-3) to generate a product description such as "A rich chocolate cake with a moderate sweetness and a moist texture."

[1216] The generated product description is then displayed on the restaurant's website or application by the server. Specifically, the product description is sent to the front end in HTML or JSON format and displayed in the user interface.

[1217] Additionally, as users select products, the device uses a camera and microphone to capture their facial expressions and tone of voice. For example, a smartphone camera captures the user's smile, and a microphone records the user's tone of voice.

[1218] The server analyzes the captured data using an emotion engine (e.g., Microsoft Azure Emotion API) to recognize the user's emotions. Based on the recognized emotions, the server recommends products that suit the user. For example, if the user shows a happy expression, the server will recommend a "fruit parfait."

[1219] As a concrete example, consider the case where a user takes a photo of a new dessert with their smartphone and uploads it to the system. The server analyzes the photo using the Google Cloud Vision API and recognizes its characteristics as "chocolate cake." The AI ​​then generates a product description: "A rich chocolate cake. It is not too sweet and has a moist texture."

[1220] If a user shows a happy expression while browsing the menu using their smartphone camera, the Microsoft Azure Emotion API will recognize that expression and the server will recommend the "fruit parfait" that the user is likely to enjoy.

[1221] Example prompt sentence:

[1222] "Upload a photo of a product taken with your smartphone. AI automatically recognizes the product's characteristics and generates a product description. It also uses the camera and microphone to recommend products based on your emotions."

[1223] In this way, a system is realized in which the server, terminals, and users work together to efficiently create restaurant menus and recommend products to customers.

[1224] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1225] Step 1:

[1226] The user takes a photo of the product.

[1227] A user uses the smartphone camera to take a photo of a product. For example, to take a photo of a new dessert, the user launches the smartphone's camera app, frames the dessert, and presses the shutter button. The input is the captured photo, and the output is image data stored in the smartphone.

[1228] Step 2:

[1229] The device uploads the photo to the server.

[1230] The device uploads the captured photo to the server. Specifically, the smartphone application sends the photo data to the server via an HTTP request. At this time, the photo data is sent in an appropriate format (e.g., JPEG, PNG). The input is the image data stored in the smartphone, and the output is the image data sent to the server.

[1231] Step 3:

[1232] The server analyzes the photo and recognizes the product's characteristics.

[1233] The server analyzes the received photo using image recognition software (e.g., image recognition API). The server recognizes the product characteristics (e.g., chocolate cake, fruit parfait) from the photo. For example, the image recognition API returns the label "chocolate cake." The input is the image data sent to the server, and the output is the recognized product characteristics information.

[1234] Step 4:

[1235] The server generates a product description.

[1236] The server generates a product description based on the recognized characteristics. Using a generative AI model (e.g., a generative AI model), it generates a product description such as "A rich chocolate cake with a moderate sweetness and a moist texture." The input is the product characteristic information, and the output is the generated product description.

[1237] Step 5:

[1238] The server displays the product description on the website or application.

[1239] The server displays the generated product description on the restaurant's website or application. Specifically, it sends the product description in HTML or JSON format to the front end and displays it in the user interface. The input is the generated product description, and the output is the product description displayed on the website or application.

[1240] Step 6:

[1241] When a user selects a product, the device captures the user's emotions.

[1242] When a user selects a product, the device uses a camera and microphone to capture the user's facial expressions and tone of voice. For example, a smartphone camera captures the user's smile, and the microphone records the user's tone of voice. The input is the user's facial expressions and tone of voice, and the output is the captured emotional data.

[1243] Step 7:

[1244] The server analyzes emotions and recommends appropriate products.

[1245] The server analyzes the captured data using an emotion engine (e.g., emotion analysis API) to recognize the user's emotions. The server then recommends products that suit the user based on the recognized emotions. For example, if the user shows a happy expression, the server recommends a "fruit parfait." The input is the captured emotion data, and the output is recommended product information.

[1246] (Application example 1)

[1247] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1248] Creating menus and improving customer experience in traditional restaurants has been a time-consuming and labor-intensive process. In particular, there are multiple challenges, such as creating menus, generating product descriptions, making recommendations based on customer preferences, and even multilingual support. Furthermore, it is difficult to provide services that respond to customer emotions, making it difficult to improve customer satisfaction.

[1249] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an AI means for generating product descriptions from photos on the smart device, an AI means for generating translation work, a means for improving the customer experience, an AI means for explaining questions when considering a menu, a means for making recommendations based on preferences, and a means for recognizing customer emotions and recommending products based on those emotions. This enables efficient menu creation, automatic generation of product descriptions, multilingual support, improved customer experience, and emotion-based recommendations.

[1250] "Smart devices" are portable electronic devices with advanced functions, such as smartphones and smart glasses.

[1251] "Menu creation means" refers to a function or method for creating a restaurant menu using a smart device.

[1252] "AI means for generating product descriptions" refers to artificial intelligence technology that automatically recognizes the characteristics and features of products from photos taken on a smart device and generates product descriptions based on that information.

[1253] "Means for AI to generate translation work" refers to a function that uses artificial intelligence to automatically perform translations that support multiple languages.

[1254] "Customer experience improvement measures" are methods and functions that improve the customer experience, such as eliminating the need to wait in line for waiters or cashiers.

[1255] "A means for AI to explain questions when considering a menu" is a function that automatically uses artificial intelligence to explain questions that arise when customers are considering a menu.

[1256] "Means for recommending based on preferences" is a function that recommends appropriate products based on the customer's preferences.

[1257] "Means of recognizing customer emotions and recommending products based on those emotions" refers to a function that recognizes emotions from the customer's facial expressions, tone of voice, etc., and recommends products that match those emotions.

[1258] A system for implementing this invention includes a menu creation means using a smart device, an AI means for generating product descriptions from photos on a smart device, an AI means for generating translation work, a means for improving the customer experience, a means for AI to explain questions that arise when considering a menu, a means for making recommendations based on preferences, and a means for recognizing customer emotions and recommending products based on those emotions.

[1259] Program processing explanation

[1260] Hardware

[1261] The server is equipped with a high-performance GPU, and the terminals are smart devices such as smartphones and smart glasses.

[1262] software

[1263] The server uses the following software:

[1264] OpenCV: Image processing library

[1265] Keras: A deep learning library

[1266] Transformers: Generative AI models (e.g., GPT-3)

[1267] Data processing and calculation

[1268] 1. Menu creation method:

[1269] Take a photo of the food using the device's camera and upload it to the server.

[1270] The server preprocesses the images using OpenCV and uses Keras to recognize the product's characteristics and features from the images.

[1271] Based on the recognition results, a product description is generated using Transformers.

[1272] 2. How AI generates translation work:

[1273] The server uses the generative AI model to translate the generated product description into multiple languages.

[1274] Multilingual support will be provided to enhance services, especially for foreign tourists.

[1275] 3. Measures to improve customer experience:

[1276] The terminal allows customers to complete orders and payments on their smart devices, eliminating the need to wait for waiters or in line at the cash register.

[1277] 4. AI-powered explanations for menu selection questions:

[1278] The device sends any questions that arise as customers consider the menu to the server, which then uses AI to automatically generate explanations.

[1279] 5. How to recommend based on your preferences:

[1280] The server recommends appropriate products based on the customer's past order history and preferences.

[1281] 6. A way to recognize customer emotions and recommend products based on those emotions:

[1282] The customer's facial expressions and tone of voice are analyzed through the device's camera, and the server uses Keras to recognize emotions.

[1283] Based on the recognized sentiment, Transformers are used to recommend appropriate products.

[1284] Examples and prompts

[1285] Examples:

[1286] When a customer wears the smart glasses and looks at the menu, the glasses analyze the customer's facial expressions and display a message saying, "You look happy, so I recommend dessert!"

[1287] Example prompt for a generative AI model:

[1288] "This dish is a delicious looking pasta dish. Please generate a detailed product description."

[1289] In this way, the mode for implementing the invention combines smart devices and AI technology to create restaurant menus and improve customer experience.

[1290] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1291] Step 1:

[1292] The user takes a photo of the food using a smart device (smartphone or smart glasses), and the photo is saved on the device.

[1293] Step 2:

[1294] The device uploads the captured photos to the server, which then uses OpenCV to preprocess the images, specifically resizing and removing noise.

[1295] Step 3:

[1296] The server analyzes the preprocessed image using Keras to recognize the product's characteristics and features. The input is the preprocessed image, and the output is data that indicates the product's characteristics and features.

[1297] Step 4:

[1298] The server generates a product description based on the recognized characteristics and features using Transformers. The input is data that describes the product's characteristics and features, and the output is the generated product description.

[1299] Step 5:

[1300] The server uses a generative AI model to translate the generated product description into multiple languages. The input is the generated product description, and the output is the translated product description into multiple languages.

[1301] Step 6:

[1302] The terminal allows customers to complete orders and payments on their smart devices, eliminating the need to wait in line for a waiter or cashier. The input is the customer's order information, and the output is a notification that the order has been completed.

[1303] Step 7:

[1304] The terminal sends questions that arise as customers consider the menu to the server, which then uses AI to automatically generate explanations. The input is the customer's question, and the output is the generated explanation.

[1305] Step 8:

[1306] The server recommends appropriate products based on the customer's past order history and preferences. The input is the customer's order history and preference data, and the output is the recommended products.

[1307] Step 9:

[1308] The server analyzes the customer's facial expression and tone of voice through the device's camera and uses Keras to recognize emotions. The input is the customer's facial expression and tone of voice data, and the output is the recognized emotion.

[1309] Step 10:

[1310] The server recommends appropriate products based on the recognized emotions using Transformers. The input is the recognized emotion and the output is the recommended product.

[1311] Example 2

[1312] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1313] Today's consumers expect to be able to quickly and accurately understand product descriptions, and they also want information to be provided in multiple languages. They also want personalized product descriptions that reflect the user's emotions. However, conventional systems have difficulty meeting these demands, and improving service for foreign tourists in particular is a challenge.

[1314] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1315] In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for generating translations using artificial intelligence, a means for improving the customer experience, a means for providing explanations using artificial intelligence for questions raised when considering a menu, a means for making recommendations based on preferences, a means for recognizing a user's emotions and generating product descriptions based on those emotions, and a means for translating the generated product descriptions into multiple languages. This allows users to quickly and accurately understand product descriptions, and information can be provided in multiple languages. Furthermore, providing personalized product descriptions based on the user's emotions can improve customer satisfaction.

[1316] A "smart device" is a portable electronic device that has Internet connectivity and can run applications.

[1317] The "menu creation means" is a function for creating menus for restaurants and service businesses using a smart device.

[1318] "Artificial intelligence means for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with a smart device and automatically generates product descriptions based on those images.

[1319] "Means for generating translation work using artificial intelligence" refers to artificial intelligence technology for translating generated product descriptions into multiple languages.

[1320] "Means to improve the customer experience" are functions that improve the convenience and satisfaction of customers when using a service.

[1321] "Means for AI to explain questions when considering a menu" is a function that enables AI to provide appropriate answers to questions that arise when customers are considering a menu.

[1322] "Preference-based recommendation means" is a function for recommending appropriate products and services based on a customer's past choices and preferences.

[1323] "Means for recognizing a user's emotions and generating a product description that corresponds to those emotions" refers to artificial intelligence technology that analyzes emotions from a user's input and behavior and generates a product description that is appropriate for those emotions.

[1324] The "means for translating the generated product description into multiple languages" is a function for translating the generated product description into multiple languages.

[1325] MODE FOR CARRYING OUT THE INVENTION

[1326] This invention is a system that includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images, a means for generating translation work using artificial intelligence, a means for improving the customer experience, a means for explaining questions that arise when considering a menu using artificial intelligence, a means for making recommendations based on preferences, a means for recognizing a user's emotions and generating product descriptions based on those emotions, and a means for translating the generated product descriptions into multiple languages.

[1327] Hardware and software used

[1328] The server generates product descriptions using a generative AI model (e.g., OpenAI's GPT-4). The generated product descriptions are translated into multiple languages ​​using a translation engine (e.g., Google Translate API). To recognize the user's emotions, an emotion engine (e.g., IBM Watson's emotion analysis API) is used. Smart devices are portable electronic devices that have Internet connectivity and can run applications.

[1329] Explanation of program processing

[1330] The server receives text entered by the user through the device and generates a product description using a generative AI model. For example, if the user enters "This product is of high quality and will last a long time," the server generates a product description based on this text.

[1331] The generated product description is then translated into multiple languages ​​using a translation engine, for example, into major tourist languages ​​such as English, Chinese, and Korean. Specifically, the description "This product is of high quality and will last a long time" is translated.

[1332] When a user inputs text or voice data related to their emotions through the device, the server uses an emotion engine to recognize the user's emotions. For example, if a user inputs "I'm feeling depressed today," the server analyzes this text and recognizes the user's emotion as "depressed."

[1333] The server uses a generative AI model to generate a product description based on the user's emotions. For example, if the user is feeling down, the server generates a product description using encouraging words.

[1334] Original description: "This item is high quality and will last a long time."

[1335] Emotional description: "This product is high quality and will brighten your day. It's long-lasting so you can use it forever!"

[1336] Examples of concrete examples and prompts

[1337] As a concrete example, consider the following scenario.

[1338] Scenario 1: Multilingual Translation

[1339] If a user types "This product is of high quality and will last a long time" in Japanese, the server will translate this product description into English, Chinese, and Korean.

[1340] Scenario 2: Emotional product descriptions

[1341] If a user types, "I'm feeling down today," the server uses an emotion engine to recognize the user's emotions and generate an uplifting product description.

[1342] Original description: "This item is high quality and will last a long time."

[1343] Emotional description: "This product is high quality and will brighten your day. It's long-lasting so you can use it forever!"

[1344] Prompt Sentence Examples

[1345] An example of a prompt for a generative AI model is as follows:

[1346] "Please translate this product description in Japanese into English, Chinese, and Korean: 'This product is of high quality and will last a long time.'"

[1347] "Generate a product description that will cheer up users when they're feeling down. The original description is 'This product is high quality and will last a long time.'"

[1348] The above is an embodiment of the present invention.

[1349] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1350] Step 1:

[1351] The user uses the terminal to input text that will be the basis for the product description. For example, the user might input "This product is of high quality and will last a long time." The input text is sent to the server.

[1352] Step 2:

[1353] The server uses a generative AI model to generate a product description based on the text entered by the user. Specifically, the server sends the following prompt to the generative AI model: "Please generate a product description based on the user's input: 'This product is high quality and will last a long time.'" The generative AI model generates a product description based on this prompt and returns it to the server. The output is the generated product description.

[1354] Step 3:

[1355] The server sends the generated product description to the translation engine for translation into multiple languages. Specifically, the server sends the following prompt to the translation engine: "Please translate this product description into English, Chinese, and Korean: 'This product is of high quality and will last a long time.'" The translation engine translates based on this prompt and returns it to the server. The output is a product description translated into multiple languages.

[1356] Step 4:

[1357] The user inputs text or voice data related to their emotions through the terminal. For example, the user inputs "I'm feeling depressed today." The input text or voice data is sent to the server.

[1358] Step 5:

[1359] The server uses the emotion engine to recognize the user's emotion. Specifically, the server sends the following prompt to the emotion engine: "Please analyze the user's input, 'I'm feeling depressed today,' and recognize the emotion." The emotion engine analyzes the emotion based on this prompt and returns it to the server. The output is the recognized emotion.

[1360] Step 6:

[1361] The server uses a generative AI model to generate an emotion-based product description based on the recognized user emotion. Specifically, the server sends the following prompt to the generative AI model: "Generate an uplifting product description for the user when they are feeling down. The original description is 'This product is of high quality and will last a long time.'" The generative AI model generates an emotion-based product description based on this prompt and returns it to the server. The output is an emotion-based product description.

[1362] Step 7:

[1363] The server sends a product description corresponding to the generated emotion to the user's device. The user can check the uplifting product description through the device. For example, if the user inputs "I'm feeling down today," the server will provide the user with a product description such as "This product is high quality and will brighten your days. It lasts a long time, so you can use it for a long time!"

[1364] The above is the specific flow of the program processing of this system.

[1365] (Application example 2)

[1366] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1367] On conventional online shopping sites, product descriptions are often provided in a single language, making them difficult for foreign tourists and multilingual users to understand. Furthermore, product descriptions are not tailored to the user's emotions, making it difficult to stimulate their purchasing motivation. Furthermore, users often spend a lot of time trying to understand the product descriptions, resulting in a poor purchasing experience.

[1368] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1369] In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, an artificial intelligence means for generating translations, a means for improving the customer experience, an artificial intelligence means for explaining questions when considering menus, a means for making recommendations based on preferences, a means for recognizing a user's emotions and generating product descriptions based on those emotions, and a means for translating the generated product descriptions into multiple languages. This makes it possible to provide product descriptions in multiple languages ​​that are based on the user's emotions and are easy to understand for foreign tourists and users who speak multiple languages. It also increases the user's purchasing motivation and improves the purchasing experience.

[1370] A "smart device" is a portable electronic device with internet connectivity, such as a smartphone, tablet, or smart glasses.

[1371] "Menu creation means" refers to functions or software that allow a user to create a menu of products or services using a smart device.

[1372] "Artificial intelligence means for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with a smart device and automatically generates product descriptions based on those images.

[1373] "Means for generating translation work using artificial intelligence" refers to artificial intelligence technology for translating generated product descriptions into multiple languages.

[1374] "Customer experience improvement measures" are functions and software that improve the experience customers have when using products and services.

[1375] "Means for AI to explain questions when considering a menu" is a function in which AI provides appropriate explanations for questions that arise when a user is considering a menu.

[1376] "Means for recommending based on preferences" is a function that recommends appropriate products and services based on the user's preferences and past behavior.

[1377] "Means for recognizing a user's emotions and generating a product description that corresponds to those emotions" refers to artificial intelligence technology that analyzes a user's emotions and generates a product description that is appropriate for those emotions.

[1378] The "means for translating the generated product description into multiple languages" refers to a function or software for translating the generated product description into multiple languages.

[1379] The following describes an embodiment of the present invention. This system includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images, a means for generating translations using artificial intelligence, a means for improving the customer experience, a means for providing explanations using artificial intelligence for questions raised when considering a menu, a means for making recommendations based on preferences, a means for recognizing a user's emotions and generating product descriptions based on those emotions, and a means for translating the generated product descriptions into multiple languages.

[1380] System configuration

[1381] Hardware

[1382] Smart Device: A portable electronic device with internet connectivity, such as a smartphone, tablet, or smart glasses.

[1383] Server: A high-performance computer responsible for data processing and storage.

[1384] software

[1385] Artificial intelligence model: Generates product descriptions using OpenAI's API.

[1386] Translation software: Multilingual translation using Google Translate API.

[1387] Emotion Recognition Software: Recognize user emotions using EmotionRecognizer.

[1388] Data processing and calculation

[1389] 1. Image analysis: A product image taken with a smart device is sent to the server, which uses image analysis algorithms to extract the product's characteristics and features and generate a product description.

[1390] 2. Emotion Recognition: Using the camera and microphone of the smart device to analyze the user's facial expressions and tone of voice, the EmotionRecognizer software recognizes the user's emotions and sends the results to the server.

[1391] 3. Product description generation: The server generates prompt sentences according to the user's emotions and generates product descriptions appropriate to the emotions using OpenAI's API.

[1392] 4. Multilingual Translation: The generated product description is translated into multiple languages ​​using Google Translate API. The translation results are available in major languages ​​such as English, Chinese, and Korean.

[1393] Specific examples

[1394] For example, if a user is feeling depressed, the smart device camera captures the user's facial expression and the EmotionRecognizer recognizes it as "depressed." The server generates the following prompt sentence:

[1395] "Generate a product description that cheers up users when they're feeling down: This product is made of high-quality materials and will last a long time."

[1396] This prompt is sent to OpenAI's API to generate an emotionally appropriate product description, which looks like this:

[1397] "This product was created to brighten your life a little. It's made from high-quality materials and is built to last."

[1398] Next, this product description is translated into multiple languages ​​using the Google Translate API.

[1399] In this way, it becomes possible to provide product descriptions in multiple languages ​​that reflect the user's emotions.

[1400] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1401] Step 1:

[1402] The user takes a photo of the product using a smart device.

[1403] Input: Product image

[1404] Output: Captured image data

[1405] How it works: A user uses a smart device such as a smartphone or tablet to take a picture of a product they are considering purchasing. This image data is then stored on the smart device.

[1406] Step 2:

[1407] The terminal transmits the captured image to the server.

[1408] Input: Captured image data

[1409] Output: Image data sent to the server

[1410] Specific operation: The smart device sends the captured image data to a server via the Internet, using protocols such as HTTP.

[1411] Step 3:

[1412] The server performs image analysis and extracts the product's characteristics and features.

[1413] Input: Submitted image data

[1414] Output: Extracted product characteristics and features

[1415] How it works: The server uses image analysis algorithms to automatically extract product characteristics and features from image data, such as product color, shape, and brand logo.

[1416] Step 4:

[1417] The server generates a product description based on the extracted properties and characteristics.

[1418] Input: Extracted product characteristics and features

[1419] Output: Generated product description

[1420] What it does: The server uses a generative AI model to generate a product description based on the extracted properties and characteristics, such as "This product is made from high-quality materials and is built to last."

[1421] Step 5:

[1422] The device recognizes the user's emotions.

[1423] Input: User facial expressions and tone of voice

[1424] Output: Perceived user emotion

[1425] How it works: Using the smart device's camera and microphone, the EmotionRecognizer software captures the user's facial expressions and tone of voice, and then recognizes their emotions, such as "depressed" or "happy."

[1426] Step 6:

[1427] The server generates a prompt sentence according to the user's emotion.

[1428] Input: Recognized user sentiment, generated product description

[1429] Output: Emotion-based prompt

[1430] Specific operation: Based on the recognized user emotion, the server generates a prompt sentence to send to the generative AI model. For example, a prompt sentence such as "Please generate a product description that will cheer up the user when they are feeling down: This product is made of high-quality materials and will last a long time."

[1431] Step 7:

[1432] The server uses a generative AI model to generate emotionally appropriate product descriptions.

[1433] Input: Emotion-specific prompt

[1434] Output: Emotionally relevant product description

[1435] What it does: The server uses a generative AI model (e.g., OpenAI API) to generate an emotionally appropriate product description based on the prompt. For example, it might generate a description like, "This product was made to make your life a little brighter. It's made from high-quality materials and is built to last."

[1436] Step 8:

[1437] The server translates the generated product description into multiple languages.

[1438] Input: Generated product description

[1439] Output: Product description translated into multiple languages

[1440] What it does: The server uses the Google Translate API to translate the generated product description into multiple languages, such as English, Chinese, and Korean. For example, the translation results might be something like, "This product is made to brighten your life a little. It is made of high-quality materials and lasts long."

[1441] Step 9:

[1442] The server transmits product descriptions translated into multiple languages ​​to the terminal.

[1443] Input: Product description translated into multiple languages

[1444] Output: Translation results sent to the device

[1445] Specific operation: The server sends the product description translated into multiple languages ​​to the smart device via the Internet, using protocols such as HTTP.

[1446] Step 10:

[1447] The device displays product descriptions translated into multiple languages ​​to the user.

[1448] Input: Translation result sent to the device

[1449] Output: Multilingual product description displayed to the user

[1450] Specific operation: The smart device receives the translated product description and displays it to the user, allowing the user to understand the product description in multiple languages ​​according to their emotions.

[1451] Example 3

[1452] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1453] Conventional customer experience improvement systems are inefficient because they often require manual tasks such as creating menus, explaining products, and translating them. Furthermore, they lack the ability to provide answers to questions customers have when considering menus, or to recommend products based on their preferences, which can lead to a decline in customer satisfaction. Furthermore, they lack the ability to recognize customer emotions and provide services in response, which hinders the quality of the customer experience.

[1454] The identification process by the identification processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means. In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, an artificial intelligence means for generating translation work, a customer experience improvement means for reducing waiting time, an artificial intelligence means for explaining questions when considering a menu, a means for recommending products according to preferences, and a means for recognizing user emotions and providing services according to those emotions. This makes it possible to quickly answer questions when customers are considering a menu and recommend products according to preferences, and further to significantly improve customer satisfaction by providing services according to customer emotions.

[1455] A "smart device" is a portable electronic device with internet connectivity, such as a smartphone or tablet.

[1456] The "menu creation means" is a function for creating menus for restaurants and service businesses using a smart device.

[1457] "Artificial intelligence means for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with a smart device and automatically generates product descriptions based on those images.

[1458] "Means for generating translation work using artificial intelligence" refers to a function that uses artificial intelligence to automatically translate text or speech.

[1459] "Customer experience improvement measures to reduce waiting time" are functions that shorten waiting times when customers use services and provide a smoother experience.

[1460] "A means for AI to explain questions when considering a menu" is a function in which AI provides appropriate answers to questions that arise when customers are considering a menu.

[1461] "Means for recommending products based on preferences" is a function that uses artificial intelligence to recommend appropriate products based on a customer's preferences and past choices.

[1462] "Means for recognizing the user's emotions and providing services according to those emotions" refers to a function that recognizes the user's emotions from their facial expressions and voice and provides appropriate services according to those emotions.

[1463] This invention is a system for improving customer experience, and includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images, a means for generating translation work using artificial intelligence, a customer experience improvement means for reducing waiting time, a means for explaining questions when considering a menu using artificial intelligence, a means for recommending products according to preferences, and a means for recognizing a user's emotions and providing services according to those emotions.

[1464] Hardware and software used

[1465] Hardware

[1466] Smart Device: A portable electronic device with internet connectivity, such as a smartphone or tablet.

[1467] Server: A high-performance server (e.g., a server equipped with an NVIDIA GPU).

[1468] software

[1469] Generative AI models: Using GPT-4 as an example.

[1470] Emotion Recognition Engine: Using Affectiva SDK as an example.

[1471] Program processing explanation

[1472] The server receives the information sent from the smart device and analyzes it using the generative AI model. For example, if a user inputs "I like spicy food," the generative AI model generates a list of spicy dishes and the server sends that list to the smart device. The smart device then displays the received list to the user.

[1473] When a user asks, "What are the ingredients in this dish?", the server uses the generative AI model to generate ingredient information and sends it to the smart device, which then displays the received ingredient information to the user.

[1474] Furthermore, the smart device uses its built-in camera and microphone to capture the user's facial expressions and voice, and uses an emotion recognition engine to recognize the user's emotions. For example, if the user shows anger, the smart device sends that information to the server. The server then uses a generative AI model to generate an apology and sends it to the smart device. The smart device then displays the received apology to the user.

[1475] Examples of concrete examples and prompts

[1476] Example 1: User enters "I like spicy food"

[1477] A user opens an app on their smartphone and types in "I like spicy food."

[1478] The server generates a list of spicy dishes using a generative AI model.

[1479] The server sends the generated list to the smart device.

[1480] A smart device displays a list of spicy dishes to the user.

[1481] Example prompt: "List dishes you would recommend for a customer who likes spicy food."

[1482] Example 2: A user asks, "What are the ingredients in this dish?"

[1483] A user opens the app on their tablet and asks, "What are the ingredients in this dish?"

[1484] The server generates ingredient information using the generative AI model.

[1485] The ingredient information generated by the server is sent to the smart device.

[1486] The smart device displays ingredient information to the user.

[1487] Example prompt: "What are the ingredients in this dish?"

[1488] Example 3: User shows angry facial expression

[1489] The user shows an angry expression while using the smartphone.

[1490] The smart device uses a camera to capture the user's facial expressions and analyzes them using an emotion recognition engine.

[1491] The smart device transmits the anger emotion information it recognizes to a server.

[1492] The server generates an apology using a generative AI model.

[1493] The server generates an apology and sends it to the smart device.

[1494] The smart device displays an apology to the user.

[1495] Example prompt: "Generate an apology for when the user is angry."

[1496] In this way, the server, smart device, and user work together to operate the system, improving the customer experience. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[1497] Step 1:

[1498] The user inputs information through a device. Using a device such as a smartphone or tablet, the user inputs questions and preferences into the system. For example, the user might input "I like spicy food." The input data is saved in text format on the device.

[1499] Step 2:

[1500] The terminal sends the input information to the server. The terminal sends the information entered by the user to the server via the Internet. The input data is sent to the server in text format.

[1501] Step 3:

[1502] The server analyzes the input information using a generative AI model. The server passes the received input information to a generative AI model (e.g., GPT-4) for analysis. The generative AI model generates appropriate answers or recommendations based on the input information. For example, for the information "I like spicy food," it generates a list of spicy dishes. The input data is in text format, and the output data is in list format.

[1503] Step 4:

[1504] The server generates appropriate answers and recommendations based on the analysis results. The server generates answers and recommendations for the user based on the analysis results obtained from the generative AI model. For example, it generates a list of spicy dishes. The output data is in list format.

[1505] Step 5:

[1506] The server sends the generated answers and recommendations to the terminal. The server then sends the generated answers and recommendations to the terminal via the Internet. The output data is sent to the terminal in list format.

[1507] Step 6:

[1508] The terminal displays answers and recommendations to the user. The terminal displays answers and recommendations received from the server to the user. The user can check the displayed information. The output data is displayed in list format.

[1509] Step 7:

[1510] The device recognizes the user's emotions using an emotion engine. The device uses the built-in camera and microphone to capture the user's facial expressions and voice, and then uses an emotion engine (e.g., Affectiva SDK) to recognize the user's emotions. The input data is in the form of images and audio, and the output data is in the form of emotion information.

[1511] Step 8:

[1512] The terminal transmits the recognized emotion information to the server. The terminal transmits the recognized emotion information to the server via the Internet. The input data is in emotion information format, and the output data is in emotion information format.

[1513] Step 9:

[1514] The server generates an appropriate service based on the emotional information. The server uses a generative AI model to generate an appropriate service based on the received emotional information. For example, if the user shows an angry expression, it generates an apology. The input data is in the form of emotional information, and the output data is in text format.

[1515] Step 10:

[1516] The server transmits the generated service to the terminal. The server transmits the generated service to the terminal via the Internet. The output data is transmitted to the terminal in text format.

[1517] Step 11:

[1518] The terminal provides the service to the user. The terminal provides the service received from the server to the user. For example, it displays an apology. The output data is displayed in text format.

[1519] (Application example 3)

[1520] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1521] Conventional food delivery services have faced challenges, such as users having to spend a lot of time selecting menu items and difficulty in providing services that respond to users' preferences and emotions. Furthermore, multilingual support for foreign tourists was insufficient, leaving a need for an improved user experience. There is a need to solve these challenges and provide a more comfortable and personalized service.

[1522] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1523] In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, an artificial intelligence means for generating translation tasks, a customer experience improvement means for reducing waiting time, an artificial intelligence means for explaining questions when considering menu items, a means for recommending products based on preferences, and an emotion recognition means for recognizing user emotions and providing services based on those emotions. This allows users to efficiently select menus and receive personalized services based on their individual preferences and emotions. Multilingual support also enhances services for foreign tourists.

[1524] A "smart device" is a portable electronic device with advanced computing capabilities, such as a smartphone or tablet.

[1525] A "menu generator" is a method or device for generating a list of products or services from which a user can choose.

[1526] "Artificial intelligence means for generating product descriptions from images" refers to an artificial intelligence system that uses image recognition technology to automatically analyze the characteristics and features of a product and generates a product description based on that analysis.

[1527] "Artificial intelligence-generated means of translation work" refers to an artificial intelligence system that automatically translates text or speech between different languages.

[1528] "Customer experience improvement means" refers to a method or device for improving the convenience and satisfaction of users when using a service.

[1529] "A means for AI to explain questions when considering a menu" is a system in which AI provides appropriate answers to questions that arise when a user is choosing a menu.

[1530] A "means for recommending products based on preferences" is a system that recommends appropriate products and services based on the user's past choices and input information.

[1531] "Emotion recognition means" is a system that analyzes the user's emotions from facial expressions, tone of voice, etc., and responds accordingly.

[1532] A system for implementing this invention includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images, an artificial intelligence means for generating translation work, a customer experience improvement means for reducing waiting time, an artificial intelligence means for explaining questions when considering a menu, a means for recommending products according to preferences, and an emotion recognition means for recognizing a user's emotions and providing services according to those emotions.

[1533] System Program

[1534] The program of this system performs the following processing.

[1535] Hardware and Software

[1536] Hardware:

[1537] Smart devices (smartphones, tablets, etc.)

[1538] Camera (to capture the user's facial expressions)

[1539] Computer (processing device for executing programs)

[1540] software:

[1541] OpenCV (image processing library)

[1542] Keras (deep learning library)

[1543] Transformers (a library of generative AI models)

[1544] Data processing and calculation

[1545] 1. Menu creation method:

[1546] The server generates a list of products and services that the user can select using the smart device.

[1547] 2. Artificial intelligence means for generating product descriptions from images:

[1548] The server analyzes images taken by the smart device's camera and automatically recognizes the product's characteristics and features.

[1549] A product description is generated based on the recognized information.

[1550] 3. AI-generated translation tasks:

[1551] The server automatically translates text and audio between different languages.

[1552] The service will be multilingual, particularly to enhance services for foreign tourists.

[1553] 4. Customer experience improvement measures:

[1554] The server reduces the waiting time when the user uses the service, improving convenience.

[1555] 5. How AI can explain questions you might have when considering a menu:

[1556] The server uses generative AI models to provide appropriate answers to questions that arise when users choose from the menu.

[1557] 6. How to recommend products based on your preferences:

[1558] The server recommends appropriate products and services based on the user's past choices and input information.

[1559] 7. Emotion recognition means:

[1560] The server analyzes the user's facial expressions and tone of voice captured by the camera to recognize their emotions.

[1561] Respond appropriately depending on the emotion recognized.

[1562] Specific examples

[1563] When a user inputs "I like spicy food," the server uses a generative AI model to recommend spicy dishes.

[1564] When a user asks, "What are the ingredients in this dish?", the server uses a generative AI model to provide ingredient information.

[1565] If the user shows an angry expression, the server uses emotion recognition means to apologize, saying, "Sorry. Is there a problem?"

[1566] Prompt Sentence Examples

[1567] "I like spicy food. What would you recommend?"

[1568] "What are the ingredients in this dish?"

[1569] "How do you respond when a user displays anger?"

[1570] In this way, users can efficiently select menus and receive personalized services that reflect their individual preferences and feelings. Multilingual support also enhances services for foreign tourists.

[1571] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1572] Step 1:

[1573] The user opens a menu using a smart device.

[1574] Input: A request to display a menu via user operation.

[1575] Data processing: The server retrieves menu information from the database and sends it to the smart device.

[1576] Output: A menu will be displayed on the smart device.

[1577] Step 2:

[1578] The user takes a photo of the food using the camera on their smart device.

[1579] Input: A user-taken image of a dish.

[1580] Data processing: The server receives the images and analyzes them using an image recognition algorithm (OpenCV).

[1581] Output: As a result of the analysis, the characteristics and features of the food are extracted.

[1582] Step 3:

[1583] The server generates a product description from the image.

[1584] Input: Data on food characteristics and properties.

[1585] Data processing: The server uses a generative AI model to generate a product description based on the product's characteristics and features.

[1586] Output: Product description text is generated and sent to the smart device.

[1587] Step 4:

[1588] Users input questions when reviewing the menu.

[1589] Input: The question typed by the user (e.g., "What are the ingredients in this dish?").

[1590] Data processing: The server generates answers to questions using a generative AI model.

[1591] Output: Answer text is generated and displayed on the smart device.

[1592] Step 5:

[1593] The server recommends products based on the user's preferences.

[1594] Input: Previous choices or inputs made by the user (e.g., "I like spicy food").

[1595] Data processing: The server uses a generative AI model to recommend products based on the user's preferences.

[1596] Output: A list of recommended products will be displayed on the smart device.

[1597] Step 6:

[1598] The user captures their facial expression using the camera on their smart device.

[1599] Input: An image of the user's facial expression.

[1600] Data processing: The server analyzes emotions using a facial expression recognition algorithm (Keras).

[1601] Output: The analysis results in the generation of user emotion data.

[1602] Step 7:

[1603] The server provides a service according to the user's emotions.

[1604] Input: User emotion data (e.g., anger emotion).

[1605] Data processing: The server generates an appropriate response based on the emotion data (e.g., an apology message).

[1606] Output: The corresponding message will be displayed on the smart device.

[1607] Step 8:

[1608] The server performs the translation work.

[1609] Input: Text or voice data entered by the user.

[1610] Data processing: The server uses a translation algorithm to translate the input data into multiple languages.

[1611] Output: The translated text and audio data will be displayed on your smart device.

[1612] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1613] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1614] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1615] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1616] [Third embodiment]

[1617] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1618] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1619] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1620] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1621] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1622] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1623] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1624] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1625] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1626] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1627] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1628] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1629] "Example 1"

[1630] In one embodiment of the present invention, restaurant operators create menus using smartphones. Specifically, they upload photos of products taken with the smartphone camera, and AI automatically recognizes the product's characteristics and features from the photos. Then, it generates product descriptions based on the recognition results. These product descriptions are displayed to customers on the restaurant's website or application.

[1631] "Example 2"

[1632] Furthermore, in this embodiment of the present invention, AI generates translation tasks. Specifically, it translates the generated product descriptions into multiple languages. This makes the product descriptions easier to understand for foreign tourists. For example, it is possible to translate into major tourist languages ​​such as English, Chinese, and Korean.

[1633] "Example 3"

[1634] This embodiment of the present invention also improves the customer experience. Specifically, when a customer is considering a menu, the AI ​​will explain their question and recommend products based on their preferences. For example, if a customer enters information such as "I like spicy food," the AI ​​will recommend spicy dishes. Also, if a customer asks a question such as "What are the ingredients in this dish?", the AI ​​will provide an answer to that question.

[1635] The processing flow of each embodiment will be described below.

[1636] "Example 1"

[1637] Step 1: The restaurant manager takes a photo of the product using a smartphone camera.

[1638] Step 2: Upload the photos you have taken to the system.

[1639] Step 3: The AI ​​in the system automatically recognizes the product's characteristics and features from the photo.

[1640] Step 4: The AI ​​generates a product description based on the recognition results.

[1641] Step 5: The generated product description is displayed to customers on the restaurant's website or application.

[1642] "Example 2"

[1643] Step 1: Get the AI-generated product description.

[1644] Step 2: AI translates the product description into multiple languages.

[1645] Step 3: The translated product description will be displayed to foreign tourists on the restaurant's website or app.

[1646] "Example 3"

[1647] Step 1: As customers consider the menu, they input their questions into the AI.

[1648] Step 2: The AI ​​generates an answer to that question.

[1649] Step 3: AI recommends products based on customer preferences.

[1650] Step 4: The AI-generated answers and recommendations are displayed to the customer.

[1651] Example 1

[1652] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1653] When restaurant operators create menus, it is time-consuming to take photos of the products, recognize the characteristics and features of the products from the photos, and generate product descriptions. Furthermore, there is a lack of multilingual support for foreign tourists and ways to improve the customer experience. This has led to a demand for more efficient menu creation and improved customer satisfaction.

[1654] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1655] In this invention, the server includes a menu creation means using a smartphone, an artificial intelligence means for generating product descriptions from smartphone photos, a means for generating translations by artificial intelligence, a means for improving the customer experience by eliminating the need to wait in line for waiters or cashiers, a means for providing artificial intelligence to answer questions when considering menu items, a means for making recommendations based on preferences, a means for uploading product photos taken with a smartphone camera and having the artificial intelligence automatically recognize the characteristics and features of the products from the photos, and a means for generating product descriptions based on the recognition results, thereby making it possible to improve the efficiency of menu creation and customer satisfaction.

[1656] A "smartphone" is a multi-functional mobile device that can connect to the Internet and use applications in addition to the functions of a mobile phone.

[1657] "Menu creation means" refers to the methods and tools used by restaurant operators to create menus, particularly those using smartphones.

[1658] "Artificial intelligence means" refers to technologies that use machine learning and data analysis to automatically perform specific tasks.

[1659] "Means for generating translation work using artificial intelligence" refers to methods and technologies for translating text into multiple languages ​​using artificial intelligence.

[1660] "Measures to improve customer experience" refers to methods and tools for improving the convenience and satisfaction of customers when using services.

[1661] "Means for AI to explain questions when considering a menu" refers to methods and technologies in which AI automatically provides answers to questions that arise when customers are considering a menu.

[1662] "Preference-based recommendation methods" refers to methods and technologies that recommend appropriate products and services based on a customer's preferences and past choices.

[1663] "Means for automatically recognizing product characteristics and features" refers to methods and technologies that use artificial intelligence to automatically extract product characteristics and features from photographs and data.

[1664] "Means for generating product descriptions" refers to methods or techniques for creating product descriptions in natural language based on recognized product characteristics and features.

[1665] This invention is a system that allows restaurant operators to create menus using smartphones. Specifically, a photo of a product taken with the smartphone camera is uploaded, and artificial intelligence (AI) automatically recognizes the product's characteristics and features from the photo and generates a product description based on the recognition results. This product description is displayed to customers on the restaurant's website or application.

[1666] Hardware and software used

[1667] Smartphone: A device for taking and uploading photos.

[1668] Server: Processes data and runs AI models.

[1669] Artificial intelligence models: Image recognition and natural language generation models based on TensorFlow and PyTorch (e.g., GPT-3, BERT).

[1670] Data processing and calculation

[1671] 1. The user takes a photo of the product with their smartphone

[1672] The user opens the camera app on their smartphone and takes a photo of a product, for example, a new dessert called "chocolate cake."

[1673] 2. The user uploads a photo to the system

[1674] The user opens the dedicated application, selects the photos they have taken, and presses the upload button, which sends the photos to the server via the Internet.

[1675] 3. The server receives the photo and inputs it into the AI ​​model

[1676] The server receives the uploaded photos and feeds them into an AI model for image processing, which is based on TensorFlow and PyTorch.

[1677] 4. The server uses an AI model to recognize the product's characteristics and features.

[1678] The server uses AI models to recognize product characteristics and features from the photo, such as the type of dessert, key ingredients, and visual features.

[1679] 5. The server generates a product description based on the recognition results.

[1680] The server generates a product description based on the AI ​​recognition results. This product description is created using natural language generation technology, using software such as GPT-3 and BERT.

[1681] 6. The server displays the product description on the website or application.

[1682] The server then displays the generated product description on the restaurant's website or application, where customers can view it.

[1683] Specific examples

[1684] A user takes a photo of a new dessert, "Chocolate Cake," and uploads it to the system. The server receives the photo and uses an AI model to recognize features such as "Chocolate Cake," "Cream Topping," and "Berry Decoration." The server then uses GPT-3 to generate a product description: "This chocolate cake is characterized by its rich chocolate and creamy topping. The berry decoration makes it a visually stunning dish." This is then displayed on the website.

[1685] Prompt Sentence Examples

[1686] Upload a photo of your new dessert and our AI will recognize its properties and characteristics and generate a product description.

[1687] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1688] Step 1:

[1689] A user takes a photo of an item with their smartphone.

[1690] Input: Product (e.g. Chocolate Cake)

[1691] Output: Product photo

[1692] Specific action: The user opens the camera app on their smartphone and takes a photo of a product, for example, a new dessert called "Chocolate Cake."

[1693] Step 2:

[1694] A user uploads a photo to the system.

[1695] Input: Product photo taken

[1696] Output: Photo data sent to the server

[1697] How it works: The user opens the application, selects the photo they have taken, and presses the upload button, which sends the photo to the server via the Internet.

[1698] Step 3:

[1699] The server receives the photos and inputs them into the AI ​​model.

[1700] Input: Photo data sent to the server

[1701] Output: Photo data fed into the AI ​​model

[1702] Specific operation: The server receives an HTTP request, temporarily stores the photo data, and inputs it into a TensorFlow or PyTorch model.

[1703] Step 4:

[1704] The server uses an AI model to recognize the product's characteristics and features.

[1705] Input: Photo data fed into the AI ​​model

[1706] Output: Recognized product characteristics or features (e.g., chocolate cake, cream topping, berry decoration)

[1707] What it does: The server runs an AI model to extract product characteristics and features from the photo, such as the type of dessert, its main ingredients, and visual features.

[1708] Step 5:

[1709] The server generates a product description based on the recognition results.

[1710] Input: Recognized product characteristics or features

[1711] Output: Generated product description (e.g., "This chocolate cake features rich chocolate and a creamy topping. The berry decoration makes it visually appealing.")

[1712] Specific operation: The server uses GPT-3 or BERT to generate a product description using the recognition results as input.

[1713] Step 6:

[1714] The server displays the product description on the website or application.

[1715] Input: Generated product description

[1716] Output: Product description displayed on your website or application

[1717] What happens: The server saves the product description in the website database and sends it to the front-end for display, where customers can view it.

[1718] (Application example 1)

[1719] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1720] Traditionally, creating menus in restaurants was often done manually, which was time-consuming and labor-intensive, and made it difficult to fully convey the appeal of the products. Other issues included a lack of multilingual support for foreign tourists and a lack of recommendation functions tailored to customer preferences. Furthermore, there were limited ways to improve the in-store customer experience, with customers having to wait in line for wait staff or cashiers, and there was a need to improve customer satisfaction.

[1721] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1722] In this invention, the server includes a menu creation means using a smartphone, an AI means for generating product descriptions from smartphone photos, an AI means for generating translations, a means for improving the customer experience by eliminating the need to wait in line for waiters or cashiers, an AI means for explaining questions when considering a menu, a means for making recommendations based on preferences, a means for recognizing product characteristics and features using an image recognition model, and a means for generating product descriptions based on the recognized features using a natural language generation model. This enables more efficient menu creation and the automatic generation of attractive product descriptions, enabling multilingual support, an improved customer experience, and recommendations based on individual preferences.

[1723] A "menu creation means using a smartphone" is a means for efficiently creating a restaurant menu using a smartphone's camera and applications.

[1724] "AI means for generating product descriptions from smartphone photos" refers to an artificial intelligence means for analyzing product photos taken with a smartphone, recognizing their characteristics and features, and automatically generating product descriptions.

[1725] "Means for AI-generated translation work" refers to a method for using artificial intelligence to translate product descriptions and menu contents into multiple languages.

[1726] "Measures to improve the customer experience by eliminating the need to wait in line for waiters or cashiers" refers to measures that allow customers to receive service smoothly without having to wait in line for waiters or cashiers.

[1727] "Means for AI to explain questions when considering a menu" refers to a means by which artificial intelligence automatically provides answers to questions that arise when customers are considering a menu.

[1728] "Means for recommendation based on preferences" refers to means for recommending appropriate products and menus based on a customer's past choices and preferences.

[1729] "Means for recognizing the characteristics and features of a product using an image recognition model" refers to means for automatically recognizing the characteristics and features of a product from a photograph of the product using image recognition technology.

[1730] "Means for generating product descriptions based on features recognized using a natural language generation model" refers to means for automatically generating attractive product descriptions using natural language generation technology based on recognized product characteristics and features.

[1731] A system for implementing this invention includes a menu creation means using a smartphone, an AI means for generating product descriptions from smartphone photos, a means for generating translation work using AI, a means for improving the customer experience by eliminating the need to wait in line for waiters or cashiers, a means for AI to explain questions that arise when considering a menu, a means for making recommendations based on preferences, a means for recognizing product characteristics and features using an image recognition model, and a means for generating product descriptions based on the features recognized using a natural language generation model.

[1732] System Program

[1733] The server implements the system using the following hardware and software.

[1734] Hardware:

[1735] Smartphone (with camera)

[1736] Server (for hosting AI models)

[1737] software:

[1738] TensorFlow: A library for image recognition

[1739] OpenAI GPT-3: An API for Natural Language Generation

[1740] Processing Description

[1741] Image Recognition:

[1742] Users take photos of dishes with their smartphone camera and upload them to the server through the application. The server then uses TensorFlow's ResNet50 model to recognize the characteristics and features of the dishes from the images. For example, features such as "chicken curry, spicy, tomato-based" can be recognized.

[1743] Natural language generation:

[1744] The server converts the recognized features into text and sends it to OpenAI GPT-3 as a prompt. An example of a prompt is "Generate an appealing product description for a dish with the following characteristics: chicken curry, spicy, tomato-based." GPT-3 generates an appealing product description based on this prompt. For example, it generates a product description such as "Spicy chicken curry features juicy chicken simmered in a tomato-based sauce. The fragrant spices will whet your appetite."

[1745] Translation work:

[1746] The generated product description can be translated into multiple languages ​​as needed. The server uses AI to translate into multiple languages, such as English and Chinese.

[1747] Improved customer experience:

[1748] Customers can use their smartphones to select menu items and place orders in the store, reducing waiting time for waiters and cashiers and enabling smoother service.

[1749] Question explanation and recommendations:

[1750] AI automatically answers questions that arise as customers consider menu items, and recommends appropriate products and menu items based on customers' past choices and preferences.

[1751] This makes it possible to create menus more efficiently and automatically generate compelling product descriptions, enabling multilingual support, improved customer experience, and personalized recommendations.

[1752] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1753] Step 1:

[1754] A user takes a photo of a dish with the camera on their smartphone and uploads it to the server through the application. The input is the photo of the dish, and the output is the image data sent to the server.

[1755] Step 2:

[1756] The server uses TensorFlow's ResNet50 model to recognize the characteristics and features of food from uploaded image data. The input is the image data, and the output is a list of recognized characteristics and features. Specifically, the server preprocesses the image data, inputs it into the model, and obtains the prediction results.

[1757] Step 3:

[1758] The server converts the recognized traits and features into a string and sends it to OpenAI GPT-3 as a prompt. The input is a list of traits and features, and the output is a prompt. The specific operation is to format the traits and features and generate a prompt.

[1759] Step 4:

[1760] The server receives the response from GPT-3 and generates an attractive product description. The input is the prompt sentence, and the output is the generated product description. Specifically, it calls the GPT-3 API, parses the response, and obtains the product description.

[1761] Step 5:

[1762] The server translates the generated product description into multiple languages ​​as needed. The input is the product description, and the output is the translated product description. Specifically, it calls the translation API to translate into multiple languages.

[1763] Step 6:

[1764] Users use their smartphones to select menu items and place orders in the store. The input is the generated product description, and the output is the user's order information. Specifically, the application displays the product description and the user selects the order.

[1765] Step 7:

[1766] The server uses AI to automatically provide answers to questions that arise when the user is considering the menu. The input is the user's question, and the output is the answer provided by the AI. Specifically, the server analyzes the question and generates an appropriate answer.

[1767] Step 8:

[1768] The server recommends appropriate products and menus based on the user's past selections and preferences. The input is the user's past selection data, and the output is the recommended products and menus. Specifically, the server analyzes the past selection data and uses a recommendation algorithm to recommend products.

[1769] Example 2

[1770] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1771] In the previous system, translation of product descriptions into multiple languages ​​was often done manually, which was a time-consuming and labor-intensive process. There was also the issue of insufficient service for foreign tourists, making product descriptions difficult to understand. Furthermore, there was a risk of customer satisfaction declining because there was insufficient effort to improve the customer experience or resolve questions when considering menu items.

[1772] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1773] In this invention, the server includes a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images on the smart device, a means for generating translations using the artificial intelligence, a means for improving the customer experience, a means for the artificial intelligence to explain questions that arise when considering a menu, a means for making recommendations based on preferences, a means for translating product descriptions into multiple languages, a means for generating prompt sentences using a generative AI model, a means for inputting the prompt sentences and product descriptions into the generative AI model and obtaining the translation results, a means for saving the translation results in a database, a means for transmitting the translation results to a terminal, and a means for the terminal to display the translation results. This automates the translation of product descriptions into multiple languages, improves service for foreign tourists, and enables an improved customer experience and resolution of questions that arise when considering a menu.

[1774] A "smart device" is a portable electronic device such as a smartphone or tablet that has advanced computing power and communication functions.

[1775] "Menu creation means" refers to functions or applications for creating menus for restaurants or retail stores using smart devices.

[1776] "Artificial intelligence means for generating product descriptions from images" refers to artificial intelligence technology that analyzes images taken with a smart device and automatically generates product descriptions based on those images.

[1777] "Means for generating translation work using artificial intelligence" refers to a function that uses artificial intelligence to translate text data into other languages.

[1778] "Means to improve the customer experience" are functions and methods for improving the convenience and satisfaction of customers when using products and services.

[1779] "A means for AI to explain questions when considering a menu" is a function in which AI automatically provides answers to questions or queries that arise when customers are considering a menu.

[1780] "Preference-based recommendations" is a feature that recommends appropriate products and services based on a customer's past choices and preferences.

[1781] "Means for translating product descriptions into multiple languages" is a function for translating product descriptions into multiple languages.

[1782] A "generative AI model" is an artificial intelligence model that has been trained to perform tasks such as text generation or translation.

[1783] A "prompt" is an instruction entered into a generative AI model to make it perform a specific task.

[1784] "Means for saving translation results in a database" refers to a function for saving translation results generated by a generative AI model in a database.

[1785] The "means for transmitting the translation result to the terminal" is a function for transmitting the translation result stored in the database to the user's terminal.

[1786] The "means for the terminal to display the translation result" is a function that displays the translation result received by the user's terminal on the screen.

[1787] This invention is a system including a menu creation means using a smart device, an artificial intelligence means for generating product descriptions from images, a means for generating translation work using artificial intelligence, a means for improving customer experience, a means for artificial intelligence to explain questions when considering menus, a means for making recommendations based on preferences, a means for translating product descriptions into multiple languages, a means for generating prompt sentences using a generative AI model, a means for inputting the prompt sentences and product descriptions into the generative AI model and obtaining the translation results, a means for saving the translation results in a database, a means for transmitting the translation results to a terminal, and a means for the terminal to display the translation results.

[1788] Hardware and software used

[1789] Hardware:

[1790] Smart devices (smartphones, tablets, etc.)

[1791] Server (a computer with high-performance computing capabilities)

[1792] software:

[1793] Generative AI models (e.g., OpenAI's GPT-4)

[1794] Database Management Systems

[1795] Communication protocol (e.g. HTTP / HTTPS)

[1796] Data processing and calculation

[1797] Generate product description:

[1798] A user takes a picture of a product using a smart device and sends the image to a server, which then uses artificial intelligence means to generate a product description from the image. Specifically, the server uses image analysis algorithms to recognize the product's characteristics and features and generates a product description based on the image.

[1799] Generate prompt statement:

[1800] The server uses the generative AI model to generate a prompt sentence, which specifies the language to be translated (e.g., English, Chinese, or Korean).

[1801] Execution of translation work:

[1802] The server inputs the generated prompt and product description into the generative AI model and obtains the translation results. The generative AI model performs multilingual translation based on the input text.

[1803] Saving and sending translation results:

[1804] The server stores the translation results in a database, which is then sent to the user's device, where they are displayed.

[1805] Specific examples

[1806] Examples:

[1807] A user enters a product description in Japanese: "This product is made with high-quality materials."

[1808] The server generates the prompt "Translate the following product description into English: This product is made from high-quality materials."

[1809] The server inputs the prompt and product description into the generated AI model and obtains the English translation result, "This product uses high-quality materials."

[1810] The server stores the translation results in a database and sends them to the terminal.

[1811] The device displays the translation results for the user to confirm.

[1812] In this way, the system enables the server, terminals, and users to work together to translate product descriptions into multiple languages.

[1813] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1814] Step 1:

[1815] The user enters a product description.

[1816] The user inputs a product description in text format using a smart device, and the input product description is sent from the smart device to the server.

[1817] Input: Product description text

[1818] Output: Product description text sent to the server

[1819] Step 2:

[1820] The server receives the product description.

[1821] The server receives the product description text sent by the user and temporarily stores it in a memory.

[1822] Input: Product description text submitted by the user

[1823] Output: Product description text stored in memory

[1824] Step 3:

[1825] The server generates a prompt.

[1826] The server generates a prompt sentence by specifying the language to be translated. For example, if translation into English, Chinese, and Korean is required, the server generates a prompt sentence corresponding to each language.

[1827] Input: Product description text, target language

[1828] Output: Generated prompt statement

[1829] Step 4:

[1830] The server inputs a prompt and product description into the generative AI model.

[1831] The server inputs the generated prompt and product description text into the generative AI model, which then translates the input text.

[1832] Input: prompt text, product description text

[1833] Output: The data fed into the generative AI model

[1834] Step 5:

[1835] The server retrieves the translation results.

[1836] The server obtains translation results from the generative AI model, which are then separated into separate languages.

[1837] Input: Data fed into a generative AI model

[1838] Output: Translation result text

[1839] Step 6:

[1840] The server stores the translation results in a database.

[1841] The server saves the translation results in a database, associating the original Japanese product description with the translation results.

[1842] Input: Translation result text, original product description text

[1843] Output: Translation results stored in a database

[1844] Step 7:

[1845] The server sends the translation results to the terminal.

[1846] The server then transmits the saved translation results to the user's device, providing the data in a format that the user can access.

[1847] Input: Translation results stored in the database

[1848] Output: Translation results sent to the device

[1849] Step 8:

[1850] The device will display the translation results.

[1851] The terminal receives the translation results sent from the server and displays them to the user, who can then check the displayed translation results and make corrections or retranslations as necessary.

[1852] Input: Translation results sent from the server

[1853] Output: Translation results displayed on the device

[1854] (Application example 2)

[1855] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1856] In modern brick-and-mortar stores, foreign tourists face language barriers when purchasing products, making it difficult for them to understand product descriptions. Visually impaired people and those with limited literacy skills also face limited options for understanding product descriptions. This can lead to a poor customer experience and potentially impact sales.

[1857] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a menu creation means using a smartphone, an artificial intelligence means for generating product descriptions from photos on the smartphone, a means for generating translation work by artificial intelligence, a means for improving the customer experience, a means for explaining questions when considering menus by artificial intelligence, a means for making recommendations according to preferences, and a means for translating product descriptions into multiple languages ​​and displaying and playing aloud the product descriptions in the user's selected language. This makes it possible to make product descriptions easier to understand for foreign tourists, visually impaired people, and people with difficulty reading and writing.

[1858] "Menu creation means using a smartphone" refers to a means for creating a menu using a smartphone.

[1859] "Artificial intelligence means for generating product descriptions from photos taken with a smartphone" refers to a means that uses artificial intelligence technology to automatically generate product descriptions from photos taken with a smartphone.

[1860] "Means for generating translation work using artificial intelligence" refers to means for translating text into multiple languages ​​using artificial intelligence.

[1861] "Measures to improve customer experience" are measures to improve the experience customers have when purchasing products.

[1862] "Means for AI to explain questions when considering a menu" refers to a means for AI to answer questions that arise when customers are considering a menu.

[1863] "Means for recommendation based on preferences" refers to means for recommending products and services based on customer preferences.

[1864] "Means for translating product descriptions into multiple languages ​​and displaying and playing audio in the language set by the user" means means for translating product descriptions into multiple languages ​​and displaying and playing audio in the language set by the user.

[1865] To implement this invention, it is necessary to build a system that combines a smartphone, artificial intelligence technology, a translation API, and audio playback functionality.

[1866] First, smartphones have a camera function that allows users to take pictures of products. When a user takes a picture of a product, the image data is sent to a server. The server then generates a product description using image recognition technology. This image recognition technology can use, for example, Google Cloud Vision API.

[1867] The generated product description is then translated into multiple languages ​​by artificial intelligence on the server using translation services such as Google Cloud Translation API, and the translated text is sent to and displayed on the user's smartphone based on their language preference.

[1868] Furthermore, the translated product descriptions can be played back aloud using the smartphone's voice playback function, making it accessible to people with visual impairments and those with limited literacy.

[1869] As a concrete example, consider a tourist looking to purchase a leather wallet at a brick-and-mortar store in Japan. When the tourist scans the tag on the leather wallet with their smartphone camera, the app automatically recognizes the product description and translates it into the user's language (e.g., English). The translated description is displayed and played aloud: "This is a high-quality leather wallet."

[1870] An example of a prompt sentence might be:

[1871] I am developing an application that translates product descriptions into multiple languages. Please translate the following text into the specified languages:

[1872] Text: "This is a high-quality leather wallet."

[1873] Target language: Japanese

[1874] In this way, foreign tourists, visually impaired people, and people with limited literacy will be able to understand product descriptions without experiencing language barriers when purchasing products in physical stores.

[1875] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1876] Step 1:

[1877] A user takes a photo of a product using the smartphone camera. The input is the product image taken with the smartp...

Claims

1. A means for generating product descriptions from photos of products taken with a smartphone; A means for inputting an instruction for translating the product description into a generating AI model and generating a translation result of the product description; A means for inputting a question that a user has when considering a menu into the generating AI model and generating an answer to the question; A means for analyzing the user's input and generating recommendation information; A system including:

2. The system of claim 1 , wherein when generating the product description, the generative AI model automatically recognizes product characteristics and features from the photo and generates a product description based on the recognition results.

3. The system of claim 1 , wherein the generative AI model for generating the translation results generates the translation results in each of multiple languages.

Images