System

Abstract

A system is provided.SOLUTION: A means for receiving a request from a user in a natural language, a means for automatically generating a question for analyzing the received request and acquiring necessary information by using a generation AI, a means for presenting the automatically generated question to the user and receiving an answer from the user, a means for adjusting specifications of an order content based on the received answer by using the generation AI, and a means for creating a completed image diagram based on the adjusted specifications by using an image generation AI; The system includes a means for confirming the final order content and a means for transmitting the confirmed order content to the seller.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The current ordering process for custom-made products faces several challenges. In particular, there are issues such as the customer not being able to visualize the finished product, not being able to form a concrete image, the meaning of the options being unclear, and the complexity of responding to inquiries. This results in inefficiency, requiring time and effort for both the seller and the customer. Furthermore, the reliance on physical stores makes it difficult to accept orders from across the country. The purpose of this invention is to solve these challenges and provide a more efficient and user-friendly ordering system for custom-made products. [Means for solving the problem]

[0005] This invention provides an inquiry response system that includes: means for accepting requests from users in natural language; means for analyzing the accepted requests and automatically generating questions to acquire necessary information using a generation AI; means for presenting the automatically generated questions to the user and receiving responses; means for adjusting the order specifications based on the received responses using the generation AI; means for creating an image of the completed product based on the adjusted specifications using an image generation AI; means for presenting the created image of the completed product to the user and allowing the user to finalize the order details; and means for transmitting the finalized order details to the seller. The system can also include means for accepting orders from across the country and means for explaining the meanings of options and suggestions in an easy-to-understand natural language. This allows customers to easily visualize the product and reduces the burden on the seller.

[0006] An "inquiry response system" is a system that manages a series of processes that accepts inquiries from users, analyzes them, and provides the necessary information.

[0007] "User" refers to any individual or entity that places an order for a custom-made product through this System.

[0008] "Natural language" refers to a language that humans use on a daily basis, including written and spoken languages.

[0009] "Generative AI" refers to technology that uses artificial intelligence to analyze user requests and input information and automatically generate appropriate questions and suggestions.

[0010] The "means for automatically generating questions" refers to a process for analyzing a user's request and automatically generating questions to acquire the required information based on that request.

[0011] "Means for adjusting specifications" refers to the process for determining optimal product specifications based on the user's responses and requests.

[0012] "Image generation AI" refers to an artificial intelligence technology that automatically generates a visual finished image based on the user's requests and specifications.

[0013] "Complete image drawing" refers to a visual image of the product created based on the user's requests.

[0014] "Seller" refers to a company or individual that offers custom-made products through this system.

[0015] "Means of accepting orders nationwide" refers to systems and processes that allow orders to be accepted from anywhere, regardless of region.

[0016] "Means for explaining the meaning of options and suggestions" refers to a process for clearly explaining the intention and content of the options suggested to the user. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention relates to a system for streamlining the ordering process for custom-made products, and aims to reduce the burden on customers by combining generative AI and image generation AI. The program for this system operates as follows.

[0039] 1. User requests

[0040] A user accesses the system through a terminal and inputs their request for a custom-made product in natural language, including details of the product type, color, material, and design. For example, they might input, "I want a simple red silk dress."

[0041] 2. Analyzing requests and generating questions

[0042] The server uses AI to analyze the user's requests and automatically generate questions to obtain the necessary information. For example, a generated question might be, "Do you have any requests for detailed decorations or special designs?"

[0043] 3. View the question and enter the answer

[0044] The device presents the user with an automatically generated question, and the user answers the question, for example, "No decoration is necessary."

[0045] 4. Adjustment of specifications

[0046] The server adjusts the order specifications based on the user's answers. For example, the specifications may be adjusted to "red silk material with a simple design without decoration." During this process, the meaning of the options and necessary information are explained to the user in an easy-to-understand manner.

[0047] 5. Sending an image generation request

[0048] The server sends the adjusted specifications to the image generation AI and requests it to generate an image of the finished product.

[0049] 6. Creating a completed image

[0050] Based on the specifications received, the image generation AI generates a visual representation of the finished product, which concretely shows what the product will look like.

[0051] 7. Display and check the image

[0052] The terminal displays the generated completed image to the user, who then confirms it, for example, by saying, "This image is fine."

[0053] 8. Final Order Confirmation

[0054] The user checks the completed image and finalizes the order by entering "I confirm my order" on the terminal.

[0055] 9. Submitting your order

[0056] The server receives the final order and automatically sends it to the seller, who can then create the product based on the specific order.

[0057] This system allows customers to easily communicate their needs in natural language and gives them a concrete image of the finished product through questions and illustrations generated along the way. This helps avoid misunderstandings and dissatisfaction, resulting in a smooth ordering process. It also allows sellers to understand the detailed order details in advance, enabling efficient product production. This system can accept orders from all over the country, which has the effect of expanding the market for custom-made products.

[0058] The processing flow will be explained below.

[0059] Step 1:

[0060] The user accesses the system through a terminal and inputs their desired custom-made product in natural language on the system's input screen. For example, "I want a simple red silk dress."

[0061] Step 2:

[0062] The device sends the user's input to the server. The server uses generative AI to analyze the user's requests. As a result of the analysis, a list of questions is automatically generated to obtain the necessary information. For example, questions such as "Do you have any requests for detailed decorations or special designs?" are generated.

[0063] Step 3:

[0064] The server sends the generated question list to the terminal. The terminal displays the questions to the user in order, and the user inputs an answer to each question. For example, the user might answer, "No decoration is necessary."

[0065] Step 4:

[0066] The device sends the user's answers to the server. The server uses generative AI to adjust the order specifications based on the answers received from the user. During this process, the meaning of the options and necessary information are explained to the user in an easy-to-understand manner. For example, the specifications are adjusted to "red silk material with a simple design without decoration."

[0067] Step 5:

[0068] The server sends the adjusted specifications to the image generation AI and requests it to generate a visual image of the completed product. The image generation AI generates a visual image of the completed product based on the received specifications.

[0069] Step 6:

[0070] The server receives the generated completed image and sends it to the terminal. The terminal displays the completed image to the user and asks for confirmation. The user confirms, "This is the design I envisioned. This is fine."

[0071] Step 7:

[0072] The terminal sends the user's confirmation to the server, which compiles the final order and automatically sends it to the seller, who can then create the product based on the specific order.

[0073] This series of steps reduces the burden on both the customer and the seller, resulting in an efficient, high-quality custom product ordering process.

[0074] Example 1

[0075] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0076] During the custom-made product ordering process, customers often have difficulty effectively communicating their desired details. This can lead to dissatisfaction with the finished product, and it can be difficult for sellers to accurately understand the customer's needs. Additionally, the process of confirming specific design elements and materials can be cumbersome and time-consuming. Furthermore, there is a need to provide appropriate options and suggestions to meet the user's needs.

[0077] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0078] In this invention, the server includes means for accepting requests from users in natural language, means for analyzing the accepted requests using a generative AI model and automatically generating questions to acquire necessary information, and means for creating a completed image based on the adjusted specifications using an image generation AI. This allows the server to efficiently analyze the user's requests and provide a concrete image of the completed product, thereby reducing dissatisfaction with the finished product and enabling the seller to accurately understand the detailed order content.

[0079] "User" refers to any individual or legal entity that uses the System to place an order for a custom-made product.

[0080] "Natural language" refers to a language used by humans on a daily basis, a language that has grammar and meaning rather than specific program code.

[0081] A "generative AI model" refers to a system and its algorithms that use artificial intelligence technology to analyze natural language and generate text.

[0082] "Request" refers to the specific requirements and wishes that a user has for a custom-made product.

[0083] "Means for automatically generating questions" refers to a function that uses a generative AI model to automatically create questions to gather information necessary to further clarify a user's needs.

[0084] An "answer" refers to specific information that a user provides in response to a posed question.

[0085] "Means to tailor order specifications" refers to the ability to use generative AI models to determine the specific design and attributes of custom-made products based on user responses.

[0086] "Image generation AI" refers to a system or algorithm that uses artificial intelligence technology to generate visual images from text information.

[0087] A "finished image" refers to a visual representation of the appearance of a custom-made product, generated by image generation AI.

[0088] "Seller" refers to an individual or legal entity that manufactures and sells custom-made products upon receiving orders from users.

[0089] "Means of explaining the meaning of options and suggestions in response to inquiries in natural language" refers to a function that uses a generative AI model to explain appropriate options and suggestions in response to a user's natural language questions.

[0090] "Prompt" refers to a textual instruction that is required of an image generation AI to generate a specific image.

[0091] This invention is a system that streamlines the ordering process for custom-made products, and aims to reduce the burden on customers by combining generative AI models and image generation AI. The system program operates as follows.

[0092] User request input

[0093] A user accesses the system using a terminal and inputs their request for a custom-made product in natural language. This input includes details of the product type, color, material, and design. For example, they might input, "I want a simple red silk dress." The terminal then sends this request to the server.

[0094] Analyzing requests and generating questions

[0095] The server analyzes the received user request using a generative AI model (for example, a general natural language processing model). Based on the analysis results, the server generates a question to collect the necessary additional information. For example, it generates a question such as, "Do you have any requests for detailed decorations or special designs?" This generated question is then sent to the device.

[0096] View questions and enter answers

[0097] The terminal displays the generated question to the user, who responds by typing, for example, "No decoration necessary." The terminal then sends the response to the server.

[0098] Specification adjustment

[0099] The server adjusts the order specifications based on the user's answers. For example, the specifications could be set to "red silk material with a simple design without decoration." During this process, the server generates text that clearly explains the meaning of the options and necessary information to the user, and sends it to the terminal.

[0100] Sending an image generation request

[0101] Based on the adjusted specifications, the server creates a prompt to send to the image generation AI (for example, a general image generation algorithm). For example, "Please generate an image of a simple dress made of red silk." This prompt is sent to the image generation AI.

[0102] Creating a completed image

[0103] Based on the specified prompts, the image generation AI generates a visual representation of the finished product, which concretely shows the appearance of the product. The generated representation is then sent to the server.

[0104] Display and check the image diagram

[0105] The terminal displays the completed image received from the server to the user. The user checks this image and confirms, for example, "This image is OK." The confirmation result is sent to the server.

[0106] Final order confirmation

[0107] After the user checks the completed image, they input "I confirm my order" through their terminal. This input is sent to the server.

[0108] Sending order details

[0109] The server receives the final order and automatically sends it to the seller, who can then create the product based on the order.

[0110] This system allows users to easily order custom-made products using natural language and confirm specific specifications based on generated questions and finished product images. This avoids misunderstandings and frustrations, resulting in a smooth ordering experience. Furthermore, sellers can efficiently produce products by knowing the details of the order in advance.

[0111] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0112] Step 1:

[0113] A user accesses the system using a terminal and inputs their request for a custom-made product in natural language. This input includes details of the product type, color, material, and design. For example, a user might input, "I want a simple red silk dress." The terminal sends this request as input data to the server. The output data generated is the user's requested information.

[0114] Step 2:

[0115] The server uses a generative AI model to analyze the user's request data it receives. Specifically, the server inputs the request data into the generative AI model and automatically generates questions to obtain the necessary additional information. For example, the server generates a question such as, "Do you have any requests for detailed decorations or special designs?" The generated question is sent from the server to the device as output data.

[0116] Step 3:

[0117] The terminal displays the question sent from the server to the user. The user inputs an answer to this question. For example, the user inputs "No decoration is necessary." The terminal sends this answer data as input to the server. The user's answer is generated as output data.

[0118] Step 4:

[0119] The server adjusts the order specifications based on the user's response data received. Specifically, the server uses a generative AI model to analyze the response data and set the specifications to "a simple design with no decorations in red silk." During this process, it also generates text to clearly explain the meaning of the options and necessary information, and sends it to the device. The output data generated is the adjusted order specifications and explanatory text.

[0120] Step 5:

[0121] Based on the adjusted specifications, the server creates a prompt to send to the image generation AI. For example, the prompt might be, "Please generate an image of a simple dress made of red silk." After the prompt is generated, the server sends it to the image generation AI. The prompt is generated and sent as output data.

[0122] Step 6:

[0123] The image generation AI generates a visual image of the finished product based on the prompt received from the server. The prompt is input into the generation AI, and a specific visual image of the dress is generated as output. The generated image is sent to the server.

[0124] Step 7:

[0125] The terminal displays the completed image sent from the server to the user. The user checks this image and enters, for example, "This image is fine." The terminal sends the user's confirmation results to the server as input data. The confirmation results and the user's decision to confirm are generated as output data.

[0126] Step 8:

[0127] After the user has made the final confirmation, they confirm the order details through the terminal. For example, they may input "I confirm my order." This input is sent to the server, which receives the final order details. The confirmed order details are generated as output data.

[0128] Step 9:

[0129] The server automatically sends the final order details to the seller, who can then create the product based on the order details. The output data is a notification of the order details to the seller.

[0130] This allows users to easily order custom-made products using natural language and confirm specific specifications based on generated questions and finished product images. Sellers can also efficiently produce products by knowing the details of the order in advance.

[0131] (Application example 1)

[0132] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0133] The ordering process for custom-made products involves many steps, which can lead to problems such as miscommunication between the user and the seller and complicated procedures. Furthermore, it is often difficult for users to form a concrete image of the product, and it often takes a lot of effort to obtain a satisfactory product. Therefore, there is a need for a system that can flexibly respond to user requests while efficiently carrying out the ordering process.

[0134] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0135] In this invention, the server includes: means for accepting requests from a user in natural language; means for analyzing the accepted request using a generative AI model and automatically generating questions to acquire necessary information; means for presenting the automatically generated questions to the user and receiving responses; means for adjusting the order specifications based on the received responses; means for creating a completed image based on the adjusted specifications using an image generation AI; means for presenting the created completed image to the user and finalizing the order details; means for transmitting the confirmed order details to the seller; means for requesting an image from the generative AI model and generating an image generation prompt based on the order details; and means for requesting the image based on the generated prompt. This allows the user to view a specific image simply by entering their request in natural language, enabling a smooth ordering process without misunderstandings. Furthermore, the use of application software compatible with smart devices can further improve user convenience.

[0136] "User" means any individual or legal entity that uses the System to order a custom-made product.

[0137] "Natural language" refers to the language that users use in their everyday communication, and is not a specific programming language, but rather the words that humans use on a daily basis.

[0138] A "generative AI model" refers to an algorithm that uses artificial intelligence technology to analyze user requests and automatically generate questions to obtain the necessary information.

[0139] "Analysis" is the process of deciphering input natural language and extracting important information from it.

[0140] "Automatic question generation" means using artificial intelligence technology to deeply understand a user's needs and create appropriate questions to solicit additional information.

[0141] "Means for receiving answers" refers to a function for receiving answers entered by a user in response to questions presented by the system.

[0142] "Order specifications" refers to the specific characteristics and features of the product that are ultimately determined based on the user's requests and responses.

[0143] "Image generation AI" refers to algorithms that use artificial intelligence technology to generate visual images based on received specifications.

[0144] "Completed image" refers to a visual representation of a product created by image generation AI based on specifications.

[0145] "Means for finalizing the order" refers to a function that allows the user to check the generated image and finalize it as the final order.

[0146] "Seller" refers to an individual or legal entity that produces and provides products to users based on confirmed orders.

[0147] "Application software compatible with smart devices" refers to a program that runs on smart devices such as smartphones and tablets and provides the functions necessary for users to order custom-made products.

[0148] A "prompt statement" refers to a command statement that uses a generative AI model to generate an image diagram for an image generation AI.

[0149] The system for realizing this invention uses the following hardware and software: The hardware uses a smart device such as a smartphone or tablet. The software uses an application developed using ANDROID (registered trademark) Studio or Xcode, a generative AI model (GPT-4 (registered trademark)), and an image generation AI (DALL-E or Stable Diffusion).

[0150] System configuration

[0151] The system includes the following programs:

[0152] 1. A means of accepting requests from users in natural language

[0153] 2. A means of using generative AI models to analyze received requests and automatically generate questions to obtain the necessary information.

[0154] 3. A means of presenting automatically generated questions to users and receiving answers

[0155] 4. A means to adjust the specifications of the order based on the response received.

[0156] 5. A method to use image generation AI to create a finished image based on adjusted specifications

[0157] 6. A means to present the completed image to the user and finalize the order details

[0158] 7. How to send the confirmed order to the seller

[0159] 8. A method to request an image from the AI ​​model and generate an image generation prompt based on the order details

[0160] 9. A method for requesting an image based on the generated prompt

[0161] Program processing explanation

[0162] User request input

[0163] Users access the application using their smart device and enter their custom product requirements in natural language, including details of product type, color, material, and design.

[0164] Analyzing requests and generating questions

[0165] The server uses a generative AI model (GPT-4) to analyze the user's request. Based on the analysis results, it automatically generates questions to complement the information. For example, if the request is "I want a simple red silk dress," the server generates a related question: "Do you want any detailed decorations or special designs?"

[0166] View questions and enter answers

[0167] The automatically generated questions are presented to the user, who answers them via their smart device, for example, "No decoration is needed."

[0168] Specification adjustment

[0169] Based on the user's response, the server adjusts the order specifications, for example, a red silk dress with a simple design without any decorations.

[0170] Sending an image generation request

[0171] Based on the adjusted specifications, the server generates a request to the image generation AI. First, the generative AI model generates a prompt sentence. For example, the prompt sentence is generated as "Create an image of a simple, red silk dress with no decorations."

[0172] Creating and confirming a completed image

[0173] The image generation AI (DALL-E or Stable Diffusion) creates an image based on the generated prompt. This image is displayed to the user for confirmation.

[0174] Finalize and submit your order

[0175] The user checks the image and finalizes the order details, which are then sent to the seller by the server.

[0176] Specific examples

[0177] For example, a user might input, "I want a simple red silk dress." The generative AI model analyzes this and generates the question, "Do you have any requests for detailed decorations or special designs?" The user responds, "No decorations necessary." Based on this information, the server determines the specifications as "a dress made of red silk with a simple design and no decorations," and sends the prompt "Create an image of a simple, red silk dress with no decorations." The generated image is displayed to the user, and the final order is confirmed.

[0178] In this way, the user's wishes can be quickly and accurately reflected, and a smooth ordering process for custom-made products can be realized.

[0179] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0180] Step 1:

[0181] Users access the application using their smart device and input their custom product requirements in natural language, including details of product type, color, material, and design.

[0182] Step 2:

[0183] The device sends the input request to a server. The server uses a generative AI model (GPT-4) to analyze the received request and automatically generate questions to obtain the necessary information. For example, in response to a request for a simple red silk dress, the server generates the question, "Do you have any requests for detailed decorations or special designs?"

[0184] Step 3:

[0185] The server sends an automatically generated question to the terminal. The terminal displays the question to the user. The user then inputs an answer to the displayed question. For example, the user might answer, "No decoration is necessary."

[0186] Step 4:

[0187] The terminal sends the user's response to the server, which then uses the response to adjust the order specifications. Specifically, the server determines the specific characteristics of the product based on the user's request and response. For example, the specifications may be adjusted to "a simple, unadorned dress made of red silk."

[0188] Step 5:

[0189] The server uses a generative AI model to generate image prompts based on the adjusted specifications. For example, the prompt might read, "Create an image of a simple, red silk dress with no decorations."

[0190] Step 6:

[0191] The server sends the generated prompt to an image generation AI (DALL-E or Stable Diffusion), which generates a visual image based on the prompt.

[0192] Step 7:

[0193] The server sends the generated image to the terminal, which displays it to the user and asks for confirmation, for example, by saying, "This image is OK."

[0194] Step 8:

[0195] The user checks the displayed image and finalizes the order details. The terminal accepts input to confirm the order.

[0196] Step 9:

[0197] The terminal sends the confirmed order details to the server, which then notifies the seller of the final order details. The seller then produces the product based on the order details and provides it to the user.

[0198] These steps ensure that the user's needs are reflected quickly and accurately, and that the ordering process for custom-made products is smooth.

[0199] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0200] This invention is a system for streamlining the ordering process for custom-made products, and aims to reduce the burden on customers by combining generative AI and image generation AI, and to improve the customer experience by adding an emotion engine that recognizes the user's emotions. The program processing of this system is explained in detail below.

[0201] 1. User requests

[0202] A user accesses the system through a terminal and inputs their request for a custom-made product in natural language. For example, they might input, "I want a simple red silk dress."

[0203] 2. Analyzing requests and generating questions

[0204] The device sends the user's input to the server, which uses generative AI to analyze the user's request and automatically generate questions to obtain the necessary information. For example, it generates questions such as, "Do you have any requests for detailed decorations or special designs?"

[0205] 3. Emotion Recognition by Emotion Engine

[0206] When a user answers a question, the device uses an emotion engine to recognize the emotion of the user's input. For example, when a user types "No decoration needed," the device determines whether the emotion is positive or negative.

[0207] 4. View the question and enter the answer

[0208] The device then displays the generated questions to the user in order based on the analysis results of the emotion engine and receives the user's answers. For example, if the user answers "No decoration is necessary" and this answer is recognized as having a negative sentiment, the server automatically generates additional follow-up questions.

[0209] 5. Adjustment of specifications

[0210] The server adjusts the order specifications based on the user's answers and the results of the emotion engine. For example, it adjusts the specifications to "red silk material with a simple design without decoration." During this process, the meaning of the options and necessary information are explained to the user in an easy-to-understand manner.

[0211] 6. Sending an image generation request

[0212] The server sends the adjusted specifications to the image generation AI and requests it to generate a visual image of the completed product. The image generation AI generates a visual image of the completed product based on the received specifications.

[0213] 7. Creation and display of completed image

[0214] The server receives the generated completed image and sends it to the terminal. The terminal displays the completed image to the user so that the user can confirm it. For example, the user can confirm, "This is the design I had in mind. This is fine."

[0215] 8. Final Order Confirmation

[0216] The user checks the completed image and finalizes the order by entering "I confirm my order" on the terminal.

[0217] 9. Submitting your order

[0218] The server receives the final order and automatically sends it to the seller, who can then create the product based on the specific order.

[0219] This system allows customers to easily communicate their requests in natural language, and an emotion engine enables responses that reflect the customer's emotions. This avoids misunderstandings and dissatisfaction, resulting in a smooth ordering process. It also allows sellers to understand the details of the order in advance, enabling more efficient product production. This system can accept orders from all over the country, which has the effect of expanding the market for custom-made products.

[0220] The processing flow will be explained below.

[0221] This invention is a system for streamlining the ordering process for custom-made products, and aims to reduce the burden on customers by combining generative AI and image generation AI, and to improve the customer experience by adding an emotion engine that recognizes the user's emotions. The details of the program processing of this system are as follows.

[0222] Step 1:

[0223] A user accesses the system through a terminal and inputs their request for a custom-made product in natural language, for example, "I want a simple red silk dress."

[0224] Step 2:

[0225] The device sends the user's input to the server, which uses generative AI to analyze the user's request and automatically generate questions to obtain the necessary information. For example, it generates questions such as, "Do you have any requests for detailed decorations or special designs?"

[0226] Step 3:

[0227] The server sends the generated question list to the terminal, which displays the generated questions to the user in order.

[0228] Step 4:

[0229] The user answers the displayed question. For example, they might type "No decoration necessary." The device uses an emotion engine to recognize the user's emotion along with their answer. For example, if the user answers with a negative emotion, that information is also sent to the server.

[0230] Step 5:

[0231] The device sends the user's answers and emotional data to the server, which then uses generative AI to adjust the order specifications based on the user's answers and emotional data. For example, the specifications may be adjusted to "red silk material with a simple design without decoration."

[0232] Step 6:

[0233] The server generates a request to the image generation AI based on the adjusted specifications, and the image generation AI generates a visual representation of the finished product based on the received specifications.

[0234] Step 7:

[0235] The server receives the generated completed image and sends it to the terminal, which displays the completed image to the user and asks for their confirmation.

[0236] Step 8:

[0237] The user checks the completed image. When the user responds, "This is the design I imagined, I'd like this," the device sends the confirmation to the server.

[0238] Step 9:

[0239] The terminal sends the user's confirmation to the server, which compiles the final order and automatically sends it to the merchant.

[0240] This series of processes allows customers to have a concrete image, and the emotion engine responds appropriately. Furthermore, sellers can understand the detailed order details in advance, enabling efficient product production. This system reduces the burden on both customers and sellers, and realizes an efficient, high-quality custom-made product ordering experience.

[0241] Example 2

[0242] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0243] Traditional custom-made product ordering processes have struggled to accurately understand and efficiently process customer requests. They also lack the ability to properly understand and respond to customer emotions and intent, resulting in final products that don't meet customer expectations. Furthermore, there is a lack of established methods for utilizing image generation technology, resulting in a lack of visual feedback. Therefore, there is a need for a way to improve customer experience and efficiently process orders.

[0244] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0245] In this invention, the server includes means for accepting requests from a user in natural language, means for analyzing the accepted request using a generation AI and automatically generating questions to acquire necessary information, means for presenting the automatically generated questions to the user and receiving answers, means for recognizing the user's emotions using an emotion engine, means for adjusting the order specifications based on the received answers and the emotion recognition results using the generation AI, means for creating an image of the completed product based on the adjusted specifications using an image generation AI, means for presenting the created image of the completed product to the user and finalizing the order details, and means for transmitting the finalized order details to the seller. This makes it possible to accurately understand customer requests and respond in line with the customer's emotions, and by providing visual feedback, it is possible to improve the customer experience and process orders more efficiently.

[0246] The "means for accepting requests in natural language from the user" is an interface that allows the user to input their wishes and requests regarding custom-made products in natural language.

[0247] "Generative AI" is a system that uses artificial intelligence technology to analyze requests received in natural language, extract necessary information, and automatically generate questions.

[0248] "Means for presenting automatically generated questions to the user and receiving answers" refers to an interface that displays questions generated by the generation AI to the user and allows the user to input answers to those questions.

[0249] The "emotion engine" is a system that analyzes the emotions expressed by users when they input or respond in real time, and determines whether those emotions are positive or negative.

[0250] "Means for adjusting order specifications based on responses received using generative AI and emotion recognition results" refers to a system that appropriately adjusts order details and determines final specifications based on responses from users and the analysis results of the emotion engine.

[0251] "Image generation AI" is an artificial intelligence technology for generating visual images of completed products based on tailored order specifications.

[0252] The "means for presenting the completed image drawing to the user and allowing the user to confirm the final order details" is an interface that displays the generated completed image drawing to the user and allows the user to confirm the final order details.

[0253] The "means for transmitting the confirmed order details to the seller" is a system for automatically transmitting the order details confirmed by the user to the seller and issuing instructions for product production.

[0254] "Means for accepting orders from all over the country" refers to a system that allows orders to be accepted from all over the country without being restricted by geography.

[0255] "Means for explaining the meaning of options and suggestions in response to inquiries in natural language" refers to an interface that explains the meaning of options and related suggestions in response to the content of an inquiry made by a user in natural language.

[0256] This invention is a system for streamlining the ordering process for custom-made products and improving customer experience. The system is composed of a user terminal, a server, a generative AI model, an image generation AI, and an emotion engine.

[0257] Hardware and software used:

[0258] Hardware: User devices (PCs, smartphones, tablets, etc.), servers

[0259] Software: Generative AI model, image generation AI, emotion engine

[0260] System process flow and specific explanation:

[0261] 1. User requests

[0262] The user uses the device to open a web browser or dedicated application.

[0263] The terminal displays an input form to the user, and the user inputs their request for a custom-made product in natural language, for example, "I want a simple red silk dress."

[0264] The terminal sends this information to the server.

[0265] 2. Analyzing requests and generating questions

[0266] The server runs a generative AI model to analyze the received user request. For example, we use GPT-4 from OpenAI (registered trademark).

[0267] The server uses a generative AI model to analyze the request and automatically generate questions to gather additional information.

[0268] For example, generate a question like, "Do you have any requests for detailed decorations or special designs?"

[0269] The server sends the generated question to the terminal.

[0270] 3. Emotion Recognition by Emotion Engine

[0271] When a user answers questions through the device, the device uses an emotion engine to analyze the user's emotions in real time. For example, it uses Affectiva as emotion recognition software.

[0272] For example, if the user answers "No decoration is needed," it is determined whether the answer is positive or negative.

[0273] 4. View the question and enter the answer

[0274] The terminal sequentially displays questions generated based on the results of the emotion engine to the user and receives the user's answers.

[0275] For example, if a user answers "No decoration necessary" and the answer is perceived as negative, the server will automatically generate additional follow-up questions.

[0276] For example, follow-up questions such as "Would you consider other colors or materials?" are generated.

[0277] 5. Adjustment of specifications

[0278] The server adjusts the order specifications based on the user's answers and the results of the emotion engine.

[0279] For example, the specifications are determined as "red silk material with a simple design without decoration."

[0280] During this process, the server explains the meaning of the options and the necessary information to the user in an easy-to-understand manner.

[0281] 6. Sending an image generation request

[0282] The server sends the adjusted specifications to the image generation AI and requests it to generate a completed image. As an example, we will use DALL-E 2.

[0283] The image generation AI generates a visual representation of the finished product based on the received specifications.

[0284] 7. Creation and display of completed image

[0285] The server receives the generated completed image and sends it to the terminal.

[0286] The terminal displays the completed image to the user, who then confirms it, for example by saying, "This is the design I had in mind. This is fine."

[0287] 8. Final Order Confirmation

[0288] The user checks the completed image and finalizes the order details.

[0289] The user inputs "I confirm the order" through the terminal.

[0290] 9. Submitting your order

[0291] The server receives the final confirmed order and automatically sends it to the merchant.

[0292] The seller receives this information and begins producing the product based on the specific order.

[0293] Examples and prompts:

[0294] Examples:

[0295] Suppose a user inputs "a red silk dress," which generates the question "Do you need detailed embellishments?" If the user answers "No embellishments needed," and the emotion engine determines this as negative, an additional question is generated: "Would you consider other colors or materials?" The specifications are then finalized, and an image of the finished product is generated.

[0296] Example prompt sentence:

[0297] Suppose a user orders a custom dress based on the following criteria:

[0298] "I want a simple red silk dress."

[0299] In response, do you have any detailed decorations or special design requests?

[0300] Also, how should we respond if negative sentiment is detected in response to the response that decoration is not necessary?

[0301] The system allows for accurate understanding of customer needs, emotional response, and visual feedback, resulting in an improved customer experience and efficient order processing.

[0302] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0303] Step 1: User input

[0304] Input: User's natural language request

[0305] Specific operation: The user opens a web browser or a dedicated application on the device. The device displays an input form, and the user inputs their requirements for a custom-made product in natural language. For example, they might input, "I want a simple red silk dress."

[0306] Output: User request data

[0307] Detailed description: The terminal receives the user's input and sends this request data to the server.

[0308] Step 2: Analyze the needs and generate questions

[0309] Input: User's request data

[0310] How it works: The server launches a generative AI model (e.g., GPT-4) to analyze the user's request. The generative AI model is used to extract necessary information from the request and automatically generate follow-up questions.

[0311] Output: Auto-generated questions

[0312] Detailed explanation: For example, a question such as "Do you have any requests for detailed decorations or special designs?" is generated, and the server sends this question to the terminal.

[0313] Step 3: Emotion recognition by the emotion engine

[0314] Input: User response data

[0315] Specific operation: When the user answers a question generated through the device, the device launches an emotion engine (e.g., Affectiva) and analyzes the user's emotions in real time.

[0316] Output: Emotion recognition result

[0317] Longer explanation: For example, if a user answers "No decoration needed," the sentiment engine determines whether the answer is positive or negative.

[0318] Step 4: View the question and enter the answer

[0319] Input: Auto-generated question and emotion recognition result

[0320] Specific operation: The device refers to the results of the emotion engine, displays the generated questions to the user in sequence, and receives the user's answers. The user enters answers to each question.

[0321] Output: User response data

[0322] Detailed explanation: For example, if a user answers "I don't need decorations," and the answer is recognized as negative, the server automatically generates an additional follow-up question, such as "Would you consider other colors or materials?"

[0323] Step 5: Adjust the specifications

[0324] Input: User response data and emotion recognition results

[0325] Specific operation: The server adjusts the order specifications based on the user's answers and the results of the emotion engine.

[0326] Output: Adjusted order specifications

[0327] Detailed explanation: For example, the specifications are determined as "red silk material with a simple design without decorations," and the server explains the meaning of the options and necessary information to the user in an easy-to-understand manner during this process.

[0328] Step 6: Sending an image generation request

[0329] Input: Adjusted order specifications

[0330] Specific operation: The server sends the adjusted specifications to the image generation AI (e.g., DALL-E 2) and requests the generation of a completed image.

[0331] Output: Finished image

[0332] Detailed description: Image generation AI generates a visual representation of the finished product based on the received specifications.

[0333] Step 7: Create and display a finished image

[0334] Input: Image of completed product

[0335] Specific operation: The server receives the generated completed image and sends it to the terminal, which displays the image to the user so that the user can check it.

[0336] Output: Final confirmation by the user

[0337] Detailed explanation: For example, the user confirms, "This is the design I envisioned, I'd like this."

[0338] Step 8: Finalize your order

[0339] Input: Final confirmation by the user

[0340] Specific operation: The user checks the completed image and confirms the final order details. The user enters "I confirm the order" on the terminal.

[0341] Output: Confirmed order details

[0342] Detailed description: The order details confirmed by the user are sent to the server.

[0343] Step 9: Submit your order

[0344] Input: Confirmed order details

[0345] What happens: The server receives the final confirmed order details and automatically sends them to the merchant.

[0346] Output: Order received by merchant

[0347] Detailed Description: The seller will receive this information and begin producing the product based on your specific order.

[0348] This system allows customers to easily communicate their needs in natural language and uses an emotion engine to respond appropriately based on the customer's emotions, resulting in a smooth ordering process. It also allows sellers to efficiently produce products by knowing the details of the order in advance.

[0349] (Application example 2)

[0350] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0351] The modern ordering process for custom-made products is extremely time-consuming and tedious, requiring customers to visit a store and provide detailed requests. Furthermore, requests are collected without taking into account the customer's emotional state, which can easily lead to customer dissatisfaction and misunderstandings. Furthermore, there is a lack of visual feedback, making it difficult for customers to visualize the final product. These issues significantly impair the customer experience, especially in physical stores.

[0352] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for accepting a request from a user in natural language; means for automatically generating questions to acquire necessary information by analyzing the accepted request using a generation AI; means for presenting the automatically generated questions to the user and receiving answers; means for adjusting the order specifications based on the received answers using the generation AI; means for creating an image of the completed product based on the adjusted specifications using an image generation AI; means for presenting the created image of the completed product to the user and finalizing the order details; means for transmitting the finalized order details to the seller; means for analyzing the user's emotions at the time of user input using an emotion engine that recognizes the user's emotions; means for analyzing the customer's emotions in real time and automatically generating follow-up questions; and means for presenting information to the user using a glasses-type terminal and confirming the order process. This realizes an ordering process that takes customer emotions into consideration, enabling intuitive and efficient ordering of custom-made products.

[0353] The "means for accepting requests from users in natural language" is a system that allows customers to input details of the custom-made products they desire in natural language via voice or text.

[0354] "Generative AI" is a system that uses artificial intelligence to analyze user input and generate relevant questions and content.

[0355] The "means for automatically generating questions" is a system that has the function of automatically generating questions to acquire necessary information based on the user's request.

[0356] The "means for presenting automatically generated questions to users and receiving answers" is a mechanism for displaying automatically generated questions to users and collecting answers from the users.

[0357] The "means for adjusting the specifications of the order" is a system that has an adjustment function for determining the final product specifications based on the user's responses.

[0358] "Image generation AI" is an artificial intelligence that generates a rendering of a finished product based on adjusted specifications.

[0359] The "means for creating a completed image drawing" is a system having a function for creating and saving the generated completed image drawing.

[0360] "Means of presenting the completed image to the user and finalizing the order details" refers to a mechanism that shows the user an image of the completed product and leads them through the process of confirming and confirming the final order details.

[0361] The "means for transmitting the confirmed order details to the seller" is a mechanism for automatically transferring the confirmed order details to the seller.

[0362] The "emotion engine" is a system for analyzing the emotions of users in real time when they input their emotions.

[0363] The "means for analyzing customer emotions in real time and automatically generating follow-up questions" is a system that analyzes the emotional state of a customer and automatically generates additional questions as needed.

[0364] An "eyeglasses-type terminal" is a wearable device that visually provides information to a user.

[0365] The present invention is a system for enabling users to streamline the ordering process for custom-made products using natural language, and can be implemented as follows.

[0366] 1. User requests

[0367] The user wears the glasses-type device and inputs their request for a custom-made product by voice or text. For example, they might say, "I'd like a simple red silk dress." The microphone in the glasses-type device captures the voice and sends the data to a smartphone connected via Bluetooth or to the store's server. This uses a voice recognition API (for example, Google® Cloud Speech-to-Text).

[0368] 2. Analyzing requests and generating questions

[0369] The server analyzes the received voice data and uses a generative AI (such as OpenAI's GPT-4) to analyze the user's requests. As a result of the analysis, questions are automatically generated to obtain the necessary information. For example, a question such as "Do you have any requests for detailed decorations or special designs?" is generated.

[0370] 3. Emotion Recognition by Emotion Engine

[0371] As the user answers the questions, the camera on the glasses uses facial expression recognition software (e.g., Microsoft® Azure® Cognitive Services emotion recognition API) to recognize the user's emotions in real time. For example, when the user types "No decoration needed," the camera determines whether the emotion is positive or negative.

[0372] 4. View the question and enter the answer

[0373] The generated questions are displayed on the display of the glasses-type device, and the user answers using voice or a touch interface. If the emotion engine detects a negative emotion, the server automatically generates additional follow-up questions.

[0374] 5. Adjustment of specifications

[0375] The server adjusts the order specifications based on the user's answers and the results of the emotion engine. For example, it may adjust the specifications to "red silk material with a simple design without decoration," and also explain the meaning of the options and necessary information to the user in an easy-to-understand manner.

[0376] 6. Sending an image generation request

[0377] The adjusted specifications are sent to an image generation AI (e.g., DALL-E) and a request is made to generate a visual image of the completed product. The image generation AI generates a visual image of the completed product based on the received specifications.

[0378] 7. Creation and display of completed image

[0379] The server receives the generated completed image and sends it to the eyeglasses. The eyeglasses display the completed image to the user, allowing them to confirm it. For example, the user can confirm, "This is the design I had in mind. This is fine."

[0380] 8. Final Order Confirmation

[0381] The user checks the completed image and confirms the final order details by voice command. The data from the glasses-type device is sent to the server.

[0382] 9. Submitting your order

[0383] The server receives the final order and automatically sends it to the seller, who can then create the product based on the specific order.

[0384] Specific examples

[0385] A user enters a physical store, puts on the eyeglasses-type device, and says, "I want a simple red silk dress."

[0386] The glasses-type device displays a question, asking, "Do you have any decoration preferences?"

[0387] If the user answers "Nothing in particular," the emotion engine recognizes this as positive.

[0388] The final specifications were decided as "red silk material with a simple design without any decoration."

[0389] Image generation AI generates visual images and displays them on the glasses-type device.

[0390] The user checks the image and confirms the final order by saying, "This is it."

[0391] Prompt Sentence Examples

[0392] "I want a simple red silk dress."

[0393] "Do you have any decoration preferences?"

[0394] "Nothing in particular."

[0395] This creates an ordering process that takes customer emotions into account, making ordering custom-made products intuitive and efficient.

[0396] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0397] Step 1:

[0398] The user wears the eyeglasses and inputs their requests for custom-made products by voice or text. The microphone in the eyeglasses captures the voice and sends the data to a smartphone connected via Bluetooth or to the store's server. This input is done by the user saying, for example, "I want a simple red silk dress." The server converts the voice data into text data using a speech recognition API (for example, Google Cloud Speech-to-Text). This converts the voice input (input data) into text data (output data).

[0399] Step 2:

[0400] The server analyzes the received text data and uses a generative AI (for example, OpenAI's GPT-4) to analyze the user's request. As a result of the analysis, questions are automatically generated to obtain the required information. For example, a question such as "Do you have any requests for detailed decorations or special designs?" is generated. This involves the process in which the generative AI generates questions (output data) based on the user's request, which is input.

[0401] Step 3:

[0402] The display on the glasses-type device presents automatically generated questions to the user. The user answers the questions using voice or a touch interface. The server receives the answers as text data. In this step, the user answers "nothing in particular" to the questions, and the input data is sent to the server.

[0403] Step 4:

[0404] When a user answers a question, the camera on the glasses uses facial expression recognition software (e.g., Microsoft Azure Cognitive Services' Emotion Recognition API) to recognize the user's emotions in real time. For example, the emotion engine detects positive emotions in response to the user's answer. In this step, real-time facial expression data is used as input data, and emotion identification results are obtained as output data.

[0405] Step 5:

[0406] The server adjusts the order specifications based on the user's answers and the results of the emotion engine. For example, it determines the specifications as "red silk material with a simple design without decoration." This adjustment is done using generative AI, and includes a process of determining the specifications (output data) from the input data of the answers and emotion data.

[0407] Step 6:

[0408] The server sends the finalized specifications to an image generation AI (e.g., DALL-E) and requests it to generate a visual representation of the completed image. Based on the received specifications, the image generation AI generates a visual representation of the completed image (output data). The input data in this step are the finalized specifications.

[0409] Step 7:

[0410] The server receives the generated completed image and sends it to the eyeglasses. The eyeglasses display the completed image on the screen for the user to confirm. The user then looks at the image and confirms, "This is fine." The input data in this step is the generated completed image, and the output data is the user's final confirmation.

[0411] Step 8:

[0412] The user checks the completed image and confirms the final order details by voice command. The voice command data from the glasses-type device is sent to the server. After this confirmation, the server confirms the final order details (output data).

[0413] Step 9:

[0414] The server receives the final order and automatically sends it to the seller, so that the seller can create the product based on the specific order. In this step, the input data is the finalized order, and the output data is the data to be sent to the seller.

[0415] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0416] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0417] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0418] [Second embodiment]

[0419] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0420] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0421] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0422] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0423] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0424] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0425] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0426] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0427] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0428] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0429] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0430] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0431] This invention relates to a system for streamlining the ordering process for custom-made products, and aims to reduce the burden on customers by combining generative AI and image generation AI. The program for this system operates as follows.

[0432] 1. User requests

[0433] A user accesses the system through a terminal and inputs their request for a custom-made product in natural language, including details of the product type, color, material, and design. For example, they might input, "I want a simple red silk dress."

[0434] 2. Analyzing requests and generating questions

[0435] The server uses AI to analyze the user's requests and automatically generate questions to obtain the necessary information. For example, a generated question might be, "Do you have any requests for detailed decorations or special designs?"

[0436] 3. View the question and enter the answer

[0437] The device presents the user with an automatically generated question, and the user answers the question, for example, "No decoration is necessary."

[0438] 4. Adjustment of specifications

[0439] The server adjusts the order specifications based on the user's answers. For example, the specifications may be adjusted to "red silk material with a simple design without decoration." During this process, the meaning of the options and necessary information are explained to the user in an easy-to-understand manner.

[0440] 5. Sending an image generation request

[0441] The server sends the adjusted specifications to the image generation AI and requests it to generate an image of the finished product.

[0442] 6. Creating a completed image

[0443] Based on the specifications received, the image generation AI generates a visual representation of the finished product, which concretely shows what the product will look like.

[0444] 7. Display and check the image

[0445] The terminal displays the generated completed image to the user, who then confirms it, for example, by saying, "This image is fine."

[0446] 8. Final Order Confirmation

[0447] The user checks the completed image and finalizes the order by entering "I confirm my order" on the terminal.

[0448] 9. Submitting your order

[0449] The server receives the final order and automatically sends it to the seller, who can then create the product based on the specific order.

[0450] This system allows customers to easily communicate their needs in natural language and gives them a concrete image of the finished product through questions and illustrations generated along the way. This helps avoid misunderstandings and dissatisfaction, resulting in a smooth ordering process. It also allows sellers to understand the detailed order details in advance, enabling efficient product production. This system can accept orders from all over the country, which has the effect of expanding the market for custom-made products.

[0451] The processing flow will be explained below.

[0452] Step 1:

[0453] The user accesses the system through a terminal and inputs their desired custom-made product in natural language on the system's input screen. For example, "I want a simple red silk dress."

[0454] Step 2:

[0455] The device sends the user's input to the server. The server uses generative AI to analyze the user's requests. As a result of the analysis, a list of questions is automatically generated to obtain the necessary information. For example, questions such as "Do you have any requests for detailed decorations or special designs?" are generated.

[0456] Step 3:

[0457] The server sends the generated question list to the terminal. The terminal displays the questions to the user in order, and the user inputs an answer to each question. For example, the user might answer, "No decoration is necessary."

[0458] Step 4:

[0459] The device sends the user's answers to the server. The server uses generative AI to adjust the order specifications based on the answers received from the user. During this process, the meaning of the options and necessary information are explained to the user in an easy-to-understand manner. For example, the specifications are adjusted to "red silk material with a simple design without decoration."

[0460] Step 5:

[0461] The server sends the adjusted specifications to the image generation AI and requests it to generate a visual image of the completed product. The image generation AI generates a visual image of the completed product based on the received specifications.

[0462] Step 6:

[0463] The server receives the generated completed image and sends it to the terminal. The terminal displays the completed image to the user and asks for confirmation. The user confirms, "This is the design I envisioned. This is fine."

[0464] Step 7:

[0465] The terminal sends the user's confirmation to the server, which compiles the final order and automatically sends it to the seller, who can then create the product based on the specific order.

[0466] This series of steps reduces the burden on both the customer and the seller, resulting in an efficient, high-quality custom product ordering process.

[0467] Example 1

[0468] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0469] During the custom-made product ordering process, customers often have difficulty effectively communicating their desired details. This can lead to dissatisfaction with the finished product, and it can be difficult for sellers to accurately understand the customer's needs. Additionally, the process of confirming specific design elements and materials can be cumbersome and time-consuming. Furthermore, there is a need to provide appropriate options and suggestions to meet the user's needs.

[0470] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0471] In this invention, the server includes means for accepting requests from users in natural language, means for analyzing the accepted requests using a generative AI model and automatically generating questions to acquire necessary information, and means for creating a completed image based on the adjusted specifications using an image generation AI. This allows the server to efficiently analyze the user's requests and provide a concrete image of the completed product, thereby reducing dissatisfaction with the finished product and enabling the seller to accurately understand the detailed order content.

[0472] "User" refers to any individual or legal entity that uses the System to place an order for a custom-made product.

[0473] "Natural language" refers to a language used by humans on a daily basis, a language that has grammar and meaning rather than specific program code.

[0474] A "generative AI model" refers to a system and its algorithms that use artificial intelligence technology to analyze natural language and generate text.

[0475] "Request" refers to the specific requirements and wishes that a user has for a custom-made product.

[0476] "Means for automatically generating questions" refers to a function that uses a generative AI model to automatically create questions to gather information necessary to further clarify a user's needs.

[0477] An "answer" refers to specific information that a user provides in response to a posed question.

[0478] "Means to tailor order specifications" refers to the ability to use generative AI models to determine the specific design and attributes of custom-made products based on user responses.

[0479] "Image generation AI" refers to a system or algorithm that uses artificial intelligence technology to generate visual images from text information.

[0480] A "finished image" refers to a visual representation of the appearance of a custom-made product, generated by image generation AI.

[0481] "Seller" refers to an individual or legal entity that manufactures and sells custom-made products upon receiving orders from users.

[0482] "Means of explaining the meaning of options and suggestions in response to inquiries in natural language" refers to a function that uses a generative AI model to explain appropriate options and suggestions in response to a user's natural language questions.

[0483] "Prompt" refers to a textual instruction that is required of an image generation AI to generate a specific image.

[0484] This invention is a system that streamlines the ordering process for custom-made products, and aims to reduce the burden on customers by combining generative AI models and image generation AI. The system program operates as follows.

[0485] User request input

[0486] A user accesses the system using a terminal and inputs their request for a custom-made product in natural language. This input includes details of the product type, color, material, and design. For example, they might input, "I want a simple red silk dress." The terminal then sends this request to the server.

[0487] Analyzing requests and generating questions

[0488] The server analyzes the received user request using a generative AI model (for example, a general natural language processing model). Based on the analysis results, the server generates a question to collect the necessary additional information. For example, it generates a question such as, "Do you have any requests for detailed decorations or special designs?" This generated question is then sent to the device.

[0489] View questions and enter answers

[0490] The terminal displays the generated question to the user, who responds by typing, for example, "No decoration necessary." The terminal then sends the response to the server.

[0491] Specification adjustment

[0492] The server adjusts the order specifications based on the user's answers. For example, the specifications could be set to "red silk material with a simple design without decoration." During this process, the server generates text that clearly explains the meaning of the options and necessary information to the user, and sends it to the terminal.

[0493] Sending an image generation request

[0494] Based on the adjusted specifications, the server creates a prompt to send to the image generation AI (for example, a general image generation algorithm). For example, "Please generate an image of a simple dress made of red silk." This prompt is sent to the image generation AI.

[0495] Creating a completed image

[0496] Based on the specified prompts, the image generation AI generates a visual representation of the finished product, which concretely shows the appearance of the product. The generated representation is then sent to the server.

[0497] Display and check the image diagram

[0498] The terminal displays the completed image received from the server to the user. The user checks this image and confirms, for example, "This image is OK." The confirmation result is sent to the server.

[0499] Final order confirmation

[0500] After the user checks the completed image, they input "I confirm my order" through their terminal. This input is sent to the server.

[0501] Sending order details

[0502] The server receives the final order and automatically sends it to the seller, who can then create the product based on the order.

[0503] This system allows users to easily order custom-made products using natural language and confirm specific specifications based on generated questions and finished product images. This avoids misunderstandings and frustrations, resulting in a smooth ordering experience. Furthermore, sellers can efficiently produce products by knowing the details of the order in advance.

[0504] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0505] Step 1:

[0506] A user accesses the system using a terminal and inputs their request for a custom-made product in natural language. This input includes details of the product type, color, material, and design. For example, a user might input, "I want a simple red silk dress." The terminal sends this request as input data to the server. The output data generated is the user's requested information.

[0507] Step 2:

[0508] The server uses a generative AI model to analyze the user's request data it receives. Specifically, the server inputs the request data into the generative AI model and automatically generates questions to obtain the necessary additional information. For example, the server generates a question such as, "Do you have any requests for detailed decorations or special designs?" The generated question is sent from the server to the device as output data.

[0509] Step 3:

[0510] The terminal displays the question sent from the server to the user. The user inputs an answer to this question. For example, the user inputs "No decoration is necessary." The terminal sends this answer data as input to the server. The user's answer is generated as output data.

[0511] Step 4:

[0512] The server adjusts the order specifications based on the user's response data received. Specifically, the server uses a generative AI model to analyze the response data and set the specifications to "a simple design with no decorations in red silk." During this process, it also generates text to clearly explain the meaning of the options and necessary information, and sends it to the device. The output data generated is the adjusted order specifications and explanatory text.

[0513] Step 5:

[0514] Based on the adjusted specifications, the server creates a prompt to send to the image generation AI. For example, the prompt might be, "Please generate an image of a simple dress made of red silk." After the prompt is generated, the server sends it to the image generation AI. The prompt is generated and sent as output data.

[0515] Step 6:

[0516] The image generation AI generates a visual image of the finished product based on the prompt received from the server. The prompt is input into the generation AI, and a specific visual image of the dress is generated as output. The generated image is sent to the server.

[0517] Step 7:

[0518] The terminal displays the completed image sent from the server to the user. The user checks this image and enters, for example, "This image is fine." The terminal sends the user's confirmation results to the server as input data. The confirmation results and the user's decision to confirm are generated as output data.

[0519] Step 8:

[0520] After the user has made the final confirmation, they confirm the order details through the terminal. For example, they may input "I confirm my order." This input is sent to the server, which receives the final order details. The confirmed order details are generated as output data.

[0521] Step 9:

[0522] The server automatically sends the final order details to the seller, who can then create the product based on the order details. The output data is a notification of the order details to the seller.

[0523] This allows users to easily order custom-made products using natural language and confirm specific specifications based on generated questions and finished product images. Sellers can also efficiently produce products by knowing the details of the order in advance.

[0524] (Application example 1)

[0525] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0526] The ordering process for custom-made products involves many steps, which can lead to problems such as miscommunication between the user and the seller and complicated procedures. Furthermore, it is often difficult for users to form a concrete image of the product, and it often takes a lot of effort to obtain a satisfactory product. Therefore, there is a need for a system that can flexibly respond to user requests while efficiently carrying out the ordering process.

[0527] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0528] In this invention, the server includes: means for accepting requests from a user in natural language; means for analyzing the accepted request using a generative AI model and automatically generating questions to acquire necessary information; means for presenting the automatically generated questions to the user and receiving responses; means for adjusting the order specifications based on the received responses; means for creating a completed image based on the adjusted specifications using an image generation AI; means for presenting the created completed image to the user and finalizing the order details; means for transmitting the confirmed order details to the seller; means for requesting an image from the generative AI model and generating an image generation prompt based on the order details; and means for requesting the image based on the generated prompt. This allows the user to view a specific image simply by entering their request in natural language, enabling a smooth ordering process without misunderstandings. Furthermore, the use of application software compatible with smart devices can further improve user convenience.

[0529] "User" means any individual or legal entity that uses the System to order a custom-made product.

[0530] "Natural language" refers to the language that users use in their everyday communication, and is not a specific programming language, but rather the words that humans use on a daily basis.

[0531] A "generative AI model" refers to an algorithm that uses artificial intelligence technology to analyze user requests and automatically generate questions to obtain the necessary information.

[0532] "Analysis" is the process of deciphering input natural language and extracting important information from it.

[0533] "Automatic question generation" means using artificial intelligence technology to deeply understand a user's needs and create appropriate questions to solicit additional information.

[0534] "Means for receiving answers" refers to a function for receiving answers entered by a user in response to questions presented by the system.

[0535] "Order specifications" refers to the specific characteristics and features of the product that are ultimately determined based on the user's requests and responses.

[0536] "Image generation AI" refers to algorithms that use artificial intelligence technology to generate visual images based on received specifications.

[0537] "Completed image" refers to a visual representation of a product created by image generation AI based on specifications.

[0538] "Means for finalizing the order" refers to a function that allows the user to check the generated image and finalize it as the final order.

[0539] "Seller" refers to an individual or legal entity that produces and provides products to users based on confirmed orders.

[0540] "Application software compatible with smart devices" refers to a program that runs on smart devices such as smartphones and tablets and provides the functions necessary for users to order custom-made products.

[0541] A "prompt statement" refers to a command statement that uses a generative AI model to generate an image diagram for an image generation AI.

[0542] The system for realizing this invention uses the following hardware and software: The hardware uses a smart device such as a smartphone or tablet. The software uses an application developed using Android Studio or Xcode, a generative AI model (GPT-4), and an image generation AI (DALL-E or Stable Diffusion).

[0543] System configuration

[0544] The system includes the following programs:

[0545] 1. A means of accepting requests from users in natural language

[0546] 2. A means of using generative AI models to analyze received requests and automatically generate questions to obtain the necessary information.

[0547] 3. A means of presenting automatically generated questions to users and receiving answers

[0548] 4. A means to adjust the specifications of the order based on the response received.

[0549] 5. A method to use image generation AI to create a finished image based on adjusted specifications

[0550] 6. A means to present the completed image to the user and finalize the order details

[0551] 7. How to send the confirmed order to the seller

[0552] 8. A method to request an image from the AI ​​model and generate an image generation prompt based on the order details

[0553] 9. A method for requesting an image based on the generated prompt

[0554] Program processing explanation

[0555] User request input

[0556] Users access the application using their smart device and enter their custom product requirements in natural language, including details of product type, color, material, and design.

[0557] Analyzing requests and generating questions

[0558] The server uses a generative AI model (GPT-4) to analyze the user's request. Based on the analysis results, it automatically generates questions to complement the information. For example, if the request is "I want a simple red silk dress," the server generates a related question: "Do you want any detailed decorations or special designs?"

[0559] View questions and enter answers

[0560] The automatically generated questions are presented to the user, who answers them via their smart device, for example, "No decoration is needed."

[0561] Specification adjustment

[0562] Based on the user's response, the server adjusts the order specifications, for example, a red silk dress with a simple design without any decorations.

[0563] Sending an image generation request

[0564] Based on the adjusted specifications, the server generates a request to the image generation AI. First, the generative AI model generates a prompt sentence. For example, the prompt sentence is generated as "Create an image of a simple, red silk dress with no decorations."

[0565] Creating and confirming a completed image

[0566] The image generation AI (DALL-E or Stable Diffusion) creates an image based on the generated prompt. This image is displayed to the user for confirmation.

[0567] Finalize and submit your order

[0568] The user checks the image and finalizes the order details, which are then sent to the seller by the server.

[0569] Specific examples

[0570] For example, a user might input, "I want a simple red silk dress." The generative AI model analyzes this and generates the question, "Do you have any requests for detailed decorations or special designs?" The user responds, "No decorations necessary." Based on this information, the server determines the specifications as "a dress made of red silk with a simple design and no decorations," and sends the prompt "Create an image of a simple, red silk dress with no decorations." The generated image is displayed to the user, and the final order is confirmed.

[0571] In this way, the user's wishes can be quickly and accurately reflected, and a smooth ordering process for custom-made products can be realized.

[0572] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0573] Step 1:

[0574] Users access the application using their smart device and input their custom product requirements in natural language, including details of product type, color, material, and design.

[0575] Step 2:

[0576] The device sends the input request to a server. The server uses a generative AI model (GPT-4) to analyze the received request and automatically generate questions to obtain the necessary information. For example, in response to a request for a simple red silk dress, the server generates the question, "Do you have any requests for detailed decorations or special designs?"

[0577] Step 3:

[0578] The server sends an automatically generated question to the terminal. The terminal displays the question to the user. The user then inputs an answer to the displayed question. For example, the user might answer, "No decoration is necessary."

[0579] Step 4:

[0580] The terminal sends the user's response to the server, which then uses the response to adjust the order specifications. Specifically, the server determines the specific characteristics of the product based on the user's request and response. For example, the specifications may be adjusted to "a simple, unadorned dress made of red silk."

[0581] Step 5:

[0582] The server uses a generative AI model to generate image prompts based on the adjusted specifications. For example, the prompt might read, "Create an image of a simple, red silk dress with no decorations."

[0583] Step 6:

[0584] The server sends the generated prompt to an image generation AI (DALL-E or Stable Diffusion), which generates a visual image based on the prompt.

[0585] Step 7:

[0586] The server sends the generated image to the terminal, which displays it to the user and asks for confirmation, for example, by saying, "This image is OK."

[0587] Step 8:

[0588] The user checks the displayed image and finalizes the order details. The terminal accepts input to confirm the order.

[0589] Step 9:

[0590] The terminal sends the confirmed order details to the server, which then notifies the seller of the final order details. The seller then produces the product based on the order details and provides it to the user.

[0591] These steps ensure that the user's needs are reflected quickly and accurately, and that the ordering process for custom-made products is smooth.

[0592] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0593] This invention is a system for streamlining the ordering process for custom-made products, and aims to reduce the burden on customers by combining generative AI and image generation AI, and to improve the customer experience by adding an emotion engine that recognizes the user's emotions. The program processing of this system is explained in detail below.

[0594] 1. User requests

[0595] A user accesses the system through a terminal and inputs their request for a custom-made product in natural language. For example, they might input, "I want a simple red silk dress."

[0596] 2. Analyzing requests and generating questions

[0597] The device sends the user's input to the server, which uses generative AI to analyze the user's request and automatically generate questions to obtain the necessary information. For example, it generates questions such as, "Do you have any requests for detailed decorations or special designs?"

[0598] 3. Emotion Recognition by Emotion Engine

[0599] When a user answers a question, the device uses an emotion engine to recognize the emotion of the user's input. For example, when a user types "No decoration needed," the device determines whether the emotion is positive or negative.

[0600] 4. View the question and enter the answer

[0601] The device then displays the generated questions to the user in order based on the analysis results of the emotion engine and receives the user's answers. For example, if the user answers "No decoration is necessary" and this answer is recognized as having a negative sentiment, the server automatically generates additional follow-up questions.

[0602] 5. Adjustment of specifications

[0603] The server adjusts the order specifications based on the user's answers and the results of the emotion engine. For example, it adjusts the specifications to "red silk material with a simple design without decoration." During this process, the meaning of the options and necessary information are explained to the user in an easy-to-understand manner.

[0604] 6. Sending an image generation request

[0605] The server sends the adjusted specifications to the image generation AI and requests it to generate a visual image of the completed product. The image generation AI generates a visual image of the completed product based on the received specifications.

[0606] 7. Creation and display of completed image

[0607] The server receives the generated completed image and sends it to the terminal. The terminal displays the completed image to the user so that the user can confirm it. For example, the user can confirm, "This is the design I had in mind. This is fine."

[0608] 8. Final Order Confirmation

[0609] The user checks the completed image and finalizes the order by entering "I confirm my order" on the terminal.

[0610] 9. Submitting your order

[0611] The server receives the final order and automatically sends it to the seller, who can then create the product based on the specific order.

[0612] This system allows customers to easily communicate their requests in natural language, and an emotion engine enables responses that reflect the customer's emotions. This avoids misunderstandings and dissatisfaction, resulting in a smooth ordering process. It also allows sellers to understand the details of the order in advance, enabling more efficient product production. This system can accept orders from all over the country, which has the effect of expanding the market for custom-made products.

[0613] The processing flow will be explained below.

[0614] This invention is a system for streamlining the ordering process for custom-made products, and aims to reduce the burden on customers by combining generative AI and image generation AI, and to improve the customer experience by adding an emotion engine that recognizes the user's emotions. The details of the program processing of this system are as follows.

[0615] Step 1:

[0616] A user accesses the system through a terminal and inputs their request for a custom-made product in natural language, for example, "I want a simple red silk dress."

[0617] Step 2:

[0618] The device sends the user's input to the server, which uses generative AI to analyze the user's request and automatically generate questions to obtain the necessary information. For example, it generates questions such as, "Do you have any requests for detailed decorations or special designs?"

[0619] Step 3:

[0620] The server sends the generated question list to the terminal, which displays the generated questions to the user in order.

[0621] Step 4:

[0622] The user answers the displayed question. For example, they might type "No decoration necessary." The device uses an emotion engine to recognize the user's emotion along with their answer. For example, if the user answers with a negative emotion, that information is also sent to the server.

[0623] Step 5:

[0624] The device sends the user's answers and emotional data to the server, which then uses generative AI to adjust the order specifications based on the user's answers and emotional data. For example, the specifications may be adjusted to "red silk material with a simple design without decoration."

[0625] Step 6:

[0626] The server generates a request to the image generation AI based on the adjusted specifications, and the image generation AI generates a visual representation of the finished product based on the received specifications.

[0627] Step 7:

[0628] The server receives the generated completed image and sends it to the terminal, which displays the completed image to the user and asks for their confirmation.

[0629] Step 8:

[0630] The user checks the completed image. When the user responds, "This is the design I imagined, I'd like this," the device sends the confirmation to the server.

[0631] Step 9:

[0632] The terminal sends the user's confirmation to the server, which compiles the final order and automatically sends it to the merchant.

[0633] This series of processes allows customers to have a concrete image, and the emotion engine responds appropriately. Furthermore, sellers can understand the detailed order details in advance, enabling efficient product production. This system reduces the burden on both customers and sellers, and realizes an efficient, high-quality custom-made product ordering experience.

[0634] Example 2

[0635] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0636] Traditional custom-made product ordering processes have struggled to accurately understand and efficiently process customer requests. They also lack the ability to properly understand and respond to customer emotions and intent, resulting in final products that don't meet customer expectations. Furthermore, there is a lack of established methods for utilizing image generation technology, resulting in a lack of visual feedback. Therefore, there is a need for a way to improve customer experience and efficiently process orders.

[0637] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0638] In this invention, the server includes means for accepting requests from a user in natural language, means for analyzing the accepted request using a generation AI and automatically generating questions to acquire necessary information, means for presenting the automatically generated questions to the user and receiving answers, means for recognizing the user's emotions using an emotion engine, means for adjusting the order specifications based on the received answers and the emotion recognition results using the generation AI, means for creating an image of the completed product based on the adjusted specifications using an image generation AI, means for presenting the created image of the completed product to the user and finalizing the order details, and means for transmitting the finalized order details to the seller. This makes it possible to accurately understand customer requests and respond in line with the customer's emotions, and by providing visual feedback, it is possible to improve the customer experience and process orders more efficiently.

[0639] The "means for accepting requests in natural language from the user" is an interface that allows the user to input their wishes and requests regarding custom-made products in natural language.

[0640] "Generative AI" is a system that uses artificial intelligence technology to analyze requests received in natural language, extract necessary information, and automatically generate questions.

[0641] "Means for presenting automatically generated questions to the user and receiving answers" refers to an interface that displays questions generated by the generation AI to the user and allows the user to input answers to those questions.

[0642] The "emotion engine" is a system that analyzes the emotions expressed by users when they input or respond in real time, and determines whether those emotions are positive or negative.

[0643] "Means for adjusting order specifications based on responses received using generative AI and emotion recognition results" refers to a system that appropriately adjusts order details and determines final specifications based on responses from users and the analysis results of the emotion engine.

[0644] "Image generation AI" is an artificial intelligence technology for generating visual images of completed products based on tailored order specifications.

[0645] The "means for presenting the completed image drawing to the user and allowing the user to confirm the final order details" is an interface that displays the generated completed image drawing to the user and allows the user to confirm the final order details.

[0646] The "means for transmitting the confirmed order details to the seller" is a system for automatically transmitting the order details confirmed by the user to the seller and issuing instructions for product production.

[0647] "Means for accepting orders from all over the country" refers to a system that allows orders to be accepted from all over the country without being restricted by geography.

[0648] "Means for explaining the meaning of options and suggestions in response to inquiries in natural language" refers to an interface that explains the meaning of options and related suggestions in response to the content of an inquiry made by a user in natural language.

[0649] This invention is a system for streamlining the ordering process for custom-made products and improving customer experience. The system is composed of a user terminal, a server, a generative AI model, an image generation AI, and an emotion engine.

[0650] Hardware and software used:

[0651] Hardware: User devices (PCs, smartphones, tablets, etc.), servers

[0652] Software: Generative AI model, image generation AI, emotion engine

[0653] System process flow and specific explanation:

[0654] 1. User requests

[0655] The user uses the device to open a web browser or dedicated application.

[0656] The terminal displays an input form to the user, and the user inputs their request for a custom-made product in natural language, for example, "I want a simple red silk dress."

[0657] The terminal sends this information to the server.

[0658] 2. Analyzing requests and generating questions

[0659] The server then runs a generative AI model, such as OpenAI's GPT-4, to analyze the received user request.

[0660] The server uses a generative AI model to analyze the request and automatically generate questions to gather additional information.

[0661] For example, generate a question like, "Do you have any requests for detailed decorations or special designs?"

[0662] The server sends the generated question to the terminal.

[0663] 3. Emotion Recognition by Emotion Engine

[0664] When a user answers questions through the device, the device uses an emotion engine to analyze the user's emotions in real time. For example, it uses Affectiva as emotion recognition software.

[0665] For example, if the user answers "No decoration is needed," it is determined whether the answer is positive or negative.

[0666] 4. View the question and enter the answer

[0667] The terminal sequentially displays questions generated based on the results of the emotion engine to the user and receives the user's answers.

[0668] For example, if a user answers "No decoration necessary" and the answer is perceived as negative, the server will automatically generate additional follow-up questions.

[0669] For example, follow-up questions such as "Would you consider other colors or materials?" are generated.

[0670] 5. Adjustment of specifications

[0671] The server adjusts the order specifications based on the user's answers and the results of the emotion engine.

[0672] For example, the specifications are determined as "red silk material with a simple design without decoration."

[0673] During this process, the server explains the meaning of the options and the necessary information to the user in an easy-to-understand manner.

[0674] 6. Sending an image generation request

[0675] The server sends the adjusted specifications to the image generation AI and requests it to generate a completed image. As an example, we will use DALL-E 2.

[0676] The image generation AI generates a visual representation of the finished product based on the received specifications.

[0677] 7. Creation and display of completed image

[0678] The server receives the generated completed image and sends it to the terminal.

[0679] The terminal displays the completed image to the user, who then confirms it, for example by saying, "This is the design I had in mind. This is fine."

[0680] 8. Final Order Confirmation

[0681] The user checks the completed image and finalizes the order details.

[0682] The user inputs "I confirm the order" through the terminal.

[0683] 9. Submitting your order

[0684] The server receives the final confirmed order and automatically sends it to the merchant.

[0685] The seller receives this information and begins producing the product based on the specific order.

[0686] Examples and prompts:

[0687] Examples:

[0688] Suppose a user inputs "a red silk dress," which generates the question "Do you need detailed embellishments?" If the user answers "No embellishments needed," and the emotion engine determines this as negative, an additional question is generated: "Would you consider other colors or materials?" The specifications are then finalized, and an image of the finished product is generated.

[0689] Example prompt sentence:

[0690] Suppose a user orders a custom dress based on the following criteria:

[0691] "I want a simple red silk dress."

[0692] In response, do you have any detailed decorations or special design requests?

[0693] Also, how should we respond if negative sentiment is detected in response to the response that decoration is not necessary?

[0694] The system allows for accurate understanding of customer needs, emotional response, and visual feedback, resulting in an improved customer experience and efficient order processing.

[0695] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0696] Step 1: User input

[0697] Input: User's natural language request

[0698] Specific operation: The user opens a web browser or a dedicated application on the device. The device displays an input form, and the user inputs their requirements for a custom-made product in natural language. For example, they might input, "I want a simple red silk dress."

[0699] Output: User request data

[0700] Detailed description: The terminal receives the user's input and sends this request data to the server.

[0701] Step 2: Analyze the needs and generate questions

[0702] Input: User's request data

[0703] How it works: The server launches a generative AI model (e.g., GPT-4) to analyze the user's request. The generative AI model is used to extract necessary information from the request and automatically generate follow-up questions.

[0704] Output: Auto-generated questions

[0705] Detailed explanation: For example, a question such as "Do you have any requests for detailed decorations or special designs?" is generated, and the server sends this question to the terminal.

[0706] Step 3: Emotion recognition by the emotion engine

[0707] Input: User response data

[0708] Specific operation: When the user answers a question generated through the device, the device launches an emotion engine (e.g., Affectiva) and analyzes the user's emotions in real time.

[0709] Output: Emotion recognition result

[0710] Longer explanation: For example, if a user answers "No decoration needed," the sentiment engine determines whether the answer is positive or negative.

[0711] Step 4: View the question and enter the answer

[0712] Input: Auto-generated question and emotion recognition result

[0713] Specific operation: The device refers to the results of the emotion engine, displays the generated questions to the user in sequence, and receives the user's answers. The user enters answers to each question.

[0714] Output: User response data

[0715] Detailed explanation: For example, if a user answers "I don't need decorations," and the answer is recognized as negative, the server automatically generates an additional follow-up question, such as "Would you consider other colors or materials?"

[0716] Step 5: Adjust the specifications

[0717] Input: User response data and emotion recognition results

[0718] Specific operation: The server adjusts the order specifications based on the user's answers and the results of the emotion engine.

[0719] Output: Adjusted order specifications

[0720] Detailed explanation: For example, the specifications are determined as "red silk material with a simple design without decorations," and the server explains the meaning of the options and necessary information to the user in an easy-to-understand manner during this process.

[0721] Step 6: Sending an image generation request

[0722] Input: Adjusted order specifications

[0723] Specific operation: The server sends the adjusted specifications to the image generation AI (e.g., DALL-E 2) and requests the generation of a completed image.

[0724] Output: Finished image

[0725] Detailed description: Image generation AI generates a visual representation of the finished product based on the received specifications.

[0726] Step 7: Create and display a finished image

[0727] Input: Image of completed product

[0728] Specific operation: The server receives the generated completed image and sends it to the terminal, which displays the image to the user so that the user can check it.

[0729] Output: Final confirmation by the user

[0730] Detailed explanation: For example, the user confirms, "This is the design I envisioned, I'd like this."

[0731] Step 8: Finalize your order

[0732] Input: Final confirmation by the user

[0733] Specific operation: The user checks the completed image and confirms the final order details. The user enters "I confirm the order" on the terminal.

[0734] Output: Confirmed order details

[0735] Detailed description: The order details confirmed by the user are sent to the server.

[0736] Step 9: Submit your order

[0737] Input: Confirmed order details

[0738] What happens: The server receives the final confirmed order details and automatically sends them to the merchant.

[0739] Output: Order received by merchant

[0740] Detailed Description: The seller will receive this information and begin producing the product based on your specific order.

[0741] This system allows customers to easily communicate their needs in natural language and uses an emotion engine to respond appropriately based on the customer's emotions, resulting in a smooth ordering process. It also allows sellers to efficiently produce products by knowing the details of the order in advance.

[0742] (Application example 2)

[0743] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0744] The modern ordering process for custom-made products is extremely time-consuming and tedious, requiring customers to visit a store and provide detailed requests. Furthermore, requests are collected without taking into account the customer's emotional state, which can easily lead to customer dissatisfaction and misunderstandings. Furthermore, there is a lack of visual feedback, making it difficult for customers to visualize the final product. These issues significantly impair the customer experience, especially in physical stores.

[0745] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for accepting a request from a user in natural language; means for automatically generating questions to acquire necessary information by analyzing the accepted request using a generation AI; means for presenting the automatically generated questions to the user and receiving answers; means for adjusting the order specifications based on the received answers using the generation AI; means for creating an image of the completed product based on the adjusted specifications using an image generation AI; means for presenting the created image of the completed product to the user and finalizing the order details; means for transmitting the finalized order details to the seller; means for analyzing the user's emotions at the time of user input using an emotion engine that recognizes the user's emotions; means for analyzing the customer's emotions in real time and automatically generating follow-up questions; and means for presenting information to the user using a glasses-type terminal and confirming the order process. This realizes an ordering process that takes customer emotions into consideration, enabling intuitive and efficient ordering of custom-made products.

[0746] The "means for accepting requests from users in natural language" is a system that allows customers to input details of the custom-made products they desire in natural language via voice or text.

[0747] "Generative AI" is a system that uses artificial intelligence to analyze user input and generate relevant questions and content.

[0748] The "means for automatically generating questions" is a system that has the function of automatically generating questions to acquire necessary information based on the user's request.

[0749] The "means for presenting automatically generated questions to users and receiving answers" is a mechanism for displaying automatically generated questions to users and collecting answers from the users.

[0750] The "means for adjusting the specifications of the order" is a system that has an adjustment function for determining the final product specifications based on the user's responses.

[0751] "Image generation AI" is an artificial intelligence that generates a rendering of a finished product based on adjusted specifications.

[0752] The "means for creating a completed image drawing" is a system having a function for creating and saving the generated completed image drawing.

[0753] "Means of presenting the completed image to the user and finalizing the order details" refers to a mechanism that shows the user an image of the completed product and leads them through the process of confirming and confirming the final order details.

[0754] The "means for transmitting the confirmed order details to the seller" is a mechanism for automatically transferring the confirmed order details to the seller.

[0755] The "emotion engine" is a system for analyzing the emotions of users in real time when they input their emotions.

[0756] The "means for analyzing customer emotions in real time and automatically generating follow-up questions" is a system that analyzes the emotional state of a customer and automatically generates additional questions as needed.

[0757] An "eyeglasses-type terminal" is a wearable device that visually provides information to a user.

[0758] The present invention is a system for enabling users to streamline the ordering process for custom-made products using natural language, and can be implemented as follows.

[0759] 1. User requests

[0760] The user wears the glasses and inputs their requirements for a custom-made product using voice or text. For example, they might say, "I'd like a simple red silk dress." The microphone in the glasses captures the voice and sends the data to a smartphone connected via Bluetooth or to the store's server. This uses a speech recognition API (for example, Google Cloud Speech-to-Text).

[0761] 2. Analyzing requests and generating questions

[0762] The server analyzes the received voice data and uses a generative AI (such as OpenAI's GPT-4) to analyze the user's requests. As a result of the analysis, questions are automatically generated to obtain the necessary information. For example, a question such as "Do you have any requests for detailed decorations or special designs?" is generated.

[0763] 3. Emotion Recognition by Emotion Engine

[0764] As the user answers the questions, the camera on the glasses uses facial expression recognition software (for example, the emotion recognition API from Microsoft Azure Cognitive Services) to recognize the user's emotions in real time. For example, when the user types "No decoration needed," it determines whether the emotion is positive or negative.

[0765] 4. View the question and enter the answer

[0766] The generated questions are displayed on the display of the glasses-type device, and the user answers using voice or a touch interface. If the emotion engine detects a negative emotion, the server automatically generates additional follow-up questions.

[0767] 5. Adjustment of specifications

[0768] The server adjusts the order specifications based on the user's answers and the results of the emotion engine. For example, it may adjust the specifications to "red silk material with a simple design without decoration," and also explain the meaning of the options and necessary information to the user in an easy-to-understand manner.

[0769] 6. Sending an image generation request

[0770] The adjusted specifications are sent to an image generation AI (e.g., DALL-E) and a request is made to generate a visual image of the completed product. The image generation AI generates a visual image of the completed product based on the received specifications.

[0771] 7. Creation and display of completed image

[0772] The server receives the generated completed image and sends it to the eyeglasses. The eyeglasses display the completed image to the user, allowing them to confirm it. For example, the user can confirm, "This is the design I had in mind. This is fine."

[0773] 8. Final Order Confirmation

[0774] The user checks the completed image and confirms the final order details by voice command. The data from the glasses-type device is sent to the server.

[0775] 9. Submitting your order

[0776] The server receives the final order and automatically sends it to the seller, who can then create the product based on the specific order.

[0777] Specific examples

[0778] A user enters a physical store, puts on the eyeglasses-type device, and says, "I want a simple red silk dress."

[0779] The glasses-type device displays a question, asking, "Do you have any decoration preferences?"

[0780] If the user answers "Nothing in particular," the emotion engine recognizes this as positive.

[0781] The final specifications were decided as "red silk material with a simple design without any decoration."

[0782] Image generation AI generates visual images and displays them on the glasses-type device.

[0783] The user checks the image and confirms the final order by saying, "This is it."

[0784] Prompt Sentence Examples

[0785] "I want a simple red silk dress."

[0786] "Do you have any decoration preferences?"

[0787] "Nothing in particular."

[0788] This creates an ordering process that takes customer emotions into account, making ordering custom-made products intuitive and efficient.

[0789] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0790] Step 1:

[0791] The user wears the eyeglasses and inputs their requests for custom-made products by voice or text. The microphone in the eyeglasses captures the voice and sends the data to a smartphone connected via Bluetooth or to the store's server. This input is done by the user saying, for example, "I want a simple red silk dress." The server converts the voice data into text data using a speech recognition API (for example, Google Cloud Speech-to-Text). This converts the voice input (input data) into text data (output data).

[0792] Step 2:

[0793] The server analyzes the received text data and uses a generative AI (for example, OpenAI's GPT-4) to analyze the user's request. As a result of the analysis, questions are automatically generated to obtain the required information. For example, a question such as "Do you have any requests for detailed decorations or special designs?" is generated. This involves the process in which the generative AI generates questions (output data) based on the user's request, which is input.

[0794] Step 3:

[0795] The display on the glasses-type device presents automatically generated questions to the user. The user answers the questions using voice or a touch interface. The server receives the answers as text data. In this step, the user answers "nothing in particular" to the questions, and the input data is sent to the server.

[0796] Step 4:

[0797] When a user answers a question, the camera on the glasses uses facial expression recognition software (e.g., Microsoft Azure Cognitive Services' Emotion Recognition API) to recognize the user's emotions in real time. For example, the emotion engine detects positive emotions in response to the user's answer. In this step, real-time facial expression data is used as input data, and emotion identification results are obtained as output data.

[0798] Step 5:

[0799] The server adjusts the order specifications based on the user's answers and the results of the emotion engine. For example, it determines the specifications as "red silk material with a simple design without decoration." This adjustment is done using generative AI, and includes a process of determining the specifications (output data) from the input data of the answers and emotion data.

[0800] Step 6:

[0801] The server sends the finalized specifications to an image generation AI (e.g., DALL-E) and requests it to generate a visual representation of the completed image. Based on the received specifications, the image generation AI generates a visual representation of the completed image (output data). The input data in this step are the finalized specifications.

[0802] Step 7:

[0803] The server receives the generated completed image and sends it to the eyeglasses. The eyeglasses display the completed image on the screen for the user to confirm. The user then looks at the image and confirms, "This is fine." The input data in this step is the generated completed image, and the output data is the user's final confirmation.

[0804] Step 8:

[0805] The user checks the completed image and confirms the final order details by voice command. The voice command data from the glasses-type device is sent to the server. After this confirmation, the server confirms the final order details (output data).

[0806] Step 9:

[0807] The server receives the final order and automatically sends it to the seller, so that the seller can create the product based on the specific order. In this step, the input data is the finalized order, and the output data is the data to be sent to the seller.

[0808] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0809] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0810] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0811] [Third embodiment]

[0812] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0813] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0814] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0815] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0816] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0817] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0818] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0819] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0820] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0821] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0822] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0823] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0824] This invention relates to a system for streamlining the ordering process for custom-made products, and aims to reduce the burden on customers by combining generative AI and image generation AI. The program for this system operates as follows.

[0825] 1. User requests

[0826] A user accesses the system through a terminal and inputs their request for a custom-made product in natural language, including details of the product type, color, material, and design. For example, they might input, "I want a simple red silk dress."

[0827] 2. Analyzing requests and generating questions

[0828] The server uses AI to analyze the user's requests and automatically generate questions to obtain the necessary information. For example, a generated question might be, "Do you have any requests for detailed decorations or special designs?"

[0829] 3. View the question and enter the answer

[0830] The device presents the user with an automatically generated question, and the user answers the question, for example, "No decoration is necessary."

[0831] 4. Adjustment of specifications

[0832] The server adjusts the order specifications based on the user's answers. For example, the specifications may be adjusted to "red silk material with a simple design without decoration." During this process, the meaning of the options and necessary information are explained to the user in an easy-to-understand manner.

[0833] 5. Sending an image generation request

[0834] The server sends the adjusted specifications to the image generation AI and requests it to generate an image of the finished product.

[0835] 6. Creating a completed image

[0836] Based on the specifications received, the image generation AI generates a visual representation of the finished product, which concretely shows what the product will look like.

[0837] 7. Display and check the image

[0838] The terminal displays the generated completed image to the user, who then confirms it, for example, by saying, "This image is fine."

[0839] 8. Final Order Confirmation

[0840] The user checks the completed image and finalizes the order by entering "I confirm my order" on the terminal.

[0841] 9. Submitting your order

[0842] The server receives the final order and automatically sends it to the seller, who can then create the product based on the specific order.

[0843] This system allows customers to easily communicate their needs in natural language and gives them a concrete image of the finished product through questions and illustrations generated along the way. This helps avoid misunderstandings and dissatisfaction, resulting in a smooth ordering process. It also allows sellers to understand the detailed order details in advance, enabling efficient product production. This system can accept orders from all over the country, which has the effect of expanding the market for custom-made products.

[0844] The processing flow will be explained below.

[0845] Step 1:

[0846] The user accesses the system through a terminal and inputs their desired custom-made product in natural language on the system's input screen. For example, "I want a simple red silk dress."

[0847] Step 2:

[0848] The device sends the user's input to the server. The server uses generative AI to analyze the user's requests. As a result of the analysis, a list of questions is automatically generated to obtain the necessary information. For example, questions such as "Do you have any requests for detailed decorations or special designs?" are generated.

[0849] Step 3:

[0850] The server sends the generated question list to the terminal. The terminal displays the questions to the user in order, and the user inputs an answer to each question. For example, the user might answer, "No decoration is necessary."

[0851] Step 4:

[0852] The device sends the user's answers to the server. The server uses generative AI to adjust the order specifications based on the answers received from the user. During this process, the meaning of the options and necessary information are explained to the user in an easy-to-understand manner. For example, the specifications are adjusted to "red silk material with a simple design without decoration."

[0853] Step 5:

[0854] The server sends the adjusted specifications to the image generation AI and requests it to generate a visual image of the completed product. The image generation AI generates a visual image of the completed product based on the received specifications.

[0855] Step 6:

[0856] The server receives the generated completed image and sends it to the terminal. The terminal displays the completed image to the user and asks for confirmation. The user confirms, "This is the design I envisioned. This is fine."

[0857] Step 7:

[0858] The terminal sends the user's confirmation to the server, which compiles the final order and automatically sends it to the seller, who can then create the product based on the specific order.

[0859] This series of steps reduces the burden on both the customer and the seller, resulting in an efficient, high-quality custom product ordering process.

[0860] Example 1

[0861] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0862] During the custom-made product ordering process, customers often have difficulty effectively communicating their desired details. This can lead to dissatisfaction with the finished product, and it can be difficult for sellers to accurately understand the customer's needs. Additionally, the process of confirming specific design elements and materials can be cumbersome and time-consuming. Furthermore, there is a need to provide appropriate options and suggestions to meet the user's needs.

[0863] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0864] In this invention, the server includes means for accepting requests from users in natural language, means for analyzing the accepted requests using a generative AI model and automatically generating questions to acquire necessary information, and means for creating a completed image based on the adjusted specifications using an image generation AI. This allows the server to efficiently analyze the user's requests and provide a concrete image of the completed product, thereby reducing dissatisfaction with the finished product and enabling the seller to accurately understand the detailed order content.

[0865] "User" refers to any individual or legal entity that uses the System to place an order for a custom-made product.

[0866] "Natural language" refers to a language used by humans on a daily basis, a language that has grammar and meaning rather than specific program code.

[0867] A "generative AI model" refers to a system and its algorithms that use artificial intelligence technology to analyze natural language and generate text.

[0868] "Request" refers to the specific requirements and wishes that a user has for a custom-made product.

[0869] "Means for automatically generating questions" refers to a function that uses a generative AI model to automatically create questions to gather information necessary to further clarify a user's needs.

[0870] An "answer" refers to specific information that a user provides in response to a posed question.

[0871] "Means to tailor order specifications" refers to the ability to use generative AI models to determine the specific design and attributes of custom-made products based on user responses.

[0872] "Image generation AI" refers to a system or algorithm that uses artificial intelligence technology to generate visual images from text information.

[0873] A "finished image" refers to a visual representation of the appearance of a custom-made product, generated by image generation AI.

[0874] "Seller" refers to an individual or legal entity that manufactures and sells custom-made products upon receiving orders from users.

[0875] "Means of explaining the meaning of options and suggestions in response to inquiries in natural language" refers to a function that uses a generative AI model to explain appropriate options and suggestions in response to a user's natural language questions.

[0876] "Prompt" refers to a textual instruction that is required of an image generation AI to generate a specific image.

[0877] This invention is a system that streamlines the ordering process for custom-made products, and aims to reduce the burden on customers by combining generative AI models and image generation AI. The system program operates as follows.

[0878] User request input

[0879] A user accesses the system using a terminal and inputs their request for a custom-made product in natural language. This input includes details of the product type, color, material, and design. For example, they might input, "I want a simple red silk dress." The terminal then sends this request to the server.

[0880] Analyzing requests and generating questions

[0881] The server analyzes the received user request using a generative AI model (for example, a general natural language processing model). Based on the analysis results, the server generates a question to collect the necessary additional information. For example, it generates a question such as, "Do you have any requests for detailed decorations or special designs?" This generated question is then sent to the device.

[0882] View questions and enter answers

[0883] The terminal displays the generated question to the user, who responds by typing, for example, "No decoration necessary." The terminal then sends the response to the server.

[0884] Specification adjustment

[0885] The server adjusts the order specifications based on the user's answers. For example, the specifications could be set to "red silk material with a simple design without decoration." During this process, the server generates text that clearly explains the meaning of the options and necessary information to the user, and sends it to the terminal.

[0886] Sending an image generation request

[0887] Based on the adjusted specifications, the server creates a prompt to send to the image generation AI (for example, a general image generation algorithm). For example, "Please generate an image of a simple dress made of red silk." This prompt is sent to the image generation AI.

[0888] Creating a completed image

[0889] Based on the specified prompts, the image generation AI generates a visual representation of the finished product, which concretely shows the appearance of the product. The generated representation is then sent to the server.

[0890] Display and check the image diagram

[0891] The terminal displays the completed image received from the server to the user. The user checks this image and confirms, for example, "This image is OK." The confirmation result is sent to the server.

[0892] Final order confirmation

[0893] After the user checks the completed image, they input "I confirm my order" through their terminal. This input is sent to the server.

[0894] Sending order details

[0895] The server receives the final order and automatically sends it to the seller, who can then create the product based on the order.

[0896] This system allows users to easily order custom-made products using natural language and confirm specific specifications based on generated questions and finished product images. This avoids misunderstandings and frustrations, resulting in a smooth ordering experience. Furthermore, sellers can efficiently produce products by knowing the details of the order in advance.

[0897] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0898] Step 1:

[0899] A user accesses the system using a terminal and inputs their request for a custom-made product in natural language. This input includes details of the product type, color, material, and design. For example, a user might input, "I want a simple red silk dress." The terminal sends this request as input data to the server. The output data generated is the user's requested information.

[0900] Step 2:

[0901] The server uses a generative AI model to analyze the user's request data it receives. Specifically, the server inputs the request data into the generative AI model and automatically generates questions to obtain the necessary additional information. For example, the server generates a question such as, "Do you have any requests for detailed decorations or special designs?" The generated question is sent from the server to the device as output data.

[0902] Step 3:

[0903] The terminal displays the question sent from the server to the user. The user inputs an answer to this question. For example, the user inputs "No decoration is necessary." The terminal sends this answer data as input to the server. The user's answer is generated as output data.

[0904] Step 4:

[0905] The server adjusts the order specifications based on the user's response data received. Specifically, the server uses a generative AI model to analyze the response data and set the specifications to "a simple design with no decorations in red silk." During this process, it also generates text to clearly explain the meaning of the options and necessary information, and sends it to the device. The output data generated is the adjusted order specifications and explanatory text.

[0906] Step 5:

[0907] Based on the adjusted specifications, the server creates a prompt to send to the image generation AI. For example, the prompt might be, "Please generate an image of a simple dress made of red silk." After the prompt is generated, the server sends it to the image generation AI. The prompt is generated and sent as output data.

[0908] Step 6:

[0909] The image generation AI generates a visual image of the finished product based on the prompt received from the server. The prompt is input into the generation AI, and a specific visual image of the dress is generated as output. The generated image is sent to the server.

[0910] Step 7:

[0911] The terminal displays the completed image sent from the server to the user. The user checks this image and enters, for example, "This image is fine." The terminal sends the user's confirmation results to the server as input data. The confirmation results and the user's decision to confirm are generated as output data.

[0912] Step 8:

[0913] After the user has made the final confirmation, they confirm the order details through the terminal. For example, they may input "I confirm my order." This input is sent to the server, which receives the final order details. The confirmed order details are generated as output data.

[0914] Step 9:

[0915] The server automatically sends the final order details to the seller, who can then create the product based on the order details. The output data is a notification of the order details to the seller.

[0916] This allows users to easily order custom-made products using natural language and confirm specific specifications based on generated questions and finished product images. Sellers can also efficiently produce products by knowing the details of the order in advance.

[0917] (Application example 1)

[0918] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0919] The ordering process for custom-made products involves many steps, which can lead to problems such as miscommunication between the user and the seller and complicated procedures. Furthermore, it is often difficult for users to form a concrete image of the product, and it often takes a lot of effort to obtain a satisfactory product. Therefore, there is a need for a system that can flexibly respond to user requests while efficiently carrying out the ordering process.

[0920] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0921] In this invention, the server includes: means for accepting requests from a user in natural language; means for analyzing the accepted request using a generative AI model and automatically generating questions to acquire necessary information; means for presenting the automatically generated questions to the user and receiving responses; means for adjusting the order specifications based on the received responses; means for creating a completed image based on the adjusted specifications using an image generation AI; means for presenting the created completed image to the user and finalizing the order details; means for transmitting the confirmed order details to the seller; means for requesting an image from the generative AI model and generating an image generation prompt based on the order details; and means for requesting the image based on the generated prompt. This allows the user to view a specific image simply by entering their request in natural language, enabling a smooth ordering process without misunderstandings. Furthermore, the use of application software compatible with smart devices can further improve user convenience.

[0922] "User" means any individual or legal entity that uses the System to order a custom-made product.

[0923] "Natural language" refers to the language that users use in their everyday communication, and is not a specific programming language, but rather the words that humans use on a daily basis.

[0924] A "generative AI model" refers to an algorithm that uses artificial intelligence technology to analyze user requests and automatically generate questions to obtain the necessary information.

[0925] "Analysis" is the process of deciphering input natural language and extracting important information from it.

[0926] "Automatic question generation" means using artificial intelligence technology to deeply understand a user's needs and create appropriate questions to solicit additional information.

[0927] "Means for receiving answers" refers to a function for receiving answers entered by a user in response to questions presented by the system.

[0928] "Order specifications" refers to the specific characteristics and features of the product that are ultimately determined based on the user's requests and responses.

[0929] "Image generation AI" refers to algorithms that use artificial intelligence technology to generate visual images based on received specifications.

[0930] "Completed image" refers to a visual representation of a product created by image generation AI based on specifications.

[0931] "Means for finalizing the order" refers to a function that allows the user to check the generated image and finalize it as the final order.

[0932] "Seller" refers to an individual or legal entity that produces and provides products to users based on confirmed orders.

[0933] "Application software compatible with smart devices" refers to a program that runs on smart devices such as smartphones and tablets and provides the functions necessary for users to order custom-made products.

[0934] A "prompt statement" refers to a command statement that uses a generative AI model to generate an image diagram for an image generation AI.

[0935] The system for realizing this invention uses the following hardware and software: The hardware uses a smart device such as a smartphone or tablet. The software uses an application developed using Android Studio or Xcode, a generative AI model (GPT-4), and an image generation AI (DALL-E or Stable Diffusion).

[0936] System configuration

[0937] The system includes the following programs:

[0938] 1. A means of accepting requests from users in natural language

[0939] 2. A means of using generative AI models to analyze received requests and automatically generate questions to obtain the necessary information.

[0940] 3. A means of presenting automatically generated questions to users and receiving answers

[0941] 4. A means to adjust the specifications of the order based on the response received.

[0942] 5. A method to use image generation AI to create a finished image based on adjusted specifications

[0943] 6. A means to present the completed image to the user and finalize the order details

[0944] 7. How to send the confirmed order to the seller

[0945] 8. A method to request an image from the AI ​​model and generate an image generation prompt based on the order details

[0946] 9. A method for requesting an image based on the generated prompt

[0947] Program processing explanation

[0948] User request input

[0949] Users access the application using their smart device and enter their custom product requirements in natural language, including details of product type, color, material, and design.

[0950] Analyzing requests and generating questions

[0951] The server uses a generative AI model (GPT-4) to analyze the user's request. Based on the analysis results, it automatically generates questions to complement the information. For example, if the request is "I want a simple red silk dress," the server generates a related question: "Do you want any detailed decorations or special designs?"

[0952] View questions and enter answers

[0953] The automatically generated questions are presented to the user, who answers them via their smart device, for example, "No decoration is needed."

[0954] Specification adjustment

[0955] Based on the user's response, the server adjusts the order specifications, for example, a red silk dress with a simple design without any decorations.

[0956] Sending an image generation request

[0957] Based on the adjusted specifications, the server generates a request to the image generation AI. First, the generative AI model generates a prompt sentence. For example, the prompt sentence is generated as "Create an image of a simple, red silk dress with no decorations."

[0958] Creating and confirming a completed image

[0959] The image generation AI (DALL-E or Stable Diffusion) creates an image based on the generated prompt. This image is displayed to the user for confirmation.

[0960] Finalize and submit your order

[0961] The user checks the image and finalizes the order details, which are then sent to the seller by the server.

[0962] Specific examples

[0963] For example, a user might input, "I want a simple red silk dress." The generative AI model analyzes this and generates the question, "Do you have any requests for detailed decorations or special designs?" The user responds, "No decorations necessary." Based on this information, the server determines the specifications as "a dress made of red silk with a simple design and no decorations," and sends the prompt "Create an image of a simple, red silk dress with no decorations." The generated image is displayed to the user, and the final order is confirmed.

[0964] In this way, the user's wishes can be quickly and accurately reflected, and a smooth ordering process for custom-made products can be realized.

[0965] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0966] Step 1:

[0967] Users access the application using their smart device and input their custom product requirements in natural language, including details of product type, color, material, and design.

[0968] Step 2:

[0969] The device sends the input request to a server. The server uses a generative AI model (GPT-4) to analyze the received request and automatically generate questions to obtain the necessary information. For example, in response to a request for a simple red silk dress, the server generates the question, "Do you have any requests for detailed decorations or special designs?"

[0970] Step 3:

[0971] The server sends an automatically generated question to the terminal. The terminal displays the question to the user. The user then inputs an answer to the displayed question. For example, the user might answer, "No decoration is necessary."

[0972] Step 4:

[0973] The terminal sends the user's response to the server, which then uses the response to adjust the order specifications. Specifically, the server determines the specific characteristics of the product based on the user's request and response. For example, the specifications may be adjusted to "a simple, unadorned dress made of red silk."

[0974] Step 5:

[0975] The server uses a generative AI model to generate image prompts based on the adjusted specifications. For example, the prompt might read, "Create an image of a simple, red silk dress with no decorations."

[0976] Step 6:

[0977] The server sends the generated prompt to an image generation AI (DALL-E or Stable Diffusion), which generates a visual image based on the prompt.

[0978] Step 7:

[0979] The server sends the generated image to the terminal, which displays it to the user and asks for confirmation, for example, by saying, "This image is OK."

[0980] Step 8:

[0981] The user checks the displayed image and finalizes the order details. The terminal accepts input to confirm the order.

[0982] Step 9:

[0983] The terminal sends the confirmed order details to the server, which then notifies the seller of the final order details. The seller then produces the product based on the order details and provides it to the user.

[0984] These steps ensure that the user's needs are reflected quickly and accurately, and that the ordering process for custom-made products is smooth.

[0985] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0986] This invention is a system for streamlining the ordering process for custom-made products, and aims to reduce the burden on customers by combining generative AI and image generation AI, and to improve the customer experience by adding an emotion engine that recognizes the user's emotions. The program processing of this system is explained in detail below.

[0987] 1. User requests

[0988] A user accesses the system through a terminal and inputs their request for a custom-made product in natural language. For example, they might input, "I want a simple red silk dress."

[0989] 2. Analyzing requests and generating questions

[0990] The device sends the user's input to the server, which uses generative AI to analyze the user's request and automatically generate questions to obtain the necessary information. For example, it generates questions such as, "Do you have any requests for detailed decorations or special designs?"

[0991] 3. Emotion Recognition by Emotion Engine

[0992] When a user answers a question, the device uses an emotion engine to recognize the emotion of the user's input. For example, when a user types "No decoration needed," the device determines whether the emotion is positive or negative.

[0993] 4. View the question and enter the answer

[0994] The device then displays the generated questions to the user in order based on the analysis results of the emotion engine and receives the user's answers. For example, if the user answers "No decoration is necessary" and this answer is recognized as having a negative sentiment, the server automatically generates additional follow-up questions.

[0995] 5. Adjustment of specifications

[0996] The server adjusts the order specifications based on the user's answers and the results of the emotion engine. For example, it adjusts the specifications to "red silk material with a simple design without decoration." During this process, the meaning of the options and necessary information are explained to the user in an easy-to-understand manner.

[0997] 6. Sending an image generation request

[0998] The server sends the adjusted specifications to the image generation AI and requests it to generate a visual image of the completed product. The image generation AI generates a visual image of the completed product based on the received specifications.

[0999] 7. Creation and display of completed image

[1000] The server receives the generated completed image and sends it to the terminal. The terminal displays the completed image to the user so that the user can confirm it. For example, the user can confirm, "This is the design I had in mind. This is fine."

[1001] 8. Final Order Confirmation

[1002] The user checks the completed image and finalizes the order by entering "I confirm my order" on the terminal.

[1003] 9. Submitting your order

[1004] The server receives the final order and automatically sends it to the seller, who can then create the product based on the specific order.

[1005] This system allows customers to easily communicate their requests in natural language, and an emotion engine enables responses that reflect the customer's emotions. This avoids misunderstandings and dissatisfaction, resulting in a smooth ordering process. It also allows sellers to understand the details of the order in advance, enabling more efficient product production. This system can accept orders from all over the country, which has the effect of expanding the market for custom-made products.

[1006] The processing flow will be explained below.

[1007] This invention is a system for streamlining the ordering process for custom-made products, and aims to reduce the burden on customers by combining generative AI and image generation AI, and to improve the customer experience by adding an emotion engine that recognizes the user's emotions. The details of the program processing of this system are as follows.

[1008] Step 1:

[1009] A user accesses the system through a terminal and inputs their request for a custom-made product in natural language, for example, "I want a simple red silk dress."

[1010] Step 2:

[1011] The device sends the user's input to the server, which uses generative AI to analyze the user's request and automatically generate questions to obtain the necessary information. For example, it generates questions such as, "Do you have any requests for detailed decorations or special designs?"

[1012] Step 3:

[1013] The server sends the generated question list to the terminal, which displays the generated questions to the user in order.

[1014] Step 4:

[1015] The user answers the displayed question. For example, they might type "No decoration necessary." The device uses an emotion engine to recognize the user's emotion along with their answer. For example, if the user answers with a negative emotion, that information is also sent to the server.

[1016] Step 5:

[1017] The device sends the user's answers and emotional data to the server, which then uses generative AI to adjust the order specifications based on the user's answers and emotional data. For example, the specifications may be adjusted to "red silk material with a simple design without decoration."

[1018] Step 6:

[1019] The server generates a request to the image generation AI based on the adjusted specifications, and the image generation AI generates a visual representation of the finished product based on the received specifications.

[1020] Step 7:

[1021] The server receives the generated completed image and sends it to the terminal, which displays the completed image to the user and asks for their confirmation.

[1022] Step 8:

[1023] The user checks the completed image. When the user responds, "This is the design I imagined, I'd like this," the device sends the confirmation to the server.

[1024] Step 9:

[1025] The terminal sends the user's confirmation to the server, which compiles the final order and automatically sends it to the merchant.

[1026] This series of processes allows customers to have a concrete image, and the emotion engine responds appropriately. Furthermore, sellers can understand the detailed order details in advance, enabling efficient product production. This system reduces the burden on both customers and sellers, and realizes an efficient, high-quality custom-made product ordering experience.

[1027] Example 2

[1028] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1029] Traditional custom-made product ordering processes have struggled to accurately understand and efficiently process customer requests. They also lack the ability to properly understand and respond to customer emotions and intent, resulting in final products that don't meet customer expectations. Furthermore, there is a lack of established methods for utilizing image generation technology, resulting in a lack of visual feedback. Therefore, there is a need for a way to improve customer experience and efficiently process orders.

[1030] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1031] In this invention, the server includes means for accepting requests from a user in natural language, means for analyzing the accepted request using a generation AI and automatically generating questions to acquire necessary information, means for presenting the automatically generated questions to the user and receiving answers, means for recognizing the user's emotions using an emotion engine, means for adjusting the order specifications based on the received answers and the emotion recognition results using the generation AI, means for creating an image of the completed product based on the adjusted specifications using an image generation AI, means for presenting the created image of the completed product to the user and finalizing the order details, and means for transmitting the finalized order details to the seller. This makes it possible to accurately understand customer requests and respond in line with the customer's emotions, and by providing visual feedback, it is possible to improve the customer experience and process orders more efficiently.

[1032] The "means for accepting requests in natural language from the user" is an interface that allows the user to input their wishes and requests regarding custom-made products in natural language.

[1033] "Generative AI" is a system that uses artificial intelligence technology to analyze requests received in natural language, extract necessary information, and automatically generate questions.

[1034] "Means for presenting automatically generated questions to the user and receiving answers" refers to an interface that displays questions generated by the generation AI to the user and allows the user to input answers to those questions.

[1035] The "emotion engine" is a system that analyzes the emotions expressed by users when they input or respond in real time, and determines whether those emotions are positive or negative.

[1036] "Means for adjusting order specifications based on responses received using generative AI and emotion recognition results" refers to a system that appropriately adjusts order details and determines final specifications based on responses from users and the analysis results of the emotion engine.

[1037] "Image generation AI" is an artificial intelligence technology for generating visual images of completed products based on tailored order specifications.

[1038] The "means for presenting the completed image drawing to the user and allowing the user to confirm the final order details" is an interface that displays the generated completed image drawing to the user and allows the user to confirm the final order details.

[1039] The "means for transmitting the confirmed order details to the seller" is a system for automatically transmitting the order details confirmed by the user to the seller and issuing instructions for product production.

[1040] "Means for accepting orders from all over the country" refers to a system that allows orders to be accepted from all over the country without being restricted by geography.

[1041] "Means for explaining the meaning of options and suggestions in response to inquiries in natural language" refers to an interface that explains the meaning of options and related suggestions in response to the content of an inquiry made by a user in natural language.

[1042] This invention is a system for streamlining the ordering process for custom-made products and improving customer experience. The system is composed of a user terminal, a server, a generative AI model, an image generation AI, and an emotion engine.

[1043] Hardware and software used:

[1044] Hardware: User devices (PCs, smartphones, tablets, etc.), servers

[1045] Software: Generative AI model, image generation AI, emotion engine

[1046] System process flow and specific explanation:

[1047] 1. User requests

[1048] The user uses the device to open a web browser or dedicated application.

[1049] The terminal displays an input form to the user, and the user inputs their request for a custom-made product in natural language, for example, "I want a simple red silk dress."

[1050] The terminal sends this information to the server.

[1051] 2. Analyzing requests and generating questions

[1052] The server then runs a generative AI model, such as OpenAI's GPT-4, to analyze the received user request.

[1053] The server uses a generative AI model to analyze the request and automatically generate questions to gather additional information.

[1054] For example, generate a question like, "Do you have any requests for detailed decorations or special designs?"

[1055] The server sends the generated question to the terminal.

[1056] 3. Emotion Recognition by Emotion Engine

[1057] When a user answers questions through the device, the device uses an emotion engine to analyze the user's emotions in real time. For example, it uses Affectiva as emotion recognition software.

[1058] For example, if the user answers "No decoration is needed," it is determined whether the answer is positive or negative.

[1059] 4. View the question and enter the answer

[1060] The terminal sequentially displays questions generated based on the results of the emotion engine to the user and receives the user's answers.

[1061] For example, if a user answers "No decoration necessary" and the answer is perceived as negative, the server will automatically generate additional follow-up questions.

[1062] For example, follow-up questions such as "Would you consider other colors or materials?" are generated.

[1063] 5. Adjustment of specifications

[1064] The server adjusts the order specifications based on the user's answers and the results of the emotion engine.

[1065] For example, the specifications are determined as "red silk material with a simple design without decoration."

[1066] During this process, the server explains the meaning of the options and the necessary information to the user in an easy-to-understand manner.

[1067] 6. Sending an image generation request

[1068] The server sends the adjusted specifications to the image generation AI and requests it to generate a completed image. As an example, we will use DALL-E 2.

[1069] The image generation AI generates a visual representation of the finished product based on the received specifications.

[1070] 7. Creation and display of completed image

[1071] The server receives the generated completed image and sends it to the terminal.

[1072] The terminal displays the completed image to the user, who then confirms it, for example by saying, "This is the design I had in mind. This is fine."

[1073] 8. Final Order Confirmation

[1074] The user checks the completed image and finalizes the order details.

[1075] The user inputs "I confirm the order" through the terminal.

[1076] 9. Submitting your order

[1077] The server receives the final confirmed order and automatically sends it to the merchant.

[1078] The seller receives this information and begins producing the product based on the specific order.

[1079] Examples and prompts:

[1080] Examples:

[1081] Suppose a user inputs "a red silk dress," which generates the question "Do you need detailed embellishments?" If the user answers "No embellishments needed," and the emotion engine determines this as negative, an additional question is generated: "Would you consider other colors or materials?" The specifications are then finalized, and an image of the finished product is generated.

[1082] Example prompt sentence:

[1083] Suppose a user orders a custom dress based on the following criteria:

[1084] "I want a simple red silk dress."

[1085] In response, do you have any detailed decorations or special design requests?

[1086] Also, how should we respond if negative sentiment is detected in response to the response that decoration is not necessary?

[1087] The system allows for accurate understanding of customer needs, emotional response, and visual feedback, resulting in an improved customer experience and efficient order processing.

[1088] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1089] Step 1: User input

[1090] Input: User's natural language request

[1091] Specific operation: The user opens a web browser or a dedicated application on the device. The device displays an input form, and the user inputs their requirements for a custom-made product in natural language. For example, they might input, "I want a simple red silk dress."

[1092] Output: User request data

[1093] Detailed description: The terminal receives the user's input and sends this request data to the server.

[1094] Step 2: Analyze the needs and generate questions

[1095] Input: User's request data

[1096] How it works: The server launches a generative AI model (e.g., GPT-4) to analyze the user's request. The generative AI model is used to extract necessary information from the request and automatically generate follow-up questions.

[1097] Output: Auto-generated questions

[1098] Detailed explanation: For example, a question such as "Do you have any requests for detailed decorations or special designs?" is generated, and the server sends this question to the terminal.

[1099] Step 3: Emotion recognition by the emotion engine

[1100] Input: User response data

[1101] Specific operation: When the user answers a question generated through the device, the device launches an emotion engine (e.g., Affectiva) and analyzes the user's emotions in real time.

[1102] Output: Emotion recognition result

[1103] Longer explanation: For example, if a user answers "No decoration needed," the sentiment engine determines whether the answer is positive or negative.

[1104] Step 4: View the question and enter the answer

[1105] Input: Auto-generated question and emotion recognition result

[1106] Specific operation: The device refers to the results of the emotion engine, displays the generated questions to the user in sequence, and receives the user's answers. The user enters answers to each question.

[1107] Output: User response data

[1108] Detailed explanation: For example, if a user answers "I don't need decorations," and the answer is recognized as negative, the server automatically generates an additional follow-up question, such as "Would you consider other colors or materials?"

[1109] Step 5: Adjust the specifications

[1110] Input: User response data and emotion recognition results

[1111] Specific operation: The server adjusts the order specifications based on the user's answers and the results of the emotion engine.

[1112] Output: Adjusted order specifications

[1113] Detailed explanation: For example, the specifications are determined as "red silk material with a simple design without decorations," and the server explains the meaning of the options and necessary information to the user in an easy-to-understand manner during this process.

[1114] Step 6: Sending an image generation request

[1115] Input: Adjusted order specifications

[1116] Specific operation: The server sends the adjusted specifications to the image generation AI (e.g., DALL-E 2) and requests the generation of a completed image.

[1117] Output: Finished image

[1118] Detailed description: Image generation AI generates a visual representation of the finished product based on the received specifications.

[1119] Step 7: Create and display a finished image

[1120] Input: Image of completed product

[1121] Specific operation: The server receives the generated completed image and sends it to the terminal, which displays the image to the user so that the user can check it.

[1122] Output: Final confirmation by the user

[1123] Detailed explanation: For example, the user confirms, "This is the design I envisioned, I'd like this."

[1124] Step 8: Finalize your order

[1125] Input: Final confirmation by the user

[1126] Specific operation: The user checks the completed image and confirms the final order details. The user enters "I confirm the order" on the terminal.

[1127] Output: Confirmed order details

[1128] Detailed description: The order details confirmed by the user are sent to the server.

[1129] Step 9: Submit your order

[1130] Input: Confirmed order details

[1131] What happens: The server receives the final confirmed order details and automatically sends them to the merchant.

[1132] Output: Order received by merchant

[1133] Detailed Description: The seller will receive this information and begin producing the product based on your specific order.

[1134] This system allows customers to easily communicate their needs in natural language and uses an emotion engine to respond appropriately based on the customer's emotions, resulting in a smooth ordering process. It also allows sellers to efficiently produce products by knowing the details of the order in advance.

[1135] (Application example 2)

[1136] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1137] The modern ordering process for custom-made products is extremely time-consuming and tedious, requiring customers to visit a store and provide detailed requests. Furthermore, requests are collected without taking into account the customer's emotional state, which can easily lead to customer dissatisfaction and misunderstandings. Furthermore, there is a lack of visual feedback, making it difficult for customers to visualize the final product. These issues significantly impair the customer experience, especially in physical stores.

[1138] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for accepting a request from a user in natural language; means for automatically generating questions to acquire necessary information by analyzing the accepted request using a generation AI; means for presenting the automatically generated questions to the user and receiving answers; means for adjusting the order specifications based on the received answers using the generation AI; means for creating an image of the completed product based on the adjusted specifications using an image generation AI; means for presenting the created image of the completed product to the user and finalizing the order details; means for transmitting the finalized order details to the seller; means for analyzing the user's emotions at the time of user input using an emotion engine that recognizes the user's emotions; means for analyzing the customer's emotions in real time and automatically generating follow-up questions; and means for presenting information to the user using a glasses-type terminal and confirming the order process. This realizes an ordering process that takes customer emotions into consideration, enabling intuitive and efficient ordering of custom-made products.

[1139] The "means for accepting requests from users in natural language" is a system that allows customers to input details of the custom-made products they desire in natural language via voice or text.

[1140] "Generative AI" is a system that uses artificial intelligence to analyze user input and generate relevant questions and content.

[1141] The "means for automatically generating questions" is a system that has the function of automatically generating questions to acquire necessary information based on the user's request.

[1142] The "means for presenting automatically generated questions to users and receiving answers" is a mechanism for displaying automatically generated questions to users and collecting answers from the users.

[1143] The "means for adjusting the specifications of the order" is a system that has an adjustment function for determining the final product specifications based on the user's responses.

[1144] "Image generation AI" is an artificial intelligence that generates a rendering of a finished product based on adjusted specifications.

[1145] The "means for creating a completed image drawing" is a system having a function for creating and saving the generated completed image drawing.

[1146] "Means of presenting the completed image to the user and finalizing the order details" refers to a mechanism that shows the user an image of the completed product and leads them through the process of confirming and confirming the final order details.

[1147] The "means for transmitting the confirmed order details to the seller" is a mechanism for automatically transferring the confirmed order details to the seller.

[1148] The "emotion engine" is a system for analyzing the emotions of users in real time when they input their emotions.

[1149] The "means for analyzing customer emotions in real time and automatically generating follow-up questions" is a system that analyzes the emotional state of a customer and automatically generates additional questions as needed.

[1150] An "eyeglasses-type terminal" is a wearable device that visually provides information to a user.

[1151] The present invention is a system for enabling users to streamline the ordering process for custom-made products using natural language, and can be implemented as follows.

[1152] 1. User requests

[1153] The user wears the glasses and inputs their requirements for a custom-made product using voice or text. For example, they might say, "I'd like a simple red silk dress." The microphone in the glasses captures the voice and sends the data to a smartphone connected via Bluetooth or to the store's server. This uses a speech recognition API (for example, Google Cloud Speech-to-Text).

[1154] 2. Analyzing requests and generating questions

[1155] The server analyzes the received voice data and uses a generative AI (such as OpenAI's GPT-4) to analyze the user's requests. As a result of the analysis, questions are automatically generated to obtain the necessary information. For example, a question such as "Do you have any requests for detailed decorations or special designs?" is generated.

[1156] 3. Emotion Recognition by Emotion Engine

[1157] As the user answers the questions, the camera on the glasses uses facial expression recognition software (for example, the emotion recognition API from Microsoft Azure Cognitive Services) to recognize the user's emotions in real time. For example, when the user types "No decoration needed," it determines whether the emotion is positive or negative.

[1158] 4. View the question and enter the answer

[1159] The generated questions are displayed on the display of the glasses-type device, and the user answers using voice or a touch interface. If the emotion engine detects a negative emotion, the server automatically generates additional follow-up questions.

[1160] 5. Adjustment of specifications

[1161] The server adjusts the order specifications based on the user's answers and the results of the emotion engine. For example, it may adjust the specifications to "red silk material with a simple design without decoration," and also explain the meaning of the options and necessary information to the user in an easy-to-understand manner.

[1162] 6. Sending an image generation request

[1163] The adjusted specifications are sent to an image generation AI (e.g., DALL-E) and a request is made to generate a visual image of the completed product. The image generation AI generates a visual image of the completed product based on the received specifications.

[1164] 7. Creation and display of completed image

[1165] The server receives the generated completed image and sends it to the eyeglasses. The eyeglasses display the completed image to the user, allowing them to confirm it. For example, the user can confirm, "This is the design I had in mind. This is fine."

[1166] 8. Final Order Confirmation

[1167] The user checks the completed image and confirms the final order details by voice command. The data from the glasses-type device is sent to the server.

[1168] 9. Submitting your order

[1169] The server receives the final order and automatically sends it to the seller, who can then create the product based on the specific order.

[1170] Specific examples

[1171] A user enters a physical store, puts on the eyeglasses-type device, and says, "I want a simple red silk dress."

[1172] The glasses-type device displays a question, asking, "Do you have any decoration preferences?"

[1173] If the user answers "Nothing in particular," the emotion engine recognizes this as positive.

[1174] The final specifications were decided as "red silk material with a simple design without any decoration."

[1175] Image generation AI generates visual images and displays them on the glasses-type device.

[1176] The user checks the image and confirms the final order by saying, "This is it."

[1177] Prompt Sentence Examples

[1178] "I want a simple red silk dress."

[1179] "Do you have any decoration preferences?"

[1180] "Nothing in particular."

[1181] This creates an ordering process that takes customer emotions into account, making ordering custom-made products intuitive and efficient.

[1182] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1183] Step 1:

[1184] The user wears the eyeglasses and inputs their requests for custom-made products by voice or text. The microphone in the eyeglasses captures the voice and sends the data to a smartphone connected via Bluetooth or to the store's server. This input is done by the user saying, for example, "I want a simple red silk dress." The server converts the voice data into text data using a speech recognition API (for example, Google Cloud Speech-to-Text). This converts the voice input (input data) into text data (output data).

[1185] Step 2:

[1186] The server analyzes the received text data and uses a generative AI (for example, OpenAI's GPT-4) to analyze the user's request. As a result of the analysis, questions are automatically generated to obtain the required information. For example, a question such as "Do you have any requests for detailed decorations or special designs?" is generated. This involves the process in which the generative AI generates questions (output data) based on the user's request, which is input.

[1187] Step 3:

[1188] The display on the glasses-type device presents automatically generated questions to the user. The user answers the questions using voice or a touch interface. The server receives the answers as text data. In this step, the user answers "nothing in particular" to the questions, and the input data is sent to the server.

[1189] Step 4:

[1190] When a user answers a question, the camera on the glasses uses facial expression recognition software (e.g., Microsoft Azure Cognitive Services' Emotion Recognition API) to recognize the user's emotions in real time. For example, the emotion engine detects positive emotions in response to the user's answer. In this step, real-time facial expression data is used as input data, and emotion identification results are obtained as output data.

[1191] Step 5:

[1192] The server adjusts the order specifications based on the user's answers and the results of the emotion engine. For example, it determines the specifications as "red silk material with a simple design without decoration." This adjustment is done using generative AI, and includes a process of determining the specifications (output data) from the input data of the answers and emotion data.

[1193] Step 6:

[1194] The server sends the finalized specifications to an image generation AI (e.g., DALL-E) and requests it to generate a visual representation of the completed image. Based on the received specifications, the image generation AI generates a visual representation of the completed image (output data). The input data in this step are the finalized specifications.

[1195] Step 7:

[1196] The server receives the generated completed image and sends it to the eyeglasses. The eyeglasses display the completed image on the screen for the user to confirm. The user then looks at the image and confirms, "This is fine." The input data in this step is the generated completed image, and the output data is the user's final confirmation.

[1197] Step 8:

[1198] The user checks the completed image and confirms the final order details by voice command. The voice command data from the glasses-type device is sent to the server. After this confirmation, the server confirms the final order details (output data).

[1199] Step 9:

[1200] The server receives the final order and automatically sends it to the seller, so that the seller can create the product based on the specific order. In this step, the input data is the finalized order, and the output data is the data to be sent to the seller.

[1201] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1202] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1203] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1204] [Fourth embodiment]

[1205] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1206] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1207] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1208] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1209] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1210] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1211] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1212] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1213] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1214] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1215] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1216] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1217] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1218] This invention relates to a system for streamlining the ordering process for custom-made products, and aims to reduce the burden on customers by combining generative AI and image generation AI. The program for this system operates as follows.

[1219] 1. User requests

[1220] A user accesses the system through a terminal and inputs their request for a custom-made product in natural language, including details of the product type, color, material, and design. For example, they might input, "I want a simple red silk dress."

[1221] 2. Analyzing requests and generating questions

[1222] The server uses AI to analyze the user's requests and automatically generate questions to obtain the necessary information. For example, a generated question might be, "Do you have any requests for detailed decorations or special designs?"

[1223] 3. View the question and enter the answer

[1224] The device presents the user with an automatically generated question, and the user answers the question, for example, "No decoration is necessary."

[1225] 4. Adjustment of specifications

[1226] The server adjusts the order specifications based on the user's answers. For example, the specifications may be adjusted to "red silk material with a simple design without decoration." During this process, the meaning of the options and necessary information are explained to the user in an easy-to-understand manner.

[1227] 5. Sending an image generation request

[1228] The server sends the adjusted specifications to the image generation AI and requests it to generate an image of the finished product.

[1229] 6. Creating a completed image

[1230] Based on the specifications received, the image generation AI generates a visual representation of the finished product, which concretely shows what the product will look like.

[1231] 7. Display and check the image

[1232] The terminal displays the generated completed image to the user, who then confirms it, for example, by saying, "This image is fine."

[1233] 8. Final Order Confirmation

[1234] The user checks the completed image and finalizes the order by entering "I confirm my order" on the terminal.

[1235] 9. Submitting your order

[1236] The server receives the final order and automatically sends it to the seller, who can then create the product based on the specific order.

[1237] This system allows customers to easily communicate their needs in natural language and gives them a concrete image of the finished product through questions and illustrations generated along the way. This helps avoid misunderstandings and dissatisfaction, resulting in a smooth ordering process. It also allows sellers to understand the detailed order details in advance, enabling efficient product production. This system can accept orders from all over the country, which has the effect of expanding the market for custom-made products.

[1238] The processing flow will be explained below.

[1239] Step 1:

[1240] The user accesses the system through a terminal and inputs their desired custom-made product in natural language on the system's input screen. For example, "I want a simple red silk dress."

[1241] Step 2:

[1242] The device sends the user's input to the server. The server uses generative AI to analyze the user's requests. As a result of the analysis, a list of questions is automatically generated to obtain the necessary information. For example, questions such as "Do you have any requests for detailed decorations or special designs?" are generated.

[1243] Step 3:

[1244] The server sends the generated question list to the terminal. The terminal displays the questions to the user in order, and the user inputs an answer to each question. For example, the user might answer, "No decoration is necessary."

[1245] Step 4:

[1246] The device sends the user's answers to the server. The server uses generative AI to adjust the order specifications based on the answers received from the user. During this process, the meaning of the options and necessary information are explained to the user in an easy-to-understand manner. For example, the specifications are adjusted to "red silk material with a simple design without decoration."

[1247] Step 5:

[1248] The server sends the adjusted specifications to the image generation AI and requests it to generate a visual image of the completed product. The image generation AI generates a visual image of the completed product based on the received specifications.

[1249] Step 6:

[1250] The server receives the generated completed image and sends it to the terminal. The terminal displays the completed image to the user and asks for confirmation. The user confirms, "This is the design I envisioned. This is fine."

[1251] Step 7:

[1252] The terminal sends the user's confirmation to the server, which compiles the final order and automatically sends it to the seller, who can then create the product based on the specific order.

[1253] This series of steps reduces the burden on both the customer and the seller, resulting in an efficient, high-quality custom product ordering process.

[1254] Example 1

[1255] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1256] During the custom-made product ordering process, customers often have difficulty effectively communicating their desired details. This can lead to dissatisfaction with the finished product, and it can be difficult for sellers to accurately understand the customer's needs. Additionally, the process of confirming specific design elements and materials can be cumbersome and time-consuming. Furthermore, there is a need to provide appropriate options and suggestions to meet the user's needs.

[1257] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1258] In this invention, the server includes means for accepting requests from users in natural language, means for analyzing the accepted requests using a generative AI model and automatically generating questions to acquire necessary information, and means for creating a completed image based on the adjusted specifications using an image generation AI. This allows the server to efficiently analyze the user's requests and provide a concrete image of the completed product, thereby reducing dissatisfaction with the finished product and enabling the seller to accurately understand the detailed order content.

[1259] "User" refers to any individual or legal entity that uses the System to place an order for a custom-made product.

[1260] "Natural language" refers to a language used by humans on a daily basis, a language that has grammar and meaning rather than specific program code.

[1261] A "generative AI model" refers to a system and its algorithms that use artificial intelligence technology to analyze natural language and generate text.

[1262] "Request" refers to the specific requirements and wishes that a user has for a custom-made product.

[1263] "Means for automatically generating questions" refers to a function that uses a generative AI model to automatically create questions to gather information necessary to further clarify a user's needs.

[1264] An "answer" refers to specific information that a user provides in response to a posed question.

[1265] "Means to tailor order specifications" refers to the ability to use generative AI models to determine the specific design and attributes of custom-made products based on user responses.

[1266] "Image generation AI" refers to a system or algorithm that uses artificial intelligence technology to generate visual images from text information.

[1267] A "finished image" refers to a visual representation of the appearance of a custom-made product, generated by image generation AI.

[1268] "Seller" refers to an individual or legal entity that manufactures and sells custom-made products upon receiving orders from users.

[1269] "Means of explaining the meaning of options and suggestions in response to inquiries in natural language" refers to a function that uses a generative AI model to explain appropriate options and suggestions in response to a user's natural language questions.

[1270] "Prompt" refers to a textual instruction that is required of an image generation AI to generate a specific image.

[1271] This invention is a system that streamlines the ordering process for custom-made products, and aims to reduce the burden on customers by combining generative AI models and image generation AI. The system program operates as follows.

[1272] User request input

[1273] A user accesses the system using a terminal and inputs their request for a custom-made product in natural language. This input includes details of the product type, color, material, and design. For example, they might input, "I want a simple red silk dress." The terminal then sends this request to the server.

[1274] Analyzing requests and generating questions

[1275] The server analyzes the received user request using a generative AI model (for example, a general natural language processing model). Based on the analysis results, the server generates a question to collect the necessary additional information. For example, it generates a question such as, "Do you have any requests for detailed decorations or special designs?" This generated question is then sent to the device.

[1276] View questions and enter answers

[1277] The terminal displays the generated question to the user, who responds by typing, for example, "No decoration necessary." The terminal then sends the response to the server.

[1278] Specification adjustment

[1279] The server adjusts the order specifications based on the user's answers. For example, the specifications could be set to "red silk material with a simple design without decoration." During this process, the server generates text that clearly explains the meaning of the options and necessary information to the user, and sends it to the terminal.

[1280] Sending an image generation request

[1281] Based on the adjusted specifications, the server creates a prompt to send to the image generation AI (for example, a general image generation algorithm). For example, "Please generate an image of a simple dress made of red silk." This prompt is sent to the image generation AI.

[1282] Creating a completed image

[1283] Based on the specified prompts, the image generation AI generates a visual representation of the finished product, which concretely shows the appearance of the product. The generated representation is then sent to the server.

[1284] Display and check the image diagram

[1285] The terminal displays the completed image received from the server to the user. The user checks this image and confirms, for example, "This image is OK." The confirmation result is sent to the server.

[1286] Final order confirmation

[1287] After the user checks the completed image, they input "I confirm my order" through their terminal. This input is sent to the server.

[1288] Sending order details

[1289] The server receives the final order and automatically sends it to the seller, who can then create the product based on the order.

[1290] This system allows users to easily order custom-made products using natural language and confirm specific specifications based on generated questions and finished product images. This avoids misunderstandings and frustrations, resulting in a smooth ordering experience. Furthermore, sellers can efficiently produce products by knowing the details of the order in advance.

[1291] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1292] Step 1:

[1293] A user accesses the system using a terminal and inputs their request for a custom-made product in natural language. This input includes details of the product type, color, material, and design. For example, a user might input, "I want a simple red silk dress." The terminal sends this request as input data to the server. The output data generated is the user's requested information.

[1294] Step 2:

[1295] The server uses a generative AI model to analyze the user's request data it receives. Specifically, the server inputs the request data into the generative AI model and automatically generates questions to obtain the necessary additional information. For example, the server generates a question such as, "Do you have any requests for detailed decorations or special designs?" The generated question is sent from the server to the device as output data.

[1296] Step 3:

[1297] The terminal displays the question sent from the server to the user. The user inputs an answer to this question. For example, the user inputs "No decoration is necessary." The terminal sends this answer data as input to the server. The user's answer is generated as output data.

[1298] Step 4:

[1299] The server adjusts the order specifications based on the user's response data received. Specifically, the server uses a generative AI model to analyze the response data and set the specifications to "a simple design with no decorations in red silk." During this process, it also generates text to clearly explain the meaning of the options and necessary information, and sends it to the device. The output data generated is the adjusted order specifications and explanatory text.

[1300] Step 5:

[1301] Based on the adjusted specifications, the server creates a prompt to send to the image generation AI. For example, the prompt might be, "Please generate an image of a simple dress made of red silk." After the prompt is generated, the server sends it to the image generation AI. The prompt is generated and sent as output data.

[1302] Step 6:

[1303] The image generation AI generates a visual image of the finished product based on the prompt received from the server. The prompt is input into the generation AI, and a specific visual image of the dress is generated as output. The generated image is sent to the server.

[1304] Step 7:

[1305] The terminal displays the completed image sent from the server to the user. The user checks this image and enters, for example, "This image is fine." The terminal sends the user's confirmation results to the server as input data. The confirmation results and the user's decision to confirm are generated as output data.

[1306] Step 8:

[1307] After the user has made the final confirmation, they confirm the order details through the terminal. For example, they may input "I confirm my order." This input is sent to the server, which receives the final order details. The confirmed order details are generated as output data.

[1308] Step 9:

[1309] The server automatically sends the final order details to the seller, who can then create the product based on the order details. The output data is a notification of the order details to the seller.

[1310] This allows users to easily order custom-made products using natural language and confirm specific specifications based on generated questions and finished product images. Sellers can also efficiently produce products by knowing the details of the order in advance.

[1311] (Application example 1)

[1312] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1313] The ordering process for custom-made products involves many steps, which can lead to problems such as miscommunication between the user and the seller and complicated procedures. Furthermore, it is often difficult for users to form a concrete image of the product, and it often takes a lot of effort to obtain a satisfactory product. Therefore, there is a need for a system that can flexibly respond to user requests while efficiently carrying out the ordering process.

[1314] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1315] In this invention, the server includes: means for accepting requests from a user in natural language; means for analyzing the accepted request using a generative AI model and automatically generating questions to acquire necessary information; means for presenting the automatically generated questions to the user and receiving responses; means for adjusting the order specifications based on the received responses; means for creating a completed image based on the adjusted specifications using an image generation AI; means for presenting the created completed image to the user and finalizing the order details; means for transmitting the confirmed order details to the seller; means for requesting an image from the generative AI model and generating an image generation prompt based on the order details; and means for requesting the image based on the generated prompt. This allows the user to view a specific image simply by entering their request in natural language, enabling a smooth ordering process without misunderstandings. Furthermore, the use of application software compatible with smart devices can further improve user convenience.

[1316] "User" means any individual or legal entity that uses the System to order a custom-made product.

[1317] "Natural language" refers to the language that users use in their everyday communication, and is not a specific programming language, but rather the words that humans use on a daily basis.

[1318] A "generative AI model" refers to an algorithm that uses artificial intelligence technology to analyze user requests and automatically generate questions to obtain the necessary information.

[1319] "Analysis" is the process of deciphering input natural language and extracting important information from it.

[1320] "Automatic question generation" means using artificial intelligence technology to deeply understand a user's needs and create appropriate questions to solicit additional information.

[1321] "Means for receiving answers" refers to a function for receiving answers entered by a user in response to questions presented by the system.

[1322] "Order specifications" refers to the specific characteristics and features of the product that are ultimately determined based on the user's requests and responses.

[1323] "Image generation AI" refers to algorithms that use artificial intelligence technology to generate visual images based on received specifications.

[1324] "Completed image" refers to a visual representation of a product created by image generation AI based on specifications.

[1325] "Means for finalizing the order" refers to a function that allows the user to check the generated image and finalize it as the final order.

[1326] "Seller" refers to an individual or legal entity that produces and provides products to users based on confirmed orders.

[1327] "Application software compatible with smart devices" refers to a program that runs on smart devices such as smartphones and tablets and provides the functions necessary for users to order custom-made products.

[1328] A "prompt statement" refers to a command statement that uses a generative AI model to generate an image diagram for an image generation AI.

[1329] The system for realizing this invention uses the following hardware and software: The hardware uses a smart device such as a smartphone or tablet. The software uses an application developed using Android Studio or Xcode, a generative AI model (GPT-4), and an image generation AI (DALL-E or Stable Diffusion).

[1330] System configuration

[1331] The system includes the following programs:

[1332] 1. A means of accepting requests from users in natural language

[1333] 2. A means of using generative AI models to analyze received requests and automatically generate questions to obtain the necessary information.

[1334] 3. A means of presenting automatically generated questions to users and receiving answers

[1335] 4. A means to adjust the specifications of the order based on the response received.

[1336] 5. A method to use image generation AI to create a finished image based on adjusted specifications

[1337] 6. A means to present the completed image to the user and finalize the order details

[1338] 7. How to send the confirmed order to the seller

[1339] 8. A method to request an image from the AI ​​model and generate an image generation prompt based on the order details

[1340] 9. A method for requesting an image based on the generated prompt

[1341] Program processing explanation

[1342] User request input

[1343] Users access the application using their smart device and enter their custom product requirements in natural language, including details of product type, color, material, and design.

[1344] Analyzing requests and generating questions

[1345] The server uses a generative AI model (GPT-4) to analyze the user's request. Based on the analysis results, it automatically generates questions to complement the information. For example, if the request is "I want a simple red silk dress," the server generates a related question: "Do you want any detailed decorations or special designs?"

[1346] View questions and enter answers

[1347] The automatically generated questions are presented to the user, who answers them via their smart device, for example, "No decoration is needed."

[1348] Specification adjustment

[1349] Based on the user's response, the server adjusts the order specifications, for example, a red silk dress with a simple design without any decorations.

[1350] Sending an image generation request

[1351] Based on the adjusted specifications, the server generates a request to the image generation AI. First, the generative AI model generates a prompt sentence. For example, the prompt sentence is generated as "Create an image of a simple, red silk dress with no decorations."

[1352] Creating and confirming a completed image

[1353] The image generation AI (DALL-E or Stable Diffusion) creates an image based on the generated prompt. This image is displayed to the user for confirmation.

[1354] Finalize and submit your order

[1355] The user checks the image and finalizes the order details, which are then sent to the seller by the server.

[1356] Specific examples

[1357] For example, a user might input, "I want a simple red silk dress." The generative AI model analyzes this and generates the question, "Do you have any requests for detailed decorations or special designs?" The user responds, "No decorations necessary." Based on this information, the server determines the specifications as "a dress made of red silk with a simple design and no decorations," and sends the prompt "Create an image of a simple, red silk dress with no decorations." The generated image is displayed to the user, and the final order is confirmed.

[1358] In this way, the user's wishes can be quickly and accurately reflected, and a smooth ordering process for custom-made products can be realized.

[1359] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1360] Step 1:

[1361] Users access the application using their smart device and input their custom product requirements in natural language, including details of product type, color, material, and design.

[1362] Step 2:

[1363] The device sends the input request to a server. The server uses a generative AI model (GPT-4) to analyze the received request and automatically generate questions to obtain the necessary information. For example, in response to a request for a simple red silk dress, the server generates the question, "Do you have any requests for detailed decorations or special designs?"

[1364] Step 3:

[1365] The server sends an automatically generated question to the terminal. The terminal displays the question to the user. The user then inputs an answer to the displayed question. For example, the user might answer, "No decoration is necessary."

[1366] Step 4:

[1367] The terminal sends the user's response to the server, which then uses the response to adjust the order specifications. Specifically, the server determines the specific characteristics of the product based on the user's request and response. For example, the specifications may be adjusted to "a simple, unadorned dress made of red silk."

[1368] Step 5:

[1369] The server uses a generative AI model to generate image prompts based on the adjusted specifications. For example, the prompt might read, "Create an image of a simple, red silk dress with no decorations."

[1370] Step 6:

[1371] The server sends the generated prompt to an image generation AI (DALL-E or Stable Diffusion), which generates a visual image based on the prompt.

[1372] Step 7:

[1373] The server sends the generated image to the terminal, which displays it to the user and asks for confirmation, for example, by saying, "This image is OK."

[1374] Step 8:

[1375] The user checks the displayed image and finalizes the order details. The terminal accepts input to confirm the order.

[1376] Step 9:

[1377] The terminal sends the confirmed order details to the server, which then notifies the seller of the final order details. The seller then produces the product based on the order details and provides it to the user.

[1378] These steps ensure that the user's needs are reflected quickly and accurately, and that the ordering process for custom-made products is smooth.

[1379] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1380] This invention is a system for streamlining the ordering process for custom-made products, and aims to reduce the burden on customers by combining generative AI and image generation AI, and to improve the customer experience by adding an emotion engine that recognizes the user's emotions. The program processing of this system is explained in detail below.

[1381] 1. User requests

[1382] A user accesses the system through a terminal and inputs their request for a custom-made product in natural language. For example, they might input, "I want a simple red silk dress."

[1383] 2. Analyzing requests and generating questions

[1384] The device sends the user's input to the server, which uses generative AI to analyze the user's request and automatically generate questions to obtain the necessary information. For example, it generates questions such as, "Do you have any requests for detailed decorations or special designs?"

[1385] 3. Emotion Recognition by Emotion Engine

[1386] When a user answers a question, the device uses an emotion engine to recognize the emotion of the user's input. For example, when a user types "No decoration needed," the device determines whether the emotion is positive or negative.

[1387] 4. View the question and enter the answer

[1388] The device then displays the generated questions to the user in order based on the analysis results of the emotion engine and receives the user's answers. For example, if the user answers "No decoration is necessary" and this answer is recognized as having a negative sentiment, the server automatically generates additional follow-up questions.

[1389] 5. Adjustment of specifications

[1390] The server adjusts the order specifications based on the user's answers and the results of the emotion engine. For example, it adjusts the specifications to "red silk material with a simple design without decoration." During this process, the meaning of the options and necessary information are explained to the user in an easy-to-understand manner.

[1391] 6. Sending an image generation request

[1392] The server sends the adjusted specifications to the image generation AI and requests it to generate a visual image of the completed product. The image generation AI generates a visual image of the completed product based on the received specifications.

[1393] 7. Creation and display of completed image

[1394] The server receives the generated completed image and sends it to the terminal. The terminal displays the completed image to the user so that the user can confirm it. For example, the user can confirm, "This is the design I had in mind. This is fine."

[1395] 8. Final Order Confirmation

[1396] The user checks the completed image and finalizes the order by entering "I confirm my order" on the terminal.

[1397] 9. Submitting your order

[1398] The server receives the final order and automatically sends it to the seller, who can then create the product based on the specific order.

[1399] This system allows customers to easily communicate their requests in natural language, and an emotion engine enables responses that reflect the customer's emotions. This avoids misunderstandings and dissatisfaction, resulting in a smooth ordering process. It also allows sellers to understand the details of the order in advance, enabling more efficient product production. This system can accept orders from all over the country, which has the effect of expanding the market for custom-made products.

[1400] The processing flow will be explained below.

[1401] This invention is a system for streamlining the ordering process for custom-made products, and aims to reduce the burden on customers by combining generative AI and image generation AI, and to improve the customer experience by adding an emotion engine that recognizes the user's emotions. The details of the program processing of this system are as follows.

[1402] Step 1:

[1403] A user accesses the system through a terminal and inputs their request for a custom-made product in natural language, for example, "I want a simple red silk dress."

[1404] Step 2:

[1405] The device sends the user's input to the server, which uses generative AI to analyze the user's request and automatically generate questions to obtain the necessary information. For example, it generates questions such as, "Do you have any requests for detailed decorations or special designs?"

[1406] Step 3:

[1407] The server sends the generated question list to the terminal, which displays the generated questions to the user in order.

[1408] Step 4:

[1409] The user answers the displayed question. For example, they might type "No decoration necessary." The device uses an emotion engine to recognize the user's emotion along with their answer. For example, if the user answers with a negative emotion, that information is also sent to the server.

[1410] Step 5:

[1411] The device sends the user's answers and emotional data to the server, which then uses generative AI to adjust the order specifications based on the user's answers and emotional data. For example, the specifications may be adjusted to "red silk material with a simple design without decoration."

[1412] Step 6:

[1413] The server generates a request to the image generation AI based on the adjusted specifications, and the image generation AI generates a visual representation of the finished product based on the received specifications.

[1414] Step 7:

[1415] The server receives the generated completed image and sends it to the terminal, which displays the completed image to the user and asks for their confirmation.

[1416] Step 8:

[1417] The user checks the completed image. When the user responds, "This is the design I imagined, I'd like this," the device sends the confirmation to the server.

[1418] Step 9:

[1419] The terminal sends the user's confirmation to the server, which compiles the final order and automatically sends it to the merchant.

[1420] This series of processes allows customers to have a concrete image, and the emotion engine responds appropriately. Furthermore, sellers can understand the detailed order details in advance, enabling efficient product production. This system reduces the burden on both customers and sellers, and realizes an efficient, high-quality custom-made product ordering experience.

[1421] Example 2

[1422] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1423] Traditional custom-made product ordering processes have struggled to accurately understand and efficiently process customer requests. They also lack the ability to properly understand and respond to customer emotions and intent, resulting in final products that don't meet customer expectations. Furthermore, there is a lack of established methods for utilizing image generation technology, resulting in a lack of visual feedback. Therefore, there is a need for a way to improve customer experience and efficiently process orders.

[1424] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1425] In this invention, the server includes means for accepting requests from a user in natural language, means for analyzing the accepted request using a generation AI and automatically generating questions to acquire necessary information, means for presenting the automatically generated questions to the user and receiving answers, means for recognizing the user's emotions using an emotion engine, means for adjusting the order specifications based on the received answers and the emotion recognition results using the generation AI, means for creating an image of the completed product based on the adjusted specifications using an image generation AI, means for presenting the created image of the completed product to the user and finalizing the order details, and means for transmitting the finalized order details to the seller. This makes it possible to accurately understand customer requests and respond in line with the customer's emotions, and by providing visual feedback, it is possible to improve the customer experience and process orders more efficiently.

[1426] The "means for accepting requests in natural language from the user" is an interface that allows the user to input their wishes and requests regarding custom-made products in natural language.

[1427] "Generative AI" is a system that uses artificial intelligence technology to analyze requests received in natural language, extract necessary information, and automatically generate questions.

[1428] "Means for presenting automatically generated questions to the user and receiving answers" refers to an interface that displays questions generated by the generation AI to the user and allows the user to input answers to those questions.

[1429] The "emotion engine" is a system that analyzes the emotions expressed by users when they input or respond in real time, and determines whether those emotions are positive or negative.

[1430] "Means for adjusting order specifications based on responses received using generative AI and emotion recognition results" refers to a system that appropriately adjusts order details and determines final specifications based on responses from users and the analysis results of the emotion engine.

[1431] "Image generation AI" is an artificial intelligence technology for generating visual images of completed products based on tailored order specifications.

[1432] The "means for presenting the completed image drawing to the user and allowing the user to confirm the final order details" is an interface that displays the generated completed image drawing to the user and allows the user to confirm the final order details.

[1433] The "means for transmitting the confirmed order details to the seller" is a system for automatically transmitting the order details confirmed by the user to the seller and issuing instructions for product production.

[1434] "Means for accepting orders from all over the country" refers to a system that allows orders to be accepted from all over the country without being restricted by geography.

[1435] "Means for explaining the meaning of options and suggestions in response to inquiries in natural language" refers to an interface that explains the meaning of options and related suggestions in response to the content of an inquiry made by a user in natural language.

[1436] This invention is a system for streamlining the ordering process for custom-made products and improving customer experience. The system is composed of a user terminal, a server, a generative AI model, an image generation AI, and an emotion engine.

[1437] Hardware and software used:

[1438] Hardware: User devices (PCs, smartphones, tablets, etc.), servers

[1439] Software: Generative AI model, image generation AI, emotion engine

[1440] System process flow and specific explanation:

[1441] 1. User requests

[1442] The user uses the device to open a web browser or dedicated application.

[1443] The terminal displays an input form to the user, and the user inputs their request for a custom-made product in natural language, for example, "I want a simple red silk dress."

[1444] The terminal sends this information to the server.

[1445] 2. Analyzing requests and generating questions

[1446] The server then runs a generative AI model, such as OpenAI's GPT-4, to analyze the received user request.

[1447] The server uses a generative AI model to analyze the request and automatically generate questions to gather additional information.

[1448] For example, generate a question like, "Do you have any requests for detailed decorations or special designs?"

[1449] The server sends the generated question to the terminal.

[1450] 3. Emotion Recognition by Emotion Engine

[1451] When a user answers questions through the device, the device uses an emotion engine to analyze the user's emotions in real time. For example, it uses Affectiva as emotion recognition software.

[1452] For example, if the user answers "No decoration is needed," it is determined whether the answer is positive or negative.

[1453] 4. View the question and enter the answer

[1454] The terminal sequentially displays questions generated based on the results of the emotion engine to the user and receives the user's answers.

[1455] For example, if a user answers "No decoration necessary" and the answer is perceived as negative, the server will automatically generate additional follow-up questions.

[1456] For example, follow-up questions such as "Would you consider other colors or materials?" are generated.

[1457] 5. Adjustment of specifications

[1458] The server adjusts the order specifications based on the user's answers and the results of the emotion engine.

[1459] For example, the specifications are determined as "red silk material with a simple design without decoration."

[1460] During this process, the server explains the meaning of the options and the necessary information to the user in an easy-to-understand manner.

[1461] 6. Sending an image generation request

[1462] The server sends the adjusted specifications to the image generation AI and requests it to generate a completed image. As an example, we will use DALL-E 2.

[1463] The image generation AI generates a visual representation of the finished product based on the received specifications.

[1464] 7. Creation and display of completed image

[1465] The server receives the generated completed image and sends it to the terminal.

[1466] The terminal displays the completed image to the user, who then confirms it, for example by saying, "This is the design I had in mind. This is fine."

[1467] 8. Final Order Confirmation

[1468] The user checks the completed image and finalizes the order details.

[1469] The user inputs "I confirm the order" through the terminal.

[1470] 9. Submitting your order

[1471] The server receives the final confirmed order and automatically sends it to the merchant.

[1472] The seller receives this information and begins producing the product based on the specific order.

[1473] Examples and prompts:

[1474] Examples:

[1475] Suppose a user inputs "a red silk dress," which generates the question "Do you need detailed embellishments?" If the user answers "No embellishments needed," and the emotion engine determines this as negative, an additional question is generated: "Would you consider other colors or materials?" The specifications are then finalized, and an image of the finished product is generated.

[1476] Example prompt sentence:

[1477] Suppose a user orders a custom dress based on the following criteria:

[1478] "I want a simple red silk dress."

[1479] In response, do you have any detailed decorations or special design requests?

[1480] Also, how should we respond if negative sentiment is detected in response to the response that decoration is not necessary?

[1481] The system allows for accurate understanding of customer needs, emotional response, and visual feedback, resulting in an improved customer experience and efficient order processing.

[1482] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1483] Step 1: User input

[1484] Input: User's natural language request

[1485] Specific operation: The user opens a web browser or a dedicated application on the device. The device displays an input form, and the user inputs their requirements for a custom-made product in natural language. For example, they might input, "I want a simple red silk dress."

[1486] Output: User request data

[1487] Detailed description: The terminal receives the user's input and sends this request data to the server.

[1488] Step 2: Analyze the needs and generate questions

[1489] Input: User's request data

[1490] How it works: The server launches a generative AI model (e.g., GPT-4) to analyze the user's request. The generative AI model is used to extract necessary information from the request and automatically generate follow-up questions.

[1491] Output: Auto-generated questions

[1492] Detailed explanation: For example, a question such as "Do you have any requests for detailed decorations or special designs?" is generated, and the server sends this question to the terminal.

[1493] Step 3: Emotion recognition by the emotion engine

[1494] Input: User response data

[1495] Specific operation: When the user answers a question generated through the device, the device launches an emotion engine (e.g., Affectiva) and analyzes the user's emotions in real time.

[1496] Output: Emotion recognition result

[1497] Longer explanation: For example, if a user answers "No decoration needed," the sentiment engine determines whether the answer is positive or negative.

[1498] Step 4: View the question and enter the answer

[1499] Input: Auto-generated question and emotion recognition result

[1500] Specific operation: The device refers to the results of the emotion engine, displays the generated questions to the user in sequence, and receives the user's answers. The user enters answers to each question.

[1501] Output: User response data

[1502] Detailed explanation: For example, if a user answers "I don't need decorations," and the answer is recognized as negative, the server automatically generates an additional follow-up question, such as "Would you consider other colors or materials?"

[1503] Step 5: Adjust the specifications

[1504] Input: User response data and emotion recognition results

[1505] Specific operation: The server adjusts the order specifications based on the user's answers and the results of the emotion engine.

[1506] Output: Adjusted order specifications

[1507] Detailed explanation: For example, the specifications are determined as "red silk material with a simple design without decorations," and the server explains the meaning of the options and necessary information to the user in an easy-to-understand manner during this process.

[1508] Step 6: Sending an image generation request

[1509] Input: Adjusted order specifications

[1510] Specific operation: The server sends the adjusted specifications to the image generation AI (e.g., DALL-E 2) and requests the generation of a completed image.

[1511] Output: Finished image

[1512] Detailed description: Image generation AI generates a visual representation of the finished product based on the received specifications.

[1513] Step 7: Create and display a finished image

[1514] Input: Image of completed product

[1515] Specific operation: The server receives the generated completed image and sends it to the terminal, which displays the image to the user so that the user can check it.

[1516] Output: Final confirmation by the user

[1517] Detailed explanation: For example, the user confirms, "This is the design I envisioned, I'd like this."

[1518] Step 8: Finalize your order

[1519] Input: Final confirmation by the user

[1520] Specific operation: The user checks the completed image and confirms the final order details. The user enters "I confirm the order" on the terminal.

[1521] Output: Confirmed order details

[1522] Detailed description: The order details confirmed by the user are sent to the server.

[1523] Step 9: Submit your order

[1524] Input: Confirmed order details

[1525] What happens: The server receives the final confirmed order details and automatically sends them to the merchant.

[1526] Output: Order received by merchant

[1527] Detailed Description: The seller will receive this information and begin producing the product based on your specific order.

[1528] This system allows customers to easily communicate their needs in natural language and uses an emotion engine to respond appropriately based on the customer's emotions, resulting in a smooth ordering process. It also allows sellers to efficiently produce products by knowing the details of the order in advance.

[1529] (Application example 2)

[1530] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1531] The modern ordering process for custom-made products is extremely time-consuming and tedious, requiring customers to visit a store and provide detailed requests. Furthermore, requests are collected without taking into account the customer's emotional state, which can easily lead to customer dissatisfaction and misunderstandings. Furthermore, there is a lack of visual feedback, making it difficult for customers to visualize the final product. These issues significantly impair the customer experience, especially in physical stores.

[1532] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for accepting a request from a user in natural language; means for automatically generating questions to acquire necessary information by analyzing the accepted request using a generation AI; means for presenting the automatically generated questions to the user and receiving answers; means for adjusting the order specifications based on the received answers using the generation AI; means for creating an image of the completed product based on the adjusted specifications using an image generation AI; means for presenting the created image of the completed product to the user and finalizing the order details; means for transmitting the finalized order details to the seller; means for analyzing the user's emotions at the time of user input using an emotion engine that recognizes the user's emotions; means for analyzing the customer's emotions in real time and automatically generating follow-up questions; and means for presenting information to the user using a glasses-type terminal and confirming the order process. This realizes an ordering process that takes customer emotions into consideration, enabling intuitive and efficient ordering of custom-made products.

[1533] The "means for accepting requests from users in natural language" is a system that allows customers to input details of the custom-made products they desire in natural language via voice or text.

[1534] "Generative AI" is a system that uses artificial intelligence to analyze user input and generate relevant questions and content.

[1535] The "means for automatically generating questions" is a system that has the function of automatically generating questions to acquire necessary information based on the user's request.

[1536] The "means for presenting automatically generated questions to users and receiving answers" is a mechanism for displaying automatically generated questions to users and collecting answers from the users.

[1537] The "means for adjusting the specifications of the order" is a system that has an adjustment function for determining the final product specifications based on the user's responses.

[1538] "Image generation AI" is an artificial intelligence that generates a rendering of a finished product based on adjusted specifications.

[1539] The "means for creating a completed image drawing" is a system having a function for creating and saving the generated completed image drawing.

[1540] "Means of presenting the completed image to the user and finalizing the order details" refers to a mechanism that shows the user an image of the completed product and leads them through the process of confirming and confirming the final order details.

[1541] The "means for transmitting the confirmed order details to the seller" is a mechanism for automatically transferring the confirmed order details to the seller.

[1542] The "emotion engine" is a system for analyzing the emotions of users in real time when they input their emotions.

[1543] The "means for analyzing customer emotions in real time and automatically generating follow-up questions" is a system that analyzes the emotional state of a customer and automatically generates additional questions as needed.

[1544] An "eyeglasses-type terminal" is a wearable device that visually provides information to a user.

[1545] The present invention is a system for enabling users to streamline the ordering process for custom-made products using natural language, and can be implemented as follows.

[1546] 1. User requests

[1547] The user wears the glasses and inputs their requirements for a custom-made product using voice or text. For example, they might say, "I'd like a simple red silk dress." The microphone in the glasses captures the voice and sends the data to a smartphone connected via Bluetooth or to the store's server. This uses a speech recognition API (for example, Google Cloud Speech-to-Text).

[1548] 2. Analyzing requests and generating questions

[1549] The server analyzes the received voice data and uses a generative AI (such as OpenAI's GPT-4) to analyze the user's requests. As a result of the analysis, questions are automatically generated to obtain the necessary information. For example, a question such as "Do you have any requests for detailed decorations or special designs?" is generated.

[1550] 3. Emotion Recognition by Emotion Engine

[1551] As the user answers the questions, the camera on the glasses uses facial expression recognition software (for example, the emotion recognition API from Microsoft Azure Cognitive Services) to recognize the user's emotions in real time. For example, when the user types "No decoration needed," it determines whether the emotion is positive or negative.

[1552] 4. View the question and enter the answer

[1553] The generated questions are displayed on the display of the glasses-type device, and the user answers using voice or a touch interface. If the emotion engine detects a negative emotion, the server automatically generates additional follow-up questions.

[1554] 5. Adjustment of specifications

[1555] The server adjusts the order specifications based on the user's answers and the results of the emotion engine. For example, it may adjust the specifications to "red silk material with a simple design without decoration," and also explain the meaning of the options and necessary information to the user in an easy-to-understand manner.

[1556] 6. Sending an image generation request

[1557] The adjusted specifications are sent to an image generation AI (e.g., DALL-E) and a request is made to generate a visual image of the completed product. The image generation AI generates a visual image of the completed product based on the received specifications.

[1558] 7. Creation and display of completed image

[1559] The server receives the generated completed image and sends it to the eyeglasses. The eyeglasses display the completed image to the user, allowing them to confirm it. For example, the user can confirm, "This is the design I had in mind. This is fine."

[1560] 8. Final Order Confirmation

[1561] The user checks the completed image and confirms the final order details by voice command. The data from the glasses-type device is sent to the server.

[1562] 9. Submitting your order

[1563] The server receives the final order and automatically sends it to the seller, who can then create the product based on the specific order.

[1564] Specific examples

[1565] A user enters a physical store, puts on the eyeglasses-type device, and says, "I want a simple red silk dress."

[1566] The glasses-type device displays a question, asking, "Do you have any decoration preferences?"

[1567] If the user answers "Nothing in particular," the emotion engine recognizes this as positive.

[1568] The final specifications were decided as "red silk material with a simple design without any decoration."

[1569] Image generation AI generates visual images and displays them on the glasses-type device.

[1570] The user checks the image and confirms the final order by saying, "This is it."

[1571] Prompt Sentence Examples

[1572] "I want a simple red silk dress."

[1573] "Do you have any decoration preferences?"

[1574] "Nothing in particular."

[1575] This creates an ordering process that takes customer emotions into account, making ordering custom-made products intuitive and efficient.

[1576] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1577] Step 1:

[1578] The user wears the eyeglasses and inputs their requests for custom-made products by voice or text. The microphone in the eyeglasses captures the voice and sends the data to a smartphone connected via Bluetooth or to the store's server. This input is done by the user saying, for example, "I want a simple red silk dress." The server converts the voice data into text data using a speech recognition API (for example, Google Cloud Speech-to-Text). This converts the voice input (input data) into text data (output data).

[1579] Step 2:

[1580] The server analyzes the received text data and uses a generative AI (for example, OpenAI's GPT-4) to analyze the user's request. As a result of the analysis, questions are automatically generated to obtain the required information. For example, a question such as "Do you have any requests for detailed decorations or special designs?" is generated. This involves the process in which the generative AI generates questions (output data) based on the user's request, which is input.

[1581] Step 3:

[1582] The display on the glasses-type device presents automatically generated questions to the user. The user answers the questions using voice or a touch interface. The server receives the answers as text data. In this step, the user answers "nothing in particular" to the questions, and the input data is sent to the server.

[1583] Step 4:

[1584] When a user answers a question, the camera on the glasses uses facial expression recognition software (e.g., Microsoft Azure Cognitive Services' Emotion Recognition API) to recognize the user's emotions in real time. For example, the emotion engine detects positive emotions in response to the user's answer. In this step, real-time facial expression data is used as input data, and emotion identification results are obtained as output data.

[1585] Step 5:

[1586] The server adjusts the order specifications based on the user's answers and the results of the emotion engine. For example, it determines the specifications as "red silk material with a simple design without decoration." This adjustment is done using generative AI, and includes a process of determining the specifications (output data) from the input data of the answers and emotion data.

[1587] Step 6:

[1588] The server sends the finalized specifications to an image generation AI (e.g., DALL-E) and requests it to generate a visual representation of the completed image. Based on the received specifications, the image generation AI generates a visual representation of the completed image (output data). The input data in this step are the finalized specifications.

[1589] Step 7:

[1590] The server receives the generated completed image and sends it to the eyeglasses. The eyeglasses display the completed image on the screen for the user to confirm. The user then looks at the image and confirms, "This is fine." The input data in this step is the generated completed image, and the output data is the user's final confirmation.

[1591] Step 8:

[1592] The user checks the completed image and confirms the final order details by voice command. The voice command data from the glasses-type device is sent to the server. After this confirmation, the server confirms the final order details (output data).

[1593] Step 9:

[1594] The server receives the final order and automatically sends it to the seller, so that the seller can create the product based on the specific order. In this step, the input data is the finalized order, and the output data is the data to be sent to the seller.

[1595] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1596] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1597] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1598] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1599] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1600] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1601] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1602] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1603] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1604] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1605] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1606] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1607] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1608] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1609] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1610] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1611] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1612] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1613] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1614] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1615] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1616] The following is further disclosed regarding the above embodiment.

[1617] (Claim 1)

[1618] The inquiry response system includes: a means for receiving requests from users in natural language;

[1619] A means of using generative AI to analyze received requests and automatically generate questions to obtain necessary information;

[1620] means for presenting automatically generated questions to a user and receiving answers;

[1621] A means to adjust the specifications of the order based on the answers received using generative AI;

[1622] A means of creating a completed image based on the adjusted specifications using image generation AI;

[1623] A means for presenting the created completed image drawing to the user and finalizing the order details;

[1624] and means for transmitting the confirmed order details to the seller.

[1625] (Claim 2)

[1626] 10. The system of claim 1, further comprising means for accepting orders nationwide.

[1627] (Claim 3)

[1628] 10. The system of claim 1, further comprising means for explaining the meaning of options and suggestions in response to a natural language query.

[1629] "Example 1"

[1630] (Claim 1)

[1631] A means for receiving requests from a user in natural language;

[1632] A means of analyzing received requests using a generative AI model and automatically generating questions to obtain necessary information;

[1633] means for presenting automatically generated questions to a user and receiving answers;

[1634] A means to adjust order specifications based on the answers received using a generative AI model; and

[1635] A means of creating a completed image based on the adjusted specifications using image generation AI;

[1636] A means for presenting the created completed image drawing to the user and finalizing the order details;

[1637] a means for transmitting the confirmed order to the seller;

[1638] A means of explaining the meaning of options and suggestions in response to natural language queries;

[1639] A system including:

[1640] (Claim 2)

[1641] 10. The system of claim 1, further comprising means for accepting orders nationwide.

[1642] (Claim 3)

[1643] 10. The system of claim 1, further comprising: means for sending a request to the image generation AI based on the generated prompt sentence.

[1644] "Application Example 1"

[1645] (Claim 1)

[1646] A means for receiving requests from a user in natural language;

[1647] A means of analyzing received requests using a generative AI model and automatically generating questions to obtain necessary information;

[1648] means for presenting automatically generated questions to a user and receiving answers;

[1649] A means of adjusting the specifications of the order based on the responses received;

[1650] A means of creating a completed image based on the adjusted specifications using image generation AI;

[1651] A means for presenting the created completed image drawing to the user and finalizing the order details;

[1652] a means for transmitting the confirmed order to the seller;

[1653] A means for requesting an image drawing from a generating AI model and generating an image generation prompt sentence based on the order content;

[1654] A means for requesting an image based on the generated prompt;

[1655] A system including:

[1656] (Claim 2)

[1657] 10. The system of claim 1, including application software compatible with the smart device.

[1658] (Claim 3)

[1659] 10. The system of claim 1, further comprising means for explaining the meaning of options and suggestions in response to a natural language question.

[1660] "Example 2: Combining Emotion Engines"

[1661] (Claim 1)

[1662] A means for receiving requests from a user in natural language;

[1663] A means of using generative AI to analyze received requests and automatically generate questions to obtain necessary information;

[1664] means for presenting automatically generated questions to a user and receiving answers;

[1665] a means for recognizing a user's emotion using an emotion engine;

[1666] A means to adjust the specifications of the order based on the answers received and the emotion recognition results using generative AI;

[1667] A means of creating a completed image based on the adjusted specifications using image generation AI;

[1668] A means for presenting the created completed image drawing to the user and finalizing the order details;

[1669] and means for transmitting the confirmed order details to the seller.

[1670] (Claim 2)

[1671] 10. The system of claim 1, further comprising means for accepting orders nationwide.

[1672] (Claim 3)

[1673] 10. The system of claim 1, further comprising means for explaining the meaning of options and suggestions in response to a natural language query.

[1674] "Application example 2 when combining emotion engines"

[1675] (Claim 1)

[1676] A means for receiving requests from a user in natural language;

[1677] A means of using generative AI to analyze received requests and automatically generate questions to obtain necessary information;

[1678] means for presenting automatically generated questions to a user and receiving answers;

[1679] A means to adjust the specifications of the order based on the answers received using generative AI;

[1680] A means of creating a completed image based on the adjusted specifications using image generation AI;

[1681] A means for presenting the created completed image drawing to the user and finalizing the order details;

[1682] a means for transmitting the confirmed order to the seller;

[1683] A means for analyzing emotions input by a user using an emotion engine that recognizes the user's emotions;

[1684] A means to analyze customer sentiment in real time and automatically generate follow-up questions,

[1685] A means for presenting information to a user using a glasses-type terminal and confirming an ordering process;

[1686] A system including:

[1687] (Claim 2)

[1688] 10. The system of claim 1, further comprising means for accepting orders nationwide.

[1689] (Claim 3)

[1690] 10. The system of claim 1, further comprising means for explaining the meaning of options and suggestions in response to a natural language query. [Explanation of symbols]

[1691] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. The inquiry response system includes: a means for receiving requests from users in natural language; A means of using generative AI to analyze received requests and automatically generate questions to obtain necessary information; means for presenting automatically generated questions to a user and receiving answers; A means to adjust the specifications of the order based on the answers received using generative AI; A means of creating a completed image based on the adjusted specifications using image generation AI; A means for presenting the created completed image drawing to the user and finalizing the order details; and means for transmitting the confirmed order details to the seller.

2. The system of claim 1 further comprising means for accepting orders nationwide.

3. The system of claim 1 further comprising means for explaining the meaning of options and suggestions in response to a natural language query.

Images