system

Abstract

We provide the system. [Solution] A means for the user to input an inquiry, A means of sending the inquiry content to the server, The server analyzes the query content using a natural language processing engine and extracts the intent and target, A means of obtaining company-specific information from a database based on the extracted information, A means by which a generative AI generates the optimal response, A means for sending the generated response to the terminal, The means by which the terminal displays a response to the user A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In corporate front-line customer service, quickly and appropriately responding to inquiries from users is important for improving customer satisfaction. However, conventional systems have problems in that the quality of response depends on the response ability and experience of front-line staff, so the quality of response is not constant and is inefficient in terms of time and cost. Furthermore, it has been difficult to quickly grasp a lot of company-specific information and past response histories and provide accurate responses.

Means for Solving the Problems

[0005] To solve this problem, the present invention provides the following means. First, it provides means for the user to input an inquiry and means for sending the inputted inquiry content to a server. Next, it provides means for the server to analyze the inquiry content with a natural language processing engine and extract the intent and target. Based on the extracted information, it provides means for obtaining company-specific information from a database and further provides means for a generative AI to generate an optimal response. Finally, it provides means for sending the generated response to a terminal and for the terminal to display the response to the user, thereby enabling the user to receive a quick and appropriate response. In this way, it is possible to maintain a consistent quality of customer service and reduce response time and costs.

[0006] An "inquiry" is text data entered by a user seeking specific information or support.

[0007] A "user" is an individual or organization that uses the system to make inquiries.

[0008] A "terminal" is a device used by a user to input inquiries and display responses from a server.

[0009] A "server" is a processing unit that analyzes the content of inquiries received from users and generates and sends responses using generative AI.

[0010] A "natural language processing engine" is software that analyzes the content of a query to identify its intent and target.

[0011] "Intention" refers to the main purpose or request that indicates what the inquiry is seeking.

[0012] "Subject" refers to the specific things or events included in the inquiry.

[0013] A "database" is a data storage system that stores company-specific information and past interaction history.

[0014] "Company-specific information" refers to data such as product information, service information, and inventory information that are specific to the target company.

[0015] A "generative AI" is an artificial intelligence system that generates the optimal response based on the content of the inquiry and related information.

[0016] "Tokenization" is a part of natural language processing, a process that divides text into meaningful units (tokens).

[0017] "Past interaction history" refers to data that includes records of interactions and responses that have taken place at the counter up to now.

[0018] A "structured data format" is a method of converting data into a regular format, and commonly includes JSON and XML. [Brief explanation of the drawing]

[0019] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] Shows an emotion map to which a plurality of emotions are mapped. [Figure 10] Shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0020] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be described.

[0022] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0023] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0024] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0025] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0027] [First Embodiment]

[0028] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0029] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0032] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0035] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0039] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0040] In this embodiment of the invention, a system is provided that enables efficient customer service through the coordinated operation of the user, terminal, and server components. The specific operation of the system is described below.

[0041] 1. Inquiry Reception

[0042] User: The user enters their inquiry using the device's interface (e.g., a web browser or dedicated application). The inquiry may include questions about specific product names or services.

[0043] Terminal: Converts the user's input into JSON format and sends it to the server as an HTTP POST request. This request includes the user's identification information and the inquiry's timestamp.

[0044] 2. Analysis of the inquiry

[0045] Server: The server uses a natural language processing engine to parse incoming HTTP POST requests. This engine tokenizes the query content, extracts specific keywords, and determines the intent of the query. It also identifies the product or service being inquired about.

[0046] 3. Database Query

[0047] Server: Executes queries against the database based on the intent and target identified by natural language processing. For example, if the query is for checking inventory, it generates an SQL query to retrieve inventory information about the product and retrieves the necessary information from the database.

[0048] 4. Generating a response

[0049] Server: Based on information retrieved from the database, a generative AI generates an appropriate response. This AI also refers to past response history to generate more appropriate and user-friendly responses. For example, it might generate a specific response such as, "Product A is currently in stock."

[0050] 5. Sending a response

[0051] Server: Packages the generated response into JSON format and sends it to the terminal as an HTTP response.

[0052] Terminal: The terminal analyzes the received response and displays it appropriately on the user interface. Through this display, the user can see the answer to their inquiry in real time.

[0053] Specific example

[0054] Example 1: Checking product inventory

[0055] User: Enters "Do you have product A in stock?"

[0056] Terminal: Convert this query into JSON format and send it to the server.

[0057] Server: The server analyzes the content of the inquiry using a natural language processing engine to extract the intent of "inventory check" and the target of "product A".

[0058] Server: Retrieves inventory information for "Product A" from the database and generates a response stating "Product A is currently in stock."

[0059] Server: Sends the generated response to the terminal.

[0060] Terminal: Displays "Product A is currently in stock" to the user.

[0061] Example 2: Support inquiry (technical question)

[0062] User: Enters "Please tell me how to configure device B."

[0063] Terminal: Convert this query into JSON format and send it to the server.

[0064] Server: Analyzes the query content using a natural language processing engine and extracts the intent of "configuration method" and the target of "device B".

[0065] Server: Retrieves information about "How to configure device B" from the database and generates a response that says, "The following are the steps to configure device B..."

[0066] Server: Sends the generated response to the terminal.

[0067] Terminal: Displays the user with the message, "The following steps will guide you through configuring device B..."

[0068] As described above, this system can significantly improve the efficiency of a company's customer service by automatically and quickly processing user inquiries.

[0069] The following describes the processing flow.

[0070] Step 1:

[0071] User: Enter your inquiry using the terminal's user interface. For example, enter "Do you have product A in stock?".

[0072] Step 2:

[0073] Terminal: Converts the entered query content into JSON format. This includes user identification information and the query's timestamp.

[0074] Step 3:

[0075] Terminal: Sends the converted JSON data to the server as an HTTP POST request.

[0076] Step 4:

[0077] Server: Parses the received HTTP POST request and extracts the query content. Processing begins based on this content.

[0078] Step 5:

[0079] Server: Uses a natural language processing engine to tokenize the extracted query content. For example, it separates it into words such as "product A," "in stock," and "available."

[0080] Step 6:

[0081] Server: Based on tokenized words, it identifies the intent of the inquiry (in this case, inventory check) and the target (in this case, product A).

[0082] Step 7:

[0083] Server: Generates queries against the database based on intent and target. For example, it generates an SQL query to "retrieve inventory information for product A".

[0084] Step 8:

[0085] Server: Executes the generated query against the database to retrieve relevant inventory information. For example, it retrieves information such as "Product A is currently in stock."

[0086] Step 9:

[0087] Server: Uses generative AI to generate the optimal response based on the acquired inventory information. For example, it generates the sentence, "Product A is currently in stock."

[0088] Step 10:

[0089] Server: Converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0090] Step 11:

[0091] Terminal: Analyzes the received HTTP response and converts it into a format that can be displayed in the user interface.

[0092] Step 12:

[0093] Terminal: Displays a formatted response in the user interface. For example, it displays the message "Product A is currently in stock."

[0094] Step 13:

[0095] User: Review the displayed response and take the next action as needed. For example, confirm that the item is in stock and proceed with the purchase.

[0096] The above outlines the specific processing steps of this system. Each step works in conjunction to enable efficient and accurate responses to user inquiries.

[0097] (Example 1)

[0098] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0099] Traditional customer service systems often have slow response times to user inquiries and may fail to provide appropriate answers. This leads to decreased efficiency in corporate customer service and reduced customer satisfaction. Furthermore, the lack of features to effectively utilize past interaction history makes it difficult to provide consistent answers to similar inquiries.

[0100] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0101] In this invention, the server includes means for analyzing with a language processing engine to extract intent and target,

[0102] A means of obtaining company-specific information from a data set based on extracted information,

[0103] A means by which a response generation engine generates an optimal response,

[0104] This includes enabling rapid and accurate analysis of inquiries and providing consistent and appropriate responses by leveraging past interaction history.

[0105] An "inquiry" refers to a request for information about a specific product or service that a user enters into an information processing device.

[0106] An "information processing device" is a terminal device used by users to input inquiries, such as a web browser or a device that utilizes a dedicated application.

[0107] A "host computer" refers to a central system that receives inquiries transmitted from information processing devices, analyzes them, and generates responses.

[0108] A "language processing engine" refers to the natural language processing technology used by a host computer to analyze query content, specifically an engine that performs tasks such as tokenization and intent extraction.

[0109] "Intention" refers to the purpose or objective that can be inferred from the user's inquiry.

[0110] "Target" refers to the product or service identified by the user's inquiry.

[0111] A "data collection" refers to an information source, such as a database, where multiple pieces of information are aggregated and stored.

[0112] A "response generation engine" refers to artificial intelligence technology that creates the optimal answer based on data acquired by a host computer.

[0113] "Optimal response" refers to information generated to answer a user's inquiry quickly and accurately.

[0114] "Presentation" refers to displaying the generated response to the user using an information processing device.

[0115] This system enables efficient customer service through the coordinated operation of users, terminals, and servers. The embodiments of this invention will be described in detail below.

[0116] Inquiry reception

[0117] User: Users enter their inquiries using the device's interface (e.g., a web browser or dedicated application). These inquiries may include questions about specific product names or services.

[0118] Terminal: The terminal converts the user's input into JSON format. This conversion includes the query content, user identification information, and query timestamp. This JSON data is sent from the terminal to the server as an HTTP POST request.

[0119] Inquiry analysis

[0120] Server: The server uses a natural language processing engine (e.g., Hugging Face's Transformers model) to parse incoming HTTP POST requests. This engine tokenizes the query content, extracts specific keywords, and determines the intent of the query. It also identifies the product or service being inquired about.

[0121] Database query

[0122] Server: Executes queries against the database based on the intent and target identified by natural language processing. For example, if the query is for checking inventory, it generates an SQL query to retrieve inventory information about the product and retrieves the necessary information from the database.

[0123] Response generation

[0124] Server: The server generates a response using a generative AI model (e.g., OpenAI®'s GPT-3®) based on information retrieved from the database. This response is provided in a format that is easy for the user to understand. For example, a specific response such as "Product A is currently in stock" is generated.

[0125] Sending a response

[0126] Server: The server packages the generated response into JSON format and sends it to the terminal as an HTTP response.

[0127] Terminal: The terminal analyzes the received response and displays it appropriately on the user interface. The user can then see the answer to their inquiry in real time from this display.

[0128] Specific example

[0129] Example 1: Checking product inventory

[0130] User: Enters "Do you have product A in stock?" into the terminal.

[0131] Terminal: Convert this query into JSON format and send it to the server.

[0132] Server: The server uses a natural language processing engine to analyze the query content and extract the intent of "stock confirmation" and the target of "product A".

[0133] Server: Retrieves inventory information for "Product A" from the database and generates a response stating "Product A is currently in stock."

[0134] Server: Sends the generated response to the terminal.

[0135] Terminal: Displays "Product A is currently in stock" to the user.

[0136] Example 2: Support inquiry (technical question)

[0137] User: Enters "Please tell me how to configure device B" into the terminal.

[0138] Terminal: Convert this query into JSON format and send it to the server.

[0139] Server: The server analyzes the query content using a natural language processing engine and extracts the intent of "configuration method" and the target of "device B".

[0140] Server: Retrieves information about "How to configure device B" from the database and generates a response that says, "The following are the steps to configure device B..."

[0141] Server: Sends the generated response to the terminal.

[0142] Terminal: Displays the user with the message, "The following steps will guide you through configuring device B..."

[0143] As described above, this system can significantly improve the efficiency of a company's customer service by automatically and quickly processing user inquiries.

[0144] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0145] Step 1:

[0146] The user enters the inquiry details into the information processing device.

[0147] Input: A specific question about a product or service (e.g., "Do you have product A in stock?")

[0148] Output: The user's inquiry is input into the information processing device.

[0149] Step 2:

[0150] The terminal converts the user's inquiry into JSON format.

[0151] Input: Inquiry content entered by the user

[0152] Output: JSON data containing query details, user identification information, and query timestamp.

[0153] Specific operation: The terminal software analyzes the user's input, extracts the necessary information, and generates data in JSON format.

[0154] Step 3:

[0155] The device sends JSON data to the server as an HTTP POST request.

[0156] Input: User inquiry content converted to JSON format

[0157] Output: HTTP POST request received by the server

[0158] Specific action: The device sends an HTTP request to the server over the network.

[0159] Step 4:

[0160] The server receives an HTTP POST request and parses the query content.

[0161] Input: HTTP POST request sent from the terminal

[0162] Output: Tokenization of query content and extraction of intent and target.

[0163] Specific operation: The natural language processing engine on the server tokenizes the query content, extracts specific keywords, and determines the intent and target.

[0164] Step 5:

[0165] The server executes SQL queries against the database based on its intent and target.

[0166] Input: Tokenized inquiry content with identified intent and target.

[0167] Output: Relevant information retrieved from the database (e.g., inventory information for product A)

[0168] Specific operation: The server dynamically generates SQL queries, executes them against the database, and retrieves the necessary information.

[0169] Step 6:

[0170] Based on the information acquired by the server, the response generation engine generates an appropriate response.

[0171] Input: Related information retrieved from the database

[0172] Output: Generated response (Example: "Product A is currently in stock")

[0173] Specific operation: The server uses a generation AI model based on specific data to generate a response in a format that is easy for the user to understand.

[0174] Step 7:

[0175] The server packages the generated response into JSON format and sends it to the terminal as an HTTP response.

[0176] Input: Generated response

[0177] Output: HTTP response received by the terminal

[0178] Specific operation: The response generated by the server is converted to JSON format, packaged as an HTTP response, and sent back to the terminal.

[0179] Step 8:

[0180] The system analyzes the response received by the terminal and displays it on the user interface.

[0181] Input: HTTP response received from the server

[0182] Output: The response displayed to the user (e.g., "Product A is currently in stock")

[0183] Specific operation: The terminal's software analyzes the response and displays it appropriately on the user interface, allowing the user to see the answer in real time.

[0184] (Application Example 1)

[0185] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0186] This invention relates to a system for responding quickly and efficiently to customer inquiries. In particular, it aims to provide a means for enabling staff in physical stores to appropriately answer customer questions and requests on the spot. In current systems, inquiries are often handled manually, which is time-consuming and may lead to a decrease in customer satisfaction. Therefore, this invention aims to provide a system that reduces the burden on staff and improves the speed and accuracy of customer service.

[0187] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0188] In this invention, the server includes means for the user to input an inquiry, means for sending the content of the inquiry to the server, means for the server to analyze the content of the inquiry using a natural language processing engine and extract the intent and target, means for obtaining information from a database based on the extracted information, means for a generative AI to generate an optimal response, means for sending the generated response to a terminal, means for the terminal to display the response to the user, and means for enabling staff to respond immediately to customer inquiries using a smart device. This makes it possible to respond quickly and accurately to customer inquiries.

[0189] An "inquiry" is the act of a customer expressing questions or requests regarding a product or service.

[0190] "Users" refer to store staff and customers who use the system to enter inquiries.

[0191] A "terminal" is a device used by a user to input and submit inquiries, and typically includes smartphones and computers.

[0192] A "server" is a central processing unit that analyzes query content and generates and sends appropriate responses.

[0193] A "natural language processing engine" is a software component that tokenizes user-inputted inquiries and extracts their intent and target.

[0194] "Intention" refers to the purpose or content of the question that the user is trying to achieve through their inquiry.

[0195] "Target" refers to the specific items of products or services related to the inquiry.

[0196] A "database" is an information management system that stores the information necessary to generate responses to queries.

[0197] "Generative AI" refers to artificial intelligence models that generate appropriate responses based on information obtained from a database.

[0198] A "smart device" is an interactive device used by staff in a physical store, and typically includes smartphones and tablet devices.

[0199] A "response" is a message generated by the server in response to a user's inquiry.

[0200] "Display" refers to the act of providing the user with a visual representation of the response generated on the device.

[0201] In this embodiment of the invention, a system is provided that enables efficient customer service in physical stores through the coordinated operation of a server, terminal, and user. This system automates a series of processes from receiving inquiries to generating and displaying responses.

[0202] 1. Program Overview

[0203] The program operates in the following sequence.

[0204] 1. Inquiry reception:

[0205] Users enter their inquiries into smart devices (such as smartphones or tablets) used within the physical store. For example, they might ask, "Do you have this product in stock?"

[0206] 2. Data transmission:

[0207] The terminal converts the entered query content into JSON format and sends it to the server. This includes the query's timestamp and user identification information.

[0208] 3. Analysis:

[0209] The server parses the received JSON data and uses a natural language processing engine (e.g., SpaCy, NLTK) to extract the intent and target of the query. For example, it might extract "stock check" and "product A".

[0210] 4. Database query:

[0211] Based on the extracted information, the relevant information is retrieved from a database (e.g., MySQL®, PostgreSQL). For example, inventory information for "Product A" is retrieved using an SQL query.

[0212] 5. Response generation:

[0213] Generative AI (e.g., OpenAI GPT-3) generates appropriate responses based on information obtained from a database. This AI also refers to past response history to provide specific and user-friendly responses. For example, it might generate a response such as, "Product A is currently in stock."

[0214] 6. Send a reply:

[0215] The generated response is converted to JSON format and sent from the server to the terminal as an HTTP response.

[0216] 7. Response display:

[0217] The terminal analyzes the received response and displays it on the smart device screen. Through this display, the user provides the customer with the appropriate answer. For example, a staff member might look at their smartphone and tell the customer, "Product A is currently in stock."

[0218] Hardware and software to be used

[0219] Smart devices: Smartphones, tablets

[0220] Server: Cloud service (e.g., AWS®, GCP)

[0221] Natural language processing engines: SpaCy, NLTK

[0222] Generative AI: OpenAI GPT-3

[0223] Database: MySQL, PostgreSQL

[0224] Specific example

[0225] Consider a scenario where a customer asks "Do you have this product in stock?" in a physical store. This question is sent from a smart device to a server, where a natural language processing engine extracts "stock check" and "product A." The inventory information for product A is retrieved from the database, and a generative AI generates the response "Product A is currently in stock." This response is sent to the terminal, which displays it on the screen, allowing staff to answer the customer based on that information.

[0226] Example of a prompt

[0227] Please generate a response based on the user's inquiry. The inquiry is "Do you have product A in stock?" Please refer to past response history and provide a specific and clear response.

[0228] Thus, the present invention contributes to improving customer satisfaction by streamlining customer service in physical stores through the coordinated operation of the user, terminal, and server.

[0229] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0230] Step 1:

[0231] Inquiry reception: Users enter their inquiry details using a smart device within the physical store. This input includes questions and requests, which are converted to JSON format by the device.

[0232] Input: User's inquiry

[0233] Data processing: Convert the entered inquiry content into JSON format.

[0234] Output: Query data in JSON format

[0235] Specific action: The user enters "Is this product in stock?" into their smartphone.

[0236] Step 2:

[0237] Data transmission: The terminal sends the converted JSON-formatted query data to the server. This transmission is performed using an HTTP POST request.

[0238] Input: Query data in JSON format

[0239] Data processing: Sent to the server as an HTTP POST request.

[0240] Output: JSON data received by the server

[0241] Specific operation: A JSON query asking "Is this product in stock?" is sent from the terminal to the server.

[0242] Step 3:

[0243] Analysis: The server analyzes the received JSON data and uses a natural language processing engine to extract the intent and target of the query. During this process, the input text is tokenized and keywords are extracted.

[0244] Input: JSON data received by the server

[0245] Data processing: Tokenize the data using a natural language processing engine and extract the intent and target.

[0246] Output: Extracted intent and target

[0247] Specific operation: The server extracts "Inventory Check" and "Product A".

[0248] Step 4:

[0249] Database query: The server executes a query against the database based on the extracted intent and target. It retrieves information from the database that corresponds to the query.

[0250] Input: Extracted intent and target

[0251] Data operations: Executing SQL queries against a database

[0252] Output: Information retrieved from the database

[0253] Specific operation: The server retrieves inventory information for "Product A" from the database.

[0254] Step 5:

[0255] Response Generation: The server uses a generative AI to generate the optimal response based on the acquired information. This AI also refers to past response history to provide specific and user-friendly responses.

[0256] Input: Information obtained from the database

[0257] Data processing: Generative AI is used to generate the optimal response.

[0258] Output: Generated response

[0259] Specific operation: The generative AI generates the response "Product A is currently in stock."

[0260] Step 6:

[0261] Response transmission: The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0262] Input: Generated response

[0263] Data processing: Convert the response to JSON format and send it to the terminal.

[0264] Output: Response received by the terminal

[0265] Specific action: The generated response "Product A is currently in stock" is sent from the server to the terminal.

[0266] Step 7:

[0267] Response Display: The terminal analyzes the received response and displays it on the smart device screen. The user provides the customer with an appropriate answer through this display.

[0268] Input: Response received by the terminal

[0269] Data processing: Displaying received responses appropriately in the user interface.

[0270] Output: Displayed response

[0271] Specific action: The staff member informs the customer of the response displayed on their smartphone screen, "Product A is currently in stock."

[0272] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0273] This invention provides a more appropriate response to user inquiries by recognizing the user's emotions and adjusting the response accordingly in a system that automates responses to user inquiries. The specific operation of the system is described below.

[0274] 1. Inquiry Reception

[0275] User: The user enters their inquiry using the device's interface (e.g., a web browser or dedicated application). The inquiry may include questions about specific products or services.

[0276] Terminal: Convert the inquiry content entered by the user into JSON format and send it as an HTTP POST request to the server. This request also includes the user's identification information and the inquiry timestamp.

[0277] 2. Analysis of the Inquiry

[0278] Server: Parse the received HTTP POST request and extract the inquiry content. Start processing based on this content.

[0279] Server: Use the natural language processing engine to tokenize the extracted inquiry content. For example, divide it into words such as "Product A", "Inventory", and "There is".

[0280] Server: Based on the tokenized words, identify the intention of the inquiry (in this case, inventory check) and the target (in this case, Product A).

[0281] 3. Recognition of Emotions

[0282] Server: Use the emotion engine to recognize the user's emotion from the inquiry content. This emotion is classified into multiple categories such as joy, anger, sadness, surprise, etc.

[0283] 4. Database Query

[0284] Server: Generate a query to the database based on the intention and target identified by natural language processing and emotion recognition. For example, for an inventory check inquiry, generate an SQL query to obtain inventory information about the product and retrieve the necessary information from the database.

[0285] 5. Generation of Response

[0286] Server: Based on the acquired inventory information and recognized sentiment information, use generative AI to generate an optimal response. This AI also refers to the past response history and generates responses with tones and contents corresponding to the sentiment. For example, adjust a sentence like "Product A is currently in stock" into a form like "Please rest assured. Product A is currently in stock."

[0287] 6. Transmission of Response

[0288] Server: Convert the generated response into JSON format and send it to the terminal as an HTTP response.

[0289] Terminal: Analyze the received response and convert it into a format that can be displayed on the user interface.

[0290] 7. Display of Response

[0291] Terminal: Display the formatted response on the user interface. For example, display a message like "Please rest assured. Product A is currently in stock."

[0292] User: Check the displayed response and take the next action if necessary. For example, confirm the availability of stock and proceed with the purchase procedure.

[0293] Specific Example

[0294] Example 1: Checking the Stock of a Product

[0295] User: Input "Is there any stock of Product A?"

[0296] Terminal: Convert the content of this inquiry into JSON format and send it to the server.

[0297] Server: Analyze the content of the inquiry with a natural language processing engine, and extract the intention of "stock check" and the target of "Product A".

[0298] Server: Using the emotion engine, it recognizes the user's emotion as "uneasy" from the inquiry.

[0299] Server: Retrieves the inventory information of "Product A" from the database and generates a response saying "Please rest assured, there is currently inventory of Product A".

[0300] Server: Sends the generated response to the terminal.

[0301] Terminal: Displays "Please rest assured, there is currently inventory of Product A" to the user.

[0302] Example 2: Support inquiry (technical question)

[0303] User: Enters "Please teach me how to set up Device B".

[0304] Terminal: Converts the content of this inquiry into JSON format and sends it to the server.

[0305] Server: Analyzes the content of the inquiry using the natural language processing engine and extracts the intention of "setup method" and the target of "Device B".

[0306] Server: Using the emotion engine, it recognizes the user's emotion as "confused" from the inquiry.

[0307] Server: Retrieves information about "the setup method of Device B" from the database and generates a response saying "Don't worry, the setup method of Device B is as follows...".

[0308] Server: Sends the generated response to the terminal.

[0309] Terminal: Displays "Don't worry, the setup method of Device B is as follows..." to the user.

[0310] In this way, by recognizing user emotions and adjusting responses accordingly, a more user-friendly approach becomes possible. This system can improve the quality of customer service at corporate customer centers and contribute to increased customer satisfaction.

[0311] The following describes the processing flow.

[0312] Step 1:

[0313] User: Enter your inquiry using the terminal's user interface. For example, enter "Do you have product A in stock?".

[0314] Step 2:

[0315] Terminal: Converts the entered query content into JSON format. This includes user identification information and the query's timestamp.

[0316] Step 3:

[0317] Terminal: Sends the converted JSON data to the server as an HTTP POST request.

[0318] Step 4:

[0319] Server: Parses the received HTTP POST request and extracts the query content. Processing begins based on this content.

[0320] Step 5:

[0321] Server: Uses a natural language processing engine to tokenize the extracted query content. For example, it separates it into words such as "product A," "in stock," and "available."

[0322] Step 6:

[0323] Server: Based on tokenized words, it identifies the intent of the inquiry (in this case, inventory check) and the target (in this case, product A).

[0324] Step 7:

[0325] Server: Uses an emotion engine to recognize the user's emotions from the query content. For example, it classifies emotions such as "anxiety," "confusion," and "anger."

[0326] Step 8:

[0327] Server: Analyzes the emotional information recognized by the emotion engine and uses it to adjust the tone and content of the response. This information is referenced in a later response generation step.

[0328] Step 9:

[0329] Server: Generates queries against the database based on intent, target, and sentiment information. For example, it generates an SQL query to "retrieve inventory information for product A" and retrieves the necessary information from the database.

[0330] Step 10:

[0331] Server: Based on inventory information retrieved from the database, a generative AI is used to generate the optimal response. This AI refers to past interaction history and sentiment information to generate a response with a tone and content that matches the user's emotions. For example, it adjusts "Product A is currently in stock" to "Don't worry, Product A is currently in stock."

[0332] Step 11:

[0333] Server: Converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0334] Step 12:

[0335] Terminal: Analyzes the received HTTP response and converts it into a format that can be displayed in the user interface.

[0336] Step 13:

[0337] Terminal: Displays a formatted response in the user interface. For example, it might display the message, "Don't worry, product A is currently in stock."

[0338] Step 14:

[0339] User: Review the displayed response and take the next action as needed. For example, confirm that the item is in stock and proceed with the purchase.

[0340] The above outlines the specific processing steps of this system. Each step works in conjunction with the others, and combined with emotion recognition, this enables efficient and user-friendly customer service.

[0341] (Example 2)

[0342] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0343] Modern inquiry systems often generate uniform responses without considering user emotions, which can lead to decreased user satisfaction. Furthermore, the process of accurately interpreting inquiries and automatically generating appropriate responses often lacks precision and flexibility. This invention aims to provide more appropriate and user-friendly responses by utilizing user emotion recognition and natural language processing.

[0344] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0345] In this invention, the server includes means for analyzing the query content with a natural language processing engine and extracting the intent and target; means for recognizing the user's emotions based on the extracted information; and means for obtaining necessary information from a database based on the emotion recognition. This makes it possible to generate appropriate responses that correspond to the user's emotions, thereby improving the quality of responses and user satisfaction.

[0346] "User" refers to an end user who uses the system to make inquiries.

[0347] "Device" refers to the hardware or software that a user uses to enter information. Examples include smartphones, tablets, and personal computers.

[0348] A "server" refers to a computer system that receives data sent from a terminal, analyzes the content of the inquiry, accesses a database, and generates a response.

[0349] "Inquiry content" refers to the text data of questions and requests that users enter through the system.

[0350] A "natural language processing engine" refers to a computer program that analyzes text data to understand its grammar and meaning.

[0351] "Intent" refers to the action or request indicated by the user's inquiry. Examples include "checking inventory" or "inquiring about setup methods."

[0352] The "subject" refers to the specific item or service that the user's inquiry is referring to. For example, this could include "Product A" or "Device B."

[0353] An "emotion recognition engine" refers to a computer program that identifies a user's emotions from text data.

[0354] A "database" refers to a structured data set used to store information related to a query.

[0355] "Generative AI" refers to artificial intelligence that automatically generates responses and text based on input data.

[0356] "Response" refers to the content of the reply that the system generates in response to a user's inquiry.

[0357] This invention relates to a system that automates responses to user inquiries and adjusts those responses based on the user's emotions. The specific operation of the system is described below.

[0358] Hardware and software to be used

[0359] This system uses the following hardware and software:

[0360] Terminal: A device that displays user input and responses. Specifically, smartphones, tablets, and personal computers are used.

[0361] Server: A computer system that receives, analyzes, and generates responses from data. Software that runs on a server includes the following:

[0362] Natural language processing engines (e.g., Stanford NLP)

[0363] Emotion recognition engines (e.g., IBM Watson®, Google® Cloud Natural Language API)

[0364] Database management systems (e.g., MySQL, PostgreSQL)

[0365] Generative AI models (e.g., OpenAI GPT-4(registered trademark))

[0366] System Operation Description

[0367] 1. Means by which users can enter inquiries:

[0368] Users use their device's interface (e.g., a web browser or a dedicated application) to enter inquiries about specific products or services. For example, they might enter a specific question such as, "Do you have product A in stock?"

[0369] 2. Means by which the terminal sends the inquiry content to the server:

[0370] The terminal receives user input, converts it to JSON format, and sends it to the server as an HTTP POST request. This includes user identification information and a timestamp of the query.

[0371] 3. A means by which the server analyzes the query content using a natural language processing engine and extracts the intent and target:

[0372] The server parses the received HTTP POST request, uses a natural language processing engine to tokenize the query content, and extracts the intent (e.g., inventory check) and the target (e.g., product A).

[0373] 4. Means by which the server recognizes the user's emotions based on the extracted information:

[0374] The server uses an emotion recognition engine to identify the user's emotions from the content of their inquiry. For example, emotions such as "anxiety" or "confusion" may be identified.

[0375] 5. Means by which the server retrieves necessary information from the database based on emotion recognition:

[0376] Based on the results of natural language processing and sentiment recognition, SQL queries are generated against the database to retrieve the necessary information (e.g., inventory information and configuration instructions).

[0377] 6. Means for generating the optimal response based on information and emotional information acquired by a generative AI:

[0378] The server uses a generative AI model to generate responses based on acquired data and sentiment information. For example, it might adjust the response to say, "Don't worry, product A is currently in stock."

[0379] 7. Means for the server to send the generated response to the terminal:

[0380] The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0381] 8. Means by which the terminal displays a response to the user:

[0382] The terminal analyzes the received response, converts it into a viewable format, and displays it to the user. The user then takes the next action (e.g., proceed with a purchase) based on this information.

[0383] Examples of specific cases and prompt statements

[0384] Example 1: Checking product inventory

[0385] If a user asks, "Do you have product A in stock?":

[0386] 1. The device sends the inquiry details to the server.

[0387] 2. The server analyzes the query and extracts the intent (inventory check) and the target (product A).

[0388] 3. The emotion recognition engine identifies the user's emotion as "anxiety."

[0389] 4. Retrieve the necessary inventory information from the database.

[0390] 5. The generative AI generates the response, "Don't worry, product A is currently in stock."

[0391] 6. Send the generated response to the terminal.

[0392] 7. The device displays to the user, "Don't worry, product A is currently in stock."

[0393] Example of a prompt

[0394] If a user asks, "Do you have product A in stock?":

[0395] 1. Please check the stock availability of product A.

[0396] 2. Since the user is feeling anxious, please generate a response in a reassuring tone.

[0397] This system enables appropriate responses tailored to the user's emotions, thereby improving response quality and user satisfaction.

[0398] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0399] Program processing flow

[0400] Step 1:

[0401] The user enters the inquiry details using the terminal's interface.

[0402] Input: Text entered by the user (e.g., "Do you have product A in stock?").

[0403] Output: The entered text is retrieved as terminal data.

[0404] Operation: The user enters the inquiry via a web browser or a dedicated application.

[0405] Step 2:

[0406] The terminal converts the user's input into JSON format and sends it to the server.

[0407] Input: Text entered by the user.

[0408] Output: Data in JSON format (Example: {"userId": "12345", "timestamp": "2023-10-05T10:00:00Z", "query": "Do you have product A in stock?"}).

[0409] Operation: The terminal converts the input content into JSON format and sends it to the server as an HTTP POST request.

[0410] Step 3:

[0411] The server parses the received JSON data and extracts the query details.

[0412] Input: Data in JSON format.

[0413] Output: Inquiry details and user information (e.g., {'query': 'Do you have product A in stock?', 'userId': '12345'}).

[0414] Operation: The server receives an HTTP request and extracts the query content by parsing the JSON data.

[0415] Step 4:

[0416] The server uses a natural language processing engine to tokenize the query content, identifying its intent and target.

[0417] Input: Extracted query content.

[0418] Output: Tokenized data (e.g., {'intent': 'Check Inventory', 'target': 'Product A'}).

[0419] Operation: The server uses a natural language processing engine to break down the query into words and phrases, identifying the intent and target.

[0420] Step 5:

[0421] The server uses an emotion recognition engine to recognize the user's emotions.

[0422] Input: Extracted query content.

[0423] Output: Emotion classification data (e.g., {'emotion': 'anxiety'}).

[0424] Operation: The server uses an emotion recognition engine to recognize the user's emotions from the content of the inquiry.

[0425] Step 6:

[0426] The server queries the database based on intent, target, and sentiment data to retrieve the necessary information.

[0427] Input: Tokenized data and sentiment classification data.

[0428] Output: Information retrieved from the database (e.g., {'stock': 'available'}).

[0429] Operation: The server generates and executes SQL queries against the database to retrieve necessary information such as inventory data.

[0430] Step 7:

[0431] The server uses a generative AI model to generate the optimal response.

[0432] Input: Information and sentiment classification data obtained from a database.

[0433] Output: Generated response (e.g., "Rest assured, product A is currently in stock").

[0434] Operation: The server uses a generative AI model to generate appropriate responses based on emotions, using input data.

[0435] Step 8:

[0436] The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0437] Input: The generated response.

[0438] Output: Response data in JSON format.

[0439] Operation: The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0440] Step 9:

[0441] The terminal parses the response data in JSON format and converts it into a format that can be displayed in the user interface.

[0442] Input: Response data in JSON format.

[0443] Output: Displayable text (e.g., "Rest assured, product A is currently in stock").

[0444] Operation: The device parses the JSON data it receives and converts it into a human-readable text format.

[0445] Step 10:

[0446] The terminal displays the formatted response in the user interface.

[0447] Input: Displayable text.

[0448] Output: The response displayed to the user.

[0449] Operation: The terminal displays a response in the user interface, and the user confirms it.

[0450] These steps allow the system to recognize user emotions and provide appropriate responses, thereby improving the user experience.

[0451] (Application Example 2)

[0452] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0453] Conventional automated response systems can generate answers to inquiries, but they cannot recognize and adapt responses to user emotions. As a result, they fail to alleviate user anxiety and confusion, leading to decreased user satisfaction. This is especially true in virtual stores, where users are required to ask questions in real time and receive appropriate support on the spot, making emotionally sensitive responses crucial. To address this challenge, there is a need to provide a system that recognizes user emotions and adjusts the tone and content of responses accordingly.

[0454] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0455] In this invention, the server includes means for analyzing the query content with a natural language processing engine to extract the intent and target, means for recognizing the user's emotions, and means for adjusting the tone and content of the response based on the emotion recognition. This makes it possible to generate an appropriate response that takes the user's emotions into consideration.

[0456] An "inquiry" refers to a question or request that a user enters when seeking information or support regarding a product or service.

[0457] A "natural language processing engine" refers to a set of algorithms and technologies used to analyze text entered by a user and identify its intent and target.

[0458] "Emotion recognition" is a technology that determines emotions from user input and identifies feelings such as joy, anger, anxiety, and confusion.

[0459] "Tokenization" is the process of dividing input text into a series of words or phrases.

[0460] A "database" is a system that systematically stores information related to products and services, making it searchable and retrievalable as needed.

[0461] "Generative AI" refers to artificial intelligence technology that generates the optimal response to user inquiries.

[0462] A "device" refers to a device operated by a user (e.g., a smartphone, smart glasses, etc.).

[0463] A "response" refers to the information or answer provided by a server to a user.

[0464] A "virtual store" is a purchasing environment built in digital space, rather than a physical store.

[0465] The system of this invention is designed to enhance the user's shopping experience in virtual stores. Using smart glasses, it can process user inquiries in real time and provide responses based on emotion recognition. The specific operation of the system is described below.

[0466] 1. System Configuration

[0467] This system uses the following hardware and software components.

[0468] Hardware:

[0469] Smart glasses: Devices that capture the user's gaze and voice input. They have a built-in AR display and voice input microphone.

[0470] software:

[0471] A natural language processing engine (NLTK) is an engine that analyzes text entered by a user to identify its intent and target. Specific examples include NLTK and spaCy.

[0472] Emotion Recognition Engine: An engine that analyzes user input to identify emotions such as joy, anger, anxiety, and confusion. A concrete example is the IBM Watson Emotion Analysis API.

[0473] A database (Product Database) is a system that stores information related to products and services, and allows users to search and retrieve it as needed. A specific example is MySQL.

[0474] Generative AI (Response Generator): Artificial intelligence technology used to generate optimal responses to user inquiries. Specific examples include OpenAI GPT-3 or 4.

[0475] 2. Flow of Operation

[0476] 1. Inquiry Reception

[0477] Users ask questions about products and services through smart glasses. The voice-input inquiries are converted into text format and sent to the server.

[0478] 2. Inquiry Analysis

[0479] The server analyzes the submitted query using a natural language processing engine. It tokenizes the query and extracts the intent (e.g., inventory check) and the target (e.g., product A).

[0480] 3. Recognition of emotions

[0481] The server uses an emotion recognition engine to identify the user's emotions from the query. For example, it can detect if the user is anxious or confused.

[0482] 4. Database Query

[0483] Based on the extracted information, the server retrieves relevant company-specific information (e.g., inventory status) from the database.

[0484] 5. Response generation

[0485] The server uses generative AI to generate the optimal response based on the information it receives and the user's emotions. The response is adjusted in tone and content to be sensitive to the user's feelings.

[0486] 6. Sending and displaying responses

[0487] The generated response is converted to JSON format and sent to the smart glasses. The smart glasses then display the received response in the user interface.

[0488] 3. Specific examples

[0489] Example 1: Checking product inventory

[0490] User: "Do you have product A in stock?"

[0491] Server: The natural language processing engine identifies the intent of "check inventory" and the target of "product A". The emotion recognition engine identifies "anxiety". Inventory information is retrieved from the database, and the response generation engine generates "Don't worry, product A is currently in stock". This message is sent to the smart glasses and displayed.

[0492] Example of a prompt

[0493] User: "Do you have product A in stock?"

[0494] NLP Engine: Intention = "Check Inventory", Target = "Product A"

[0495] Emotional engine: Emotion = "Anxiety"

[0496] Database: SQL query = "SELECT stock FROM products WHERE name = 'Product A'"

[0497] AI generation: "Don't worry, product A is currently in stock."

[0498] In this way, this system allows users to receive emotionally sensitive, real-time responses, improving their shopping experience in virtual stores.

[0499] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0500] Step 1:

[0501] Inquiry reception

[0502] Users input questions about products and services via voice through smart glasses. The smart glasses convert this voice input into text format and send it to the server as JSON data. Here, the input is voice, and the output is text data. This makes it possible to treat the inquiry content as data.

[0503] Step 2:

[0504] Inquiry analysis

[0505] The server receives the transmitted JSON text data and parses it using a natural language processing engine (e.g., NLTK, spaCy). Specifically, it tokenizes the text and extracts the intent of the inquiry (e.g., inventory check) and the target (e.g., product A). The input is JSON data, and the output is tokenized words and intent / target. Through this analysis, the server can obtain the basic data necessary to take appropriate action.

[0506] Step 3:

[0507] Recognition of emotions

[0508] The server inputs the analyzed text data into an emotion recognition engine (e.g., IBM Watson Emotion Analysis API) to identify the user's emotions (e.g., anxiety, joy). The input is the analyzed text data, and the output is the emotion category. Emotion recognition provides the data needed to generate responses that correspond to the user's emotions.

[0509] Step 4:

[0510] Database query

[0511] The server sends queries to a database (e.g., MySQL) based on the user's intent and target to retrieve information about products and services. For example, if the target is "Product A" and the intent is "check inventory," the server will retrieve inventory information for Product A using an SQL query. The input is the query, and the output is the information retrieved from the database. This step provides specific information in response to the user's inquiry.

[0512] Step 5:

[0513] Response generation

[0514] The server uses generative AI (e.g., OpenAI GPT-3, GPT-4) to generate the optimal response based on acquired information and the user's emotions. For example, if inventory information is available, it will generate a response in a tone that considers the user's emotions, such as, "Don't worry, product A is currently in stock." The input is the acquired data and emotional information, and the output is the generated response text. This response generation enhances user satisfaction.

[0515] Step 6:

[0516] Sending and displaying responses

[0517] The server converts the generated response into JSON format and sends it to the smart glasses. The smart glasses parse the received JSON data and display it on the user interface. The input is the generated response text, and the output is the response message displayed on the smart glasses' screen. This step allows the user to receive answers in real time.

[0518] In this way, a system is realized that automatically generates and provides appropriate responses to user inquiries through each step.

[0519] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0520] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0521] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0522] [Second Embodiment]

[0523] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0524] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0525] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0526] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0527] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0528] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0529] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0530] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0531] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0532] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0533] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0534] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0535] In this embodiment of the invention, a system is provided that enables efficient customer service through the coordinated operation of the user, terminal, and server components. The specific operation of the system is described below.

[0536] 1. Inquiry Reception

[0537] User: The user enters their inquiry using the device's interface (e.g., a web browser or dedicated application). The inquiry may include questions about specific product names or services.

[0538] Terminal: Converts the user's input into JSON format and sends it to the server as an HTTP POST request. This request includes the user's identification information and the inquiry's timestamp.

[0539] 2. Inquiry Analysis

[0540] Server: The server uses a natural language processing engine to parse incoming HTTP POST requests. This engine tokenizes the query content, extracts specific keywords, and determines the intent of the query. It also identifies the product or service being inquired about.

[0541] 3. Database Query

[0542] Server: Executes queries against the database based on the intent and target identified by natural language processing. For example, if the query is for checking inventory, it generates an SQL query to retrieve inventory information about the product and retrieves the necessary information from the database.

[0543] 4. Generating a response

[0544] Server: Based on information retrieved from the database, a generative AI generates an appropriate response. This AI also refers to past response history to generate more appropriate and user-friendly responses. For example, it might generate a specific response such as, "Product A is currently in stock."

[0545] 5. Sending a response

[0546] Server: Packages the generated response into JSON format and sends it to the terminal as an HTTP response.

[0547] Terminal: The terminal analyzes the received response and displays it appropriately on the user interface. Through this display, the user can see the answer to their inquiry in real time.

[0548] Specific example

[0549] Example 1: Checking product inventory

[0550] User: Enters "Do you have product A in stock?"

[0551] Terminal: Convert this query into JSON format and send it to the server.

[0552] Server: The server analyzes the content of the inquiry using a natural language processing engine to extract the intent of "inventory check" and the target of "product A".

[0553] Server: Retrieves inventory information for "Product A" from the database and generates a response stating "Product A is currently in stock."

[0554] Server: Sends the generated response to the terminal.

[0555] Terminal: Displays "Product A is currently in stock" to the user.

[0556] Example 2: Support inquiry (technical question)

[0557] User: Enters "Please tell me how to configure device B."

[0558] Terminal: Convert this query into JSON format and send it to the server.

[0559] Server: Analyzes the query content using a natural language processing engine and extracts the intent of "configuration method" and the target of "device B".

[0560] Server: Retrieves information about "How to configure device B" from the database and generates a response that says, "The following are the steps to configure device B..."

[0561] Server: Sends the generated response to the terminal.

[0562] Terminal: Displays the user with the message, "The following steps will guide you through configuring device B..."

[0563] As described above, this system can significantly improve the efficiency of a company's customer service by automatically and quickly processing user inquiries.

[0564] The following describes the processing flow.

[0565] Step 1:

[0566] User: Enter your inquiry using the terminal's user interface. For example, enter "Do you have product A in stock?".

[0567] Step 2:

[0568] Terminal: Converts the entered query content into JSON format. This includes user identification information and the query's timestamp.

[0569] Step 3:

[0570] Terminal: Sends the converted JSON data to the server as an HTTP POST request.

[0571] Step 4:

[0572] Server: Parses the received HTTP POST request and extracts the query content. Processing begins based on this content.

[0573] Step 5:

[0574] Server: Uses a natural language processing engine to tokenize the extracted query content. For example, it separates it into words such as "product A," "in stock," and "available."

[0575] Step 6:

[0576] Server: Based on tokenized words, it identifies the intent of the inquiry (in this case, inventory check) and the target (in this case, product A).

[0577] Step 7:

[0578] Server: Generates queries against the database based on intent and target. For example, it generates an SQL query to "retrieve inventory information for product A".

[0579] Step 8:

[0580] Server: Executes the generated query against the database to retrieve relevant inventory information. For example, it retrieves information such as "Product A is currently in stock."

[0581] Step 9:

[0582] Server: Uses generative AI to generate the optimal response based on the acquired inventory information. For example, it generates the sentence, "Product A is currently in stock."

[0583] Step 10:

[0584] Server: Converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0585] Step 11:

[0586] Terminal: Analyzes the received HTTP response and converts it into a format that can be displayed in the user interface.

[0587] Step 12:

[0588] Terminal: Displays a formatted response in the user interface. For example, it displays the message "Product A is currently in stock."

[0589] Step 13:

[0590] User: Review the displayed response and take the next action as needed. For example, confirm that the item is in stock and proceed with the purchase.

[0591] The above outlines the specific processing steps of this system. Each step works in conjunction to enable efficient and accurate responses to user inquiries.

[0592] (Example 1)

[0593] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0594] Traditional customer service systems often have slow response times to user inquiries and may fail to provide appropriate answers. This leads to decreased efficiency in corporate customer service and reduced customer satisfaction. Furthermore, the lack of features to effectively utilize past interaction history makes it difficult to provide consistent answers to similar inquiries.

[0595] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0596] In this invention, the server includes means for analyzing with a language processing engine to extract intent and target,

[0597] A means of obtaining company-specific information from a data set based on extracted information,

[0598] A means by which a response generation engine generates an optimal response,

[0599] This includes enabling rapid and accurate analysis of inquiries and providing consistent and appropriate responses by leveraging past interaction history.

[0600] An "inquiry" refers to a request for information about a specific product or service that a user enters into an information processing device.

[0601] An "information processing device" is a terminal device used by users to input inquiries, such as a web browser or a device that utilizes a dedicated application.

[0602] A "host computer" refers to a central system that receives inquiries transmitted from information processing devices, analyzes them, and generates responses.

[0603] A "language processing engine" refers to the natural language processing technology used by a host computer to analyze query content, specifically an engine that performs tasks such as tokenization and intent extraction.

[0604] "Intention" refers to the purpose or objective that can be inferred from the user's inquiry.

[0605] "Target" refers to the product or service identified by the user's inquiry.

[0606] A "data collection" refers to an information source, such as a database, where multiple pieces of information are aggregated and stored.

[0607] A "response generation engine" refers to artificial intelligence technology that creates the optimal answer based on data acquired by a host computer.

[0608] "Optimal response" refers to information generated to answer a user's inquiry quickly and accurately.

[0609] "Presentation" refers to displaying the generated response to the user using an information processing device.

[0610] This system enables efficient customer service through the coordinated operation of users, terminals, and servers. The embodiments of this invention will be described in detail below.

[0611] Inquiry reception

[0612] User: Users enter their inquiries using the device's interface (e.g., a web browser or dedicated application). These inquiries may include questions about specific product names or services.

[0613] Terminal: The terminal converts the user's input into JSON format. This conversion includes the query content, user identification information, and query timestamp. This JSON data is sent from the terminal to the server as an HTTP POST request.

[0614] Inquiry analysis

[0615] Server: The server uses a natural language processing engine (e.g., Hugging Face's Transformers model) to parse incoming HTTP POST requests. This engine tokenizes the query content, extracts specific keywords, and determines the intent of the query. It also identifies the product or service being inquired about.

[0616] Database query

[0617] Server: Executes queries against the database based on the intent and target identified by natural language processing. For example, if the query is for checking inventory, it generates an SQL query to retrieve inventory information about the product and retrieves the necessary information from the database.

[0618] Response generation

[0619] Server: The server uses a generative AI model (e.g., OpenAI's GPT-3) to generate a response based on information retrieved from the database. This response is provided in a format that is easy for the user to understand. For example, a specific response such as "Product A is currently in stock" is generated.

[0620] Sending a response

[0621] Server: The server packages the generated response into JSON format and sends it to the terminal as an HTTP response.

[0622] Terminal: The terminal analyzes the received response and displays it appropriately on the user interface. The user can then see the answer to their inquiry in real time from this display.

[0623] Specific example

[0624] Example 1: Checking product inventory

[0625] User: Enters "Do you have product A in stock?" into the terminal.

[0626] Terminal: Convert this query into JSON format and send it to the server.

[0627] Server: The server uses a natural language processing engine to analyze the query content and extract the intent of "stock confirmation" and the target of "product A".

[0628] Server: Retrieves inventory information for "Product A" from the database and generates a response stating "Product A is currently in stock."

[0629] Server: Sends the generated response to the terminal.

[0630] Terminal: Displays "Product A is currently in stock" to the user.

[0631] Example 2: Support inquiry (technical question)

[0632] User: Enters "Please tell me how to configure device B" into the terminal.

[0633] Terminal: Convert this query into JSON format and send it to the server.

[0634] Server: The server analyzes the query content using a natural language processing engine and extracts the intent of "configuration method" and the target of "device B".

[0635] Server: Retrieves information about "How to configure device B" from the database and generates a response that says, "The following are the steps to configure device B..."

[0636] Server: Sends the generated response to the terminal.

[0637] Terminal: Displays the user with the message, "The following steps will guide you through configuring device B..."

[0638] As described above, this system can significantly improve the efficiency of a company's customer service by automatically and quickly processing user inquiries.

[0639] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0640] Step 1:

[0641] The user enters the inquiry details into the information processing device.

[0642] Input: A specific question about a product or service (e.g., "Do you have product A in stock?")

[0643] Output: The user's inquiry is input into the information processing device.

[0644] Step 2:

[0645] The terminal converts the user's inquiry into JSON format.

[0646] Input: Inquiry content entered by the user

[0647] Output: JSON data containing query details, user identification information, and query timestamp.

[0648] Specific operation: The terminal software analyzes the user's input, extracts the necessary information, and generates data in JSON format.

[0649] Step 3:

[0650] The device sends JSON data to the server as an HTTP POST request.

[0651] Input: User inquiry content converted to JSON format

[0652] Output: HTTP POST request received by the server

[0653] Specific action: The device sends an HTTP request to the server over the network.

[0654] Step 4:

[0655] The server receives an HTTP POST request and parses the query content.

[0656] Input: HTTP POST request sent from the terminal

[0657] Output: Tokenization of query content and extraction of intent and target.

[0658] Specific operation: The natural language processing engine on the server tokenizes the query content, extracts specific keywords, and determines the intent and target.

[0659] Step 5:

[0660] The server executes SQL queries against the database based on its intent and target.

[0661] Input: Tokenized inquiry content with identified intent and target.

[0662] Output: Relevant information retrieved from the database (e.g., inventory information for product A)

[0663] Specific operation: The server dynamically generates SQL queries, executes them against the database, and retrieves the necessary information.

[0664] Step 6:

[0665] Based on the information acquired by the server, the response generation engine generates an appropriate response.

[0666] Input: Related information retrieved from the database

[0667] Output: Generated response (Example: "Product A is currently in stock")

[0668] Specific operation: The server uses a generation AI model based on specific data to generate a response in a format that is easy for the user to understand.

[0669] Step 7:

[0670] The server packages the generated response into JSON format and sends it to the terminal as an HTTP response.

[0671] Input: Generated response

[0672] Output: HTTP response received by the terminal

[0673] Specific operation: The response generated by the server is converted to JSON format, packaged as an HTTP response, and sent back to the terminal.

[0674] Step 8:

[0675] The system analyzes the response received by the terminal and displays it on the user interface.

[0676] Input: HTTP response received from the server

[0677] Output: The response displayed to the user (e.g., "Product A is currently in stock")

[0678] Specific operation: The terminal's software analyzes the response and displays it appropriately on the user interface, allowing the user to see the answer in real time.

[0679] (Application Example 1)

[0680] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0681] This invention relates to a system for responding quickly and efficiently to customer inquiries. In particular, it aims to provide a means for enabling staff in physical stores to appropriately answer customer questions and requests on the spot. In current systems, inquiries are often handled manually, which is time-consuming and may lead to a decrease in customer satisfaction. Therefore, this invention aims to provide a system that reduces the burden on staff and improves the speed and accuracy of customer service.

[0682] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0683] In this invention, the server includes means for the user to input an inquiry, means for sending the content of the inquiry to the server, means for the server to analyze the content of the inquiry using a natural language processing engine and extract the intent and target, means for obtaining information from a database based on the extracted information, means for a generative AI to generate an optimal response, means for sending the generated response to a terminal, means for the terminal to display the response to the user, and means for enabling staff to respond immediately to customer inquiries using a smart device. This makes it possible to respond quickly and accurately to customer inquiries.

[0684] An "inquiry" is the act of a customer expressing questions or requests regarding a product or service.

[0685] "Users" refer to store staff and customers who use the system to enter inquiries.

[0686] A "terminal" is a device used by a user to input and submit inquiries, and typically includes smartphones and computers.

[0687] A "server" is a central processing unit that analyzes query content and generates and sends appropriate responses.

[0688] A "natural language processing engine" is a software component that tokenizes user-inputted inquiries and extracts their intent and target.

[0689] "Intention" refers to the purpose or content of the question that the user is trying to achieve through their inquiry.

[0690] "Target" refers to the specific items of products or services related to the inquiry.

[0691] A "database" is an information management system that stores the information necessary to generate responses to queries.

[0692] "Generative AI" refers to artificial intelligence models that generate appropriate responses based on information obtained from a database.

[0693] A "smart device" is an interactive device used by staff in a physical store, and typically includes smartphones and tablet devices.

[0694] A "response" is a message generated by the server in response to a user's inquiry.

[0695] "Display" refers to the act of providing the user with a visual representation of the response generated on the device.

[0696] In this embodiment of the invention, a system is provided that enables efficient customer service in physical stores through the coordinated operation of a server, terminal, and user. This system automates a series of processes from receiving inquiries to generating and displaying responses.

[0697] 1. Program Overview

[0698] The program operates in the following sequence:

[0699] 1. Inquiry reception:

[0700] Users enter their inquiries into smart devices (such as smartphones or tablets) used within the physical store. For example, they might ask, "Do you have this product in stock?"

[0701] 2. Data transmission:

[0702] The terminal converts the entered query content into JSON format and sends it to the server. This includes the query's timestamp and user identification information.

[0703] 3. Analysis:

[0704] The server parses the received JSON data and uses a natural language processing engine (e.g., SpaCy, NLTK) to extract the intent and target of the query. For example, it might extract "stock check" and "product A".

[0705] 4. Database query:

[0706] Based on the extracted information, retrieve the relevant information from the database (e.g., MySQL, PostgreSQL). For example, retrieve inventory information for "Product A" using an SQL query.

[0707] 5. Response generation:

[0708] Generative AI (e.g., OpenAI GPT-3) generates appropriate responses based on information obtained from a database. This AI also refers to past response history to provide specific and user-friendly responses. For example, it might generate a response such as, "Product A is currently in stock."

[0709] 6. Send a reply:

[0710] The generated response is converted to JSON format and sent from the server to the terminal as an HTTP response.

[0711] 7. Response display:

[0712] The terminal analyzes the received response and displays it on the smart device screen. Through this display, the user provides the customer with the appropriate answer. For example, a staff member might look at their smartphone and tell the customer, "Product A is currently in stock."

[0713] Hardware and software to be used

[0714] Smart devices: Smartphones, tablets

[0715] Server: Cloud service (e.g., AWS, GCP)

[0716] Natural language processing engines: SpaCy, NLTK

[0717] Generative AI: OpenAI GPT-3

[0718] Database: MySQL, PostgreSQL

[0719] Specific example

[0720] Consider a scenario where a customer asks "Do you have this product in stock?" in a physical store. This question is sent from a smart device to a server, where a natural language processing engine extracts "stock check" and "product A." The inventory information for product A is retrieved from the database, and a generative AI generates the response "Product A is currently in stock." This response is sent to the terminal, which displays it on the screen, allowing staff to answer the customer based on that information.

[0721] Example of a prompt

[0722] Please generate a response based on the user's inquiry. The inquiry is "Do you have product A in stock?" Please refer to past response history and provide a specific and clear response.

[0723] Thus, the present invention contributes to improving customer satisfaction by streamlining customer service in physical stores through the coordinated operation of the user, terminal, and server.

[0724] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0725] Step 1:

[0726] Inquiry Reception: Users enter their inquiry details using a smart device within the physical store. This input includes questions and requests, which are converted to JSON format by the device.

[0727] Input: User's inquiry

[0728] Data processing: Convert the entered inquiry content into JSON format.

[0729] Output: Query data in JSON format

[0730] Specific action: The user enters "Is this product in stock?" into their smartphone.

[0731] Step 2:

[0732] Data transmission: The terminal sends the converted JSON-formatted query data to the server. This transmission is performed using an HTTP POST request.

[0733] Input: Query data in JSON format

[0734] Data processing: Sent to the server as an HTTP POST request.

[0735] Output: JSON data received by the server

[0736] Specific operation: A JSON query asking "Is this product in stock?" is sent from the terminal to the server.

[0737] Step 3:

[0738] Analysis: The server analyzes the received JSON data and uses a natural language processing engine to extract the intent and target of the query. During this process, the input text is tokenized and keywords are extracted.

[0739] Input: JSON data received by the server

[0740] Data processing: Tokenize the data using a natural language processing engine and extract the intent and target.

[0741] Output: Extracted intent and target

[0742] Specific operation: The server extracts "Inventory Check" and "Product A".

[0743] Step 4:

[0744] Database query: The server executes a query against the database based on the extracted intent and target. It retrieves information from the database that corresponds to the query.

[0745] Input: Extracted intent and target

[0746] Data operations: Executing SQL queries against a database

[0747] Output: Information retrieved from the database

[0748] Specific operation: The server retrieves inventory information for "Product A" from the database.

[0749] Step 5:

[0750] Response Generation: The server uses a generative AI to generate the optimal response based on the acquired information. This AI also refers to past response history to provide specific and user-friendly responses.

[0751] Input: Information obtained from the database

[0752] Data processing: Generative AI is used to generate the optimal response.

[0753] Output: Generated response

[0754] Specific operation: The generative AI generates the response "Product A is currently in stock."

[0755] Step 6:

[0756] Response transmission: The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0757] Input: Generated response

[0758] Data processing: Convert the response to JSON format and send it to the terminal.

[0759] Output: Response received by the terminal

[0760] Specific action: The generated response "Product A is currently in stock" is sent from the server to the terminal.

[0761] Step 7:

[0762] Response Display: The terminal analyzes the received response and displays it on the smart device screen. The user provides the customer with an appropriate answer through this display.

[0763] Input: Response received by the terminal

[0764] Data processing: Displaying received responses appropriately in the user interface.

[0765] Output: Displayed response

[0766] Specific action: The staff member informs the customer of the response displayed on their smartphone screen, "Product A is currently in stock."

[0767] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0768] This invention provides a more appropriate response to user inquiries by recognizing the user's emotions and adjusting the response accordingly in a system that automates responses to user inquiries. The specific operation of the system is described below.

[0769] 1. Inquiry Reception

[0770] User: The user enters their inquiry using the device's interface (e.g., a web browser or dedicated application). The inquiry may include questions about specific products or services.

[0771] Terminal: Converts the user's input into JSON format and sends it to the server as an HTTP POST request. This request includes the user's identification information and the inquiry's timestamp.

[0772] 2. Inquiry Analysis

[0773] Server: Parses the received HTTP POST request and extracts the query content. Processing begins based on this content.

[0774] Server: Uses a natural language processing engine to tokenize the extracted query content. For example, it separates it into words such as "product A," "in stock," and "available."

[0775] Server: Based on tokenized words, it identifies the intent of the inquiry (in this case, inventory check) and the target (in this case, product A).

[0776] 3. Recognition of emotions

[0777] Server: Uses an emotion engine to recognize the user's emotions from the query content. These emotions are categorized into multiple categories, such as joy, anger, sadness, and surprise.

[0778] 4. Database Query

[0779] Server: Generates queries against the database based on intentions and targets identified through natural language processing and sentiment recognition. For example, if the query is for checking inventory, it generates an SQL query to retrieve inventory information about the product and retrieves the necessary information from the database.

[0780] 5. Response generation

[0781] Server: Based on acquired inventory information and recognized sentiment information, a generative AI is used to generate the optimal response. This AI also refers to past interaction history and generates responses with a tone and content appropriate to the sentiment. For example, it adjusts the sentence "Product A is currently in stock" to "Rest assured, Product A is currently in stock."

[0782] 6. Sending a response

[0783] Server: Converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0784] Terminal: Analyzes the received response and converts it into a format that can be displayed in the user interface.

[0785] 7. Display of response

[0786] Terminal: Displays a formatted response in the user interface. For example, it might display the message, "Don't worry, product A is currently in stock."

[0787] User: Review the displayed response and take the next action as needed. For example, confirm that the item is in stock and proceed with the purchase.

[0788] Specific example

[0789] Example 1: Checking product inventory

[0790] User: Enters "Do you have product A in stock?"

[0791] Terminal: Convert this query into JSON format and send it to the server.

[0792] Server: The server analyzes the content of the inquiry using a natural language processing engine to extract the intent of "inventory check" and the target of "product A".

[0793] Server: The emotion engine recognizes the user's emotion as "anxiety" based on the query.

[0794] Server: Retrieves inventory information for "Product A" from the database and generates a response saying, "Don't worry, Product A is currently in stock."

[0795] Server: Sends the generated response to the terminal.

[0796] Terminal: Displays the user with the message, "Don't worry, product A is currently in stock."

[0797] Example 2: Support inquiry (technical question)

[0798] User: Enters "Please tell me how to configure device B."

[0799] Terminal: Convert this query into JSON format and send it to the server.

[0800] Server: Analyzes the query content using a natural language processing engine and extracts the intent of "configuration method" and the target of "device B".

[0801] Server: The emotion engine recognizes the user's emotion as "confused" based on the query.

[0802] Server: Retrieves information about "How to configure device B" from the database and generates a response saying, "Don't worry, here's how to configure device B..."

[0803] Server: Sends the generated response to the terminal.

[0804] Terminal: Displays the user with the message, "Don't worry, here's how to set up device B..."

[0805] In this way, by recognizing user emotions and adjusting responses accordingly, a more user-friendly approach becomes possible. This system can improve the quality of customer service at corporate customer centers and contribute to increased customer satisfaction.

[0806] The following describes the processing flow.

[0807] Step 1:

[0808] User: Enter your inquiry using the terminal's user interface. For example, enter "Do you have product A in stock?".

[0809] Step 2:

[0810] Terminal: Converts the entered query content into JSON format. This includes user identification information and the query's timestamp.

[0811] Step 3:

[0812] Terminal: Sends the converted JSON data to the server as an HTTP POST request.

[0813] Step 4:

[0814] Server: Parses the received HTTP POST request and extracts the query content. Processing begins based on this content.

[0815] Step 5:

[0816] Server: Uses a natural language processing engine to tokenize the extracted query content. For example, it separates it into words such as "product A," "in stock," and "available."

[0817] Step 6:

[0818] Server: Based on tokenized words, it identifies the intent of the inquiry (in this case, inventory check) and the target (in this case, product A).

[0819] Step 7:

[0820] Server: Uses an emotion engine to recognize the user's emotions from the query content. For example, it classifies emotions such as "anxiety," "confusion," and "anger."

[0821] Step 8:

[0822] Server: Analyzes the emotional information recognized by the emotion engine and uses it to adjust the tone and content of the response. This information is referenced in a later response generation step.

[0823] Step 9:

[0824] Server: Generates queries against the database based on intent, target, and sentiment information. For example, it generates an SQL query to "retrieve inventory information for product A" and retrieves the necessary information from the database.

[0825] Step 10:

[0826] Server: Based on inventory information retrieved from the database, a generative AI is used to generate the optimal response. This AI refers to past interaction history and sentiment information to generate a response with a tone and content that matches the user's emotions. For example, it adjusts "Product A is currently in stock" to "Don't worry, Product A is currently in stock."

[0827] Step 11:

[0828] Server: Converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0829] Step 12:

[0830] Terminal: Analyzes the received HTTP response and converts it into a format that can be displayed in the user interface.

[0831] Step 13:

[0832] Terminal: Displays a formatted response in the user interface. For example, it might display the message, "Don't worry, product A is currently in stock."

[0833] Step 14:

[0834] User: Review the displayed response and take the next action as needed. For example, confirm that the item is in stock and proceed with the purchase.

[0835] The above outlines the specific processing steps of this system. Each step works in conjunction with the others, and combined with emotion recognition, this enables efficient and user-friendly customer service.

[0836] (Example 2)

[0837] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0838] Modern inquiry systems often generate uniform responses without considering user emotions, which can lead to decreased user satisfaction. Furthermore, the process of accurately interpreting inquiries and automatically generating appropriate responses often lacks precision and flexibility. This invention aims to provide more appropriate and user-friendly responses by utilizing user emotion recognition and natural language processing.

[0839] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0840] In this invention, the server includes means for analyzing the query content with a natural language processing engine and extracting the intent and target; means for recognizing the user's emotions based on the extracted information; and means for obtaining necessary information from a database based on the emotion recognition. This makes it possible to generate appropriate responses that correspond to the user's emotions, thereby improving the quality of responses and user satisfaction.

[0841] "User" refers to an end user who uses the system to make inquiries.

[0842] "Device" refers to the hardware or software that a user uses to enter information. Examples include smartphones, tablets, and personal computers.

[0843] A "server" refers to a computer system that receives data sent from a terminal, analyzes the content of the inquiry, accesses a database, and generates a response.

[0844] "Inquiry content" refers to the text data of questions and requests that users enter through the system.

[0845] A "natural language processing engine" refers to a computer program that analyzes text data to understand its grammar and meaning.

[0846] "Intent" refers to the action or request indicated by the user's inquiry. Examples include "checking inventory" or "inquiring about setup methods."

[0847] The "subject" refers to the specific item or service that the user's inquiry is referring to. For example, this could include "Product A" or "Device B."

[0848] An "emotion recognition engine" refers to a computer program that identifies a user's emotions from text data.

[0849] A "database" refers to a structured data set used to store information related to a query.

[0850] "Generative AI" refers to artificial intelligence that automatically generates responses and text based on input data.

[0851] "Response" refers to the content of the reply that the system generates in response to a user's inquiry.

[0852] This invention relates to a system that automates responses to user inquiries and adjusts those responses based on the user's emotions. The specific operation of the system is described below.

[0853] Hardware and software to be used

[0854] This system uses the following hardware and software:

[0855] Terminal: A device that displays user input and responses. Specifically, smartphones, tablets, and personal computers are used.

[0856] Server: A computer system that receives, analyzes, and generates responses from data. Software that runs on a server includes the following:

[0857] Natural language processing engines (e.g., Stanford NLP)

[0858] Emotion recognition engines (e.g., IBM Watson, Google Cloud Natural Language API)

[0859] Database management systems (e.g., MySQL, PostgreSQL)

[0860] Generative AI models (e.g., OpenAI GPT-4)

[0861] System Operation Description

[0862] 1. Means by which users can enter inquiries:

[0863] Users use their device's interface (e.g., a web browser or a dedicated application) to enter inquiries about specific products or services. For example, they might enter a specific question such as, "Do you have product A in stock?"

[0864] 2. Means by which the terminal sends the inquiry content to the server:

[0865] The terminal receives user input, converts it to JSON format, and sends it to the server as an HTTP POST request. This includes user identification information and a timestamp of the query.

[0866] 3. A means by which the server analyzes the query content using a natural language processing engine and extracts the intent and target:

[0867] The server parses the received HTTP POST request, uses a natural language processing engine to tokenize the query content, and extracts the intent (e.g., inventory check) and the target (e.g., product A).

[0868] 4. Means by which the server recognizes the user's emotions based on the extracted information:

[0869] The server uses an emotion recognition engine to identify the user's emotions from the content of their inquiry. For example, emotions such as "anxiety" or "confusion" may be identified.

[0870] 5. Means by which the server retrieves necessary information from the database based on emotion recognition:

[0871] Based on the results of natural language processing and sentiment recognition, SQL queries are generated against the database to retrieve the necessary information (e.g., inventory information and configuration instructions).

[0872] 6. Means for generating the optimal response based on information and emotional information acquired by a generative AI:

[0873] The server uses a generative AI model to generate responses based on acquired data and sentiment information. For example, it might adjust the response to say, "Don't worry, product A is currently in stock."

[0874] 7. Means for the server to send the generated response to the terminal:

[0875] The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0876] 8. Means by which the terminal displays a response to the user:

[0877] The terminal analyzes the received response, converts it into a viewable format, and displays it to the user. The user then takes the next action (e.g., proceed with a purchase) based on this information.

[0878] Examples of specific cases and prompt statements

[0879] Example 1: Checking product inventory

[0880] If a user asks, "Do you have product A in stock?":

[0881] 1. The device sends the inquiry details to the server.

[0882] 2. The server analyzes the query and extracts the intent (inventory check) and the target (product A).

[0883] 3. The emotion recognition engine identifies the user's emotion as "anxiety."

[0884] 4. Retrieve the necessary inventory information from the database.

[0885] 5. The generative AI generates the response, "Don't worry, product A is currently in stock."

[0886] 6. Send the generated response to the terminal.

[0887] 7. The device displays to the user, "Don't worry, product A is currently in stock."

[0888] Example of a prompt

[0889] If a user asks, "Do you have product A in stock?":

[0890] 1. Please check the stock availability of product A.

[0891] 2. Since the user is feeling anxious, please generate a response in a reassuring tone.

[0892] This system enables appropriate responses tailored to the user's emotions, thereby improving response quality and user satisfaction.

[0893] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0894] Program processing flow

[0895] Step 1:

[0896] The user enters the inquiry details using the terminal's interface.

[0897] Input: Text entered by the user (e.g., "Do you have product A in stock?").

[0898] Output: The entered text is retrieved as terminal data.

[0899] Operation: The user enters the inquiry via a web browser or a dedicated application.

[0900] Step 2:

[0901] The terminal converts the user's input into JSON format and sends it to the server.

[0902] Input: Text entered by the user.

[0903] Output: Data in JSON format (Example: {"userId": "12345", "timestamp": "2023-10-05T10:00:00Z", "query": "Do you have product A in stock?"}).

[0904] Operation: The terminal converts the input content into JSON format and sends it to the server as an HTTP POST request.

[0905] Step 3:

[0906] The server parses the received JSON data and extracts the query details.

[0907] Input: Data in JSON format.

[0908] Output: Inquiry details and user information (e.g., {'query': 'Do you have product A in stock?', 'userId': '12345'}).

[0909] Operation: The server receives an HTTP request and extracts the query content by parsing the JSON data.

[0910] Step 4:

[0911] The server uses a natural language processing engine to tokenize the query content, identifying its intent and target.

[0912] Input: Extracted query content.

[0913] Output: Tokenized data (e.g., {'intent': 'Check Inventory', 'target': 'Product A'}).

[0914] Operation: The server uses a natural language processing engine to break down the query into words and phrases, identifying the intent and target.

[0915] Step 5:

[0916] The server uses an emotion recognition engine to recognize the user's emotions.

[0917] Input: Extracted query content.

[0918] Output: Emotion classification data (e.g., {'emotion': 'anxiety'}).

[0919] Operation: The server uses an emotion recognition engine to recognize the user's emotions from the content of the inquiry.

[0920] Step 6:

[0921] The server queries the database based on intent, target, and sentiment data to retrieve the necessary information.

[0922] Input: Tokenized data and sentiment classification data.

[0923] Output: Information retrieved from the database (e.g., {'stock': 'available'}).

[0924] Operation: The server generates and executes SQL queries against the database to retrieve necessary information such as inventory data.

[0925] Step 7:

[0926] The server uses a generative AI model to generate the optimal response.

[0927] Input: Information and sentiment classification data obtained from a database.

[0928] Output: Generated response (e.g., "Rest assured, product A is currently in stock").

[0929] Operation: The server uses a generative AI model to generate appropriate responses based on emotions, using input data.

[0930] Step 8:

[0931] The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0932] Input: The generated response.

[0933] Output: Response data in JSON format.

[0934] Operation: The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[0935] Step 9:

[0936] The terminal parses the response data in JSON format and converts it into a format that can be displayed in the user interface.

[0937] Input: Response data in JSON format.

[0938] Output: Displayable text (e.g., "Rest assured, product A is currently in stock").

[0939] Operation: The device parses the JSON data it receives and converts it into a human-readable text format.

[0940] Step 10:

[0941] The terminal displays the formatted response in the user interface.

[0942] Input: Displayable text.

[0943] Output: The response displayed to the user.

[0944] Operation: The terminal displays a response in the user interface, and the user confirms it.

[0945] These steps allow the system to recognize user emotions and provide appropriate responses, thereby improving the user experience.

[0946] (Application Example 2)

[0947] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0948] Conventional automated response systems can generate answers to inquiries, but they cannot recognize and adapt responses to user emotions. As a result, they fail to alleviate user anxiety and confusion, leading to decreased user satisfaction. This is especially true in virtual stores, where users are required to ask questions in real time and receive appropriate support on the spot, making emotionally sensitive responses crucial. To address this challenge, there is a need to provide a system that recognizes user emotions and adjusts the tone and content of responses accordingly.

[0949] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0950] In this invention, the server includes means for analyzing the query content with a natural language processing engine to extract the intent and target, means for recognizing the user's emotions, and means for adjusting the tone and content of the response based on the emotion recognition. This makes it possible to generate an appropriate response that takes the user's emotions into consideration.

[0951] An "inquiry" refers to a question or request that a user enters when seeking information or support regarding a product or service.

[0952] A "natural language processing engine" refers to a set of algorithms and technologies used to analyze text entered by a user and identify its intent and target.

[0953] "Emotion recognition" is a technology that determines emotions from user input and identifies feelings such as joy, anger, anxiety, and confusion.

[0954] "Tokenization" is the process of dividing input text into a series of words or phrases.

[0955] A "database" is a system that systematically stores information related to products and services, making it searchable and retrievalable as needed.

[0956] "Generative AI" refers to artificial intelligence technology that generates the optimal response to user inquiries.

[0957] A "device" refers to a device operated by a user (e.g., a smartphone, smart glasses, etc.).

[0958] A "response" refers to the information or answer provided by a server to a user.

[0959] A "virtual store" is a purchasing environment built in digital space, rather than a physical store.

[0960] The system of this invention is designed to enhance the user's shopping experience in virtual stores. Using smart glasses, it can process user inquiries in real time and provide responses based on emotion recognition. The specific operation of the system is described below.

[0961] 1. System Configuration

[0962] This system uses the following hardware and software components.

[0963] Hardware:

[0964] Smart glasses: Devices that capture the user's gaze and voice input. They have a built-in AR display and voice input microphone.

[0965] software:

[0966] A natural language processing engine (NLTK) is an engine that analyzes text entered by a user to identify its intent and target. Specific examples include NLTK and spaCy.

[0967] Emotion Recognition Engine: An engine that analyzes user input to identify emotions such as joy, anger, anxiety, and confusion. A concrete example is the IBM Watson Emotion Analysis API.

[0968] A database (Product Database) is a system that stores information related to products and services, and allows users to search and retrieve it as needed. A specific example is MySQL.

[0969] Generative AI (Response Generator): Artificial intelligence technology used to generate optimal responses to user inquiries. Specific examples include OpenAI GPT-3 or 4.

[0970] 2. Flow of Operation

[0971] 1. Inquiry Reception

[0972] Users ask questions about products and services through smart glasses. The voice-input inquiries are converted into text format and sent to the server.

[0973] 2. Inquiry Analysis

[0974] The server analyzes the submitted query using a natural language processing engine. It tokenizes the query and extracts the intent (e.g., inventory check) and the target (e.g., product A).

[0975] 3. Recognition of emotions

[0976] The server uses an emotion recognition engine to identify the user's emotions from the query. For example, it can detect if the user is anxious or confused.

[0977] 4. Database Query

[0978] Based on the extracted information, the server retrieves relevant company-specific information (e.g., inventory status) from the database.

[0979] 5. Response generation

[0980] The server uses generative AI to generate the optimal response based on the information it receives and the user's emotions. The response is adjusted in tone and content to be sensitive to the user's feelings.

[0981] 6. Sending and displaying responses

[0982] The generated response is converted to JSON format and sent to the smart glasses. The smart glasses then display the received response in the user interface.

[0983] 3. Specific examples

[0984] Example 1: Checking product inventory

[0985] User: "Do you have product A in stock?"

[0986] Server: The natural language processing engine identifies the intent of "check inventory" and the target of "product A". The emotion recognition engine identifies "anxiety". Inventory information is retrieved from the database, and the response generation engine generates "Don't worry, product A is currently in stock". This message is sent to the smart glasses and displayed.

[0987] Example of a prompt

[0988] User: "Do you have product A in stock?"

[0989] NLP Engine: Intention = "Check Inventory", Target = "Product A"

[0990] Emotional engine: Emotion = "Anxiety"

[0991] Database: SQL query = "SELECT stock FROM products WHERE name = 'Product A'"

[0992] AI generation: "Don't worry, product A is currently in stock."

[0993] In this way, this system allows users to receive emotionally sensitive, real-time responses, improving their shopping experience in virtual stores.

[0994] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0995] Step 1:

[0996] Inquiry reception

[0997] Users input questions about products and services via voice through smart glasses. The smart glasses convert this voice input into text format and send it to the server as JSON data. Here, the input is voice, and the output is text data. This makes it possible to treat the inquiry content as data.

[0998] Step 2:

[0999] Inquiry analysis

[1000] The server receives the transmitted JSON text data and parses it using a natural language processing engine (e.g., NLTK, spaCy). Specifically, it tokenizes the text and extracts the intent of the inquiry (e.g., inventory check) and the target (e.g., product A). The input is JSON data, and the output is tokenized words and intent / target. Through this analysis, the server can obtain the basic data necessary to take appropriate action.

[1001] Step 3:

[1002] Recognition of emotions

[1003] The server inputs the analyzed text data into an emotion recognition engine (e.g., IBM Watson Emotion Analysis API) to identify the user's emotions (e.g., anxiety, joy). The input is the analyzed text data, and the output is the emotion category. Emotion recognition provides the data needed to generate responses that correspond to the user's emotions.

[1004] Step 4:

[1005] Database query

[1006] The server sends queries to a database (e.g., MySQL) based on the user's intent and target to retrieve information about products and services. For example, if the target is "Product A" and the intent is "check inventory," the server will retrieve inventory information for Product A using an SQL query. The input is the query, and the output is the information retrieved from the database. This step provides specific information in response to the user's inquiry.

[1007] Step 5:

[1008] Response generation

[1009] The server uses generative AI (e.g., OpenAI GPT-3, GPT-4) to generate the optimal response based on acquired information and the user's emotions. For example, if inventory information is available, it will generate a response in a tone that considers the user's emotions, such as, "Don't worry, product A is currently in stock." The input is the acquired data and emotional information, and the output is the generated response text. This response generation enhances user satisfaction.

[1010] Step 6:

[1011] Sending and displaying responses

[1012] The server converts the generated response into JSON format and sends it to the smart glasses. The smart glasses parse the received JSON data and display it on the user interface. The input is the generated response text, and the output is the response message displayed on the smart glasses' screen. This step allows the user to receive answers in real time.

[1013] In this way, a system is realized that automatically generates and provides appropriate responses to user inquiries through each step.

[1014] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1015] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1016] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1017] [Third Embodiment]

[1018] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1019] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1020] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1021] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1022] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1023] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1024] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1025] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1026] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1027] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1028] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1029] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1030] In this embodiment of the invention, a system is provided that enables efficient customer service through the coordinated operation of the user, terminal, and server components. The specific operation of the system is described below.

[1031] 1. Inquiry Reception

[1032] User: The user enters their inquiry using the device's interface (e.g., a web browser or dedicated application). The inquiry may include questions about specific product names or services.

[1033] Terminal: Converts the user's input into JSON format and sends it to the server as an HTTP POST request. This request includes the user's identification information and the inquiry's timestamp.

[1034] 2. Inquiry Analysis

[1035] Server: The server uses a natural language processing engine to parse incoming HTTP POST requests. This engine tokenizes the query content, extracts specific keywords, and determines the intent of the query. It also identifies the product or service being inquired about.

[1036] 3. Database Query

[1037] Server: Executes queries against the database based on the intent and target identified by natural language processing. For example, if the query is for checking inventory, it generates an SQL query to retrieve inventory information about the product and retrieves the necessary information from the database.

[1038] 4. Generating a response

[1039] Server: Based on information retrieved from the database, a generative AI generates an appropriate response. This AI also refers to past response history to generate more appropriate and user-friendly responses. For example, it might generate a specific response such as, "Product A is currently in stock."

[1040] 5. Sending a response

[1041] Server: Packages the generated response into JSON format and sends it to the terminal as an HTTP response.

[1042] Terminal: The terminal analyzes the received response and displays it appropriately on the user interface. Through this display, the user can see the answer to their inquiry in real time.

[1043] Specific example

[1044] Example 1: Checking product inventory

[1045] User: Enters "Do you have product A in stock?"

[1046] Terminal: Convert this query into JSON format and send it to the server.

[1047] Server: The server analyzes the content of the inquiry using a natural language processing engine to extract the intent of "inventory check" and the target of "product A".

[1048] Server: Retrieves inventory information for "Product A" from the database and generates a response stating "Product A is currently in stock."

[1049] Server: Sends the generated response to the terminal.

[1050] Terminal: Displays "Product A is currently in stock" to the user.

[1051] Example 2: Support inquiry (technical question)

[1052] User: Enters "Please tell me how to configure device B."

[1053] Terminal: Convert this query into JSON format and send it to the server.

[1054] Server: Analyzes the query content using a natural language processing engine and extracts the intent of "configuration method" and the target of "device B".

[1055] Server: Retrieves information about "How to configure device B" from the database and generates a response that says, "The following are the steps to configure device B..."

[1056] Server: Sends the generated response to the terminal.

[1057] Terminal: Displays the user with the message, "The following steps will guide you through configuring device B..."

[1058] As described above, this system can significantly improve the efficiency of a company's customer service by automatically and quickly processing user inquiries.

[1059] The following describes the processing flow.

[1060] Step 1:

[1061] User: Enter your inquiry using the terminal's user interface. For example, enter "Do you have product A in stock?".

[1062] Step 2:

[1063] Terminal: Converts the entered query content into JSON format. This includes user identification information and the query's timestamp.

[1064] Step 3:

[1065] Terminal: Sends the converted JSON data to the server as an HTTP POST request.

[1066] Step 4:

[1067] Server: Parses the received HTTP POST request and extracts the query content. Processing begins based on this content.

[1068] Step 5:

[1069] Server: Uses a natural language processing engine to tokenize the extracted query content. For example, it separates it into words such as "product A," "in stock," and "available."

[1070] Step 6:

[1071] Server: Based on tokenized words, it identifies the intent of the inquiry (in this case, inventory check) and the target (in this case, product A).

[1072] Step 7:

[1073] Server: Generates queries against the database based on intent and target. For example, it generates an SQL query to "retrieve inventory information for product A".

[1074] Step 8:

[1075] Server: Executes the generated query against the database to retrieve relevant inventory information. For example, it retrieves information such as "Product A is currently in stock."

[1076] Step 9:

[1077] Server: Uses generative AI to generate the optimal response based on the acquired inventory information. For example, it generates the sentence, "Product A is currently in stock."

[1078] Step 10:

[1079] Server: Converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[1080] Step 11:

[1081] Terminal: Analyzes the received HTTP response and converts it into a format that can be displayed in the user interface.

[1082] Step 12:

[1083] Terminal: Displays a formatted response in the user interface. For example, it displays the message "Product A is currently in stock."

[1084] Step 13:

[1085] User: Review the displayed response and take the next action as needed. For example, confirm that the item is in stock and proceed with the purchase.

[1086] The above outlines the specific processing steps of this system. Each step works in conjunction to enable efficient and accurate responses to user inquiries.

[1087] (Example 1)

[1088] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1089] Traditional customer service systems often have slow response times to user inquiries and may fail to provide appropriate answers. This leads to decreased efficiency in corporate customer service and reduced customer satisfaction. Furthermore, the lack of features to effectively utilize past interaction history makes it difficult to provide consistent answers to similar inquiries.

[1090] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1091] In this invention, the server includes means for analyzing with a language processing engine to extract intent and target,

[1092] A means of obtaining company-specific information from a data set based on extracted information,

[1093] A means by which a response generation engine generates an optimal response,

[1094] This includes enabling rapid and accurate analysis of inquiries and providing consistent and appropriate responses by leveraging past interaction history.

[1095] An "inquiry" refers to a request for information about a specific product or service that a user enters into an information processing device.

[1096] An "information processing device" is a terminal device used by users to input inquiries, such as a web browser or a device that utilizes a dedicated application.

[1097] A "host computer" refers to a central system that receives inquiries transmitted from information processing devices, analyzes them, and generates responses.

[1098] A "language processing engine" refers to the natural language processing technology used by a host computer to analyze query content, specifically an engine that performs tasks such as tokenization and intent extraction.

[1099] "Intention" refers to the purpose or objective that can be inferred from the user's inquiry.

[1100] "Target" refers to the product or service identified by the user's inquiry.

[1101] A "data collection" refers to an information source, such as a database, where multiple pieces of information are aggregated and stored.

[1102] A "response generation engine" refers to artificial intelligence technology that creates the optimal answer based on data acquired by a host computer.

[1103] "Optimal response" refers to information generated to answer a user's inquiry quickly and accurately.

[1104] "Presentation" refers to displaying the generated response to the user using an information processing device.

[1105] This system enables efficient customer service through the coordinated operation of users, terminals, and servers. The embodiments of this invention will be described in detail below.

[1106] Inquiry reception

[1107] User: Users enter their inquiries using the device's interface (e.g., a web browser or dedicated application). These inquiries may include questions about specific product names or services.

[1108] Terminal: The terminal converts the user's input into JSON format. This conversion includes the query content, user identification information, and query timestamp. This JSON data is sent from the terminal to the server as an HTTP POST request.

[1109] Inquiry analysis

[1110] Server: The server uses a natural language processing engine (e.g., Hugging Face's Transformers model) to parse incoming HTTP POST requests. This engine tokenizes the query content, extracts specific keywords, and determines the intent of the query. It also identifies the product or service being inquired about.

[1111] Database query

[1112] Server: Executes queries against the database based on the intent and target identified by natural language processing. For example, if the query is for checking inventory, it generates an SQL query to retrieve inventory information about the product and retrieves the necessary information from the database.

[1113] Response generation

[1114] Server: The server uses a generative AI model (e.g., OpenAI's GPT-3) to generate a response based on information retrieved from the database. This response is provided in a format that is easy for the user to understand. For example, a specific response such as "Product A is currently in stock" is generated.

[1115] Sending a response

[1116] Server: The server packages the generated response into JSON format and sends it to the terminal as an HTTP response.

[1117] Terminal: The terminal analyzes the received response and displays it appropriately on the user interface. The user can then see the answer to their inquiry in real time from this display.

[1118] Specific example

[1119] Example 1: Checking product inventory

[1120] User: Enters "Do you have product A in stock?" into the terminal.

[1121] Terminal: Convert this query into JSON format and send it to the server.

[1122] Server: The server uses a natural language processing engine to analyze the query content and extract the intent of "stock confirmation" and the target of "product A".

[1123] Server: Retrieves inventory information for "Product A" from the database and generates a response stating "Product A is currently in stock."

[1124] Server: Sends the generated response to the terminal.

[1125] Terminal: Displays "Product A is currently in stock" to the user.

[1126] Example 2: Support inquiry (technical question)

[1127] User: Enters "Please tell me how to configure device B" into the terminal.

[1128] Terminal: Convert this query into JSON format and send it to the server.

[1129] Server: The server analyzes the query content using a natural language processing engine and extracts the intent of "configuration method" and the target of "device B".

[1130] Server: Retrieves information about "How to configure device B" from the database and generates a response that says, "The following are the steps to configure device B..."

[1131] Server: Sends the generated response to the terminal.

[1132] Terminal: Displays the user with the message, "The following steps will guide you through configuring device B..."

[1133] As described above, this system can significantly improve the efficiency of a company's customer service by automatically and quickly processing user inquiries.

[1134] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1135] Step 1:

[1136] The user enters the inquiry details into the information processing device.

[1137] Input: A specific question about a product or service (e.g., "Do you have product A in stock?")

[1138] Output: The user's inquiry is input into the information processing device.

[1139] Step 2:

[1140] The terminal converts the user's inquiry into JSON format.

[1141] Input: Inquiry content entered by the user

[1142] Output: JSON data containing query details, user identification information, and query timestamp.

[1143] Specific operation: The terminal software analyzes the user's input, extracts the necessary information, and generates data in JSON format.

[1144] Step 3:

[1145] The device sends JSON data to the server as an HTTP POST request.

[1146] Input: User inquiry content converted to JSON format

[1147] Output: HTTP POST request received by the server

[1148] Specific action: The device sends an HTTP request to the server over the network.

[1149] Step 4:

[1150] The server receives an HTTP POST request and parses the query content.

[1151] Input: HTTP POST request sent from the terminal

[1152] Output: Tokenization of query content and extraction of intent and target.

[1153] Specific operation: The natural language processing engine on the server tokenizes the query content, extracts specific keywords, and determines the intent and target.

[1154] Step 5:

[1155] The server executes SQL queries against the database based on its intent and target.

[1156] Input: Tokenized inquiry content with identified intent and target.

[1157] Output: Relevant information retrieved from the database (e.g., inventory information for product A)

[1158] Specific operation: The server dynamically generates SQL queries, executes them against the database, and retrieves the necessary information.

[1159] Step 6:

[1160] Based on the information acquired by the server, the response generation engine generates an appropriate response.

[1161] Input: Related information retrieved from the database

[1162] Output: Generated response (Example: "Product A is currently in stock")

[1163] Specific operation: The server uses a generation AI model based on specific data to generate a response in a format that is easy for the user to understand.

[1164] Step 7:

[1165] The server packages the generated response into JSON format and sends it to the terminal as an HTTP response.

[1166] Input: Generated response

[1167] Output: HTTP response received by the terminal

[1168] Specific operation: The response generated by the server is converted to JSON format, packaged as an HTTP response, and sent back to the terminal.

[1169] Step 8:

[1170] The system analyzes the response received by the terminal and displays it on the user interface.

[1171] Input: HTTP response received from the server

[1172] Output: The response displayed to the user (e.g., "Product A is currently in stock")

[1173] Specific operation: The terminal's software analyzes the response and displays it appropriately on the user interface, allowing the user to see the answer in real time.

[1174] (Application Example 1)

[1175] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1176] This invention relates to a system for responding quickly and efficiently to customer inquiries. In particular, it aims to provide a means for enabling staff in physical stores to appropriately answer customer questions and requests on the spot. In current systems, inquiries are often handled manually, which is time-consuming and may lead to a decrease in customer satisfaction. Therefore, this invention aims to provide a system that reduces the burden on staff and improves the speed and accuracy of customer service.

[1177] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1178] In this invention, the server includes means for the user to input an inquiry, means for sending the content of the inquiry to the server, means for the server to analyze the content of the inquiry using a natural language processing engine and extract the intent and target, means for obtaining information from a database based on the extracted information, means for a generative AI to generate an optimal response, means for sending the generated response to a terminal, means for the terminal to display the response to the user, and means for enabling staff to respond immediately to customer inquiries using a smart device. This makes it possible to respond quickly and accurately to customer inquiries.

[1179] An "inquiry" is the act of a customer expressing questions or requests regarding a product or service.

[1180] "Users" refer to store staff and customers who use the system to enter inquiries.

[1181] A "terminal" is a device used by a user to input and submit inquiries, and typically includes smartphones and computers.

[1182] A "server" is a central processing unit that analyzes query content and generates and sends appropriate responses.

[1183] A "natural language processing engine" is a software component that tokenizes user-inputted inquiries and extracts their intent and target.

[1184] "Intention" refers to the purpose or content of the question that the user is trying to achieve through their inquiry.

[1185] "Target" refers to the specific items of products or services related to the inquiry.

[1186] A "database" is an information management system that stores the information necessary to generate responses to queries.

[1187] "Generative AI" refers to artificial intelligence models that generate appropriate responses based on information obtained from a database.

[1188] A "smart device" is an interactive device used by staff in a physical store, and typically includes smartphones and tablet devices.

[1189] A "response" is a message generated by the server in response to a user's inquiry.

[1190] "Display" refers to the act of providing the user with a visual representation of the response generated on the device.

[1191] In this embodiment of the invention, a system is provided that enables efficient customer service in physical stores through the coordinated operation of a server, terminal, and user. This system automates a series of processes from receiving inquiries to generating and displaying responses.

[1192] 1. Program Overview

[1193] The program operates in the following sequence:

[1194] 1. Inquiry reception:

[1195] Users enter their inquiries into smart devices (such as smartphones or tablets) used within the physical store. For example, they might ask, "Do you have this product in stock?"

[1196] 2. Data transmission:

[1197] The terminal converts the entered query content into JSON format and sends it to the server. This includes the query's timestamp and user identification information.

[1198] 3. Analysis:

[1199] The server parses the received JSON data and uses a natural language processing engine (e.g., SpaCy, NLTK) to extract the intent and target of the query. For example, it might extract "stock check" and "product A".

[1200] 4. Database query:

[1201] Based on the extracted information, retrieve the relevant information from the database (e.g., MySQL, PostgreSQL). For example, retrieve inventory information for "Product A" using an SQL query.

[1202] 5. Response generation:

[1203] Generative AI (e.g., OpenAI GPT-3) generates appropriate responses based on information obtained from a database. This AI also refers to past response history to provide specific and user-friendly responses. For example, it might generate a response such as, "Product A is currently in stock."

[1204] 6. Send a reply:

[1205] The generated response is converted to JSON format and sent from the server to the terminal as an HTTP response.

[1206] 7. Response display:

[1207] The terminal analyzes the received response and displays it on the smart device screen. Through this display, the user provides the customer with the appropriate answer. For example, a staff member might look at their smartphone and tell the customer, "Product A is currently in stock."

[1208] Hardware and software to be used

[1209] Smart devices: Smartphones, tablets

[1210] Server: Cloud service (e.g., AWS, GCP)

[1211] Natural language processing engines: SpaCy, NLTK

[1212] Generative AI: OpenAI GPT-3

[1213] Database: MySQL, PostgreSQL

[1214] Specific example

[1215] Consider a scenario where a customer asks "Do you have this product in stock?" in a physical store. This question is sent from a smart device to a server, where a natural language processing engine extracts "stock check" and "product A." The inventory information for product A is retrieved from the database, and a generative AI generates the response "Product A is currently in stock." This response is sent to the terminal, which displays it on the screen, allowing staff to answer the customer based on that information.

[1216] Example of a prompt

[1217] Please generate a response based on the user's inquiry. The inquiry is "Do you have product A in stock?" Please refer to past response history and provide a specific and clear response.

[1218] Thus, the present invention contributes to improving customer satisfaction by streamlining customer service in physical stores through the coordinated operation of the user, terminal, and server.

[1219] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1220] Step 1:

[1221] Inquiry Reception: Users enter their inquiry details using a smart device within the physical store. This input includes questions and requests, which are converted to JSON format by the device.

[1222] Input: User's inquiry

[1223] Data processing: Convert the entered inquiry content into JSON format.

[1224] Output: Query data in JSON format

[1225] Specific action: The user enters "Is this product in stock?" into their smartphone.

[1226] Step 2:

[1227] Data transmission: The terminal sends the converted JSON-formatted query data to the server. This transmission is performed using an HTTP POST request.

[1228] Input: Query data in JSON format

[1229] Data processing: Sent to the server as an HTTP POST request.

[1230] Output: JSON data received by the server

[1231] Specific operation: A JSON query asking "Is this product in stock?" is sent from the terminal to the server.

[1232] Step 3:

[1233] Analysis: The server analyzes the received JSON data and uses a natural language processing engine to extract the intent and target of the query. During this process, the input text is tokenized and keywords are extracted.

[1234] Input: JSON data received by the server

[1235] Data processing: Tokenize the data using a natural language processing engine and extract the intent and target.

[1236] Output: Extracted intent and target

[1237] Specific operation: The server extracts "Inventory Check" and "Product A".

[1238] Step 4:

[1239] Database query: The server executes a query against the database based on the extracted intent and target. It retrieves information from the database that corresponds to the query.

[1240] Input: Extracted intent and target

[1241] Data operations: Executing SQL queries against a database

[1242] Output: Information retrieved from the database

[1243] Specific operation: The server retrieves inventory information for "Product A" from the database.

[1244] Step 5:

[1245] Response Generation: The server uses a generative AI to generate the optimal response based on the acquired information. This AI also refers to past response history to provide specific and user-friendly responses.

[1246] Input: Information obtained from the database

[1247] Data processing: Generative AI is used to generate the optimal response.

[1248] Output: Generated response

[1249] Specific operation: The generative AI generates the response "Product A is currently in stock."

[1250] Step 6:

[1251] Response transmission: The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[1252] Input: Generated response

[1253] Data processing: Convert the response to JSON format and send it to the terminal.

[1254] Output: Response received by the terminal

[1255] Specific action: The generated response "Product A is currently in stock" is sent from the server to the terminal.

[1256] Step 7:

[1257] Response Display: The terminal analyzes the received response and displays it on the smart device screen. The user provides the customer with an appropriate answer through this display.

[1258] Input: Response received by the terminal

[1259] Data processing: Displaying received responses appropriately in the user interface.

[1260] Output: Displayed response

[1261] Specific action: The staff member informs the customer of the response displayed on their smartphone screen, "Product A is currently in stock."

[1262] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1263] This invention provides a more appropriate response to user inquiries by recognizing the user's emotions and adjusting the response accordingly in a system that automates responses to user inquiries. The specific operation of the system is described below.

[1264] 1. Inquiry Reception

[1265] User: The user enters their inquiry using the device's interface (e.g., a web browser or dedicated application). The inquiry may include questions about specific products or services.

[1266] Terminal: Converts the user's input into JSON format and sends it to the server as an HTTP POST request. This request includes the user's identification information and the inquiry's timestamp.

[1267] 2. Inquiry Analysis

[1268] Server: Parses the received HTTP POST request and extracts the query content. Processing begins based on this content.

[1269] Server: Uses a natural language processing engine to tokenize the extracted query content. For example, it separates it into words such as "product A," "in stock," and "available."

[1270] Server: Based on tokenized words, it identifies the intent of the inquiry (in this case, inventory check) and the target (in this case, product A).

[1271] 3. Recognition of emotions

[1272] Server: Uses an emotion engine to recognize the user's emotions from the query content. These emotions are categorized into multiple categories, such as joy, anger, sadness, and surprise.

[1273] 4. Database Query

[1274] Server: Generates queries against the database based on intentions and targets identified through natural language processing and sentiment recognition. For example, if the query is for checking inventory, it generates an SQL query to retrieve inventory information about the product and retrieves the necessary information from the database.

[1275] 5. Response generation

[1276] Server: Based on acquired inventory information and recognized sentiment information, a generative AI is used to generate the optimal response. This AI also refers to past interaction history and generates responses with a tone and content appropriate to the sentiment. For example, it adjusts the sentence "Product A is currently in stock" to "Rest assured, Product A is currently in stock."

[1277] 6. Sending a response

[1278] Server: Converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[1279] Terminal: Analyzes the received response and converts it into a format that can be displayed in the user interface.

[1280] 7. Display of response

[1281] Terminal: Displays a formatted response in the user interface. For example, it might display the message, "Don't worry, product A is currently in stock."

[1282] User: Review the displayed response and take the next action as needed. For example, confirm that the item is in stock and proceed with the purchase.

[1283] Specific example

[1284] Example 1: Checking product inventory

[1285] User: Enters "Do you have product A in stock?"

[1286] Terminal: Convert this query into JSON format and send it to the server.

[1287] Server: The server analyzes the content of the inquiry using a natural language processing engine to extract the intent of "inventory check" and the target of "product A".

[1288] Server: The emotion engine recognizes the user's emotion as "anxiety" based on the query.

[1289] Server: Retrieves inventory information for "Product A" from the database and generates a response saying, "Don't worry, Product A is currently in stock."

[1290] Server: Sends the generated response to the terminal.

[1291] Terminal: Displays the user with the message, "Don't worry, product A is currently in stock."

[1292] Example 2: Support inquiry (technical question)

[1293] User: Enters "Please tell me how to configure device B."

[1294] Terminal: Convert this query into JSON format and send it to the server.

[1295] Server: Analyzes the query content using a natural language processing engine and extracts the intent of "configuration method" and the target of "device B".

[1296] Server: The emotion engine recognizes the user's emotion as "confused" based on the query.

[1297] Server: Retrieves information about "How to configure device B" from the database and generates a response saying, "Don't worry, here's how to configure device B..."

[1298] Server: Sends the generated response to the terminal.

[1299] Terminal: Displays the user with the message, "Don't worry, here's how to set up device B..."

[1300] In this way, by recognizing user emotions and adjusting responses accordingly, a more user-friendly approach becomes possible. This system can improve the quality of customer service at corporate customer centers and contribute to increased customer satisfaction.

[1301] The following describes the processing flow.

[1302] Step 1:

[1303] User: Enter your inquiry using the terminal's user interface. For example, enter "Do you have product A in stock?".

[1304] Step 2:

[1305] Terminal: Converts the entered query content into JSON format. This includes user identification information and the query's timestamp.

[1306] Step 3:

[1307] Terminal: Sends the converted JSON data to the server as an HTTP POST request.

[1308] Step 4:

[1309] Server: Parses the received HTTP POST request and extracts the query content. Processing begins based on this content.

[1310] Step 5:

[1311] Server: Uses a natural language processing engine to tokenize the extracted query content. For example, it separates it into words such as "product A," "in stock," and "available."

[1312] Step 6:

[1313] Server: Based on tokenized words, it identifies the intent of the inquiry (in this case, inventory check) and the target (in this case, product A).

[1314] Step 7:

[1315] Server: Uses an emotion engine to recognize the user's emotions from the query content. For example, it classifies emotions such as "anxiety," "confusion," and "anger."

[1316] Step 8:

[1317] Server: Analyzes the emotional information recognized by the emotion engine and uses it to adjust the tone and content of the response. This information is referenced in a later response generation step.

[1318] Step 9:

[1319] Server: Generates queries against the database based on intent, target, and sentiment information. For example, it generates an SQL query to "retrieve inventory information for product A" and retrieves the necessary information from the database.

[1320] Step 10:

[1321] Server: Based on inventory information retrieved from the database, a generative AI is used to generate the optimal response. This AI refers to past interaction history and sentiment information to generate a response with a tone and content that matches the user's emotions. For example, it adjusts "Product A is currently in stock" to "Don't worry, Product A is currently in stock."

[1322] Step 11:

[1323] Server: Converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[1324] Step 12:

[1325] Terminal: Analyzes the received HTTP response and converts it into a format that can be displayed in the user interface.

[1326] Step 13:

[1327] Terminal: Displays a formatted response in the user interface. For example, it might display the message, "Don't worry, product A is currently in stock."

[1328] Step 14:

[1329] User: Review the displayed response and take the next action as needed. For example, confirm that the item is in stock and proceed with the purchase.

[1330] The above outlines the specific processing steps of this system. Each step works in conjunction with the others, and combined with emotion recognition, this enables efficient and user-friendly customer service.

[1331] (Example 2)

[1332] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1333] Modern inquiry systems often generate uniform responses without considering user emotions, which can lead to decreased user satisfaction. Furthermore, the process of accurately interpreting inquiries and automatically generating appropriate responses often lacks precision and flexibility. This invention aims to provide more appropriate and user-friendly responses by utilizing user emotion recognition and natural language processing.

[1334] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1335] In this invention, the server includes means for analyzing the query content with a natural language processing engine and extracting the intent and target; means for recognizing the user's emotions based on the extracted information; and means for obtaining necessary information from a database based on the emotion recognition. This makes it possible to generate appropriate responses that correspond to the user's emotions, thereby improving the quality of responses and user satisfaction.

[1336] "User" refers to an end user who uses the system to make inquiries.

[1337] "Device" refers to the hardware or software that a user uses to enter information. Examples include smartphones, tablets, and personal computers.

[1338] A "server" refers to a computer system that receives data sent from a terminal, analyzes the content of the inquiry, accesses a database, and generates a response.

[1339] "Inquiry content" refers to the text data of questions and requests that users enter through the system.

[1340] A "natural language processing engine" refers to a computer program that analyzes text data to understand its grammar and meaning.

[1341] "Intent" refers to the action or request indicated by the user's inquiry. Examples include "checking inventory" or "inquiring about setup methods."

[1342] The "subject" refers to the specific item or service that the user's inquiry is referring to. For example, this could include "Product A" or "Device B."

[1343] An "emotion recognition engine" refers to a computer program that identifies a user's emotions from text data.

[1344] A "database" refers to a structured data set used to store information related to a query.

[1345] "Generative AI" refers to artificial intelligence that automatically generates responses and text based on input data.

[1346] "Response" refers to the content of the reply that the system generates in response to a user's inquiry.

[1347] This invention relates to a system that automates responses to user inquiries and adjusts those responses based on the user's emotions. The specific operation of the system is described below.

[1348] Hardware and software to be used

[1349] This system uses the following hardware and software:

[1350] Terminal: A device that displays user input and responses. Specifically, smartphones, tablets, and personal computers are used.

[1351] Server: A computer system that receives, analyzes, and generates responses from data. Software that runs on a server includes the following:

[1352] Natural language processing engines (e.g., Stanford NLP)

[1353] Emotion recognition engines (e.g., IBM Watson, Google Cloud Natural Language API)

[1354] Database management systems (e.g., MySQL, PostgreSQL)

[1355] Generative AI models (e.g., OpenAI GPT-4)

[1356] System Operation Description

[1357] 1. Means by which users can enter inquiries:

[1358] Users use their device's interface (e.g., a web browser or a dedicated application) to enter inquiries about specific products or services. For example, they might enter a specific question such as, "Do you have product A in stock?"

[1359] 2. Means by which the terminal sends the inquiry content to the server:

[1360] The terminal receives user input, converts it to JSON format, and sends it to the server as an HTTP POST request. This includes user identification information and a timestamp of the query.

[1361] 3. A means by which the server analyzes the query content using a natural language processing engine and extracts the intent and target:

[1362] The server parses the received HTTP POST request, uses a natural language processing engine to tokenize the query content, and extracts the intent (e.g., inventory check) and the target (e.g., product A).

[1363] 4. Means by which the server recognizes the user's emotions based on the extracted information:

[1364] The server uses an emotion recognition engine to identify the user's emotions from the content of their inquiry. For example, emotions such as "anxiety" or "confusion" may be identified.

[1365] 5. Means by which the server retrieves necessary information from the database based on emotion recognition:

[1366] Based on the results of natural language processing and sentiment recognition, SQL queries are generated against the database to retrieve the necessary information (e.g., inventory information and configuration instructions).

[1367] 6. Means for generating the optimal response based on information and emotional information acquired by a generative AI:

[1368] The server uses a generative AI model to generate responses based on acquired data and sentiment information. For example, it might adjust the response to say, "Don't worry, product A is currently in stock."

[1369] 7. Means for the server to send the generated response to the terminal:

[1370] The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[1371] 8. Means by which the terminal displays a response to the user:

[1372] The terminal analyzes the received response, converts it into a viewable format, and displays it to the user. The user then takes the next action (e.g., proceed with a purchase) based on this information.

[1373] Examples of specific cases and prompt statements

[1374] Example 1: Checking product inventory

[1375] If a user asks, "Do you have product A in stock?":

[1376] 1. The device sends the inquiry details to the server.

[1377] 2. The server analyzes the query and extracts the intent (inventory check) and the target (product A).

[1378] 3. The emotion recognition engine identifies the user's emotion as "anxiety."

[1379] 4. Retrieve the necessary inventory information from the database.

[1380] 5. The generative AI generates the response, "Don't worry, product A is currently in stock."

[1381] 6. Send the generated response to the terminal.

[1382] 7. The device displays to the user, "Don't worry, product A is currently in stock."

[1383] Example of a prompt

[1384] If a user asks, "Do you have product A in stock?":

[1385] 1. Please check the stock availability of product A.

[1386] 2. Since the user is feeling anxious, please generate a response in a reassuring tone.

[1387] This system enables appropriate responses tailored to the user's emotions, thereby improving response quality and user satisfaction.

[1388] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1389] Program processing flow

[1390] Step 1:

[1391] The user enters the inquiry details using the terminal's interface.

[1392] Input: Text entered by the user (e.g., "Do you have product A in stock?").

[1393] Output: The entered text is retrieved as terminal data.

[1394] Operation: The user enters the inquiry via a web browser or a dedicated application.

[1395] Step 2:

[1396] The terminal converts the user's input into JSON format and sends it to the server.

[1397] Input: Text entered by the user.

[1398] Output: Data in JSON format (Example: {"userId": "12345", "timestamp": "2023-10-05T10:00:00Z", "query": "Do you have product A in stock?"}).

[1399] Operation: The terminal converts the input content into JSON format and sends it to the server as an HTTP POST request.

[1400] Step 3:

[1401] The server parses the received JSON data and extracts the query details.

[1402] Input: Data in JSON format.

[1403] Output: Inquiry details and user information (e.g., {'query': 'Do you have product A in stock?', 'userId': '12345'}).

[1404] Operation: The server receives an HTTP request and extracts the query content by parsing the JSON data.

[1405] Step 4:

[1406] The server uses a natural language processing engine to tokenize the query content, identifying its intent and target.

[1407] Input: Extracted query content.

[1408] Output: Tokenized data (e.g., {'intent': 'Check Inventory', 'target': 'Product A'}).

[1409] Operation: The server uses a natural language processing engine to break down the query into words and phrases, identifying the intent and target.

[1410] Step 5:

[1411] The server uses an emotion recognition engine to recognize the user's emotions.

[1412] Input: Extracted query content.

[1413] Output: Emotion classification data (e.g., {'emotion': 'anxiety'}).

[1414] Operation: The server uses an emotion recognition engine to recognize the user's emotions from the content of the inquiry.

[1415] Step 6:

[1416] The server queries the database based on intent, target, and sentiment data to retrieve the necessary information.

[1417] Input: Tokenized data and sentiment classification data.

[1418] Output: Information retrieved from the database (e.g., {'stock': 'available'}).

[1419] Operation: The server generates and executes SQL queries against the database to retrieve necessary information such as inventory data.

[1420] Step 7:

[1421] The server uses a generative AI model to generate the optimal response.

[1422] Input: Information and sentiment classification data obtained from a database.

[1423] Output: Generated response (e.g., "Rest assured, product A is currently in stock").

[1424] Operation: The server uses a generative AI model to generate appropriate responses based on emotions, using input data.

[1425] Step 8:

[1426] The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[1427] Input: The generated response.

[1428] Output: Response data in JSON format.

[1429] Operation: The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[1430] Step 9:

[1431] The terminal parses the response data in JSON format and converts it into a format that can be displayed in the user interface.

[1432] Input: Response data in JSON format.

[1433] Output: Displayable text (e.g., "Rest assured, product A is currently in stock").

[1434] Operation: The device parses the JSON data it receives and converts it into a human-readable text format.

[1435] Step 10:

[1436] The terminal displays the formatted response in the user interface.

[1437] Input: Displayable text.

[1438] Output: The response displayed to the user.

[1439] Operation: The terminal displays a response in the user interface, and the user confirms it.

[1440] These steps allow the system to recognize user emotions and provide appropriate responses, thereby improving the user experience.

[1441] (Application Example 2)

[1442] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1443] Conventional automated response systems can generate answers to inquiries, but they cannot recognize and adapt responses to user emotions. As a result, they fail to alleviate user anxiety and confusion, leading to decreased user satisfaction. This is especially true in virtual stores, where users are required to ask questions in real time and receive appropriate support on the spot, making emotionally sensitive responses crucial. To address this challenge, there is a need to provide a system that recognizes user emotions and adjusts the tone and content of responses accordingly.

[1444] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1445] In this invention, the server includes means for analyzing the query content with a natural language processing engine to extract the intent and target, means for recognizing the user's emotions, and means for adjusting the tone and content of the response based on the emotion recognition. This makes it possible to generate an appropriate response that takes the user's emotions into consideration.

[1446] An "inquiry" refers to a question or request that a user enters when seeking information or support regarding a product or service.

[1447] A "natural language processing engine" refers to a set of algorithms and technologies used to analyze text entered by a user and identify its intent and target.

[1448] "Emotion recognition" is a technology that determines emotions from user input and identifies feelings such as joy, anger, anxiety, and confusion.

[1449] "Tokenization" is the process of dividing input text into a series of words or phrases.

[1450] A "database" is a system that systematically stores information related to products and services, making it searchable and retrievalable as needed.

[1451] "Generative AI" refers to artificial intelligence technology that generates the optimal response to user inquiries.

[1452] A "device" refers to a device operated by a user (e.g., a smartphone, smart glasses, etc.).

[1453] A "response" refers to the information or answer provided by a server to a user.

[1454] A "virtual store" is a purchasing environment built in digital space, rather than a physical store.

[1455] The system of this invention is designed to enhance the user's shopping experience in virtual stores. Using smart glasses, it can process user inquiries in real time and provide responses based on emotion recognition. The specific operation of the system is described below.

[1456] 1. System Configuration

[1457] This system uses the following hardware and software components.

[1458] Hardware:

[1459] Smart glasses: Devices that capture the user's gaze and voice input. They have a built-in AR display and voice input microphone.

[1460] software:

[1461] A natural language processing engine (NLTK) is an engine that analyzes text entered by a user to identify its intent and target. Specific examples include NLTK and spaCy.

[1462] Emotion Recognition Engine: An engine that analyzes user input to identify emotions such as joy, anger, anxiety, and confusion. A concrete example is the IBM Watson Emotion Analysis API.

[1463] A database (Product Database) is a system that stores information related to products and services, and allows users to search and retrieve it as needed. A specific example is MySQL.

[1464] Generative AI (Response Generator): Artificial intelligence technology used to generate optimal responses to user inquiries. Specific examples include OpenAI GPT-3 or 4.

[1465] 2. Flow of Operation

[1466] 1. Inquiry Reception

[1467] Users ask questions about products and services through smart glasses. The voice-input inquiries are converted into text format and sent to the server.

[1468] 2. Inquiry Analysis

[1469] The server analyzes the submitted query using a natural language processing engine. It tokenizes the query and extracts the intent (e.g., inventory check) and the target (e.g., product A).

[1470] 3. Recognition of emotions

[1471] The server uses an emotion recognition engine to identify the user's emotions from the query. For example, it can detect if the user is anxious or confused.

[1472] 4. Database Query

[1473] Based on the extracted information, the server retrieves relevant company-specific information (e.g., inventory status) from the database.

[1474] 5. Response generation

[1475] The server uses generative AI to generate the optimal response based on the information it receives and the user's emotions. The response is adjusted in tone and content to be sensitive to the user's feelings.

[1476] 6. Sending and displaying responses

[1477] The generated response is converted to JSON format and sent to the smart glasses. The smart glasses then display the received response in the user interface.

[1478] 3. Specific examples

[1479] Example 1: Checking product inventory

[1480] User: "Do you have product A in stock?"

[1481] Server: The natural language processing engine identifies the intent of "check inventory" and the target of "product A". The emotion recognition engine identifies "anxiety". Inventory information is retrieved from the database, and the response generation engine generates "Don't worry, product A is currently in stock". This message is sent to the smart glasses and displayed.

[1482] Example of a prompt

[1483] User: "Do you have product A in stock?"

[1484] NLP Engine: Intention = "Check Inventory", Target = "Product A"

[1485] Emotional engine: Emotion = "Anxiety"

[1486] Database: SQL query = "SELECT stock FROM products WHERE name = 'Product A'"

[1487] AI generation: "Don't worry, product A is currently in stock."

[1488] In this way, this system allows users to receive emotionally sensitive, real-time responses, improving their shopping experience in virtual stores.

[1489] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1490] Step 1:

[1491] Inquiry reception

[1492] Users input questions about products and services via voice through smart glasses. The smart glasses convert this voice input into text format and send it to the server as JSON data. Here, the input is voice, and the output is text data. This makes it possible to treat the inquiry content as data.

[1493] Step 2:

[1494] Inquiry analysis

[1495] The server receives the transmitted JSON text data and parses it using a natural language processing engine (e.g., NLTK, spaCy). Specifically, it tokenizes the text and extracts the intent of the inquiry (e.g., inventory check) and the target (e.g., product A). The input is JSON data, and the output is tokenized words and intent / target. Through this analysis, the server can obtain the basic data necessary to take appropriate action.

[1496] Step 3:

[1497] Recognition of emotions

[1498] The server inputs the analyzed text data into an emotion recognition engine (e.g., IBM Watson Emotion Analysis API) to identify the user's emotions (e.g., anxiety, joy). The input is the analyzed text data, and the output is the emotion category. Emotion recognition provides the data needed to generate responses that correspond to the user's emotions.

[1499] Step 4:

[1500] Database query

[1501] The server sends queries to a database (e.g., MySQL) based on the user's intent and target to retrieve information about products and services. For example, if the target is "Product A" and the intent is "check inventory," the server will retrieve inventory information for Product A using an SQL query. The input is the query, and the output is the information retrieved from the database. This step provides specific information in response to the user's inquiry.

[1502] Step 5:

[1503] Response generation

[1504] The server uses generative AI (e.g., OpenAI GPT-3, GPT-4) to generate the optimal response based on acquired information and the user's emotions. For example, if inventory information is available, it will generate a response in a tone that considers the user's emotions, such as, "Don't worry, product A is currently in stock." The input is the acquired data and emotional information, and the output is the generated response text. This response generation enhances user satisfaction.

[1505] Step 6:

[1506] Sending and displaying responses

[1507] The server converts the generated response into JSON format and sends it to the smart glasses. The smart glasses parse the received JSON data and display it on the user interface. The input is the generated response text, and the output is the response message displayed on the smart glasses' screen. This step allows the user to receive answers in real time.

[1508] In this way, a system is realized that automatically generates and provides appropriate responses to user inquiries through each step.

[1509] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1510] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1511] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1512] [Fourth Embodiment]

[1513] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1514] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1515] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1516] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1517] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1518] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1519] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1520] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1521] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1522] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1523] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1524] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1525] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1526] In this embodiment of the invention, a system is provided that enables efficient customer service through the coordinated operation of the user, terminal, and server components. The specific operation of the system is described below.

[1527] 1. Inquiry Reception

[1528] User: The user enters their inquiry using the device's interface (e.g., a web browser or dedicated application). The inquiry may include questions about specific product names or services.

[1529] Terminal: Converts the user's input into JSON format and sends it to the server as an HTTP POST request. This request includes the user's identification information and the inquiry's timestamp.

[1530] 2. Inquiry Analysis

[1531] Server: The server uses a natural language processing engine to parse incoming HTTP POST requests. This engine tokenizes the query content, extracts specific keywords, and determines the intent of the query. It also identifies the product or service being inquired about.

[1532] 3. Database Query

[1533] Server: Executes queries against the database based on the intent and target identified by natural language processing. For example, if the query is for checking inventory, it generates an SQL query to retrieve inventory information about the product and retrieves the necessary information from the database.

[1534] 4. Generating a response

[1535] Server: Based on information retrieved from the database, a generative AI generates an appropriate response. This AI also refers to past response history to generate more appropriate and user-friendly responses. For example, it might generate a specific response such as, "Product A is currently in stock."

[1536] 5. Sending a response

[1537] Server: Packages the generated response into JSON format and sends it to the terminal as an HTTP response.

[1538] Terminal: The terminal analyzes the received response and displays it appropriately on the user interface. Through this display, the user can see the answer to their inquiry in real time.

[1539] Specific example

[1540] Example 1: Checking product inventory

[1541] User: Enters "Do you have product A in stock?"

[1542] Terminal: Convert this query into JSON format and send it to the server.

[1543] Server: The server analyzes the content of the inquiry using a natural language processing engine to extract the intent of "inventory check" and the target of "product A".

[1544] Server: Retrieves inventory information for "Product A" from the database and generates a response stating "Product A is currently in stock."

[1545] Server: Sends the generated response to the terminal.

[1546] Terminal: Displays "Product A is currently in stock" to the user.

[1547] Example 2: Support inquiry (technical question)

[1548] User: Enters "Please tell me how to configure device B."

[1549] Terminal: Convert this query into JSON format and send it to the server.

[1550] Server: Analyzes the query content using a natural language processing engine and extracts the intent of "configuration method" and the target of "device B".

[1551] Server: Retrieves information about "How to configure device B" from the database and generates a response that says, "The following are the steps to configure device B..."

[1552] Server: Sends the generated response to the terminal.

[1553] Terminal: Displays the user with the message, "The following steps will guide you through configuring device B..."

[1554] As described above, this system can significantly improve the efficiency of a company's customer service by automatically and quickly processing user inquiries.

[1555] The following describes the processing flow.

[1556] Step 1:

[1557] User: Enter your inquiry using the terminal's user interface. For example, enter "Do you have product A in stock?".

[1558] Step 2:

[1559] Terminal: Converts the entered query content into JSON format. This includes user identification information and the query's timestamp.

[1560] Step 3:

[1561] Terminal: Sends the converted JSON data to the server as an HTTP POST request.

[1562] Step 4:

[1563] Server: Parses the received HTTP POST request and extracts the query content. Processing begins based on this content.

[1564] Step 5:

[1565] Server: Uses a natural language processing engine to tokenize the extracted query content. For example, it separates it into words such as "product A," "in stock," and "available."

[1566] Step 6:

[1567] Server: Based on tokenized words, it identifies the intent of the inquiry (in this case, inventory check) and the target (in this case, product A).

[1568] Step 7:

[1569] Server: Generates queries against the database based on intent and target. For example, it generates an SQL query to "retrieve inventory information for product A".

[1570] Step 8:

[1571] Server: Executes the generated query against the database to retrieve relevant inventory information. For example, it retrieves information such as "Product A is currently in stock."

[1572] Step 9:

[1573] Server: Uses generative AI to generate the optimal response based on the acquired inventory information. For example, it generates the sentence, "Product A is currently in stock."

[1574] Step 10:

[1575] Server: Converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[1576] Step 11:

[1577] Terminal: Analyzes the received HTTP response and converts it into a format that can be displayed in the user interface.

[1578] Step 12:

[1579] Terminal: Displays a formatted response in the user interface. For example, it displays the message "Product A is currently in stock."

[1580] Step 13:

[1581] User: Review the displayed response and take the next action as needed. For example, confirm that the item is in stock and proceed with the purchase.

[1582] The above outlines the specific processing steps of this system. Each step works in conjunction to enable efficient and accurate responses to user inquiries.

[1583] (Example 1)

[1584] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1585] Traditional customer service systems often have slow response times to user inquiries and may fail to provide appropriate answers. This leads to decreased efficiency in corporate customer service and reduced customer satisfaction. Furthermore, the lack of features to effectively utilize past interaction history makes it difficult to provide consistent answers to similar inquiries.

[1586] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1587] In this invention, the server includes means for analyzing with a language processing engine to extract intent and target,

[1588] A means of obtaining company-specific information from a data set based on extracted information,

[1589] A means by which a response generation engine generates an optimal response,

[1590] This includes enabling rapid and accurate analysis of inquiries and providing consistent and appropriate responses by leveraging past interaction history.

[1591] An "inquiry" refers to a request for information about a specific product or service that a user enters into an information processing device.

[1592] An "information processing device" is a terminal device used by users to input inquiries, such as a web browser or a device that utilizes a dedicated application.

[1593] A "host computer" refers to a central system that receives inquiries transmitted from information processing devices, analyzes them, and generates responses.

[1594] A "language processing engine" refers to the natural language processing technology used by a host computer to analyze query content, specifically an engine that performs tasks such as tokenization and intent extraction.

[1595] "Intention" refers to the purpose or objective that can be inferred from the user's inquiry.

[1596] "Target" refers to the product or service identified by the user's inquiry.

[1597] A "data collection" refers to an information source, such as a database, where multiple pieces of information are aggregated and stored.

[1598] A "response generation engine" refers to artificial intelligence technology that creates the optimal answer based on data acquired by a host computer.

[1599] "Optimal response" refers to information generated to answer a user's inquiry quickly and accurately.

[1600] "Presentation" refers to displaying the generated response to the user using an information processing device.

[1601] This system enables efficient customer service through the coordinated operation of users, terminals, and servers. The embodiments of this invention will be described in detail below.

[1602] Inquiry reception

[1603] User: Users enter their inquiries using the device's interface (e.g., a web browser or dedicated application). These inquiries may include questions about specific product names or services.

[1604] Terminal: The terminal converts the user's input into JSON format. This conversion includes the query content, user identification information, and query timestamp. This JSON data is sent from the terminal to the server as an HTTP POST request.

[1605] Inquiry analysis

[1606] Server: The server uses a natural language processing engine (e.g., Hugging Face's Transformers model) to parse incoming HTTP POST requests. This engine tokenizes the query content, extracts specific keywords, and determines the intent of the query. It also identifies the product or service being inquired about.

[1607] Database query

[1608] Server: Executes queries against the database based on the intent and target identified by natural language processing. For example, if the query is for checking inventory, it generates an SQL query to retrieve inventory information about the product and retrieves the necessary information from the database.

[1609] Response generation

[1610] Server: The server uses a generative AI model (e.g., OpenAI's GPT-3) to generate a response based on information retrieved from the database. This response is provided in a format that is easy for the user to understand. For example, a specific response such as "Product A is currently in stock" is generated.

[1611] Sending a response

[1612] Server: The server packages the generated response into JSON format and sends it to the terminal as an HTTP response.

[1613] Terminal: The terminal analyzes the received response and displays it appropriately on the user interface. The user can then see the answer to their inquiry in real time from this display.

[1614] Specific example

[1615] Example 1: Checking product inventory

[1616] User: Enters "Do you have product A in stock?" into the terminal.

[1617] Terminal: Convert this query into JSON format and send it to the server.

[1618] Server: The server uses a natural language processing engine to analyze the query content and extract the intent of "stock confirmation" and the target of "product A".

[1619] Server: Retrieves inventory information for "Product A" from the database and generates a response stating "Product A is currently in stock."

[1620] Server: Sends the generated response to the terminal.

[1621] Terminal: Displays "Product A is currently in stock" to the user.

[1622] Example 2: Support inquiry (technical question)

[1623] User: Enters "Please tell me how to configure device B" into the terminal.

[1624] Terminal: Convert this query into JSON format and send it to the server.

[1625] Server: The server analyzes the query content using a natural language processing engine and extracts the intent of "configuration method" and the target of "device B".

[1626] Server: Retrieves information about "How to configure device B" from the database and generates a response that says, "The following are the steps to configure device B..."

[1627] Server: Sends the generated response to the terminal.

[1628] Terminal: Displays the user with the message, "The following steps will guide you through configuring device B..."

[1629] As described above, this system can significantly improve the efficiency of a company's customer service by automatically and quickly processing user inquiries.

[1630] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1631] Step 1:

[1632] The user enters the inquiry details into the information processing device.

[1633] Input: A specific question about a product or service (e.g., "Do you have product A in stock?")

[1634] Output: The user's inquiry is input into the information processing device.

[1635] Step 2:

[1636] The terminal converts the user's inquiry into JSON format.

[1637] Input: Inquiry content entered by the user

[1638] Output: JSON data containing query details, user identification information, and query timestamp.

[1639] Specific operation: The terminal software analyzes the user's input, extracts the necessary information, and generates data in JSON format.

[1640] Step 3:

[1641] The device sends JSON data to the server as an HTTP POST request.

[1642] Input: User inquiry content converted to JSON format

[1643] Output: HTTP POST request received by the server

[1644] Specific action: The device sends an HTTP request to the server over the network.

[1645] Step 4:

[1646] The server receives an HTTP POST request and parses the query content.

[1647] Input: HTTP POST request sent from the terminal

[1648] Output: Tokenization of query content and extraction of intent and target.

[1649] Specific operation: The natural language processing engine on the server tokenizes the query content, extracts specific keywords, and determines the intent and target.

[1650] Step 5:

[1651] The server executes SQL queries against the database based on its intent and target.

[1652] Input: Tokenized inquiry content with identified intent and target.

[1653] Output: Relevant information retrieved from the database (e.g., inventory information for product A)

[1654] Specific operation: The server dynamically generates SQL queries, executes them against the database, and retrieves the necessary information.

[1655] Step 6:

[1656] Based on the information acquired by the server, the response generation engine generates an appropriate response.

[1657] Input: Related information retrieved from the database

[1658] Output: Generated response (Example: "Product A is currently in stock")

[1659] Specific operation: The server uses a generation AI model based on specific data to generate a response in a format that is easy for the user to understand.

[1660] Step 7:

[1661] The server packages the generated response into JSON format and sends it to the terminal as an HTTP response.

[1662] Input: Generated response

[1663] Output: HTTP response received by the terminal

[1664] Specific operation: The response generated by the server is converted to JSON format, packaged as an HTTP response, and sent back to the terminal.

[1665] Step 8:

[1666] The system analyzes the response received by the terminal and displays it on the user interface.

[1667] Input: HTTP response received from the server

[1668] Output: The response displayed to the user (e.g., "Product A is currently in stock")

[1669] Specific operation: The terminal's software analyzes the response and displays it appropriately on the user interface, allowing the user to see the answer in real time.

[1670] (Application Example 1)

[1671] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1672] This invention relates to a system for responding quickly and efficiently to customer inquiries. In particular, it aims to provide a means for enabling staff in physical stores to appropriately answer customer questions and requests on the spot. In current systems, inquiries are often handled manually, which is time-consuming and may lead to a decrease in customer satisfaction. Therefore, this invention aims to provide a system that reduces the burden on staff and improves the speed and accuracy of customer service.

[1673] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1674] In this invention, the server includes means for the user to input an inquiry, means for sending the content of the inquiry to the server, means for the server to analyze the content of the inquiry using a natural language processing engine and extract the intent and target, means for obtaining information from a database based on the extracted information, means for a generative AI to generate an optimal response, means for sending the generated response to a terminal, means for the terminal to display the response to the user, and means for enabling staff to respond immediately to customer inquiries using a smart device. This makes it possible to respond quickly and accurately to customer inquiries.

[1675] An "inquiry" is the act of a customer expressing questions or requests regarding a product or service.

[1676] "Users" refer to store staff and customers who use the system to enter inquiries.

[1677] A "terminal" is a device used by a user to input and submit inquiries, and typically includes smartphones and computers.

[1678] A "server" is a central processing unit that analyzes query content and generates and sends appropriate responses.

[1679] A "natural language processing engine" is a software component that tokenizes user-inputted inquiries and extracts their intent and target.

[1680] "Intention" refers to the purpose or content of the question that the user is trying to achieve through their inquiry.

[1681] "Target" refers to the specific items of products or services related to the inquiry.

[1682] A "database" is an information management system that stores the information necessary to generate responses to queries.

[1683] "Generative AI" refers to artificial intelligence models that generate appropriate responses based on information obtained from a database.

[1684] A "smart device" is an interactive device used by staff in a physical store, and typically includes smartphones and tablet devices.

[1685] A "response" is a message generated by the server in response to a user's inquiry.

[1686] "Display" refers to the act of providing the user with a visual representation of the response generated on the device.

[1687] In this embodiment of the invention, a system is provided that enables efficient customer service in physical stores through the coordinated operation of a server, terminal, and user. This system automates a series of processes from receiving inquiries to generating and displaying responses.

[1688] 1. Program Overview

[1689] The program operates in the following sequence:

[1690] 1. Inquiry reception:

[1691] Users enter their inquiries into smart devices (such as smartphones or tablets) used within the physical store. For example, they might ask, "Do you have this product in stock?"

[1692] 2. Data transmission:

[1693] The terminal converts the entered query content into JSON format and sends it to the server. This includes the query's timestamp and user identification information.

[1694] 3. Analysis:

[1695] The server parses the received JSON data and uses a natural language processing engine (e.g., SpaCy, NLTK) to extract the intent and target of the query. For example, it might extract "stock check" and "product A".

[1696] 4. Database query:

[1697] Based on the extracted information, retrieve the relevant information from the database (e.g., MySQL, PostgreSQL). For example, retrieve inventory information for "Product A" using an SQL query.

[1698] 5. Response generation:

[1699] Generative AI (e.g., OpenAI GPT-3) generates appropriate responses based on information obtained from a database. This AI also refers to past response history to provide specific and user-friendly responses. For example, it might generate a response such as, "Product A is currently in stock."

[1700] 6. Send a reply:

[1701] The generated response is converted to JSON format and sent from the server to the terminal as an HTTP response.

[1702] 7. Response display:

[1703] The terminal analyzes the received response and displays it on the smart device screen. Through this display, the user provides the customer with the appropriate answer. For example, a staff member might look at their smartphone and tell the customer, "Product A is currently in stock."

[1704] Hardware and software to be used

[1705] Smart devices: Smartphones, tablets

[1706] Server: Cloud service (e.g., AWS, GCP)

[1707] Natural language processing engines: SpaCy, NLTK

[1708] Generative AI: OpenAI GPT-3

[1709] Database: MySQL, PostgreSQL

[1710] Specific example

[1711] Consider a scenario where a customer asks "Do you have this product in stock?" in a physical store. This question is sent from a smart device to a server, where a natural language processing engine extracts "stock check" and "product A." The inventory information for product A is retrieved from the database, and a generative AI generates the response "Product A is currently in stock." This response is sent to the terminal, which displays it on the screen, allowing staff to answer the customer based on that information.

[1712] Example of a prompt

[1713] Please generate a response based on the user's inquiry. The inquiry is "Do you have product A in stock?" Please refer to past response history and provide a specific and clear response.

[1714] Thus, the present invention contributes to improving customer satisfaction by streamlining customer service in physical stores through the coordinated operation of the user, terminal, and server.

[1715] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1716] Step 1:

[1717] Inquiry Reception: Users enter their inquiry details using a smart device within the physical store. This input includes questions and requests, which are converted to JSON format by the device.

[1718] Input: User's inquiry

[1719] Data processing: Convert the entered inquiry content into JSON format.

[1720] Output: Query data in JSON format

[1721] Specific action: The user enters "Is this product in stock?" into their smartphone.

[1722] Step 2:

[1723] Data transmission: The terminal sends the converted JSON-formatted query data to the server. This transmission is performed using an HTTP POST request.

[1724] Input: Query data in JSON format

[1725] Data processing: Sent to the server as an HTTP POST request.

[1726] Output: JSON data received by the server

[1727] Specific operation: A JSON query asking "Is this product in stock?" is sent from the terminal to the server.

[1728] Step 3:

[1729] Analysis: The server analyzes the received JSON data and uses a natural language processing engine to extract the intent and target of the query. During this process, the input text is tokenized and keywords are extracted.

[1730] Input: JSON data received by the server

[1731] Data processing: Tokenize the data using a natural language processing engine and extract the intent and target.

[1732] Output: Extracted intent and target

[1733] Specific operation: The server extracts "Inventory Check" and "Product A".

[1734] Step 4:

[1735] Database query: The server executes a query against the database based on the extracted intent and target. It retrieves information from the database that corresponds to the query.

[1736] Input: Extracted intent and target

[1737] Data operations: Executing SQL queries against a database

[1738] Output: Information retrieved from the database

[1739] Specific operation: The server retrieves inventory information for "Product A" from the database.

[1740] Step 5:

[1741] Response Generation: The server uses a generative AI to generate the optimal response based on the acquired information. This AI also refers to past response history to provide specific and user-friendly responses.

[1742] Input: Information obtained from the database

[1743] Data processing: Generative AI is used to generate the optimal response.

[1744] Output: Generated response

[1745] Specific operation: The generative AI generates the response "Product A is currently in stock."

[1746] Step 6:

[1747] Response transmission: The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[1748] Input: Generated response

[1749] Data processing: Convert the response to JSON format and send it to the terminal.

[1750] Output: Response received by the terminal

[1751] Specific action: The generated response "Product A is currently in stock" is sent from the server to the terminal.

[1752] Step 7:

[1753] Response Display: The terminal analyzes the received response and displays it on the smart device screen. The user provides the customer with an appropriate answer through this display.

[1754] Input: Response received by the terminal

[1755] Data processing: Displaying received responses appropriately in the user interface.

[1756] Output: Displayed response

[1757] Specific action: The staff member informs the customer of the response displayed on their smartphone screen, "Product A is currently in stock."

[1758] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1759] This invention provides a more appropriate response to user inquiries by recognizing the user's emotions and adjusting the response accordingly in a system that automates responses to user inquiries. The specific operation of the system is described below.

[1760] 1. Inquiry Reception

[1761] User: The user enters their inquiry using the device's interface (e.g., a web browser or dedicated application). The inquiry may include questions about specific products or services.

[1762] Terminal: Converts the user's input into JSON format and sends it to the server as an HTTP POST request. This request includes the user's identification information and the inquiry's timestamp.

[1763] 2. Inquiry Analysis

[1764] Server: Parses the received HTTP POST request and extracts the query content. Processing begins based on this content.

[1765] Server: Uses a natural language processing engine to tokenize the extracted query content. For example, it separates it into words such as "product A," "in stock," and "available."

[1766] Server: Based on tokenized words, it identifies the intent of the inquiry (in this case, inventory check) and the target (in this case, product A).

[1767] 3. Recognition of emotions

[1768] Server: Uses an emotion engine to recognize the user's emotions from the query content. These emotions are categorized into multiple categories, such as joy, anger, sadness, and surprise.

[1769] 4. Database Query

[1770] Server: Generates queries against the database based on intentions and targets identified through natural language processing and sentiment recognition. For example, if the query is for checking inventory, it generates an SQL query to retrieve inventory information about the product and retrieves the necessary information from the database.

[1771] 5. Response generation

[1772] Server: Based on acquired inventory information and recognized sentiment information, a generative AI is used to generate the optimal response. This AI also refers to past interaction history and generates responses with a tone and content appropriate to the sentiment. For example, it adjusts the sentence "Product A is currently in stock" to "Rest assured, Product A is currently in stock."

[1773] 6. Sending a response

[1774] Server: Converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[1775] Terminal: Analyzes the received response and converts it into a format that can be displayed in the user interface.

[1776] 7. Display of response

[1777] Terminal: Displays a formatted response in the user interface. For example, it might display the message, "Don't worry, product A is currently in stock."

[1778] User: Review the displayed response and take the next action as needed. For example, confirm that the item is in stock and proceed with the purchase.

[1779] Specific example

[1780] Example 1: Checking product inventory

[1781] User: Enters "Do you have product A in stock?"

[1782] Terminal: Convert this query into JSON format and send it to the server.

[1783] Server: The server analyzes the content of the inquiry using a natural language processing engine to extract the intent of "inventory check" and the target of "product A".

[1784] Server: The emotion engine recognizes the user's emotion as "anxiety" based on the query.

[1785] Server: Retrieves inventory information for "Product A" from the database and generates a response saying, "Don't worry, Product A is currently in stock."

[1786] Server: Sends the generated response to the terminal.

[1787] Terminal: Displays the user with the message, "Don't worry, product A is currently in stock."

[1788] Example 2: Support inquiry (technical question)

[1789] User: Enters "Please tell me how to configure device B."

[1790] Terminal: Convert this query into JSON format and send it to the server.

[1791] Server: Analyzes the query content using a natural language processing engine and extracts the intent of "configuration method" and the target of "device B".

[1792] Server: The emotion engine recognizes the user's emotion as "confused" based on the query.

[1793] Server: Retrieves information about "How to configure device B" from the database and generates a response saying, "Don't worry, here's how to configure device B..."

[1794] Server: Sends the generated response to the terminal.

[1795] Terminal: Displays the user with the message, "Don't worry, here's how to set up device B..."

[1796] In this way, by recognizing user emotions and adjusting responses accordingly, a more user-friendly approach becomes possible. This system can improve the quality of customer service at corporate customer centers and contribute to increased customer satisfaction.

[1797] The following describes the processing flow.

[1798] Step 1:

[1799] User: Enter your inquiry using the terminal's user interface. For example, enter "Do you have product A in stock?".

[1800] Step 2:

[1801] Terminal: Converts the entered query content into JSON format. This includes user identification information and the query's timestamp.

[1802] Step 3:

[1803] Terminal: Sends the converted JSON data to the server as an HTTP POST request.

[1804] Step 4:

[1805] Server: Parses the received HTTP POST request and extracts the query content. Processing begins based on this content.

[1806] Step 5:

[1807] Server: Uses a natural language processing engine to tokenize the extracted query content. For example, it separates it into words such as "product A," "in stock," and "available."

[1808] Step 6:

[1809] Server: Based on tokenized words, it identifies the intent of the inquiry (in this case, inventory check) and the target (in this case, product A).

[1810] Step 7:

[1811] Server: Uses an emotion engine to recognize the user's emotions from the query content. For example, it classifies emotions such as "anxiety," "confusion," and "anger."

[1812] Step 8:

[1813] Server: Analyzes the emotional information recognized by the emotion engine and uses it to adjust the tone and content of the response. This information is referenced in a later response generation step.

[1814] Step 9:

[1815] Server: Generates queries against the database based on intent, target, and sentiment information. For example, it generates an SQL query to "retrieve inventory information for product A" and retrieves the necessary information from the database.

[1816] Step 10:

[1817] Server: Based on inventory information retrieved from the database, a generative AI is used to generate the optimal response. This AI refers to past interaction history and sentiment information to generate a response with a tone and content that matches the user's emotions. For example, it adjusts "Product A is currently in stock" to "Don't worry, Product A is currently in stock."

[1818] Step 11:

[1819] Server: Converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[1820] Step 12:

[1821] Terminal: Analyzes the received HTTP response and converts it into a format that can be displayed in the user interface.

[1822] Step 13:

[1823] Terminal: Displays a formatted response in the user interface. For example, it might display the message, "Don't worry, product A is currently in stock."

[1824] Step 14:

[1825] User: Review the displayed response and take the next action as needed. For example, confirm that the item is in stock and proceed with the purchase.

[1826] The above outlines the specific processing steps of this system. Each step works in conjunction with the others, and combined with emotion recognition, this enables efficient and user-friendly customer service.

[1827] (Example 2)

[1828] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1829] Modern inquiry systems often generate uniform responses without considering user emotions, which can lead to decreased user satisfaction. Furthermore, the process of accurately interpreting inquiries and automatically generating appropriate responses often lacks precision and flexibility. This invention aims to provide more appropriate and user-friendly responses by utilizing user emotion recognition and natural language processing.

[1830] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1831] In this invention, the server includes means for analyzing the query content with a natural language processing engine and extracting the intent and target; means for recognizing the user's emotions based on the extracted information; and means for obtaining necessary information from a database based on the emotion recognition. This makes it possible to generate appropriate responses that correspond to the user's emotions, thereby improving the quality of responses and user satisfaction.

[1832] "User" refers to an end user who uses the system to make inquiries.

[1833] "Device" refers to the hardware or software that a user uses to enter information. Examples include smartphones, tablets, and personal computers.

[1834] A "server" refers to a computer system that receives data sent from a terminal, analyzes the content of the inquiry, accesses a database, and generates a response.

[1835] "Inquiry content" refers to the text data of questions and requests that users enter through the system.

[1836] A "natural language processing engine" refers to a computer program that analyzes text data to understand its grammar and meaning.

[1837] "Intent" refers to the action or request indicated by the user's inquiry. Examples include "checking inventory" or "inquiring about setup methods."

[1838] The "subject" refers to the specific item or service that the user's inquiry is referring to. For example, this could include "Product A" or "Device B."

[1839] An "emotion recognition engine" refers to a computer program that identifies a user's emotions from text data.

[1840] A "database" refers to a structured data set used to store information related to a query.

[1841] "Generative AI" refers to artificial intelligence that automatically generates responses and text based on input data.

[1842] "Response" refers to the content of the reply that the system generates in response to a user's inquiry.

[1843] This invention relates to a system that automates responses to user inquiries and adjusts those responses based on the user's emotions. The specific operation of the system is described below.

[1844] Hardware and software to be used

[1845] This system uses the following hardware and software:

[1846] Terminal: A device that displays user input and responses. Specifically, smartphones, tablets, and personal computers are used.

[1847] Server: A computer system that receives, analyzes, and generates responses from data. Software that runs on a server includes the following:

[1848] Natural language processing engines (e.g., Stanford NLP)

[1849] Emotion recognition engines (e.g., IBM Watson, Google Cloud Natural Language API)

[1850] Database management systems (e.g., MySQL, PostgreSQL)

[1851] Generative AI models (e.g., OpenAI GPT-4)

[1852] System Operation Description

[1853] 1. Means by which users can enter inquiries:

[1854] Users use their device's interface (e.g., a web browser or a dedicated application) to enter inquiries about specific products or services. For example, they might enter a specific question such as, "Do you have product A in stock?"

[1855] 2. Means by which the terminal sends the inquiry content to the server:

[1856] The terminal receives user input, converts it to JSON format, and sends it to the server as an HTTP POST request. This includes user identification information and a timestamp of the query.

[1857] 3. A means by which the server analyzes the query content using a natural language processing engine and extracts the intent and target:

[1858] The server parses the received HTTP POST request, uses a natural language processing engine to tokenize the query content, and extracts the intent (e.g., inventory check) and the target (e.g., product A).

[1859] 4. Means by which the server recognizes the user's emotions based on the extracted information:

[1860] The server uses an emotion recognition engine to identify the user's emotions from the content of their inquiry. For example, emotions such as "anxiety" or "confusion" may be identified.

[1861] 5. Means by which the server retrieves necessary information from the database based on emotion recognition:

[1862] Based on the results of natural language processing and sentiment recognition, SQL queries are generated against the database to retrieve the necessary information (e.g., inventory information and configuration instructions).

[1863] 6. Means for generating the optimal response based on information and emotional information acquired by a generative AI:

[1864] The server uses a generative AI model to generate responses based on acquired data and sentiment information. For example, it might adjust the response to say, "Don't worry, product A is currently in stock."

[1865] 7. Means for the server to send the generated response to the terminal:

[1866] The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[1867] 8. Means by which the terminal displays a response to the user:

[1868] The terminal analyzes the received response, converts it into a viewable format, and displays it to the user. The user then takes the next action (e.g., proceed with a purchase) based on this information.

[1869] Examples of specific cases and prompt statements

[1870] Example 1: Checking product inventory

[1871] If a user asks, "Do you have product A in stock?":

[1872] 1. The device sends the inquiry details to the server.

[1873] 2. The server analyzes the query and extracts the intent (inventory check) and the target (product A).

[1874] 3. The emotion recognition engine identifies the user's emotion as "anxiety."

[1875] 4. Retrieve the necessary inventory information from the database.

[1876] 5. The generative AI generates the response, "Don't worry, product A is currently in stock."

[1877] 6. Send the generated response to the terminal.

[1878] 7. The device displays to the user, "Don't worry, product A is currently in stock."

[1879] Example of a prompt

[1880] If a user asks, "Do you have product A in stock?":

[1881] 1. Please check the stock availability of product A.

[1882] 2. Since the user is feeling anxious, please generate a response in a reassuring tone.

[1883] This system enables appropriate responses tailored to the user's emotions, thereby improving response quality and user satisfaction.

[1884] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1885] Program processing flow

[1886] Step 1:

[1887] The user enters the inquiry details using the terminal's interface.

[1888] Input: Text entered by the user (e.g., "Do you have product A in stock?").

[1889] Output: The entered text is retrieved as terminal data.

[1890] Operation: The user enters the inquiry via a web browser or a dedicated application.

[1891] Step 2:

[1892] The terminal converts the user's input into JSON format and sends it to the server.

[1893] Input: Text entered by the user.

[1894] Output: Data in JSON format (Example: {"userId": "12345", "timestamp": "2023-10-05T10:00:00Z", "query": "Do you have product A in stock?"}).

[1895] Operation: The terminal converts the input content into JSON format and sends it to the server as an HTTP POST request.

[1896] Step 3:

[1897] The server parses the received JSON data and extracts the query details.

[1898] Input: Data in JSON format.

[1899] Output: Inquiry details and user information (e.g., {'query': 'Do you have product A in stock?', 'userId': '12345'}).

[1900] Operation: The server receives an HTTP request and extracts the query content by parsing the JSON data.

[1901] Step 4:

[1902] The server uses a natural language processing engine to tokenize the query content, identifying its intent and target.

[1903] Input: Extracted query content.

[1904] Output: Tokenized data (e.g., {'intent': 'Check Inventory', 'target': 'Product A'}).

[1905] Operation: The server uses a natural language processing engine to break down the query into words and phrases, identifying the intent and target.

[1906] Step 5:

[1907] The server uses an emotion recognition engine to recognize the user's emotions.

[1908] Input: Extracted query content.

[1909] Output: Emotion classification data (e.g., {'emotion': 'anxiety'}).

[1910] Operation: The server uses an emotion recognition engine to recognize the user's emotions from the content of the inquiry.

[1911] Step 6:

[1912] The server queries the database based on intent, target, and sentiment data to retrieve the necessary information.

[1913] Input: Tokenized data and sentiment classification data.

[1914] Output: Information retrieved from the database (e.g., {'stock': 'available'}).

[1915] Operation: The server generates and executes SQL queries against the database to retrieve necessary information such as inventory data.

[1916] Step 7:

[1917] The server uses a generative AI model to generate the optimal response.

[1918] Input: Information and sentiment classification data obtained from a database.

[1919] Output: Generated response (e.g., "Rest assured, product A is currently in stock").

[1920] Operation: The server uses a generative AI model to generate appropriate responses based on emotions, using input data.

[1921] Step 8:

[1922] The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[1923] Input: The generated response.

[1924] Output: Response data in JSON format.

[1925] Operation: The server converts the generated response into JSON format and sends it to the terminal as an HTTP response.

[1926] Step 9:

[1927] The terminal parses the response data in JSON format and converts it into a format that can be displayed in the user interface.

[1928] Input: Response data in JSON format.

[1929] Output: Displayable text (e.g., "Rest assured, product A is currently in stock").

[1930] Operation: The device parses the JSON data it receives and converts it into a human-readable text format.

[1931] Step 10:

[1932] The terminal displays the formatted response in the user interface.

[1933] Input: Displayable text.

[1934] Output: The response displayed to the user.

[1935] Operation: The terminal displays a response in the user interface, and the user confirms it.

[1936] These steps allow the system to recognize user emotions and provide appropriate responses, thereby improving the user experience.

[1937] (Application Example 2)

[1938] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1939] Conventional automated response systems can generate answers to inquiries, but they cannot recognize and adapt responses to user emotions. As a result, they fail to alleviate user anxiety and confusion, leading to decreased user satisfaction. This is especially true in virtual stores, where users are required to ask questions in real time and receive appropriate support on the spot, making emotionally sensitive responses crucial. To address this challenge, there is a need to provide a system that recognizes user emotions and adjusts the tone and content of responses accordingly.

[1940] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1941] In this invention, the server includes means for analyzing the query content with a natural language processing engine to extract the intent and target, means for recognizing the user's emotions, and means for adjusting the tone and content of the response based on the emotion recognition. This makes it possible to generate an appropriate response that takes the user's emotions into consideration.

[1942] An "inquiry" refers to a question or request that a user enters when seeking information or support regarding a product or service.

[1943] A "natural language processing engine" refers to a set of algorithms and technologies used to analyze text entered by a user and identify its intent and target.

[1944] "Emotion recognition" is a technology that determines emotions from user input and identifies feelings such as joy, anger, anxiety, and confusion.

[1945] "Tokenization" is the process of dividing input text into a series of words or phrases.

[1946] A "database" is a system that systematically stores information related to products and services, making it searchable and retrievalable as needed.

[1947] "Generative AI" refers to artificial intelligence technology that generates the optimal response to user inquiries.

[1948] A "device" refers to a device operated by a user (e.g., a smartphone, smart glasses, etc.).

[1949] A "response" refers to the information or answer provided by a server to a user.

[1950] A "virtual store" is a purchasing environment built in digital space, rather than a physical store.

[1951] The system of this invention is designed to enhance the user's shopping experience in virtual stores. Using smart glasses, it can process user inquiries in real time and provide responses based on emotion recognition. The specific operation of the system is described below.

[1952] 1. System Configuration

[1953] This system uses the following hardware and software components.

[1954] Hardware:

[1955] Smart glasses: Devices that capture the user's gaze and voice input. They have a built-in AR display and voice input microphone.

[1956] software:

[1957] A natural language processing engine (NLTK) is an engine that analyzes text entered by a user to identify its intent and target. Specific examples include NLTK and spaCy.

[1958] Emotion Recognition Engine: An engine that analyzes user input to identify emotions such as joy, anger, anxiety, and confusion. A concrete example is the IBM Watson Emotion Analysis API.

[1959] A database (Product Database) is a system that stores information related to products and services, and allows users to search and retrieve it as needed. A specific example is MySQL.

[1960] Generative AI (Response Generator): Artificial intelligence technology used to generate optimal responses to user inquiries. Specific examples include OpenAI GPT-3 or 4.

[1961] 2. Flow of Operation

[1962] 1. Inquiry Reception

[1963] Users ask questions about products and services through smart glasses. The voice-input inquiries are converted into text format and sent to the server.

[1964] 2. Inquiry Analysis

[1965] The server analyzes the submitted query using a natural language processing engine. It tokenizes the query and extracts the intent (e.g., inventory check) and the target (e.g., product A).

[1966] 3. Recognition of emotions

[1967] The server uses an emotion recognition engine to identify the user's emotions from the query. For example, it can detect if the user is anxious or confused.

[1968] 4. Database Query

[1969] Based on the extracted information, the server retrieves relevant company-specific information (e.g., inventory status) from the database.

[1970] 5. Response generation

[1971] The server uses generative AI to generate the optimal response based on the information it receives and the user's emotions. The response is adjusted in tone and content to be sensitive to the user's feelings.

[1972] 6. Sending and displaying responses

[1973] The generated response is converted to JSON format and sent to the smart glasses. The smart glasses then display the received response in the user interface.

[1974] 3. Specific examples

[1975] Example 1: Checking product inventory

[1976] User: "Do you have product A in stock?"

[1977] Server: The natural language processing engine identifies the intent of "check inventory" and the target of "product A". The emotion recognition engine identifies "anxiety". Inventory information is retrieved from the database, and the response generation engine generates "Don't worry, product A is currently in stock". This message is sent to the smart glasses and displayed.

[1978] Example of a prompt

[1979] User: "Do you have product A in stock?"

[1980] NLP Engine: Intention = "Check Inventory", Target = "Product A"

[1981] Emotional engine: Emotion = "Anxiety"

[1982] Database: SQL query = "SELECT stock FROM products WHERE name = 'Product A'"

[1983] AI generation: "Don't worry, product A is currently in stock."

[1984] In this way, this system allows users to receive emotionally sensitive, real-time responses, improving their shopping experience in virtual stores.

[1985] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1986] Step 1:

[1987] Inquiry reception

[1988] Users input questions about products and services via voice through smart glasses. The smart glasses convert this voice input into text format and send it to the server as JSON data. Here, the input is voice, and the output is text data. This makes it possible to treat the inquiry content as data.

[1989] Step 2:

[1990] Inquiry analysis

[1991] The server receives the transmitted JSON text data and parses it using a natural language processing engine (e.g., NLTK, spaCy). Specifically, it tokenizes the text and extracts the intent of the inquiry (e.g., inventory check) and the target (e.g., product A). The input is JSON data, and the output is tokenized words and intent / target. Through this analysis, the server can obtain the basic data necessary to take appropriate action.

[1992] Step 3:

[1993] Recognition of emotions

[1994] The server inputs the analyzed text data into an emotion recognition engine (e.g., IBM Watson Emotion Analysis API) to identify the user's emotions (e.g., anxiety, joy). The input is the analyzed text data, and the output is the emotion category. Emotion recognition provides the data needed to generate responses that correspond to the user's emotions.

[1995] Step 4:

[1996] Database query

[1997] The server sends queries to a database (e.g., MySQL) based on the user's intent and target to retrieve information about products and services. For example, if the target is "Product A" and the intent is "check inventory," the server will retrieve inventory information for Product A using an SQL query. The input is the query, and the output is the information retrieved from the database. This step provides specific information in response to the user's inquiry.

[1998] Step 5:

[1999] Response generation

[2000] The server uses generative AI (e.g., OpenAI GPT-3, GPT-4) to generate the optimal response based on acquired information and the user's emotions. For example, if inventory information is available, it will generate a response in a tone that considers the user's emotions, such as, "Don't worry, product A is currently in stock." The input is the acquired data and emotional information, and the output is the generated response text. This response generation enhances user satisfaction.

[2001] Step 6:

[2002] Sending and displaying responses

[2003] The server converts the generated response into JSON format and sends it to the smart glasses. The smart glasses parse the received JSON data and display it on the user interface. The input is the generated response text, and the output is the response message displayed on the smart glasses' screen. This step allows the user to receive answers in real time.

[2004] In this way, a system is realized that automatically generates and provides appropriate responses to user inquiries through each step.

[2005] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2006] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2007] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[2008] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2009] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[2010] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[2011] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[2012] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[2013] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[2014] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[2015] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[2016] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[2017] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[2018] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2019] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[2020] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[2021] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[2022] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[2023] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[2024] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[2025] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[2026] The following is further disclosed regarding the embodiments described above.

[2027] (Claim 1)

[2028] The means by which users enter inquiries,

[2029] A means of sending the inquiry content to the server,

[2030] The server analyzes the query content using a natural language processing engine and extracts the intent and target,

[2031] A means of obtaining company-specific information from a database based on the extracted information,

[2032] A means by which a generative AI generates the optimal response,

[2033] A means for sending the generated response to the terminal,

[2034] The means by which the terminal displays a response to the user

[2035] A system that includes this.

[2036] (Claim 2)

[2037] The system according to claim 1, which includes a means for a natural language processing engine to tokenize and identify the intent and target of a query.

[2038] (Claim 3)

[2039] The system according to claim 1, which includes means for a generative AI to generate a response by referring to past response history.

[2040] "Example 1"

[2041] (Claim 1)

[2042] A means by which the user inputs an inquiry into the information processing device,

[2043] A means of transmitting the inquiry content from the information processing device to the host computer,

[2044] A host computer analyzes the query content using a language processing engine and extracts the intent and target,

[2045] A means of obtaining company-specific information from a data set based on extracted information,

[2046] A means by which a response generation engine generates an optimal response,

[2047] Means for transmitting the generated response to an information processing device,

[2048] The means by which an information processing device presents a response to the user

[2049] A system that includes this.

[2050] (Claim 2)

[2051] The system according to claim 1, which includes a means for a language processing engine to tokenize and identify the intent and target of a query.

[2052] (Claim 3)

[2053] The system according to claim 1, comprising means for a response generation engine to generate a response by referring to past correspondence history.

[2054] "Application Example 1"

[2055] (Claim 1)

[2056] The means by which users enter inquiries,

[2057] A means of sending the inquiry content to the server,

[2058] The server analyzes the query content using a natural language processing engine and extracts the intent and target,

[2059] A means of obtaining information from a database based on the extracted information,

[2060] A means by which a generative AI generates the optimal response,

[2061] A means for sending the generated response to the terminal,

[2062] A means by which the terminal displays a response to the user,

[2063] A means to enable staff to use smart devices to respond immediately to customer inquiries,

[2064] A system that includes this.

[2065] (Claim 2)

[2066] The system according to claim 1, which includes a means for a natural language processing engine to tokenize and identify the intent and target of a query.

[2067] (Claim 3)

[2068] The system according to claim 1, which includes means for a generative AI to generate a response by referring to past response history.

[2069] "Example 2 of combining an emotion engine"

[2070] (Claim 1)

[2071] The means by which users enter inquiries,

[2072] A means by which the terminal sends the inquiry details to the server,

[2073] The server analyzes the query content using a natural language processing engine and extracts the intent and target,

[2074] A means by which the server recognizes the user's emotions based on the extracted information,

[2075] A means by which the server retrieves necessary information from the database based on emotion recognition,

[2076] A means for generating an optimal response based on information and emotional information acquired by a generative AI,

[2077] A means by which the server sends the generated response to the terminal,

[2078] The means by which the terminal displays a response to the user

[2079] A system that includes this.

[2080] (Claim 2)

[2081] The system according to claim 1, which includes a means for a natural language processing engine to tokenize and identify the intent and target of a query.

[2082] (Claim 3)

[2083] The system according to claim 1, which includes means for a generative AI to generate a response by referring to past response history.

[2084] "Application example 2 when combining with an emotional engine"

[2085] (Claim 1)

[2086] The means by which users enter inquiries,

[2087] A means of sending the inquiry content to the server,

[2088] The server analyzes the query content using a natural language processing engine and extracts the intent and target,

[2089] A means by which the server recognizes the user's emotions based on the extracted information,

[2090] A means of adjusting the tone and content of responses based on the user's emotional perception,

[2091] A means of obtaining company-specific information from a database based on the extracted information,

[2092] A means by which a generative AI generates the optimal response,

[2093] A means for sending the generated response to the terminal,

[2094] The means by which the terminal displays a response to the user

[2095] A system that includes this.

[2096] (Claim 2)

[2097] The system according to claim 1, which includes a means for a natural language processing engine to tokenize and identify the intent and target of a query.

[2098] (Claim 3)

[2099] The system according to claim 1, which includes means for a generative AI to generate a response by referring to past response history. [Explanation of symbols]

[2100] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. The means by which users enter inquiries, A means of sending the inquiry content to the server, The server analyzes the query content using a natural language processing engine and extracts the intent and target, A means of obtaining company-specific information from a database based on the extracted information, A means by which a generative AI generates the optimal response, A means for sending the generated response to the terminal, The means by which the terminal displays a response to the user A system that includes this.

2. The system according to claim 1, which includes a means for a natural language processing engine to tokenize and identify the intent and target of a query.

3. The system according to claim 1, which includes means for a generative AI to generate a response by referring to past response history.

Images