system

Abstract

We provide the system. [Solution] A means of sending questions entered by the user from the terminal to the server, The server includes means for analyzing the received question using a natural language processing engine, The server provides means for querying a database based on the analysis results and generating a response, The server provides means for sending the generated response to the user's terminal, A system including means for displaying the response received by the terminal on a user interface.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In order to provide a quick and accurate real-time response to inquiries from customers to a company, the conventional system has problems such as delays in customer service and limitations of human resources. The present invention aims to solve such problems and provide an efficient system that can immediately respond to questions from a large number of customers.

Means for Solving the Problems

[0005] <00000३०>The present invention solves the above-mentioned problems with a system that includes means for sending a question entered by a user from a terminal to a server, means for the server to analyze the received question using a natural language processing engine, means for the server to query a database based on the analysis results and generate an answer, means for the server to send the generated answer to the user's terminal, and means for the terminal to display the received answer on a user interface.

[0006] A "user" refers to an individual or legal entity that uses the system to input and submit questions.

[0007] "Terminal" refers to devices used by users, such as computers, smartphones, and tablets.

[0008] A "question" refers to the content of an inquiry entered by the user from their device.

[0009] A "server" refers to a central processing unit that receives questions, analyzes them, and generates answers.

[0010] A "natural language processing engine" refers to a program or algorithm used within a server to analyze a question.

[0011] "Analysis" refers to the process of identifying and understanding keywords and intent from received questions.

[0012] A "database" refers to an information storage system used to retrieve information based on analysis results.

[0013] "Response" refers to the content of the response generated based on the analysis results and information obtained from the database.

[0014] "Generation" refers to the process of constructing an answer based on analysis results and information obtained from the database.

[0015] "Sending" refers to the process of sending a generated response from the server to the terminal.

[0016] The "user interface" refers to the display part on the terminal where the user can visually confirm the inquiry content and answers.

[0017] "Display" refers to the process of visually outputting the answers generated on the user interface of the terminal.

Brief Explanation of Drawings

[0018] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Modes for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0022] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0023] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0024] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0026] [First Embodiment]

[0027] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0028] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0031] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0034] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0038] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0039] The present invention is a system for providing real-time answers to customer questions, and is primarily carried out through a series of processes including question submission, analysis, database querying, answer generation, and answer submission and display.

[0040] Submit a question

[0041] When a user enters a question on their device and clicks the send button, the device sends the question to the server. The device generates an API request, packages the question and associated information (user ID, timestamp, etc.), and sends it to the server.

[0042] Receiving and analyzing questions

[0043] The server receives user queries through the API endpoint. The server analyzes the received queries using a natural language processing engine to identify keywords and user intent. This analysis enables the server to query the appropriate database.

[0044] Database query and response generation

[0045] Based on the analysis results, the server queries the relevant database. For example, if the question "Is it in stock?" is analyzed, the server connects to the database to retrieve the inventory information for the relevant product and uses an SQL query to obtain the inventory information.

[0046] The server generates a response based on the acquired data. The generated response may use a predefined phrase or it may be dynamically constructed. For example, the response "We have it in stock" might be generated.

[0047] Submit and display of responses

[0048] The generated response is sent from the server to the user's device. The device receives this response and displays it in the user interface. The user can then view the answer to the question on the device screen.

[0049] Specific example

[0050] For example, if a user enters the question "Is this item in stock?" and presses the submit button, the device sends the question to the server. The server receives the question and uses natural language processing to analyze the keyword "stock." Next, the server connects to the database and queries for the stock information of that item. If the database returns "In stock," the server uses this information to generate the answer "This item is in stock" and sends it to the user's device. The device displays the received answer on the screen to inform the user.

[0051] This system allows users to get quick and accurate answers to their questions in real time, significantly improving customer service.

[0052] The following describes the processing flow.

[0053] Step 1:

[0054] The user enters their question from their device. The user enters their question into the input form and clicks the submit button. Example: "Is this product in stock?"

[0055] Step 2:

[0056] The device sends the question to the server. The device generates an API request and sends it to the server along with the entered question, user ID, timestamp, and other associated information.

[0057] Step 3:

[0058] The server receives the question. The server's API endpoint receives the request and extracts the question and associated information.

[0059] Step 4:

[0060] The server analyzes the question. The server's natural language processing engine analyzes the question and identifies keywords and intent. Example: Extract the keyword "inventory" from the question.

[0061] Step 5:

[0062] The server queries the database. Based on the identified keywords, the server connects to the database and executes an SQL query to retrieve the relevant information.

[0063] Step 6:

[0064] The server generates the response. Based on information retrieved from the database, the server dynamically generates a response for the user. For example, it might generate the response "In stock."

[0065] Step 7:

[0066] The server sends the answer to the terminal. The response containing the generated answer is sent to the terminal.

[0067] Step 8:

[0068] The device receives the response. The device parses the received response and formats the content to be displayed.

[0069] Step 9:

[0070] The device displays the answer in the user interface. The user can confirm the answer on the device screen. For example, they might see a message like, "This item is in stock!"

[0071] (Example 1)

[0072] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0073] Traditional systems often failed to provide quick and accurate answers to customer inquiries, posing a challenge to improving customer service. Furthermore, the mechanisms for properly analyzing questions entered in natural language, acquiring necessary data, and generating accurate responses were insufficient. As a result, user experience deteriorated, and business efficiency was sometimes compromised.

[0074] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0075] In this invention, the server includes means for transmitting a question entered by a user from a terminal, means for analyzing the question using a natural language processing engine, and means for querying a database and generating an answer based on the analysis results. This makes it possible to receive questions from customers in real time and provide quick and accurate answers.

[0076] A "terminal" refers to a device used by a user for input, and includes, for example, smartphones, personal computers, and tablets.

[0077] A "server" refers to a computer system that receives data sent by users and performs analysis and processing on it.

[0078] A "natural language processing engine" refers to a software component that analyzes human language and understands its meaning and intent.

[0079] An "API request" refers to a communication request from a device to a server to request operations or data.

[0080] A "database" refers to a system that manages organized collections of data, and an example of this is a relational database management system (RDBMS).

[0081] "Analysis results" refer to the information obtained by a natural language processing engine through its analysis of questions and input data.

[0082] "Answer" refers to the response that the server generates based on the user's question.

[0083] "User interface" refers to the screen and software portion of a device that a user uses to perform operations and input.

[0084] "JSON format" refers to a text-based data format used to structure and represent data information.

[0085] This invention is a system that provides real-time answers to customer questions and is implemented through a main processing flow. The system begins with the user inputting and sending a question from a terminal. The terminal generates the input question as an API request and sends it to the server. The API request includes information such as the question content, user ID, and timestamp.

[0086] The server receives requests through the specified API endpoint and uses a natural language processing engine to analyze the question content. Commonly used natural language processing engines such as Google® Cloud Natural Language API and OpenAI® generative AI models can be used. This allows the server to identify keywords and user intent from the question.

[0087] Based on the analysis results, the server connects to the relevant database and retrieves the necessary information. This involves issuing SQL queries using relational database management systems (RDBMS) such as MySQL® or PostgreSQL. For example, if the question "Is this in stock?" is analyzed, a database query is made to retrieve the inventory information for the relevant product. Based on the retrieved information, the server generates an answer either dynamically or using a predefined template.

[0088] The generated response is packaged again in API request format and sent to the user's device. The device parses the received response and displays it in the user interface. The user interface can be a web page or mobile application built using HTML, CSS, JavaScript (registered trademark), etc.

[0089] For example, if a user enters the question "Is this item in stock?" and presses the submit button, the device sends the question to the server. The server receives the question, analyzes it using a natural language processing engine, and identifies the keyword "stock." Next, the server connects to a database and queries for stock information about the item. If the database returns "in stock," the server generates the answer "This item is in stock" and sends it to the user's device. The device then displays the received answer on the user interface to inform the user.

[0090] Examples of prompt statements include the following:

[0091] "Do you have this item in stock?"

[0092] "Could you tell me your business hours?"

[0093] "What's the latest promotion?"

[0094] This system allows users to receive quick and accurate answers to their questions in real time, thereby improving customer service.

[0095] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0096] Step 1:

[0097] The user enters a question from their device and clicks the submit button. This input includes information such as the question, user ID, and timestamp. When the user enters "Is this product in stock?" and clicks the submit button, the device retrieves the input and generates an API request in JSON format. The API request is then sent to the server as output.

[0098] Step 2:

[0099] The server processes requests received at the API endpoint. The input here is the API request sent from the terminal. The server first parses the JSON data to extract information such as the question content, user ID, and timestamp. Based on this extracted information, it prepares data to pass to the natural language processing engine, and generates data for analysis as output.

[0100] Step 3:

[0101] The server analyzes the question using a natural language processing engine. The input is the data prepared in step 2. The server uses a natural language processing engine, such as the Google Cloud Natural Language API or OpenAI's generative AI model, to identify keywords and intent from the question. If the keyword "inventory" is identified through this analysis, the analysis result is obtained as output.

[0102] Step 4:

[0103] The server queries the database based on the analysis results. The input here is the analysis result of natural language processing, which contains information that includes the identified keywords. The server connects to a relational database management system (RDBMS) such as MySQL or PostgreSQL, generates an SQL query, and sends it. For example, a query like "SELECT in_stock FROM products WHERE product_id = 98765" is issued. The output is inventory information returned from the database.

[0104] Step 5:

[0105] The server generates a response based on information retrieved from the database. The input is inventory information retrieved from the database. Based on the retrieved data, the server generates a response such as "This item is in stock" if the item is in stock. The output is the response text for the user.

[0106] Step 6:

[0107] The server generates a response and sends it to the device. The input is the response text generated by the server. The server packages the response in JSON format, generates another API request, and sends it to the device. The output is the API request sent to the device.

[0108] Step 7:

[0109] The terminal displays the received response on the user interface. The input is response data in JSON format received from the server. The terminal parses this data and formats it for display on the user interface. For example, HTML and JavaScript are used to display the response on the screen. As output, the user can see the answer to the question on the screen.

[0110] Through these steps, users can obtain quick and accurate answers to their questions in real time.

[0111] (Application Example 1)

[0112] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0113] Currently, when customers in physical stores want to know product information or inventory information, they typically ask store staff directly. However, it is difficult to get a quick response during busy periods or when staff are engaged in other tasks. This invention aims to solve this problem and provide a system that enables customers to obtain information quickly and accurately.

[0114] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0115] In this invention, the server includes means for transmitting a question entered by a user from a mobile device to the server; means for analyzing the received question using a natural language processing system; means for querying an information storage device based on the analysis results and generating an answer; means for transmitting the generated answer to the user's mobile device; means for displaying the answer received by the mobile device on a user interface; input means for enabling the user to input questions using voice or text; and means for receiving and displaying answers to questions in real time via software installed on the mobile device. This enables customers to quickly and accurately obtain product and inventory information even in physical stores.

[0116] A "personal information terminal" is a portable device such as a smartphone or smart glasses that a user uses to input questions and receive and display answers.

[0117] A "natural language processing system" is an artificial intelligence technology that analyzes questions received from users to identify their intent and keywords.

[0118] "Information storage device" refers to a database or storage device that stores data for generating answers to questions.

[0119] An "input method" is a function that allows users to input questions using voice or text.

[0120] A "user interface" is a screen or display area on a mobile device that allows the user to visually confirm the answers they have generated.

[0121] "Software" refers to a program installed on a mobile device that controls the sending of questions and the receiving and display of answers.

[0122] "Real-time" refers to responding quickly to user questions and providing answers without delay.

[0123] This invention provides a system that allows customers to quickly obtain product and inventory information in physical stores. This system includes a mobile information terminal, a server, a natural language processing system, an information storage device, input means, and a user interface.

[0124] System Configuration

[0125] 1. Mobile device:

[0126] In this system, a personal digital assistant (PDA) is a device used by the user to input questions and receive and display answers. Specifically, this includes smartphones and smart glasses.

[0127] 2. Server:

[0128] The server receives a question from the user and analyzes it using a natural language processing system. It then queries its information storage device based on the analysis results and generates an answer. Finally, it sends the generated answer to the user's mobile device.

[0129] 3. Natural Language Processing Systems:

[0130] The natural language processing system used on the server analyzes received questions to identify keywords and user intent. This analysis enables appropriate database queries. Specific software used includes Spacy, among others.

[0131] 4. Information storage device:

[0132] A database or storage device that holds data for generating answers to questions. For example, it might store product inventory information or detailed information.

[0133] 5. Input method:

[0134] A feature that allows users to input questions using voice or text. This includes voice input devices and text input interfaces.

[0135] 6. User Interface:

[0136] This refers to a screen or display area on a mobile device that allows users to visually confirm the answers they have generated. This enables users to instantly obtain the necessary information within the store.

[0137] System operation

[0138] When a user inputs a question via voice or text using a mobile device, the question is sent from the device to the server. The server receives the question and analyzes it using a natural language processing system. Based on the keywords in the analyzed question, the server queries its information storage device and generates an appropriate answer. The generated answer is sent from the server to the mobile device and displayed on the device's user interface.

[0139] Specific example

[0140] For example, if a user asks their smart glasses, "Do you have this item in stock?", the question is sent to the server. The server analyzes the question and identifies the keyword "stock". It then connects to a data storage device and retrieves the stock information for that item. If the database returns "in stock", the server generates the response "This item is in stock" and displays it on the smart glasses.

[0141] Example of a prompt

[0142] Question: "Do you have this item in stock?"

[0143] Answer: "This item is in stock."

[0144] This allows customers to easily obtain product information in physical stores, resulting in an efficient shopping experience.

[0145] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0146] Step 1:

[0147] The user enters the question using voice or text via a mobile device.

[0148] Input: User's question (e.g., "Do you have this item in stock?")

[0149] Output: Question data (audio data or text data)

[0150] Operation: Questions are entered using the input function of a smartphone or smart glasses. The user interface utilizes voice recognition and text input.

[0151] Step 2:

[0152] The terminal sends the question data to the server.

[0153] Input: Question data

[0154] Output: API Request

[0155] Operation: The terminal packages the question data, generates an API request, and sends it to the server. Specifically, the request includes information such as the question text, user ID, and timestamp.

[0156] Step 3:

[0157] The server receives the question data through the API endpoint.

[0158] Input: API Request

[0159] Output: Question data (received on the server side)

[0160] Operation: The server receives question data via the API endpoint and places it in a queue for preparation for analysis.

[0161] Step 4:

[0162] The server uses a natural language processing system to analyze the question.

[0163] Input: Question data

[0164] Output: Analysis results (keywords and intent)

[0165] Operation: Uses a natural language processing system (e.g., Spacy) to analyze the received question text and identify keywords and user intent. For example, it extracts nouns such as "inventory" and verbs such as "do you have it?".

[0166] Step 5:

[0167] The server queries the information storage device based on the analysis results.

[0168] Input: Analysis results (keywords and intent)

[0169] Output: Database query results (e.g., inventory information)

[0170] Operation: Based on the analysis results, it generates an appropriate SQL query and queries the information storage device (database). It retrieves inventory information and related data for the relevant product from the database.

[0171] Step 6:

[0172] The server generates an answer based on the data it has acquired.

[0173] Input: Database query results (e.g., inventory information)

[0174] Output: Generated response (Example: "This item is in stock.")

[0175] Function: Processes query results and generates appropriate boilerplate or dynamically constructed responses. For example, from the data "In Stock," it creates a response such as "This product is in stock."

[0176] Step 7:

[0177] The server sends the generated response to the terminal.

[0178] Input: Generated answer

[0179] Output: API response (response data)

[0180] Operation: The server packages the generated response and sends it to the terminal as an API response.

[0181] Step 8:

[0182] The device displays the received response on the user interface.

[0183] Input: API response (response data)

[0184] Output: Displayed answer

[0185] Operation: The device receives a response from the server and displays it on the user interface. For example, "This item is in stock" might be displayed on the screen of smart glasses or a smartphone.

[0186] Through these steps, users can quickly obtain product and inventory information even in physical stores.

[0187] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0188] The present invention is a system for providing real-time answers to customer questions, and is particularly characterized by its ability to recognize the user's emotions by combining it with an emotion engine, thereby providing more appropriate answers. Specific embodiments of the system of the present invention are shown below.

[0189] Question submission and sentiment recognition

[0190] The user enters a question on the device and clicks the send button. The device prepares to send the entered question to the server and activates an emotion engine to recognize the user's emotions from the question. The emotion engine analyzes the text of the question and generates emotion parameters such as joy, anger, and sadness.

[0191] Reception and analysis on the server

[0192] The device sends an API request to the server, which includes the question, user ID, timestamp, and sentiment parameter. The server receives the request through the API endpoint and extracts the question and sentiment parameter.

[0193] Question analysis and database queries

[0194] The server's natural language processing engine analyzes the question, identifying keywords and intent. Sentiment parameters are also considered and used as additional information to generate an appropriate response. Based on the analyzed keywords, the server queries the database. For example, even if the question is "out of stock," data is retrieved to provide a gentle notification that takes the user's emotions into consideration.

[0195] Answer generation

[0196] Based on information retrieved from the database and emotion parameters, the server generates an appropriate response. Based on the output of the emotion engine, the response is adjusted to be considerate of the user's emotions. For example, a response that is considerate of the user's emotions might be created, such as, "We are sorry, but this item is currently out of stock. We will notify you as soon as it becomes available."

[0197] Submit and display of responses

[0198] The server sends the generated response to the user's device. The device receives the response and displays it in the user interface. The user can see the specific answer to the question, along with a message that takes their feelings into consideration, on the device screen.

[0199] Specific example

[0200] For example, a user enters the question, "Do you have this item in stock? I'm tired of waiting," and presses the submit button. The device sends the question and emotion parameter (in this case, the emotion "tired") to the server. The server receives the question and analyzes it using a natural language processing engine and an emotion engine, extracting the keyword "stock" and the emotion "tired." Next, the server queries the database for stock information and, if the item is out of stock, generates an emotion-sensitive response. Finally, the device displays a message to the user saying, "We're sorry, but this item is currently out of stock. We will notify you as soon as it becomes available."

[0201] This system allows users to receive quick and accurate answers to their questions in real time, as well as emotionally sensitive service, significantly improving customer satisfaction.

[0202] The following describes the processing flow.

[0203] Step 1:

[0204] The user enters a question from their device. The user enters the question into the input form and clicks the submit button. Example: Enter "Is this product in stock?"

[0205] Step 2:

[0206] The device sends the question to the server. The device generates an API request, packages the entered question with accompanying information such as the user ID and timestamp, and sends it to the server.

[0207] Step 3:

[0208] The device activates its emotion engine. Based on the entered questions, the device uses the emotion engine to analyze the user's emotions. For example, it extracts emotions such as "anger" or "dissatisfaction" from text.

[0209] Step 4:

[0210] The device sends emotion parameters to the server. In addition to the question, the analyzed emotion parameters are also included in the API request and sent to the server.

[0211] Step 5:

[0212] The server receives the question and sentiment parameters. The server's API endpoint receives the request and extracts the question and sentiment parameters.

[0213] Step 6:

[0214] The server analyzes the question. A natural language processing engine built into the server analyzes the question, identifying key keywords and user intent. For example, it identifies the keyword "inventory."

[0215] Step 7:

[0216] The server analyzes the emotional parameters. The server analyzes the emotional parameters received from the emotion engine to understand the user's emotional state. Example: It determines that the user is "highly dissatisfied."

[0217] Step 8:

[0218] The server queries the database. Based on the specified keywords, the server connects to the relevant database and executes an SQL query to retrieve the necessary information. Example: Retrieving product inventory information.

[0219] Step 9:

[0220] The server generates the response. Based on information retrieved from the database and sentiment parameters, the server generates an appropriate response. It constructs a message that takes the user's feelings into consideration, taking sentiment parameters into account. Example: It generates "We are sorry, but this item is currently out of stock. We will notify you as soon as it becomes available."

[0221] Step 10:

[0222] The server sends the generated response to the device. The API response containing the generated response is then sent to the user's device.

[0223] Step 11:

[0224] The device receives the response. The device receives the API response, parses the content of the response, and formats it into data for display.

[0225] Step 12:

[0226] The device displays the answer in the user interface. The user can see a specific answer to the question and an emotionally sensitive message on the device screen. For example, it might display: "We are sorry, but this item is currently out of stock. We will notify you as soon as it becomes available."

[0227] Through this series of processes, users can not only receive real-time answers to their questions, but also receive responses that take their emotional state at the time into consideration. This significantly improves customer satisfaction.

[0228] (Example 2)

[0229] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0230] Conventional question-answering systems failed to consider the user's emotions when they entered and submitted a question. This resulted in the inability to generate appropriate answers that took the user's feelings into account, leading to decreased customer satisfaction. Furthermore, the system lacked the means to provide answers that reflected the user's emotions, resulting in inefficient communication with users.

[0231] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0232] In this invention, the server includes means for sending a question entered by the user from a terminal to the server, means for the terminal to analyze the content of the question and generate sentiment parameters, means for the server to analyze the received question using a natural language processing engine, means for the server to query a database based on the analysis results and sentiment parameters and generate an answer, means for the server to send the generated answer to the user's terminal, and means for the terminal to display the received answer on a user interface. This makes it possible to provide answers that take the user's emotions into consideration in real time, thereby improving customer satisfaction and enabling efficient communication.

[0233] A "user" refers to an individual or organization that operates the system and enters questions.

[0234] "Terminal" refers to electronic devices, including input / output devices such as computers and smartphones used by users.

[0235] A "question" refers to the content of an inquiry that a user enters into the system via their device.

[0236] A "server" refers to a central processing unit that receives, analyzes, processes, and responds to user inquiries.

[0237] "Transmission method" refers to the function for sending user questions from the terminal to the server.

[0238] "Emotional parameters" refer to data that indicates the emotional state (e.g., joy, anger, sadness, etc.) analyzed from the user's questions.

[0239] A "natural language processing engine" refers to software or programs that a server uses to analyze a question and identify keywords and intent.

[0240] A "database" refers to an information storage system that stores answers to questions and manages them so that they can be searched when needed.

[0241] "Query method" refers to the function that allows a server to instruct a database to search for information related to the query.

[0242] "Generation means" refers to the function that allows the server to create an appropriate response based on information and sentiment parameters obtained from the database.

[0243] "Display means" refers to the function that allows the terminal to display the response received from the server on the user interface.

[0244] The system of this invention provides emotionally sensitive and appropriate answers in real time to questions sent by users from their terminals. The system of this invention is implemented by combining the following hardware and software.

[0245] Entering and submitting questions via a terminal.

[0246] The user accesses the system from a device (such as a computer or smartphone) and enters a question. For example, the user enters the following prompt:

[0247] Do you have this item in stock? I'm tired of waiting.

[0248] When the user clicks the "Submit" button, the device receives this question and proceeds to the next step.

[0249] Emotion recognition by devices

[0250] The device uses a cloud-based natural language processing API to analyze the question and recognize the user's emotions. Specifically, it uses the Google Cloud Natural Language API to analyze the question and extract emotional parameters such as joy, anger, and sadness.

[0251] The emotion parameters obtained through this process (e.g., the emotion "tired") are sent to the server in JSON format along with the question data.

[0252] Reception and analysis on the server

[0253] The server receives requests sent from the terminal through an API endpoint built using the Spring Boot framework. The received JSON data includes the question content, user ID, timestamp, and sentiment parameter. The server extracts this data and uses it for the next processing step.

[0254] Question analysis and database queries

[0255] The server analyzes the question using the Hugging Face Transformers library. A natural language processing engine identifies the main keywords of the question and the user's intent. Sentiment parameters are also taken into consideration.

[0256] Based on the analyzed keywords, the server queries the MySQL database. For example, it generates an SQL query to retrieve inventory information and obtains the necessary information from the database.

[0257] Answer generation

[0258] The server generates an appropriate response based on the retrieved database information and emotion parameters. During this process, the response is adjusted based on the emotion engine's output to ensure it reflects the user's emotions. For example, if an item is out of stock, a comment such as "We apologize, but this item is currently out of stock. We will notify you as soon as it becomes available" is generated.

[0259] Submit and display of responses

[0260] The server sends the generated response to the device in JSON format. The device receives the response asynchronously using JavaScript (AJAX) and displays it in the user interface (for example, a web application using React.js). The user can then see an appropriate and emotionally sensitive response to their question on the device screen.

[0261] In this way, the system of the present invention allows users to obtain quick and accurate answers in real time, and furthermore, these answers take into consideration the user's feelings. As a result, customer satisfaction is significantly improved.

[0262] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0263] Step 1:

[0264] The user enters and submits the question from their terminal. The user enters the question on the system interface and clicks the submit button. For example, the user might enter the following prompt:

[0265] Do you have this item in stock? I'm tired of waiting.

[0266] After this input is received, the terminal retains the question data and proceeds to the next processing step.

[0267] Step 2:

[0268] The device prepares to send a question to the server. Simultaneously, it starts sentiment recognition using the Google Cloud Natural Language API. The device analyzes the question and extracts sentiment parameters. For example, the sentiment "tired" might be recognized. Based on this analysis, the device converts the question and sentiment parameters into JSON format and constructs the data to send to the server. Input: User's question, Output: JSON data with sentiment parameters.

[0269] Step 3:

[0270] The device sends JSON data it has constructed to the server using a RESTful API. Input: JSON data with sentiment parameters; Output: Request sent to the server. This request includes the user ID, question content, timestamp, and sentiment parameters.

[0271] Step 4:

[0272] The server receives the request. It uses a Spring Boot-based API endpoint to receive the request and parse the data. Input: Request sent to the server; Output: Extracted question content, user ID, timestamp, and sentiment parameter.

[0273] Step 5:

[0274] The server analyzes the extracted question content. Using the Hugging Face Transformers library, key keywords and user intent are identified. For example, the keyword "inventory" might be extracted. Sentiment parameters are also included in the processing. Input: Question content; Output: Identified keywords and sentiment parameters.

[0275] Step 6:

[0276] The server queries the database based on the question. It executes an SQL query against the MySQL database to retrieve relevant information. For example, an SQL query to check inventory information is executed. Input: Identified keyword, Output: Inventory information retrieved from the database.

[0277] Step 7:

[0278] The server generates an appropriate response based on information retrieved from the database and emotion parameters. Based on the output of the emotion engine, the response is adjusted to be sensitive to the user's emotions. For example, a response such as "We are sorry, but we are currently out of stock. We will notify you as soon as it becomes available" might be generated. Input: Stock information and emotion parameters, Output: Emotionally sensitive response.

[0279] Step 8:

[0280] The server generates the response and sends it to the terminal in JSON format. Input: Generated response, Output: Response sent to the user's terminal.

[0281] Step 9:

[0282] The terminal displays the response received from the server on the user interface. For example, the response is received via asynchronous communication (AJAX) using JavaScript and displayed using React.js or the like. The user can confirm, on the screen of the terminal, a message that takes into account their feelings along with the specific response to the question. Input: Response sent from the server, Output: Response displayed on the user interface.

[0283] (Application Example 2)

[0284] Next, Application Example 2 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart device 14 is referred to as the "terminal".

[0285] In a conventional real-time question-and-answer system, it is difficult to provide an answer that takes into account the user's feelings, and as a result, there is a problem that user satisfaction decreases. In particular, an answer that ignores the user's feelings embedded in the question is a factor that degrades the quality of the customer experience. Also, from the perspective of sales promotion, service provision that takes into account feelings is required.

[0286] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0287] In this invention, the server includes means for transmitting a question input by the user from the terminal to the server, means for activating an emotion engine for recognizing the user's feelings from the input question, means for analyzing the received question using a natural language processing engine, means for querying a database based on the analysis result and generating an answer, means for adjusting the generated answer based on emotion parameters, means for transmitting the generated answer to the user's terminal, and means for displaying the received answer on the user interface. Thereby, real-time question-and-answer that takes into account the user's feelings becomes possible.

[0288] The "user" refers to a person who inputs a question using the system.

[0289] A "device" refers to a device used by a user to input questions and receive answers. Examples include smartphones and tablets.

[0290] A "question" refers to the content of an inquiry that a user enters from their device and sends to the system.

[0291] A "server" refers to a central processing unit that analyzes received questions, generates answers, and sends them to terminals.

[0292] A "natural language processing engine" refers to a processing unit that analyzes input questions and understands their meaning and intent.

[0293] An "emotion engine" refers to a device that generates emotional parameters from questions entered by the user.

[0294] "Emotional parameters" refer to data generated by the emotion engine that indicates the user's emotional state.

[0295] A "database" refers to an information storage system that stores the information necessary to generate answers to questions.

[0296] "Answer" refers to the response to an inquiry that the server generates and provides to the user.

[0297] "User interface" refers to the means of displaying answers visually on a device.

[0298] System Configuration

[0299] A system for carrying out this invention includes the following main components:

[0300] User's terminal

[0301] server

[0302] Natural Language Processing Engine

[0303] Emotional Engine

[0304] Database

[0305] Program Execution

[0306] 1. Sending a Question from the User's Terminal

[0307] The user enters a question from the terminal and presses the send button. This operation activates the emotional engine for recognizing the user's emotion along with the question content.

[0308] 2. Emotion Recognition by the Emotional Engine

[0309] The emotional engine analyzes the question text and generates the user's emotion parameters (e.g., joy, anger, sadness). For this process, for example, the "sentiment - analysis" model of Transformer is used.

[0310] 3. Data Transmission to the Server

[0311] The terminal sends an API request containing the question content, user ID, and emotion parameters to the server. The request is sent in JSON format.

[0312] 4. Receiving and Analyzing at the Server

[0313] The server receives the request at the API endpoint and extracts the question and emotion parameters. Next, it analyzes the question using a natural language processing engine (e.g., SpaCy or BERT) to identify keywords and intentions.

[0314] 5. Retrieving Information from the Database

[0315] Based on the analyzed keywords, the server queries the database. For example, if the keyword "inventory" is included, inventory information is retrieved from the database.

[0316] 6. Generating responses and adjusting them with consideration for emotions.

[0317] Based on information retrieved from the database and emotion parameters, the server generates an appropriate response. A generative AI model (e.g., GPT-3®) is used to refine the response to be more emotionally conscious.

[0318] 7. Submitting and displaying responses

[0319] The server sends the generated response to the user's terminal. The terminal formats the received response and displays it on the user interface.

[0320] Specific example

[0321] For example, suppose a user enters the question, "Do you have this item in stock? I'm tired of waiting," and presses the submit button. The emotion engine recognizes the emotion "tired." An API request containing the question and emotion parameters is sent to the server, which analyzes the keyword "stock" and the emotion "tired." Next, it queries the database for stock information, and if it does not have the item in stock, it generates an emotion-sensitive response such as, "We're sorry, but we don't have this item in stock at the moment. We will notify you as soon as it becomes available." This response is then sent to the device and displayed on the user interface.

[0322] Example prompt

[0323] The following is an example of a prompt used with the emotion engine:

[0324] "Do you have this item in stock? I'm tired of waiting."

[0325] By allowing users to input specific questions in this way, the system can provide quick and emotionally sensitive answers. This entire process improves customer satisfaction and delivers a better user experience.

[0326] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0327] Step 1:

[0328] The user enters a question from their device and presses the send button. The user might enter a specific question, such as, "Do you have this item in stock? I'm tired of waiting." The entered question is temporarily stored as text data on the device.

[0329] Step 2:

[0330] The device passes the entered question to the sentiment engine to begin analysis. The sentiment engine analyzes the question text using, for example, Transformer's "sentiment-analysis" model, and generates the user's sentiment parameters (e.g., joy, anger, sadness). The generated sentiment parameters are represented as a label such as "tired" and its confidence score (e.g., 0.95).

[0331] Step 3:

[0332] The device sends an API request to the server that includes the question content and sentiment parameters. This request includes the question text, user ID, sentiment label, and sentiment score.

[0333] Step 4:

[0334] The server receives the request at the API endpoint and extracts the question, user ID, sentiment label, and sentiment score. The extracted data is then split into its respective fields and passed on to subsequent processing.

[0335] Step 5:

[0336] The server analyzes the question text using a natural language processing engine. For example, it uses SpaCy or BERT to identify keywords and intent in the question. From the question text "Do you have this item in stock? I'm tired of waiting," keywords such as "stock" and "tired" are extracted.

[0337] Step 6:

[0338] The server queries the database for relevant information based on the analyzed keywords. For example, a database query containing the keyword "inventory" is generated, and inventory information for the corresponding product is retrieved.

[0339] Step 7:

[0340] The server generates appropriate responses based on information retrieved from the database and sentiment parameters. In this process, a generative AI model (e.g., GPT-3) is used to generate responses that take into account the intent of the question and the user's emotions. For example, a response such as, "We apologize, but this item is currently out of stock. We will notify you as soon as it becomes available," might be generated.

[0341] Step 8:

[0342] The server sends the generated response to the user's device. The data sent includes the text of the response.

[0343] Step 9:

[0344] The device formats the received response and displays it on the user interface. Specifically, the formatted text will read, "We apologize, but this item is currently out of stock. We will notify you as soon as it becomes available."

[0345] This processing flow allows users to receive quick, emotionally sensitive answers to their questions in real time.

[0346] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0347] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0348] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0349] [Second Embodiment]

[0350] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0351] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0352] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0353] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0354] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0355] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0356] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0357] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0358] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0359] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0360] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0361] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0362] The present invention is a system for providing real-time answers to customer questions, and is primarily implemented through a series of processes including question submission, analysis, database querying, answer generation, and answer submission and display.

[0363] Submit a question

[0364] When a user enters a question on their device and clicks the send button, the device sends the question to the server. The device generates an API request, packages the question and associated information (user ID, timestamp, etc.), and sends it to the server.

[0365] Receiving and analyzing questions

[0366] The server receives user queries through the API endpoint. The server analyzes the received queries using a natural language processing engine to identify keywords and user intent. This analysis enables the server to query the appropriate database.

[0367] Database query and response generation

[0368] Based on the analysis results, the server queries the relevant database. For example, if the question "Is this in stock?" is analyzed, the server connects to the database to retrieve the inventory information for the relevant product and uses an SQL query to obtain the inventory information.

[0369] The server generates a response based on the acquired data. The generated response may use a predefined phrase or it may be dynamically constructed. For example, the response "We have it in stock" might be generated.

[0370] Submit and display of responses

[0371] The generated response is sent from the server to the user's device. The device receives this response and displays it in the user interface. The user can then view the answer to the question on the device screen.

[0372] Specific example

[0373] For example, if a user enters the question "Is this item in stock?" and presses the submit button, the device sends the question to the server. The server receives the question and uses natural language processing to analyze the keyword "stock." Next, the server connects to the database and queries for the stock information of that item. If the database returns "In stock," the server uses this information to generate the answer "This item is in stock" and sends it to the user's device. The device displays the received answer on the screen to inform the user.

[0374] This system allows users to get quick and accurate answers to their questions in real time, significantly improving customer service.

[0375] The following describes the processing flow.

[0376] Step 1:

[0377] The user enters their question from their device. The user enters their question into the input form and clicks the submit button. Example: "Is this product in stock?"

[0378] Step 2:

[0379] The device sends the question to the server. The device generates an API request and sends it to the server along with the entered question, user ID, timestamp, and other associated information.

[0380] Step 3:

[0381] The server receives the question. The server's API endpoint receives the request and extracts the question and associated information.

[0382] Step 4:

[0383] The server analyzes the question. The server's natural language processing engine analyzes the question and identifies keywords and intent. Example: Extract the keyword "inventory" from the question.

[0384] Step 5:

[0385] The server queries the database. Based on the identified keywords, the server connects to the database and executes an SQL query to retrieve the relevant information.

[0386] Step 6:

[0387] The server generates the response. Based on information retrieved from the database, the server dynamically generates a response for the user. For example, it might generate the response "In stock."

[0388] Step 7:

[0389] The server sends the answer to the terminal. The response containing the generated answer is sent to the terminal.

[0390] Step 8:

[0391] The device receives the response. The device parses the received response and formats the content to be displayed.

[0392] Step 9:

[0393] The device displays the answer in the user interface. The user can confirm the answer on the device screen. For example, they might see a message like, "This item is in stock!"

[0394] (Example 1)

[0395] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0396] Traditional systems often failed to provide quick and accurate answers to customer inquiries, posing a challenge to improving customer service. Furthermore, the mechanisms for properly analyzing questions entered in natural language, acquiring necessary data, and generating accurate responses were insufficient. As a result, user experience deteriorated, and business efficiency was sometimes compromised.

[0397] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0398] In this invention, the server includes means for transmitting a question entered by a user from a terminal, means for analyzing the question using a natural language processing engine, and means for querying a database and generating an answer based on the analysis results. This makes it possible to receive questions from customers in real time and provide quick and accurate answers.

[0399] A "terminal" refers to a device used by a user for input, and includes, for example, smartphones, personal computers, and tablets.

[0400] A "server" refers to a computer system that receives data sent by users and performs analysis and processing on it.

[0401] A "natural language processing engine" refers to a software component that analyzes human language and understands its meaning and intent.

[0402] An "API request" refers to a communication request from a device to a server to request operations or data.

[0403] A "database" refers to a system that manages organized collections of data, and an example of this is a relational database management system (RDBMS).

[0404] "Analysis results" refer to the information obtained by a natural language processing engine through its analysis of questions and input data.

[0405] "Answer" refers to the response that the server generates based on the user's question.

[0406] "User interface" refers to the screen and software portion of a device that a user uses to perform operations and input.

[0407] "JSON format" refers to a text-based data format used to structure and represent data information.

[0408] This invention is a system that provides real-time answers to customer questions and is implemented through a main processing flow. The system begins with the user inputting and sending a question from a terminal. The terminal generates the input question as an API request and sends it to the server. The API request includes information such as the question content, user ID, and timestamp.

[0409] The server receives requests through the specified API endpoint and uses a natural language processing engine to analyze the question content. Commonly used natural language processing engines such as the Google Cloud Natural Language API or OpenAI's generative AI models can be used. This allows the server to identify keywords and user intent from the question.

[0410] Based on the analysis results, the server connects to the relevant database and retrieves the necessary information. This involves issuing SQL queries using a relational database management system (RDBMS) such as MySQL or PostgreSQL. For example, if the question "Is this in stock?" is analyzed, a database query is made to retrieve the inventory information for the relevant product. Based on the retrieved information, the server generates an answer either dynamically or using a predefined template.

[0411] The generated responses are packaged again in API request format and sent to the user's device. The device parses the received responses and displays them in the user interface. The user interface can be a web page or mobile application built using HTML, CSS, JavaScript, etc.

[0412] For example, if a user enters the question "Is this item in stock?" and presses the submit button, the device sends the question to the server. The server receives the question, analyzes it using a natural language processing engine, and identifies the keyword "stock." Next, the server connects to a database and queries for stock information about the item. If the database returns "in stock," the server generates the answer "This item is in stock" and sends it to the user's device. The device then displays the received answer on the user interface to inform the user.

[0413] Examples of prompt statements include the following:

[0414] "Do you have this item in stock?"

[0415] "Could you tell me your business hours?"

[0416] "What's the latest promotion?"

[0417] This system allows users to receive quick and accurate answers to their questions in real time, thereby improving customer service.

[0418] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0419] Step 1:

[0420] The user enters a question from their device and clicks the submit button. This input includes information such as the question, user ID, and timestamp. When the user enters "Is this product in stock?" and clicks the submit button, the device retrieves the input and generates an API request in JSON format. The API request is then sent to the server as output.

[0421] Step 2:

[0422] The server processes requests received at the API endpoint. The input here is the API request sent from the terminal. The server first parses the JSON data to extract information such as the question content, user ID, and timestamp. Based on this extracted information, it prepares data to pass to the natural language processing engine, and generates data for analysis as output.

[0423] Step 3:

[0424] The server analyzes the question using a natural language processing engine. The input is the data prepared in step 2. The server uses a natural language processing engine, such as the Google Cloud Natural Language API or OpenAI's generative AI model, to identify keywords and intent from the question. If the keyword "inventory" is identified through this analysis, the analysis result is obtained as output.

[0425] Step 4:

[0426] The server queries the database based on the analysis results. The input here is the analysis result of natural language processing, which contains information that includes the identified keywords. The server connects to a relational database management system (RDBMS) such as MySQL or PostgreSQL, generates an SQL query, and sends it. For example, a query like "SELECT in_stock FROM products WHERE product_id = 98765" is issued. The output is inventory information returned from the database.

[0427] Step 5:

[0428] The server generates a response based on information retrieved from the database. The input is inventory information retrieved from the database. Based on the retrieved data, the server generates a response such as "This item is in stock" if the item is in stock. The output is the response text for the user.

[0429] Step 6:

[0430] The server generates a response and sends it to the device. The input is the response text generated by the server. The server packages the response in JSON format, generates another API request, and sends it to the device. The output is the API request sent to the device.

[0431] Step 7:

[0432] The terminal displays the received response on the user interface. The input is response data in JSON format received from the server. The terminal parses this data and formats it for display on the user interface. For example, HTML and JavaScript are used to display the response on the screen. As output, the user can see the answer to the question on the screen.

[0433] Through these steps, users can obtain quick and accurate answers to their questions in real time.

[0434] (Application Example 1)

[0435] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0436] Currently, when customers in physical stores want to know product information or inventory information, they typically ask store staff directly. However, it is difficult to get a quick response during busy periods or when staff are engaged in other tasks. This invention aims to solve this problem and provide a system that enables customers to obtain information quickly and accurately.

[0437] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0438] In this invention, the server includes means for transmitting a question entered by a user from a mobile device to the server; means for analyzing the received question using a natural language processing system; means for querying an information storage device based on the analysis results and generating an answer; means for transmitting the generated answer to the user's mobile device; means for displaying the answer received by the mobile device on a user interface; input means for enabling the user to input questions using voice or text; and means for receiving and displaying answers to questions in real time via software installed on the mobile device. This enables customers to quickly and accurately obtain product and inventory information even in physical stores.

[0439] A "personal information terminal" is a portable device such as a smartphone or smart glasses that a user uses to input questions and receive and display answers.

[0440] A "natural language processing system" is an artificial intelligence technology that analyzes questions received from users to identify their intent and keywords.

[0441] "Information storage device" refers to a database or storage device that stores data for generating answers to questions.

[0442] An "input method" is a function that allows users to input questions using voice or text.

[0443] A "user interface" is a screen or display area on a mobile device that allows the user to visually confirm the answers they have generated.

[0444] "Software" refers to a program installed on a mobile device that controls the sending of questions and the receiving and display of answers.

[0445] "Real-time" refers to responding quickly to user questions and providing answers without delay.

[0446] This invention provides a system that allows customers to quickly obtain product and inventory information in physical stores. This system includes a mobile information terminal, a server, a natural language processing system, an information storage device, input means, and a user interface.

[0447] System Configuration

[0448] 1. Mobile device:

[0449] In this system, a personal digital assistant (PDA) is a device used by the user to input questions and receive and display answers. Specifically, this includes smartphones and smart glasses.

[0450] 2. Server:

[0451] The server receives a question from the user and analyzes it using a natural language processing system. It then queries its information storage device based on the analysis results and generates an answer. Finally, it sends the generated answer to the user's mobile device.

[0452] 3. Natural Language Processing Systems:

[0453] The natural language processing system used on the server analyzes received questions to identify keywords and user intent. This analysis enables appropriate database queries. Specific software used includes Spacy, among others.

[0454] 4. Information storage:

[0455] A database or storage device that holds data for generating answers to questions. For example, it might store product inventory information or detailed information.

[0456] 5. Input method:

[0457] A feature that allows users to input questions using voice or text. This includes voice input devices and text input interfaces.

[0458] 6. User Interface:

[0459] This refers to a screen or display area on a mobile device that allows users to visually confirm the answers they have generated. This enables users to instantly obtain the necessary information within the store.

[0460] System operation

[0461] When a user inputs a question via voice or text using a mobile device, the question is sent from the device to the server. The server receives the question and analyzes it using a natural language processing system. Based on the keywords in the analyzed question, the server queries its information storage device and generates an appropriate answer. The generated answer is sent from the server to the mobile device and displayed on the device's user interface.

[0462] Specific example

[0463] For example, if a user asks their smart glasses, "Do you have this item in stock?", the question is sent to the server. The server analyzes the question and identifies the keyword "stock". It then connects to a data storage device and retrieves the stock information for that item. If the database returns "in stock", the server generates the response "This item is in stock" and displays it on the smart glasses.

[0464] Example of a prompt

[0465] Question: "Do you have this item in stock?"

[0466] Answer: "This item is in stock."

[0467] This allows customers to easily obtain product information in physical stores, resulting in an efficient shopping experience.

[0468] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0469] Step 1:

[0470] The user enters the question using voice or text via a mobile device.

[0471] Input: User's question (e.g., "Do you have this item in stock?")

[0472] Output: Question data (audio data or text data)

[0473] Operation: Questions are entered using the input function of a smartphone or smart glasses. The user interface utilizes voice recognition and text input.

[0474] Step 2:

[0475] The terminal sends the question data to the server.

[0476] Input: Question data

[0477] Output: API Request

[0478] Operation: The terminal packages the question data, generates an API request, and sends it to the server. Specifically, the request includes information such as the question text, user ID, and timestamp.

[0479] Step 3:

[0480] The server receives the question data through the API endpoint.

[0481] Input: API Request

[0482] Output: Question data (received on the server side)

[0483] Operation: The server receives question data via the API endpoint and places it in a queue for preparation for analysis.

[0484] Step 4:

[0485] The server uses a natural language processing system to analyze the question.

[0486] Input: Question data

[0487] Output: Analysis results (keywords and intent)

[0488] Operation: Uses a natural language processing system (e.g., Spacy) to analyze the received question text and identify keywords and user intent. For example, it extracts nouns such as "inventory" and verbs such as "do you have any?".

[0489] Step 5:

[0490] The server queries the information storage device based on the analysis results.

[0491] Input: Analysis results (keywords and intent)

[0492] Output: Database query results (e.g., inventory information)

[0493] Operation: Based on the analysis results, it generates an appropriate SQL query and queries the information storage device (database). It retrieves inventory information and related data for the relevant product from the database.

[0494] Step 6:

[0495] The server generates an answer based on the data it has acquired.

[0496] Input: Database query results (e.g., inventory information)

[0497] Output: Generated response (Example: "This item is in stock.")

[0498] Function: Processes query results and generates appropriate canned responses or dynamically constructed answers. For example, from the data "In stock," it creates a response such as "This product is in stock."

[0499] Step 7:

[0500] The server sends the generated response to the terminal.

[0501] Input: Generated answer

[0502] Output: API response (response data)

[0503] Operation: The server packages the generated response and sends it to the terminal as an API response.

[0504] Step 8:

[0505] The device displays the received response on the user interface.

[0506] Input: API response (response data)

[0507] Output: Displayed answer

[0508] Operation: The device receives a response from the server and displays it on the user interface. For example, "This item is in stock" might be displayed on the screen of smart glasses or a smartphone.

[0509] Through these steps, users can quickly obtain product and inventory information even in physical stores.

[0510] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0511] The present invention is a system for providing real-time answers to customer questions, and is particularly characterized by its ability to recognize the user's emotions in conjunction with the system by combining it with an emotion engine, thereby providing more appropriate answers. Specific embodiments of the system of the present invention are shown below.

[0512] Question submission and sentiment recognition

[0513] The user enters a question on the device and clicks the send button. The device prepares to send the entered question to the server and activates an emotion engine to recognize the user's emotions from the question. The emotion engine analyzes the text of the question and generates emotion parameters such as joy, anger, and sadness.

[0514] Reception and analysis on the server

[0515] The device sends an API request to the server, which includes the question, user ID, timestamp, and sentiment parameter. The server receives the request through the API endpoint and extracts the question and sentiment parameter.

[0516] Question analysis and database queries

[0517] The server's natural language processing engine analyzes the question, identifying keywords and intent. Sentiment parameters are also considered and used as additional information to generate an appropriate response. Based on the analyzed keywords, the server queries the database. For example, even if the question is "out of stock," data is retrieved to provide a gentle notification that takes the user's emotions into consideration.

[0518] Generating an answer

[0519] Based on information retrieved from the database and emotion parameters, the server generates an appropriate response. Based on the output of the emotion engine, the response is adjusted to be considerate of the user's emotions. For example, a response that is considerate of the user's emotions might be created, such as, "We are sorry, but this item is currently out of stock. We will notify you as soon as it becomes available."

[0520] Submit and display of responses

[0521] The server sends the generated response to the user's device. The device receives the response and displays it in the user interface. The user can see the specific answer to the question, along with a message that takes their feelings into consideration, on the device screen.

[0522] Specific example

[0523] For example, a user enters the question, "Do you have this item in stock? I'm tired of waiting," and presses the submit button. The device sends the question and emotion parameter (in this case, the emotion "tired") to the server. The server receives the question and analyzes it using a natural language processing engine and an emotion engine, extracting the keyword "stock" and the emotion "tired." Next, the server queries the database for stock information and, if the item is out of stock, generates an emotion-sensitive response. Finally, the device displays a message to the user saying, "We're sorry, but this item is currently out of stock. We will notify you as soon as it becomes available."

[0524] This system allows users to receive quick and accurate answers to their questions in real time, as well as emotionally sensitive service, significantly improving customer satisfaction.

[0525] The following describes the processing flow.

[0526] Step 1:

[0527] The user enters a question from their device. The user enters the question into the input form and clicks the submit button. Example: Enter "Is this product in stock?"

[0528] Step 2:

[0529] The device sends the question to the server. The device generates an API request, packages the entered question with accompanying information such as the user ID and timestamp, and sends it to the server.

[0530] Step 3:

[0531] The device activates its emotion engine. Based on the entered questions, the device uses the emotion engine to analyze the user's emotions. For example, it extracts emotions such as "anger" or "dissatisfaction" from text.

[0532] Step 4:

[0533] The device sends emotion parameters to the server. In addition to the question, the analyzed emotion parameters are also included in the API request and sent to the server.

[0534] Step 5:

[0535] The server receives the question and sentiment parameters. The server's API endpoint receives the request and extracts the question and sentiment parameters.

[0536] Step 6:

[0537] The server analyzes the question. A natural language processing engine built into the server analyzes the question, identifying key keywords and user intent. For example, it identifies the keyword "inventory."

[0538] Step 7:

[0539] The server analyzes the emotional parameters. The server analyzes the emotional parameters received from the emotion engine to understand the user's emotional state. Example: It determines that the user is "highly dissatisfied."

[0540] Step 8:

[0541] The server queries the database. Based on the specified keywords, the server connects to the relevant database and executes an SQL query to retrieve the necessary information. Example: Retrieving product inventory information.

[0542] Step 9:

[0543] The server generates the response. Based on information retrieved from the database and sentiment parameters, the server generates an appropriate response. It constructs a message that takes the user's feelings into consideration, taking sentiment parameters into account. Example: It generates "We are sorry, but this item is currently out of stock. We will notify you as soon as it becomes available."

[0544] Step 10:

[0545] The server sends the generated response to the device. The API response containing the generated response is then sent to the user's device.

[0546] Step 11:

[0547] The device receives the response. The device receives the API response, parses the content of the response, and formats it into data for display.

[0548] Step 12:

[0549] The device displays the answer in the user interface. The user can see a specific answer to the question and an emotionally sensitive message on the device screen. For example, it might display: "We are sorry, but this item is currently out of stock. We will notify you as soon as it becomes available."

[0550] Through this series of processes, users can not only receive real-time answers to their questions, but also receive responses that take their emotional state at the time into consideration. This significantly improves customer satisfaction.

[0551] (Example 2)

[0552] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0553] Conventional question-answering systems failed to consider the user's emotions when they entered and submitted a question. This resulted in the inability to generate appropriate answers that took the user's feelings into account, leading to decreased customer satisfaction. Furthermore, the system lacked the means to provide answers that reflected the user's emotions, resulting in inefficient communication with users.

[0554] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0555] In this invention, the server includes means for sending a question entered by the user from a terminal to the server, means for the terminal to analyze the content of the question and generate sentiment parameters, means for the server to analyze the received question using a natural language processing engine, means for the server to query a database based on the analysis results and sentiment parameters and generate an answer, means for the server to send the generated answer to the user's terminal, and means for the terminal to display the received answer on a user interface. This makes it possible to provide answers that take the user's emotions into consideration in real time, thereby improving customer satisfaction and enabling efficient communication.

[0556] A "user" refers to an individual or organization that operates the system and enters questions.

[0557] "Terminal" refers to electronic devices, including input / output devices such as computers and smartphones used by users.

[0558] A "question" refers to the content of an inquiry that a user enters into the system via their device.

[0559] A "server" refers to a central processing unit that receives, analyzes, processes, and responds to user inquiries.

[0560] "Transmission method" refers to the function for sending user questions from the terminal to the server.

[0561] "Emotional parameters" refer to data that indicates the emotional state (e.g., joy, anger, sadness, etc.) analyzed from the user's questions.

[0562] A "natural language processing engine" refers to software or programs that a server uses to analyze a question and identify keywords and intent.

[0563] A "database" refers to an information storage system that stores answers to questions and manages them so that they can be searched when needed.

[0564] "Query method" refers to the function that allows a server to instruct a database to search for information related to the query.

[0565] "Generation means" refers to the function that allows the server to create an appropriate response based on information and sentiment parameters obtained from the database.

[0566] "Display means" refers to the function that allows the terminal to display the response received from the server on the user interface.

[0567] The system of this invention is designed to provide emotionally appropriate and real-time responses to questions sent by users from their terminals. The system of this invention is implemented by combining the following hardware and software.

[0568] Entering and submitting questions via a terminal.

[0569] The user accesses the system from a device (such as a computer or smartphone) and enters a question. For example, the user enters the following prompt:

[0570] Do you have this item in stock? I'm tired of waiting.

[0571] When the user clicks the "Submit" button, the device receives this question and proceeds to the next step.

[0572] Emotion recognition by devices

[0573] The device uses a cloud-based natural language processing API to analyze the question and recognize the user's emotions. Specifically, it uses the Google Cloud Natural Language API to analyze the question and extract emotional parameters such as joy, anger, and sadness.

[0574] The emotion parameters obtained through this process (e.g., the emotion "tired") are sent to the server in JSON format along with the question data.

[0575] Reception and analysis on the server

[0576] The server receives requests sent from the terminal through an API endpoint built using the Spring Boot framework. The received JSON data includes the question content, user ID, timestamp, and sentiment parameter. The server extracts this data and uses it for the next processing step.

[0577] Question analysis and database queries

[0578] The server analyzes the question using the Hugging Face Transformers library. A natural language processing engine identifies the main keywords of the question and the user's intent. Sentiment parameters are also taken into consideration.

[0579] Based on the analyzed keywords, the server queries the MySQL database. For example, it generates an SQL query to retrieve inventory information and obtains the necessary information from the database.

[0580] Generating an answer

[0581] The server generates an appropriate response based on the retrieved database information and emotion parameters. During this process, the response is adjusted based on the emotion engine's output to ensure it reflects the user's emotions. For example, if an item is out of stock, a comment such as "We apologize, but this item is currently out of stock. We will notify you as soon as it becomes available" is generated.

[0582] Submit and display of responses

[0583] The server sends the generated response to the device in JSON format. The device receives the response asynchronously using JavaScript (AJAX) and displays it in the user interface (for example, a web application using React.js). The user can then see an appropriate and emotionally sensitive response to their question on the device screen.

[0584] In this way, the system of the present invention allows users to obtain quick and accurate answers in real time, and furthermore, these answers take into consideration the user's feelings. As a result, customer satisfaction is significantly improved.

[0585] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0586] Step 1:

[0587] The user enters and submits the question from their terminal. The user enters the question on the system interface and clicks the submit button. For example, the user might enter the following prompt:

[0588] Do you have this item in stock? I'm tired of waiting.

[0589] After this input is received, the terminal retains the question data and proceeds to the next processing step.

[0590] Step 2:

[0591] The device prepares to send a question to the server. Simultaneously, it starts sentiment recognition using the Google Cloud Natural Language API. The device analyzes the question and extracts sentiment parameters. For example, the sentiment "tired" might be recognized. Based on this analysis, the device converts the question and sentiment parameters into JSON format and constructs the data to send to the server. Input: User's question, Output: JSON data with sentiment parameters.

[0592] Step 3:

[0593] The device sends JSON data it has constructed to the server using a RESTful API. Input: JSON data with sentiment parameters; Output: Request sent to the server. This request includes the user ID, question content, timestamp, and sentiment parameters.

[0594] Step 4:

[0595] The server receives the request. It uses a Spring Boot-based API endpoint to receive the request and parse the data. Input: Request sent to the server; Output: Extracted question content, user ID, timestamp, and sentiment parameter.

[0596] Step 5:

[0597] The server analyzes the extracted question content. Using the Hugging Face Transformers library, key keywords and user intent are identified. For example, the keyword "inventory" might be extracted. Sentiment parameters are also included in the processing. Input: Question content; Output: Identified keywords and sentiment parameters.

[0598] Step 6:

[0599] The server queries the database based on the question. It executes an SQL query against the MySQL database to retrieve relevant information. For example, an SQL query to check inventory information is executed. Input: Identified keyword, Output: Inventory information retrieved from the database.

[0600] Step 7:

[0601] The server generates an appropriate response based on information retrieved from the database and emotion parameters. Based on the output of the emotion engine, the response is adjusted to be sensitive to the user's emotions. For example, a response such as "We are sorry, but we are currently out of stock. We will notify you as soon as it becomes available" might be generated. Input: Stock information and emotion parameters, Output: Emotionally sensitive response.

[0602] Step 8:

[0603] The server generates the response and sends it to the terminal in JSON format. Input: Generated response, Output: Response sent to the user's terminal.

[0604] Step 9:

[0605] The device displays the response received from the server on the user interface. For example, responses are received asynchronously using JavaScript (AJAX) and displayed using React.js or similar. The user can see a specific answer to the question, along with a message that takes their emotions into consideration, on the device screen. Input: Response sent from the server; Output: Response displayed on the user interface.

[0606] (Application Example 2)

[0607] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0608] Conventional real-time question answering systems struggle to provide answers that take user emotions into consideration, resulting in decreased user satisfaction. In particular, answers that ignore the emotions embedded in the user's questions degrade the quality of the customer experience. Furthermore, from a sales promotion perspective, providing services that are sensitive to emotions is essential.

[0609] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0610] In this invention, the server includes means for sending a question entered by the user from a terminal to the server, means for activating an emotion engine for recognizing the user's emotions from the question entered by the terminal, means for analyzing the received question using a natural language processing engine, means for querying a database based on the analysis results and generating an answer, means for adjusting the generated answer based on emotion parameters, means for sending the generated answer to the user's terminal, and means for displaying the received answer on a user interface. This enables real-time question answering that takes the user's emotions into consideration.

[0611] A "user" refers to a person who uses the system to input questions.

[0612] A "device" refers to a device used by a user to input questions and receive answers. Examples include smartphones and tablets.

[0613] A "question" refers to the content of an inquiry that a user enters from their device and sends to the system.

[0614] A "server" refers to a central processing unit that analyzes received questions, generates answers, and sends them to terminals.

[0615] A "natural language processing engine" refers to a processing unit that analyzes input questions and understands their meaning and intent.

[0616] An "emotion engine" refers to a device that generates emotional parameters from questions entered by the user.

[0617] "Emotional parameters" refer to data generated by the emotion engine that indicates the user's emotional state.

[0618] A "database" refers to an information storage system that stores the information necessary to generate answers to questions.

[0619] "Answer" refers to the response to an inquiry that the server generates and provides to the user.

[0620] "User interface" refers to the means of displaying answers visually on a device.

[0621] System Configuration

[0622] A system for carrying out this invention includes the following main components:

[0623] User's terminal

[0624] server

[0625] Natural Language Processing Engine

[0626] Emotional Engine

[0627] database

[0628] Program execution

[0629] 1. Send a question from the user's device.

[0630] The user enters a question on their device and presses the send button. This action activates an emotion engine that recognizes the user's emotions along with the question content.

[0631] 2. Emotion recognition by an emotion engine

[0632] The emotion engine analyzes the question text and generates the user's emotion parameters (e.g., joy, anger, sadness). This process uses, for example, the "sentiment-analysis" model from Transformer.

[0633] 3. Sending data to the server

[0634] The device sends an API request to the server that includes the question content, user ID, and sentiment parameters. The request is sent in JSON format.

[0635] 4. Receiving and analyzing data on the server

[0636] The server receives requests at the API endpoint and extracts the question and sentiment parameters. Next, it analyzes the question using a natural language processing engine (e.g., SpaCy or BERT) to identify keywords and intent.

[0637] 5. Retrieving information from the database

[0638] Based on the analyzed keywords, the server queries the database. For example, if the keyword "inventory" is included, it retrieves inventory information from the database.

[0639] 6. Generating responses and adjusting them with consideration for emotions.

[0640] Based on information retrieved from the database and emotion parameters, the server generates an appropriate response. A generative AI model (e.g., GPT-3) is used to refine the response to be more emotionally conscious.

[0641] 7. Submitting and displaying responses

[0642] The server sends the generated response to the user's terminal. The terminal formats the received response and displays it on the user interface.

[0643] Specific example

[0644] For example, suppose a user enters the question, "Do you have this item in stock? I'm tired of waiting," and presses the submit button. The emotion engine recognizes the emotion "tired." An API request containing the question and emotion parameters is sent to the server, which analyzes the keyword "stock" and the emotion "tired." Next, it queries the database for stock information, and if it does not have the item in stock, it generates an emotion-sensitive response such as, "We're sorry, but we don't have this item in stock at the moment. We will notify you as soon as it becomes available." This response is then sent to the device and displayed on the user interface.

[0645] Example prompt

[0646] The following is an example of a prompt used with the emotion engine:

[0647] "Do you have this item in stock? I'm tired of waiting."

[0648] By allowing users to input specific questions in this way, the system can provide quick and emotionally sensitive answers. This entire process improves customer satisfaction and delivers a better user experience.

[0649] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0650] Step 1:

[0651] The user enters a question from their device and presses the send button. The user might enter a specific question, such as, "Do you have this item in stock? I'm tired of waiting." The entered question is temporarily stored as text data on the device.

[0652] Step 2:

[0653] The device passes the entered question to the sentiment engine to begin analysis. The sentiment engine analyzes the question text using, for example, Transformer's "sentiment-analysis" model, and generates the user's sentiment parameters (e.g., joy, anger, sadness). The generated sentiment parameters are represented as a label such as "tired" and its confidence score (e.g., 0.95).

[0654] Step 3:

[0655] The device sends an API request to the server that includes the question content and sentiment parameters. This request includes the question text, user ID, sentiment label, and sentiment score.

[0656] Step 4:

[0657] The server receives the request at the API endpoint and extracts the question, user ID, sentiment label, and sentiment score. The extracted data is then split into its respective fields and passed on to subsequent processing.

[0658] Step 5:

[0659] The server analyzes the question text using a natural language processing engine. For example, it uses SpaCy or BERT to identify keywords and intent in the question. From the question text "Do you have this item in stock? I'm tired of waiting," keywords such as "stock" and "tired" are extracted.

[0660] Step 6:

[0661] Based on the analyzed keywords, the server queries the database for relevant information. For example, a database query containing the keyword "inventory" is generated, and inventory information for the corresponding product is retrieved.

[0662] Step 7:

[0663] The server generates appropriate responses based on information retrieved from the database and sentiment parameters. In this process, a generative AI model (e.g., GPT-3) is used to generate responses that take into account the intent of the question and the user's emotions. For example, a response such as, "We apologize, but this item is currently out of stock. We will notify you as soon as it becomes available," might be generated.

[0664] Step 8:

[0665] The server sends the generated response to the user's device. The data sent includes the text of the response.

[0666] Step 9:

[0667] The device formats the received response and displays it on the user interface. Specifically, the formatted text will read, "We apologize, but this item is currently out of stock. We will notify you as soon as it becomes available."

[0668] This processing flow allows users to receive quick, emotionally sensitive answers to their questions in real time.

[0669] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0670] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0671] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0672] [Third Embodiment]

[0673] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0674] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0675] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0676] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0677] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0678] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0679] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0680] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0681] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0682] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0683] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0684] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0685] The present invention is a system for providing real-time answers to customer questions, and is primarily implemented through a series of processes including question submission, analysis, database querying, answer generation, and answer submission and display.

[0686] Submit a question

[0687] When a user enters a question on their device and clicks the send button, the device sends the question to the server. The device generates an API request, packages the question and associated information (user ID, timestamp, etc.), and sends it to the server.

[0688] Receiving and analyzing questions

[0689] The server receives user queries through the API endpoint. The server analyzes the received queries using a natural language processing engine to identify keywords and user intent. This analysis enables the server to query the appropriate database.

[0690] Database query and response generation

[0691] Based on the analysis results, the server queries the relevant database. For example, if the question "Is this in stock?" is analyzed, the server connects to the database to retrieve the inventory information for the relevant product and uses an SQL query to obtain the inventory information.

[0692] The server generates a response based on the acquired data. The generated response may use a predefined phrase or it may be dynamically constructed. For example, the response "We have it in stock" might be generated.

[0693] Submit and display of responses

[0694] The generated response is sent from the server to the user's device. The device receives this response and displays it in the user interface. The user can then view the answer to the question on the device screen.

[0695] Specific example

[0696] For example, if a user enters the question "Is this item in stock?" and presses the submit button, the device sends the question to the server. The server receives the question and uses natural language processing to analyze the keyword "stock." Next, the server connects to the database and queries for the stock information of that item. If the database returns "In stock," the server uses this information to generate the answer "This item is in stock" and sends it to the user's device. The device displays the received answer on the screen to inform the user.

[0697] This system allows users to get quick and accurate answers to their questions in real time, significantly improving customer service.

[0698] The following describes the processing flow.

[0699] Step 1:

[0700] The user enters their question from their device. The user enters their question into the input form and clicks the submit button. Example: "Is this product in stock?"

[0701] Step 2:

[0702] The device sends the question to the server. The device generates an API request and sends it to the server along with the entered question, user ID, timestamp, and other associated information.

[0703] Step 3:

[0704] The server receives the question. The server's API endpoint receives the request and extracts the question and associated information.

[0705] Step 4:

[0706] The server analyzes the question. The server's natural language processing engine analyzes the question and identifies keywords and intent. Example: Extract the keyword "inventory" from the question.

[0707] Step 5:

[0708] The server queries the database. Based on the identified keywords, the server connects to the database and executes an SQL query to retrieve the relevant information.

[0709] Step 6:

[0710] The server generates the response. Based on information retrieved from the database, the server dynamically generates a response for the user. For example, it might generate the response "In stock."

[0711] Step 7:

[0712] The server sends the answer to the terminal. The response containing the generated answer is sent to the terminal.

[0713] Step 8:

[0714] The device receives the response. The device parses the received response and formats the content to be displayed.

[0715] Step 9:

[0716] The device displays the answer in the user interface. The user can confirm the answer on the device screen. For example, they might see a message like, "This item is in stock!"

[0717] (Example 1)

[0718] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0719] Traditional systems often failed to provide quick and accurate answers to customer inquiries, posing a challenge to improving customer service. Furthermore, the mechanisms for properly analyzing questions entered in natural language, acquiring necessary data, and generating accurate responses were insufficient. As a result, user experience deteriorated, and business efficiency was sometimes compromised.

[0720] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0721] In this invention, the server includes means for transmitting a question entered by a user from a terminal, means for analyzing the question using a natural language processing engine, and means for querying a database and generating an answer based on the analysis results. This makes it possible to receive questions from customers in real time and provide quick and accurate answers.

[0722] A "terminal" refers to a device used by a user for input, and includes, for example, smartphones, personal computers, and tablets.

[0723] A "server" refers to a computer system that receives data sent by users and performs analysis and processing on it.

[0724] A "natural language processing engine" refers to a software component that analyzes human language and understands its meaning and intent.

[0725] An "API request" refers to a communication request from a device to a server to request operations or data.

[0726] A "database" refers to a system that manages organized collections of data, and an example of this is a relational database management system (RDBMS).

[0727] "Analysis results" refer to the information obtained by a natural language processing engine through its analysis of questions and input data.

[0728] "Answer" refers to the response that the server generates based on the user's question.

[0729] "User interface" refers to the screen and software portion of a device that a user uses to perform operations and input.

[0730] "JSON format" refers to a text-based data format used to structure and represent data information.

[0731] This invention is a system that provides real-time answers to customer questions and is implemented through a main processing flow. The system begins with the user inputting and sending a question from a terminal. The terminal generates the input question as an API request and sends it to the server. The API request includes information such as the question content, user ID, and timestamp.

[0732] The server receives requests through the specified API endpoint and uses a natural language processing engine to analyze the question content. Commonly used natural language processing engines such as the Google Cloud Natural Language API or OpenAI's generative AI models can be used. This allows the server to identify keywords and user intent from the question.

[0733] Based on the analysis results, the server connects to the relevant database and retrieves the necessary information. This involves issuing SQL queries using a relational database management system (RDBMS) such as MySQL or PostgreSQL. For example, if the question "Is this in stock?" is analyzed, a database query is made to retrieve the inventory information for the relevant product. Based on the retrieved information, the server generates an answer either dynamically or using a predefined template.

[0734] The generated responses are packaged again in API request format and sent to the user's device. The device parses the received responses and displays them in the user interface. The user interface can be a web page or mobile application built using HTML, CSS, JavaScript, etc.

[0735] For example, if a user enters the question "Is this item in stock?" and presses the submit button, the device sends the question to the server. The server receives the question, analyzes it using a natural language processing engine, and identifies the keyword "stock." Next, the server connects to a database and queries for stock information about the item. If the database returns "in stock," the server generates the answer "This item is in stock" and sends it to the user's device. The device then displays the received answer on the user interface to inform the user.

[0736] Examples of prompt statements include the following:

[0737] "Do you have this item in stock?"

[0738] "Could you tell me your business hours?"

[0739] "What's the latest promotion?"

[0740] This system allows users to receive quick and accurate answers to their questions in real time, thereby improving customer service.

[0741] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0742] Step 1:

[0743] The user enters a question from their device and clicks the submit button. This input includes information such as the question, user ID, and timestamp. When the user enters "Is this product in stock?" and clicks the submit button, the device retrieves the input and generates an API request in JSON format. The API request is then sent to the server as output.

[0744] Step 2:

[0745] The server processes requests received at the API endpoint. The input here is the API request sent from the terminal. The server first parses the JSON data to extract information such as the question content, user ID, and timestamp. Based on this extracted information, it prepares data to pass to the natural language processing engine, and generates data for analysis as output.

[0746] Step 3:

[0747] The server analyzes the question using a natural language processing engine. The input is the data prepared in step 2. The server uses a natural language processing engine, such as the Google Cloud Natural Language API or OpenAI's generative AI model, to identify keywords and intent from the question. If the keyword "inventory" is identified through this analysis, the analysis result is obtained as output.

[0748] Step 4:

[0749] The server queries the database based on the analysis results. The input here is the analysis result of natural language processing, which contains information that includes the identified keywords. The server connects to a relational database management system (RDBMS) such as MySQL or PostgreSQL, generates an SQL query, and sends it. For example, a query like "SELECT in_stock FROM products WHERE product_id = 98765" is issued. The output is inventory information returned from the database.

[0750] Step 5:

[0751] The server generates a response based on information retrieved from the database. The input is inventory information retrieved from the database. Based on the retrieved data, the server generates a response such as "This item is in stock" if the item is in stock. The output is the response text for the user.

[0752] Step 6:

[0753] The server generates a response and sends it to the device. The input is the response text generated by the server. The server packages the response in JSON format, generates another API request, and sends it to the device. The output is the API request sent to the device.

[0754] Step 7:

[0755] The terminal displays the received response on the user interface. The input is response data in JSON format received from the server. The terminal parses this data and formats it for display on the user interface. For example, HTML and JavaScript are used to display the response on the screen. As output, the user can see the answer to the question on the screen.

[0756] Through these steps, users can obtain quick and accurate answers to their questions in real time.

[0757] (Application Example 1)

[0758] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0759] Currently, when customers in physical stores want to know product information or inventory information, they typically ask store staff directly. However, it is difficult to get a quick response during busy periods or when staff are engaged in other tasks. This invention aims to solve this problem and provide a system that enables customers to obtain information quickly and accurately.

[0760] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0761] In this invention, the server includes means for transmitting a question entered by a user from a mobile device to the server; means for analyzing the received question using a natural language processing system; means for querying an information storage device based on the analysis results and generating an answer; means for transmitting the generated answer to the user's mobile device; means for displaying the answer received by the mobile device on a user interface; input means for enabling the user to input questions using voice or text; and means for receiving and displaying answers to questions in real time via software installed on the mobile device. This enables customers to quickly and accurately obtain product and inventory information even in physical stores.

[0762] A "personal information terminal" is a portable device such as a smartphone or smart glasses that a user uses to input questions and receive and display answers.

[0763] A "natural language processing system" is an artificial intelligence technology that analyzes questions received from users to identify their intent and keywords.

[0764] "Information storage device" refers to a database or storage device that stores data for generating answers to questions.

[0765] An "input method" is a function that allows users to input questions using voice or text.

[0766] A "user interface" is a screen or display area on a mobile device that allows the user to visually confirm the answers they have generated.

[0767] "Software" refers to a program installed on a mobile device that controls the sending of questions and the receiving and display of answers.

[0768] "Real-time" refers to responding quickly to user questions and providing answers without delay.

[0769] This invention provides a system that allows customers to quickly obtain product and inventory information in physical stores. This system includes a mobile information terminal, a server, a natural language processing system, an information storage device, input means, and a user interface.

[0770] System Configuration

[0771] 1. Mobile device:

[0772] In this system, a personal digital assistant (PDA) is a device used by the user to input questions and receive and display answers. Specifically, this includes smartphones and smart glasses.

[0773] 2. Server:

[0774] The server receives a question from the user and analyzes it using a natural language processing system. It then queries its information storage device based on the analysis results and generates an answer. Finally, it sends the generated answer to the user's mobile device.

[0775] 3. Natural Language Processing Systems:

[0776] The natural language processing system used on the server analyzes received questions to identify keywords and user intent. This analysis enables appropriate database queries. Specific software used includes Spacy, among others.

[0777] 4. Information storage:

[0778] A database or storage device that holds data for generating answers to questions. For example, it might store product inventory information or detailed information.

[0779] 5. Input method:

[0780] A feature that allows users to input questions using voice or text. This includes voice input devices and text input interfaces.

[0781] 6. User Interface:

[0782] This refers to a screen or display area on a mobile device that allows users to visually confirm the answers they have generated. This enables users to instantly obtain the necessary information within the store.

[0783] System operation

[0784] When a user inputs a question via voice or text using a mobile device, the question is sent from the device to the server. The server receives the question and analyzes it using a natural language processing system. Based on the keywords in the analyzed question, the server queries its information storage device and generates an appropriate answer. The generated answer is sent from the server to the mobile device and displayed on the device's user interface.

[0785] Specific example

[0786] For example, if a user asks their smart glasses, "Do you have this item in stock?", the question is sent to the server. The server analyzes the question and identifies the keyword "stock". It then connects to a data storage device and retrieves the stock information for that item. If the database returns "in stock", the server generates the response "This item is in stock" and displays it on the smart glasses.

[0787] Example of a prompt

[0788] Question: "Do you have this item in stock?"

[0789] Answer: "This item is in stock."

[0790] This allows customers to easily obtain product information in physical stores, resulting in an efficient shopping experience.

[0791] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0792] Step 1:

[0793] The user enters the question using voice or text via a mobile device.

[0794] Input: User's question (e.g., "Do you have this item in stock?")

[0795] Output: Question data (audio data or text data)

[0796] Operation: Questions are entered using the input function of a smartphone or smart glasses. The user interface utilizes voice recognition and text input.

[0797] Step 2:

[0798] The terminal sends the question data to the server.

[0799] Input: Question data

[0800] Output: API Request

[0801] Operation: The terminal packages the question data, generates an API request, and sends it to the server. Specifically, the request includes information such as the question text, user ID, and timestamp.

[0802] Step 3:

[0803] The server receives the question data through the API endpoint.

[0804] Input: API Request

[0805] Output: Question data (received on the server side)

[0806] Operation: The server receives question data via the API endpoint and places it in a queue for preparation for analysis.

[0807] Step 4:

[0808] The server uses a natural language processing system to analyze the question.

[0809] Input: Question data

[0810] Output: Analysis results (keywords and intent)

[0811] Operation: Uses a natural language processing system (e.g., Spacy) to analyze the received question text and identify keywords and user intent. For example, it extracts nouns such as "inventory" and verbs such as "do you have any?".

[0812] Step 5:

[0813] The server queries the information storage device based on the analysis results.

[0814] Input: Analysis results (keywords and intent)

[0815] Output: Database query results (e.g., inventory information)

[0816] Operation: Based on the analysis results, it generates an appropriate SQL query and queries the information storage device (database). It retrieves inventory information and related data for the relevant product from the database.

[0817] Step 6:

[0818] The server generates an answer based on the data it has acquired.

[0819] Input: Database query results (e.g., inventory information)

[0820] Output: Generated response (Example: "This item is in stock.")

[0821] Function: Processes query results and generates appropriate canned responses or dynamically constructed answers. For example, from the data "In stock," it creates a response such as "This product is in stock."

[0822] Step 7:

[0823] The server sends the generated response to the terminal.

[0824] Input: Generated answer

[0825] Output: API response (response data)

[0826] Operation: The server packages the generated response and sends it to the terminal as an API response.

[0827] Step 8:

[0828] The device displays the received response on the user interface.

[0829] Input: API response (response data)

[0830] Output: Displayed answer

[0831] Operation: The device receives a response from the server and displays it on the user interface. For example, "This item is in stock" might be displayed on the screen of smart glasses or a smartphone.

[0832] Through these steps, users can quickly obtain product and inventory information even in physical stores.

[0833] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0834] The present invention is a system for providing real-time answers to customer questions, and is particularly characterized by its ability to recognize the user's emotions in conjunction with the system by combining it with an emotion engine, thereby providing more appropriate answers. Specific embodiments of the system of the present invention are shown below.

[0835] Question submission and sentiment recognition

[0836] The user enters a question on the device and clicks the send button. The device prepares to send the entered question to the server and activates an emotion engine to recognize the user's emotions from the question. The emotion engine analyzes the text of the question and generates emotion parameters such as joy, anger, and sadness.

[0837] Reception and analysis on the server

[0838] The device sends an API request to the server, which includes the question, user ID, timestamp, and sentiment parameter. The server receives the request through the API endpoint and extracts the question and sentiment parameter.

[0839] Question analysis and database queries

[0840] The server's natural language processing engine analyzes the question, identifying keywords and intent. Sentiment parameters are also considered and used as additional information to generate an appropriate response. Based on the analyzed keywords, the server queries the database. For example, even if the question is "out of stock," data is retrieved to provide a gentle notification that takes the user's emotions into consideration.

[0841] Generating an answer

[0842] Based on information retrieved from the database and emotion parameters, the server generates an appropriate response. Based on the output of the emotion engine, the response is adjusted to be considerate of the user's emotions. For example, a response that is considerate of the user's emotions might be created, such as, "We are sorry, but this item is currently out of stock. We will notify you as soon as it becomes available."

[0843] Submit and display of responses

[0844] The server sends the generated response to the user's device. The device receives the response and displays it in the user interface. The user can see the specific answer to the question, along with a message that takes their feelings into consideration, on the device screen.

[0845] Specific example

[0846] For example, a user enters the question, "Do you have this item in stock? I'm tired of waiting," and presses the submit button. The device sends the question and emotion parameter (in this case, the emotion "tired") to the server. The server receives the question and analyzes it using a natural language processing engine and an emotion engine, extracting the keyword "stock" and the emotion "tired." Next, the server queries the database for stock information and, if the item is out of stock, generates an emotion-sensitive response. Finally, the device displays a message to the user saying, "We're sorry, but this item is currently out of stock. We will notify you as soon as it becomes available."

[0847] This system allows users to receive quick and accurate answers to their questions in real time, as well as emotionally sensitive service, significantly improving customer satisfaction.

[0848] The following describes the processing flow.

[0849] Step 1:

[0850] The user enters a question from their device. The user enters the question into the input form and clicks the submit button. Example: Enter "Is this product in stock?"

[0851] Step 2:

[0852] The device sends the question to the server. The device generates an API request, packages the entered question with accompanying information such as the user ID and timestamp, and sends it to the server.

[0853] Step 3:

[0854] The device activates its emotion engine. Based on the entered questions, the device uses the emotion engine to analyze the user's emotions. For example, it extracts emotions such as "anger" or "dissatisfaction" from text.

[0855] Step 4:

[0856] The device sends emotion parameters to the server. In addition to the question, the analyzed emotion parameters are also included in the API request and sent to the server.

[0857] Step 5:

[0858] The server receives the question and sentiment parameters. The server's API endpoint receives the request and extracts the question and sentiment parameters.

[0859] Step 6:

[0860] The server analyzes the question. A natural language processing engine built into the server analyzes the question, identifying key keywords and user intent. For example, it identifies the keyword "inventory."

[0861] Step 7:

[0862] The server analyzes the emotional parameters. The server analyzes the emotional parameters received from the emotion engine to understand the user's emotional state. Example: It determines that the user is "highly dissatisfied."

[0863] Step 8:

[0864] The server queries the database. Based on the specified keywords, the server connects to the relevant database and executes an SQL query to retrieve the necessary information. Example: Retrieving product inventory information.

[0865] Step 9:

[0866] The server generates the response. Based on information retrieved from the database and sentiment parameters, the server generates an appropriate response. It constructs a message that takes the user's feelings into consideration, taking sentiment parameters into account. Example: It generates "We are sorry, but this item is currently out of stock. We will notify you as soon as it becomes available."

[0867] Step 10:

[0868] The server sends the generated response to the device. The API response containing the generated response is then sent to the user's device.

[0869] Step 11:

[0870] The device receives the response. The device receives the API response, parses the content of the response, and formats it into data for display.

[0871] Step 12:

[0872] The device displays the answer in the user interface. The user can see a specific answer to the question and an emotionally sensitive message on the device screen. For example, it might display: "We are sorry, but this item is currently out of stock. We will notify you as soon as it becomes available."

[0873] Through this series of processes, users can not only receive real-time answers to their questions, but also receive responses that take their emotional state at the time into consideration. This significantly improves customer satisfaction.

[0874] (Example 2)

[0875] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0876] Conventional question-answering systems failed to consider the user's emotions when they entered and submitted a question. This resulted in the inability to generate appropriate answers that took the user's feelings into account, leading to decreased customer satisfaction. Furthermore, the system lacked the means to provide answers that reflected the user's emotions, resulting in inefficient communication with users.

[0877] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0878] In this invention, the server includes means for sending a question entered by the user from a terminal to the server, means for the terminal to analyze the content of the question and generate sentiment parameters, means for the server to analyze the received question using a natural language processing engine, means for the server to query a database based on the analysis results and sentiment parameters and generate an answer, means for the server to send the generated answer to the user's terminal, and means for the terminal to display the received answer on a user interface. This makes it possible to provide answers that take the user's emotions into consideration in real time, thereby improving customer satisfaction and enabling efficient communication.

[0879] A "user" refers to an individual or organization that operates the system and enters questions.

[0880] "Terminal" refers to electronic devices, including input / output devices such as computers and smartphones used by users.

[0881] A "question" refers to the content of an inquiry that a user enters into the system via their device.

[0882] A "server" refers to a central processing unit that receives, analyzes, processes, and responds to user inquiries.

[0883] "Transmission method" refers to the function for sending user questions from the terminal to the server.

[0884] "Emotional parameters" refer to data that indicates the emotional state (e.g., joy, anger, sadness, etc.) analyzed from the user's questions.

[0885] A "natural language processing engine" refers to software or programs that a server uses to analyze a question and identify keywords and intent.

[0886] A "database" refers to an information storage system that stores answers to questions and manages them so that they can be searched when needed.

[0887] "Query method" refers to the function that allows a server to instruct a database to search for information related to the query.

[0888] "Generation means" refers to the function that allows the server to create an appropriate response based on information and sentiment parameters obtained from the database.

[0889] "Display means" refers to the function that allows the terminal to display the response received from the server on the user interface.

[0890] The system of this invention is designed to provide emotionally appropriate and real-time responses to questions sent by users from their terminals. The system of this invention is implemented by combining the following hardware and software.

[0891] Entering and submitting questions via a terminal.

[0892] The user accesses the system from a device (such as a computer or smartphone) and enters a question. For example, the user enters the following prompt:

[0893] Do you have this item in stock? I'm tired of waiting.

[0894] When the user clicks the "Submit" button, the device receives this question and proceeds to the next step.

[0895] Emotion recognition by devices

[0896] The device uses a cloud-based natural language processing API to analyze the question and recognize the user's emotions. Specifically, it uses the Google Cloud Natural Language API to analyze the question and extract emotional parameters such as joy, anger, and sadness.

[0897] The emotion parameters obtained through this process (e.g., the emotion "tired") are sent to the server in JSON format along with the question data.

[0898] Reception and analysis on the server

[0899] The server receives requests sent from the terminal through an API endpoint built using the Spring Boot framework. The received JSON data includes the question content, user ID, timestamp, and sentiment parameter. The server extracts this data and uses it for the next processing step.

[0900] Question analysis and database queries

[0901] The server analyzes the question using the Hugging Face Transformers library. A natural language processing engine identifies the main keywords of the question and the user's intent. Sentiment parameters are also taken into consideration.

[0902] Based on the analyzed keywords, the server queries the MySQL database. For example, it generates an SQL query to retrieve inventory information and obtains the necessary information from the database.

[0903] Generating an answer

[0904] The server generates an appropriate response based on the retrieved database information and emotion parameters. During this process, the response is adjusted based on the emotion engine's output to ensure it reflects the user's emotions. For example, if an item is out of stock, a comment such as "We apologize, but this item is currently out of stock. We will notify you as soon as it becomes available" is generated.

[0905] Submit and display of responses

[0906] The server sends the generated response to the device in JSON format. The device receives the response asynchronously using JavaScript (AJAX) and displays it in the user interface (for example, a web application using React.js). The user can then see an appropriate and emotionally sensitive response to their question on the device screen.

[0907] In this way, the system of the present invention allows users to obtain quick and accurate answers in real time, and furthermore, these answers take into consideration the user's feelings. As a result, customer satisfaction is significantly improved.

[0908] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0909] Step 1:

[0910] The user enters and submits the question from their terminal. The user enters the question on the system interface and clicks the submit button. For example, the user might enter the following prompt:

[0911] Do you have this item in stock? I'm tired of waiting.

[0912] After this input is received, the terminal retains the question data and proceeds to the next processing step.

[0913] Step 2:

[0914] The device prepares to send a question to the server. Simultaneously, it starts sentiment recognition using the Google Cloud Natural Language API. The device analyzes the question and extracts sentiment parameters. For example, the sentiment "tired" might be recognized. Based on this analysis, the device converts the question and sentiment parameters into JSON format and constructs the data to send to the server. Input: User's question, Output: JSON data with sentiment parameters.

[0915] Step 3:

[0916] The device sends JSON data it has constructed to the server using a RESTful API. Input: JSON data with sentiment parameters; Output: Request sent to the server. This request includes the user ID, question content, timestamp, and sentiment parameters.

[0917] Step 4:

[0918] The server receives the request. It uses a Spring Boot-based API endpoint to receive the request and parse the data. Input: Request sent to the server; Output: Extracted question content, user ID, timestamp, and sentiment parameter.

[0919] Step 5:

[0920] The server analyzes the extracted question content. Using the Hugging Face Transformers library, key keywords and user intent are identified. For example, the keyword "inventory" might be extracted. Sentiment parameters are also included in the processing. Input: Question content; Output: Identified keywords and sentiment parameters.

[0921] Step 6:

[0922] The server queries the database based on the question. It executes an SQL query against the MySQL database to retrieve relevant information. For example, an SQL query to check inventory information is executed. Input: Identified keyword, Output: Inventory information retrieved from the database.

[0923] Step 7:

[0924] The server generates an appropriate response based on information retrieved from the database and emotion parameters. Based on the output of the emotion engine, the response is adjusted to be sensitive to the user's emotions. For example, a response such as "We are sorry, but we are currently out of stock. We will notify you as soon as it becomes available" might be generated. Input: Stock information and emotion parameters, Output: Emotionally sensitive response.

[0925] Step 8:

[0926] The server generates the response and sends it to the terminal in JSON format. Input: Generated response, Output: Response sent to the user's terminal.

[0927] Step 9:

[0928] The device displays the response received from the server on the user interface. For example, responses are received asynchronously using JavaScript (AJAX) and displayed using React.js or similar. The user can see a specific answer to the question, along with a message that takes their emotions into consideration, on the device screen. Input: Response sent from the server; Output: Response displayed on the user interface.

[0929] (Application Example 2)

[0930] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0931] Conventional real-time question answering systems struggle to provide answers that take user emotions into consideration, resulting in decreased user satisfaction. In particular, answers that ignore the emotions embedded in the user's questions degrade the quality of the customer experience. Furthermore, from a sales promotion perspective, providing services that are sensitive to emotions is essential.

[0932] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0933] In this invention, the server includes means for sending a question entered by the user from a terminal to the server, means for activating an emotion engine for recognizing the user's emotions from the question entered by the terminal, means for analyzing the received question using a natural language processing engine, means for querying a database based on the analysis results and generating an answer, means for adjusting the generated answer based on emotion parameters, means for sending the generated answer to the user's terminal, and means for displaying the received answer on a user interface. This enables real-time question answering that takes the user's emotions into consideration.

[0934] A "user" refers to a person who uses the system to input questions.

[0935] A "device" refers to a device used by a user to input questions and receive answers. Examples include smartphones and tablets.

[0936] A "question" refers to the content of an inquiry that a user enters from their device and sends to the system.

[0937] A "server" refers to a central processing unit that analyzes received questions, generates answers, and sends them to terminals.

[0938] A "natural language processing engine" refers to a processing unit that analyzes input questions and understands their meaning and intent.

[0939] An "emotion engine" refers to a device that generates emotional parameters from questions entered by the user.

[0940] "Emotional parameters" refer to data generated by the emotion engine that indicates the user's emotional state.

[0941] A "database" refers to an information storage system that stores the information necessary to generate answers to questions.

[0942] "Answer" refers to the response to an inquiry that the server generates and provides to the user.

[0943] "User interface" refers to the means of displaying answers visually on a device.

[0944] System Configuration

[0945] A system for carrying out this invention includes the following main components:

[0946] User's terminal

[0947] server

[0948] Natural Language Processing Engine

[0949] Emotional Engine

[0950] database

[0951] Program execution

[0952] 1. Send a question from the user's device.

[0953] The user enters a question on their device and presses the send button. This action activates an emotion engine that recognizes the user's emotions along with the question content.

[0954] 2. Emotion recognition by an emotion engine

[0955] The emotion engine analyzes the question text and generates the user's emotion parameters (e.g., joy, anger, sadness). This process uses, for example, the "sentiment-analysis" model from Transformer.

[0956] 3. Sending data to the server

[0957] The device sends an API request to the server that includes the question content, user ID, and sentiment parameters. The request is sent in JSON format.

[0958] 4. Receiving and analyzing data on the server

[0959] The server receives requests at the API endpoint and extracts the question and sentiment parameters. Next, it analyzes the question using a natural language processing engine (e.g., SpaCy or BERT) to identify keywords and intent.

[0960] 5. Retrieving information from the database

[0961] Based on the analyzed keywords, the server queries the database. For example, if the keyword "inventory" is included, it retrieves inventory information from the database.

[0962] 6. Generating responses and adjusting them with consideration for emotions.

[0963] Based on information retrieved from the database and emotion parameters, the server generates an appropriate response. A generative AI model (e.g., GPT-3) is used to refine the response to be more emotionally conscious.

[0964] 7. Submitting and displaying responses

[0965] The server sends the generated response to the user's terminal. The terminal formats the received response and displays it on the user interface.

[0966] Specific example

[0967] For example, suppose a user enters the question, "Do you have this item in stock? I'm tired of waiting," and presses the submit button. The emotion engine recognizes the emotion "tired." An API request containing the question and emotion parameters is sent to the server, which analyzes the keyword "stock" and the emotion "tired." Next, it queries the database for stock information, and if it does not have the item in stock, it generates an emotion-sensitive response such as, "We're sorry, but we don't have this item in stock at the moment. We will notify you as soon as it becomes available." This response is then sent to the device and displayed on the user interface.

[0968] Example prompt

[0969] The following is an example of a prompt used with the emotion engine:

[0970] "Do you have this item in stock? I'm tired of waiting."

[0971] By allowing users to input specific questions in this way, the system can provide quick and emotionally sensitive answers. This entire process improves customer satisfaction and delivers a better user experience.

[0972] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0973] Step 1:

[0974] The user enters a question from their device and presses the send button. The user might enter a specific question, such as, "Do you have this item in stock? I'm tired of waiting." The entered question is temporarily stored as text data on the device.

[0975] Step 2:

[0976] The device passes the entered question to the sentiment engine to begin analysis. The sentiment engine analyzes the question text using, for example, Transformer's "sentiment-analysis" model, and generates the user's sentiment parameters (e.g., joy, anger, sadness). The generated sentiment parameters are represented as a label such as "tired" and its confidence score (e.g., 0.95).

[0977] Step 3:

[0978] The device sends an API request to the server that includes the question content and sentiment parameters. This request includes the question text, user ID, sentiment label, and sentiment score.

[0979] Step 4:

[0980] The server receives the request at the API endpoint and extracts the question, user ID, sentiment label, and sentiment score. The extracted data is then split into its respective fields and passed on to subsequent processing.

[0981] Step 5:

[0982] The server analyzes the question text using a natural language processing engine. For example, it uses SpaCy or BERT to identify keywords and intent in the question. From the question text "Do you have this item in stock? I'm tired of waiting," keywords such as "stock" and "tired" are extracted.

[0983] Step 6:

[0984] Based on the analyzed keywords, the server queries the database for relevant information. For example, a database query containing the keyword "inventory" is generated, and inventory information for the corresponding product is retrieved.

[0985] Step 7:

[0986] The server generates appropriate responses based on information retrieved from the database and sentiment parameters. In this process, a generative AI model (e.g., GPT-3) is used to generate responses that take into account the intent of the question and the user's emotions. For example, a response such as, "We apologize, but this item is currently out of stock. We will notify you as soon as it becomes available," might be generated.

[0987] Step 8:

[0988] The server sends the generated response to the user's device. The data sent includes the text of the response.

[0989] Step 9:

[0990] The device formats the received response and displays it on the user interface. Specifically, the formatted text will read, "We apologize, but this item is currently out of stock. We will notify you as soon as it becomes available."

[0991] This processing flow allows users to receive quick, emotionally sensitive answers to their questions in real time.

[0992] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0993] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0994] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0995] [Fourth Embodiment]

[0996] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0997] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0998] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0999] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1000] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1001] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1002] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1003] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1004] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1005] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1006] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1007] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1008] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1009] The present invention is a system for providing real-time answers to customer questions, and is primarily implemented through a series of processes including question submission, analysis, database querying, answer generation, and answer submission and display.

[1010] Submit a question

[1011] When a user enters a question on their device and clicks the send button, the device sends the question to the server. The device generates an API request, packages the question and associated information (user ID, timestamp, etc.), and sends it to the server.

[1012] Receiving and analyzing questions

[1013] The server receives user queries through the API endpoint. The server analyzes the received queries using a natural language processing engine to identify keywords and user intent. This analysis enables the server to query the appropriate database.

[1014] Database query and response generation

[1015] Based on the analysis results, the server queries the relevant database. For example, if the question "Is this in stock?" is analyzed, the server connects to the database to retrieve the inventory information for the relevant product and uses an SQL query to obtain the inventory information.

[1016] The server generates a response based on the acquired data. The generated response may use a predefined phrase or it may be dynamically constructed. For example, the response "We have it in stock" might be generated.

[1017] Submit and display of responses

[1018] The generated response is sent from the server to the user's device. The device receives this response and displays it in the user interface. The user can then view the answer to the question on the device screen.

[1019] Specific example

[1020] For example, if a user enters the question "Is this item in stock?" and presses the submit button, the device sends the question to the server. The server receives the question and uses natural language processing to analyze the keyword "stock." Next, the server connects to the database and queries for the stock information of that item. If the database returns "In stock," the server uses this information to generate the answer "This item is in stock" and sends it to the user's device. The device displays the received answer on the screen to inform the user.

[1021] This system allows users to get quick and accurate answers to their questions in real time, significantly improving customer service.

[1022] The following describes the processing flow.

[1023] Step 1:

[1024] The user enters their question from their device. The user enters their question into the input form and clicks the submit button. Example: "Is this product in stock?"

[1025] Step 2:

[1026] The device sends the question to the server. The device generates an API request and sends it to the server along with the entered question, user ID, timestamp, and other associated information.

[1027] Step 3:

[1028] The server receives the question. The server's API endpoint receives the request and extracts the question and associated information.

[1029] Step 4:

[1030] The server analyzes the question. The server's natural language processing engine analyzes the question and identifies keywords and intent. Example: Extract the keyword "inventory" from the question.

[1031] Step 5:

[1032] The server queries the database. Based on the identified keywords, the server connects to the database and executes an SQL query to retrieve the relevant information.

[1033] Step 6:

[1034] The server generates the response. Based on information retrieved from the database, the server dynamically generates a response for the user. For example, it might generate the response "In stock."

[1035] Step 7:

[1036] The server sends the answer to the terminal. The response containing the generated answer is sent to the terminal.

[1037] Step 8:

[1038] The device receives the response. The device parses the received response and formats the content to be displayed.

[1039] Step 9:

[1040] The device displays the answer in the user interface. The user can confirm the answer on the device screen. For example, they might see a message like, "This item is in stock!"

[1041] (Example 1)

[1042] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1043] Traditional systems often failed to provide quick and accurate answers to customer inquiries, posing a challenge to improving customer service. Furthermore, the mechanisms for properly analyzing questions entered in natural language, acquiring necessary data, and generating accurate responses were insufficient. As a result, user experience deteriorated, and business efficiency was sometimes compromised.

[1044] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1045] In this invention, the server includes means for transmitting a question entered by a user from a terminal, means for analyzing the question using a natural language processing engine, and means for querying a database and generating an answer based on the analysis results. This makes it possible to receive questions from customers in real time and provide quick and accurate answers.

[1046] A "terminal" refers to a device used by a user for input, and includes, for example, smartphones, personal computers, and tablets.

[1047] A "server" refers to a computer system that receives data sent by users and performs analysis and processing on it.

[1048] A "natural language processing engine" refers to a software component that analyzes human language and understands its meaning and intent.

[1049] An "API request" refers to a communication request from a device to a server to request operations or data.

[1050] A "database" refers to a system that manages organized collections of data, and an example of this is a relational database management system (RDBMS).

[1051] "Analysis results" refer to the information obtained by a natural language processing engine through its analysis of questions and input data.

[1052] "Answer" refers to the response that the server generates based on the user's question.

[1053] "User interface" refers to the screen and software portion of a device that a user uses to perform operations and input.

[1054] "JSON format" refers to a text-based data format used to structure and represent data information.

[1055] This invention is a system that provides real-time answers to customer questions and is implemented through a main processing flow. The system begins with the user inputting and sending a question from a terminal. The terminal generates the input question as an API request and sends it to the server. The API request includes information such as the question content, user ID, and timestamp.

[1056] The server receives requests through the specified API endpoint and uses a natural language processing engine to analyze the question content. Commonly used natural language processing engines such as the Google Cloud Natural Language API or OpenAI's generative AI models can be used. This allows the server to identify keywords and user intent from the question.

[1057] Based on the analysis results, the server connects to the relevant database and retrieves the necessary information. This involves issuing SQL queries using a relational database management system (RDBMS) such as MySQL or PostgreSQL. For example, if the question "Is this in stock?" is analyzed, a database query is made to retrieve the inventory information for the relevant product. Based on the retrieved information, the server generates an answer either dynamically or using a predefined template.

[1058] The generated responses are packaged again in API request format and sent to the user's device. The device parses the received responses and displays them in the user interface. The user interface can be a web page or mobile application built using HTML, CSS, JavaScript, etc.

[1059] For example, if a user enters the question "Is this item in stock?" and presses the submit button, the device sends the question to the server. The server receives the question, analyzes it using a natural language processing engine, and identifies the keyword "stock." Next, the server connects to a database and queries for stock information about the item. If the database returns "in stock," the server generates the answer "This item is in stock" and sends it to the user's device. The device then displays the received answer on the user interface to inform the user.

[1060] Examples of prompt statements include the following:

[1061] "Do you have this item in stock?"

[1062] "Could you tell me your business hours?"

[1063] "What's the latest promotion?"

[1064] This system allows users to receive quick and accurate answers to their questions in real time, thereby improving customer service.

[1065] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1066] Step 1:

[1067] The user enters a question from their device and clicks the submit button. This input includes information such as the question, user ID, and timestamp. When the user enters "Is this product in stock?" and clicks the submit button, the device retrieves the input and generates an API request in JSON format. The API request is then sent to the server as output.

[1068] Step 2:

[1069] The server processes requests received at the API endpoint. The input here is the API request sent from the terminal. The server first parses the JSON data to extract information such as the question content, user ID, and timestamp. Based on this extracted information, it prepares data to pass to the natural language processing engine, and generates data for analysis as output.

[1070] Step 3:

[1071] The server analyzes the question using a natural language processing engine. The input is the data prepared in step 2. The server uses a natural language processing engine, such as the Google Cloud Natural Language API or OpenAI's generative AI model, to identify keywords and intent from the question. If the keyword "inventory" is identified through this analysis, the analysis result is obtained as output.

[1072] Step 4:

[1073] The server queries the database based on the analysis results. The input here is the analysis result of natural language processing, which contains information that includes the identified keywords. The server connects to a relational database management system (RDBMS) such as MySQL or PostgreSQL, generates an SQL query, and sends it. For example, a query like "SELECT in_stock FROM products WHERE product_id = 98765" is issued. The output is inventory information returned from the database.

[1074] Step 5:

[1075] The server generates a response based on information retrieved from the database. The input is inventory information retrieved from the database. Based on the retrieved data, the server generates a response such as "This item is in stock" if the item is in stock. The output is the response text for the user.

[1076] Step 6:

[1077] The server generates a response and sends it to the device. The input is the response text generated by the server. The server packages the response in JSON format, generates another API request, and sends it to the device. The output is the API request sent to the device.

[1078] Step 7:

[1079] The terminal displays the received response on the user interface. The input is response data in JSON format received from the server. The terminal parses this data and formats it for display on the user interface. For example, HTML and JavaScript are used to display the response on the screen. As output, the user can see the answer to the question on the screen.

[1080] Through these steps, users can obtain quick and accurate answers to their questions in real time.

[1081] (Application Example 1)

[1082] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1083] Currently, when customers in physical stores want to know product information or inventory information, they typically ask store staff directly. However, it is difficult to get a quick response during busy periods or when staff are engaged in other tasks. This invention aims to solve this problem and provide a system that enables customers to obtain information quickly and accurately.

[1084] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1085] In this invention, the server includes means for transmitting a question entered by a user from a mobile device to the server; means for analyzing the received question using a natural language processing system; means for querying an information storage device based on the analysis results and generating an answer; means for transmitting the generated answer to the user's mobile device; means for displaying the answer received by the mobile device on a user interface; input means for enabling the user to input questions using voice or text; and means for receiving and displaying answers to questions in real time via software installed on the mobile device. This enables customers to quickly and accurately obtain product and inventory information even in physical stores.

[1086] A "personal information terminal" is a portable device such as a smartphone or smart glasses that a user uses to input questions and receive and display answers.

[1087] A "natural language processing system" is an artificial intelligence technology that analyzes questions received from users to identify their intent and keywords.

[1088] "Information storage device" refers to a database or storage device that stores data for generating answers to questions.

[1089] An "input method" is a function that allows users to input questions using voice or text.

[1090] A "user interface" is a screen or display area on a mobile device that allows the user to visually confirm the answers they have generated.

[1091] "Software" refers to a program installed on a mobile device that controls the sending of questions and the receiving and display of answers.

[1092] "Real-time" refers to responding quickly to user questions and providing answers without delay.

[1093] This invention provides a system that allows customers to quickly obtain product and inventory information in physical stores. This system includes a mobile information terminal, a server, a natural language processing system, an information storage device, input means, and a user interface.

[1094] System Configuration

[1095] 1. Mobile device:

[1096] In this system, a personal digital assistant (PDA) is a device used by the user to input questions and receive and display answers. Specifically, this includes smartphones and smart glasses.

[1097] 2. Server:

[1098] The server receives a question from the user and analyzes it using a natural language processing system. It then queries its information storage device based on the analysis results and generates an answer. Finally, it sends the generated answer to the user's mobile device.

[1099] 3. Natural Language Processing Systems:

[1100] The natural language processing system used on the server analyzes received questions to identify keywords and user intent. This analysis enables appropriate database queries. Specific software used includes Spacy, among others.

[1101] 4. Information storage:

[1102] A database or storage device that holds data for generating answers to questions. For example, it might store product inventory information or detailed information.

[1103] 5. Input method:

[1104] A feature that allows users to input questions using voice or text. This includes voice input devices and text input interfaces.

[1105] 6. User Interface:

[1106] This refers to a screen or display area on a mobile device that allows users to visually confirm the answers they have generated. This enables users to instantly obtain the necessary information within the store.

[1107] System operation

[1108] When a user inputs a question via voice or text using a mobile device, the question is sent from the device to the server. The server receives the question and analyzes it using a natural language processing system. Based on the keywords in the analyzed question, the server queries its information storage device and generates an appropriate answer. The generated answer is sent from the server to the mobile device and displayed on the device's user interface.

[1109] Specific example

[1110] For example, if a user asks their smart glasses, "Do you have this item in stock?", the question is sent to the server. The server analyzes the question and identifies the keyword "stock". It then connects to a data storage device and retrieves the stock information for that item. If the database returns "in stock", the server generates the response "This item is in stock" and displays it on the smart glasses.

[1111] Example of a prompt

[1112] Question: "Do you have this item in stock?"

[1113] Answer: "This item is in stock."

[1114] This allows customers to easily obtain product information in physical stores, resulting in an efficient shopping experience.

[1115] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1116] Step 1:

[1117] The user enters the question using voice or text via a mobile device.

[1118] Input: User's question (e.g., "Do you have this item in stock?")

[1119] Output: Question data (audio data or text data)

[1120] Operation: Questions are entered using the input function of a smartphone or smart glasses. The user interface utilizes voice recognition and text input.

[1121] Step 2:

[1122] The terminal sends the question data to the server.

[1123] Input: Question data

[1124] Output: API Request

[1125] Operation: The terminal packages the question data, generates an API request, and sends it to the server. Specifically, the request includes information such as the question text, user ID, and timestamp.

[1126] Step 3:

[1127] The server receives the question data through the API endpoint.

[1128] Input: API Request

[1129] Output: Question data (received on the server side)

[1130] Operation: The server receives question data via the API endpoint and places it in a queue for preparation for analysis.

[1131] Step 4:

[1132] The server uses a natural language processing system to analyze the question.

[1133] Input: Question data

[1134] Output: Analysis results (keywords and intent)

[1135] Operation: Uses a natural language processing system (e.g., Spacy) to analyze the received question text and identify keywords and user intent. For example, it extracts nouns such as "inventory" and verbs such as "do you have any?".

[1136] Step 5:

[1137] The server queries the information storage device based on the analysis results.

[1138] Input: Analysis results (keywords and intent)

[1139] Output: Database query results (e.g., inventory information)

[1140] Operation: Based on the analysis results, it generates an appropriate SQL query and queries the information storage device (database). It retrieves inventory information and related data for the relevant product from the database.

[1141] Step 6:

[1142] The server generates an answer based on the data it has acquired.

[1143] Input: Database query results (e.g., inventory information)

[1144] Output: Generated response (Example: "This item is in stock.")

[1145] Function: Processes query results and generates appropriate canned responses or dynamically constructed answers. For example, from the data "In stock," it creates a response such as "This product is in stock."

[1146] Step 7:

[1147] The server sends the generated response to the terminal.

[1148] Input: Generated answer

[1149] Output: API response (response data)

[1150] Operation: The server packages the generated response and sends it to the terminal as an API response.

[1151] Step 8:

[1152] The device displays the received response on the user interface.

[1153] Input: API response (response data)

[1154] Output: Displayed answer

[1155] Operation: The device receives a response from the server and displays it on the user interface. For example, "This item is in stock" might be displayed on the screen of smart glasses or a smartphone.

[1156] Through these steps, users can quickly obtain product and inventory information even in physical stores.

[1157] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1158] The present invention is a system for providing real-time answers to customer questions, and is particularly characterized by its ability to recognize the user's emotions in conjunction with the system by combining it with an emotion engine, thereby providing more appropriate answers. Specific embodiments of the system of the present invention are shown below.

[1159] Question submission and sentiment recognition

[1160] The user enters a question on the device and clicks the send button. The device prepares to send the entered question to the server and activates an emotion engine to recognize the user's emotions from the question. The emotion engine analyzes the text of the question and generates emotion parameters such as joy, anger, and sadness.

[1161] Reception and analysis on the server

[1162] The device sends an API request to the server, which includes the question, user ID, timestamp, and sentiment parameter. The server receives the request through the API endpoint and extracts the question and sentiment parameter.

[1163] Question analysis and database queries

[1164] The server's natural language processing engine analyzes the question, identifying keywords and intent. Sentiment parameters are also considered and used as additional information to generate an appropriate response. Based on the analyzed keywords, the server queries the database. For example, even if the question is "out of stock," data is retrieved to provide a gentle notification that takes the user's emotions into consideration.

[1165] Generating an answer

[1166] Based on information retrieved from the database and emotion parameters, the server generates an appropriate response. Based on the output of the emotion engine, the response is adjusted to be considerate of the user's emotions. For example, a response that is considerate of the user's emotions might be created, such as, "We are sorry, but this item is currently out of stock. We will notify you as soon as it becomes available."

[1167] Submit and display of responses

[1168] The server sends the generated response to the user's device. The device receives the response and displays it in the user interface. The user can see the specific answer to the question, along with a message that takes their feelings into consideration, on the device screen.

[1169] Specific example

[1170] For example, a user enters the question, "Do you have this item in stock? I'm tired of waiting," and presses the submit button. The device sends the question and emotion parameter (in this case, the emotion "tired") to the server. The server receives the question and analyzes it using a natural language processing engine and an emotion engine, extracting the keyword "stock" and the emotion "tired." Next, the server queries the database for stock information and, if the item is out of stock, generates an emotion-sensitive response. Finally, the device displays a message to the user saying, "We're sorry, but this item is currently out of stock. We will notify you as soon as it becomes available."

[1171] This system allows users to receive quick and accurate answers to their questions in real time, as well as emotionally sensitive service, significantly improving customer satisfaction.

[1172] The following describes the processing flow.

[1173] Step 1:

[1174] The user enters a question from their device. The user enters the question into the input form and clicks the submit button. Example: Enter "Is this product in stock?"

[1175] Step 2:

[1176] The device sends the question to the server. The device generates an API request, packages the entered question with accompanying information such as the user ID and timestamp, and sends it to the server.

[1177] Step 3:

[1178] The device activates its emotion engine. Based on the entered questions, the device uses the emotion engine to analyze the user's emotions. For example, it extracts emotions such as "anger" or "dissatisfaction" from text.

[1179] Step 4:

[1180] The device sends emotion parameters to the server. In addition to the question, the analyzed emotion parameters are also included in the API request and sent to the server.

[1181] Step 5:

[1182] The server receives the question and sentiment parameters. The server's API endpoint receives the request and extracts the question and sentiment parameters.

[1183] Step 6:

[1184] The server analyzes the question. A natural language processing engine built into the server analyzes the question, identifying key keywords and user intent. For example, it identifies the keyword "inventory."

[1185] Step 7:

[1186] The server analyzes the emotional parameters. The server analyzes the emotional parameters received from the emotion engine to understand the user's emotional state. Example: It determines that the user is "highly dissatisfied."

[1187] Step 8:

[1188] The server queries the database. Based on the specified keywords, the server connects to the relevant database and executes an SQL query to retrieve the necessary information. Example: Retrieving product inventory information.

[1189] Step 9:

[1190] The server generates the response. Based on information retrieved from the database and sentiment parameters, the server generates an appropriate response. It constructs a message that takes the user's feelings into consideration, taking sentiment parameters into account. Example: It generates "We are sorry, but this item is currently out of stock. We will notify you as soon as it becomes available."

[1191] Step 10:

[1192] The server sends the generated response to the device. The API response containing the generated response is then sent to the user's device.

[1193] Step 11:

[1194] The device receives the response. The device receives the API response, parses the content of the response, and formats it into data for display.

[1195] Step 12:

[1196] The device displays the answer in the user interface. The user can see a specific answer to the question and an emotionally sensitive message on the device screen. For example, it might display: "We are sorry, but this item is currently out of stock. We will notify you as soon as it becomes available."

[1197] Through this series of processes, users can not only receive real-time answers to their questions, but also receive responses that take their emotional state at the time into consideration. This significantly improves customer satisfaction.

[1198] (Example 2)

[1199] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1200] Conventional question-answering systems failed to consider the user's emotions when they entered and submitted a question. This resulted in the inability to generate appropriate answers that took the user's feelings into account, leading to decreased customer satisfaction. Furthermore, the system lacked the means to provide answers that reflected the user's emotions, resulting in inefficient communication with users.

[1201] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1202] In this invention, the server includes means for sending a question entered by the user from a terminal to the server, means for the terminal to analyze the content of the question and generate sentiment parameters, means for the server to analyze the received question using a natural language processing engine, means for the server to query a database based on the analysis results and sentiment parameters and generate an answer, means for the server to send the generated answer to the user's terminal, and means for the terminal to display the received answer on a user interface. This makes it possible to provide answers that take the user's emotions into consideration in real time, thereby improving customer satisfaction and enabling efficient communication.

[1203] A "user" refers to an individual or organization that operates the system and enters questions.

[1204] "Terminal" refers to electronic devices, including input / output devices such as computers and smartphones used by users.

[1205] A "question" refers to the content of an inquiry that a user enters into the system via their device.

[1206] A "server" refers to a central processing unit that receives, analyzes, processes, and responds to user inquiries.

[1207] "Transmission method" refers to the function for sending user questions from the terminal to the server.

[1208] "Emotional parameters" refer to data that indicates the emotional state (e.g., joy, anger, sadness, etc.) analyzed from the user's questions.

[1209] A "natural language processing engine" refers to software or programs that a server uses to analyze a question and identify keywords and intent.

[1210] A "database" refers to an information storage system that stores answers to questions and manages them so that they can be searched when needed.

[1211] "Query method" refers to the function that allows a server to instruct a database to search for information related to the query.

[1212] "Generation means" refers to the function that allows the server to create an appropriate response based on information and sentiment parameters obtained from the database.

[1213] "Display means" refers to the function that allows the terminal to display the response received from the server on the user interface.

[1214] The system of this invention is designed to provide emotionally appropriate and real-time responses to questions sent by users from their terminals. The system of this invention is implemented by combining the following hardware and software.

[1215] Entering and submitting questions via a terminal.

[1216] The user accesses the system from a device (such as a computer or smartphone) and enters a question. For example, the user enters the following prompt:

[1217] Do you have this item in stock? I'm tired of waiting.

[1218] When the user clicks the "Submit" button, the device receives this question and proceeds to the next step.

[1219] Emotion recognition by devices

[1220] The device uses a cloud-based natural language processing API to analyze the question and recognize the user's emotions. Specifically, it uses the Google Cloud Natural Language API to analyze the question and extract emotional parameters such as joy, anger, and sadness.

[1221] The emotion parameters obtained through this process (e.g., the emotion "tired") are sent to the server in JSON format along with the question data.

[1222] Reception and analysis on the server

[1223] The server receives requests sent from the terminal through an API endpoint built using the Spring Boot framework. The received JSON data includes the question content, user ID, timestamp, and sentiment parameter. The server extracts this data and uses it for the next processing step.

[1224] Question analysis and database queries

[1225] The server analyzes the question using the Hugging Face Transformers library. A natural language processing engine identifies the main keywords of the question and the user's intent. Sentiment parameters are also taken into consideration.

[1226] Based on the analyzed keywords, the server queries the MySQL database. For example, it generates an SQL query to retrieve inventory information and obtains the necessary information from the database.

[1227] Generating an answer

[1228] The server generates an appropriate response based on the retrieved database information and emotion parameters. During this process, the response is adjusted based on the emotion engine's output to ensure it reflects the user's emotions. For example, if an item is out of stock, a comment such as "We apologize, but this item is currently out of stock. We will notify you as soon as it becomes available" is generated.

[1229] Submit and display of responses

[1230] The server sends the generated response to the device in JSON format. The device receives the response asynchronously using JavaScript (AJAX) and displays it in the user interface (for example, a web application using React.js). The user can then see an appropriate and emotionally sensitive response to their question on the device screen.

[1231] In this way, the system of the present invention allows users to obtain quick and accurate answers in real time, and furthermore, these answers take into consideration the user's feelings. As a result, customer satisfaction is significantly improved.

[1232] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1233] Step 1:

[1234] The user enters and submits the question from their terminal. The user enters the question on the system interface and clicks the submit button. For example, the user might enter the following prompt:

[1235] Do you have this item in stock? I'm tired of waiting.

[1236] After this input is received, the terminal retains the question data and proceeds to the next processing step.

[1237] Step 2:

[1238] The device prepares to send a question to the server. Simultaneously, it starts sentiment recognition using the Google Cloud Natural Language API. The device analyzes the question and extracts sentiment parameters. For example, the sentiment "tired" might be recognized. Based on this analysis, the device converts the question and sentiment parameters into JSON format and constructs the data to send to the server. Input: User's question, Output: JSON data with sentiment parameters.

[1239] Step 3:

[1240] The device sends JSON data it has constructed to the server using a RESTful API. Input: JSON data with sentiment parameters; Output: Request sent to the server. This request includes the user ID, question content, timestamp, and sentiment parameters.

[1241] Step 4:

[1242] The server receives the request. It uses a Spring Boot-based API endpoint to receive the request and parse the data. Input: Request sent to the server; Output: Extracted question content, user ID, timestamp, and sentiment parameter.

[1243] Step 5:

[1244] The server analyzes the extracted question content. Using the Hugging Face Transformers library, key keywords and user intent are identified. For example, the keyword "inventory" might be extracted. Sentiment parameters are also included in the processing. Input: Question content; Output: Identified keywords and sentiment parameters.

[1245] Step 6:

[1246] The server queries the database based on the question. It executes an SQL query against the MySQL database to retrieve relevant information. For example, an SQL query to check inventory information is executed. Input: Identified keyword, Output: Inventory information retrieved from the database.

[1247] Step 7:

[1248] The server generates an appropriate response based on information retrieved from the database and emotion parameters. Based on the output of the emotion engine, the response is adjusted to be sensitive to the user's emotions. For example, a response such as "We are sorry, but we are currently out of stock. We will notify you as soon as it becomes available" might be generated. Input: Stock information and emotion parameters, Output: Emotionally sensitive response.

[1249] Step 8:

[1250] The server generates the response and sends it to the terminal in JSON format. Input: Generated response, Output: Response sent to the user's terminal.

[1251] Step 9:

[1252] The device displays the response received from the server on the user interface. For example, responses are received asynchronously using JavaScript (AJAX) and displayed using React.js or similar. The user can see a specific answer to the question, along with a message that takes their emotions into consideration, on the device screen. Input: Response sent from the server; Output: Response displayed on the user interface.

[1253] (Application Example 2)

[1254] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1255] Conventional real-time question answering systems struggle to provide answers that take user emotions into consideration, resulting in decreased user satisfaction. In particular, answers that ignore the emotions embedded in the user's questions degrade the quality of the customer experience. Furthermore, from a sales promotion perspective, providing services that are sensitive to emotions is essential.

[1256] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1257] In this invention, the server includes means for sending a question entered by the user from a terminal to the server, means for activating an emotion engine for recognizing the user's emotions from the question entered by the terminal, means for analyzing the received question using a natural language processing engine, means for querying a database based on the analysis results and generating an answer, means for adjusting the generated answer based on emotion parameters, means for sending the generated answer to the user's terminal, and means for displaying the received answer on a user interface. This enables real-time question answering that takes the user's emotions into consideration.

[1258] A "user" refers to a person who uses the system to input questions.

[1259] A "device" refers to a device used by a user to input questions and receive answers. Examples include smartphones and tablets.

[1260] A "question" refers to the content of an inquiry that a user enters from their device and sends to the system.

[1261] A "server" refers to a central processing unit that analyzes received questions, generates answers, and sends them to terminals.

[1262] A "natural language processing engine" refers to a processing unit that analyzes input questions and understands their meaning and intent.

[1263] An "emotion engine" refers to a device that generates emotional parameters from questions entered by the user.

[1264] "Emotional parameters" refer to data generated by the emotion engine that indicates the user's emotional state.

[1265] A "database" refers to an information storage system that stores the information necessary to generate answers to questions.

[1266] "Answer" refers to the response to an inquiry that the server generates and provides to the user.

[1267] "User interface" refers to the means of displaying answers visually on a device.

[1268] System Configuration

[1269] A system for carrying out this invention includes the following main components:

[1270] User's terminal

[1271] server

[1272] Natural Language Processing Engine

[1273] Emotional Engine

[1274] database

[1275] Program execution

[1276] 1. Send a question from the user's device.

[1277] The user enters a question on their device and presses the send button. This action activates an emotion engine that recognizes the user's emotions along with the question content.

[1278] 2. Emotion recognition by an emotion engine

[1279] The emotion engine analyzes the question text and generates the user's emotion parameters (e.g., joy, anger, sadness). This process uses, for example, the "sentiment-analysis" model from Transformer.

[1280] 3. Sending data to the server

[1281] The device sends an API request to the server that includes the question content, user ID, and sentiment parameters. The request is sent in JSON format.

[1282] 4. Receiving and analyzing data on the server

[1283] The server receives requests at the API endpoint and extracts the question and sentiment parameters. Next, it analyzes the question using a natural language processing engine (e.g., SpaCy or BERT) to identify keywords and intent.

[1284] 5. Retrieving information from the database

[1285] Based on the analyzed keywords, the server queries the database. For example, if the keyword "inventory" is included, it retrieves inventory information from the database.

[1286] 6. Generating responses and adjusting them with consideration for emotions.

[1287] Based on information retrieved from the database and emotion parameters, the server generates an appropriate response. A generative AI model (e.g., GPT-3) is used to refine the response to be more emotionally conscious.

[1288] 7. Submitting and displaying responses

[1289] The server sends the generated response to the user's terminal. The terminal formats the received response and displays it on the user interface.

[1290] Specific example

[1291] For example, suppose a user enters the question, "Do you have this item in stock? I'm tired of waiting," and presses the submit button. The emotion engine recognizes the emotion "tired." An API request containing the question and emotion parameters is sent to the server, which analyzes the keyword "stock" and the emotion "tired." Next, it queries the database for stock information, and if it does not have the item in stock, it generates an emotion-sensitive response such as, "We're sorry, but we don't have this item in stock at the moment. We will notify you as soon as it becomes available." This response is then sent to the device and displayed on the user interface.

[1292] Example prompt

[1293] The following is an example of a prompt used with the emotion engine:

[1294] "Do you have this item in stock? I'm tired of waiting."

[1295] By allowing users to input specific questions in this way, the system can provide quick and emotionally sensitive answers. This entire process improves customer satisfaction and delivers a better user experience.

[1296] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1297] Step 1:

[1298] The user enters a question from their device and presses the send button. The user might enter a specific question, such as, "Do you have this item in stock? I'm tired of waiting." The entered question is temporarily stored as text data on the device.

[1299] Step 2:

[1300] The device passes the entered question to the sentiment engine to begin analysis. The sentiment engine analyzes the question text using, for example, Transformer's "sentiment-analysis" model, and generates the user's sentiment parameters (e.g., joy, anger, sadness). The generated sentiment parameters are represented as a label such as "tired" and its confidence score (e.g., 0.95).

[1301] Step 3:

[1302] The device sends an API request to the server that includes the question content and sentiment parameters. This request includes the question text, user ID, sentiment label, and sentiment score.

[1303] Step 4:

[1304] The server receives the request at the API endpoint and extracts the question, user ID, sentiment label, and sentiment score. The extracted data is then split into its respective fields and passed on to subsequent processing.

[1305] Step 5:

[1306] The server analyzes the question text using a natural language processing engine. For example, it uses SpaCy or BERT to identify keywords and intent in the question. From the question text "Do you have this item in stock? I'm tired of waiting," keywords such as "stock" and "tired" are extracted.

[1307] Step 6:

[1308] Based on the analyzed keywords, the server queries the database for relevant information. For example, a database query containing the keyword "inventory" is generated, and inventory information for the corresponding product is retrieved.

[1309] Step 7:

[1310] The server generates appropriate responses based on information retrieved from the database and sentiment parameters. In this process, a generative AI model (e.g., GPT-3) is used to generate responses that take into account the intent of the question and the user's emotions. For example, a response such as, "We apologize, but this item is currently out of stock. We will notify you as soon as it becomes available," might be generated.

[1311] Step 8:

[1312] The server sends the generated response to the user's device. The data sent includes the text of the response.

[1313] Step 9:

[1314] The device formats the received response and displays it on the user interface. Specifically, the formatted text will read, "We apologize, but this item is currently out of stock. We will notify you as soon as it becomes available."

[1315] This processing flow allows users to receive quick, emotionally sensitive answers to their questions in real time.

[1316] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1317] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1318] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1319] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1320] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1321] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1322] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1323] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1324] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1325] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1326] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1327] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1328] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1329] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1330] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1331] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1332] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1333] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1334] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1335] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1336] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1337] The following is further disclosed regarding the embodiments described above.

[1338] (Claim 1)

[1339] A means of sending questions entered by the user from the terminal to the server,

[1340] The server includes means for analyzing the received question using a natural language processing engine,

[1341] The server provides means for querying a database based on the analysis results and generating a response,

[1342] The server provides means for sending the generated response to the user's terminal,

[1343] A system including means for displaying the response received by the terminal on a user interface.

[1344] (Claim 2)

[1345] A means for identifying keywords included in the aforementioned question,

[1346] The system according to claim 1, further comprising means for performing a database search based on the identified keywords.

[1347] (Claim 3)

[1348] The server provides a means for dynamically constructing the generated response,

[1349] The system according to claim 1, wherein the terminal comprises means for formatting and displaying the answer.

[1350] "Example 1"

[1351] (Claim 1)

[1352] A means of sending questions entered by the user from the terminal to the server,

[1353] The server includes means for analyzing the received question using a natural language processing engine,

[1354] The server provides means for querying a database based on the analysis results and generating a response,

[1355] The server provides means for sending the generated response to the user's terminal,

[1356] A system including means for displaying the response received by the terminal on a user interface.

[1357] (Claim 2)

[1358] Means for identifying the information included in the aforementioned question,

[1359] The system according to claim 1, further comprising means for performing a database search based on the identified information.

[1360] (Claim 3)

[1361] The server provides a means for dynamically constructing the generated response,

[1362] The system according to claim 1, wherein the terminal comprises means for formatting and displaying the answer.

[1363] "Application Example 1"

[1364] (Claim 1)

[1365] A means of sending questions entered by the user from a mobile device to a server,

[1366] The server provides means for analyzing the received question using a natural language processing system,

[1367] The server provides means for querying an information storage device based on the analysis results and generating a response,

[1368] The server provides means for transmitting the generated response to the user's mobile device,

[1369] Means for displaying the response received by the mobile information terminal on the user interface,

[1370] An input means that allows the user to input questions using voice or text,

[1371] A system including means for receiving and displaying answers to questions in real time via software installed on the aforementioned mobile information terminal.

[1372] (Claim 2)

[1373] A means for identifying keywords included in the aforementioned question,

[1374] The system according to claim 1, further comprising means for performing an information storage device search based on the identified keywords.

[1375] (Claim 3)

[1376] The server provides a means for dynamically constructing the generated response,

[1377] The system according to claim 1, wherein the mobile information terminal is equipped with means for formatting and displaying the response.

[1378] "Example 2 of combining an emotion engine"

[1379] (Claim 1)

[1380] A means of sending questions entered by the user from the terminal to the server,

[1381] The aforementioned terminal includes means for analyzing the content of the question and generating emotion parameters,

[1382] The server includes means for analyzing the received question using a natural language processing engine,

[1383] The server provides means for querying a database based on the analysis results and emotion parameters and generating a response,

[1384] The server provides means for sending the generated response to the user's terminal,

[1385] A system including means for displaying the response received by the terminal on a user interface.

[1386] (Claim 2)

[1387] A means for identifying keywords included in the aforementioned question,

[1388] The system according to claim 1, further comprising means for performing a database search based on the identified keywords and sentiment parameters.

[1389] (Claim 3)

[1390] The server dynamically constructs the generated response and adjusts it with consideration for emotions,

[1391] The system according to claim 1, wherein the terminal comprises means for formatting and displaying the answer.

[1392] "Application example 2 of combining emotional engines"

[1393] (Claim 1)

[1394] A means of sending questions entered by the user from the terminal to the server,

[1395] The aforementioned terminal includes means for activating an emotion engine to recognize the user's emotions from the questions entered,

[1396] The server includes means for analyzing the received question using a natural language processing engine,

[1397] The server provides means for querying a database based on the analysis results and generating a response,

[1398] The server includes means for adjusting the generated response based on emotion parameters,

[1399] The server provides means for sending the generated response to the user's terminal,

[1400] A system including means for displaying the response received by the terminal on a user interface.

[1401] (Claim 2)

[1402] A means for identifying keywords included in the aforementioned question,

[1403] The system according to claim 1, further comprising means for performing a database search based on the identified keywords.

[1404] (Claim 3)

[1405] The server provides a means for dynamically constructing the generated response,

[1406] The server includes means for adjusting the response using emotion parameters,

[1407] The system according to claim 1, wherein the terminal comprises means for formatting and displaying the answer. [Explanation of symbols]

[1408] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of sending questions entered by the user from the terminal to the server, The server includes means for analyzing the received question using a natural language processing engine, The server provides means for querying a database based on the analysis results and generating a response, The server provides means for sending the generated response to the user's terminal, A system including means for displaying the response received by the terminal on a user interface.

2. A means for identifying keywords included in the aforementioned question, The system according to claim 1, further comprising means for performing a database search based on the specified keywords.

3. The server provides a means for dynamically constructing the generated response, The system according to claim 1, wherein the terminal comprises means for formatting and displaying the response.

Images