system

Abstract

Provide a system. 【Solution means】Means for providing an interface for a user to input a question, Means for sending the question to a server, Means for the server to analyze the question using a natural language processing engine, Means for classifying the question, Means for obtaining relevant information from a database based on the classification, Means for generating the obtained relevant information as an answer using a natural language generation engine, Means for sending the generated answer to a user terminal, Means for the user terminal to display the answer, A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Currently, at administrative and corporate counters, questions regarding national systems and IT technical specifications are frequently received. However, there are problems such as the content being ambiguous, difficulty in handling infrequent or out-of-scope questions, and the response taking a long time, increasing the burden on the counter staff and potentially degrading the service quality for end-users. The present invention aims to solve these problems, provide quick and accurate answers to questions, reduce the burden on the counter staff, and improve the convenience for end-users.

Means for Solving the Problems

[0005] To solve the above problems, the present invention provides the following means: a system including means for providing an interface for a user to input a question, means for sending a question to a server, means for the server to analyze the received question using a natural language processing engine, means for classifying the question, means for obtaining relevant information from a database based on the classification, means for generating an answer from the obtained information using a natural language generation engine, means for sending the generated answer to a user terminal, and means for the user terminal to display the answer. Furthermore, by having means for the server to classify questions using a machine learning model and means for the user terminal to send questions to the server using the HTTPS protocol, it becomes possible to respond quickly and accurately to ambiguous questions, infrequent questions, and questions outside the scope.

[0006] A "user interface" is a means by which a user can input information into a system and view the output from that system.

[0007] A "server" is a computer system that provides services and resources to other computer systems (clients) via a network.

[0008] A "natural language processing engine" is a software tool or algorithm used to analyze human language and understand its meaning.

[0009] "Question classification" refers to the process of categorizing received questions according to certain criteria.

[0010] A "database" is a system that manages organized collections of data, enabling rapid searching and updating.

[0011] A "natural language generation engine" is a software tool or algorithm that automatically generates text in a form that humans can read naturally.

[0012] The HTTPS protocol is a communication protocol used to securely exchange data over the internet.

[0013] A "machine learning model" is a mathematical model that learns patterns from data and uses those learning results to make predictions and classifications about new data.

[0014] A "user terminal" is a hardware device (such as a PC or smartphone) operated by a user, and is a device used to perform interactions with the system.

[0015] "Response generation" refers to the process of creating response text based on appropriate information for analyzed and categorized questions. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] Shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] [[ID=^{7}]]It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained

[0019] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention provides a system that allows users to quickly and accurately obtain answers to questions at government and corporate service counters. The system centers on a series of processes in which the user inputs a question, which is then sent to a server for analysis and classification, and an appropriate answer is generated and provided to the user.

[0038] Overall System Overview

[0039] This system mainly consists of the following components:

[0040] 1. User Interface

[0041] 2. The terminal that sends the question data to the server.

[0042] 3. Server that receives and analyzes question data

[0043] 4. Natural Language Processing and Generation Engines

[0044] 5. Database

[0045] 6. Interface for sending and displaying responses on the user's terminal.

[0046] Program processing flow

[0047] Input Interface

[0048] The user launches the application provided on their PC or smartphone and enters a question into the text box. For example, after entering the question, "How do I apply for childcare support funds?", they click the submit button.

[0049] Submit a question

[0050] The terminal sends the questions entered by the user to the server using the HTTPS protocol.

[0051] Question analysis

[0052] The server parses the received question in JSON format and uses a natural language processing engine (e.g., spaCy or NLTK) to analyze the question text. Here, the keyword "childcare support funds" and the action "how to apply" are extracted.

[0053] Question classification

[0054] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the question into a predefined category. In this case, the question is classified as "national system."

[0055] Answer generation

[0056] The server queries the FAQ database and retrieves the relevant answer information. For example, "How to apply for childcare support funds" is extracted from the database. Based on the extracted information, a natural language generation engine (e.g., GPT-3®) generates text in an easy-to-understand format. This results in the answer being "You can apply for childcare support funds by following these steps..."

[0057] Submit your response

[0058] The server sends the generated response to the user's terminal as an HTTP response in JSON format.

[0059] Display the answer

[0060] The device parses the received response data and displays it in the application's user interface. The user can see the response, "You can apply for childcare support funds by following these steps..."

[0061] Specific example

[0062] Example 1: Questions regarding IT technical specifications

[0063] A user types "How do I renew my SSL certificate?" on their smartphone and sends it. The device sends this question to the server, which analyzes and categorizes the question. If the question is classified as "IT technical specifications," information on "how to renew an SSL certificate" is retrieved from the database. A natural language generation engine generates an answer, which is displayed on the user interface as "To renew your SSL certificate, follow these steps...".

[0064] Example 2: Questions about national systems

[0065] A user types "How do I apply for childcare support funds?" on their PC and sends it. The terminal sends this question to the server, which analyzes and classifies the question. The question is classified as "National System," and information on "How to apply for childcare support funds" is retrieved from the database. A natural language generation engine generates an answer, which is displayed on the user interface as "You can apply for childcare support funds by following these steps...".

[0066] The above describes the embodiments for carrying out the present invention. This system efficiently performs a series of processes from question analysis to answer generation, enabling users to obtain quick and accurate answers.

[0067] The following describes the processing flow.

[0068] Step 1:

[0069] The user launches the application on their PC or smartphone and enters a question into the text box. For example, they might enter the question, "How do I apply for childcare support funds?" and click the submit button.

[0070] Step 2:

[0071] The terminal sends the entered question to the server as an HTTP POST request using the HTTPS protocol. The question data is encoded in JSON format during this process.

[0072] Step 3:

[0073] The server parses the received question data into JSON format. The server temporarily stores this data and passes it to a natural language processing engine (e.g., spaCy or NLTK).

[0074] Step 4:

[0075] The server uses a natural language processing engine to perform morphological analysis on the question text. Specifically, it analyzes the sentence and extracts keywords and important verbs. For example, keywords such as "childcare support funds" and "application method" are extracted.

[0076] Step 5:

[0077] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the question into a predefined category. For example, the question might be classified as "national systems."

[0078] Step 6:

[0079] The server queries the corresponding FAQ database based on the classification result. The server searches for and extracts relevant information. In this case, information regarding "how to apply for childcare support funds" is retrieved from the database.

[0080] Step 7:

[0081] Based on the data acquired by the server, a natural language generation engine (e.g., GPT-3) is used to generate a response in a user-friendly format. For example, it might generate a sentence like, "To apply for childcare support funds, follow these steps..."

[0082] Step 8:

[0083] The server generates the response and sends it to the user's terminal as an HTTP response in JSON format. The response data is encoded before transmission.

[0084] Step 9:

[0085] The device parses the response data it receives. The parsed data is then dynamically inserted into the application's UI components.

[0086] Step 10:

[0087] The device displays the answer in the user interface. The user can see the answer, which reads, "To apply for childcare support funds, follow these steps..."

[0088] (Example 1)

[0089] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0090] Conventional question-answering systems have struggled to provide quick and accurate answers to user-inputted questions. This is because the processes required for question analysis, classification, and optimal answer generation are complex, and appropriate means are lacking for each stage of the process. The present invention aims to solve these problems and provide a system that can provide users with quick and accurate answers.

[0091] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0092] In this invention, the server includes means for receiving and parsing questions in JSON format, means for analyzing them using a natural language processing engine, and means for classifying them using a machine learning model. This makes it possible to process questions in a consistent flow from reception to analysis and classification.

[0093] A "user" is a person or entity that uses this system to input questions and obtain answers.

[0094] "Interface" refers to the screen or input area of ​​an application where a user enters a question.

[0095] A "server" is a computing system that receives, analyzes, classifies, and generates answers to questions.

[0096] "JSON format" is an abbreviation for JavaScript (registered trademark) Object Notation, and is a format for representing data in a structured manner.

[0097] "Parsing" refers to the process of analyzing data and converting it into a specific format.

[0098] A "natural language processing engine" is software and algorithms used to analyze natural language text and understand its meaning and context.

[0099] A "machine learning model" refers to an algorithm that learns patterns and knowledge from large amounts of data and uses that knowledge to make predictions and classifications on new data.

[0100] A "database" is a data storage system used to centrally manage answer information and related data for questions.

[0101] A "natural language generation engine" is software and algorithms used to convert information into easily understandable text.

[0102] The "HTTPS protocol" is a protocol used to encrypt data in web communications, and is a technology that enables secure data transmission and reception.

[0103] This invention provides a system that allows users to quickly and accurately obtain answers to questions at government and corporate service counters. The system centers on a series of processes in which the user inputs a question, which is then sent to a server for analysis and classification, and an appropriate answer is generated and provided to the user.

[0104] Overall System Overview

[0105] This system mainly consists of the following components:

[0106] 1. User Interface

[0107] 2. The terminal that sends the question data to the server.

[0108] 3. Server that receives and analyzes question data

[0109] 4. Natural Language Processing Engine

[0110] 5. Machine Learning Models

[0111] 6. Database

[0112] 7. Natural Language Generation Engine

[0113] 8. Interface for sending and displaying responses on the user's terminal.

[0114] User question input

[0115] Users can launch the application provided on a device such as a PC or smartphone and enter a question into a text box. For example, they can enter the question, "How do I apply for childcare support funds?" and then click the submit button. The entered question is temporarily stored in the device's memory.

[0116] Submit a question

[0117] The device sends the user-entered question to the server using the HTTPS protocol. Specifically, the question data is included as a JSON payload in the HTTP request and sent to the server's specified URL using the POST method.

[0118] Receiving and parsing questions

[0119] The server receives an HTTP request sent from the terminal. The server parses the received payload in JSON format and stores it in a database or temporary storage on the server.

[0120] Question analysis

[0121] The server passes the stored question data to a natural language processing engine (e.g., spaCy or NLTK) for text analysis. Here, the noun phrase "childcare support funds" and the verb phrase "application method" are extracted. Specifically, processes such as tokenization, part-of-speech tagging, and dependency parsing are performed.

[0122] Question classification

[0123] The server inputs the analyzed data into a machine learning model (e.g., random forest or support vector machine) and classifies it into a predefined category (in this case, "national systems"). This classification result is then added to the question data.

[0124] Answer generation

[0125] The server queries the FAQ database based on the classification results and retrieves answer information for the relevant questions. As a result of the query, information such as "How to apply for childcare support funds" is extracted. Next, a natural language generation engine (e.g., GPT-3) reconstructs the extracted information into natural-sounding sentences. At this stage, a sentence such as "You can apply for childcare support funds by following these steps..." is generated.

[0126] Submit your response

[0127] The server sends the generated response to the user's terminal as an HTTP response in JSON format. The server creates a response including HTTP headers and payload and sends it to the terminal.

[0128] Display the answer

[0129] The device receives a response from the server and parses the answer data in the payload. The parsed data is displayed in the application's user interface. The user can then see the answer, "You can apply for childcare support funds by following these steps...", on the screen.

[0130] Explanation of specific examples

[0131] Example 1: Questions regarding IT technical specifications

[0132] A user types "How do I renew my SSL certificate?" on their smartphone and sends it. The device sends this question to the server, which analyzes and categorizes the question. If the question is classified as "IT technical specifications," information on "how to renew an SSL certificate" is retrieved from the database. A natural language generation engine generates an answer, which is displayed on the user interface as "To renew your SSL certificate, follow these steps...".

[0133] Example 2: Questions about national systems

[0134] A user types "How do I apply for childcare support funds?" on their PC and sends it. The terminal sends this question to the server, which analyzes and classifies the question. The question is classified as "National System," and information on "How to apply for childcare support funds" is retrieved from the database. A natural language generation engine generates an answer, which is displayed on the user interface as "You can apply for childcare support funds by following these steps...".

[0135] Examples of prompts for generative AI models

[0136] Example of a prompt message for a generative AI model (e.g., GPT-3):

[0137] Question: How do I apply for childcare support funds?

[0138] Previous text in the model: Childcare support grants can be applied for by following these steps…

[0139] This prompt is used to generate specific text that provides additional information for the generated AI model.

[0140] The above describes the embodiments for carrying out the present invention. By using this system, a series of processes from question analysis to answer generation can be performed efficiently, enabling users to obtain quick and accurate answers.

[0141] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0142] Step 1: User Question Input

[0143] The user opens the application on their PC or smartphone and enters a question into the text box. The entered question might be text data such as, "Please tell me how to apply for childcare support funds." Clicking the submit button prepares the input data needed to proceed to the next step.

[0144] Step 2: Submit your question

[0145] The terminal sends the user-entered question to the server using the HTTPS protocol. The input data is a text-based question, which is stored as a JSON payload in the body of the HTTP request and sent to the server's specified URL using the POST method. The output is this HTTP request.

[0146] Step 3: Receiving and Parsing Questions

[0147] The server receives an HTTP request sent from the terminal. The received data is parsed in JSON format, and the text data of the question is extracted. This extracted text data becomes the input data used for analysis in the next step. The output is the parseable text data of the question.

[0148] Step 4: Analyzing the Question

[0149] The server passes the extracted text data of the questions to a natural language processing engine (e.g., spaCy or NLTK) for text analysis. The input data is the extracted text data, which undergoes processes such as tokenization, part-of-speech tagging, and dependency parsing to obtain, for example, the noun phrase "childcare support funds" and the verb phrase "application method". The output is structural information of the analyzed text data.

[0150] Step 5: Classifying Questions

[0151] The server inputs the structural information of the analyzed text data into a machine learning model (e.g., Random Forest or Support Vector Machine) to perform categorization. The input data is the structural information of the analysis results, which is classified into predefined categories. In this case, it is classified as "National Institutions." The output is the classification result.

[0152] Step 6: Generating the answer

[0153] The server queries the FAQ database based on the classification results and retrieves relevant answer information. The input data is the classification results, and the retrieved answer information is input into a natural language generation engine (e.g., GPT-3) to reconstruct it into natural-sounding sentences. For example, a specific answer text such as "You can apply for childcare support funds by following these steps..." is generated. The output is the generated answer text.

[0154] Step 7: Submit your response

[0155] The server sends the generated response as an HTTP response in JSON format to the user's terminal. The input data is the generated response text, which is stored in the HTTP response payload and sent to the terminal. The output is this HTTP response.

[0156] Step 8: Display the answer

[0157] The terminal receives a response from the server and parses the answer data within the JSON payload. The input data is the received HTTP response, which is then displayed in the application's user interface after parsing. For example, it might display something like, "You can apply for childcare support funds by following these steps..." The output is the answer displayed in the user interface.

[0158] The above outlines the specific processing flow of the program in this system. This allows users to obtain quick and accurate answers.

[0159] (Application Example 1)

[0160] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0161] Currently, it is difficult for customers to get quick and accurate answers about product information and services in physical stores. In particular, when customers ask questions about specific products or services in stores, staff members are often busy, making it difficult to get an appropriate answer immediately. In such situations, customer satisfaction decreases, which can lead to a decline in sales. Therefore, there is a need for a system that allows customers to get quick and accurate answers to their questions in physical stores.

[0162] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0163] In this invention, the server includes means for analyzing a question using a natural language processing engine, means for classifying the question, means for obtaining relevant information from a database based on the classification, means for generating an answer from the obtained relevant information using a natural language generation engine, means for transmitting the generated answer to a user terminal, means for the user terminal to display the answer, and means for a user to input a question using a smartphone application in a physical store and receive and display the generated answer in real time. This makes it possible for customers in physical stores to obtain accurate answers about product information and services in real time.

[0164] A "system" is a collection of devices and software that perform a series of processes to allow a user to input a question and obtain an appropriate answer.

[0165] A "user terminal" is a device that a user uses to input questions and receive and display answers, and includes, for example, smartphones and tablets.

[0166] An "interface" refers to the means by which data is entered between the user and the system, such as a screen or input field for the user to enter a question.

[0167] A "server" is a central processing unit that analyzes questions submitted by users and generates answers, and it is equipped with a database and natural language processing and generation engines.

[0168] A "natural language processing engine" is a software module that analyzes user-inputted questions, extracts keywords and categories, and understands their meaning.

[0169] "Classification" is the process of assigning questions analyzed by a natural language processing engine to predefined categories.

[0170] A "database" is a storage system that stores information for retrieving related information based on the aforementioned classification.

[0171] A "natural language generation engine" is a software module that automatically generates natural-sounding response sentences based on acquired relevant information.

[0172] A "physical store" is a sales and service facility that provides goods and services in a physical location, where customers visit in person to make purchases.

[0173] A "smartphone application" is a software program that runs on a smartphone, allowing the user to input questions and receive generated answers.

[0174] "Real-time" refers to the timing at which a system processes user input immediately and provides an instant response.

[0175] This invention provides a system that offers quick and accurate answers to user questions in physical stores. This system mainly consists of the following components.

[0176] First, an interface for users to input questions is provided as a smartphone application. Users launch the application in a physical store and input their questions in text format.

[0177] Next, the user terminal sends the entered question to the server using the HTTPS protocol. Here, the server's role is crucial. The server is equipped with a natural language processing engine (for example, SpaCy or NLTK) that analyzes the received question and extracts keywords and sentence components.

[0178] The server then uses a machine learning model (e.g., random forest or support vector machine) to classify the questions into predefined categories. Based on this classification, it retrieves relevant information from the database.

[0179] Based on the acquired relevant information, a natural language generation engine (for example, a generative AI model such as GPT-3) generates a response in a format that is easy for the user to understand. This generated response is then sent to the user's terminal by the server in JSON format.

[0180] Finally, the user's device parses the received response and displays it on the smartphone application's user interface. This allows the user to obtain answers to their questions in real time.

[0181] As a concrete example, consider the case where a user asks, "How do I use this product?" When the user enters the question into their smartphone and sends it, the server analyzes and classifies it. The question is classified as "How to use," and information on "How to use this product" is retrieved from the database. Subsequently, an answer is generated using a generative AI model (GPT-3), and it is displayed on the user interface as, "This product can be used in the following ways..."

[0182] Examples of prompt statements include the following:

[0183] "Category: How to use\nQuestion: Please tell me how to use this product\nAnswer:"

[0184] In this way, a system is realized that provides immediate and accurate answers to user questions in physical stores.

[0185] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0186] Step 1:

[0187] The user enters a question. Specifically, the user enters the question in text format into the interface of a smartphone application. A question such as "How do I use this product?" is entered. In this step, the input is the user's question, and the output is the entered text-formatted question.

[0188] Step 2:

[0189] The user terminal sends the entered question to the server. Specifically, the user terminal uses the HTTPS protocol to send the user-entered question to the server. The input in this step is a text-based question, and the output is an HTTP request sent to the server.

[0190] Step 3:

[0191] The server receives and parses the question. Specifically, the server receives an HTTP request and uses a natural language processing engine (e.g., SpaCy or NLTK) to parse the question and extract keywords and sentence components. The input for this step is the question text of the HTTP request, and the output is the parsed keywords and sentence components.

[0192] Step 4:

[0193] The server classifies the questions. Specifically, the server uses a machine learning model (e.g., random forest or support vector machine) to classify the analyzed question sentences into predefined categories (e.g., usage, price, specifications, etc.). The input for this step is the keywords and sentence components of the analyzed question, and the output is the appropriate category label.

[0194] Step 5:

[0195] The server retrieves relevant information from the database based on the classification. Specifically, the server queries the database and retrieves information corresponding to the classified category. The input for this step is the category label, and the output is the relevant information retrieved from the database.

[0196] Step 6:

[0197] The server uses a natural language generation engine to generate answers based on the relevant information it has acquired. Specifically, it uses a generative AI model (e.g., GPT-3) to generate answers in a natural format based on the acquired information. The input for this step is relevant information acquired from a database, and the output is an answer generated in a way that is easy for the user to understand.

[0198] Step 7:

[0199] The server sends the generated response to the user's terminal. Specifically, it sends the generated response to the user's terminal as an HTTP response in JSON format. The input for this step is the generated response text, and the output is the HTTP response sent to the user's terminal.

[0200] Step 8:

[0201] The user terminal displays the response it received. Specifically, the user terminal receives an HTTP response, parses it, and displays the response on the user interface. It might be displayed in the format, "This product can be used in the following ways..." The input for this step is the HTTP response from the server, and the output is the response displayed on the user terminal.

[0202] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0203] This invention provides a system that quickly and appropriately answers questions that users ask to government or corporate contact points, and further incorporates an emotion engine that recognizes the user's emotions. The system's core process involves the user inputting a question, which is then sent to a server for analysis and classification, generating an appropriate answer, and providing it to the user. In addition, it includes a function to recognize the user's emotions and utilize the recognition results to provide even more appropriate answers.

[0204] Overall System Overview

[0205] This system mainly consists of the following components:

[0206] 1. User Interface

[0207] 2. The terminal that sends the question data to the server.

[0208] 3. Server that receives and analyzes question data

[0209] 4. Natural Language Processing and Generation Engines

[0210] 5. Database

[0211] 6. Emotional Engine

[0212] 7. Interface for sending and displaying responses on the user's terminal.

[0213] Program processing flow

[0214] Input Interface

[0215] The user launches the application on their PC or smartphone and enters a question into the text box. For example, they might enter the question, "How do I renew my SSL certificate?" and click the submit button.

[0216] Submit a question

[0217] The terminal sends the questions entered by the user to the server using the HTTPS protocol.

[0218] Question analysis

[0219] The server parses the received question in JSON format and uses a natural language processing engine (e.g., spaCy or NLTK) to analyze the question text. Here, the keywords "SSL certificate" and the verb "renewal" are extracted.

[0220] Question classification

[0221] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the questions into predefined categories. In this case, the questions are classified as "IT technical specifications."

[0222] Emotional analysis

[0223] The server passes the question to the sentiment engine, which recognizes the user's emotions. The sentiment engine analyzes the emotions (e.g., joy, frustration, anger) from the text entered by the user. The analysis results influence the classification of the question.

[0224] Answer generation

[0225] The server queries the FAQ database and retrieves the relevant answer information. For example, information regarding "how to renew an SSL certificate" is extracted from the database. Based on the extracted information, a natural language generation engine (e.g., GPT-3) generates text in an easy-to-understand format. Furthermore, considering the results of sentiment analysis, a more user-friendly answer is generated. As a result, the answer may take the form of, "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us at any time."

[0226] Submit your response

[0227] The server sends the generated response to the user's terminal as an HTTP response in JSON format. The response data is encoded before transmission.

[0228] Display the answer

[0229] The device parses the response data it receives. The parsed data is then dynamically inserted into the application's UI components.

[0230] Confirmation of the answer

[0231] The device displays the response in the user interface. The user can see the response: "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us."

[0232] Specific example

[0233] Example 1: Questions regarding IT technical specifications

[0234] A user types "How do I renew my SSL certificate?" on their smartphone and sends it. The device sends this question to the server, which analyzes and classifies the question. The question is classified as "IT technical specification," and the sentiment engine analyzes it as "the user is feeling anxious." Information on "how to renew your SSL certificate" is retrieved from the database, and an emotionally sensitive response is generated. Finally, the message "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us anytime." is displayed.

[0235] Example 2: Questions about national systems

[0236] A user types "How do I apply for childcare support funds?" on their PC and sends it. The device sends this question to the server, which analyzes and classifies the question. The question is classified as "National System," and the sentiment engine analyzes that "the user is feeling positive emotions." Information on "How to apply for childcare support funds" is retrieved from the database, and an emotionally sensitive response is generated. Finally, the message "How to apply for childcare support funds is as follows. We hope your application goes smoothly." is displayed.

[0237] The above describes the embodiments for carrying out the present invention. This system efficiently performs a series of processes from question analysis to answer generation and emotion analysis, enabling users to obtain quick and accurate answers.

[0238] The following describes the processing flow.

[0239] Step 1:

[0240] The user launches the application on their PC or smartphone and enters the question "How do I renew my SSL certificate?" into the interface for entering questions. Then they click the submit button.

[0241] Step 2:

[0242] The terminal sends the entered question to the server using the HTTPS protocol. The question data is encoded in JSON format.

[0243] Step 3:

[0244] The server parses the received question data into JSON format. The server temporarily receives this data and passes it on to the question analysis process.

[0245] Step 4:

[0246] The server uses a natural language processing engine (such as spaCy or NLTK) to perform morphological analysis on the question text. Specifically, it tokenizes the sentence and extracts keywords and important verbs. For example, the keyword "SSL certificate" and the verb "renewal" are extracted.

[0247] Step 5:

[0248] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the questions into predefined categories. In this case, the questions are classified as "IT technical specifications."

[0249] Step 6:

[0250] The server passes the question to the sentiment engine, which analyzes the user's emotions. The sentiment engine analyzes the question and determines that "the user is feeling anxious."

[0251] Step 7:

[0252] The server queries the FAQ database and retrieves the relevant answer information. Specifically, information regarding "how to renew SSL certificates" is extracted from the database.

[0253] Step 8:

[0254] The server uses a natural language generation engine (e.g., GPT-3) to generate a response based on the extracted data. Considering the results of the sentiment engine's analysis, wording that is considerate of users experiencing anxiety is added. The generated response will be in the form of "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us."

[0255] Step 9:

[0256] The server generates the response and sends it to the user's terminal as an HTTP response in JSON format. The response data is encoded.

[0257] Step 10:

[0258] The terminal parses the response data it receives. The parsed data is then dynamically inserted into the application's user interface.

[0259] Step 11:

[0260] The device displays the response in the user interface. The user can see the response: "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us."

[0261] (Example 2)

[0262] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0263] In today's information society, users ask a variety of questions seeking quick and accurate answers. However, conventional systems often suffer from delays in responding to user questions, inconsistent response quality, and particular difficulty in providing appropriate answers that take user emotions into consideration. Furthermore, there is a lack of means to generate appropriate answers that respond to emotions. As a result, the user experience fails to reach a satisfactory level, which is a challenge.

[0264] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0265] In this invention, the server includes means for analyzing a question using a natural language processing engine, means for classifying the question using a machine learning model, and means for performing sentiment analysis of the question using a sentiment engine. This makes it possible to quickly generate an appropriate answer based on the content of the question and the user's sentiment.

[0266] A "user" is an individual or group that uses the system to input questions and receive answers.

[0267] An "interface" is a means that includes screens and input devices for users to input questions and view answers.

[0268] A "server" is a computer system that receives questions from users, analyzes them, classifies them, generates answers, and sends them back.

[0269] A "natural language processing engine" is a software engine that analyzes question texts to understand their meaning, using language models to extract keywords and important phrases.

[0270] "Question classification" is the process of classifying analyzed question texts into predefined categories using a machine learning model.

[0271] An "emotion engine" is a software engine that detects emotions from questions entered by the user and performs analysis based on those emotions.

[0272] A "database" is an information system that stores answer information to questions, and the necessary information is extracted through queries.

[0273] A "natural language generation engine" is a software engine that generates answers in an easy-to-understand and appropriate format for the user based on acquired information.

[0274] The HTTPS protocol is a communication protocol for securely transmitting data over the internet, encrypting and protecting user information.

[0275] "Response generation" is the process by which the server generates responses to provide to the user using a natural language generation engine, taking into account information obtained from the database and the results of the sentiment engine's analysis.

[0276] "Displaying the answer" is the process of presenting the generated answer to the user's terminal in an easy-to-read format.

[0277] This invention provides a system that offers quick and appropriate answers to questions users make to government or corporate contact points, and further incorporates an emotion engine that recognizes the user's emotions. This system mainly consists of the following key components.

[0278] 1. User Interface

[0279] 2. The terminal that sends the question data to the server.

[0280] 3. Server that receives and analyzes question data

[0281] 4. Natural Language Processing Engine

[0282] 5. Database

[0283] 6. Emotion Engine

[0284] 7. Interface for Sending and Displaying Answers to User Terminals

[0285] The process starts when the user launches the application using a PC or smartphone terminal and enters a question in the text box. For example, enter a question such as "Please teach me how to update the SSL certificate" and click the send button. Thereby, the user's terminal sends this question to the server using the HTTPS protocol. The question received by the server is parsed in JSON format and analyzed using a natural language processing engine (e.g., spaCy or NLTK). Here, the keywords "SSL certificate" and the verb "update" are extracted.

[0286] Next, the server classifies the question using a machine learning model (e.g., random forest or support vector machine). It is classified into a specific category (in this case, "IT technical specifications"). Then, the server passes the question text to the emotion engine to recognize the user's emotion. The emotion engine analyzes the emotion (e.g., joy, dissatisfaction, anger, etc.) from the text input by the user. The result of this analysis affects the classification of the question and the generation of the answer.

[0287] After that, the server queries the database to obtain the corresponding answer information. For example, information about "how to update the SSL certificate" is extracted from the database. Based on this extracted information, a natural language generation engine (e.g., a generative AI model) is used to generate a text in an easy-to-understand format. Further, considering the result of the emotion analysis, an answer is generated in an appropriate format. Specifically, it will be something like "The update of the SSL certificate is performed in the following steps. If you have any questions, please feel free to contact us at any time."

[0288] Finally, the server sends the generated response to the user's device as an HTTP response in JSON format. The device parses the received response data, dynamically inserts it into the application's UI components, and displays it in a way that the user can verify.

[0289] Specific example

[0290] (Example 1: Questions regarding IT technical specifications)

[0291] A user types "How do I renew my SSL certificate?" on their smartphone and sends it. The device sends this question to the server, which analyzes and classifies the question. The question is classified as "IT technical specification," and the sentiment engine analyzes it as "the user is feeling anxious." Information on "how to renew your SSL certificate" is retrieved from the database, and an emotionally sensitive response is generated. Finally, the message "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us anytime." is displayed.

[0292] (Example 2: Questions about national systems)

[0293] A user types "How do I apply for childcare support funds?" on their PC and sends it. The device sends this question to the server, which analyzes and classifies the question. The question is classified as "National System," and the sentiment engine analyzes that "the user is feeling positive emotions." Information on "How to apply for childcare support funds" is retrieved from the database, and an emotionally sensitive response is generated. Finally, the message "How to apply for childcare support funds is as follows. We hope your application goes smoothly." is displayed.

[0294] Example of a prompt

[0295] Point down

[0296] Q: How do I renew my SSL certificate?

[0297] A: Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us.

[0298] This system can efficiently perform a series of processes, from question analysis to answer generation and sentiment analysis. Furthermore, by analyzing and responding to user emotions, it can provide answers that are more satisfying.

[0299] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0300] Step 1: The user enters a question.

[0301] The user launches the application on their PC or smartphone and enters a question into a text box. Specifically, they might enter the question, "How do I renew my SSL certificate?" and click the submit button. The user's question text serves as the input and is the starting point for the entire process. The output is the question text stored on the device.

[0302] Step 2: Submit your question

[0303] The terminal uses the HTTPS protocol to send user-entered questions to the server. Specifically, it is a process of encrypting the entered question text and securely sending it to the server. The input is the user's question text, and the output is the encrypted question data sent to the server.

[0304] Step 3: Receive and analyze the questions

[0305] The server parses the received question in JSON format. Next, a natural language processing engine (e.g., spaCy or NLTK) is used to analyze the question text. Here, the keywords "SSL certificate" and the verb "renewal" are extracted. The input is encrypted question data, and the output is the analysis results, such as keywords and phrases.

[0306] Step 4: Classify the questions

[0307] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the questions into predefined categories. The input is the analysis result, and the output is a category such as "IT technical specifications".

[0308] Step 5: Analyze the sentiment

[0309] The server passes the question text to the sentiment engine to recognize the user's sentiment. The sentiment engine analyzes the sentiment (e.g., joy, dissatisfaction, anger, etc.) from the text input by the user. The input is the question text itself and the analysis result, and the output is the sentiment analysis result such as "uneasy" or "confused".

[0310] Step 6: Retrieve relevant information from the database

[0311] The server queries the FAQ database to obtain the corresponding answer information. For example, information about "how to update the SSL certificate" is extracted from the database. The input is the question category and the sentiment analysis result, and the output is the relevant answer information.

[0312] Step 7: Generate an answer

[0313] The server uses a natural language generation engine (e.g., a generative AI model) based on the obtained information to generate text in an easy-to-understand format. Furthermore, considering the result of the sentiment analysis, an answer text that is more considerate of the user's sentiment is generated. The input is the obtained answer information and the sentiment analysis result, and the output is an answer text that takes sentiment into account. Specifically, it will be in the form of "The SSL certificate can be updated as follows. If you have any questions, please feel free to contact us at any time."

[0314] Step 8: Send the answer <000099​The server sends the generated response to the user's terminal as an HTTP response in JSON format. The data is encoded before transmission. The input is the generated response text, and the output is the encoded response data delivered to the user's terminal.

[0316] Step 9: Show your answer

[0317] The device parses the received response data. The parsed data is dynamically inserted into the application's UI components and displayed in a format viewable by the user. The input is encoded response data, and the output is a viewable response text displayed on the device screen.

[0318] Step 10: Check your answers

[0319] The user reviews the response. Specifically, by viewing the response, "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us," the user can obtain the information they were looking for. The input is the displayed response, and the output is the user reviewing its contents.

[0320] (Application Example 2)

[0321] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0322] Traditional systems suffered from a decline in the quality of the user experience because they did not consider the user's emotions during the process from when the user entered a question to when they received an answer. Furthermore, in stressful environments such as factories, it is necessary to appropriately handle user anxiety and dissatisfaction. However, with current technology, it has been difficult to recognize user emotions and provide answers that respond accordingly.

[0323] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for providing an interface for the user to input a question, means for transmitting the question to the server, means for the server to analyze the question using a natural language processing engine, means for classifying the question, means for obtaining relevant information from a database based on the classification, means for generating an answer from the obtained relevant information using a natural language generation engine, means for transmitting the generated answer to a user terminal, means for the user terminal to display the answer, means for recognizing the user's emotions from the input question, and means for generating an answer considering the emotion recognition result. This makes it possible to improve the quality of the user experience by recognizing the user's emotions and providing an answer accordingly.

[0324] "User interface" refers to the screens and operating methods used by users to input questions.

[0325] A "server" refers to a computer system that receives and processes requests from clients via a network.

[0326] A "natural language processing engine" refers to a system that includes algorithms and models for analyzing text data and understanding its meaning.

[0327] "Question classification" refers to the process of categorizing user questions into specific categories or tags.

[0328] A "database" refers to a collection of information that is systematically organized and stored, making it easy to search and retrieve.

[0329] A "natural language generation engine" refers to a system that includes algorithms and models for generating responses in a natural format based on acquired relevant information.

[0330] "Emotion recognition" refers to the process of analyzing and identifying emotions (such as joy, dissatisfaction, or anger) from text entered by the user.

[0331] A "user terminal" refers to a device (such as a smartphone, tablet, or PC) used by a user to access the server and exchange questions and answers.

[0332] This invention relates to an interactive QA system for factory robots. This system allows factory workers to ask questions about robot operation and maintenance, and provides appropriate and prompt answers. It also has a function to recognize the user's emotions and generate corresponding responses.

[0333] System Configuration

[0334] This system consists of the following components.

[0335] 1. User Interface: An interface for the worker to input questions. Examples include using smart glasses or voice input devices.

[0336] 2. Server: A central processing unit that receives questions, analyzes them, and generates answers. The server includes a natural language processing engine (e.g., spaCy), an emotion recognition engine (e.g., Hugging Face Transformers), and a natural language generation engine (e.g., OpenAI® GPT-3).

[0337] 3. Database: A collection of information that stores relevant information for a query. Examples include PostgreSQL and MySQL®.

[0338] 4. User terminal: A device that displays answers to the worker. This includes smartphones, tablets, and PCs.

[0339] Program processing flow

[0340] User Interface

[0341] The worker wears smart glasses and inputs a question by voice, such as, "Please tell me how to maintain the robot."

[0342] Submit a question

[0343] The smart glasses convert the voice data into text format and send the question to the server. For speech recognition, Google® Cloud Speech-to-Text API or similar services can be used.

[0344] Question analysis

[0345] The server analyzes the received question using a natural language processing engine (e.g., spaCy). Keywords such as "maintenance" and "robot" are extracted from the text data.

[0346] Question classification

[0347] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the question. The question is classified as "technical support".

[0348] emotion recognition

[0349] The server uses an emotion analysis engine (e.g., Hugging Face Transformers) to recognize the worker's emotions from the input text. The emotion analysis result is determined to be "anxiety."

[0350] Answer generation

[0351] The server retrieves relevant information from the database and generates a response using a natural language generation engine (e.g., OpenAI GPT-3). The response is generated in the format of "Robot maintenance is performed using the following steps. If you have any questions, please contact support."

[0352] Submit your response

[0353] The server sends the generated response to the user's terminal. The user's terminal displays the received response so that the worker can confirm it.

[0354] Specific example

[0355] Example 1: Questions about robot maintenance

[0356] If a worker asks, "How do I calibrate the robot arm?", the system will operate as follows: The question is analyzed, relevant information is retrieved from the database, and, taking sentiment into consideration, an answer is generated stating, "The robot arm calibration procedure is as follows. If you have any questions, please contact support."

[0357] Example of a prompt

[0358] The user is asking about the calibration method for a robot arm. Emotion detected: Anxiety. Provide a detailed but friendly response.

[0359] This invention recognizes user emotions and provides corresponding responses, enabling factory workers to quickly obtain information and improve work efficiency. This, in turn, enhances overall factory productivity and reduces anxiety and stress.

[0360] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0361] Step 1:

[0362] The user wears smart glasses and uses a voice input device to enter a question. For example, they might ask, "How do I calibrate the robot arm?" The smart glasses capture the voice data as input and convert it into text data. The converted text data is then output.

[0363] Step 2:

[0364] The terminal sends the text data converted from the speech to the server using the HTTPS protocol. The server receives this text data and takes it as input. The output is a request body containing the text data.

[0365] Step 3:

[0366] The server analyzes the received text data using a natural language processing engine (e.g., spaCy). Specifically, it splits the text data and extracts keywords and important phrases. This process produces a list of analyzed keywords as output.

[0367] Step 4:

[0368] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the analyzed text data into a specific category. A keyword list is used as input, and the output category is "Technical Support".

[0369] Step 5:

[0370] The server uses an emotion analysis engine (e.g., Hugging Face Transformers) to recognize the worker's emotions from the input text. In this step, text data is used as input, and the emotion analysis result (e.g., "anxious") is output.

[0371] Step 6:

[0372] The server retrieves relevant information from the database. The classified categories and keywords are used as input, and a database query is executed to retrieve the necessary information. The output is the relevant information (e.g., "Calibration Method").

[0373] Step 7:

[0374] The server uses a natural language generation engine (e.g., OpenAI GPT-3) to generate a response based on the acquired relevant information and sentiment recognition results. This involves inputting a prompt sentence into a generation AI model, and obtaining the generated response text as its output. For example, the prompt sentence is "The user is asking about the calibration method for a robot arm. Emotion detected: Anxiety. Provide a detailed but friendly response."

[0375] Step 8:

[0376] The server sends the generated response to the user's terminal. The HTTPS protocol is used for transmission, and the generated response text is included as output.

[0377] Step 9:

[0378] The user's device displays the received response. The response text is displayed on the smart glasses or smartphone screen, allowing the user to confirm the answer to the question. In this step, the response text is passed to the device as input and is visually displayed as output.

[0379] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0380] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0381] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0382] [Second Embodiment]

[0383] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0384] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0385] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0386] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0387] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0388] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0389] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0390] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0391] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0392] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0393] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0394] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0395] This invention provides a system that allows users to quickly and accurately obtain answers to questions at government and corporate service counters. The system centers on a series of processes in which the user inputs a question, which is then sent to a server for analysis and classification, and an appropriate answer is generated and provided to the user.

[0396] Overall System Overview

[0397] This system mainly consists of the following components:

[0398] 1. User Interface

[0399] 2. The terminal that sends the question data to the server.

[0400] 3. Server that receives and analyzes question data

[0401] 4. Natural Language Processing and Generation Engines

[0402] 5. Database

[0403] 6. Interface for sending and displaying responses on the user's terminal.

[0404] Program processing flow

[0405] Input Interface

[0406] The user launches the application provided on their PC or smartphone and enters a question into the text box. For example, after entering the question, "How do I apply for childcare support funds?", they click the submit button.

[0407] Submit a question

[0408] The terminal sends the questions entered by the user to the server using the HTTPS protocol.

[0409] Question analysis

[0410] The server parses the received question in JSON format and uses a natural language processing engine (e.g., spaCy or NLTK) to analyze the question text. Here, the keyword "childcare support funds" and the action "how to apply" are extracted.

[0411] Question classification

[0412] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the question into a predefined category. In this case, the question is classified as "national system."

[0413] Answer generation

[0414] The server queries the FAQ database and retrieves the relevant answer information. For example, "How to apply for childcare support funds" is extracted from the database. Based on the extracted information, a natural language generation engine (e.g., GPT-3) generates text in an easy-to-understand format. This results in the answer being "You can apply for childcare support funds by following these steps..."

[0415] Submit your response

[0416] The server sends the generated response to the user's terminal as an HTTP response in JSON format.

[0417] Display the answer

[0418] The device parses the received response data and displays it in the application's user interface. The user can see the response, "You can apply for childcare support funds by following these steps..."

[0419] Specific example

[0420] Example 1: Questions regarding IT technical specifications

[0421] A user types "How do I renew my SSL certificate?" on their smartphone and sends it. The device sends this question to the server, which analyzes and categorizes the question. If the question is classified as "IT technical specifications," information on "how to renew an SSL certificate" is retrieved from the database. A natural language generation engine generates an answer, which is displayed on the user interface as "To renew your SSL certificate, follow these steps...".

[0422] Example 2: Questions about national systems

[0423] A user types "How do I apply for childcare support funds?" on their PC and sends it. The terminal sends this question to the server, which analyzes and classifies the question. The question is classified as "National System," and information on "How to apply for childcare support funds" is retrieved from the database. A natural language generation engine generates an answer, which is displayed on the user interface as "You can apply for childcare support funds by following these steps...".

[0424] The above describes the embodiments for carrying out the present invention. This system efficiently performs a series of processes from question analysis to answer generation, enabling users to obtain quick and accurate answers.

[0425] The following describes the processing flow.

[0426] Step 1:

[0427] The user launches the application on their PC or smartphone and enters a question into the text box. For example, they might enter the question, "How do I apply for childcare support funds?" and click the submit button.

[0428] Step 2:

[0429] The terminal sends the entered question to the server as an HTTP POST request using the HTTPS protocol. The question data is encoded in JSON format during this process.

[0430] Step 3:

[0431] The server parses the received question data into JSON format. The server temporarily stores this data and passes it to a natural language processing engine (e.g., spaCy or NLTK).

[0432] Step 4:

[0433] The server uses a natural language processing engine to perform morphological analysis on the question text. Specifically, it analyzes the sentence and extracts keywords and important verbs. For example, keywords such as "childcare support funds" and "application method" are extracted.

[0434] Step 5:

[0435] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the question into a predefined category. For example, the question might be classified as "national systems."

[0436] Step 6:

[0437] The server queries the corresponding FAQ database based on the classification result. The server searches for and extracts relevant information. In this case, information regarding "how to apply for childcare support funds" is retrieved from the database.

[0438] Step 7:

[0439] Based on the data acquired by the server, a natural language generation engine (e.g., GPT-3) is used to generate a response in a user-friendly format. For example, it might generate a sentence like, "To apply for childcare support funds, follow these steps..."

[0440] Step 8:

[0441] The server generates the response and sends it to the user's terminal as an HTTP response in JSON format. The response data is encoded before transmission.

[0442] Step 9:

[0443] The device parses the response data it receives. The parsed data is then dynamically inserted into the application's UI components.

[0444] Step 10:

[0445] The device displays the answer in the user interface. The user can see the answer, which reads, "To apply for childcare support funds, follow these steps..."

[0446] (Example 1)

[0447] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0448] Conventional question-answering systems have struggled to provide quick and accurate answers to user-inputted questions. This is because the processes required for question analysis, classification, and optimal answer generation are complex, and appropriate means are lacking for each stage of the process. The present invention aims to solve these problems and provide a system that can provide users with quick and accurate answers.

[0449] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0450] In this invention, the server includes means for receiving and parsing questions in JSON format, means for analyzing them using a natural language processing engine, and means for classifying them using a machine learning model. This makes it possible to process questions in a consistent flow from reception to analysis and classification.

[0451] A "user" is a person or entity that uses this system to input questions and obtain answers.

[0452] "Interface" refers to the screen or input area of ​​an application where a user enters a question.

[0453] A "server" is a computing system that receives, analyzes, classifies, and generates answers to questions.

[0454] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a format for representing data in a structured manner.

[0455] "Parsing" refers to the process of analyzing data and converting it into a specific format.

[0456] A "natural language processing engine" is software and algorithms used to analyze natural language text and understand its meaning and context.

[0457] A "machine learning model" refers to an algorithm that learns patterns and knowledge from large amounts of data and uses that knowledge to make predictions and classifications on new data.

[0458] A "database" is a data storage system used to centrally manage answer information and related data for questions.

[0459] A "natural language generation engine" is software and algorithms used to convert information into easily understandable text.

[0460] The "HTTPS protocol" is a protocol used to encrypt data in web communications, and is a technology that enables secure data transmission and reception.

[0461] This invention provides a system that allows users to quickly and accurately obtain answers to questions at government and corporate service counters. The system centers on a series of processes in which the user inputs a question, which is then sent to a server for analysis and classification, and an appropriate answer is generated and provided to the user.

[0462] Overall System Overview

[0463] This system mainly consists of the following components:

[0464] 1. User Interface

[0465] 2. The terminal that sends the question data to the server.

[0466] 3. Server that receives and analyzes question data

[0467] 4. Natural Language Processing Engine

[0468] 5. Machine Learning Models

[0469] 6. Database

[0470] 7. Natural Language Generation Engine

[0471] 8. Interface for sending and displaying responses on the user's terminal.

[0472] User question input

[0473] Users can launch the application provided on a device such as a PC or smartphone and enter a question into a text box. For example, they can enter the question, "How do I apply for childcare support funds?" and then click the submit button. The entered question is temporarily stored in the device's memory.

[0474] Submit a question

[0475] The device sends the user-entered question to the server using the HTTPS protocol. Specifically, the question data is included as a JSON payload in the HTTP request and sent to the server's specified URL using the POST method.

[0476] Receiving and parsing questions

[0477] The server receives an HTTP request sent from the terminal. The server parses the received payload in JSON format and stores it in a database or temporary storage on the server.

[0478] Question analysis

[0479] The server passes the stored question data to a natural language processing engine (e.g., spaCy or NLTK) for text analysis. Here, the noun phrase "childcare support funds" and the verb phrase "application method" are extracted. Specifically, processes such as tokenization, part-of-speech tagging, and dependency parsing are performed.

[0480] Question classification

[0481] The server inputs the analyzed data into a machine learning model (e.g., random forest or support vector machine) and classifies it into a predefined category (in this case, "national systems"). This classification result is then added to the question data.

[0482] Answer generation

[0483] The server queries the FAQ database based on the classification results and retrieves answer information for the relevant questions. As a result of the query, information such as "How to apply for childcare support funds" is extracted. Next, a natural language generation engine (e.g., GPT-3) reconstructs the extracted information into natural-sounding sentences. At this stage, a sentence such as "You can apply for childcare support funds by following these steps..." is generated.

[0484] Submit your response

[0485] The server sends the generated response to the user's terminal as an HTTP response in JSON format. The server creates a response including HTTP headers and payload and sends it to the terminal.

[0486] Display the answer

[0487] The device receives a response from the server and parses the answer data in the payload. The parsed data is displayed in the application's user interface. The user can then see the answer, "You can apply for childcare support funds by following these steps...", on the screen.

[0488] Explanation of specific examples

[0489] Example 1: Questions regarding IT technical specifications

[0490] A user types "How do I renew my SSL certificate?" on their smartphone and sends it. The device sends this question to the server, which analyzes and categorizes the question. If the question is classified as "IT technical specifications," information on "how to renew an SSL certificate" is retrieved from the database. A natural language generation engine generates an answer, which is displayed on the user interface as "To renew your SSL certificate, follow these steps...".

[0491] Example 2: Questions about national systems

[0492] A user types "How do I apply for childcare support funds?" on their PC and sends it. The terminal sends this question to the server, which analyzes and classifies the question. The question is classified as "National System," and information on "How to apply for childcare support funds" is retrieved from the database. A natural language generation engine generates an answer, which is displayed on the user interface as "You can apply for childcare support funds by following these steps...".

[0493] Examples of prompts for generative AI models

[0494] Example of a prompt message for a generative AI model (e.g., GPT-3):

[0495] Question: How do I apply for childcare support funds?

[0496] Previous text in the model: Childcare support grants can be applied for by following these steps…

[0497] This prompt is used to generate specific text that provides additional information for the generated AI model.

[0498] The above describes the embodiments for carrying out the present invention. By using this system, a series of processes from question analysis to answer generation can be performed efficiently, enabling users to obtain quick and accurate answers.

[0499] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0500] Step 1: User Question Input

[0501] The user opens the application on their PC or smartphone and enters a question into the text box. The entered question might be text data such as, "Please tell me how to apply for childcare support funds." Clicking the submit button prepares the input data needed to proceed to the next step.

[0502] Step 2: Submit your question

[0503] The terminal sends the user-entered question to the server using the HTTPS protocol. The input data is a text-based question, which is stored as a JSON payload in the body of the HTTP request and sent to the server's specified URL using the POST method. The output is this HTTP request.

[0504] Step 3: Receiving and Parsing Questions

[0505] The server receives an HTTP request sent from the terminal. The received data is parsed in JSON format, and the text data of the question is extracted. This extracted text data becomes the input data used for analysis in the next step. The output is the parseable text data of the question.

[0506] Step 4: Analyzing the Question

[0507] The server passes the extracted text data of the questions to a natural language processing engine (e.g., spaCy or NLTK) for text analysis. The input data is the extracted text data, which undergoes processes such as tokenization, part-of-speech tagging, and dependency parsing to obtain, for example, the noun phrase "childcare support funds" and the verb phrase "application method". The output is structural information of the analyzed text data.

[0508] Step 5: Classifying Questions

[0509] The server inputs the structural information of the analyzed text data into a machine learning model (e.g., Random Forest or Support Vector Machine) to perform categorization. The input data is the structural information of the analysis results, which is classified into predefined categories. In this case, it is classified as "National Institutions." The output is the classification result.

[0510] Step 6: Generating the answer

[0511] The server queries the FAQ database based on the classification results and retrieves relevant answer information. The input data is the classification results, and the retrieved answer information is input into a natural language generation engine (e.g., GPT-3) to reconstruct it into natural-sounding sentences. For example, a specific answer text such as "You can apply for childcare support funds by following these steps..." is generated. The output is the generated answer text.

[0512] Step 7: Submit your response

[0513] The server sends the generated response as an HTTP response in JSON format to the user's terminal. The input data is the generated response text, which is stored in the HTTP response payload and sent to the terminal. The output is this HTTP response.

[0514] Step 8: Display the answer

[0515] The terminal receives a response from the server and parses the answer data within the JSON payload. The input data is the received HTTP response, which is then displayed in the application's user interface after parsing. For example, it might display something like, "You can apply for childcare support funds by following these steps..." The output is the answer displayed in the user interface.

[0516] The above outlines the specific processing flow of the program in this system. This allows users to obtain quick and accurate answers.

[0517] (Application Example 1)

[0518] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0519] Currently, it is difficult for customers to get quick and accurate answers about product information and services in physical stores. In particular, when customers ask questions about specific products or services in stores, staff members are often busy, making it difficult to get an appropriate answer immediately. In such situations, customer satisfaction decreases, which can lead to a decline in sales. Therefore, there is a need for a system that allows customers to get quick and accurate answers to their questions in physical stores.

[0520] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0521] In this invention, the server includes means for analyzing a question using a natural language processing engine, means for classifying the question, means for obtaining relevant information from a database based on the classification, means for generating an answer from the obtained relevant information using a natural language generation engine, means for transmitting the generated answer to a user terminal, means for the user terminal to display the answer, and means for a user to input a question using a smartphone application in a physical store and receive and display the generated answer in real time. This makes it possible for customers in physical stores to obtain accurate answers about product information and services in real time.

[0522] A "system" is a collection of devices and software that perform a series of processes to allow a user to input a question and obtain an appropriate answer.

[0523] A "user terminal" is a device that a user uses to input questions and receive and display answers, and includes, for example, smartphones and tablets.

[0524] An "interface" refers to the means by which data is entered between the user and the system, such as a screen or input field for the user to enter a question.

[0525] A "server" is a central processing unit that analyzes questions submitted by users and generates answers, and it is equipped with a database and natural language processing and generation engines.

[0526] A "natural language processing engine" is a software module that analyzes user-inputted questions, extracts keywords and categories, and understands their meaning.

[0527] "Classification" is the process of assigning questions analyzed by a natural language processing engine to predefined categories.

[0528] A "database" is a storage system that stores information for retrieving related information based on the aforementioned classification.

[0529] A "natural language generation engine" is a software module that automatically generates natural-sounding response sentences based on acquired relevant information.

[0530] A "physical store" is a sales and service facility that provides goods and services in a physical location, where customers visit in person to make purchases.

[0531] A "smartphone application" is a software program that runs on a smartphone, allowing the user to input questions and receive generated answers.

[0532] "Real-time" refers to the timing at which a system processes user input immediately and provides an instant response.

[0533] This invention provides a system that offers quick and accurate answers to user questions in physical stores. This system mainly consists of the following components.

[0534] First, an interface for users to input questions is provided as a smartphone application. Users launch the application in a physical store and input their questions in text format.

[0535] Next, the user terminal sends the entered question to the server using the HTTPS protocol. Here, the server's role is crucial. The server is equipped with a natural language processing engine (for example, SpaCy or NLTK) that analyzes the received question and extracts keywords and sentence components.

[0536] The server then uses a machine learning model (e.g., random forest or support vector machine) to classify the questions into predefined categories. Based on this classification, it retrieves relevant information from the database.

[0537] Based on the acquired relevant information, a natural language generation engine (for example, a generative AI model such as GPT-3) generates a response in a format that is easy for the user to understand. This generated response is then sent to the user's terminal by the server in JSON format.

[0538] Finally, the user's device parses the received response and displays it on the smartphone application's user interface. This allows the user to obtain answers to their questions in real time.

[0539] As a concrete example, consider the case where a user asks, "How do I use this product?" When the user enters the question into their smartphone and sends it, the server analyzes and classifies it. The question is classified as "How to use," and information on "How to use this product" is retrieved from the database. Subsequently, an answer is generated using a generative AI model (GPT-3), and it is displayed on the user interface as, "This product can be used in the following ways..."

[0540] Examples of prompt statements include the following:

[0541] "Category: How to use\nQuestion: Please tell me how to use this product\nAnswer:"

[0542] In this way, a system is realized that provides immediate and accurate answers to user questions in physical stores.

[0543] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0544] Step 1:

[0545] The user enters a question. Specifically, the user enters the question in text format into the interface of a smartphone application. A question such as "How do I use this product?" is entered. In this step, the input is the user's question, and the output is the entered text-formatted question.

[0546] Step 2:

[0547] The user terminal sends the entered question to the server. Specifically, the user terminal uses the HTTPS protocol to send the user-entered question to the server. The input in this step is a text-based question, and the output is an HTTP request sent to the server.

[0548] Step 3:

[0549] The server receives and parses the question. Specifically, the server receives an HTTP request and uses a natural language processing engine (e.g., SpaCy or NLTK) to parse the question and extract keywords and sentence components. The input for this step is the question text of the HTTP request, and the output is the parsed keywords and sentence components.

[0550] Step 4:

[0551] The server classifies the questions. Specifically, the server uses a machine learning model (e.g., random forest or support vector machine) to classify the analyzed question sentences into predefined categories (e.g., usage, price, specifications, etc.). The input for this step is the keywords and sentence components of the analyzed question, and the output is the appropriate category label.

[0552] Step 5:

[0553] The server retrieves relevant information from the database based on the classification. Specifically, the server queries the database and retrieves information corresponding to the classified category. The input for this step is the category label, and the output is the relevant information retrieved from the database.

[0554] Step 6:

[0555] The server uses a natural language generation engine to generate answers based on the relevant information it has acquired. Specifically, it uses a generative AI model (e.g., GPT-3) to generate answers in a natural format based on the acquired information. The input for this step is relevant information acquired from a database, and the output is an answer generated in a way that is easy for the user to understand.

[0556] Step 7:

[0557] The server sends the generated response to the user's terminal. Specifically, it sends the generated response to the user's terminal as an HTTP response in JSON format. The input for this step is the generated response text, and the output is the HTTP response sent to the user's terminal.

[0558] Step 8:

[0559] The user terminal displays the response it received. Specifically, the user terminal receives an HTTP response, parses it, and displays the response on the user interface. It might be displayed in the format, "This product can be used in the following ways..." The input for this step is the HTTP response from the server, and the output is the response displayed on the user terminal.

[0560] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0561] This invention provides a system that quickly and appropriately answers questions that users ask to government or corporate contact points, and further incorporates an emotion engine that recognizes the user's emotions. The system's core process involves the user inputting a question, which is then sent to a server for analysis and classification, generating an appropriate answer, and providing it to the user. In addition, it includes a function to recognize the user's emotions and utilize the recognition results to provide even more appropriate answers.

[0562] Overall System Overview

[0563] This system mainly consists of the following components:

[0564] 1. User Interface

[0565] 2. The terminal that sends the question data to the server.

[0566] 3. Server that receives and analyzes question data

[0567] 4. Natural Language Processing and Generation Engines

[0568] 5. Database

[0569] 6. Emotional Engine

[0570] 7. Interface for sending and displaying responses on the user's terminal.

[0571] Program processing flow

[0572] Input Interface

[0573] The user launches the application on their PC or smartphone and enters a question into the text box. For example, they might enter the question, "How do I renew my SSL certificate?" and click the submit button.

[0574] Submit a question

[0575] The terminal sends the questions entered by the user to the server using the HTTPS protocol.

[0576] Question analysis

[0577] The server parses the received question in JSON format and uses a natural language processing engine (e.g., spaCy or NLTK) to analyze the question text. Here, the keywords "SSL certificate" and the verb "renewal" are extracted.

[0578] Question classification

[0579] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the questions into predefined categories. In this case, the questions are classified as "IT technical specifications."

[0580] Emotional analysis

[0581] The server passes the question to the sentiment engine, which recognizes the user's emotions. The sentiment engine analyzes the emotions (e.g., joy, frustration, anger) from the text entered by the user. The analysis results influence the classification of the question.

[0582] Answer generation

[0583] The server queries the FAQ database and retrieves the relevant answer information. For example, information regarding "how to renew an SSL certificate" is extracted from the database. Based on the extracted information, a natural language generation engine (e.g., GPT-3) generates text in an easy-to-understand format. Furthermore, considering the results of sentiment analysis, a more user-friendly answer is generated. As a result, the answer may take the form of, "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us at any time."

[0584] Submit your response

[0585] The server sends the generated response to the user's terminal as an HTTP response in JSON format. The response data is encoded before transmission.

[0586] Display the answer

[0587] The device parses the response data it receives. The parsed data is then dynamically inserted into the application's UI components.

[0588] Confirmation of the answer

[0589] The device displays the response in the user interface. The user can see the response: "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us."

[0590] Specific example

[0591] Example 1: Questions regarding IT technical specifications

[0592] A user types "How do I renew my SSL certificate?" on their smartphone and sends it. The device sends this question to the server, which analyzes and classifies the question. The question is classified as "IT technical specification," and the sentiment engine analyzes it as "the user is feeling anxious." Information on "how to renew your SSL certificate" is retrieved from the database, and an emotionally sensitive response is generated. Finally, the message "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us anytime." is displayed.

[0593] Example 2: Questions about national systems

[0594] A user types "How do I apply for childcare support funds?" on their PC and sends it. The device sends this question to the server, which analyzes and classifies the question. The question is classified as "National System," and the sentiment engine analyzes that "the user is feeling positive emotions." Information on "How to apply for childcare support funds" is retrieved from the database, and an emotionally sensitive response is generated. Finally, the message "How to apply for childcare support funds is as follows. We hope your application goes smoothly." is displayed.

[0595] The above describes the embodiments for carrying out the present invention. This system efficiently performs a series of processes from question analysis to answer generation and emotion analysis, enabling users to obtain quick and accurate answers.

[0596] The following describes the processing flow.

[0597] Step 1:

[0598] The user launches the application on their PC or smartphone and enters the question "How do I renew my SSL certificate?" into the interface for entering questions. Then they click the submit button.

[0599] Step 2:

[0600] The terminal sends the entered question to the server using the HTTPS protocol. The question data is encoded in JSON format.

[0601] Step 3:

[0602] The server parses the received question data into JSON format. The server temporarily receives this data and passes it on to the question analysis process.

[0603] Step 4:

[0604] The server uses a natural language processing engine (such as spaCy or NLTK) to perform morphological analysis on the question text. Specifically, it tokenizes the sentence and extracts keywords and important verbs. For example, the keyword "SSL certificate" and the verb "renewal" are extracted.

[0605] Step 5:

[0606] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the questions into predefined categories. In this case, the questions are classified as "IT technical specifications."

[0607] Step 6:

[0608] The server passes the question to the sentiment engine, which analyzes the user's emotions. The sentiment engine analyzes the question and determines that "the user is feeling anxious."

[0609] Step 7:

[0610] The server queries the FAQ database and retrieves the relevant answer information. Specifically, information regarding "how to renew SSL certificates" is extracted from the database.

[0611] Step 8:

[0612] The server uses a natural language generation engine (e.g., GPT-3) to generate a response based on the extracted data. Considering the results of the sentiment engine's analysis, wording that is considerate of users experiencing anxiety is added. The generated response will be in the form of "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us."

[0613] Step 9:

[0614] The server generates the response and sends it to the user's terminal as an HTTP response in JSON format. The response data is encoded.

[0615] Step 10:

[0616] The terminal parses the response data it receives. The parsed data is then dynamically inserted into the application's user interface.

[0617] Step 11:

[0618] The device displays the response in the user interface. The user can see the response: "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us."

[0619] (Example 2)

[0620] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0621] In today's information society, users ask a variety of questions seeking quick and accurate answers. However, conventional systems often suffer from delays in responding to user questions, inconsistent response quality, and particular difficulty in providing appropriate answers that take user emotions into consideration. Furthermore, there is a lack of means to generate appropriate answers that respond to emotions. As a result, the user experience fails to reach a satisfactory level, which is a challenge.

[0622] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0623] In this invention, the server includes means for analyzing a question using a natural language processing engine, means for classifying the question using a machine learning model, and means for performing sentiment analysis of the question using a sentiment engine. This makes it possible to quickly generate an appropriate answer based on the content of the question and the user's sentiment.

[0624] A "user" is an individual or group that uses the system to input questions and receive answers.

[0625] An "interface" is a means that includes screens and input devices for users to input questions and view answers.

[0626] A "server" is a computer system that receives questions from users, analyzes them, classifies them, generates answers, and sends them back.

[0627] A "natural language processing engine" is a software engine that analyzes question texts to understand their meaning, using language models to extract keywords and important phrases.

[0628] "Question classification" is the process of classifying analyzed question texts into predefined categories using a machine learning model.

[0629] An "emotion engine" is a software engine that detects emotions from questions entered by the user and performs analysis based on those emotions.

[0630] A "database" is an information system that stores answer information to questions, and the necessary information is extracted through queries.

[0631] A "natural language generation engine" is a software engine that generates answers in an easy-to-understand and appropriate format for the user based on acquired information.

[0632] The HTTPS protocol is a communication protocol for securely transmitting data over the internet, encrypting and protecting user information.

[0633] "Response generation" is the process by which the server generates responses to provide to the user using a natural language generation engine, taking into account information obtained from the database and the results of the sentiment engine's analysis.

[0634] "Displaying the answer" is the process of presenting the generated answer to the user's terminal in an easy-to-read format.

[0635] This invention provides a system that offers quick and appropriate answers to questions users make to government or corporate contact points, and further incorporates an emotion engine that recognizes the user's emotions. This system mainly consists of the following key components.

[0636] 1. User Interface

[0637] 2. The terminal that sends the question data to the server.

[0638] 3. Server that receives and analyzes question data

[0639] 4. Natural Language Processing Engine

[0640] 5. Database

[0641] 6. Emotional Engine

[0642] 7. Interface for sending and displaying responses on the user's terminal.

[0643] The process begins when a user launches the application on their PC or smartphone and enters a question into a text box. For example, they might enter the question, "How do I renew my SSL certificate?" and click the submit button. This causes the user's device to send the question to the server using the HTTPS protocol. The server parses the received question in JSON format and analyzes it using a natural language processing engine (e.g., spaCy or NLTK). Here, the keywords "SSL certificate" and the verb "renew" are extracted.

[0644] Next, the server uses a machine learning model (e.g., random forest or support vector machine) to classify the question. It is then categorized into a specific category (in this case, "IT technical specifications"). The server then passes the question to the sentiment engine to recognize the user's emotions. The sentiment engine analyzes the emotions (e.g., joy, frustration, anger, etc.) from the text entered by the user. This analysis influences the classification of the question and the generation of answers.

[0645] The server then queries the database to retrieve the relevant response information. For example, information regarding "how to renew an SSL certificate" might be extracted from the database. Based on this extracted information, a natural language generation engine (e.g., a generative AI model) is used to generate text in an easily understandable format. Furthermore, the response is generated in an appropriate format, taking into account the results of sentiment analysis. Specifically, the response might be something like, "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us."

[0646] Finally, the server sends the generated response to the user's device as an HTTP response in JSON format. The device parses the received response data, dynamically inserts it into the application's UI components, and displays it in a way that the user can verify.

[0647] Specific example

[0648] (Example 1: Questions regarding IT technical specifications)

[0649] A user types "How do I renew my SSL certificate?" on their smartphone and sends it. The device sends this question to the server, which analyzes and classifies the question. The question is classified as "IT technical specification," and the sentiment engine analyzes it as "the user is feeling anxious." Information on "how to renew your SSL certificate" is retrieved from the database, and an emotionally sensitive response is generated. Finally, the message "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us anytime." is displayed.

[0650] (Example 2: Questions about national systems)

[0651] A user types "How do I apply for childcare support funds?" on their PC and sends it. The device sends this question to the server, which analyzes and classifies the question. The question is classified as "National System," and the sentiment engine analyzes that "the user is feeling positive emotions." Information on "How to apply for childcare support funds" is retrieved from the database, and an emotionally sensitive response is generated. Finally, the message "How to apply for childcare support funds is as follows. We hope your application goes smoothly." is displayed.

[0652] Example of a prompt

[0653] Point down

[0654] Q: How do I renew my SSL certificate?

[0655] A: Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us.

[0656] This system can efficiently perform a series of processes, from question analysis to answer generation and sentiment analysis. Furthermore, by analyzing and responding to user emotions, it can provide answers that are more satisfying.

[0657] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0658] Step 1: The user enters a question.

[0659] The user launches the application on their PC or smartphone and enters a question into a text box. Specifically, they might enter the question, "How do I renew my SSL certificate?" and click the submit button. The user's question text serves as the input and is the starting point for the entire process. The output is the question text stored on the device.

[0660] Step 2: Submit your question

[0661] The terminal uses the HTTPS protocol to send user-entered questions to the server. Specifically, it is a process of encrypting the entered question text and securely sending it to the server. The input is the user's question text, and the output is the encrypted question data sent to the server.

[0662] Step 3: Receive and analyze the questions

[0663] The server parses the received question in JSON format. Next, a natural language processing engine (e.g., spaCy or NLTK) is used to analyze the question text. Here, the keywords "SSL certificate" and the verb "renewal" are extracted. The input is encrypted question data, and the output is the analysis results, such as keywords and phrases.

[0664] Step 4: Classify the questions

[0665] The server uses a machine learning model (e.g., random forest or support vector machine) to classify questions into predefined categories. The input is the analysis results, and the output is a determined category such as "IT technical specifications."

[0666] Step 5: Analyze your emotions

[0667] The server passes the question to the emotion engine, which then recognizes the user's emotions. The emotion engine analyzes the emotions (e.g., joy, frustration, anger, etc.) from the text entered by the user. The input includes the question itself and the analysis results, and the output is an emotion analysis result such as "anxiety" or "confusion."

[0668] Step 6: Retrieve relevant information from the database.

[0669] The server queries the FAQ database and retrieves relevant answer information. For example, information about "how to renew an SSL certificate" is extracted from the database. The input includes the question category and sentiment analysis results, and the output is the relevant answer information.

[0670] Step 7: Generate the answer

[0671] Based on the information acquired by the server, a natural language generation engine (e.g., a generative AI model) is used to generate text in an easily understandable format. Furthermore, the results of sentiment analysis are taken into consideration to generate a response that is more sensitive to the user's emotions. The input consists of acquired response information and sentiment analysis results, and the output is a response that is sensitive to emotions. Specifically, it would take the form of, "Please follow the steps below to renew your SSL certificate. If you have any questions, please feel free to contact us at any time."

[0672] Step 8: Submit your response

[0673] The server sends the generated response to the user's terminal as an HTTP response in JSON format. The data is encoded before transmission. The input is the generated response text, and the output is the encoded response data delivered to the user's terminal.

[0674] Step 9: Show your answer

[0675] The device parses the received response data. The parsed data is dynamically inserted into the application's UI components and displayed in a format viewable by the user. The input is encoded response data, and the output is a viewable response text displayed on the device screen.

[0676] Step 10: Check your answers

[0677] The user reviews the response. Specifically, by viewing the response, "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us," the user can obtain the information they were looking for. The input is the displayed response, and the output is the user reviewing its contents.

[0678] (Application Example 2)

[0679] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0680] Traditional systems suffered from a decline in the quality of the user experience because they did not consider the user's emotions during the process from when the user entered a question to when they received an answer. Furthermore, in stressful environments such as factories, it is necessary to appropriately handle user anxiety and dissatisfaction. However, with current technology, it has been difficult to recognize user emotions and provide answers that respond accordingly.

[0681] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for providing an interface for the user to input a question, means for transmitting the question to the server, means for the server to analyze the question using a natural language processing engine, means for classifying the question, means for obtaining relevant information from a database based on the classification, means for generating an answer from the obtained relevant information using a natural language generation engine, means for transmitting the generated answer to a user terminal, means for the user terminal to display the answer, means for recognizing the user's emotions from the input question, and means for generating an answer considering the emotion recognition result. This makes it possible to improve the quality of the user experience by recognizing the user's emotions and providing an answer accordingly.

[0682] "User interface" refers to the screens and operating methods used by users to input questions.

[0683] A "server" refers to a computer system that receives and processes requests from clients via a network.

[0684] A "natural language processing engine" refers to a system that includes algorithms and models for analyzing text data and understanding its meaning.

[0685] "Question classification" refers to the process of categorizing user questions into specific categories or tags.

[0686] A "database" refers to a collection of information that is systematically organized and stored, making it easy to search and retrieve.

[0687] A "natural language generation engine" refers to a system that includes algorithms and models for generating responses in a natural format based on acquired relevant information.

[0688] "Emotion recognition" refers to the process of analyzing and identifying emotions (such as joy, dissatisfaction, or anger) from text entered by the user.

[0689] A "user terminal" refers to a device (such as a smartphone, tablet, or PC) used by a user to access the server and exchange questions and answers.

[0690] This invention relates to an interactive QA system for factory robots. This system allows factory workers to ask questions about robot operation and maintenance, and provides appropriate and prompt answers. It also has a function to recognize the user's emotions and generate corresponding responses.

[0691] System Configuration

[0692] This system consists of the following components.

[0693] 1. User Interface: An interface for the worker to input questions. Examples include using smart glasses or voice input devices.

[0694] 2. Server: A central processing unit that receives questions, analyzes them, and generates answers. The server includes a natural language processing engine (e.g., spaCy), an emotion recognition engine (e.g., Hugging Face Transformers), and a natural language generation engine (e.g., OpenAI GPT-3).

[0695] 3. Database: A collection of information that stores relevant information for a query. Examples include PostgreSQL and MySQL.

[0696] 4. User terminal: A device that displays answers to the worker. This includes smartphones, tablets, and PCs.

[0697] Program processing flow

[0698] User Interface

[0699] The worker wears smart glasses and inputs a question by voice, such as, "Please tell me how to maintain the robot."

[0700] Submit a question

[0701] The smart glasses convert the voice data into text format and send the question to the server. For speech recognition, APIs such as Google Cloud Speech-to-Text can be used.

[0702] Question analysis

[0703] The server analyzes the received question using a natural language processing engine (e.g., spaCy). Keywords such as "maintenance" and "robot" are extracted from the text data.

[0704] Question classification

[0705] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the question. The question is classified as "technical support".

[0706] emotion recognition

[0707] The server uses an emotion analysis engine (e.g., Hugging Face Transformers) to recognize the worker's emotions from the input text. The emotion analysis result is determined to be "anxiety."

[0708] Answer generation

[0709] The server retrieves relevant information from the database and generates a response using a natural language generation engine (e.g., OpenAI GPT-3). The response is generated in the format of "Robot maintenance is performed using the following steps. If you have any questions, please contact support."

[0710] Submit your response

[0711] The server sends the generated response to the user's terminal. The user's terminal displays the received response so that the worker can confirm it.

[0712] Specific example

[0713] Example 1: Questions about robot maintenance

[0714] If a worker asks, "How do I calibrate the robot arm?", the system will operate as follows: The question is analyzed, relevant information is retrieved from the database, and, taking sentiment into consideration, an answer is generated stating, "The robot arm calibration procedure is as follows. If you have any questions, please contact support."

[0715] Example of a prompt

[0716] The user is asking about the calibration method for a robot arm. Emotion detected: Anxiety. Provide a detailed but friendly response.

[0717] This invention recognizes user emotions and provides corresponding responses, enabling factory workers to quickly obtain information and improve work efficiency. This, in turn, enhances overall factory productivity and reduces anxiety and stress.

[0718] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0719] Step 1:

[0720] The user wears smart glasses and uses a voice input device to enter a question. For example, they might ask, "How do I calibrate the robot arm?" The smart glasses capture the voice data as input and convert it into text data. The converted text data is then output.

[0721] Step 2:

[0722] The terminal sends the text data converted from the speech to the server using the HTTPS protocol. The server receives this text data and takes it as input. The output is a request body containing the text data.

[0723] Step 3:

[0724] The server analyzes the received text data using a natural language processing engine (e.g., spaCy). Specifically, it splits the text data and extracts keywords and important phrases. This process produces a list of analyzed keywords as output.

[0725] Step 4:

[0726] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the analyzed text data into a specific category. A keyword list is used as input, and the output category is "Technical Support".

[0727] Step 5:

[0728] The server uses an emotion analysis engine (e.g., Hugging Face Transformers) to recognize the worker's emotions from the input text. In this step, text data is used as input, and the emotion analysis result (e.g., "anxious") is output.

[0729] Step 6:

[0730] The server retrieves relevant information from the database. The classified categories and keywords are used as input, and a database query is executed to retrieve the necessary information. The output is the relevant information (e.g., "Calibration Method").

[0731] Step 7:

[0732] The server uses a natural language generation engine (e.g., OpenAI GPT-3) to generate a response based on the acquired relevant information and sentiment recognition results. This involves inputting a prompt sentence into a generation AI model, and obtaining the generated response text as its output. For example, the prompt sentence is "The user is asking about the calibration method for a robot arm. Emotion detected: Anxiety. Provide a detailed but friendly response."

[0733] Step 8:

[0734] The server sends the generated response to the user's terminal. The HTTPS protocol is used for transmission, and the generated response text is included as output.

[0735] Step 9:

[0736] The user's device displays the received response. The response text is displayed on the smart glasses or smartphone screen, allowing the user to confirm the answer to the question. In this step, the response text is passed to the device as input and is visually displayed as output.

[0737] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0738] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0739] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0740] [Third Embodiment]

[0741] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0742] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0743] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0744] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0745] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0746] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0747] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0748] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0749] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0750] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0751] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0752] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0753] This invention provides a system that allows users to quickly and accurately obtain answers to questions at government and corporate service counters. The system centers on a series of processes in which the user inputs a question, which is then sent to a server for analysis and classification, and an appropriate answer is generated and provided to the user.

[0754] Overall System Overview

[0755] This system mainly consists of the following components:

[0756] 1. User Interface

[0757] 2. The terminal that sends the question data to the server.

[0758] 3. Server that receives and analyzes question data

[0759] 4. Natural Language Processing and Generation Engines

[0760] 5. Database

[0761] 6. Interface for sending and displaying responses on the user's terminal.

[0762] Program processing flow

[0763] Input Interface

[0764] The user launches the application provided on their PC or smartphone and enters a question into the text box. For example, after entering the question, "How do I apply for childcare support funds?", they click the submit button.

[0765] Submit a question

[0766] The terminal sends the questions entered by the user to the server using the HTTPS protocol.

[0767] Question analysis

[0768] The server parses the received question in JSON format and uses a natural language processing engine (e.g., spaCy or NLTK) to analyze the question text. Here, the keyword "childcare support funds" and the action "how to apply" are extracted.

[0769] Question classification

[0770] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the question into a predefined category. In this case, the question is classified as "national system."

[0771] Answer generation

[0772] The server queries the FAQ database and retrieves the relevant answer information. For example, "How to apply for childcare support funds" is extracted from the database. Based on the extracted information, a natural language generation engine (e.g., GPT-3) generates text in an easy-to-understand format. This results in the answer being "You can apply for childcare support funds by following these steps..."

[0773] Submit your response

[0774] The server sends the generated response to the user's terminal as an HTTP response in JSON format.

[0775] Display the answer

[0776] The device parses the received response data and displays it in the application's user interface. The user can see the response, "You can apply for childcare support funds by following these steps..."

[0777] Specific example

[0778] Example 1: Questions regarding IT technical specifications

[0779] A user types "How do I renew my SSL certificate?" on their smartphone and sends it. The device sends this question to the server, which analyzes and categorizes the question. If the question is classified as "IT technical specifications," information on "how to renew an SSL certificate" is retrieved from the database. A natural language generation engine generates an answer, which is displayed on the user interface as "To renew your SSL certificate, follow these steps...".

[0780] Example 2: Questions about national systems

[0781] A user types "How do I apply for childcare support funds?" on their PC and sends it. The terminal sends this question to the server, which analyzes and classifies the question. The question is classified as "National System," and information on "How to apply for childcare support funds" is retrieved from the database. A natural language generation engine generates an answer, which is displayed on the user interface as "You can apply for childcare support funds by following these steps...".

[0782] The above describes the embodiments for carrying out the present invention. This system efficiently performs a series of processes from question analysis to answer generation, enabling users to obtain quick and accurate answers.

[0783] The following describes the processing flow.

[0784] Step 1:

[0785] The user launches the application on their PC or smartphone and enters a question into the text box. For example, they might enter the question, "How do I apply for childcare support funds?" and click the submit button.

[0786] Step 2:

[0787] The terminal sends the entered question to the server as an HTTP POST request using the HTTPS protocol. The question data is encoded in JSON format during this process.

[0788] Step 3:

[0789] The server parses the received question data into JSON format. The server temporarily stores this data and passes it to a natural language processing engine (e.g., spaCy or NLTK).

[0790] Step 4:

[0791] The server uses a natural language processing engine to perform morphological analysis on the question text. Specifically, it analyzes the sentence and extracts keywords and important verbs. For example, keywords such as "childcare support funds" and "application method" are extracted.

[0792] Step 5:

[0793] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the question into a predefined category. For example, the question might be classified as "national systems."

[0794] Step 6:

[0795] The server queries the corresponding FAQ database based on the classification result. The server searches for and extracts relevant information. In this case, information regarding "how to apply for childcare support funds" is retrieved from the database.

[0796] Step 7:

[0797] Based on the data acquired by the server, a natural language generation engine (e.g., GPT-3) is used to generate a response in a user-friendly format. For example, it might generate a sentence like, "To apply for childcare support funds, follow these steps..."

[0798] Step 8:

[0799] The server generates the response and sends it to the user's terminal as an HTTP response in JSON format. The response data is encoded before transmission.

[0800] Step 9:

[0801] The device parses the response data it receives. The parsed data is then dynamically inserted into the application's UI components.

[0802] Step 10:

[0803] The device displays the answer in the user interface. The user can see the answer, which reads, "To apply for childcare support funds, follow these steps..."

[0804] (Example 1)

[0805] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0806] Conventional question-answering systems have struggled to provide quick and accurate answers to user-inputted questions. This is because the processes required for question analysis, classification, and optimal answer generation are complex, and appropriate means are lacking for each stage of the process. The present invention aims to solve these problems and provide a system that can provide users with quick and accurate answers.

[0807] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0808] In this invention, the server includes means for receiving and parsing questions in JSON format, means for analyzing them using a natural language processing engine, and means for classifying them using a machine learning model. This makes it possible to process questions in a consistent flow from reception to analysis and classification.

[0809] A "user" is a person or entity that uses this system to input questions and obtain answers.

[0810] "Interface" refers to the screen or input area of ​​an application where a user enters a question.

[0811] A "server" is a computing system that receives, analyzes, classifies, and generates answers to questions.

[0812] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a format for representing data in a structured manner.

[0813] "Parsing" refers to the process of analyzing data and converting it into a specific format.

[0814] A "natural language processing engine" is software and algorithms used to analyze natural language text and understand its meaning and context.

[0815] A "machine learning model" refers to an algorithm that learns patterns and knowledge from large amounts of data and uses that knowledge to make predictions and classifications on new data.

[0816] A "database" is a data storage system used to centrally manage answer information and related data for questions.

[0817] A "natural language generation engine" is software and algorithms used to convert information into easily understandable text.

[0818] The "HTTPS protocol" is a protocol used to encrypt data in web communications, and is a technology that enables secure data transmission and reception.

[0819] This invention provides a system that allows users to quickly and accurately obtain answers to questions at government and corporate service counters. The system centers on a series of processes in which the user inputs a question, which is then sent to a server for analysis and classification, and an appropriate answer is generated and provided to the user.

[0820] Overall System Overview

[0821] This system mainly consists of the following components:

[0822] 1. User Interface

[0823] 2. The terminal that sends the question data to the server.

[0824] 3. Server that receives and analyzes question data

[0825] 4. Natural Language Processing Engine

[0826] 5. Machine Learning Models

[0827] 6. Database

[0828] 7. Natural Language Generation Engine

[0829] 8. Interface for sending and displaying responses on the user's terminal.

[0830] User question input

[0831] Users can launch the application provided on a device such as a PC or smartphone and enter a question into a text box. For example, they can enter the question, "How do I apply for childcare support funds?" and then click the submit button. The entered question is temporarily stored in the device's memory.

[0832] Submit a question

[0833] The device sends the user-entered question to the server using the HTTPS protocol. Specifically, the question data is included as a JSON payload in the HTTP request and sent to the server's specified URL using the POST method.

[0834] Receiving and parsing questions

[0835] The server receives an HTTP request sent from the terminal. The server parses the received payload in JSON format and stores it in a database or temporary storage on the server.

[0836] Question analysis

[0837] The server passes the stored question data to a natural language processing engine (e.g., spaCy or NLTK) for text analysis. Here, the noun phrase "childcare support funds" and the verb phrase "application method" are extracted. Specifically, processes such as tokenization, part-of-speech tagging, and dependency parsing are performed.

[0838] Question classification

[0839] The server inputs the analyzed data into a machine learning model (e.g., random forest or support vector machine) and classifies it into a predefined category (in this case, "national systems"). This classification result is then added to the question data.

[0840] Answer generation

[0841] The server queries the FAQ database based on the classification results and retrieves answer information for the relevant questions. As a result of the query, information such as "How to apply for childcare support funds" is extracted. Next, a natural language generation engine (e.g., GPT-3) reconstructs the extracted information into natural-sounding sentences. At this stage, a sentence such as "You can apply for childcare support funds by following these steps..." is generated.

[0842] Submit your response

[0843] The server sends the generated response to the user's terminal as an HTTP response in JSON format. The server creates a response including HTTP headers and payload and sends it to the terminal.

[0844] Display the answer

[0845] The device receives a response from the server and parses the answer data in the payload. The parsed data is displayed in the application's user interface. The user can then see the answer, "You can apply for childcare support funds by following these steps...", on the screen.

[0846] Explanation of specific examples

[0847] Example 1: Questions regarding IT technical specifications

[0848] A user types "How do I renew my SSL certificate?" on their smartphone and sends it. The device sends this question to the server, which analyzes and categorizes the question. If the question is classified as "IT technical specifications," information on "how to renew an SSL certificate" is retrieved from the database. A natural language generation engine generates an answer, which is displayed on the user interface as "To renew your SSL certificate, follow these steps...".

[0849] Example 2: Questions about national systems

[0850] A user types "How do I apply for childcare support funds?" on their PC and sends it. The terminal sends this question to the server, which analyzes and classifies the question. The question is classified as "National System," and information on "How to apply for childcare support funds" is retrieved from the database. A natural language generation engine generates an answer, which is displayed on the user interface as "You can apply for childcare support funds by following these steps...".

[0851] Examples of prompts for generative AI models

[0852] Example of a prompt message for a generative AI model (e.g., GPT-3):

[0853] Question: How do I apply for childcare support funds?

[0854] Previous text in the model: Childcare support grants can be applied for by following these steps…

[0855] This prompt is used to generate specific text that provides additional information for the generated AI model.

[0856] The above describes the embodiments for carrying out the present invention. By using this system, a series of processes from question analysis to answer generation can be performed efficiently, enabling users to obtain quick and accurate answers.

[0857] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0858] Step 1: User Question Input

[0859] The user opens the application on their PC or smartphone and enters a question into the text box. The entered question might be text data such as, "Please tell me how to apply for childcare support funds." Clicking the submit button prepares the input data needed to proceed to the next step.

[0860] Step 2: Submit your question

[0861] The terminal sends the user-entered question to the server using the HTTPS protocol. The input data is a text-based question, which is stored as a JSON payload in the body of the HTTP request and sent to the server's specified URL using the POST method. The output is this HTTP request.

[0862] Step 3: Receiving and Parsing Questions

[0863] The server receives an HTTP request sent from the terminal. The received data is parsed in JSON format, and the text data of the question is extracted. This extracted text data becomes the input data used for analysis in the next step. The output is the parseable text data of the question.

[0864] Step 4: Analyzing the Question

[0865] The server passes the extracted text data of the questions to a natural language processing engine (e.g., spaCy or NLTK) for text analysis. The input data is the extracted text data, which undergoes processes such as tokenization, part-of-speech tagging, and dependency parsing to obtain, for example, the noun phrase "childcare support funds" and the verb phrase "application method". The output is structural information of the analyzed text data.

[0866] Step 5: Classifying Questions

[0867] The server inputs the structural information of the analyzed text data into a machine learning model (e.g., Random Forest or Support Vector Machine) to perform categorization. The input data is the structural information of the analysis results, which is classified into predefined categories. In this case, it is classified as "National Institutions." The output is the classification result.

[0868] Step 6: Generating the answer

[0869] The server queries the FAQ database based on the classification results and retrieves relevant answer information. The input data is the classification results, and the retrieved answer information is input into a natural language generation engine (e.g., GPT-3) to reconstruct it into natural-sounding sentences. For example, a specific answer text such as "You can apply for childcare support funds by following these steps..." is generated. The output is the generated answer text.

[0870] Step 7: Submit your response

[0871] The server sends the generated response as an HTTP response in JSON format to the user's terminal. The input data is the generated response text, which is stored in the HTTP response payload and sent to the terminal. The output is this HTTP response.

[0872] Step 8: Display the answer

[0873] The terminal receives a response from the server and parses the answer data within the JSON payload. The input data is the received HTTP response, which is then displayed in the application's user interface after parsing. For example, it might display something like, "You can apply for childcare support funds by following these steps..." The output is the answer displayed in the user interface.

[0874] The above outlines the specific processing flow of the program in this system. This allows users to obtain quick and accurate answers.

[0875] (Application Example 1)

[0876] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0877] Currently, it is difficult for customers to get quick and accurate answers about product information and services in physical stores. In particular, when customers ask questions about specific products or services in stores, staff members are often busy, making it difficult to get an appropriate answer immediately. In such situations, customer satisfaction decreases, which can lead to a decline in sales. Therefore, there is a need for a system that allows customers to get quick and accurate answers to their questions in physical stores.

[0878] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0879] In this invention, the server includes means for analyzing a question using a natural language processing engine, means for classifying the question, means for obtaining relevant information from a database based on the classification, means for generating an answer from the obtained relevant information using a natural language generation engine, means for transmitting the generated answer to a user terminal, means for the user terminal to display the answer, and means for a user to input a question using a smartphone application in a physical store and receive and display the generated answer in real time. This makes it possible for customers in physical stores to obtain accurate answers about product information and services in real time.

[0880] A "system" is a collection of devices and software that perform a series of processes to allow a user to input a question and obtain an appropriate answer.

[0881] A "user terminal" is a device that a user uses to input questions and receive and display answers, and includes, for example, smartphones and tablets.

[0882] An "interface" refers to the means by which data is entered between the user and the system, such as a screen or input field for the user to enter a question.

[0883] A "server" is a central processing unit that analyzes questions submitted by users and generates answers, and it is equipped with a database and natural language processing and generation engines.

[0884] A "natural language processing engine" is a software module that analyzes user-inputted questions, extracts keywords and categories, and understands their meaning.

[0885] "Classification" is the process of assigning questions analyzed by a natural language processing engine to predefined categories.

[0886] A "database" is a storage system that stores information for retrieving related information based on the aforementioned classification.

[0887] A "natural language generation engine" is a software module that automatically generates natural-sounding response sentences based on acquired relevant information.

[0888] A "physical store" is a sales and service facility that provides goods and services in a physical location, where customers visit in person to make purchases.

[0889] A "smartphone application" is a software program that runs on a smartphone, allowing the user to input questions and receive generated answers.

[0890] "Real-time" refers to the timing at which a system processes user input immediately and provides an instant response.

[0891] This invention provides a system that offers quick and accurate answers to user questions in physical stores. This system mainly consists of the following components.

[0892] First, an interface for users to input questions is provided as a smartphone application. Users launch the application in a physical store and input their questions in text format.

[0893] Next, the user terminal sends the entered question to the server using the HTTPS protocol. Here, the server's role is crucial. The server is equipped with a natural language processing engine (for example, SpaCy or NLTK) that analyzes the received question and extracts keywords and sentence components.

[0894] The server then uses a machine learning model (e.g., random forest or support vector machine) to classify the questions into predefined categories. Based on this classification, it retrieves relevant information from the database.

[0895] Based on the acquired relevant information, a natural language generation engine (for example, a generative AI model such as GPT-3) generates a response in a format that is easy for the user to understand. This generated response is then sent to the user's terminal by the server in JSON format.

[0896] Finally, the user's device parses the received response and displays it on the smartphone application's user interface. This allows the user to obtain answers to their questions in real time.

[0897] As a concrete example, consider the case where a user asks, "How do I use this product?" When the user enters the question into their smartphone and sends it, the server analyzes and classifies it. The question is classified as "How to use," and information on "How to use this product" is retrieved from the database. Subsequently, an answer is generated using a generative AI model (GPT-3), and it is displayed on the user interface as, "This product can be used in the following ways..."

[0898] Examples of prompt statements include the following:

[0899] "Category: How to use\nQuestion: Please tell me how to use this product\nAnswer:"

[0900] In this way, a system is realized that provides immediate and accurate answers to user questions in physical stores.

[0901] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0902] Step 1:

[0903] The user enters a question. Specifically, the user enters the question in text format into the interface of a smartphone application. A question such as "How do I use this product?" is entered. In this step, the input is the user's question, and the output is the entered text-formatted question.

[0904] Step 2:

[0905] The user terminal sends the entered question to the server. Specifically, the user terminal uses the HTTPS protocol to send the user-entered question to the server. The input in this step is a text-based question, and the output is an HTTP request sent to the server.

[0906] Step 3:

[0907] The server receives and parses the question. Specifically, the server receives an HTTP request and uses a natural language processing engine (e.g., SpaCy or NLTK) to parse the question and extract keywords and sentence components. The input for this step is the question text of the HTTP request, and the output is the parsed keywords and sentence components.

[0908] Step 4:

[0909] The server classifies the questions. Specifically, the server uses a machine learning model (e.g., random forest or support vector machine) to classify the analyzed question sentences into predefined categories (e.g., usage, price, specifications, etc.). The input for this step is the keywords and sentence components of the analyzed question, and the output is the appropriate category label.

[0910] Step 5:

[0911] The server retrieves relevant information from the database based on the classification. Specifically, the server queries the database and retrieves information corresponding to the classified category. The input for this step is the category label, and the output is the relevant information retrieved from the database.

[0912] Step 6:

[0913] The server uses a natural language generation engine to generate answers based on the relevant information it has acquired. Specifically, it uses a generative AI model (e.g., GPT-3) to generate answers in a natural format based on the acquired information. The input for this step is relevant information acquired from a database, and the output is an answer generated in a way that is easy for the user to understand.

[0914] Step 7:

[0915] The server sends the generated response to the user's terminal. Specifically, it sends the generated response to the user's terminal as an HTTP response in JSON format. The input for this step is the generated response text, and the output is the HTTP response sent to the user's terminal.

[0916] Step 8:

[0917] The user terminal displays the response it received. Specifically, the user terminal receives an HTTP response, parses it, and displays the response on the user interface. It might be displayed in the format, "This product can be used in the following ways..." The input for this step is the HTTP response from the server, and the output is the response displayed on the user terminal.

[0918] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0919] This invention provides a system that quickly and appropriately answers questions that users ask to government or corporate contact points, and further incorporates an emotion engine that recognizes the user's emotions. The system's core process involves the user inputting a question, which is then sent to a server for analysis and classification, generating an appropriate answer, and providing it to the user. In addition, it includes a function to recognize the user's emotions and utilize the recognition results to provide even more appropriate answers.

[0920] Overall System Overview

[0921] This system mainly consists of the following components:

[0922] 1. User Interface

[0923] 2. The terminal that sends the question data to the server.

[0924] 3. Server that receives and analyzes question data

[0925] 4. Natural Language Processing and Generation Engines

[0926] 5. Database

[0927] 6. Emotional Engine

[0928] 7. Interface for sending and displaying responses on the user's terminal.

[0929] Program processing flow

[0930] Input Interface

[0931] The user launches the application on their PC or smartphone and enters a question into the text box. For example, they might enter the question, "How do I renew my SSL certificate?" and click the submit button.

[0932] Submit a question

[0933] The terminal sends the questions entered by the user to the server using the HTTPS protocol.

[0934] Question analysis

[0935] The server parses the received question in JSON format and uses a natural language processing engine (e.g., spaCy or NLTK) to analyze the question text. Here, the keywords "SSL certificate" and the verb "renewal" are extracted.

[0936] Question classification

[0937] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the questions into predefined categories. In this case, the questions are classified as "IT technical specifications."

[0938] Emotional analysis

[0939] The server passes the question to the sentiment engine, which recognizes the user's emotions. The sentiment engine analyzes the emotions (e.g., joy, frustration, anger) from the text entered by the user. The analysis results influence the classification of the question.

[0940] Answer generation

[0941] The server queries the FAQ database and retrieves the relevant answer information. For example, information regarding "how to renew an SSL certificate" is extracted from the database. Based on the extracted information, a natural language generation engine (e.g., GPT-3) generates text in an easy-to-understand format. Furthermore, considering the results of sentiment analysis, a more user-friendly answer is generated. As a result, the answer may take the form of, "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us at any time."

[0942] Submit your response

[0943] The server sends the generated response to the user's terminal as an HTTP response in JSON format. The response data is encoded before transmission.

[0944] Display the answer

[0945] The device parses the response data it receives. The parsed data is then dynamically inserted into the application's UI components.

[0946] Confirmation of the answer

[0947] The device displays the response in the user interface. The user can see the response: "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us."

[0948] Specific example

[0949] Example 1: Questions regarding IT technical specifications

[0950] A user types "How do I renew my SSL certificate?" on their smartphone and sends it. The device sends this question to the server, which analyzes and classifies the question. The question is classified as "IT technical specification," and the sentiment engine analyzes it as "the user is feeling anxious." Information on "how to renew your SSL certificate" is retrieved from the database, and an emotionally sensitive response is generated. Finally, the message "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us anytime." is displayed.

[0951] Example 2: Questions about national systems

[0952] A user types "How do I apply for childcare support funds?" on their PC and sends it. The device sends this question to the server, which analyzes and classifies the question. The question is classified as "National System," and the sentiment engine analyzes that "the user is feeling positive emotions." Information on "How to apply for childcare support funds" is retrieved from the database, and an emotionally sensitive response is generated. Finally, the message "How to apply for childcare support funds is as follows. We hope your application goes smoothly." is displayed.

[0953] The above describes the embodiments for carrying out the present invention. This system efficiently performs a series of processes from question analysis to answer generation and emotion analysis, enabling users to obtain quick and accurate answers.

[0954] The following describes the processing flow.

[0955] Step 1:

[0956] The user launches the application on their PC or smartphone and enters the question "How do I renew my SSL certificate?" into the interface for entering questions. Then they click the submit button.

[0957] Step 2:

[0958] The terminal sends the entered question to the server using the HTTPS protocol. The question data is encoded in JSON format.

[0959] Step 3:

[0960] The server parses the received question data into JSON format. The server temporarily receives this data and passes it on to the question analysis process.

[0961] Step 4:

[0962] The server uses a natural language processing engine (such as spaCy or NLTK) to perform morphological analysis on the question text. Specifically, it tokenizes the sentence and extracts keywords and important verbs. For example, the keyword "SSL certificate" and the verb "renewal" are extracted.

[0963] Step 5:

[0964] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the questions into predefined categories. In this case, the questions are classified as "IT technical specifications."

[0965] Step 6:

[0966] The server passes the question to the sentiment engine, which analyzes the user's emotions. The sentiment engine analyzes the question and determines that "the user is feeling anxious."

[0967] Step 7:

[0968] The server queries the FAQ database and retrieves the relevant answer information. Specifically, information regarding "how to renew SSL certificates" is extracted from the database.

[0969] Step 8:

[0970] The server uses a natural language generation engine (e.g., GPT-3) to generate a response based on the extracted data. Considering the results of the sentiment engine's analysis, wording that is considerate of users experiencing anxiety is added. The generated response will be in the form of "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us."

[0971] Step 9:

[0972] The server generates the response and sends it to the user's terminal as an HTTP response in JSON format. The response data is encoded.

[0973] Step 10:

[0974] The terminal parses the response data it receives. The parsed data is then dynamically inserted into the application's user interface.

[0975] Step 11:

[0976] The device displays the response in the user interface. The user can see the response: "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us."

[0977] (Example 2)

[0978] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0979] In today's information society, users ask a variety of questions seeking quick and accurate answers. However, conventional systems often suffer from delays in responding to user questions, inconsistent response quality, and particular difficulty in providing appropriate answers that take user emotions into consideration. Furthermore, there is a lack of means to generate appropriate answers that respond to emotions. As a result, the user experience fails to reach a satisfactory level, which is a challenge.

[0980] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0981] In this invention, the server includes means for analyzing a question using a natural language processing engine, means for classifying the question using a machine learning model, and means for performing sentiment analysis of the question using a sentiment engine. This makes it possible to quickly generate an appropriate answer based on the content of the question and the user's sentiment.

[0982] A "user" is an individual or group that uses the system to input questions and receive answers.

[0983] An "interface" is a means that includes screens and input devices for users to input questions and view answers.

[0984] A "server" is a computer system that receives questions from users, analyzes them, classifies them, generates answers, and sends them back.

[0985] A "natural language processing engine" is a software engine that analyzes question texts to understand their meaning, using language models to extract keywords and important phrases.

[0986] "Question classification" is the process of classifying analyzed question texts into predefined categories using a machine learning model.

[0987] An "emotion engine" is a software engine that detects emotions from questions entered by the user and performs analysis based on those emotions.

[0988] A "database" is an information system that stores answer information to questions, and the necessary information is extracted through queries.

[0989] A "natural language generation engine" is a software engine that generates answers in an easy-to-understand and appropriate format for the user based on acquired information.

[0990] The HTTPS protocol is a communication protocol for securely transmitting data over the internet, encrypting and protecting user information.

[0991] "Response generation" is the process by which the server generates responses to provide to the user using a natural language generation engine, taking into account information obtained from the database and the results of the sentiment engine's analysis.

[0992] "Displaying the answer" is the process of presenting the generated answer to the user's terminal in an easy-to-read format.

[0993] This invention provides a system that offers quick and appropriate answers to questions users make to government or corporate contact points, and further incorporates an emotion engine that recognizes the user's emotions. This system mainly consists of the following key components.

[0994] 1. User Interface

[0995] 2. The terminal that sends the question data to the server.

[0996] 3. Server that receives and analyzes question data

[0997] 4. Natural Language Processing Engine

[0998] 5. Database

[0999] 6. Emotional Engine

[1000] 7. Interface for sending and displaying responses on the user's terminal.

[1001] The process begins when a user launches the application on their PC or smartphone and enters a question into a text box. For example, they might enter the question, "How do I renew my SSL certificate?" and click the submit button. This causes the user's device to send the question to the server using the HTTPS protocol. The server parses the received question in JSON format and analyzes it using a natural language processing engine (e.g., spaCy or NLTK). Here, the keywords "SSL certificate" and the verb "renew" are extracted.

[1002] Next, the server uses a machine learning model (e.g., random forest or support vector machine) to classify the question. It is then categorized into a specific category (in this case, "IT technical specifications"). The server then passes the question to the sentiment engine to recognize the user's emotions. The sentiment engine analyzes the emotions (e.g., joy, frustration, anger, etc.) from the text entered by the user. This analysis influences the classification of the question and the generation of answers.

[1003] The server then queries the database to retrieve the relevant response information. For example, information regarding "how to renew an SSL certificate" might be extracted from the database. Based on this extracted information, a natural language generation engine (e.g., a generative AI model) is used to generate text in an easily understandable format. Furthermore, the response is generated in an appropriate format, taking into account the results of sentiment analysis. Specifically, the response might be something like, "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us."

[1004] Finally, the server sends the generated response to the user's device as an HTTP response in JSON format. The device parses the received response data, dynamically inserts it into the application's UI components, and displays it in a way that the user can verify.

[1005] Specific example

[1006] (Example 1: Questions regarding IT technical specifications)

[1007] A user types "How do I renew my SSL certificate?" on their smartphone and sends it. The device sends this question to the server, which analyzes and classifies the question. The question is classified as "IT technical specification," and the sentiment engine analyzes it as "the user is feeling anxious." Information on "how to renew your SSL certificate" is retrieved from the database, and an emotionally sensitive response is generated. Finally, the message "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us anytime." is displayed.

[1008] (Example 2: Questions about national systems)

[1009] A user types "How do I apply for childcare support funds?" on their PC and sends it. The device sends this question to the server, which analyzes and classifies the question. The question is classified as "National System," and the sentiment engine analyzes that "the user is feeling positive emotions." Information on "How to apply for childcare support funds" is retrieved from the database, and an emotionally sensitive response is generated. Finally, the message "How to apply for childcare support funds is as follows. We hope your application goes smoothly." is displayed.

[1010] Example of a prompt

[1011] Point down

[1012] Q: How do I renew my SSL certificate?

[1013] A: Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us.

[1014] This system can efficiently perform a series of processes, from question analysis to answer generation and sentiment analysis. Furthermore, by analyzing and responding to user emotions, it can provide answers that are more satisfying.

[1015] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1016] Step 1: The user enters a question.

[1017] The user launches the application on their PC or smartphone and enters a question into a text box. Specifically, they might enter the question, "How do I renew my SSL certificate?" and click the submit button. The user's question text serves as the input and is the starting point for the entire process. The output is the question text stored on the device.

[1018] Step 2: Submit your question

[1019] The terminal uses the HTTPS protocol to send user-entered questions to the server. Specifically, it is a process of encrypting the entered question text and securely sending it to the server. The input is the user's question text, and the output is the encrypted question data sent to the server.

[1020] Step 3: Receive and analyze the questions

[1021] The server parses the received question in JSON format. Next, a natural language processing engine (e.g., spaCy or NLTK) is used to analyze the question text. Here, the keywords "SSL certificate" and the verb "renewal" are extracted. The input is encrypted question data, and the output is the analysis results, such as keywords and phrases.

[1022] Step 4: Classify the questions

[1023] The server uses a machine learning model (e.g., random forest or support vector machine) to classify questions into predefined categories. The input is the analysis results, and the output is a determined category such as "IT technical specifications."

[1024] Step 5: Analyze your emotions

[1025] The server passes the question to the emotion engine, which then recognizes the user's emotions. The emotion engine analyzes the emotions (e.g., joy, frustration, anger, etc.) from the text entered by the user. The input includes the question itself and the analysis results, and the output is an emotion analysis result such as "anxiety" or "confusion."

[1026] Step 6: Retrieve relevant information from the database.

[1027] The server queries the FAQ database and retrieves relevant answer information. For example, information about "how to renew an SSL certificate" is extracted from the database. The input includes the question category and sentiment analysis results, and the output is the relevant answer information.

[1028] Step 7: Generate the answer

[1029] Based on the information acquired by the server, a natural language generation engine (e.g., a generative AI model) is used to generate text in an easily understandable format. Furthermore, the results of sentiment analysis are taken into consideration to generate a response that is more sensitive to the user's emotions. The input consists of acquired response information and sentiment analysis results, and the output is a response that is sensitive to emotions. Specifically, it would take the form of, "Please follow the steps below to renew your SSL certificate. If you have any questions, please feel free to contact us at any time."

[1030] Step 8: Submit your response

[1031] The server sends the generated response to the user's terminal as an HTTP response in JSON format. The data is encoded before transmission. The input is the generated response text, and the output is the encoded response data delivered to the user's terminal.

[1032] Step 9: Show your answer

[1033] The device parses the received response data. The parsed data is dynamically inserted into the application's UI components and displayed in a format viewable by the user. The input is encoded response data, and the output is a viewable response text displayed on the device screen.

[1034] Step 10: Check your answers

[1035] The user reviews the response. Specifically, by viewing the response, "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us," the user can obtain the information they were looking for. The input is the displayed response, and the output is the user reviewing its contents.

[1036] (Application Example 2)

[1037] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1038] Traditional systems suffered from a decline in the quality of the user experience because they did not consider the user's emotions during the process from when the user entered a question to when they received an answer. Furthermore, in stressful environments such as factories, it is necessary to appropriately handle user anxiety and dissatisfaction. However, with current technology, it has been difficult to recognize user emotions and provide answers that respond accordingly.

[1039] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for providing an interface for the user to input a question, means for transmitting the question to the server, means for the server to analyze the question using a natural language processing engine, means for classifying the question, means for obtaining relevant information from a database based on the classification, means for generating an answer from the obtained relevant information using a natural language generation engine, means for transmitting the generated answer to a user terminal, means for the user terminal to display the answer, means for recognizing the user's emotions from the input question, and means for generating an answer considering the emotion recognition result. This makes it possible to improve the quality of the user experience by recognizing the user's emotions and providing an answer accordingly.

[1040] "User interface" refers to the screens and operating methods used by users to input questions.

[1041] A "server" refers to a computer system that receives and processes requests from clients via a network.

[1042] A "natural language processing engine" refers to a system that includes algorithms and models for analyzing text data and understanding its meaning.

[1043] "Question classification" refers to the process of categorizing user questions into specific categories or tags.

[1044] A "database" refers to a collection of information that is systematically organized and stored, making it easy to search and retrieve.

[1045] A "natural language generation engine" refers to a system that includes algorithms and models for generating responses in a natural format based on acquired relevant information.

[1046] "Emotion recognition" refers to the process of analyzing and identifying emotions (such as joy, dissatisfaction, or anger) from text entered by the user.

[1047] A "user terminal" refers to a device (such as a smartphone, tablet, or PC) used by a user to access the server and exchange questions and answers.

[1048] This invention relates to an interactive QA system for factory robots. This system allows factory workers to ask questions about robot operation and maintenance, and provides appropriate and prompt answers. It also has a function to recognize the user's emotions and generate corresponding responses.

[1049] System Configuration

[1050] This system consists of the following components.

[1051] 1. User Interface: An interface for the worker to input questions. Examples include using smart glasses or voice input devices.

[1052] 2. Server: A central processing unit that receives questions, analyzes them, and generates answers. The server includes a natural language processing engine (e.g., spaCy), an emotion recognition engine (e.g., Hugging Face Transformers), and a natural language generation engine (e.g., OpenAI GPT-3).

[1053] 3. Database: A collection of information that stores relevant information for a query. Examples include PostgreSQL and MySQL.

[1054] 4. User terminal: A device that displays answers to the worker. This includes smartphones, tablets, and PCs.

[1055] Program processing flow

[1056] User Interface

[1057] The worker wears smart glasses and inputs a question by voice, such as, "Please tell me how to maintain the robot."

[1058] Submit a question

[1059] The smart glasses convert the voice data into text format and send the question to the server. For speech recognition, APIs such as Google Cloud Speech-to-Text can be used.

[1060] Question analysis

[1061] The server analyzes the received question using a natural language processing engine (e.g., spaCy). Keywords such as "maintenance" and "robot" are extracted from the text data.

[1062] Question classification

[1063] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the question. The question is classified as "technical support".

[1064] emotion recognition

[1065] The server uses an emotion analysis engine (e.g., Hugging Face Transformers) to recognize the worker's emotions from the input text. The emotion analysis result is determined to be "anxiety."

[1066] Answer generation

[1067] The server retrieves relevant information from the database and generates a response using a natural language generation engine (e.g., OpenAI GPT-3). The response is generated in the format of "Robot maintenance is performed using the following steps. If you have any questions, please contact support."

[1068] Submit your response

[1069] The server sends the generated response to the user's terminal. The user's terminal displays the received response so that the worker can confirm it.

[1070] Specific example

[1071] Example 1: Questions about robot maintenance

[1072] If a worker asks, "How do I calibrate the robot arm?", the system will operate as follows: The question is analyzed, relevant information is retrieved from the database, and, taking sentiment into consideration, an answer is generated stating, "The robot arm calibration procedure is as follows. If you have any questions, please contact support."

[1073] Example of a prompt

[1074] The user is asking about the calibration method for a robot arm. Emotion detected: Anxiety. Provide a detailed but friendly response.

[1075] This invention recognizes user emotions and provides corresponding responses, enabling factory workers to quickly obtain information and improve work efficiency. This, in turn, enhances overall factory productivity and reduces anxiety and stress.

[1076] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1077] Step 1:

[1078] The user wears smart glasses and uses a voice input device to enter a question. For example, they might ask, "How do I calibrate the robot arm?" The smart glasses capture the voice data as input and convert it into text data. The converted text data is then output.

[1079] Step 2:

[1080] The terminal sends the text data converted from the speech to the server using the HTTPS protocol. The server receives this text data and takes it as input. The output is a request body containing the text data.

[1081] Step 3:

[1082] The server analyzes the received text data using a natural language processing engine (e.g., spaCy). Specifically, it splits the text data and extracts keywords and important phrases. This process produces a list of analyzed keywords as output.

[1083] Step 4:

[1084] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the analyzed text data into a specific category. A keyword list is used as input, and the output category is "Technical Support".

[1085] Step 5:

[1086] The server uses an emotion analysis engine (e.g., Hugging Face Transformers) to recognize the worker's emotions from the input text. In this step, text data is used as input, and the emotion analysis result (e.g., "anxious") is output.

[1087] Step 6:

[1088] The server retrieves relevant information from the database. The classified categories and keywords are used as input, and a database query is executed to retrieve the necessary information. The output is the relevant information (e.g., "Calibration Method").

[1089] Step 7:

[1090] The server uses a natural language generation engine (e.g., OpenAI GPT-3) to generate a response based on the acquired relevant information and sentiment recognition results. This involves inputting a prompt sentence into a generation AI model, and obtaining the generated response text as its output. For example, the prompt sentence is "The user is asking about the calibration method for a robot arm. Emotion detected: Anxiety. Provide a detailed but friendly response."

[1091] Step 8:

[1092] The server sends the generated response to the user's terminal. The HTTPS protocol is used for transmission, and the generated response text is included as output.

[1093] Step 9:

[1094] The user's device displays the received response. The response text is displayed on the smart glasses or smartphone screen, allowing the user to confirm the answer to the question. In this step, the response text is passed to the device as input and is visually displayed as output.

[1095] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1096] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1097] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1098] [Fourth Embodiment]

[1099] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1100] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1101] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1102] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1103] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1104] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1105] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1106] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1107] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1108] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1109] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1110] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1111] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1112] This invention provides a system that allows users to quickly and accurately obtain answers to questions at government and corporate service counters. The system centers on a series of processes in which the user inputs a question, which is then sent to a server for analysis and classification, and an appropriate answer is generated and provided to the user.

[1113] Overall System Overview

[1114] This system mainly consists of the following components:

[1115] 1. User Interface

[1116] 2. The terminal that sends the question data to the server.

[1117] 3. Server that receives and analyzes question data

[1118] 4. Natural Language Processing and Generation Engines

[1119] 5. Database

[1120] 6. Interface for sending and displaying responses on the user's terminal.

[1121] Program processing flow

[1122] Input Interface

[1123] The user launches the application provided on their PC or smartphone and enters a question into the text box. For example, after entering the question, "How do I apply for childcare support funds?", they click the submit button.

[1124] Submit a question

[1125] The terminal sends the questions entered by the user to the server using the HTTPS protocol.

[1126] Question analysis

[1127] The server parses the received question in JSON format and uses a natural language processing engine (e.g., spaCy or NLTK) to analyze the question text. Here, the keyword "childcare support funds" and the action "how to apply" are extracted.

[1128] Question classification

[1129] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the question into a predefined category. In this case, the question is classified as "national system."

[1130] Answer generation

[1131] The server queries the FAQ database and retrieves the relevant answer information. For example, "How to apply for childcare support funds" is extracted from the database. Based on the extracted information, a natural language generation engine (e.g., GPT-3) generates text in an easy-to-understand format. This results in the answer being "You can apply for childcare support funds by following these steps..."

[1132] Submit your response

[1133] The server sends the generated response to the user's terminal as an HTTP response in JSON format.

[1134] Display the answer

[1135] The device parses the received response data and displays it in the application's user interface. The user can see the response, "You can apply for childcare support funds by following these steps..."

[1136] Specific example

[1137] Example 1: Questions regarding IT technical specifications

[1138] A user types "How do I renew my SSL certificate?" on their smartphone and sends it. The device sends this question to the server, which analyzes and categorizes the question. If the question is classified as "IT technical specifications," information on "how to renew an SSL certificate" is retrieved from the database. A natural language generation engine generates an answer, which is displayed on the user interface as "To renew your SSL certificate, follow these steps...".

[1139] Example 2: Questions about national systems

[1140] A user types "How do I apply for childcare support funds?" on their PC and sends it. The terminal sends this question to the server, which analyzes and classifies the question. The question is classified as "National System," and information on "How to apply for childcare support funds" is retrieved from the database. A natural language generation engine generates an answer, which is displayed on the user interface as "You can apply for childcare support funds by following these steps...".

[1141] The above describes the embodiments for carrying out the present invention. This system efficiently performs a series of processes from question analysis to answer generation, enabling users to obtain quick and accurate answers.

[1142] The following describes the processing flow.

[1143] Step 1:

[1144] The user launches the application on their PC or smartphone and enters a question into the text box. For example, they might enter the question, "How do I apply for childcare support funds?" and click the submit button.

[1145] Step 2:

[1146] The terminal sends the entered question to the server as an HTTP POST request using the HTTPS protocol. The question data is encoded in JSON format during this process.

[1147] Step 3:

[1148] The server parses the received question data into JSON format. The server temporarily stores this data and passes it to a natural language processing engine (e.g., spaCy or NLTK).

[1149] Step 4:

[1150] The server uses a natural language processing engine to perform morphological analysis on the question text. Specifically, it analyzes the sentence and extracts keywords and important verbs. For example, keywords such as "childcare support funds" and "application method" are extracted.

[1151] Step 5:

[1152] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the question into a predefined category. For example, the question might be classified as "national systems."

[1153] Step 6:

[1154] The server queries the corresponding FAQ database based on the classification result. The server searches for and extracts relevant information. In this case, information regarding "how to apply for childcare support funds" is retrieved from the database.

[1155] Step 7:

[1156] Based on the data acquired by the server, a natural language generation engine (e.g., GPT-3) is used to generate a response in a user-friendly format. For example, it might generate a sentence like, "To apply for childcare support funds, follow these steps..."

[1157] Step 8:

[1158] The server generates the response and sends it to the user's terminal as an HTTP response in JSON format. The response data is encoded before transmission.

[1159] Step 9:

[1160] The device parses the response data it receives. The parsed data is then dynamically inserted into the application's UI components.

[1161] Step 10:

[1162] The device displays the answer in the user interface. The user can see the answer, which reads, "To apply for childcare support funds, follow these steps..."

[1163] (Example 1)

[1164] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1165] Conventional question-answering systems have struggled to provide quick and accurate answers to user-inputted questions. This is because the processes required for question analysis, classification, and optimal answer generation are complex, and appropriate means are lacking for each stage of the process. The present invention aims to solve these problems and provide a system that can provide users with quick and accurate answers.

[1166] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1167] In this invention, the server includes means for receiving and parsing questions in JSON format, means for analyzing them using a natural language processing engine, and means for classifying them using a machine learning model. This makes it possible to process questions in a consistent flow from reception to analysis and classification.

[1168] A "user" is a person or entity that uses this system to input questions and obtain answers.

[1169] "Interface" refers to the screen or input area of ​​an application where a user enters a question.

[1170] A "server" is a computing system that receives, analyzes, classifies, and generates answers to questions.

[1171] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a format for representing data in a structured manner.

[1172] "Parsing" refers to the process of analyzing data and converting it into a specific format.

[1173] A "natural language processing engine" is software and algorithms used to analyze natural language text and understand its meaning and context.

[1174] A "machine learning model" refers to an algorithm that learns patterns and knowledge from large amounts of data and uses that knowledge to make predictions and classifications on new data.

[1175] A "database" is a data storage system used to centrally manage answer information and related data for questions.

[1176] A "natural language generation engine" is software and algorithms used to convert information into easily understandable text.

[1177] The "HTTPS protocol" is a protocol used to encrypt data in web communications, and is a technology that enables secure data transmission and reception.

[1178] This invention provides a system that allows users to quickly and accurately obtain answers to questions at government and corporate service counters. The system centers on a series of processes in which the user inputs a question, which is then sent to a server for analysis and classification, and an appropriate answer is generated and provided to the user.

[1179] Overall System Overview

[1180] This system mainly consists of the following components:

[1181] 1. User Interface

[1182] 2. The terminal that sends the question data to the server.

[1183] 3. Server that receives and analyzes question data

[1184] 4. Natural Language Processing Engine

[1185] 5. Machine Learning Models

[1186] 6. Database

[1187] 7. Natural Language Generation Engine

[1188] 8. Interface for sending and displaying responses on the user's terminal.

[1189] User question input

[1190] Users can launch the application provided on a device such as a PC or smartphone and enter a question into a text box. For example, they can enter the question, "How do I apply for childcare support funds?" and then click the submit button. The entered question is temporarily stored in the device's memory.

[1191] Submit a question

[1192] The device sends the user-entered question to the server using the HTTPS protocol. Specifically, the question data is included as a JSON payload in the HTTP request and sent to the server's specified URL using the POST method.

[1193] Receiving and parsing questions

[1194] The server receives an HTTP request sent from the terminal. The server parses the received payload in JSON format and stores it in a database or temporary storage on the server.

[1195] Question analysis

[1196] The server passes the stored question data to a natural language processing engine (e.g., spaCy or NLTK) for text analysis. Here, the noun phrase "childcare support funds" and the verb phrase "application method" are extracted. Specifically, processes such as tokenization, part-of-speech tagging, and dependency parsing are performed.

[1197] Question classification

[1198] The server inputs the analyzed data into a machine learning model (e.g., random forest or support vector machine) and classifies it into a predefined category (in this case, "national systems"). This classification result is then added to the question data.

[1199] Answer generation

[1200] The server queries the FAQ database based on the classification results and retrieves answer information for the relevant questions. As a result of the query, information such as "How to apply for childcare support funds" is extracted. Next, a natural language generation engine (e.g., GPT-3) reconstructs the extracted information into natural-sounding sentences. At this stage, a sentence such as "You can apply for childcare support funds by following these steps..." is generated.

[1201] Submit your response

[1202] The server sends the generated response to the user's terminal as an HTTP response in JSON format. The server creates a response including HTTP headers and payload and sends it to the terminal.

[1203] Display the answer

[1204] The device receives a response from the server and parses the answer data in the payload. The parsed data is displayed in the application's user interface. The user can then see the answer, "You can apply for childcare support funds by following these steps...", on the screen.

[1205] Explanation of specific examples

[1206] Example 1: Questions regarding IT technical specifications

[1207] A user types "How do I renew my SSL certificate?" on their smartphone and sends it. The device sends this question to the server, which analyzes and categorizes the question. If the question is classified as "IT technical specifications," information on "how to renew an SSL certificate" is retrieved from the database. A natural language generation engine generates an answer, which is displayed on the user interface as "To renew your SSL certificate, follow these steps...".

[1208] Example 2: Questions about national systems

[1209] A user types "How do I apply for childcare support funds?" on their PC and sends it. The terminal sends this question to the server, which analyzes and classifies the question. The question is classified as "National System," and information on "How to apply for childcare support funds" is retrieved from the database. A natural language generation engine generates an answer, which is displayed on the user interface as "You can apply for childcare support funds by following these steps...".

[1210] Examples of prompts for generative AI models

[1211] Example of a prompt message for a generative AI model (e.g., GPT-3):

[1212] Question: How do I apply for childcare support funds?

[1213] Previous text in the model: Childcare support grants can be applied for by following these steps…

[1214] This prompt is used to generate specific text that provides additional information for the generated AI model.

[1215] The above describes the embodiments for carrying out the present invention. By using this system, a series of processes from question analysis to answer generation can be performed efficiently, enabling users to obtain quick and accurate answers.

[1216] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1217] Step 1: User Question Input

[1218] The user opens the application on their PC or smartphone and enters a question into the text box. The entered question might be text data such as, "Please tell me how to apply for childcare support funds." Clicking the submit button prepares the input data needed to proceed to the next step.

[1219] Step 2: Submit your question

[1220] The terminal sends the user-entered question to the server using the HTTPS protocol. The input data is a text-based question, which is stored as a JSON payload in the body of the HTTP request and sent to the server's specified URL using the POST method. The output is this HTTP request.

[1221] Step 3: Receiving and Parsing Questions

[1222] The server receives an HTTP request sent from the terminal. The received data is parsed in JSON format, and the text data of the question is extracted. This extracted text data becomes the input data used for analysis in the next step. The output is the parseable text data of the question.

[1223] Step 4: Analyzing the Question

[1224] The server passes the extracted text data of the questions to a natural language processing engine (e.g., spaCy or NLTK) for text analysis. The input data is the extracted text data, which undergoes processes such as tokenization, part-of-speech tagging, and dependency parsing to obtain, for example, the noun phrase "childcare support funds" and the verb phrase "application method". The output is structural information of the analyzed text data.

[1225] Step 5: Classifying Questions

[1226] The server inputs the structural information of the analyzed text data into a machine learning model (e.g., Random Forest or Support Vector Machine) to perform categorization. The input data is the structural information of the analysis results, which is classified into predefined categories. In this case, it is classified as "National Institutions." The output is the classification result.

[1227] Step 6: Generating the answer

[1228] The server queries the FAQ database based on the classification results and retrieves relevant answer information. The input data is the classification results, and the retrieved answer information is input into a natural language generation engine (e.g., GPT-3) to reconstruct it into natural-sounding sentences. For example, a specific answer text such as "You can apply for childcare support funds by following these steps..." is generated. The output is the generated answer text.

[1229] Step 7: Submit your response

[1230] The server sends the generated response as an HTTP response in JSON format to the user's terminal. The input data is the generated response text, which is stored in the HTTP response payload and sent to the terminal. The output is this HTTP response.

[1231] Step 8: Display the answer

[1232] The terminal receives a response from the server and parses the answer data within the JSON payload. The input data is the received HTTP response, which is then displayed in the application's user interface after parsing. For example, it might display something like, "You can apply for childcare support funds by following these steps..." The output is the answer displayed in the user interface.

[1233] The above outlines the specific processing flow of the program in this system. This allows users to obtain quick and accurate answers.

[1234] (Application Example 1)

[1235] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1236] Currently, it is difficult for customers to get quick and accurate answers about product information and services in physical stores. In particular, when customers ask questions about specific products or services in stores, staff members are often busy, making it difficult to get an appropriate answer immediately. In such situations, customer satisfaction decreases, which can lead to a decline in sales. Therefore, there is a need for a system that allows customers to get quick and accurate answers to their questions in physical stores.

[1237] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1238] In this invention, the server includes means for analyzing a question using a natural language processing engine, means for classifying the question, means for obtaining relevant information from a database based on the classification, means for generating an answer from the obtained relevant information using a natural language generation engine, means for transmitting the generated answer to a user terminal, means for the user terminal to display the answer, and means for a user to input a question using a smartphone application in a physical store and receive and display the generated answer in real time. This makes it possible for customers in physical stores to obtain accurate answers about product information and services in real time.

[1239] A "system" is a collection of devices and software that perform a series of processes to allow a user to input a question and obtain an appropriate answer.

[1240] A "user terminal" is a device that a user uses to input questions and receive and display answers, and includes, for example, smartphones and tablets.

[1241] An "interface" refers to the means by which data is entered between the user and the system, such as a screen or input field for the user to enter a question.

[1242] A "server" is a central processing unit that analyzes questions submitted by users and generates answers, and it is equipped with a database and natural language processing and generation engines.

[1243] A "natural language processing engine" is a software module that analyzes user-inputted questions, extracts keywords and categories, and understands their meaning.

[1244] "Classification" is the process of assigning questions analyzed by a natural language processing engine to predefined categories.

[1245] A "database" is a storage system that stores information for retrieving related information based on the aforementioned classification.

[1246] A "natural language generation engine" is a software module that automatically generates natural-sounding response sentences based on acquired relevant information.

[1247] A "physical store" is a sales and service facility that provides goods and services in a physical location, where customers visit in person to make purchases.

[1248] A "smartphone application" is a software program that runs on a smartphone, allowing the user to input questions and receive generated answers.

[1249] "Real-time" refers to the timing at which a system processes user input immediately and provides an instant response.

[1250] This invention provides a system that offers quick and accurate answers to user questions in physical stores. This system mainly consists of the following components.

[1251] First, an interface for users to input questions is provided as a smartphone application. Users launch the application in a physical store and input their questions in text format.

[1252] Next, the user terminal sends the entered question to the server using the HTTPS protocol. Here, the server's role is crucial. The server is equipped with a natural language processing engine (for example, SpaCy or NLTK) that analyzes the received question and extracts keywords and sentence components.

[1253] The server then uses a machine learning model (e.g., random forest or support vector machine) to classify the questions into predefined categories. Based on this classification, it retrieves relevant information from the database.

[1254] Based on the acquired relevant information, a natural language generation engine (for example, a generative AI model such as GPT-3) generates a response in a format that is easy for the user to understand. This generated response is then sent to the user's terminal by the server in JSON format.

[1255] Finally, the user's device parses the received response and displays it on the smartphone application's user interface. This allows the user to obtain answers to their questions in real time.

[1256] As a concrete example, consider the case where a user asks, "How do I use this product?" When the user enters the question into their smartphone and sends it, the server analyzes and classifies it. The question is classified as "How to use," and information on "How to use this product" is retrieved from the database. Subsequently, an answer is generated using a generative AI model (GPT-3), and it is displayed on the user interface as, "This product can be used in the following ways..."

[1257] Examples of prompt statements include the following:

[1258] "Category: How to use\nQuestion: Please tell me how to use this product\nAnswer:"

[1259] In this way, a system is realized that provides immediate and accurate answers to user questions in physical stores.

[1260] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1261] Step 1:

[1262] The user enters a question. Specifically, the user enters the question in text format into the interface of a smartphone application. A question such as "How do I use this product?" is entered. In this step, the input is the user's question, and the output is the entered text-formatted question.

[1263] Step 2:

[1264] The user terminal sends the entered question to the server. Specifically, the user terminal uses the HTTPS protocol to send the user-entered question to the server. The input in this step is a text-based question, and the output is an HTTP request sent to the server.

[1265] Step 3:

[1266] The server receives and parses the question. Specifically, the server receives an HTTP request and uses a natural language processing engine (e.g., SpaCy or NLTK) to parse the question and extract keywords and sentence components. The input for this step is the question text of the HTTP request, and the output is the parsed keywords and sentence components.

[1267] Step 4:

[1268] The server classifies the questions. Specifically, the server uses a machine learning model (e.g., random forest or support vector machine) to classify the analyzed question sentences into predefined categories (e.g., usage, price, specifications, etc.). The input for this step is the keywords and sentence components of the analyzed question, and the output is the appropriate category label.

[1269] Step 5:

[1270] The server retrieves relevant information from the database based on the classification. Specifically, the server queries the database and retrieves information corresponding to the classified category. The input for this step is the category label, and the output is the relevant information retrieved from the database.

[1271] Step 6:

[1272] The server uses a natural language generation engine to generate answers based on the relevant information it has acquired. Specifically, it uses a generative AI model (e.g., GPT-3) to generate answers in a natural format based on the acquired information. The input for this step is relevant information acquired from a database, and the output is an answer generated in a way that is easy for the user to understand.

[1273] Step 7:

[1274] The server sends the generated response to the user's terminal. Specifically, it sends the generated response to the user's terminal as an HTTP response in JSON format. The input for this step is the generated response text, and the output is the HTTP response sent to the user's terminal.

[1275] Step 8:

[1276] The user terminal displays the response it received. Specifically, the user terminal receives an HTTP response, parses it, and displays the response on the user interface. It might be displayed in the format, "This product can be used in the following ways..." The input for this step is the HTTP response from the server, and the output is the response displayed on the user terminal.

[1277] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1278] This invention provides a system that quickly and appropriately answers questions that users ask to government or corporate contact points, and further incorporates an emotion engine that recognizes the user's emotions. The system's core process involves the user inputting a question, which is then sent to a server for analysis and classification, generating an appropriate answer, and providing it to the user. In addition, it includes a function to recognize the user's emotions and utilize the recognition results to provide even more appropriate answers.

[1279] Overall System Overview

[1280] This system mainly consists of the following components:

[1281] 1. User Interface

[1282] 2. The terminal that sends the question data to the server.

[1283] 3. Server that receives and analyzes question data

[1284] 4. Natural Language Processing and Generation Engines

[1285] 5. Database

[1286] 6. Emotional Engine

[1287] 7. Interface for sending and displaying responses on the user's terminal.

[1288] Program processing flow

[1289] Input Interface

[1290] The user launches the application on their PC or smartphone and enters a question into the text box. For example, they might enter the question, "How do I renew my SSL certificate?" and click the submit button.

[1291] Submit a question

[1292] The terminal sends the questions entered by the user to the server using the HTTPS protocol.

[1293] Question analysis

[1294] The server parses the received question in JSON format and uses a natural language processing engine (e.g., spaCy or NLTK) to analyze the question text. Here, the keywords "SSL certificate" and the verb "renewal" are extracted.

[1295] Question classification

[1296] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the questions into predefined categories. In this case, the questions are classified as "IT technical specifications."

[1297] Emotional analysis

[1298] The server passes the question to the sentiment engine, which recognizes the user's emotions. The sentiment engine analyzes the emotions (e.g., joy, frustration, anger) from the text entered by the user. The analysis results influence the classification of the question.

[1299] Answer generation

[1300] The server queries the FAQ database and retrieves the relevant answer information. For example, information regarding "how to renew an SSL certificate" is extracted from the database. Based on the extracted information, a natural language generation engine (e.g., GPT-3) generates text in an easy-to-understand format. Furthermore, considering the results of sentiment analysis, a more user-friendly answer is generated. As a result, the answer may take the form of, "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us at any time."

[1301] Submit your response

[1302] The server sends the generated response to the user's terminal as an HTTP response in JSON format. The response data is encoded before transmission.

[1303] Display the answer

[1304] The device parses the response data it receives. The parsed data is then dynamically inserted into the application's UI components.

[1305] Confirmation of the answer

[1306] The device displays the response in the user interface. The user can see the response: "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us."

[1307] Specific example

[1308] Example 1: Questions regarding IT technical specifications

[1309] A user types "How do I renew my SSL certificate?" on their smartphone and sends it. The device sends this question to the server, which analyzes and classifies the question. The question is classified as "IT technical specification," and the sentiment engine analyzes it as "the user is feeling anxious." Information on "how to renew your SSL certificate" is retrieved from the database, and an emotionally sensitive response is generated. Finally, the message "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us anytime." is displayed.

[1310] Example 2: Questions about national systems

[1311] A user types "How do I apply for childcare support funds?" on their PC and sends it. The device sends this question to the server, which analyzes and classifies the question. The question is classified as "National System," and the sentiment engine analyzes that "the user is feeling positive emotions." Information on "How to apply for childcare support funds" is retrieved from the database, and an emotionally sensitive response is generated. Finally, the message "How to apply for childcare support funds is as follows. We hope your application goes smoothly." is displayed.

[1312] The above describes the embodiments for carrying out the present invention. This system efficiently performs a series of processes from question analysis to answer generation and emotion analysis, enabling users to obtain quick and accurate answers.

[1313] The following describes the processing flow.

[1314] Step 1:

[1315] The user launches the application on their PC or smartphone and enters the question "How do I renew my SSL certificate?" into the interface for entering questions. Then they click the submit button.

[1316] Step 2:

[1317] The terminal sends the entered question to the server using the HTTPS protocol. The question data is encoded in JSON format.

[1318] Step 3:

[1319] The server parses the received question data into JSON format. The server temporarily receives this data and passes it on to the question analysis process.

[1320] Step 4:

[1321] The server uses a natural language processing engine (such as spaCy or NLTK) to perform morphological analysis on the question text. Specifically, it tokenizes the sentence and extracts keywords and important verbs. For example, the keyword "SSL certificate" and the verb "renewal" are extracted.

[1322] Step 5:

[1323] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the questions into predefined categories. In this case, the questions are classified as "IT technical specifications."

[1324] Step 6:

[1325] The server passes the question to the sentiment engine, which analyzes the user's emotions. The sentiment engine analyzes the question and determines that "the user is feeling anxious."

[1326] Step 7:

[1327] The server queries the FAQ database and retrieves the relevant answer information. Specifically, information regarding "how to renew SSL certificates" is extracted from the database.

[1328] Step 8:

[1329] The server uses a natural language generation engine (e.g., GPT-3) to generate a response based on the extracted data. Considering the results of the sentiment engine's analysis, wording that is considerate of users experiencing anxiety is added. The generated response will be in the form of "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us."

[1330] Step 9:

[1331] The server generates the response and sends it to the user's terminal as an HTTP response in JSON format. The response data is encoded.

[1332] Step 10:

[1333] The terminal parses the response data it receives. The parsed data is then dynamically inserted into the application's user interface.

[1334] Step 11:

[1335] The device displays the response in the user interface. The user can see the response: "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us."

[1336] (Example 2)

[1337] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1338] In today's information society, users ask a variety of questions seeking quick and accurate answers. However, conventional systems often suffer from delays in responding to user questions, inconsistent response quality, and particular difficulty in providing appropriate answers that take user emotions into consideration. Furthermore, there is a lack of means to generate appropriate answers that respond to emotions. As a result, the user experience fails to reach a satisfactory level, which is a challenge.

[1339] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1340] In this invention, the server includes means for analyzing a question using a natural language processing engine, means for classifying the question using a machine learning model, and means for performing sentiment analysis of the question using a sentiment engine. This makes it possible to quickly generate an appropriate answer based on the content of the question and the user's sentiment.

[1341] A "user" is an individual or group that uses the system to input questions and receive answers.

[1342] An "interface" is a means that includes screens and input devices for users to input questions and view answers.

[1343] A "server" is a computer system that receives questions from users, analyzes them, classifies them, generates answers, and sends them back.

[1344] A "natural language processing engine" is a software engine that analyzes question texts to understand their meaning, using language models to extract keywords and important phrases.

[1345] "Question classification" is the process of classifying analyzed question texts into predefined categories using a machine learning model.

[1346] An "emotion engine" is a software engine that detects emotions from questions entered by the user and performs analysis based on those emotions.

[1347] A "database" is an information system that stores answer information to questions, and the necessary information is extracted through queries.

[1348] A "natural language generation engine" is a software engine that generates answers in an easy-to-understand and appropriate format for the user based on acquired information.

[1349] The HTTPS protocol is a communication protocol for securely transmitting data over the internet, encrypting and protecting user information.

[1350] "Response generation" is the process by which the server generates responses to provide to the user using a natural language generation engine, taking into account information obtained from the database and the results of the sentiment engine's analysis.

[1351] "Displaying the answer" is the process of presenting the generated answer to the user's terminal in an easy-to-read format.

[1352] This invention provides a system that offers quick and appropriate answers to questions users make to government or corporate contact points, and further incorporates an emotion engine that recognizes the user's emotions. This system mainly consists of the following key components.

[1353] 1. User Interface

[1354] 2. The terminal that sends the question data to the server.

[1355] 3. Server that receives and analyzes question data

[1356] 4. Natural Language Processing Engine

[1357] 5. Database

[1358] 6. Emotional Engine

[1359] 7. Interface for sending and displaying responses on the user's terminal.

[1360] The process begins when a user launches the application on their PC or smartphone and enters a question into a text box. For example, they might enter the question, "How do I renew my SSL certificate?" and click the submit button. This causes the user's device to send the question to the server using the HTTPS protocol. The server parses the received question in JSON format and analyzes it using a natural language processing engine (e.g., spaCy or NLTK). Here, the keywords "SSL certificate" and the verb "renew" are extracted.

[1361] Next, the server uses a machine learning model (e.g., random forest or support vector machine) to classify the question. It is then categorized into a specific category (in this case, "IT technical specifications"). The server then passes the question to the sentiment engine to recognize the user's emotions. The sentiment engine analyzes the emotions (e.g., joy, frustration, anger, etc.) from the text entered by the user. This analysis influences the classification of the question and the generation of answers.

[1362] The server then queries the database to retrieve the relevant response information. For example, information regarding "how to renew an SSL certificate" might be extracted from the database. Based on this extracted information, a natural language generation engine (e.g., a generative AI model) is used to generate text in an easily understandable format. Furthermore, the response is generated in an appropriate format, taking into account the results of sentiment analysis. Specifically, the response might be something like, "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us."

[1363] Finally, the server sends the generated response to the user's device as an HTTP response in JSON format. The device parses the received response data, dynamically inserts it into the application's UI components, and displays it in a way that the user can verify.

[1364] Specific example

[1365] (Example 1: Questions regarding IT technical specifications)

[1366] A user types "How do I renew my SSL certificate?" on their smartphone and sends it. The device sends this question to the server, which analyzes and classifies the question. The question is classified as "IT technical specification," and the sentiment engine analyzes it as "the user is feeling anxious." Information on "how to renew your SSL certificate" is retrieved from the database, and an emotionally sensitive response is generated. Finally, the message "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us anytime." is displayed.

[1367] (Example 2: Questions about national systems)

[1368] A user types "How do I apply for childcare support funds?" on their PC and sends it. The device sends this question to the server, which analyzes and classifies the question. The question is classified as "National System," and the sentiment engine analyzes that "the user is feeling positive emotions." Information on "How to apply for childcare support funds" is retrieved from the database, and an emotionally sensitive response is generated. Finally, the message "How to apply for childcare support funds is as follows. We hope your application goes smoothly." is displayed.

[1369] Example of a prompt

[1370] Point down

[1371] Q: How do I renew my SSL certificate?

[1372] A: Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us.

[1373] This system can efficiently perform a series of processes, from question analysis to answer generation and sentiment analysis. Furthermore, by analyzing and responding to user emotions, it can provide answers that are more satisfying.

[1374] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1375] Step 1: The user enters a question.

[1376] The user launches the application on their PC or smartphone and enters a question into a text box. Specifically, they might enter the question, "How do I renew my SSL certificate?" and click the submit button. The user's question text serves as the input and is the starting point for the entire process. The output is the question text stored on the device.

[1377] Step 2: Submit your question

[1378] The terminal uses the HTTPS protocol to send user-entered questions to the server. Specifically, it is a process of encrypting the entered question text and securely sending it to the server. The input is the user's question text, and the output is the encrypted question data sent to the server.

[1379] Step 3: Receive and analyze the questions

[1380] The server parses the received question in JSON format. Next, a natural language processing engine (e.g., spaCy or NLTK) is used to analyze the question text. Here, the keywords "SSL certificate" and the verb "renewal" are extracted. The input is encrypted question data, and the output is the analysis results, such as keywords and phrases.

[1381] Step 4: Classify the questions

[1382] The server uses a machine learning model (e.g., random forest or support vector machine) to classify questions into predefined categories. The input is the analysis results, and the output is a determined category such as "IT technical specifications."

[1383] Step 5: Analyze your emotions

[1384] The server passes the question to the emotion engine, which then recognizes the user's emotions. The emotion engine analyzes the emotions (e.g., joy, frustration, anger, etc.) from the text entered by the user. The input includes the question itself and the analysis results, and the output is an emotion analysis result such as "anxiety" or "confusion."

[1385] Step 6: Retrieve relevant information from the database.

[1386] The server queries the FAQ database and retrieves relevant answer information. For example, information about "how to renew an SSL certificate" is extracted from the database. The input includes the question category and sentiment analysis results, and the output is the relevant answer information.

[1387] Step 7: Generate the answer

[1388] Based on the information acquired by the server, a natural language generation engine (e.g., a generative AI model) is used to generate text in an easily understandable format. Furthermore, the results of sentiment analysis are taken into consideration to generate a response that is more sensitive to the user's emotions. The input consists of acquired response information and sentiment analysis results, and the output is a response that is sensitive to emotions. Specifically, it would take the form of, "Please follow the steps below to renew your SSL certificate. If you have any questions, please feel free to contact us at any time."

[1389] Step 8: Submit your response

[1390] The server sends the generated response to the user's terminal as an HTTP response in JSON format. The data is encoded before transmission. The input is the generated response text, and the output is the encoded response data delivered to the user's terminal.

[1391] Step 9: Show your answer

[1392] The device parses the received response data. The parsed data is dynamically inserted into the application's UI components and displayed in a format viewable by the user. The input is encoded response data, and the output is a viewable response text displayed on the device screen.

[1393] Step 10: Check your answers

[1394] The user reviews the response. Specifically, by viewing the response, "Please follow these steps to renew your SSL certificate. If you have any questions, please feel free to contact us," the user can obtain the information they were looking for. The input is the displayed response, and the output is the user reviewing its contents.

[1395] (Application Example 2)

[1396] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1397] Traditional systems suffered from a decline in the quality of the user experience because they did not consider the user's emotions during the process from when the user entered a question to when they received an answer. Furthermore, in stressful environments such as factories, it is necessary to appropriately handle user anxiety and dissatisfaction. However, with current technology, it has been difficult to recognize user emotions and provide answers that respond accordingly.

[1398] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for providing an interface for the user to input a question, means for transmitting the question to the server, means for the server to analyze the question using a natural language processing engine, means for classifying the question, means for obtaining relevant information from a database based on the classification, means for generating an answer from the obtained relevant information using a natural language generation engine, means for transmitting the generated answer to a user terminal, means for the user terminal to display the answer, means for recognizing the user's emotions from the input question, and means for generating an answer considering the emotion recognition result. This makes it possible to improve the quality of the user experience by recognizing the user's emotions and providing an answer accordingly.

[1399] "User interface" refers to the screens and operating methods used by users to input questions.

[1400] A "server" refers to a computer system that receives and processes requests from clients via a network.

[1401] A "natural language processing engine" refers to a system that includes algorithms and models for analyzing text data and understanding its meaning.

[1402] "Question classification" refers to the process of categorizing user questions into specific categories or tags.

[1403] A "database" refers to a collection of information that is systematically organized and stored, making it easy to search and retrieve.

[1404] A "natural language generation engine" refers to a system that includes algorithms and models for generating responses in a natural format based on acquired relevant information.

[1405] "Emotion recognition" refers to the process of analyzing and identifying emotions (such as joy, dissatisfaction, or anger) from text entered by the user.

[1406] A "user terminal" refers to a device (such as a smartphone, tablet, or PC) used by a user to access the server and exchange questions and answers.

[1407] This invention relates to an interactive QA system for factory robots. This system allows factory workers to ask questions about robot operation and maintenance, and provides appropriate and prompt answers. It also has a function to recognize the user's emotions and generate corresponding responses.

[1408] System Configuration

[1409] This system consists of the following components.

[1410] 1. User Interface: An interface for the worker to input questions. Examples include using smart glasses or voice input devices.

[1411] 2. Server: A central processing unit that receives questions, analyzes them, and generates answers. The server includes a natural language processing engine (e.g., spaCy), an emotion recognition engine (e.g., Hugging Face Transformers), and a natural language generation engine (e.g., OpenAI GPT-3).

[1412] 3. Database: A collection of information that stores relevant information for a query. Examples include PostgreSQL and MySQL.

[1413] 4. User terminal: A device that displays answers to the worker. This includes smartphones, tablets, and PCs.

[1414] Program processing flow

[1415] User Interface

[1416] The worker wears smart glasses and inputs a question by voice, such as, "Please tell me how to maintain the robot."

[1417] Submit a question

[1418] The smart glasses convert the voice data into text format and send the question to the server. For speech recognition, APIs such as Google Cloud Speech-to-Text can be used.

[1419] Question analysis

[1420] The server analyzes the received question using a natural language processing engine (e.g., spaCy). Keywords such as "maintenance" and "robot" are extracted from the text data.

[1421] Question classification

[1422] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the question. The question is classified as "technical support".

[1423] emotion recognition

[1424] The server uses an emotion analysis engine (e.g., Hugging Face Transformers) to recognize the worker's emotions from the input text. The emotion analysis result is determined to be "anxiety."

[1425] Answer generation

[1426] The server retrieves relevant information from the database and generates a response using a natural language generation engine (e.g., OpenAI GPT-3). The response is generated in the format of "Robot maintenance is performed using the following steps. If you have any questions, please contact support."

[1427] Submit your response

[1428] The server sends the generated response to the user's terminal. The user's terminal displays the received response so that the worker can confirm it.

[1429] Specific example

[1430] Example 1: Questions about robot maintenance

[1431] If a worker asks, "How do I calibrate the robot arm?", the system will operate as follows: The question is analyzed, relevant information is retrieved from the database, and, taking sentiment into consideration, an answer is generated stating, "The robot arm calibration procedure is as follows. If you have any questions, please contact support."

[1432] Example of a prompt

[1433] The user is asking about the calibration method for a robot arm. Emotion detected: Anxiety. Provide a detailed but friendly response.

[1434] This invention recognizes user emotions and provides corresponding responses, enabling factory workers to quickly obtain information and improve work efficiency. This, in turn, enhances overall factory productivity and reduces anxiety and stress.

[1435] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1436] Step 1:

[1437] The user wears smart glasses and uses a voice input device to enter a question. For example, they might ask, "How do I calibrate the robot arm?" The smart glasses capture the voice data as input and convert it into text data. The converted text data is then output.

[1438] Step 2:

[1439] The terminal sends the text data converted from the speech to the server using the HTTPS protocol. The server receives this text data and takes it as input. The output is a request body containing the text data.

[1440] Step 3:

[1441] The server analyzes the received text data using a natural language processing engine (e.g., spaCy). Specifically, it splits the text data and extracts keywords and important phrases. This process produces a list of analyzed keywords as output.

[1442] Step 4:

[1443] The server uses a machine learning model (e.g., random forest or support vector machine) to classify the analyzed text data into a specific category. A keyword list is used as input, and the output category is "Technical Support".

[1444] Step 5:

[1445] The server uses an emotion analysis engine (e.g., Hugging Face Transformers) to recognize the worker's emotions from the input text. In this step, text data is used as input, and the emotion analysis result (e.g., "anxious") is output.

[1446] Step 6:

[1447] The server retrieves relevant information from the database. The classified categories and keywords are used as input, and a database query is executed to retrieve the necessary information. The output is the relevant information (e.g., "Calibration Method").

[1448] Step 7:

[1449] The server uses a natural language generation engine (e.g., OpenAI GPT-3) to generate a response based on the acquired relevant information and sentiment recognition results. This involves inputting a prompt sentence into a generation AI model, and obtaining the generated response text as its output. For example, the prompt sentence is "The user is asking about the calibration method for a robot arm. Emotion detected: Anxiety. Provide a detailed but friendly response."

[1450] Step 8:

[1451] The server sends the generated response to the user's terminal. The HTTPS protocol is used for transmission, and the generated response text is included as output.

[1452] Step 9:

[1453] The user's device displays the received response. The response text is displayed on the smart glasses or smartphone screen, allowing the user to confirm the answer to the question. In this step, the response text is passed to the device as input and is visually displayed as output.

[1454] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1455] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1456] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1457] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1458] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1459] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1460] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1461] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1462] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1463] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1464] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1465] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1466] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1467] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1468] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1469] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1470] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1471] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1472] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1473] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1474] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[1475] The following is further disclosed regarding the embodiments described above.

[1476] (Claim 1)

[1477] A means of providing an interface for users to input questions,

[1478] Means for sending the aforementioned question to the server,

[1479] The server provides means for analyzing the question using a natural language processing engine,

[1480] A means for classifying the aforementioned questions,

[1481] A means for obtaining relevant information from a database based on the aforementioned classification,

[1482] A means for generating an answer using a natural language generation engine based on the acquired related information,

[1483] Means for sending the generated response to the user terminal,

[1484] The user terminal includes means for displaying the answer,

[1485] A system that includes this.

[1486] (Claim 2)

[1487] The system according to claim 1, wherein the server has means for classifying the questions using a machine learning model.

[1488] (Claim 3)

[1489] The system according to claim 1, wherein the user terminal has means for transmitting the question to the server using the HTTPS protocol.

[1490] "Example 1"

[1491] (Claim 1)

[1492] A means of providing an interface for users to input questions,

[1493] Means for sending the aforementioned question to the server,

[1494] The server has means for receiving and parsing the question in JSON format,

[1495] A means for analyzing the aforementioned question using a natural language processing engine,

[1496] A means for classifying the aforementioned questions using a machine learning model,

[1497] A means for obtaining relevant information from a database based on the aforementioned classification,

[1498] A means for generating an answer using a natural language generation engine based on the acquired related information,

[1499] Means for sending the generated response to the user terminal,

[1500] The user terminal includes means for parsing and displaying the response,

[1501] A system that includes this.

[1502] (Claim 2)

[1503] The system according to claim 1, wherein the server has means of using a text analysis engine as a natural language processing engine.

[1504] (Claim 3)

[1505] The system according to claim 1, wherein the user terminal has means for transmitting the question to the server using the HTTPS protocol.

[1506] "Application Example 1"

[1507] (Claim 1)

[1508] A means of providing an interface for users to input questions,

[1509] Means for sending the aforementioned question to the server,

[1510] The server provides means for analyzing the question using a natural language processing engine,

[1511] A means for classifying the aforementioned questions,

[1512] A means for obtaining relevant information from a database based on the aforementioned classification,

[1513] A means for generating an answer using a natural language generation engine based on the acquired related information,

[1514] Means for sending the generated response to the user terminal,

[1515] The user terminal includes means for displaying the answer,

[1516] A means for a user to input the aforementioned questions using a smartphone application in a physical store, and for receiving and displaying the generated answers in real time,

[1517] A system that includes this.

[1518] (Claim 2)

[1519] The system according to claim 1, wherein the server has means for classifying the questions using a machine learning model.

[1520] (Claim 3)

[1521] The system according to claim 1, wherein the user terminal has means for transmitting the question to the server using the HTTPS protocol.

[1522] "Example 2 of combining an emotion engine"

[1523] (Claim 1)

[1524] A means of providing an interface for users to input questions,

[1525] Means for sending the aforementioned question to the server,

[1526] The server provides means for analyzing the question using a natural language processing engine,

[1527] A means for classifying the aforementioned questions,

[1528] The server provides means for performing sentiment analysis on the question using a sentiment engine,

[1529] A means for obtaining relevant information from a database based on the aforementioned classification and sentiment analysis,

[1530] A means for generating an emotionally sensitive response from the acquired related information using a natural language generation engine,

[1531] Means for sending the generated response to the user terminal,

[1532] The user terminal includes means for displaying the answer,

[1533] A system that includes this.

[1534] (Claim 2)

[1535] The system according to claim 1, wherein the server has means for classifying the questions using a machine learning model.

[1536] (Claim 3)

[1537] The system according to claim 1, wherein the user terminal has means for transmitting the question to the server using the HTTPS protocol.

[1538] "Application example 2 when combining with an emotional engine"

[1539] (Claim 1)

[1540] A means of providing an interface for users to input questions,

[1541] Means for sending the aforementioned question to the server,

[1542] The server provides means for analyzing the question using a natural language processing engine,

[1543] A means for classifying the aforementioned questions,

[1544] A means for obtaining relevant information from a database based on the aforementioned classification,

[1545] A means for generating an answer using a natural language generation engine based on the acquired related information,

[1546] Means for sending the generated response to the user terminal,

[1547] The user terminal includes means for displaying the answer,

[1548] A means for recognizing the user's emotions from the aforementioned questions that were entered,

[1549] Means for generating a response considering the aforementioned emotion recognition results,

[1550] A system that includes this.

[1551] (Claim 2)

[1552] The system according to claim 1, wherein the server has means for classifying the questions using a machine learning model.

[1553] (Claim 3)

[1554] The system according to claim 1, wherein the user terminal has means for transmitting the question to the server using the HTTPS protocol. [Explanation of Symbols]

[1555] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of providing an interface for users to input questions, Means for sending the aforementioned question to the server, The server provides means for analyzing the question using a natural language processing engine, A means for classifying the aforementioned questions, A means for obtaining relevant information from a database based on the aforementioned classification, A means for generating an answer using a natural language generation engine based on the acquired related information, Means for sending the generated response to the user terminal, The user terminal includes means for displaying the answer, A system that includes this.

2. The system according to claim 1, wherein the server has means for classifying the questions using a machine learning model.

3. The system according to claim 1, wherein the user terminal has means for transmitting the question to the server using the HTTPS protocol.

Images