system

Abstract

We provide the system. [Solution] A means for collecting inquiry data, A means for preprocessing the collected query data, A method for training a machine learning model using preprocessed data, A means of receiving real-time inquiries from users, A means for analyzing received inquiries and generating the optimal answer using a trained model, A system that includes means of providing the generated response to the user.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern times, customer support has become an important factor influencing a company's competitiveness. However, in many companies, it is difficult to respond promptly and accurately to inquiries from users, especially the response to free-form questions is insufficient, which has become a problem. Conventional systems cannot cope with the diversity of inquiry contents, which has become a factor reducing user satisfaction.

[0005] In addition, there are different needs in the inquiry contents of new users and existing users, and there is a lack of a system that can handle both. As a result, the quality of inquiry response deteriorates, and there is a risk of losing users' trust. Furthermore, since there is no mechanism to improve the accuracy of the system based on feedback, it is difficult to make long-term improvements.

[0006] A new system is needed to solve the above problems and provide prompt and accurate user support. [Means for solving the problem]

[0007] The present invention provides a system that includes means for collecting and preprocessing past inquiry data, and means for training a machine learning model based on that data. It also includes means for receiving real-time inquiries from users, analyzing them, and generating optimal answers. Furthermore, it includes means for providing the generated answers to users.

[0008] Specifically, the problem will be solved by combining the following methods:

[0009] 1. Means of collecting inquiry data: Collect previously accumulated inquiry data from a database.

[0010] 2. Means of preprocessing collected query data: Data cleansing, normalization, and tokenization are performed to improve data quality and consistency.

[0011] 3. Method for training a machine learning model using preprocessed data: Train a model using natural language processing techniques to learn the correspondence between inquiries and answers.

[0012] 4. Means for receiving real-time inquiries from users: The server receives user questions sent from the terminal.

[0013] 5. Means for analyzing received inquiries and generating optimal answers using a trained model: Analyze the content of the received inquiry and generate relevant answers.

[0014] 6. Means of providing the generated response to the user: Send the generated response to the user's device and display it.

[0015] Furthermore, by providing means to collect feedback from users and update the machine learning model based on it, it is possible to continuously improve the accuracy and performance of the system. In this way, the present invention improves the efficiency of responding to inquiries from new and existing users and enhances user satisfaction.

[0016] "Inquiry data" refers to information regarding questions and requests made by users to companies or service providers.

[0017] "Preprocessing" is a process for preparing data in a format suitable for a machine learning model, and specifically includes data cleaning, normalization, tokenization, etc.

[0018] A "machine learning model" is an algorithm that learns patterns and rules based on past data and makes predictions or classifications for newly input data.

[0019] "Real-time inquiry" refers to the on-the-spot input of questions and requests made by users through terminals.

[0020] "Analysis" is an information processing means for understanding the content of inquiry data and determining appropriate countermeasures.

[0021] "Generate" is a process of creating an appropriate answer to a user's question based on the results of analysis.

[0022] "Provide to the user" is an act of sending the generated answer to the user's terminal so that the user can view it.

[0023] "Feedback" is an evaluation or reaction from the user and is information that serves as a basis for improving the system and enhancing accuracy.

Brief Description of the Drawings

[0024] [Figure 1]It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

MODE FOR CARRYING OUT THE INVENTION

[0025] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0026] First, let's explain the terminology used in the following explanation.

[0027] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0028] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0029] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0030] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0031] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0032] [First Embodiment]

[0033] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0034] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0035] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0036] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0037] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0038] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0039] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0040] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0041] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0042] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0043] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0044] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0045] This invention relates to a system that collects and preprocesses past inquiry data and trains a machine learning model based on it, thereby generating and providing immediate and accurate answers to real-time inquiries from users. The general program of the system and its processing are described below.

[0046] System Overview

[0047] Collection and preprocessing of query data

[0048] The server collects user inquiry data accumulated in the past from the database. Since the collected data is often incomplete as is, data cleansing is used to remove noise and unnecessary data and to prepare it in an appropriate format. Next, tokenization is performed to divide the text data into tokens (words and phrases), converting it into a form that can be easily processed by machines.

[0049] Model training

[0050] The server trains a machine learning model using preprocessed data. Specifically, it selects a model using natural language processing techniques (e.g., BERT, GPT), splits the data into training and validation datasets, and trains the model to learn the relationship between queries and corresponding answers. Through this process, the model acquires the ability to generate appropriate answers for various types of queries.

[0051] Real-time inquiry analysis

[0052] When a user enters a question from their device, the data is sent to the server via the device. The server receives the submitted query, performs tokenization and normalization to transform it into a format that is easy to analyze. Next, it uses a trained model to analyze the query and generate the optimal answer.

[0053] Providing a response

[0054] The server sends the generated response to the terminal, which then displays the response to the user. This allows the user to obtain an accurate answer immediately.

[0055] Specific example

[0056] Example 1: Inquiry about login methods for new users

[0057] The user types "I don't know how to log in for the first time" from their device. The device sends this question to the server, which parses the received query. The trained model generates the best answer regarding "how to log in," and the server sends the answer "For your first login, please use the email address you registered and enter the verification code sent to you" to the device. The device then displays this answer to the user.

[0058] Example 2: Inquiry about update details for existing users

[0059] The user types "What were the updates from last week?" into their device. The device sends this question to the server, which analyzes the received query. The trained model generates the best possible answer regarding "updates" and sends the response "Last week's update included the addition of XX as a new feature and YY as a bug fix" from the server to the device. The device then displays this response to the user.

[0060] Feedback and Model Updates

[0061] It is also possible to add a feature to the device that allows users to provide feedback on the answers they have provided. This feedback is sent to the server and used to improve the system. The server can use this data to retrain machine learning models and improve the system's accuracy and user satisfaction.

[0062] As described above, this system efficiently analyzes inquiry data and enables quick and accurate responses to inquiries from both new and existing users.

[0063] The following describes the processing flow.

[0064] Step 1:

[0065] The server collects past query data from the database. The database contains a wide variety of queries and their corresponding answers.

[0066] Step 2:

[0067] The server performs data cleansing on the collected query data. At this stage, noise (inappropriate data and errors) is removed, improving the quality of the data.

[0068] Step 3:

[0069] The server normalizes the data. Specifically, it converts uppercase English letters to lowercase and removes special characters to arrange the data into a consistent format.

[0070] Step 4:

[0071] The server performs tokenization, which divides the data into tokens (words or phrases). This converts the text data into a format that is easier to parse.

[0072] Step 5:

[0073] The server trains a machine learning model using preprocessed data. For training, it selects an algorithm that uses natural language processing techniques (e.g., BERT, GPT, etc.) and trains the model by dividing the dataset.

[0074] Step 6:

[0075] The server starts a real-time analysis system to process new queries, making it ready to receive user inquiries immediately.

[0076] Step 7:

[0077] Users will enter questions in a free-form format via their device. Specific questions such as "I don't know how to log in for the first time" are expected.

[0078] Step 8:

[0079] The terminal sends the questions entered by the user to the server as text data.

[0080] Step 9:

[0081] The server preprocesses the text data received from the terminal for analysis. Tokenization and normalization are performed again at this stage.

[0082] Step 10:

[0083] The server uses a machine learning model to generate the optimal response based on pre-processed query data. The model predicts the response based on historical data.

[0084] Step 11:

[0085] The server sends the generated response as text data to the terminal.

[0086] Step 12:

[0087] The device displays the received response in the user interface. For example, it might say, "For your first login, please use the email address you registered and enter the verification code that was sent to you."

[0088] Step 13:

[0089] Users can rate their satisfaction with the provided answers. Satisfaction feedback is entered on the device and used to improve the system.

[0090] Step 14:

[0091] The device sends user feedback to the server.

[0092] Step 15:

[0093] The server analyzes the received feedback and uses it to update its machine learning model. This improves the accuracy and quality of future inquiries.

[0094] The above outlines the specific processing steps from collecting inquiry data to providing responses and further improving the system based on feedback.

[0095] (Example 1)

[0096] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0097] Conventional inquiry handling systems often suffered from delays in responding to user inquiries and had limited performance in generating appropriate answers. Furthermore, updating models based on feedback was cumbersome, posing a risk of decreased accuracy. Therefore, there was a need for a system that could provide real-time, rapid, and accurate answers.

[0098] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0099] In this invention, the server includes means for collecting query data, means for preprocessing the collected query data, means for training a machine learning model using the preprocessed data, means for receiving real-time queries from users, means for analyzing the received queries and generating the optimal answer using the trained model, means for denoising and formatting the data, means for using natural language processing technology as the machine learning model, and means for providing the generated answer to the user. This makes it possible to provide fast and accurate answers in real time.

[0100] "Inquiry data" refers to data that includes information about questions and requests from users.

[0101] "Preprocessing" refers to the process of applying noise reduction and formatting to collected data to make it easier to analyze.

[0102] A "machine learning model" is an algorithm that learns from input data and performs predictions and classifications for specific tasks.

[0103] "Training" is the process by which a machine learning model learns using pre-processed data.

[0104] A "real-time inquiry" is a question or request that a user makes to a system instantly.

[0105] "Noise reduction" is the process of removing unwanted or inaccurate information from data.

[0106] "Formatting" refers to the process of aligning data to a specific format or structure.

[0107] "Natural language processing technology" is the technology that enables computers to understand and generate human language.

[0108] "Feedback" refers to the evaluations and comments that users make regarding the answers provided.

[0109] "Tokenization" is the process of dividing text data into units such as words and phrases.

[0110] "Normalization" is the process of converting data into a standard format in order to maintain data consistency.

[0111] This invention is a system that efficiently collects and analyzes inquiry data and provides quick and accurate responses to real-time inquiries from users. This system uses the following hardware and software.

[0112] hardware

[0113] Server: A central processing unit for data collection, preprocessing, model training, query analysis, response generation, and delivery. This server can utilize data center-level computers equipped with high-performance CPUs and GPUs.

[0114] Terminal: A device used by users to input inquiries and receive responses. This includes common computing devices such as personal computers, tablets, or smartphones.

[0115] software

[0116] Database system: Stores and manages query data. Specifically, it uses relational databases such as MySQL (registered trademark) or PostgreSQL.

[0117] Data cleansing tools are software that removes noise and formats collected data. An example is OpenRefine.

[0118] Natural language processing libraries: These perform tokenization and normalization of text data. Specifically, libraries such as NLTK and spaCy are used.

[0119] Machine learning frameworks: These frameworks train machine learning models based on preprocessed data and analyze queries. Specific examples include Tensorflow®, PyTorch, and Transformers libraries (BERT, GPT, etc.).

[0120] Web server: Receives and provides responses to queries in real time. Python frameworks such as FastAPI and Flask are used.

[0121] Specific example

[0122] When a user types "I don't know how to log in for the first time" from their computer, the device sends this inquiry to the server. The server tokenizes and normalizes the received inquiry data using natural language processing libraries (NLTK, spaCy). A trained machine learning model (e.g., BERT, GPT) is used to analyze the inquiry and generate the most appropriate response. In this case, the response generated is "For your first login, please use your registered email address and enter the verification code sent to you." The generated response is sent from the server to the device and displayed to the user.

[0123] Example of a prompt

[0124] For example, the following types of user inquiries can be handled in a similar manner:

[0125] "What should I do if I forget my password?"

[0126] "I want to know what's in the latest update."

[0127] This allows users to receive accurate answers immediately and solve problems efficiently. Furthermore, user feedback can be collected, and the server can use this data to retrain the model and improve the system's accuracy.

[0128] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0129] Step 1: Collecting inquiry data

[0130] The server collects user query data accumulated in the past from a database. This database is a relational database system (e.g., MySQL, PostgreSQL). The server executes SQL queries to extract the query data and converts it into a data frame format (e.g., Pandas DataFrame).

[0131] Input: Query data in the database

[0132] Output: Query data in data frame format

[0133] Step 2: Data preprocessing

[0134] The server performs data cleansing on the collected data. Specifically, it uses Pandas to remove noisy data (e.g., missing values, redundant data) and prepares the data in a clean state. Next, it uses a natural language processing library (e.g., NLTK, spaCy) to tokenize the text data and then format it.

[0135] Input: Query data in data frame format

[0136] Output: Tokenized and cleansed query data

[0137] Step 3: Training the machine learning model

[0138] The server trains a machine learning model using preprocessed data. Here, we implement a model using natural language processing techniques (e.g., BERT, GPT). The data is split into training and validation datasets, and the model is trained using query-and-appropriate response pairs. Frameworks such as TensorFlow and PyTorch are used.

[0139] Input: Tokenized query data and corresponding response data

[0140] Output: Trained machine learning model

[0141] Step 4: Receiving real-time inquiries

[0142] When a user enters a query from their device, that data is sent to the server via the device. Here, JavaScript (registered trademark) is used for the frontend, and a lightweight web framework such as FastAPI is used for the backend.

[0143] Input: Inquiry text entered by the user

[0144] Output: Sending query data to the server

[0145] Step 5: Analyze the inquiry and generate the answer.

[0146] The server analyzes incoming queries in real time. First, it uses a natural language processing library to tokenize and normalize the queries, and then uses a trained machine learning model to generate the optimal response.

[0147] Input: Received query data

[0148] Output: Generated response data

[0149] Step 6: Provide your response

[0150] The server sends the generated response to the terminal, and the terminal displays that response to the user. HTTP or WebSocket is used as the communication protocol for this process.

[0151] Input: Server-generated response data

[0152] Output: Answer displayed to the user

[0153] Step 7: Gathering Feedback and Updating the Model

[0154] Users provide feedback on the provided answers. This feedback is sent to the server via the terminal. The server retrains the model based on the feedback data to improve the system's accuracy.

[0155] Input: User-entered feedback data

[0156] Output: Updated machine learning model

[0157] (Application Example 1)

[0158] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0159] Traditional customer support systems struggled to provide quick and accurate responses to real-time user inquiries. Furthermore, there was a lack of means to improve the quality and efficiency of customer service in physical stores. In particular, during peak hours, staff shortages led to customers having to wait, which was a significant problem.

[0160] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0161] In this invention, the server includes means for collecting inquiry data, means for preprocessing the collected inquiry data, means for training a machine learning model using the preprocessed data, means for receiving real-time inquiries from users, means for analyzing the received inquiries and generating the optimal answer using the trained model, means for providing the generated answer to the user, means for installing it on a robot placed in a physical store, and means for the robot to receive customer inquiries and provide answers. This enables quick and accurate customer service even in physical stores.

[0162] "Inquiry data" refers to data related to questions and requests from customers and users.

[0163] "Preprocessing" refers to a series of processes that convert raw data into a format suitable for analysis and training.

[0164] A "machine learning model" is an algorithm that learns from large amounts of data and performs predictions and classifications for specific tasks.

[0165] A "real-time inquiry" is an immediate question or request made by a user during an ongoing session.

[0166] "Analysis" is the process of interpreting data and information to derive meaning in accordance with a specific purpose.

[0167] "Answer generation" is the process of constructing an appropriate response to an inquiry.

[0168] "Providing" refers to the act of showing or sending the generated response to the user.

[0169] A "physical store" is a commercial facility that provides goods and services in a physical location.

[0170] A "robot" is a mechanical device that performs specific tasks autonomously or remotely.

[0171] "Feedback" refers to user opinions and evaluations regarding the system's operation and the responses provided.

[0172] "Tokenization" is the process of dividing text data into words or phrases.

[0173] "Normalization" is the process of arranging the format and content of data into a consistent and standard form.

[0174] This invention relates to a system that uses robots placed in physical stores to provide appropriate answers to customer inquiries in real time.

[0175] Collection and preprocessing of query data

[0176] The server collects past query data from the database. Since the collected data is often incomplete, data cleansing techniques are used to remove noise and unnecessary data, and to prepare it in an appropriate format. Then, tokenization is performed to divide the text data into words and phrases, and further normalization is carried out to convert it into a form that is easy to analyze.

[0177] Model training

[0178] The server trains a machine learning model using pre-processed data. This process utilizes natural language processing techniques, and models such as BERT and GPT are selected. The collected data is divided into a training dataset and a validation dataset, and the relationship between queries and corresponding answers is learned.

[0179] Real-time inquiry analysis

[0180] When a customer makes a request to a robot placed in a physical store, the data is sent by the robot to a server. The server tokenizes and normalizes the received data, and then uses a trained model to analyze the request and generate the best possible answer.

[0181] Providing a response

[0182] The server sends the generated response to the robot, which then provides the response to the customer. This allows the customer to receive the appropriate answer immediately.

[0183] Specific hardware and software

[0184] This invention primarily uses Python 3.x, the Transformers library (provided by Hugging Face), and PyTorch as its software. For hardware, the CPU and GPU installed in the customer service robot are used. The customer service robot is equipped with a touchscreen to allow customers to easily input questions.

[0185] Specific example

[0186] For example, if a customer types "What are your store's opening hours?" into the robot's touchscreen, the robot sends the question to the server. The server analyzes the question, generates an answer such as "Our store's opening hours are from 10 AM to 8 PM," and sends it to the robot. The robot then provides this answer to the customer.

[0187] Example of a prompt

[0188] "Please explain how to provide appropriate answers to customer inquiries regarding store hours in a physical store setting."

[0189] "How can we develop a system that can respond immediately in real time to customer inquiries about product inventory status?"

[0190] This invention enables prompt and accurate customer service in physical stores, improving the user experience and increasing the efficiency of store operations.

[0191] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0192] Step 1:

[0193] The server collects past query data from the database. The collected query data, in its raw form, may contain noise and unnecessary data. This data is then cleansed using data cleansing techniques to remove the unnecessary parts and format it appropriately. Through this process, the server obtains pre-processed query data.

[0194] Step 2:

[0195] The server performs tokenization and normalization on the preprocessed query data. Specifically, it uses a natural language processing library to split the text data into words and phrases and convert it into a consistent format. Through this process, the server obtains data that is in a form that is easy to parse.

[0196] Step 3:

[0197] The server trains a machine learning model using preprocessed data. It primarily uses the Transformers library (provided by Hugging Face) and PyTorch. In this step, the data is split into training and validation datasets, and the relationship between queries and corresponding answers is learned. As a result of this process, the server obtains a trained machine learning model.

[0198] Step 4:

[0199] The user enters their inquiry into a customer service robot in a physical store. The inquiry, entered using the robot's touchscreen, is sent by the robot to a server. In this step, the user's question is transmitted to the server via the robot.

[0200] Step 5:

[0201] The server then performs tokenization and normalization again on the query received from the robot. It then analyzes the query using a trained machine learning model and generates the optimal response. This step ensures the server generates a suitable answer, which it then sends to the robot.

[0202] Step 6:

[0203] The robot displays the answers received from the server and provides them to the user. Specifically, the answers are displayed on the robot's touchscreen, allowing the user to instantly obtain the appropriate information.

[0204] Step 7:

[0205] Users can provide feedback on the provided answers. This feedback is sent back to the server and used to improve the model in the future. By collecting and analyzing feedback, the server can improve the accuracy of the machine learning model.

[0206] The above outlines the processing steps for a system used for customer service in physical stores.

[0207] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0208] This invention relates to a system that generates and provides immediate and accurate answers to real-time user inquiries by collecting and preprocessing past inquiry data and training a machine learning model based on that data. Furthermore, it aims to further improve user satisfaction by incorporating an emotion engine that recognizes user emotions.

[0209] System Overview

[0210] Collection and preprocessing of query data

[0211] The server collects user inquiry data accumulated in the past from a database. The database stores a wide variety of inquiries and their corresponding answers. Since much of the collected data is incomplete in its raw form, data cleansing is used to remove noise and unnecessary data and improve data quality. Next, tokenization is performed to divide the text data into tokens (words and phrases) to prepare it for easier analysis.

[0212] Model training

[0213] The server trains a machine learning model using pre-processed data. For training, it selects an algorithm using natural language processing techniques (e.g., BERT, GPT, etc.) and trains the model on a divided dataset. This allows the server to learn from past queries and corresponding answers, and generate appropriate responses to queries.

[0214] Real-time inquiry analysis

[0215] When a user enters a question from their device, the data is sent to the server via the device. The server receives the submitted query, performs tokenization and normalization to transform it into a format that is easy to analyze. Next, it uses a trained model to analyze the query and generate the optimal answer.

[0216] Introducing an emotional engine

[0217] The server analyzes the received user inquiry text using an emotion engine. The emotion engine extracts emotions from the user's text and classifies them into emotion categories such as positive, negative, and neutral. Based on this classification, it adjusts the appropriate response or reply.

[0218] Providing a response

[0219] The server sends the generated response to the terminal, which then displays the response to the user. By taking the results of the sentiment engine into account, for example, a helpful tone is provided to a user who is "in distress," while an apology or a quick solution is offered to a user who is "dissatisfied."

[0220] Specific example

[0221] Example 1: Inquiry about login methods for new users

[0222] The user types "I don't know how to log in for the first time" from their device. The device sends this question to the server, which analyzes the received query. The sentiment engine recognizes this query as "confused." The trained model generates the best answer regarding "how to log in" and sends the response "For your first login, please use the email address you registered and enter the verification code sent to you. If you have any questions, please contact us anytime" from the server to the device. The device then displays this response to the user.

[0223] Example 2: Inquiry about update details for existing users

[0224] The user types "What was updated last week?" into their device. The device sends this question to the server, which analyzes the received query. The sentiment engine categorizes it as "neutral." The trained model generates the best possible answer regarding "updates" and sends the response "Last week's update added XX as a new feature and fixed YY as a bug" from the server to the device. The device then displays this response to the user.

[0225] Feedback and Model Updates

[0226] It is also possible to add a feature to the device that allows users to provide feedback on the answers they have received. This feedback is sent to the server and used to improve the system. The server can use this data to retrain the machine learning model, improving the system's accuracy and user satisfaction. Furthermore, by incorporating the results of the emotion engine as training data, the system can learn which emotions are most effectively addressed in its responses.

[0227] As described above, this system includes specific processing steps ranging from collecting inquiry data to providing answers and further improving the system based on feedback, enabling it to recognize user emotions and provide more appropriate support.

[0228] The following describes the processing flow.

[0229] Step 1:

[0230] The server collects past query data from the database. The database contains a wide variety of queries and their corresponding answers.

[0231] Step 2:

[0232] The server performs data cleansing on the collected query data. At this stage, noise (inappropriate data and errors) is removed, improving the quality of the data.

[0233] Step 3:

[0234] The server normalizes the data. Specifically, it converts uppercase English letters to lowercase and removes special characters to arrange the data into a consistent format.

[0235] Step 4:

[0236] The server performs tokenization, which divides the data into tokens (words or phrases). This converts the text data into a format that is easier to parse.

[0237] Step 5:

[0238] The server trains a machine learning model using preprocessed data. For training, it selects an algorithm that uses natural language processing techniques (e.g., BERT, GPT, etc.) and trains the model by dividing the dataset.

[0239] Step 6:

[0240] The server starts a real-time analysis system to process new queries, making it ready to receive user inquiries immediately.

[0241] Step 7:

[0242] Users will enter questions in a free-form format via their device. Specific questions such as "I don't know how to log in for the first time" are expected.

[0243] Step 8:

[0244] The terminal sends the questions entered by the user to the server as text data.

[0245] Step 9:

[0246] The server preprocesses the text data received from the terminal for analysis. Tokenization and normalization are performed again at this stage.

[0247] Step 10:

[0248] The server uses a machine learning model to generate the optimal response based on pre-processed query data. The model predicts the response based on historical data.

[0249] Step 11:

[0250] The server analyzes the received text data using an emotion engine. The emotion engine extracts emotions from the user's text and classifies them into emotion categories such as positive, negative, and neutral.

[0251] Step 12:

[0252] The server adjusts its responses based on the results of the emotion engine's analysis. For example, if the user is "confused," it will generate a response in a kind and caring tone.

[0253] Step 13:

[0254] The server sends the generated response to the terminal.

[0255] Step 14:

[0256] The device displays the received response in the user interface. For example, it might say, "For your first login, please use the email address you registered and enter the verification code that was sent to you."

[0257] Step 15:

[0258] Users can rate their satisfaction with the provided answers. Satisfaction feedback is entered on the device and used to improve the system.

[0259] Step 16:

[0260] The device sends user feedback to the server.

[0261] Step 17:

[0262] The server analyzes the received feedback and uses it to update its machine learning model. This improves the accuracy and quality of future inquiries.

[0263] Step 18:

[0264] The server also incorporates the results from the emotion engine as training data. This allows the system to continuously learn which emotions are most effectively addressed.

[0265] The above outlines the specific processing steps for an inquiry handling system that incorporates an emotion engine.

[0266] (Example 2)

[0267] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0268] Conventional inquiry handling systems have faced challenges in responding to user inquiries quickly and accurately, and insufficiently considering user emotions. Furthermore, they have been unable to effectively utilize feedback on the answers provided, limiting improvements in system performance and user satisfaction.

[0269] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0270] In this invention, the server includes means for collecting query data from a wide range of sources, means for preprocessing the collected query data through data cleansing and tokenization, and means for training a machine learning model using the preprocessed data. This enables the server to provide quick and accurate answers to user inquiries. The server further includes means for adjusting answers using an emotion engine that analyzes user emotions, means for providing the adjusted answers to the user, means for collecting user feedback, means for updating the machine learning model based on the collected feedback, and means for performing data tokenization and normalization. This enables responses that take user emotions into consideration, and allows for system improvement and increased user satisfaction through the use of feedback.

[0271] "Inquiry data" refers to data that includes information such as questions, requests, and feedback from users.

[0272] "Data cleansing" is the process of removing noise and unnecessary data from collected data to improve data quality.

[0273] "Tokenization" is the process of dividing text data into smaller units such as words and phrases.

[0274] A "machine learning model" is a collection of algorithms that learn patterns from input data and then use that knowledge to make predictions and classifications on new data.

[0275] An "emotion engine" is a technology that extracts and classifies users' emotions from text data.

[0276] "Preprocessing" refers to a series of processes for formatting collected data into a form that can be analyzed.

[0277] A "real-time inquiry" is an inquiry sent by a user in real time.

[0278] "Optimal response" refers to the most appropriate and effective information or answer to a user's inquiry.

[0279] "Feedback" refers to the evaluations and opinions from users regarding the services or answers provided.

[0280] "Tokenization and normalization" is the process of dividing text data into words and phrases and formatting them for analysis.

[0281] This invention relates to a system that collects and preprocesses past inquiry data to train a machine learning model, and then generates and provides immediate and accurate answers to real-time inquiries from users. Furthermore, it aims to further improve user satisfaction by incorporating an emotion engine that recognizes user emotions.

[0282] The system's main components are servers, terminals, and users. The specific functions of each component are explained below.

[0283] First, the server collects a wide range of query data from the database. This database stores many queries and their corresponding answers. The collected data is then preprocessed through data cleansing and tokenization. The Python pandas library is used for data cleansing to remove noise and unnecessary data. The NLTK or spaCy library is used for tokenization to divide the data into analyzable units.

[0284] Next, the server trains a machine learning model using the preprocessed data. Libraries such as TensorFlow and PyTorch are utilized for training, and the model is constructed with algorithms (e.g., BERT, GPT, etc.) that apply natural language processing techniques. As a result, a model is created that has the ability to learn past inquiry contents and corresponding answers and generate appropriate answers.

[0285] When a user inputs a question from a terminal, the data is sent to the server in real time through the terminal. The server analyzes the received inquiry and generates an optimal answer using the trained model. In the process of answer generation, the sentiment engine analyzes the content of the user's inquiry and classifies it as positive, negative, neutral, etc. This enables appropriate responses according to the user's sentiment. TextBlob and VADER libraries are used for the sentiment engine.

[0286] The generated answer is sent from the server to the terminal, and the terminal displays the answer to the user. Feedback on the answer provided by the user is also collected, and based on this, the system is improved. The feedback is used by the server as retraining data, improving the accuracy of the system and user satisfaction.

[0287] As a specific example, when a user inputs "I don't know how to log in for the first time" into the terminal, the terminal sends this to the server. The sentiment engine recognizes it as "confused", and the trained model answers "For the first time login, use the registered email address and enter the sent verification code. If you have any questions, please feel free to contact us at any time." This answer is displayed to the user through the terminal.

[0288] Examples of prompt texts are as follows:

[0289] "User's question:'I don't know how to log in for the first time'

[0290] Emotion analysis result: 'confused'

[0291] Please generate a list of possible answers.

[0292] As a result, the present invention includes specific processing steps ranging from collecting inquiry data to providing answers and further improving the system through feedback, making it possible to recognize user emotions and provide more appropriate support.

[0293] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0294] Step 1:

[0295] The server collects past query data from the database. The database stores user inquiries and their corresponding answers. The server extracts this data by executing SQL queries. For example, it might execute a query like SELECT inquiry, response FROM inquiries;

[0296] Input: Inquiry Database

[0297] Data processing: Execute SQL queries, extract data.

[0298] Output: Retrieved query data (e.g., a set of questions and corresponding answers)

[0299] Step 2:

[0300] The server preprocesses the collected query data. It performs data cleansing, using the Python pandas library to remove noise and missing data. Furthermore, it performs tokenization using the NLTK or spaCy libraries to make the text data parseable.

[0301] Input: Collected query data

[0302] Data processing: Data cleaning, tokenization

[0303] Output: Preprocessed data (tokenized text data)

[0304] Step 3:

[0305] The server uses the preprocessed data to train a machine learning model. Utilize TensorFlow or PyTorch libraries to build and train the model with natural language processing algorithms (e.g., BERT, GPT, etc.).

[0306] Input: Preprocessed data

[0307] Data operation: Training of machine learning model

[0308] Output: Trained model

[0309] Step 4:

[0310] When the user inputs a question from the terminal, the terminal sends the data to the server. HTTP requests are commonly used for this communication. For example, the user inputs "I don't know the method of the first login".

[0311] Input: User's question text

[0312] Data processing: Send to the server via HTTP request

[0313] Output: Inquiry sent to the server

[0314] Step 5:

[0315] The server analyzes the received inquiry, performs tokenization and normalization. Furthermore, using the trained machine learning model based on the preprocessed data, it generates the optimal answer.

[0316] Input: User inquiry text

[0317] Data processing: tokenization, normalization, and model-based response generation.

[0318] Output: Generated answer

[0319] Step 6:

[0320] The server analyzes the received user inquiry text using an emotion engine. The emotion engine extracts emotions such as positive, negative, and neutral from the text and adjusts the corresponding response based on the results. For example, sentiment analysis can be performed using the TextBlob library.

[0321] Input: User's inquiry text

[0322] Data processing: Sentiment analysis (positive, negative, neutral, etc.)

[0323] Output: Emotion analysis results

[0324] Step 7:

[0325] Based on the generated responses and sentiment analysis results, the server creates a response in the most appropriate tone and delivers it to the user via the device. For example, it uses a gentle tone for users who are "troubled" and offers a quick solution, including an apology, for users who are "dissatisfied."

[0326] Input: Generated response, sentiment analysis results

[0327] Data processing: Adjusting the tone of responses

[0328] Output: Adjusted answer

[0329] Step 8:

[0330] When a user enters feedback on a provided response, the device sends it to the server. A specific feedback format is used and sent to the server via an HTTP request.

[0331] Input: User feedback

[0332] Data processing: Sending to the server via HTTP request

[0333] Output: Feedback sent to the server

[0334] Step 9:

[0335] The server updates the machine learning model based on the collected feedback. This data is then incorporated back into the training set and reflected in the model as new data.

[0336] Input: Collected feedback

[0337] Data processing: Feedback integration, retraining of machine learning models

[0338] Output: Updated machine learning model

[0339] By following these steps, the system can provide quick and accurate answers to user inquiries and continuously improve based on feedback.

[0340] (Application Example 2)

[0341] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0342] Traditional customer support systems struggled to provide timely and accurate answers to user inquiries. Furthermore, they often failed to consider user emotions, leading to decreased customer satisfaction. In such cases, particularly on e-commerce sites, customer dissatisfaction became significant, negatively impacting the business. Additionally, the lack of adequate features for utilizing feedback on inquiries resulted in slow system improvements.

[0343] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for collecting inquiry data, means for pre-processing the collected inquiry data, means for training a machine learning model using the pre-processed data, means for receiving real-time inquiries from users, means for analyzing the received inquiries and generating the optimal answer using the trained model, means for providing the generated answer to the user, emotion recognition means for analyzing the user's emotions from the inquiry content, and means for adjusting the tone of the answer based on the analyzed emotions. This makes it possible to immediately provide an appropriate answer that takes into account not only the content of the inquiry but also the user's emotions, thereby improving customer satisfaction. Furthermore, by collecting feedback from users and updating the machine learning model based on it, continuous improvement of the system is also possible.

[0344] "Inquiry data" refers to information about questions and requests provided by users.

[0345] "Preprocessing" is the process of removing noise and unnecessary data from raw data to prepare it for easier analysis.

[0346] A "machine learning model" is an algorithm that learns specific patterns and relationships based on large amounts of data, and then uses that learning to make predictions and judgments about new data.

[0347] A "real-time inquiry" is an immediate question or request that a user enters and submits at that moment.

[0348] "Training" is the process of providing data to a machine learning model, allowing the model to learn patterns in that data.

[0349] "Analysis" is the process of finding meaning and patterns based on given data.

[0350] "Generation" is the process of producing appropriate responses or results based on input data.

[0351] "Emotion recognition" is the process of analyzing and determining a user's emotional state at a given time based on the content of their inquiry.

[0352] "Response tone" refers to the nuances and tone of expression in the generated response.

[0353] "Feedback" refers to evaluations and opinions provided by users regarding the system and its responses.

[0354] This invention relates to a system that provides immediate and accurate responses to user inquiries. This system is characterized by analyzing the content of the inquiry, recognizing the user's emotions, and providing responses in a tone that matches the user's emotions.

[0355] System Overview

[0356] This system consists of the following main components:

[0357] 1. Collection and preprocessing of query data:

[0358] The server collects user inquiry data accumulated in the past from the database and performs data cleansing to remove noise and unnecessary data. Next, it performs tokenization, which divides the collected text data into tokens (words and phrases), to prepare it for easier analysis.

[0359] 2. Model training:

[0360] The server trains a machine learning model using preprocessed data. For training, it selects an algorithm using natural language processing techniques (e.g., BERT, GPT, etc.) and trains the model by dividing the dataset. This allows the server to learn past queries and their corresponding answers, and generate appropriate responses to queries.

[0361] 3. Real-time query analysis:

[0362] When a user enters a question from their device, the data is sent to the server via the device. The server receives the submitted query, performs tokenization and normalization to transform it into a format that is easy to analyze. Next, it uses a trained model to analyze the query and generate the optimal answer.

[0363] 4. Introduction of the Emotion Engine:

[0364] The server analyzes the received user inquiry text using an emotion engine. The emotion engine extracts emotions from the user's text and classifies them into emotion categories such as positive, negative, and neutral. Based on this classification, it adjusts the appropriate response or reply.

[0365] 5. Providing an answer:

[0366] The server sends the generated response to the terminal, which then displays the response to the user. By taking the results of the sentiment engine into account, for example, a helpful tone is provided to a user who is "in distress," while an apology or a quick solution is offered to a user who is "dissatisfied."

[0367] 6. Feedback and Model Updates:

[0368] A feature will be added to the device that allows users to provide feedback on the answers they receive. This feedback will be sent to the server and used to improve the system. The server can use this data to retrain the machine learning model, improving the system's accuracy and user satisfaction. Furthermore, by incorporating the results of the emotion engine as training data, the system can learn which emotions are most effectively addressed in responses.

[0369] Specific example

[0370] Inquiry about how to log in as a new user

[0371] The user types "I don't know how to log in for the first time" from their device. The device sends this question to the server, which analyzes the received query. The sentiment engine recognizes this query as "confused." The trained model generates the best answer regarding "how to log in" and sends the response "For your first login, please use the email address you registered and enter the verification code sent to you. If you have any questions, please contact us anytime" from the server to the device. The device then displays this response to the user.

[0372] Inquiry about update details for existing users

[0373] The user types "What was updated last week?" into their device. The device sends this question to the server, which analyzes the received query. The sentiment engine categorizes it as "neutral." The trained model generates the best possible answer regarding "updates" and sends the response "Last week's update added XX as a new feature and fixed YY as a bug" from the server to the device. The device then displays this response to the user.

[0374] Example of a prompt

[0375] For example, if a new user asks, "I don't know how to log in for the first time," the following prompt message will be used:

[0376] I don't know how to log in for the first time.

[0377] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0378] Step 1:

[0379] The server collects past query data from the database. The input data to be collected consists of past user queries and the corresponding responses. The output is a list of the collected raw data. This data may contain noise and incomplete information.

[0380] Step 2:

[0381] The server preprocesses the collected query data. Preprocessing begins with data cleansing to remove noise and unnecessary data. Next, the text data is tokenized to prepare it for easier analysis. The input is raw data, and the output is cleansed, tokenized data.

[0382] Step 3:

[0383] The server trains a machine learning model using preprocessed data. It selects an algorithm using natural language processing techniques (e.g., BERT, GPT), splits the dataset, and trains the model. The input is the preprocessed data set, and the output is the trained machine learning model.

[0384] Step 4:

[0385] The user enters a query in real time from their terminal. The user's input is a text-based question, and the terminal sends that question to the server. The input is the user's real-time query, and the output is data sent to the server.

[0386] Step 5:

[0387] The server analyzes the received query. First, it performs tokenization and normalization to transform the query into a format that is easier to analyze. Next, it uses a trained model to analyze the query content and generate the optimal answer. The input is the query text received in real time, and the output is the generated answer.

[0388] Step 6:

[0389] The server analyzes the user's emotions from the query content. Using an emotion engine, it classifies emotions from the text into positive, negative, neutral, etc. The input is the query text, and the output is the analyzed emotion category.

[0390] Step 7:

[0391] The server adjusts the tone of its responses based on the analyzed emotions. For example, it uses a friendly tone for confused users and offers apologies or quick solutions for users with negative emotions. The input is the analyzed emotion category and the generated response, and the output is the response adjusted according to the emotion.

[0392] Step 8:

[0393] The server sends the adjusted response to the terminal, and the terminal displays that response to the user. The input is the adjusted response, and the output is what is displayed to the user.

[0394] Step 9:

[0395] The user enters feedback on the provided answers. The input is feedback text, and the device sends that feedback to the server.

[0396] Step 10:

[0397] The server updates the machine learning model based on the collected feedback. The feedback data is added to the training dataset, and retraining is performed. The input is the feedback data, and the output is the updated machine learning model.

[0398] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0399] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0400] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0401] [Second Embodiment]

[0402] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0403] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0404] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0405] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0406] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0407] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0408] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0409] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0410] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0411] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0412] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0413] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0414] This invention relates to a system that collects and preprocesses past inquiry data and trains a machine learning model based on it, thereby generating and providing immediate and accurate answers to real-time inquiries from users. The general program of the system and its processing are described below.

[0415] System Overview

[0416] Collection and preprocessing of query data

[0417] The server collects user inquiry data accumulated in the past from the database. Since the collected data is often incomplete as is, data cleansing is used to remove noise and unnecessary data and to prepare it in an appropriate format. Next, tokenization is performed to divide the text data into tokens (words and phrases), converting it into a form that can be easily processed by machines.

[0418] Model training

[0419] The server trains a machine learning model using preprocessed data. Specifically, it selects a model using natural language processing techniques (e.g., BERT, GPT), splits the data into training and validation datasets, and trains the model to learn the relationship between queries and corresponding answers. Through this process, the model acquires the ability to generate appropriate answers for various types of queries.

[0420] Real-time inquiry analysis

[0421] When a user enters a question from their device, the data is sent to the server via the device. The server receives the submitted query, performs tokenization and normalization to transform it into a format that is easy to analyze. Next, it uses a trained model to analyze the query and generate the optimal answer.

[0422] Providing a response

[0423] The server sends the generated response to the terminal, which then displays the response to the user. This allows the user to obtain an accurate answer immediately.

[0424] Specific example

[0425] Example 1: Inquiry about login methods for new users

[0426] The user types "I don't know how to log in for the first time" from their device. The device sends this question to the server, which parses the received query. The trained model generates the best answer regarding "how to log in," and the server sends the answer "For your first login, please use the email address you registered and enter the verification code sent to you" to the device. The device then displays this answer to the user.

[0427] Example 2: Inquiry about update details for existing users

[0428] The user types "What were the updates from last week?" into their device. The device sends this question to the server, which analyzes the received query. The trained model generates the best possible answer regarding "updates" and sends the response "Last week's update included the addition of XX as a new feature and YY as a bug fix" from the server to the device. The device then displays this response to the user.

[0429] Feedback and Model Updates

[0430] It is also possible to add a feature to the device that allows users to provide feedback on the answers they have provided. This feedback is sent to the server and used to improve the system. The server can use this data to retrain machine learning models and improve the system's accuracy and user satisfaction.

[0431] As described above, this system efficiently analyzes inquiry data and enables quick and accurate responses to inquiries from both new and existing users.

[0432] The following describes the processing flow.

[0433] Step 1:

[0434] The server collects past query data from the database. The database contains a wide variety of queries and their corresponding answers.

[0435] Step 2:

[0436] The server performs data cleansing on the collected query data. At this stage, noise (inappropriate data and errors) is removed, improving the quality of the data.

[0437] Step 3:

[0438] The server normalizes the data. Specifically, it converts uppercase English letters to lowercase and removes special characters to arrange the data into a consistent format.

[0439] Step 4:

[0440] The server performs tokenization, which divides the data into tokens (words or phrases). This converts the text data into a format that is easier to parse.

[0441] Step 5:

[0442] The server trains a machine learning model using preprocessed data. For training, it selects an algorithm that uses natural language processing techniques (e.g., BERT, GPT, etc.) and trains the model by dividing the dataset.

[0443] Step 6:

[0444] The server starts a real-time analysis system to process new queries, making it ready to receive user inquiries immediately.

[0445] Step 7:

[0446] Users will enter questions in a free-form format via their device. Specific questions such as "I don't know how to log in for the first time" are expected.

[0447] Step 8:

[0448] The terminal sends the questions entered by the user to the server as text data.

[0449] Step 9:

[0450] The server preprocesses the text data received from the terminal for analysis. Tokenization and normalization are performed again at this stage.

[0451] Step 10:

[0452] The server uses a machine learning model to generate the optimal response based on pre-processed query data. The model predicts the response based on historical data.

[0453] Step 11:

[0454] The server sends the generated response as text data to the terminal.

[0455] Step 12:

[0456] The device displays the received response in the user interface. For example, it might say, "For your first login, please use the email address you registered and enter the verification code that was sent to you."

[0457] Step 13:

[0458] Users can rate their satisfaction with the provided answers. Satisfaction feedback is entered on the device and used to improve the system.

[0459] Step 14:

[0460] The device sends user feedback to the server.

[0461] Step 15:

[0462] The server analyzes the received feedback and uses it to update its machine learning model. This improves the accuracy and quality of future inquiries.

[0463] The above outlines the specific processing steps from collecting inquiry data to providing responses and further improving the system based on feedback.

[0464] (Example 1)

[0465] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0466] Conventional inquiry handling systems often suffered from delays in responding to user inquiries and had limited performance in generating appropriate answers. Furthermore, updating models based on feedback was cumbersome, posing a risk of decreased accuracy. Therefore, there was a need for a system that could provide real-time, rapid, and accurate answers.

[0467] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0468] In this invention, the server includes means for collecting query data, means for preprocessing the collected query data, means for training a machine learning model using the preprocessed data, means for receiving real-time queries from users, means for analyzing the received queries and generating the optimal answer using the trained model, means for denoising and formatting the data, means for using natural language processing technology as the machine learning model, and means for providing the generated answer to the user. This makes it possible to provide fast and accurate answers in real time.

[0469] "Inquiry data" refers to data that includes information about questions and requests from users.

[0470] "Preprocessing" refers to the process of applying noise reduction and formatting to collected data to make it easier to analyze.

[0471] A "machine learning model" is an algorithm that learns from input data and performs predictions and classifications for specific tasks.

[0472] "Training" is the process by which a machine learning model learns using pre-processed data.

[0473] A "real-time inquiry" is a question or request that a user makes to a system instantly.

[0474] "Noise reduction" is the process of removing unwanted or inaccurate information from data.

[0475] "Formatting" refers to the process of aligning data to a specific format or structure.

[0476] "Natural language processing technology" is the technology that enables computers to understand and generate human language.

[0477] "Feedback" refers to the evaluations and comments that users make regarding the answers provided.

[0478] "Tokenization" is the process of dividing text data into units such as words and phrases.

[0479] "Normalization" is the process of converting data into a standard format in order to maintain data consistency.

[0480] This invention is a system that efficiently collects and analyzes inquiry data and provides quick and accurate responses to real-time inquiries from users. This system uses the following hardware and software.

[0481] hardware

[0482] Server: A central processing unit for data collection, preprocessing, model training, query analysis, response generation, and delivery. This server can utilize data center-level computers equipped with high-performance CPUs and GPUs.

[0483] Terminal: A device used by users to input inquiries and receive responses. This includes common computing devices such as personal computers, tablets, or smartphones.

[0484] software

[0485] Database system: Stores and manages query data. Specifically, it uses relational databases such as MySQL or PostgreSQL.

[0486] Data cleansing tools are software that removes noise and formats collected data. An example is OpenRefine.

[0487] Natural language processing libraries: These perform tokenization and normalization of text data. Specifically, libraries such as NLTK and spaCy are used.

[0488] Machine learning frameworks: These frameworks train machine learning models based on preprocessed data and analyze queries. Specific examples include TensorFlow, PyTorch, and Transformers libraries (BERT, GPT, etc.).

[0489] Web server: Receives and provides responses to queries in real time. Python frameworks such as FastAPI and Flask are used.

[0490] Specific example

[0491] When a user types "I don't know how to log in for the first time" from their computer, the device sends this inquiry to the server. The server tokenizes and normalizes the received inquiry data using natural language processing libraries (NLTK, spaCy). A trained machine learning model (e.g., BERT, GPT) is used to analyze the inquiry and generate the most appropriate response. In this case, the response generated is "For your first login, please use your registered email address and enter the verification code sent to you." The generated response is sent from the server to the device and displayed to the user.

[0492] Example of a prompt

[0493] For example, the following types of user inquiries can be handled in a similar manner:

[0494] "What should I do if I forget my password?"

[0495] "I want to know what's in the latest update."

[0496] This allows users to receive accurate answers immediately and solve problems efficiently. Furthermore, user feedback can be collected, and the server can use this data to retrain the model and improve the system's accuracy.

[0497] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0498] Step 1: Collecting inquiry data

[0499] The server collects user query data accumulated in the past from a database. This database is a relational database system (e.g., MySQL, PostgreSQL). The server executes SQL queries to extract the query data and converts it into a data frame format (e.g., Pandas DataFrame).

[0500] Input: Query data in the database

[0501] Output: Query data in data frame format

[0502] Step 2: Data preprocessing

[0503] The server performs data cleansing on the collected data. Specifically, it uses Pandas to remove noisy data (e.g., missing values, redundant data) and prepares the data in a clean state. Next, it uses a natural language processing library (e.g., NLTK, spaCy) to tokenize the text data and then format it.

[0504] Input: Query data in data frame format

[0505] Output: Tokenized and cleansed query data

[0506] Step 3: Training the machine learning model

[0507] The server trains a machine learning model using preprocessed data. Here, we implement a model using natural language processing techniques (e.g., BERT, GPT). The data is split into training and validation datasets, and the model is trained using query-and-appropriate response pairs. Frameworks such as TensorFlow and PyTorch are used.

[0508] Input: Tokenized query data and corresponding response data

[0509] Output: Trained machine learning model

[0510] Step 4: Receiving real-time inquiries

[0511] When a user enters a query from their device, that data is sent to the server via the device. Here, JavaScript is used for the frontend, and a lightweight web framework such as FastAPI is used for the backend.

[0512] Input: Inquiry text entered by the user

[0513] Output: Sending query data to the server

[0514] Step 5: Analyze the inquiry and generate the answer.

[0515] The server analyzes incoming queries in real time. First, it uses a natural language processing library to tokenize and normalize the queries, and then uses a trained machine learning model to generate the optimal response.

[0516] Input: Received query data

[0517] Output: Generated response data

[0518] Step 6: Provide your response

[0519] The server sends the generated response to the terminal, and the terminal displays that response to the user. HTTP or WebSocket is used as the communication protocol for this process.

[0520] Input: Server-generated response data

[0521] Output: Answer displayed to the user

[0522] Step 7: Gathering Feedback and Updating the Model

[0523] Users provide feedback on the provided answers. This feedback is sent to the server via the terminal. The server retrains the model based on the feedback data to improve the system's accuracy.

[0524] Input: User-entered feedback data

[0525] Output: Updated machine learning model

[0526] (Application Example 1)

[0527] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0528] Traditional customer support systems struggled to provide quick and accurate responses to real-time user inquiries. Furthermore, there was a lack of means to improve the quality and efficiency of customer service in physical stores. In particular, during peak hours, staff shortages led to customers having to wait, which was a significant problem.

[0529] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0530] In this invention, the server includes means for collecting inquiry data, means for preprocessing the collected inquiry data, means for training a machine learning model using the preprocessed data, means for receiving real-time inquiries from users, means for analyzing the received inquiries and generating the optimal answer using the trained model, means for providing the generated answer to the user, means for installing it on a robot placed in a physical store, and means for the robot to receive customer inquiries and provide answers. This enables quick and accurate customer service even in physical stores.

[0531] "Inquiry data" refers to data related to questions and requests from customers and users.

[0532] "Preprocessing" refers to a series of processes that convert raw data into a format suitable for analysis and training.

[0533] A "machine learning model" is an algorithm that learns from large amounts of data and performs predictions and classifications for specific tasks.

[0534] A "real-time inquiry" is an immediate question or request made by a user during an ongoing session.

[0535] "Analysis" is the process of interpreting data and information to derive meaning in accordance with a specific purpose.

[0536] "Answer generation" is the process of constructing an appropriate response to an inquiry.

[0537] "Providing" refers to the act of showing or sending the generated response to the user.

[0538] A "physical store" is a commercial facility that provides goods and services in a physical location.

[0539] A "robot" is a mechanical device that performs specific tasks autonomously or remotely.

[0540] "Feedback" refers to user opinions and evaluations regarding the system's operation and the responses provided.

[0541] "Tokenization" is the process of dividing text data into words or phrases.

[0542] "Normalization" is the process of arranging the format and content of data into a consistent and standard form.

[0543] This invention relates to a system that uses robots placed in physical stores to provide appropriate answers to customer inquiries in real time.

[0544] Collection and preprocessing of query data

[0545] The server collects past query data from the database. Since the collected data is often incomplete, data cleansing techniques are used to remove noise and unnecessary data, and to prepare it in an appropriate format. Then, tokenization is performed to divide the text data into words and phrases, and further normalization is carried out to convert it into a form that is easy to analyze.

[0546] Model training

[0547] The server trains a machine learning model using pre-processed data. This process utilizes natural language processing techniques, and models such as BERT and GPT are selected. The collected data is divided into a training dataset and a validation dataset, and the relationship between queries and corresponding answers is learned.

[0548] Real-time inquiry analysis

[0549] When a customer makes a request to a robot placed in a physical store, the data is sent by the robot to a server. The server tokenizes and normalizes the received data, and then uses a trained model to analyze the request and generate the best possible answer.

[0550] Providing a response

[0551] The server sends the generated response to the robot, which then provides the response to the customer. This allows the customer to receive the appropriate answer immediately.

[0552] Specific hardware and software

[0553] This invention primarily uses Python 3.x, the Transformers library (provided by Hugging Face), and PyTorch as its software. For hardware, the CPU and GPU installed in the customer service robot are used. The customer service robot is equipped with a touchscreen to allow customers to easily input questions.

[0554] Specific example

[0555] For example, if a customer types "What are your store's opening hours?" into the robot's touchscreen, the robot sends the question to the server. The server analyzes the question, generates an answer such as "Our store's opening hours are from 10 AM to 8 PM," and sends it to the robot. The robot then provides this answer to the customer.

[0556] Example of a prompt

[0557] "Please explain how to provide appropriate answers to customer inquiries regarding store hours in a physical store setting."

[0558] "How can we develop a system that can respond immediately in real time to customer inquiries about product inventory status?"

[0559] This invention enables prompt and accurate customer service in physical stores, improving the user experience and increasing the efficiency of store operations.

[0560] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0561] Step 1:

[0562] The server collects past query data from the database. The collected query data, in its raw form, may contain noise and unnecessary data. This data is then cleansed using data cleansing techniques to remove the unnecessary parts and format it appropriately. Through this process, the server obtains pre-processed query data.

[0563] Step 2:

[0564] The server performs tokenization and normalization on the preprocessed query data. Specifically, it uses a natural language processing library to split the text data into words and phrases and convert it into a consistent format. Through this process, the server obtains data that is in a form that is easy to parse.

[0565] Step 3:

[0566] The server trains a machine learning model using preprocessed data. It primarily uses the Transformers library (provided by Hugging Face) and PyTorch. In this step, the data is split into training and validation datasets, and the relationship between queries and corresponding answers is learned. As a result of this process, the server obtains a trained machine learning model.

[0567] Step 4:

[0568] The user enters their inquiry into a customer service robot in a physical store. The inquiry, entered using the robot's touchscreen, is sent by the robot to a server. In this step, the user's question is transmitted to the server via the robot.

[0569] Step 5:

[0570] The server then performs tokenization and normalization again on the query received from the robot. It then analyzes the query using a trained machine learning model and generates the optimal response. This step ensures the server generates a suitable answer, which it then sends to the robot.

[0571] Step 6:

[0572] The robot displays the answers received from the server and provides them to the user. Specifically, the answers are displayed on the robot's touchscreen, allowing the user to instantly obtain the appropriate information.

[0573] Step 7:

[0574] Users can provide feedback on the provided answers. This feedback is sent back to the server and used to improve the model in the future. By collecting and analyzing feedback, the server can improve the accuracy of the machine learning model.

[0575] The above outlines the processing steps for a system used for customer service in physical stores.

[0576] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0577] This invention relates to a system that generates and provides immediate and accurate answers to real-time user inquiries by collecting and preprocessing past inquiry data and training a machine learning model based on that data. Furthermore, it aims to further improve user satisfaction by incorporating an emotion engine that recognizes user emotions.

[0578] System Overview

[0579] Collection and preprocessing of query data

[0580] The server collects user inquiry data accumulated in the past from a database. The database stores a wide variety of inquiries and their corresponding answers. Since much of the collected data is incomplete in its raw form, data cleansing is used to remove noise and unnecessary data and improve data quality. Next, tokenization is performed to divide the text data into tokens (words and phrases) to prepare it for easier analysis.

[0581] Model training

[0582] The server trains a machine learning model using pre-processed data. For training, it selects an algorithm using natural language processing techniques (e.g., BERT, GPT, etc.) and trains the model on a divided dataset. This allows the server to learn from past queries and corresponding answers, and generate appropriate responses to queries.

[0583] Real-time inquiry analysis

[0584] When a user enters a question from their device, the data is sent to the server via the device. The server receives the submitted query, performs tokenization and normalization to transform it into a format that is easy to analyze. Next, it uses a trained model to analyze the query and generate the optimal answer.

[0585] Introducing an emotional engine

[0586] The server analyzes the received user inquiry text using an emotion engine. The emotion engine extracts emotions from the user's text and classifies them into emotion categories such as positive, negative, and neutral. Based on this classification, it adjusts the appropriate response or reply.

[0587] Providing a response

[0588] The server sends the generated response to the terminal, which then displays the response to the user. By taking the results of the sentiment engine into account, for example, a helpful tone is provided to a user who is "in distress," while an apology or a quick solution is offered to a user who is "dissatisfied."

[0589] Specific example

[0590] Example 1: Inquiry about login methods for new users

[0591] The user types "I don't know how to log in for the first time" from their device. The device sends this question to the server, which analyzes the received query. The sentiment engine recognizes this query as "confused." The trained model generates the best answer regarding "how to log in" and sends the response "For your first login, please use the email address you registered and enter the verification code sent to you. If you have any questions, please contact us anytime" from the server to the device. The device then displays this response to the user.

[0592] Example 2: Inquiry about update details for existing users

[0593] The user types "What was updated last week?" into their device. The device sends this question to the server, which analyzes the received query. The sentiment engine categorizes it as "neutral." The trained model generates the best possible answer regarding "updates" and sends the response "Last week's update added XX as a new feature and fixed YY as a bug" from the server to the device. The device then displays this response to the user.

[0594] Feedback and Model Updates

[0595] It is also possible to add a feature to the device that allows users to provide feedback on the answers they have received. This feedback is sent to the server and used to improve the system. The server can use this data to retrain the machine learning model, improving the system's accuracy and user satisfaction. Furthermore, by incorporating the results of the emotion engine as training data, the system can learn which emotions are most effectively addressed in its responses.

[0596] As described above, this system includes specific processing steps ranging from collecting inquiry data to providing answers and further improving the system based on feedback, enabling it to recognize user emotions and provide more appropriate support.

[0597] The following describes the processing flow.

[0598] Step 1:

[0599] The server collects past query data from the database. The database contains a wide variety of queries and their corresponding answers.

[0600] Step 2:

[0601] The server performs data cleansing on the collected query data. At this stage, noise (inappropriate data and errors) is removed, improving the quality of the data.

[0602] Step 3:

[0603] The server normalizes the data. Specifically, it converts uppercase English letters to lowercase and removes special characters to arrange the data into a consistent format.

[0604] Step 4:

[0605] The server performs tokenization, which divides the data into tokens (words or phrases). This converts the text data into a format that is easier to parse.

[0606] Step 5:

[0607] The server trains a machine learning model using preprocessed data. For training, it selects an algorithm that uses natural language processing techniques (e.g., BERT, GPT, etc.) and trains the model by dividing the dataset.

[0608] Step 6:

[0609] The server starts a real-time analysis system to process new queries, making it ready to receive user inquiries immediately.

[0610] Step 7:

[0611] Users will enter questions in a free-form format via their device. Specific questions such as "I don't know how to log in for the first time" are expected.

[0612] Step 8:

[0613] The terminal sends the questions entered by the user to the server as text data.

[0614] Step 9:

[0615] The server preprocesses the text data received from the terminal for analysis. Tokenization and normalization are performed again at this stage.

[0616] Step 10:

[0617] The server uses a machine learning model to generate the optimal response based on pre-processed query data. The model predicts the response based on historical data.

[0618] Step 11:

[0619] The server analyzes the received text data using an emotion engine. The emotion engine extracts emotions from the user's text and classifies them into emotion categories such as positive, negative, and neutral.

[0620] Step 12:

[0621] The server adjusts its responses based on the results of the emotion engine's analysis. For example, if the user is "confused," it will generate a response in a kind and caring tone.

[0622] Step 13:

[0623] The server sends the generated response to the terminal.

[0624] Step 14:

[0625] The device displays the received response in the user interface. For example, it might say, "For your first login, please use the email address you registered and enter the verification code that was sent to you."

[0626] Step 15:

[0627] Users can rate their satisfaction with the provided answers. Satisfaction feedback is entered on the device and used to improve the system.

[0628] Step 16:

[0629] The device sends user feedback to the server.

[0630] Step 17:

[0631] The server analyzes the received feedback and uses it to update its machine learning model. This improves the accuracy and quality of future inquiries.

[0632] Step 18:

[0633] The server also incorporates the results from the emotion engine as training data. This allows the system to continuously learn which emotions are most effectively addressed.

[0634] The above outlines the specific processing steps for an inquiry handling system that incorporates an emotion engine.

[0635] (Example 2)

[0636] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0637] Conventional inquiry handling systems have faced challenges in responding to user inquiries quickly and accurately, and insufficiently considering user emotions. Furthermore, they have been unable to effectively utilize feedback on the answers provided, limiting improvements in system performance and user satisfaction.

[0638] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0639] In this invention, the server includes means for collecting query data from a wide range of sources, means for preprocessing the collected query data through data cleansing and tokenization, and means for training a machine learning model using the preprocessed data. This enables the server to provide quick and accurate answers to user inquiries. The server further includes means for adjusting answers using an emotion engine that analyzes user emotions, means for providing the adjusted answers to the user, means for collecting user feedback, means for updating the machine learning model based on the collected feedback, and means for performing data tokenization and normalization. This enables responses that take user emotions into consideration, and allows for system improvement and increased user satisfaction through the use of feedback.

[0640] "Inquiry data" refers to data that includes information such as questions, requests, and feedback from users.

[0641] "Data cleansing" is the process of removing noise and unnecessary data from collected data to improve data quality.

[0642] "Tokenization" is the process of dividing text data into smaller units such as words and phrases.

[0643] A "machine learning model" is a collection of algorithms that learn patterns from input data and then use that knowledge to make predictions and classifications on new data.

[0644] An "emotion engine" is a technology that extracts and classifies users' emotions from text data.

[0645] "Preprocessing" refers to a series of processes for formatting collected data into a form that can be analyzed.

[0646] A "real-time inquiry" is an inquiry sent by a user in real time.

[0647] "Optimal response" refers to the most appropriate and effective information or answer to a user's inquiry.

[0648] "Feedback" refers to the evaluations and opinions from users regarding the services or answers provided.

[0649] "Tokenization and normalization" is the process of dividing text data into words and phrases and formatting them for analysis.

[0650] This invention relates to a system that collects and preprocesses past inquiry data to train a machine learning model, and then generates and provides immediate and accurate answers to real-time inquiries from users. Furthermore, it aims to further improve user satisfaction by incorporating an emotion engine that recognizes user emotions.

[0651] The system's main components are servers, terminals, and users. The specific functions of each component are explained below.

[0652] First, the server collects a wide range of query data from the database. This database stores many queries and their corresponding answers. The collected data is then preprocessed through data cleansing and tokenization. The Python pandas library is used for data cleansing to remove noise and unnecessary data. The NLTK or spaCy library is used for tokenization to divide the data into analyzable units.

[0653] Next, the server trains a machine learning model using the preprocessed data. Libraries such as TensorFlow and PyTorch are used for training, and the model is built using algorithms that apply natural language processing techniques (e.g., BERT, GPT). This creates a model that can learn from past inquiries and corresponding answers and generate appropriate responses.

[0654] When a user enters a question from their device, the data is transmitted to the server in real time. The server analyzes the received inquiry and generates the optimal answer using a trained model. During this answer generation process, the sentiment engine analyzes the user's inquiry and classifies it as positive, negative, neutral, etc. This enables appropriate responses based on the user's emotions. The sentiment engine uses libraries such as TextBlob and VADER.

[0655] The generated responses are sent from the server to the terminal, which then displays the responses to the user. Feedback on the responses provided by the user is also collected and used to improve the system. The server uses this feedback as retraining data, improving the system's accuracy and user satisfaction.

[0656] For example, if a user types "I don't know how to log in for the first time" into their device, the device sends this to the server. The emotion engine recognizes this as "confusion," and the trained model responds, "For your first login, please use the email address you registered and enter the verification code sent to you. If you have any questions, please contact us anytime." This response is then displayed to the user through their device.

[0657] Examples of prompt statements include:

[0658] User question: 'I don't know how to log in for the first time.'

[0659] Emotion analysis result: 'confused'

[0660] Please generate a list of possible answers.

[0661] As a result, the present invention includes specific processing steps ranging from collecting inquiry data to providing answers and further improving the system through feedback, making it possible to recognize user emotions and provide more appropriate support.

[0662] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0663] Step 1:

[0664] The server collects past query data from the database. The database stores user inquiries and their corresponding answers. The server extracts this data by executing SQL queries. For example, it might execute a query like SELECT inquiry, response FROM inquiries;

[0665] Input: Inquiry Database

[0666] Data processing: Execute SQL queries, extract data.

[0667] Output: Retrieved query data (e.g., a set of questions and corresponding answers)

[0668] Step 2:

[0669] The server preprocesses the collected query data. It performs data cleansing, using the Python pandas library to remove noise and missing data. Furthermore, it performs tokenization using the NLTK or spaCy libraries to make the text data parseable.

[0670] Input: Collected query data

[0671] Data processing: Data cleansing, tokenization

[0672] Output: Preprocessed data (tokenized text data)

[0673] Step 3:

[0674] The server trains machine learning models using preprocessed data. It utilizes libraries such as TensorFlow and PyTorch to build and train models using natural language processing algorithms (e.g., BERT, GPT).

[0675] Input: Preprocessed data

[0676] Data processing: Training machine learning models

[0677] Output: Trained model

[0678] Step 4:

[0679] When a user enters a question from their device, the device sends that data to the server. HTTP requests are commonly used for this communication. For example, a user might enter "I don't know how to log in for the first time."

[0680] Input: User's question text

[0681] Data processing: Sending to the server via HTTP request

[0682] Output: Query sent to the server

[0683] Step 5:

[0684] The server analyzes the received query, performs tokenization and normalization, and then uses a pre-trained machine learning model based on the pre-processed data to generate the best possible response.

[0685] Input: User inquiry text

[0686] Data processing: tokenization, normalization, and model-based response generation.

[0687] Output: Generated answer

[0688] Step 6:

[0689] The server analyzes the received user inquiry text using an emotion engine. The emotion engine extracts emotions such as positive, negative, and neutral from the text and adjusts the corresponding response based on the results. For example, sentiment analysis can be performed using the TextBlob library.

[0690] Input: User's inquiry text

[0691] Data processing: Sentiment analysis (positive, negative, neutral, etc.)

[0692] Output: Emotion analysis results

[0693] Step 7:

[0694] Based on the generated responses and sentiment analysis results, the server creates a response in the most appropriate tone and delivers it to the user via the device. For example, it uses a gentle tone for users who are "troubled" and offers a quick solution, including an apology, for users who are "dissatisfied."

[0695] Input: Generated response, sentiment analysis results

[0696] Data processing: Adjusting the tone of responses

[0697] Output: Adjusted answer

[0698] Step 8:

[0699] When a user enters feedback on a provided response, the device sends it to the server. A specific feedback format is used and sent to the server via an HTTP request.

[0700] Input: User feedback

[0701] Data processing: Sending to the server via HTTP request

[0702] Output: Feedback sent to the server

[0703] Step 9:

[0704] The server updates the machine learning model based on the collected feedback. This data is then incorporated back into the training set and reflected in the model as new data.

[0705] Input: Collected feedback

[0706] Data processing: Feedback integration, retraining of machine learning models

[0707] Output: Updated machine learning model

[0708] By following these steps, the system can provide quick and accurate answers to user inquiries and continuously improve based on feedback.

[0709] (Application Example 2)

[0710] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0711] Traditional customer support systems struggled to provide timely and accurate answers to user inquiries. Furthermore, they often failed to consider user emotions, leading to decreased customer satisfaction. In such cases, particularly on e-commerce sites, customer dissatisfaction became significant, negatively impacting the business. Additionally, the lack of adequate features for utilizing feedback on inquiries resulted in slow system improvements.

[0712] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for collecting inquiry data, means for pre-processing the collected inquiry data, means for training a machine learning model using the pre-processed data, means for receiving real-time inquiries from users, means for analyzing the received inquiries and generating the optimal answer using the trained model, means for providing the generated answer to the user, emotion recognition means for analyzing the user's emotions from the inquiry content, and means for adjusting the tone of the answer based on the analyzed emotions. This makes it possible to immediately provide an appropriate answer that takes into account not only the content of the inquiry but also the user's emotions, thereby improving customer satisfaction. Furthermore, by collecting feedback from users and updating the machine learning model based on it, continuous improvement of the system is also possible.

[0713] "Inquiry data" refers to information about questions and requests provided by users.

[0714] "Preprocessing" is the process of removing noise and unnecessary data from raw data to prepare it for easier analysis.

[0715] A "machine learning model" is an algorithm that learns specific patterns and relationships based on large amounts of data, and then uses that learning to make predictions and judgments about new data.

[0716] A "real-time inquiry" is an immediate question or request that a user enters and submits at that moment.

[0717] "Training" is the process of providing data to a machine learning model, allowing the model to learn patterns in that data.

[0718] "Analysis" is the process of finding meaning and patterns based on given data.

[0719] "Generation" is the process of producing appropriate responses or results based on input data.

[0720] "Emotion recognition" is the process of analyzing and determining a user's emotional state at a given time based on the content of their inquiry.

[0721] "Response tone" refers to the nuances and tone of expression in the generated response.

[0722] "Feedback" refers to evaluations and opinions provided by users regarding the system and its responses.

[0723] This invention relates to a system that provides immediate and accurate responses to user inquiries. This system is characterized by analyzing the content of the inquiry, recognizing the user's emotions, and providing responses in a tone that matches the user's emotions.

[0724] System Overview

[0725] This system consists of the following main components:

[0726] 1. Collection and preprocessing of query data:

[0727] The server collects user inquiry data accumulated in the past from the database and performs data cleansing to remove noise and unnecessary data. Next, it performs tokenization, which divides the collected text data into tokens (words and phrases), to prepare it for easier analysis.

[0728] 2. Model training:

[0729] The server trains a machine learning model using preprocessed data. For training, it selects an algorithm using natural language processing techniques (e.g., BERT, GPT, etc.) and trains the model by dividing the dataset. This allows the server to learn past queries and their corresponding answers, and generate appropriate responses to queries.

[0730] 3. Real-time query analysis:

[0731] When a user enters a question from their device, the data is sent to the server via the device. The server receives the submitted query, performs tokenization and normalization to transform it into a format that is easy to analyze. Next, it uses a trained model to analyze the query and generate the optimal answer.

[0732] 4. Introduction of the Emotion Engine:

[0733] The server analyzes the received user inquiry text using an emotion engine. The emotion engine extracts emotions from the user's text and classifies them into emotion categories such as positive, negative, and neutral. Based on this classification, it adjusts the appropriate response or reply.

[0734] 5. Providing an answer:

[0735] The server sends the generated response to the terminal, which then displays the response to the user. By taking the results of the sentiment engine into account, for example, a helpful tone is provided to a user who is "in distress," while an apology or a quick solution is offered to a user who is "dissatisfied."

[0736] 6. Feedback and Model Updates:

[0737] A feature will be added to the device that allows users to provide feedback on the answers they receive. This feedback will be sent to the server and used to improve the system. The server can use this data to retrain the machine learning model, improving the system's accuracy and user satisfaction. Furthermore, by incorporating the results of the emotion engine as training data, the system can learn which emotions are most effectively addressed in responses.

[0738] Specific example

[0739] Inquiry about how to log in as a new user

[0740] The user types "I don't know how to log in for the first time" from their device. The device sends this question to the server, which analyzes the received query. The sentiment engine recognizes this query as "confused." The trained model generates the best answer regarding "how to log in" and sends the response "For your first login, please use the email address you registered and enter the verification code sent to you. If you have any questions, please contact us anytime" from the server to the device. The device then displays this response to the user.

[0741] Inquiry about update details for existing users

[0742] The user types "What was updated last week?" into their device. The device sends this question to the server, which analyzes the received query. The sentiment engine categorizes it as "neutral." The trained model generates the best possible answer regarding "updates" and sends the response "Last week's update added XX as a new feature and fixed YY as a bug" from the server to the device. The device then displays this response to the user.

[0743] Example of a prompt

[0744] For example, if a new user asks, "I don't know how to log in for the first time," the following prompt message will be used:

[0745] I don't know how to log in for the first time.

[0746] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0747] Step 1:

[0748] The server collects past query data from the database. The input data to be collected consists of past user queries and the corresponding responses. The output is a list of the collected raw data. This data may contain noise and incomplete information.

[0749] Step 2:

[0750] The server preprocesses the collected query data. Preprocessing begins with data cleansing to remove noise and unnecessary data. Next, the text data is tokenized to prepare it for easier analysis. The input is raw data, and the output is cleansed, tokenized data.

[0751] Step 3:

[0752] The server trains a machine learning model using preprocessed data. It selects an algorithm using natural language processing techniques (e.g., BERT, GPT), splits the dataset, and trains the model. The input is the preprocessed data set, and the output is the trained machine learning model.

[0753] Step 4:

[0754] The user enters a query in real time from their terminal. The user's input is a text-based question, and the terminal sends that question to the server. The input is the user's real-time query, and the output is data sent to the server.

[0755] Step 5:

[0756] The server analyzes the received query. First, it performs tokenization and normalization to transform the query into a format that is easier to analyze. Next, it uses a trained model to analyze the query content and generate the optimal answer. The input is the query text received in real time, and the output is the generated answer.

[0757] Step 6:

[0758] The server analyzes the user's emotions from the query content. Using an emotion engine, it classifies emotions from the text into positive, negative, neutral, etc. The input is the query text, and the output is the analyzed emotion category.

[0759] Step 7:

[0760] The server adjusts the tone of its responses based on the analyzed emotions. For example, it uses a friendly tone for confused users and offers apologies or quick solutions for users with negative emotions. The input is the analyzed emotion category and the generated response, and the output is the response adjusted according to the emotion.

[0761] Step 8:

[0762] The server sends the adjusted response to the terminal, and the terminal displays that response to the user. The input is the adjusted response, and the output is what is displayed to the user.

[0763] Step 9:

[0764] The user enters feedback on the provided answers. The input is feedback text, and the device sends that feedback to the server.

[0765] Step 10:

[0766] The server updates the machine learning model based on the collected feedback. The feedback data is added to the training dataset, and retraining is performed. The input is the feedback data, and the output is the updated machine learning model.

[0767] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0768] The data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0769] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0770] [Third Embodiment]

[0771] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0772] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0773] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0774] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0775] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0776] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0777] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0778] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0779] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0780] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0781] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0782] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0783] This invention relates to a system that collects and preprocesses past inquiry data and trains a machine learning model based on it, thereby generating and providing immediate and accurate answers to real-time inquiries from users. The general program of the system and its processing are described below.

[0784] System Overview

[0785] Collection and preprocessing of query data

[0786] The server collects user inquiry data accumulated in the past from the database. Since the collected data is often incomplete as is, data cleansing is used to remove noise and unnecessary data and to prepare it in an appropriate format. Next, tokenization is performed to divide the text data into tokens (words and phrases), converting it into a form that can be easily processed by machines.

[0787] Model training

[0788] The server trains a machine learning model using preprocessed data. Specifically, it selects a model using natural language processing techniques (e.g., BERT, GPT), splits the data into training and validation datasets, and trains the model to learn the relationship between queries and corresponding answers. Through this process, the model acquires the ability to generate appropriate answers for various types of queries.

[0789] Real-time inquiry analysis

[0790] When a user enters a question from their device, the data is sent to the server via the device. The server receives the submitted query, performs tokenization and normalization to transform it into a format that is easy to analyze. Next, it uses a trained model to analyze the query and generate the optimal answer.

[0791] Providing a response

[0792] The server sends the generated response to the terminal, which then displays the response to the user. This allows the user to obtain an accurate answer immediately.

[0793] Specific example

[0794] Example 1: Inquiry about login methods for new users

[0795] The user types "I don't know how to log in for the first time" from their device. The device sends this question to the server, which parses the received query. The trained model generates the best answer regarding "how to log in," and the server sends the answer "For your first login, please use the email address you registered and enter the verification code sent to you" to the device. The device then displays this answer to the user.

[0796] Example 2: Inquiry about update details for existing users

[0797] The user types "What were the updates from last week?" into their device. The device sends this question to the server, which analyzes the received query. The trained model generates the best possible answer regarding "updates" and sends the response "Last week's update included the addition of XX as a new feature and YY as a bug fix" from the server to the device. The device then displays this response to the user.

[0798] Feedback and Model Updates

[0799] It is also possible to add a feature to the device that allows users to provide feedback on the answers they have provided. This feedback is sent to the server and used to improve the system. The server can use this data to retrain machine learning models and improve the system's accuracy and user satisfaction.

[0800] As described above, this system efficiently analyzes inquiry data and enables quick and accurate responses to inquiries from both new and existing users.

[0801] The following describes the processing flow.

[0802] Step 1:

[0803] The server collects past query data from the database. The database contains a wide variety of queries and their corresponding answers.

[0804] Step 2:

[0805] The server performs data cleansing on the collected query data. At this stage, noise (inappropriate data and errors) is removed, improving the quality of the data.

[0806] Step 3:

[0807] The server normalizes the data. Specifically, it converts uppercase English letters to lowercase and removes special characters to arrange the data into a consistent format.

[0808] Step 4:

[0809] The server performs tokenization, which divides the data into tokens (words or phrases). This converts the text data into a format that is easier to parse.

[0810] Step 5:

[0811] The server trains a machine learning model using preprocessed data. For training, it selects an algorithm that uses natural language processing techniques (e.g., BERT, GPT, etc.) and trains the model by dividing the dataset.

[0812] Step 6:

[0813] The server starts a real-time analysis system to process new queries, making it ready to receive user inquiries immediately.

[0814] Step 7:

[0815] Users will enter questions in a free-form format via their device. Specific questions such as "I don't know how to log in for the first time" are expected.

[0816] Step 8:

[0817] The terminal sends the questions entered by the user to the server as text data.

[0818] Step 9:

[0819] The server preprocesses the text data received from the terminal for analysis. Tokenization and normalization are performed again at this stage.

[0820] Step 10:

[0821] The server uses a machine learning model to generate the optimal response based on pre-processed query data. The model predicts the response based on historical data.

[0822] Step 11:

[0823] The server sends the generated response as text data to the terminal.

[0824] Step 12:

[0825] The device displays the received response in the user interface. For example, it might say, "For your first login, please use the email address you registered and enter the verification code that was sent to you."

[0826] Step 13:

[0827] Users can rate their satisfaction with the provided answers. Satisfaction feedback is entered on the device and used to improve the system.

[0828] Step 14:

[0829] The device sends user feedback to the server.

[0830] Step 15:

[0831] The server analyzes the received feedback and uses it to update its machine learning model. This improves the accuracy and quality of future inquiries.

[0832] The above outlines the specific processing steps from collecting inquiry data to providing responses and further improving the system based on feedback.

[0833] (Example 1)

[0834] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0835] Conventional inquiry handling systems often suffered from delays in responding to user inquiries and had limited performance in generating appropriate answers. Furthermore, updating models based on feedback was cumbersome, posing a risk of decreased accuracy. Therefore, there was a need for a system that could provide real-time, rapid, and accurate answers.

[0836] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0837] In this invention, the server includes means for collecting query data, means for preprocessing the collected query data, means for training a machine learning model using the preprocessed data, means for receiving real-time queries from users, means for analyzing the received queries and generating the optimal answer using the trained model, means for denoising and formatting the data, means for using natural language processing technology as the machine learning model, and means for providing the generated answer to the user. This makes it possible to provide fast and accurate answers in real time.

[0838] "Inquiry data" refers to data that includes information about questions and requests from users.

[0839] "Preprocessing" refers to the process of applying noise reduction and formatting to collected data to make it easier to analyze.

[0840] A "machine learning model" is an algorithm that learns from input data and performs predictions and classifications for specific tasks.

[0841] "Training" is the process by which a machine learning model learns using pre-processed data.

[0842] A "real-time inquiry" is a question or request that a user makes to a system instantly.

[0843] "Noise reduction" is the process of removing unwanted or inaccurate information from data.

[0844] "Formatting" refers to the process of aligning data to a specific format or structure.

[0845] "Natural language processing technology" is the technology that enables computers to understand and generate human language.

[0846] "Feedback" refers to the evaluations and comments that users make regarding the answers provided.

[0847] "Tokenization" is the process of dividing text data into units such as words and phrases.

[0848] "Normalization" is the process of converting data into a standard format in order to maintain data consistency.

[0849] This invention is a system that efficiently collects and analyzes inquiry data and provides quick and accurate responses to real-time inquiries from users. This system uses the following hardware and software.

[0850] hardware

[0851] Server: A central processing unit for data collection, preprocessing, model training, query analysis, response generation, and delivery. This server can utilize data center-level computers equipped with high-performance CPUs and GPUs.

[0852] Terminal: A device used by users to input inquiries and receive responses. This includes common computing devices such as personal computers, tablets, or smartphones.

[0853] software

[0854] Database system: Stores and manages query data. Specifically, it uses relational databases such as MySQL or PostgreSQL.

[0855] Data cleansing tools are software that removes noise and formats collected data. An example is OpenRefine.

[0856] Natural language processing libraries: These perform tokenization and normalization of text data. Specifically, libraries such as NLTK and spaCy are used.

[0857] Machine learning frameworks: These frameworks train machine learning models based on preprocessed data and analyze queries. Specific examples include TensorFlow, PyTorch, and Transformers libraries (BERT, GPT, etc.).

[0858] Web server: Receives and provides responses to queries in real time. Python frameworks such as FastAPI and Flask are used.

[0859] Specific example

[0860] When a user types "I don't know how to log in for the first time" from their computer, the device sends this inquiry to the server. The server tokenizes and normalizes the received inquiry data using natural language processing libraries (NLTK, spaCy). A trained machine learning model (e.g., BERT, GPT) is used to analyze the inquiry and generate the most appropriate response. In this case, the response generated is "For your first login, please use your registered email address and enter the verification code sent to you." The generated response is sent from the server to the device and displayed to the user.

[0861] Example of a prompt

[0862] For example, the following types of user inquiries can be handled in a similar manner:

[0863] "What should I do if I forget my password?"

[0864] "I want to know what's in the latest update."

[0865] This allows users to receive accurate answers immediately and solve problems efficiently. Furthermore, user feedback can be collected, and the server can use this data to retrain the model and improve the system's accuracy.

[0866] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0867] Step 1: Collecting inquiry data

[0868] The server collects user query data accumulated in the past from a database. This database is a relational database system (e.g., MySQL, PostgreSQL). The server executes SQL queries to extract the query data and converts it into a data frame format (e.g., Pandas DataFrame).

[0869] Input: Query data in the database

[0870] Output: Query data in data frame format

[0871] Step 2: Data preprocessing

[0872] The server performs data cleansing on the collected data. Specifically, it uses Pandas to remove noisy data (e.g., missing values, redundant data) and prepares the data in a clean state. Next, it uses a natural language processing library (e.g., NLTK, spaCy) to tokenize the text data and then format it.

[0873] Input: Query data in data frame format

[0874] Output: Tokenized and cleansed query data

[0875] Step 3: Training the machine learning model

[0876] The server trains a machine learning model using preprocessed data. Here, we implement a model using natural language processing techniques (e.g., BERT, GPT). The data is split into training and validation datasets, and the model is trained using query-and-appropriate response pairs. Frameworks such as TensorFlow and PyTorch are used.

[0877] Input: Tokenized query data and corresponding response data

[0878] Output: Trained machine learning model

[0879] Step 4: Receiving real-time inquiries

[0880] When a user enters a query from their device, that data is sent to the server via the device. Here, JavaScript is used for the frontend, and a lightweight web framework such as FastAPI is used for the backend.

[0881] Input: Inquiry text entered by the user

[0882] Output: Sending query data to the server

[0883] Step 5: Analyze the inquiry and generate the answer.

[0884] The server analyzes incoming queries in real time. First, it uses a natural language processing library to tokenize and normalize the queries, and then uses a trained machine learning model to generate the optimal response.

[0885] Input: Received query data

[0886] Output: Generated response data

[0887] Step 6: Provide your response

[0888] The server sends the generated response to the terminal, and the terminal displays that response to the user. HTTP or WebSocket is used as the communication protocol for this process.

[0889] Input: Server-generated response data

[0890] Output: Answer displayed to the user

[0891] Step 7: Gathering Feedback and Updating the Model

[0892] Users provide feedback on the provided answers. This feedback is sent to the server via the terminal. The server retrains the model based on the feedback data to improve the system's accuracy.

[0893] Input: User-entered feedback data

[0894] Output: Updated machine learning model

[0895] (Application Example 1)

[0896] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0897] Traditional customer support systems struggled to provide quick and accurate responses to real-time user inquiries. Furthermore, there was a lack of means to improve the quality and efficiency of customer service in physical stores. In particular, during peak hours, staff shortages led to customers having to wait, which was a significant problem.

[0898] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0899] In this invention, the server includes means for collecting inquiry data, means for preprocessing the collected inquiry data, means for training a machine learning model using the preprocessed data, means for receiving real-time inquiries from users, means for analyzing the received inquiries and generating the optimal answer using the trained model, means for providing the generated answer to the user, means for installing it on a robot placed in a physical store, and means for the robot to receive customer inquiries and provide answers. This enables quick and accurate customer service even in physical stores.

[0900] "Inquiry data" refers to data related to questions and requests from customers and users.

[0901] "Preprocessing" refers to a series of processes that convert raw data into a format suitable for analysis and training.

[0902] A "machine learning model" is an algorithm that learns from large amounts of data and performs predictions and classifications for specific tasks.

[0903] A "real-time inquiry" is an immediate question or request made by a user during an ongoing session.

[0904] "Analysis" is the process of interpreting data and information to derive meaning in accordance with a specific purpose.

[0905] "Answer generation" is the process of constructing an appropriate response to an inquiry.

[0906] "Providing" refers to the act of showing or sending the generated response to the user.

[0907] A "physical store" is a commercial facility that provides goods and services in a physical location.

[0908] A "robot" is a mechanical device that performs specific tasks autonomously or remotely.

[0909] "Feedback" refers to user opinions and evaluations regarding the system's operation and the responses provided.

[0910] "Tokenization" is the process of dividing text data into words or phrases.

[0911] "Normalization" is the process of arranging the format and content of data into a consistent and standard form.

[0912] This invention relates to a system that uses robots placed in physical stores to provide appropriate answers to customer inquiries in real time.

[0913] Collection and preprocessing of query data

[0914] The server collects past query data from the database. Since the collected data is often incomplete, data cleansing techniques are used to remove noise and unnecessary data, and to prepare it in an appropriate format. Then, tokenization is performed to divide the text data into words and phrases, and further normalization is carried out to convert it into a form that is easy to analyze.

[0915] Model training

[0916] The server trains a machine learning model using pre-processed data. This process utilizes natural language processing techniques, and models such as BERT and GPT are selected. The collected data is divided into a training dataset and a validation dataset, and the relationship between queries and corresponding answers is learned.

[0917] Real-time inquiry analysis

[0918] When a customer makes a request to a robot placed in a physical store, the data is sent by the robot to a server. The server tokenizes and normalizes the received data, and then uses a trained model to analyze the request and generate the best possible answer.

[0919] Providing a response

[0920] The server sends the generated response to the robot, which then provides the response to the customer. This allows the customer to receive the appropriate answer immediately.

[0921] Specific hardware and software

[0922] This invention primarily uses Python 3.x, the Transformers library (provided by Hugging Face), and PyTorch as its software. For hardware, the CPU and GPU installed in the customer service robot are used. The customer service robot is equipped with a touchscreen to allow customers to easily input questions.

[0923] Specific example

[0924] For example, if a customer types "What are your store's opening hours?" into the robot's touchscreen, the robot sends the question to the server. The server analyzes the question, generates an answer such as "Our store's opening hours are from 10 AM to 8 PM," and sends it to the robot. The robot then provides this answer to the customer.

[0925] Example of a prompt

[0926] "Please explain how to provide appropriate answers to customer inquiries regarding store hours in a physical store setting."

[0927] "How can we develop a system that can respond immediately in real time to customer inquiries about product inventory status?"

[0928] This invention enables prompt and accurate customer service in physical stores, improving the user experience and increasing the efficiency of store operations.

[0929] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0930] Step 1:

[0931] The server collects past query data from the database. The collected query data, in its raw form, may contain noise and unnecessary data. This data is then cleansed using data cleansing techniques to remove the unnecessary parts and format it appropriately. Through this process, the server obtains pre-processed query data.

[0932] Step 2:

[0933] The server performs tokenization and normalization on the preprocessed query data. Specifically, it uses a natural language processing library to split the text data into words and phrases and convert it into a consistent format. Through this process, the server obtains data that is in a form that is easy to parse.

[0934] Step 3:

[0935] The server trains a machine learning model using preprocessed data. It primarily uses the Transformers library (provided by Hugging Face) and PyTorch. In this step, the data is split into training and validation datasets, and the relationship between queries and corresponding answers is learned. As a result of this process, the server obtains a trained machine learning model.

[0936] Step 4:

[0937] The user enters their inquiry into a customer service robot in a physical store. The inquiry, entered using the robot's touchscreen, is sent by the robot to a server. In this step, the user's question is transmitted to the server via the robot.

[0938] Step 5:

[0939] The server then performs tokenization and normalization again on the query received from the robot. It then analyzes the query using a trained machine learning model and generates the optimal response. This step ensures the server generates a suitable answer, which it then sends to the robot.

[0940] Step 6:

[0941] The robot displays the answers received from the server and provides them to the user. Specifically, the answers are displayed on the robot's touchscreen, allowing the user to instantly obtain the appropriate information.

[0942] Step 7:

[0943] Users can provide feedback on the provided answers. This feedback is sent back to the server and used to improve the model in the future. By collecting and analyzing feedback, the server can improve the accuracy of the machine learning model.

[0944] The above outlines the processing steps for a system used for customer service in physical stores.

[0945] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0946] This invention relates to a system that generates and provides immediate and accurate answers to real-time user inquiries by collecting and preprocessing past inquiry data and training a machine learning model based on that data. Furthermore, it aims to further improve user satisfaction by incorporating an emotion engine that recognizes user emotions.

[0947] System Overview

[0948] Collection and preprocessing of query data

[0949] The server collects user inquiry data accumulated in the past from a database. The database stores a wide variety of inquiries and their corresponding answers. Since much of the collected data is incomplete in its raw form, data cleansing is used to remove noise and unnecessary data and improve data quality. Next, tokenization is performed to divide the text data into tokens (words and phrases) to prepare it for easier analysis.

[0950] Model training

[0951] The server trains a machine learning model using pre-processed data. For training, it selects an algorithm using natural language processing techniques (e.g., BERT, GPT, etc.) and trains the model on a divided dataset. This allows the server to learn from past queries and corresponding answers, and generate appropriate responses to queries.

[0952] Real-time inquiry analysis

[0953] When a user enters a question from their device, the data is sent to the server via the device. The server receives the submitted query, performs tokenization and normalization to transform it into a format that is easy to analyze. Next, it uses a trained model to analyze the query and generate the optimal answer.

[0954] Introducing an emotional engine

[0955] The server analyzes the received user inquiry text using an emotion engine. The emotion engine extracts emotions from the user's text and classifies them into emotion categories such as positive, negative, and neutral. Based on this classification, it adjusts the appropriate response or reply.

[0956] Providing a response

[0957] The server sends the generated response to the terminal, which then displays the response to the user. By taking the results of the sentiment engine into account, for example, a helpful tone is provided to a user who is "in distress," while an apology or a quick solution is offered to a user who is "dissatisfied."

[0958] Specific example

[0959] Example 1: Inquiry about login methods for new users

[0960] The user types "I don't know how to log in for the first time" from their device. The device sends this question to the server, which analyzes the received query. The sentiment engine recognizes this query as "confused." The trained model generates the best answer regarding "how to log in" and sends the response "For your first login, please use the email address you registered and enter the verification code sent to you. If you have any questions, please contact us anytime" from the server to the device. The device then displays this response to the user.

[0961] Example 2: Inquiry about update details for existing users

[0962] The user types "What was updated last week?" into their device. The device sends this question to the server, which analyzes the received query. The sentiment engine categorizes it as "neutral." The trained model generates the best possible answer regarding "updates" and sends the response "Last week's update added XX as a new feature and fixed YY as a bug" from the server to the device. The device then displays this response to the user.

[0963] Feedback and Model Updates

[0964] It is also possible to add a feature to the device that allows users to provide feedback on the answers they have received. This feedback is sent to the server and used to improve the system. The server can use this data to retrain the machine learning model, improving the system's accuracy and user satisfaction. Furthermore, by incorporating the results of the emotion engine as training data, the system can learn which emotions are most effectively addressed in its responses.

[0965] As described above, this system includes specific processing steps ranging from collecting inquiry data to providing answers and further improving the system based on feedback, enabling it to recognize user emotions and provide more appropriate support.

[0966] The following describes the processing flow.

[0967] Step 1:

[0968] The server collects past query data from the database. The database contains a wide variety of queries and their corresponding answers.

[0969] Step 2:

[0970] The server performs data cleansing on the collected query data. At this stage, noise (inappropriate data and errors) is removed, improving the quality of the data.

[0971] Step 3:

[0972] The server normalizes the data. Specifically, it converts uppercase English letters to lowercase and removes special characters to arrange the data into a consistent format.

[0973] Step 4:

[0974] The server performs tokenization, which divides the data into tokens (words or phrases). This converts the text data into a format that is easier to parse.

[0975] Step 5:

[0976] The server trains a machine learning model using preprocessed data. For training, it selects an algorithm that uses natural language processing techniques (e.g., BERT, GPT, etc.) and trains the model by dividing the dataset.

[0977] Step 6:

[0978] The server starts a real-time analysis system to process new queries, making it ready to receive user inquiries immediately.

[0979] Step 7:

[0980] Users will enter questions in a free-form format via their device. Specific questions such as "I don't know how to log in for the first time" are expected.

[0981] Step 8:

[0982] The terminal sends the questions entered by the user to the server as text data.

[0983] Step 9:

[0984] The server preprocesses the text data received from the terminal for analysis. Tokenization and normalization are performed again at this stage.

[0985] Step 10:

[0986] The server uses a machine learning model to generate the optimal response based on pre-processed query data. The model predicts the response based on historical data.

[0987] Step 11:

[0988] The server analyzes the received text data using an emotion engine. The emotion engine extracts emotions from the user's text and classifies them into emotion categories such as positive, negative, and neutral.

[0989] Step 12:

[0990] The server adjusts its responses based on the results of the emotion engine's analysis. For example, if the user is "confused," it will generate a response in a kind and caring tone.

[0991] Step 13:

[0992] The server sends the generated response to the terminal.

[0993] Step 14:

[0994] The device displays the received response in the user interface. For example, it might say, "For your first login, please use the email address you registered and enter the verification code that was sent to you."

[0995] Step 15:

[0996] Users can rate their satisfaction with the provided answers. Satisfaction feedback is entered on the device and used to improve the system.

[0997] Step 16:

[0998] The device sends user feedback to the server.

[0999] Step 17:

[1000] The server analyzes the received feedback and uses it to update its machine learning model. This improves the accuracy and quality of future inquiries.

[1001] Step 18:

[1002] The server also incorporates the results from the emotion engine as training data. This allows the system to continuously learn which emotions are most effectively addressed.

[1003] The above outlines the specific processing steps for an inquiry handling system that incorporates an emotion engine.

[1004] (Example 2)

[1005] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1006] Conventional inquiry handling systems have faced challenges in responding to user inquiries quickly and accurately, and insufficiently considering user emotions. Furthermore, they have been unable to effectively utilize feedback on the answers provided, limiting improvements in system performance and user satisfaction.

[1007] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1008] In this invention, the server includes means for collecting query data from a wide range of sources, means for preprocessing the collected query data through data cleansing and tokenization, and means for training a machine learning model using the preprocessed data. This enables the server to provide quick and accurate answers to user inquiries. The server further includes means for adjusting answers using an emotion engine that analyzes user emotions, means for providing the adjusted answers to the user, means for collecting user feedback, means for updating the machine learning model based on the collected feedback, and means for performing data tokenization and normalization. This enables responses that take user emotions into consideration, and allows for system improvement and increased user satisfaction through the use of feedback.

[1009] "Inquiry data" refers to data that includes information such as questions, requests, and feedback from users.

[1010] "Data cleansing" is the process of removing noise and unnecessary data from collected data to improve data quality.

[1011] "Tokenization" is the process of dividing text data into smaller units such as words and phrases.

[1012] A "machine learning model" is a collection of algorithms that learn patterns from input data and then use that knowledge to make predictions and classifications on new data.

[1013] An "emotion engine" is a technology that extracts and classifies users' emotions from text data.

[1014] "Preprocessing" refers to a series of processes for formatting collected data into a form that can be analyzed.

[1015] A "real-time inquiry" is an inquiry sent by a user in real time.

[1016] "Optimal response" refers to the most appropriate and effective information or answer to a user's inquiry.

[1017] "Feedback" refers to the evaluations and opinions from users regarding the services or answers provided.

[1018] "Tokenization and normalization" is the process of dividing text data into words and phrases and formatting them for analysis.

[1019] This invention relates to a system that collects and preprocesses past inquiry data to train a machine learning model, and then generates and provides immediate and accurate answers to real-time inquiries from users. Furthermore, it aims to further improve user satisfaction by incorporating an emotion engine that recognizes user emotions.

[1020] The system's main components are servers, terminals, and users. The specific functions of each component are explained below.

[1021] First, the server collects a wide range of query data from the database. This database stores many queries and their corresponding answers. The collected data is then preprocessed through data cleansing and tokenization. The Python pandas library is used for data cleansing to remove noise and unnecessary data. The NLTK or spaCy library is used for tokenization to divide the data into analyzable units.

[1022] Next, the server trains a machine learning model using the preprocessed data. Libraries such as TensorFlow and PyTorch are used for training, and the model is built using algorithms that apply natural language processing techniques (e.g., BERT, GPT). This creates a model that can learn from past inquiries and corresponding answers and generate appropriate responses.

[1023] When a user enters a question from their device, the data is transmitted to the server in real time. The server analyzes the received inquiry and generates the optimal answer using a trained model. During this answer generation process, the sentiment engine analyzes the user's inquiry and classifies it as positive, negative, neutral, etc. This enables appropriate responses based on the user's emotions. The sentiment engine uses libraries such as TextBlob and VADER.

[1024] The generated responses are sent from the server to the terminal, which then displays the responses to the user. Feedback on the responses provided by the user is also collected and used to improve the system. The server uses this feedback as retraining data, improving the system's accuracy and user satisfaction.

[1025] For example, if a user types "I don't know how to log in for the first time" into their device, the device sends this to the server. The emotion engine recognizes this as "confusion," and the trained model responds, "For your first login, please use the email address you registered and enter the verification code sent to you. If you have any questions, please contact us anytime." This response is then displayed to the user through their device.

[1026] Examples of prompt statements include:

[1027] User question: 'I don't know how to log in for the first time.'

[1028] Emotion analysis result: 'confused'

[1029] Please generate a list of possible answers.

[1030] As a result, the present invention includes specific processing steps ranging from collecting inquiry data to providing answers and further improving the system through feedback, making it possible to recognize user emotions and provide more appropriate support.

[1031] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1032] Step 1:

[1033] The server collects past query data from the database. The database stores user inquiries and their corresponding answers. The server extracts this data by executing SQL queries. For example, it might execute a query like SELECT inquiry, response FROM inquiries;

[1034] Input: Inquiry Database

[1035] Data processing: Execute SQL queries, extract data.

[1036] Output: Retrieved query data (e.g., a set of questions and corresponding answers)

[1037] Step 2:

[1038] The server preprocesses the collected query data. It performs data cleansing, using the Python pandas library to remove noise and missing data. Furthermore, it performs tokenization using the NLTK or spaCy libraries to make the text data parseable.

[1039] Input: Collected query data

[1040] Data processing: Data cleansing, tokenization

[1041] Output: Preprocessed data (tokenized text data)

[1042] Step 3:

[1043] The server trains machine learning models using preprocessed data. It utilizes libraries such as TensorFlow and PyTorch to build and train models using natural language processing algorithms (e.g., BERT, GPT).

[1044] Input: Preprocessed data

[1045] Data processing: Training machine learning models

[1046] Output: Trained model

[1047] Step 4:

[1048] When a user enters a question from their device, the device sends that data to the server. HTTP requests are commonly used for this communication. For example, a user might enter "I don't know how to log in for the first time."

[1049] Input: User's question text

[1050] Data processing: Sending to the server via HTTP request

[1051] Output: Query sent to the server

[1052] Step 5:

[1053] The server analyzes the received query, performs tokenization and normalization, and then uses a pre-trained machine learning model based on the pre-processed data to generate the best possible response.

[1054] Input: User inquiry text

[1055] Data processing: tokenization, normalization, and model-based response generation.

[1056] Output: Generated answer

[1057] Step 6:

[1058] The server analyzes the received user inquiry text using an emotion engine. The emotion engine extracts emotions such as positive, negative, and neutral from the text and adjusts the corresponding response based on the results. For example, sentiment analysis can be performed using the TextBlob library.

[1059] Input: User's inquiry text

[1060] Data processing: Sentiment analysis (positive, negative, neutral, etc.)

[1061] Output: Emotion analysis results

[1062] Step 7:

[1063] Based on the generated responses and sentiment analysis results, the server creates a response in the most appropriate tone and delivers it to the user via the device. For example, it uses a gentle tone for users who are "troubled" and offers a quick solution, including an apology, for users who are "dissatisfied."

[1064] Input: Generated response, sentiment analysis results

[1065] Data processing: Adjusting the tone of responses

[1066] Output: Adjusted answer

[1067] Step 8:

[1068] When a user enters feedback on a provided response, the device sends it to the server. A specific feedback format is used and sent to the server via an HTTP request.

[1069] Input: User feedback

[1070] Data processing: Sending to the server via HTTP request

[1071] Output: Feedback sent to the server

[1072] Step 9:

[1073] The server updates the machine learning model based on the collected feedback. This data is then incorporated back into the training set and reflected in the model as new data.

[1074] Input: Collected feedback

[1075] Data processing: Feedback integration, retraining of machine learning models

[1076] Output: Updated machine learning model

[1077] By following these steps, the system can provide quick and accurate answers to user inquiries and continuously improve based on feedback.

[1078] (Application Example 2)

[1079] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1080] Traditional customer support systems struggled to provide timely and accurate answers to user inquiries. Furthermore, they often failed to consider user emotions, leading to decreased customer satisfaction. In such cases, particularly on e-commerce sites, customer dissatisfaction became significant, negatively impacting the business. Additionally, the lack of adequate features for utilizing feedback on inquiries resulted in slow system improvements.

[1081] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for collecting inquiry data, means for pre-processing the collected inquiry data, means for training a machine learning model using the pre-processed data, means for receiving real-time inquiries from users, means for analyzing the received inquiries and generating the optimal answer using the trained model, means for providing the generated answer to the user, emotion recognition means for analyzing the user's emotions from the inquiry content, and means for adjusting the tone of the answer based on the analyzed emotions. This makes it possible to immediately provide an appropriate answer that takes into account not only the content of the inquiry but also the user's emotions, thereby improving customer satisfaction. Furthermore, by collecting feedback from users and updating the machine learning model based on it, continuous improvement of the system is also possible.

[1082] "Inquiry data" refers to information about questions and requests provided by users.

[1083] "Preprocessing" is the process of removing noise and unnecessary data from raw data to prepare it for easier analysis.

[1084] A "machine learning model" is an algorithm that learns specific patterns and relationships based on large amounts of data, and then uses that learning to make predictions and judgments about new data.

[1085] A "real-time inquiry" is an immediate question or request that a user enters and submits at that moment.

[1086] "Training" is the process of providing data to a machine learning model, allowing the model to learn patterns in that data.

[1087] "Analysis" is the process of finding meaning and patterns based on given data.

[1088] "Generation" is the process of producing appropriate responses or results based on input data.

[1089] "Emotion recognition" is the process of analyzing and determining a user's emotional state at a given time based on the content of their inquiry.

[1090] "Response tone" refers to the nuances and tone of expression in the generated response.

[1091] "Feedback" refers to evaluations and opinions provided by users regarding the system and its responses.

[1092] This invention relates to a system that provides immediate and accurate responses to user inquiries. This system is characterized by analyzing the content of the inquiry, recognizing the user's emotions, and providing responses in a tone that matches the user's emotions.

[1093] System Overview

[1094] This system consists of the following main components:

[1095] 1. Collection and preprocessing of query data:

[1096] The server collects user inquiry data accumulated in the past from the database and performs data cleansing to remove noise and unnecessary data. Next, it performs tokenization, which divides the collected text data into tokens (words and phrases), to prepare it for easier analysis.

[1097] 2. Model training:

[1098] The server trains a machine learning model using preprocessed data. For training, it selects an algorithm using natural language processing techniques (e.g., BERT, GPT, etc.) and trains the model by dividing the dataset. This allows the server to learn past queries and their corresponding answers, and generate appropriate responses to queries.

[1099] 3. Real-time query analysis:

[1100] When a user enters a question from their device, the data is sent to the server via the device. The server receives the submitted query, performs tokenization and normalization to transform it into a format that is easy to analyze. Next, it uses a trained model to analyze the query and generate the optimal answer.

[1101] 4. Introduction of the Emotion Engine:

[1102] The server analyzes the received user inquiry text using an emotion engine. The emotion engine extracts emotions from the user's text and classifies them into emotion categories such as positive, negative, and neutral. Based on this classification, it adjusts the appropriate response or reply.

[1103] 5. Providing an answer:

[1104] The server sends the generated response to the terminal, which then displays the response to the user. By taking the results of the sentiment engine into account, for example, a helpful tone is provided to a user who is "in distress," while an apology or a quick solution is offered to a user who is "dissatisfied."

[1105] 6. Feedback and Model Updates:

[1106] A feature will be added to the device that allows users to provide feedback on the answers they receive. This feedback will be sent to the server and used to improve the system. The server can use this data to retrain the machine learning model, improving the system's accuracy and user satisfaction. Furthermore, by incorporating the results of the emotion engine as training data, the system can learn which emotions are most effectively addressed in responses.

[1107] Specific example

[1108] Inquiry about how to log in as a new user

[1109] The user types "I don't know how to log in for the first time" from their device. The device sends this question to the server, which analyzes the received query. The sentiment engine recognizes this query as "confused." The trained model generates the best answer regarding "how to log in" and sends the response "For your first login, please use the email address you registered and enter the verification code sent to you. If you have any questions, please contact us anytime" from the server to the device. The device then displays this response to the user.

[1110] Inquiry about update details for existing users

[1111] The user types "What was updated last week?" into their device. The device sends this question to the server, which analyzes the received query. The sentiment engine categorizes it as "neutral." The trained model generates the best possible answer regarding "updates" and sends the response "Last week's update added XX as a new feature and fixed YY as a bug" from the server to the device. The device then displays this response to the user.

[1112] Example of a prompt

[1113] For example, if a new user asks, "I don't know how to log in for the first time," the following prompt message will be used:

[1114] I don't know how to log in for the first time.

[1115] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1116] Step 1:

[1117] The server collects past query data from the database. The input data to be collected consists of past user queries and the corresponding responses. The output is a list of the collected raw data. This data may contain noise and incomplete information.

[1118] Step 2:

[1119] The server preprocesses the collected query data. Preprocessing begins with data cleansing to remove noise and unnecessary data. Next, the text data is tokenized to prepare it for easier analysis. The input is raw data, and the output is cleansed, tokenized data.

[1120] Step 3:

[1121] The server trains a machine learning model using preprocessed data. It selects an algorithm using natural language processing techniques (e.g., BERT, GPT), splits the dataset, and trains the model. The input is the preprocessed data set, and the output is the trained machine learning model.

[1122] Step 4:

[1123] The user enters a query in real time from their terminal. The user's input is a text-based question, and the terminal sends that question to the server. The input is the user's real-time query, and the output is data sent to the server.

[1124] Step 5:

[1125] The server analyzes the received query. First, it performs tokenization and normalization to transform the query into a format that is easier to analyze. Next, it uses a trained model to analyze the query content and generate the optimal answer. The input is the query text received in real time, and the output is the generated answer.

[1126] Step 6:

[1127] The server analyzes the user's emotions from the query content. Using an emotion engine, it classifies emotions from the text into positive, negative, neutral, etc. The input is the query text, and the output is the analyzed emotion category.

[1128] Step 7:

[1129] The server adjusts the tone of its responses based on the analyzed emotions. For example, it uses a friendly tone for confused users and offers apologies or quick solutions for users with negative emotions. The input is the analyzed emotion category and the generated response, and the output is the response adjusted according to the emotion.

[1130] Step 8:

[1131] The server sends the adjusted response to the terminal, and the terminal displays that response to the user. The input is the adjusted response, and the output is what is displayed to the user.

[1132] Step 9:

[1133] The user enters feedback on the provided answers. The input is feedback text, and the device sends that feedback to the server.

[1134] Step 10:

[1135] The server updates the machine learning model based on the collected feedback. The feedback data is added to the training dataset, and retraining is performed. The input is the feedback data, and the output is the updated machine learning model.

[1136] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1137] The data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1138] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1139] [Fourth Embodiment]

[1140] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1141] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1142] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1143] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1144] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1145] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1146] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1147] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1148] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1149] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1150] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1151] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1152] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1153] This invention relates to a system that collects and preprocesses past inquiry data and trains a machine learning model based on it, thereby generating and providing immediate and accurate answers to real-time inquiries from users. The general program of the system and its processing are described below.

[1154] System Overview

[1155] Collection and preprocessing of query data

[1156] The server collects user inquiry data accumulated in the past from the database. Since the collected data is often incomplete as is, data cleansing is used to remove noise and unnecessary data and to prepare it in an appropriate format. Next, tokenization is performed to divide the text data into tokens (words and phrases), converting it into a form that can be easily processed by machines.

[1157] Model training

[1158] The server trains a machine learning model using preprocessed data. Specifically, it selects a model using natural language processing techniques (e.g., BERT, GPT), splits the data into training and validation datasets, and trains the model to learn the relationship between queries and corresponding answers. Through this process, the model acquires the ability to generate appropriate answers for various types of queries.

[1159] Real-time inquiry analysis

[1160] When a user enters a question from their device, the data is sent to the server via the device. The server receives the submitted query, performs tokenization and normalization to transform it into a format that is easy to analyze. Next, it uses a trained model to analyze the query and generate the optimal answer.

[1161] Providing a response

[1162] The server sends the generated response to the terminal, which then displays the response to the user. This allows the user to obtain an accurate answer immediately.

[1163] Specific example

[1164] Example 1: Inquiry about login methods for new users

[1165] The user types "I don't know how to log in for the first time" from their device. The device sends this question to the server, which parses the received query. The trained model generates the best answer regarding "how to log in," and the server sends the answer "For your first login, please use the email address you registered and enter the verification code sent to you" to the device. The device then displays this answer to the user.

[1166] Example 2: Inquiry about update details for existing users

[1167] The user types "What were the updates from last week?" into their device. The device sends this question to the server, which analyzes the received query. The trained model generates the best possible answer regarding "updates" and sends the response "Last week's update included the addition of XX as a new feature and YY as a bug fix" from the server to the device. The device then displays this response to the user.

[1168] Feedback and Model Updates

[1169] It is also possible to add a feature to the device that allows users to provide feedback on the answers they have provided. This feedback is sent to the server and used to improve the system. The server can use this data to retrain machine learning models and improve the system's accuracy and user satisfaction.

[1170] As described above, this system efficiently analyzes inquiry data and enables quick and accurate responses to inquiries from both new and existing users.

[1171] The following describes the processing flow.

[1172] Step 1:

[1173] The server collects past query data from the database. The database contains a wide variety of queries and their corresponding answers.

[1174] Step 2:

[1175] The server performs data cleansing on the collected query data. At this stage, noise (inappropriate data and errors) is removed, improving the quality of the data.

[1176] Step 3:

[1177] The server normalizes the data. Specifically, it converts uppercase English letters to lowercase and removes special characters to arrange the data into a consistent format.

[1178] Step 4:

[1179] The server performs tokenization, which divides the data into tokens (words or phrases). This converts the text data into a format that is easier to parse.

[1180] Step 5:

[1181] The server trains a machine learning model using preprocessed data. For training, it selects an algorithm that uses natural language processing techniques (e.g., BERT, GPT, etc.) and trains the model by dividing the dataset.

[1182] Step 6:

[1183] The server starts a real-time analysis system to process new queries, making it ready to receive user inquiries immediately.

[1184] Step 7:

[1185] Users will enter questions in a free-form format via their device. Specific questions such as "I don't know how to log in for the first time" are expected.

[1186] Step 8:

[1187] The terminal sends the questions entered by the user to the server as text data.

[1188] Step 9:

[1189] The server preprocesses the text data received from the terminal for analysis. Tokenization and normalization are performed again at this stage.

[1190] Step 10:

[1191] The server uses a machine learning model to generate the optimal response based on pre-processed query data. The model predicts the response based on historical data.

[1192] Step 11:

[1193] The server sends the generated response as text data to the terminal.

[1194] Step 12:

[1195] The device displays the received response in the user interface. For example, it might say, "For your first login, please use the email address you registered and enter the verification code that was sent to you."

[1196] Step 13:

[1197] Users can rate their satisfaction with the provided answers. Satisfaction feedback is entered on the device and used to improve the system.

[1198] Step 14:

[1199] The device sends user feedback to the server.

[1200] Step 15:

[1201] The server analyzes the received feedback and uses it to update its machine learning model. This improves the accuracy and quality of future inquiries.

[1202] The above outlines the specific processing steps from collecting inquiry data to providing responses and further improving the system based on feedback.

[1203] (Example 1)

[1204] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1205] Conventional inquiry handling systems often suffered from delays in responding to user inquiries and had limited performance in generating appropriate answers. Furthermore, updating models based on feedback was cumbersome, posing a risk of decreased accuracy. Therefore, there was a need for a system that could provide real-time, rapid, and accurate answers.

[1206] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1207] In this invention, the server includes means for collecting query data, means for preprocessing the collected query data, means for training a machine learning model using the preprocessed data, means for receiving real-time queries from users, means for analyzing the received queries and generating the optimal answer using the trained model, means for denoising and formatting the data, means for using natural language processing technology as the machine learning model, and means for providing the generated answer to the user. This makes it possible to provide fast and accurate answers in real time.

[1208] "Inquiry data" refers to data that includes information about questions and requests from users.

[1209] "Preprocessing" refers to the process of applying noise reduction and formatting to collected data to make it easier to analyze.

[1210] A "machine learning model" is an algorithm that learns from input data and performs predictions and classifications for specific tasks.

[1211] "Training" is the process by which a machine learning model learns using pre-processed data.

[1212] A "real-time inquiry" is a question or request that a user makes to a system instantly.

[1213] "Noise reduction" is the process of removing unwanted or inaccurate information from data.

[1214] "Formatting" refers to the process of aligning data to a specific format or structure.

[1215] "Natural language processing technology" is the technology that enables computers to understand and generate human language.

[1216] "Feedback" refers to the evaluations and comments that users make regarding the answers provided.

[1217] "Tokenization" is the process of dividing text data into units such as words and phrases.

[1218] "Normalization" is the process of converting data into a standard format in order to maintain data consistency.

[1219] This invention is a system that efficiently collects and analyzes inquiry data and provides quick and accurate responses to real-time inquiries from users. This system uses the following hardware and software.

[1220] hardware

[1221] Server: A central processing unit for data collection, preprocessing, model training, query analysis, response generation, and delivery. This server can utilize data center-level computers equipped with high-performance CPUs and GPUs.

[1222] Terminal: A device used by users to input inquiries and receive responses. This includes common computing devices such as personal computers, tablets, or smartphones.

[1223] software

[1224] Database system: Stores and manages query data. Specifically, it uses relational databases such as MySQL or PostgreSQL.

[1225] Data cleansing tools are software that removes noise and formats collected data. An example is OpenRefine.

[1226] Natural language processing libraries: These perform tokenization and normalization of text data. Specifically, libraries such as NLTK and spaCy are used.

[1227] Machine learning frameworks: These frameworks train machine learning models based on preprocessed data and analyze queries. Specific examples include TensorFlow, PyTorch, and Transformers libraries (BERT, GPT, etc.).

[1228] Web server: Receives and provides responses to queries in real time. Python frameworks such as FastAPI and Flask are used.

[1229] Specific example

[1230] When a user types "I don't know how to log in for the first time" from their computer, the device sends this inquiry to the server. The server tokenizes and normalizes the received inquiry data using natural language processing libraries (NLTK, spaCy). A trained machine learning model (e.g., BERT, GPT) is used to analyze the inquiry and generate the most appropriate response. In this case, the response generated is "For your first login, please use your registered email address and enter the verification code sent to you." The generated response is sent from the server to the device and displayed to the user.

[1231] Example of a prompt

[1232] For example, the following types of user inquiries can be handled in a similar manner:

[1233] "What should I do if I forget my password?"

[1234] "I want to know what's in the latest update."

[1235] This allows users to receive accurate answers immediately and solve problems efficiently. Furthermore, user feedback can be collected, and the server can use this data to retrain the model and improve the system's accuracy.

[1236] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1237] Step 1: Collecting inquiry data

[1238] The server collects user query data accumulated in the past from a database. This database is a relational database system (e.g., MySQL, PostgreSQL). The server executes SQL queries to extract the query data and converts it into a data frame format (e.g., Pandas DataFrame).

[1239] Input: Query data in the database

[1240] Output: Query data in data frame format

[1241] Step 2: Data preprocessing

[1242] The server performs data cleansing on the collected data. Specifically, it uses Pandas to remove noisy data (e.g., missing values, redundant data) and prepares the data in a clean state. Next, it uses a natural language processing library (e.g., NLTK, spaCy) to tokenize the text data and then format it.

[1243] Input: Query data in data frame format

[1244] Output: Tokenized and cleansed query data

[1245] Step 3: Training the machine learning model

[1246] The server trains a machine learning model using preprocessed data. Here, we implement a model using natural language processing techniques (e.g., BERT, GPT). The data is split into training and validation datasets, and the model is trained using query-and-appropriate response pairs. Frameworks such as TensorFlow and PyTorch are used.

[1247] Input: Tokenized query data and corresponding response data

[1248] Output: Trained machine learning model

[1249] Step 4: Receiving real-time inquiries

[1250] When a user enters a query from their device, that data is sent to the server via the device. Here, JavaScript is used for the frontend, and a lightweight web framework such as FastAPI is used for the backend.

[1251] Input: Inquiry text entered by the user

[1252] Output: Sending query data to the server

[1253] Step 5: Analyze the inquiry and generate the answer.

[1254] The server analyzes incoming queries in real time. First, it uses a natural language processing library to tokenize and normalize the queries, and then uses a trained machine learning model to generate the optimal response.

[1255] Input: Received query data

[1256] Output: Generated response data

[1257] Step 6: Provide your response

[1258] The server sends the generated response to the terminal, and the terminal displays that response to the user. HTTP or WebSocket is used as the communication protocol for this process.

[1259] Input: Server-generated response data

[1260] Output: Answer displayed to the user

[1261] Step 7: Gathering Feedback and Updating the Model

[1262] Users provide feedback on the provided answers. This feedback is sent to the server via the terminal. The server retrains the model based on the feedback data to improve the system's accuracy.

[1263] Input: User-entered feedback data

[1264] Output: Updated machine learning model

[1265] (Application Example 1)

[1266] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1267] Traditional customer support systems struggled to provide quick and accurate responses to real-time user inquiries. Furthermore, there was a lack of means to improve the quality and efficiency of customer service in physical stores. In particular, during peak hours, staff shortages led to customers having to wait, which was a significant problem.

[1268] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1269] In this invention, the server includes means for collecting inquiry data, means for preprocessing the collected inquiry data, means for training a machine learning model using the preprocessed data, means for receiving real-time inquiries from users, means for analyzing the received inquiries and generating the optimal answer using the trained model, means for providing the generated answer to the user, means for installing it on a robot placed in a physical store, and means for the robot to receive customer inquiries and provide answers. This enables quick and accurate customer service even in physical stores.

[1270] "Inquiry data" refers to data related to questions and requests from customers and users.

[1271] "Preprocessing" refers to a series of processes that convert raw data into a format suitable for analysis and training.

[1272] A "machine learning model" is an algorithm that learns from large amounts of data and performs predictions and classifications for specific tasks.

[1273] A "real-time inquiry" is an immediate question or request made by a user during an ongoing session.

[1274] "Analysis" is the process of interpreting data and information to derive meaning in accordance with a specific purpose.

[1275] "Answer generation" is the process of constructing an appropriate response to an inquiry.

[1276] "Providing" refers to the act of showing or sending the generated response to the user.

[1277] A "physical store" is a commercial facility that provides goods and services in a physical location.

[1278] A "robot" is a mechanical device that performs specific tasks autonomously or remotely.

[1279] "Feedback" refers to user opinions and evaluations regarding the system's operation and the responses provided.

[1280] "Tokenization" is the process of dividing text data into words or phrases.

[1281] "Normalization" is the process of arranging the format and content of data into a consistent and standard form.

[1282] This invention relates to a system that uses robots placed in physical stores to provide appropriate answers to customer inquiries in real time.

[1283] Collection and preprocessing of query data

[1284] The server collects past query data from the database. Since the collected data is often incomplete, data cleansing techniques are used to remove noise and unnecessary data, and to prepare it in an appropriate format. Then, tokenization is performed to divide the text data into words and phrases, and further normalization is carried out to convert it into a form that is easy to analyze.

[1285] Model training

[1286] The server trains a machine learning model using pre-processed data. This process utilizes natural language processing techniques, and models such as BERT and GPT are selected. The collected data is divided into a training dataset and a validation dataset, and the relationship between queries and corresponding answers is learned.

[1287] Real-time inquiry analysis

[1288] When a customer makes a request to a robot placed in a physical store, the data is sent by the robot to a server. The server tokenizes and normalizes the received data, and then uses a trained model to analyze the request and generate the best possible answer.

[1289] Providing a response

[1290] The server sends the generated response to the robot, which then provides the response to the customer. This allows the customer to receive the appropriate answer immediately.

[1291] Specific hardware and software

[1292] This invention primarily uses Python 3.x, the Transformers library (provided by Hugging Face), and PyTorch as its software. For hardware, the CPU and GPU installed in the customer service robot are used. The customer service robot is equipped with a touchscreen to allow customers to easily input questions.

[1293] Specific example

[1294] For example, if a customer types "What are your store's opening hours?" into the robot's touchscreen, the robot sends the question to the server. The server analyzes the question, generates an answer such as "Our store's opening hours are from 10 AM to 8 PM," and sends it to the robot. The robot then provides this answer to the customer.

[1295] Example of a prompt

[1296] "Please explain how to provide appropriate answers to customer inquiries regarding store hours in a physical store setting."

[1297] "How can we develop a system that can respond immediately in real time to customer inquiries about product inventory status?"

[1298] This invention enables prompt and accurate customer service in physical stores, improving the user experience and increasing the efficiency of store operations.

[1299] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1300] Step 1:

[1301] The server collects past query data from the database. The collected query data, in its raw form, may contain noise and unnecessary data. This data is then cleansed using data cleansing techniques to remove the unnecessary parts and format it appropriately. Through this process, the server obtains pre-processed query data.

[1302] Step 2:

[1303] The server performs tokenization and normalization on the preprocessed query data. Specifically, it uses a natural language processing library to split the text data into words and phrases and convert it into a consistent format. Through this process, the server obtains data that is in a form that is easy to parse.

[1304] Step 3:

[1305] The server trains a machine learning model using preprocessed data. It primarily uses the Transformers library (provided by Hugging Face) and PyTorch. In this step, the data is split into training and validation datasets, and the relationship between queries and corresponding answers is learned. As a result of this process, the server obtains a trained machine learning model.

[1306] Step 4:

[1307] The user enters their inquiry into a customer service robot in a physical store. The inquiry, entered using the robot's touchscreen, is sent by the robot to a server. In this step, the user's question is transmitted to the server via the robot.

[1308] Step 5:

[1309] The server then performs tokenization and normalization again on the query received from the robot. It then analyzes the query using a trained machine learning model and generates the optimal response. This step ensures the server generates a suitable answer, which it then sends to the robot.

[1310] Step 6:

[1311] The robot displays the answers received from the server and provides them to the user. Specifically, the answers are displayed on the robot's touchscreen, allowing the user to instantly obtain the appropriate information.

[1312] Step 7:

[1313] Users can provide feedback on the provided answers. This feedback is sent back to the server and used to improve the model in the future. By collecting and analyzing feedback, the server can improve the accuracy of the machine learning model.

[1314] The above outlines the processing steps for a system used for customer service in physical stores.

[1315] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1316] This invention relates to a system that generates and provides immediate and accurate answers to real-time user inquiries by collecting and preprocessing past inquiry data and training a machine learning model based on that data. Furthermore, it aims to further improve user satisfaction by incorporating an emotion engine that recognizes user emotions.

[1317] System Overview

[1318] Collection and preprocessing of query data

[1319] The server collects user inquiry data accumulated in the past from a database. The database stores a wide variety of inquiries and their corresponding answers. Since much of the collected data is incomplete in its raw form, data cleansing is used to remove noise and unnecessary data and improve data quality. Next, tokenization is performed to divide the text data into tokens (words and phrases) to prepare it for easier analysis.

[1320] Model training

[1321] The server trains a machine learning model using pre-processed data. For training, it selects an algorithm using natural language processing techniques (e.g., BERT, GPT, etc.) and trains the model on a divided dataset. This allows the server to learn from past queries and corresponding answers, and generate appropriate responses to queries.

[1322] Real-time inquiry analysis

[1323] When a user enters a question from their device, the data is sent to the server via the device. The server receives the submitted query, performs tokenization and normalization to transform it into a format that is easy to analyze. Next, it uses a trained model to analyze the query and generate the optimal answer.

[1324] Introducing an emotional engine

[1325] The server analyzes the received user inquiry text using an emotion engine. The emotion engine extracts emotions from the user's text and classifies them into emotion categories such as positive, negative, and neutral. Based on this classification, it adjusts the appropriate response or reply.

[1326] Providing a response

[1327] The server sends the generated response to the terminal, which then displays the response to the user. By taking the results of the sentiment engine into account, for example, a helpful tone is provided to a user who is "in distress," while an apology or a quick solution is offered to a user who is "dissatisfied."

[1328] Specific example

[1329] Example 1: Inquiry about login methods for new users

[1330] The user types "I don't know how to log in for the first time" from their device. The device sends this question to the server, which analyzes the received query. The sentiment engine recognizes this query as "confused." The trained model generates the best answer regarding "how to log in" and sends the response "For your first login, please use the email address you registered and enter the verification code sent to you. If you have any questions, please contact us anytime" from the server to the device. The device then displays this response to the user.

[1331] Example 2: Inquiry about update details for existing users

[1332] The user types "What was updated last week?" into their device. The device sends this question to the server, which analyzes the received query. The sentiment engine categorizes it as "neutral." The trained model generates the best possible answer regarding "updates" and sends the response "Last week's update added XX as a new feature and fixed YY as a bug" from the server to the device. The device then displays this response to the user.

[1333] Feedback and Model Updates

[1334] It is also possible to add a feature to the device that allows users to provide feedback on the answers they have received. This feedback is sent to the server and used to improve the system. The server can use this data to retrain the machine learning model, improving the system's accuracy and user satisfaction. Furthermore, by incorporating the results of the emotion engine as training data, the system can learn which emotions are most effectively addressed in its responses.

[1335] As described above, this system includes specific processing steps ranging from collecting inquiry data to providing answers and further improving the system based on feedback, enabling it to recognize user emotions and provide more appropriate support.

[1336] The following describes the processing flow.

[1337] Step 1:

[1338] The server collects past query data from the database. The database contains a wide variety of queries and their corresponding answers.

[1339] Step 2:

[1340] The server performs data cleansing on the collected query data. At this stage, noise (inappropriate data and errors) is removed, improving the quality of the data.

[1341] Step 3:

[1342] The server normalizes the data. Specifically, it converts uppercase English letters to lowercase and removes special characters to arrange the data into a consistent format.

[1343] Step 4:

[1344] The server performs tokenization, which divides the data into tokens (words or phrases). This converts the text data into a format that is easier to parse.

[1345] Step 5:

[1346] The server trains a machine learning model using preprocessed data. For training, it selects an algorithm that uses natural language processing techniques (e.g., BERT, GPT, etc.) and trains the model by dividing the dataset.

[1347] Step 6:

[1348] The server starts a real-time analysis system to process new queries, making it ready to receive user inquiries immediately.

[1349] Step 7:

[1350] Users will enter questions in a free-form format via their device. Specific questions such as "I don't know how to log in for the first time" are expected.

[1351] Step 8:

[1352] The terminal sends the questions entered by the user to the server as text data.

[1353] Step 9:

[1354] The server preprocesses the text data received from the terminal for analysis. Tokenization and normalization are performed again at this stage.

[1355] Step 10:

[1356] The server uses a machine learning model to generate the optimal response based on pre-processed query data. The model predicts the response based on historical data.

[1357] Step 11:

[1358] The server analyzes the received text data using an emotion engine. The emotion engine extracts emotions from the user's text and classifies them into emotion categories such as positive, negative, and neutral.

[1359] Step 12:

[1360] The server adjusts its responses based on the results of the emotion engine's analysis. For example, if the user is "confused," it will generate a response in a kind and caring tone.

[1361] Step 13:

[1362] The server sends the generated response to the terminal.

[1363] Step 14:

[1364] The device displays the received response in the user interface. For example, it might say, "For your first login, please use the email address you registered and enter the verification code that was sent to you."

[1365] Step 15:

[1366] Users can rate their satisfaction with the provided answers. Satisfaction feedback is entered on the device and used to improve the system.

[1367] Step 16:

[1368] The device sends user feedback to the server.

[1369] Step 17:

[1370] The server analyzes the received feedback and uses it to update its machine learning model. This improves the accuracy and quality of future inquiries.

[1371] Step 18:

[1372] The server also incorporates the results from the emotion engine as training data. This allows the system to continuously learn which emotions are most effectively addressed.

[1373] The above outlines the specific processing steps for an inquiry handling system that incorporates an emotion engine.

[1374] (Example 2)

[1375] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1376] Conventional inquiry handling systems have faced challenges in responding to user inquiries quickly and accurately, and insufficiently considering user emotions. Furthermore, they have been unable to effectively utilize feedback on the answers provided, limiting improvements in system performance and user satisfaction.

[1377] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1378] In this invention, the server includes means for collecting query data from a wide range of sources, means for preprocessing the collected query data through data cleansing and tokenization, and means for training a machine learning model using the preprocessed data. This enables the server to provide quick and accurate answers to user inquiries. The server further includes means for adjusting answers using an emotion engine that analyzes user emotions, means for providing the adjusted answers to the user, means for collecting user feedback, means for updating the machine learning model based on the collected feedback, and means for performing data tokenization and normalization. This enables responses that take user emotions into consideration, and allows for system improvement and increased user satisfaction through the use of feedback.

[1379] "Inquiry data" refers to data that includes information such as questions, requests, and feedback from users.

[1380] "Data cleansing" is the process of removing noise and unnecessary data from collected data to improve data quality.

[1381] "Tokenization" is the process of dividing text data into smaller units such as words and phrases.

[1382] A "machine learning model" is a collection of algorithms that learn patterns from input data and then use that knowledge to make predictions and classifications on new data.

[1383] An "emotion engine" is a technology that extracts and classifies users' emotions from text data.

[1384] "Preprocessing" refers to a series of processes for formatting collected data into a form that can be analyzed.

[1385] A "real-time inquiry" is an inquiry sent by a user in real time.

[1386] "Optimal response" refers to the most appropriate and effective information or answer to a user's inquiry.

[1387] "Feedback" refers to the evaluations and opinions from users regarding the services or answers provided.

[1388] "Tokenization and normalization" is the process of dividing text data into words and phrases and formatting them for analysis.

[1389] This invention relates to a system that collects and preprocesses past inquiry data to train a machine learning model, and then generates and provides immediate and accurate answers to real-time inquiries from users. Furthermore, it aims to further improve user satisfaction by incorporating an emotion engine that recognizes user emotions.

[1390] The system's main components are servers, terminals, and users. The specific functions of each component are explained below.

[1391] First, the server collects a wide range of query data from the database. This database stores many queries and their corresponding answers. The collected data is then preprocessed through data cleansing and tokenization. The Python pandas library is used for data cleansing to remove noise and unnecessary data. The NLTK or spaCy library is used for tokenization to divide the data into analyzable units.

[1392] Next, the server trains a machine learning model using the preprocessed data. Libraries such as TensorFlow and PyTorch are used for training, and the model is built using algorithms that apply natural language processing techniques (e.g., BERT, GPT). This creates a model that can learn from past inquiries and corresponding answers and generate appropriate responses.

[1393] When a user enters a question from their device, the data is transmitted to the server in real time. The server analyzes the received inquiry and generates the optimal answer using a trained model. During this answer generation process, the sentiment engine analyzes the user's inquiry and classifies it as positive, negative, neutral, etc. This enables appropriate responses based on the user's emotions. The sentiment engine uses libraries such as TextBlob and VADER.

[1394] The generated responses are sent from the server to the terminal, which then displays the responses to the user. Feedback on the responses provided by the user is also collected and used to improve the system. The server uses this feedback as retraining data, improving the system's accuracy and user satisfaction.

[1395] For example, if a user types "I don't know how to log in for the first time" into their device, the device sends this to the server. The emotion engine recognizes this as "confusion," and the trained model responds, "For your first login, please use the email address you registered and enter the verification code sent to you. If you have any questions, please contact us anytime." This response is then displayed to the user through their device.

[1396] Examples of prompt statements include:

[1397] User question: 'I don't know how to log in for the first time.'

[1398] Emotion analysis result: 'confused'

[1399] Please generate a list of possible answers.

[1400] As a result, the present invention includes specific processing steps ranging from collecting inquiry data to providing answers and further improving the system through feedback, making it possible to recognize user emotions and provide more appropriate support.

[1401] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1402] Step 1:

[1403] The server collects past query data from the database. The database stores user inquiries and their corresponding answers. The server extracts this data by executing SQL queries. For example, it might execute a query like SELECT inquiry, response FROM inquiries;

[1404] Input: Inquiry Database

[1405] Data processing: Execute SQL queries, extract data.

[1406] Output: Retrieved query data (e.g., a set of questions and corresponding answers)

[1407] Step 2:

[1408] The server preprocesses the collected query data. It performs data cleansing, using the Python pandas library to remove noise and missing data. Furthermore, it performs tokenization using the NLTK or spaCy libraries to make the text data parseable.

[1409] Input: Collected query data

[1410] Data processing: Data cleansing, tokenization

[1411] Output: Preprocessed data (tokenized text data)

[1412] Step 3:

[1413] The server trains machine learning models using preprocessed data. It utilizes libraries such as TensorFlow and PyTorch to build and train models using natural language processing algorithms (e.g., BERT, GPT).

[1414] Input: Preprocessed data

[1415] Data processing: Training machine learning models

[1416] Output: Trained model

[1417] Step 4:

[1418] When a user enters a question from their device, the device sends that data to the server. HTTP requests are commonly used for this communication. For example, a user might enter "I don't know how to log in for the first time."

[1419] Input: User's question text

[1420] Data processing: Sending to the server via HTTP request

[1421] Output: Query sent to the server

[1422] Step 5:

[1423] The server analyzes the received query, performs tokenization and normalization, and then uses a pre-trained machine learning model based on the pre-processed data to generate the best possible response.

[1424] Input: User inquiry text

[1425] Data processing: tokenization, normalization, and model-based response generation.

[1426] Output: Generated answer

[1427] Step 6:

[1428] The server analyzes the received user inquiry text using an emotion engine. The emotion engine extracts emotions such as positive, negative, and neutral from the text and adjusts the corresponding response based on the results. For example, sentiment analysis can be performed using the TextBlob library.

[1429] Input: User's inquiry text

[1430] Data processing: Sentiment analysis (positive, negative, neutral, etc.)

[1431] Output: Emotion analysis results

[1432] Step 7:

[1433] Based on the generated responses and sentiment analysis results, the server creates a response in the most appropriate tone and delivers it to the user via the device. For example, it uses a gentle tone for users who are "troubled" and offers a quick solution, including an apology, for users who are "dissatisfied."

[1434] Input: Generated response, sentiment analysis results

[1435] Data processing: Adjusting the tone of responses

[1436] Output: Adjusted answer

[1437] Step 8:

[1438] When a user enters feedback on a provided response, the device sends it to the server. A specific feedback format is used and sent to the server via an HTTP request.

[1439] Input: User feedback

[1440] Data processing: Sending to the server via HTTP request

[1441] Output: Feedback sent to the server

[1442] Step 9:

[1443] The server updates the machine learning model based on the collected feedback. This data is then incorporated back into the training set and reflected in the model as new data.

[1444] Input: Collected feedback

[1445] Data processing: Feedback integration, retraining of machine learning models

[1446] Output: Updated machine learning model

[1447] By following these steps, the system can provide quick and accurate answers to user inquiries and continuously improve based on feedback.

[1448] (Application Example 2)

[1449] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1450] Traditional customer support systems struggled to provide timely and accurate answers to user inquiries. Furthermore, they often failed to consider user emotions, leading to decreased customer satisfaction. In such cases, particularly on e-commerce sites, customer dissatisfaction became significant, negatively impacting the business. Additionally, the lack of adequate features for utilizing feedback on inquiries resulted in slow system improvements.

[1451] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for collecting inquiry data, means for pre-processing the collected inquiry data, means for training a machine learning model using the pre-processed data, means for receiving real-time inquiries from users, means for analyzing the received inquiries and generating the optimal answer using the trained model, means for providing the generated answer to the user, emotion recognition means for analyzing the user's emotions from the inquiry content, and means for adjusting the tone of the answer based on the analyzed emotions. This makes it possible to immediately provide an appropriate answer that takes into account not only the content of the inquiry but also the user's emotions, thereby improving customer satisfaction. Furthermore, by collecting feedback from users and updating the machine learning model based on it, continuous improvement of the system is also possible.

[1452] "Inquiry data" refers to information about questions and requests provided by users.

[1453] "Preprocessing" is the process of removing noise and unnecessary data from raw data to prepare it for easier analysis.

[1454] A "machine learning model" is an algorithm that learns specific patterns and relationships based on large amounts of data, and then uses that learning to make predictions and judgments about new data.

[1455] A "real-time inquiry" is an immediate question or request that a user enters and submits at that moment.

[1456] "Training" is the process of providing data to a machine learning model, allowing the model to learn patterns in that data.

[1457] "Analysis" is the process of finding meaning and patterns based on given data.

[1458] "Generation" is the process of producing appropriate responses or results based on input data.

[1459] "Emotion recognition" is the process of analyzing and determining a user's emotional state at a given time based on the content of their inquiry.

[1460] "Response tone" refers to the nuances and tone of expression in the generated response.

[1461] "Feedback" refers to evaluations and opinions provided by users regarding the system and its responses.

[1462] This invention relates to a system that provides immediate and accurate responses to user inquiries. This system is characterized by analyzing the content of the inquiry, recognizing the user's emotions, and providing responses in a tone that matches the user's emotions.

[1463] System Overview

[1464] This system consists of the following main components:

[1465] 1. Collection and preprocessing of query data:

[1466] The server collects user inquiry data accumulated in the past from the database and performs data cleansing to remove noise and unnecessary data. Next, it performs tokenization, which divides the collected text data into tokens (words and phrases), to prepare it for easier analysis.

[1467] 2. Model training:

[1468] The server trains a machine learning model using preprocessed data. For training, it selects an algorithm using natural language processing techniques (e.g., BERT, GPT, etc.) and trains the model by dividing the dataset. This allows the server to learn past queries and their corresponding answers, and generate appropriate responses to queries.

[1469] 3. Real-time query analysis:

[1470] When a user enters a question from their device, the data is sent to the server via the device. The server receives the submitted query, performs tokenization and normalization to transform it into a format that is easy to analyze. Next, it uses a trained model to analyze the query and generate the optimal answer.

[1471] 4. Introduction of the Emotion Engine:

[1472] The server analyzes the received user inquiry text using an emotion engine. The emotion engine extracts emotions from the user's text and classifies them into emotion categories such as positive, negative, and neutral. Based on this classification, it adjusts the appropriate response or reply.

[1473] 5. Providing an answer:

[1474] The server sends the generated response to the terminal, which then displays the response to the user. By taking the results of the sentiment engine into account, for example, a helpful tone is provided to a user who is "in distress," while an apology or a quick solution is offered to a user who is "dissatisfied."

[1475] 6. Feedback and Model Updates:

[1476] A feature will be added to the device that allows users to provide feedback on the answers they receive. This feedback will be sent to the server and used to improve the system. The server can use this data to retrain the machine learning model, improving the system's accuracy and user satisfaction. Furthermore, by incorporating the results of the emotion engine as training data, the system can learn which emotions are most effectively addressed in responses.

[1477] Specific example

[1478] Inquiry about how to log in as a new user

[1479] The user types "I don't know how to log in for the first time" from their device. The device sends this question to the server, which analyzes the received query. The sentiment engine recognizes this query as "confused." The trained model generates the best answer regarding "how to log in" and sends the response "For your first login, please use the email address you registered and enter the verification code sent to you. If you have any questions, please contact us anytime" from the server to the device. The device then displays this response to the user.

[1480] Inquiry about update details for existing users

[1481] The user types "What was updated last week?" into their device. The device sends this question to the server, which analyzes the received query. The sentiment engine categorizes it as "neutral." The trained model generates the best possible answer regarding "updates" and sends the response "Last week's update added XX as a new feature and fixed YY as a bug" from the server to the device. The device then displays this response to the user.

[1482] Example of a prompt

[1483] For example, if a new user asks, "I don't know how to log in for the first time," the following prompt message will be used:

[1484] I don't know how to log in for the first time.

[1485] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1486] Step 1:

[1487] The server collects past query data from the database. The input data to be collected consists of past user queries and the corresponding responses. The output is a list of the collected raw data. This data may contain noise and incomplete information.

[1488] Step 2:

[1489] The server preprocesses the collected query data. Preprocessing begins with data cleansing to remove noise and unnecessary data. Next, the text data is tokenized to prepare it for easier analysis. The input is raw data, and the output is cleansed, tokenized data.

[1490] Step 3:

[1491] The server trains a machine learning model using preprocessed data. It selects an algorithm using natural language processing techniques (e.g., BERT, GPT), splits the dataset, and trains the model. The input is the preprocessed data set, and the output is the trained machine learning model.

[1492] Step 4:

[1493] The user enters a query in real time from their terminal. The user's input is a text-based question, and the terminal sends that question to the server. The input is the user's real-time query, and the output is data sent to the server.

[1494] Step 5:

[1495] The server analyzes the received query. First, it performs tokenization and normalization to transform the query into a format that is easier to analyze. Next, it uses a trained model to analyze the query content and generate the optimal answer. The input is the query text received in real time, and the output is the generated answer.

[1496] Step 6:

[1497] The server analyzes the user's emotions from the query content. Using an emotion engine, it classifies emotions from the text into positive, negative, neutral, etc. The input is the query text, and the output is the analyzed emotion category.

[1498] Step 7:

[1499] The server adjusts the tone of its responses based on the analyzed emotions. For example, it uses a friendly tone for confused users and offers apologies or quick solutions for users with negative emotions. The input is the analyzed emotion category and the generated response, and the output is the response adjusted according to the emotion.

[1500] Step 8:

[1501] The server sends the adjusted response to the terminal, and the terminal displays that response to the user. The input is the adjusted response, and the output is what is displayed to the user.

[1502] Step 9:

[1503] The user enters feedback on the provided answers. The input is feedback text, and the device sends that feedback to the server.

[1504] Step 10:

[1505] The server updates the machine learning model based on the collected feedback. The feedback data is added to the training dataset, and retraining is performed. The input is the feedback data, and the output is the updated machine learning model.

[1506] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1507] The data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1508] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1509] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1510] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1511] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1512] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1513] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1514] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1515] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1516] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1517] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1518] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1519] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1520] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1521] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1522] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1523] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1524] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1525] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1526] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[1527] The following is further disclosed regarding the embodiments described above.

[1528] (Claim 1)

[1529] Means of collecting inquiry data,

[1530] A means for preprocessing the collected query data,

[1531] A method for training a machine learning model using preprocessed data,

[1532] A means of receiving real-time inquiries from users,

[1533] A means for analyzing received inquiries and generating the optimal answer using a trained model,

[1534] A system that includes means of providing the generated response to the user.

[1535] (Claim 2)

[1536] A means of collecting user feedback,

[1537] The system according to claim 1, further comprising means for updating a machine learning model based on collected feedback.

[1538] (Claim 3)

[1539] The system according to claim 1, comprising means for performing data tokenization and normalization.

[1540] "Example 1"

[1541] (Claim 1)

[1542] Means of collecting inquiry data,

[1543] A means for preprocessing the collected query data,

[1544] A method for training a machine learning model using preprocessed data,

[1545] A means of receiving real-time inquiries from users,

[1546] A means for analyzing received inquiries and generating the optimal answer using a trained model,

[1547] A means of denoising and formatting data,

[1548] Methods for using natural language processing techniques as machine learning models,

[1549] A system that includes means of providing the generated response to the user.

[1550] (Claim 2)

[1551] A means of collecting user feedback,

[1552] The system according to claim 1, further comprising means for updating a machine learning model based on collected feedback.

[1553] (Claim 3)

[1554] The system according to claim 1, comprising means for tokenizing and normalizing data, and means for transmitting data from a device that receives a query to a device that displays a response.

[1555] "Application Example 1"

[1556] (Claim 1)

[1557] Means of collecting inquiry data,

[1558] A means for preprocessing the collected query data,

[1559] A method for training a machine learning model using preprocessed data,

[1560] A means of receiving real-time inquiries from users,

[1561] A means for analyzing received inquiries and generating the optimal answer using a trained model,

[1562] A means of providing the generated answer to the user,

[1563] Methods for installing it on robots placed in physical stores,

[1564] A system that includes means for a robot to receive customer inquiries and provide answers.

[1565] (Claim 2)

[1566] A means of collecting user feedback,

[1567] The system according to claim 1, further comprising means for updating a machine learning model based on collected feedback.

[1568] (Claim 3)

[1569] The system according to claim 1, comprising means for performing data tokenization and normalization.

[1570] "Example 2 of combining an emotion engine"

[1571] (Claim 1)

[1572] A means of collecting inquiry data from a wide range of sources,

[1573] A means for preprocessing collected query data through data cleansing and tokenization,

[1574] A means of training a machine learning model using preprocessed data,

[1575] A means of receiving real-time inquiries from users,

[1576] A means of analyzing received inquiries and generating the optimal answer using a trained model,

[1577] A system that includes means for sending generated responses to users.

[1578] (Claim 2)

[1579] A means of adjusting responses using an emotion engine that analyzes the user's emotions,

[1580] The system according to claim 1, further comprising means for providing a adjusted response to the user.

[1581] (Claim 3)

[1582] A means of collecting user feedback,

[1583] The system according to claim 1, further comprising means for updating a machine learning model based on collected feedback.

[1584] (Claim 4)

[1585] The system according to claim 1, further comprising means for performing data tokenization and normalization.

[1586] "Application example 2 when combining with an emotional engine"

[1587] (Claim 1)

[1588] Means of collecting inquiry data,

[1589] A means for preprocessing the collected query data,

[1590] A method for training a machine learning model using preprocessed data,

[1591] A means of receiving real-time inquiries from users,

[1592] A means for analyzing received inquiries and generating the optimal answer using a trained model,

[1593] A means of providing the generated answer to the user,

[1594] A means of sentiment recognition that analyzes the user's emotions from the content of the inquiry,

[1595] A system that includes means for adjusting the tone of responses based on analyzed emotions.

[1596] (Claim 2)

[1597] A means of collecting user feedback,

[1598] The system according to claim 1, further comprising means for updating a machine learning model based on collected feedback.

[1599] (Claim 3)

[1600] The system according to claim 1, comprising means for performing data tokenization and normalization. [Explanation of symbols]

[1601] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means of collecting inquiry data, A means for preprocessing the collected query data, A method for training a machine learning model using preprocessed data, A means of receiving real-time inquiries from users, A means for analyzing received inquiries and generating the optimal answer using a trained model, A system that includes means of providing the generated response to the user.

2. A means of collecting user feedback, The system according to claim 1, further comprising means for updating a machine learning model based on collected feedback.

3. The system according to claim 1, comprising means for performing data tokenization and normalization.

Images