system

Abstract

Provide a system. 【Solution means】 Input means for receiving inquiries from users, Information processing means for analyzing inquiries using natural language processing, Answer generation means for referring to a knowledge base based on the analysis result and generating an appropriate response, Answer providing means for presenting the generated response to the user, Transmission means for transferring complex inquiries to an operator, Storage means for recording the history of inquiries and responses, Response support means including analysis of inquiry intent and automatic generation of responses using a knowledge base, Process automation means for automatically transferring complex problems to a human operator, Means for quickly resolving payment troubles by providing an immediate response to the user, A system including.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In customer support, conventionally, the mainstream approach was to rely on human operators, which led to problems such as soaring labor costs, difficulties in providing 24-hour support, and variations in support quality. Also, it was difficult to handle multi-language support and quickly resolve inquiries, posing challenges in maintaining customer satisfaction. Furthermore, there was a problem in that information regarding inquiry responses could not be efficiently accumulated and utilized, resulting in delays in knowledge sharing and system improvement.

Means for Solving the Problems

[0005] The present invention provides a customer support system comprising: a terminal means for receiving user inquiries using a chatbot; a processing means for analyzing inquiries using natural language processing; a response generation means for generating appropriate responses by referring to a knowledge base based on the analysis results; a response provision means for presenting the generated responses to the user; a transfer means for transferring complex inquiries to an operator; and a recording means for recording the history of inquiries and responses. This enables cost reduction, provision of a consistent customer experience, multilingual support, and system improvement through the utilization of historical data.

[0006] A "user" is the entity that makes inquiries to the system and seeks information about products and services.

[0007] An "inquiry" refers to a question or request made by a user to a system regarding information they want to know or a problem they want to solve.

[0008] A "terminal means" refers to a device or interface that has the function of receiving inquiries from users and, if necessary, sending them to a server or other system components.

[0009] "Natural language processing" is a technology that analyzes user input and understands its meaning, involving the interpretation of context and the extraction of keywords.

[0010] The "processing mechanism" is the part that receives an inquiry, performs analysis, extracts the necessary information, and passes it on to the next processing stage.

[0011] A "knowledge base" is a database that compiles past inquiries and their answers, or information within a specific scope, and serves as a reference source for generating answers.

[0012] The "response generation means" is a process that extracts appropriate information from a knowledge base based on the analysis results obtained by the processing means and constructs an answer for the user.

[0013] A "response provision means" is a means of displaying or communicating the generated response to the user, and generally, a chatbot UI is used.

[0014] A "transfer method" is a function that allows the system to transfer complex inquiries that it cannot handle to an operator.

[0015] "Recording means" refers to a function that stores inquiries and their responses in a database and manages the history for later analysis and system improvement. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), etc.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention relates to a customer support system using generative AI and RPA, and aims to build a system that efficiently processes user inquiries. This system begins when a user makes an inquiry to customer support using a terminal.

[0038] When a user enters a query through a terminal, the terminal receives the query and sends it to the server as text data. The server then uses natural language processing (NLP) techniques to analyze the query. The analyzed data is used as input to generate the optimal response from the knowledge base, based on the user's intent and the information requested.

[0039] The server consults the knowledge base and generates an appropriate answer based on the analysis results. The generated answer is then sent back to the terminal and displayed to the user. This allows the user to get a quick response to their inquiry.

[0040] On the other hand, if the server encounters a complex inquiry that cannot be handled by its generation AI or knowledge base, it will escalate the inquiry to an operator using a transfer mechanism. In this case, the server provides the operator with detailed information about the inquiry and its history to support a smooth handover.

[0041] Furthermore, the server records all queries and their responses in a database. This recorded data is used for later analysis, knowledge base updates, and new employee training. This allows for the accumulation of query-related information, contributing to future system improvements and increased efficiency.

[0042] As a concrete example, consider the following support scenario: If a user requests to reset their account password, the server uses natural language processing to retrieve information about password resets from the knowledge base. This allows the system to provide the user with detailed instructions on how to reset their password. On the other hand, if the inquiry concerns details of an unexpected software error, the analysis detects the complexity and automatically transfers the request to a human operator for expert assistance.

[0043] The following describes the processing flow.

[0044] Step 1:

[0045] The user enters an inquiry into the chatbot UI via their device. The device receives this input and sends it to the server as text data.

[0046] Step 2:

[0047] The server passes the received query data to a natural language processing engine for analysis. Specifically, it identifies the user's intent and extracts the main keywords and related information.

[0048] Step 3:

[0049] The server uses the analysis results to execute a search query against the knowledge base, thereby retrieving relevant answer information.

[0050] Step 4:

[0051] The server generates a user-appropriate answer based on information retrieved from the knowledge base. The answer is structured and adapted to a format that is easy for the user to understand.

[0052] Step 5:

[0053] The server sends the generated response to the device. The device then displays that response to the user on the chatbot UI.

[0054] Step 6:

[0055] The user reviews their answers and enters additional questions if necessary. The terminal receives this input again and repeats the process from step 2.

[0056] Step 7:

[0057] If the server determines that an inquiry is complex or difficult to handle using its knowledge base, it will transfer the inquiry to a human operator. The server will then provide the operator with detailed information and a history of the inquiry.

[0058] Step 8:

[0059] The server records all queries and responses in a database. This accumulates data that can be used for historical reference and future improvements.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] Many current customer support systems face challenges in efficiently and quickly processing user inquiries. Furthermore, language diversity and the complexity of inquiries can impact the speed and quality of support. Additionally, the utilization of inquiry history and the automated updating and optimization of information bases are often insufficient. These challenges need to be overcome to improve the user experience.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes receiving means for receiving user inquiries via a digital device, analyzing means for analyzing inquiries using natural language processing technology, and generating means for referencing an information base based on the analysis results and generating an optimal response. This enables rapid multilingual support and effective forwarding of complex inquiries. Furthermore, by automatically improving and updating the information base based on historical data stored in an information recording device, it becomes possible to optimize responses to future inquiries.

[0065] "Receiving means" refers to a device or process that has the function of acquiring user inquiries via a digital device.

[0066] "Analysis means" refers to a technology or process for understanding and breaking down acquired queries using natural language processing techniques to identify the user's intent.

[0067] A "generation means" is a device or system that has the function of constructing an optimal response based on analyzed information and by referring to an information base.

[0068] "Presentation means" refers to technology that has the function of displaying or providing information through a digital device in order to convey the generated response to the user in an appropriate format.

[0069] A "transfer method" refers to a process or system for routing complex inquiries that are difficult to process automatically to the appropriate person.

[0070] "Recording means" refers to a technology or apparatus for systematically storing inquiries and their responses in an information recording device.

[0071] "Update method" refers to a technology or technique for automatically improving and updating an information base based on stored historical data to enhance the quality of the service.

[0072] This invention begins with a user making an inquiry to customer support using a digital device. The user enters their inquiry, for example, using a chat window on a web browser or a mobile application. This entered text data is transmitted to a server via the internet through the terminal's communication protocol. The server implements a secure protocol such as SSL / TLS to receive this data.

[0073] The server analyzes incoming queries using software specialized for natural language processing, such as generative AI models. This analysis aims to understand the user's intentions or the information they are seeking. Techniques such as tokenization and intent recognition are used for this purpose. The analysis results are then compared with data stored in the information base to derive the optimal solution.

[0074] Next, the server uses the analysis results to select appropriate answers from the knowledge base and then uses a generative AI model to format them into a format suitable for user understanding. This process utilizes a pre-trained natural language model and is provided in a way that users can easily and quickly understand.

[0075] For example, if a user requests to reset their account password, the server retrieves information related to password resets from its knowledge base and provides instructions on how to reset it. The prompt might look something like, "The user is requesting a password reset. Please provide detailed reset instructions."

[0076] Furthermore, if an inquiry is complex and the server determines that an automated response is insufficient, it will be forwarded to a human resource representative. The server will then provide the representative with details and history of the inquiry, helping to ensure that the user receives appropriate support.

[0077] Furthermore, this system stores all inquiries and responses in an information recording device, retaining them as structured data for later analysis and automated improvement of the information base. This achieves increased efficiency and improved quality in support.

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] The user enters their inquiry through a digital device. The entered text is formatted as data within the terminal and sent to the server via a communication protocol. The input is the user's question, and the output is the text data sent to the server. Specifically, the user types their message into the chat window and presses the send button.

[0081] Step 2:

[0082] The server receives text data sent from the terminal. It then analyzes the received data using natural language processing techniques. A generative AI model is used for analysis, extracting intent and identifying keywords. The input is the text data of the submitted query, and the output is the analysis result. Specifically, the server divides the data into tokens and processes them using an intent recognition algorithm.

[0083] Step 3:

[0084] The server searches for relevant answers in its information base based on the analysis results. It selects the most appropriate answer based on the data in the information base. The input is the analysis results, and the output is the selected answer information. Specifically, the server executes database queries to retrieve relevant answers.

[0085] Step 4:

[0086] The server uses an AI model to generate the acquired response information in a user-friendly format. Based on this prompt, text generation takes place, creating the final response. The input is the selected response information, and the output is formatted text for presentation to the user. Specifically, the server uses the AI ​​model to express the response in natural language.

[0087] Step 5:

[0088] The server sends the generated response to the terminal. The terminal displays the received text data and presents it to the user. The input is the formatted response text, and the output is the answer displayed on the user's terminal. Specifically, the terminal displays the answer as a pop-up on the screen.

[0089] Step 6:

[0090] The server records queries and their responses in a database. This record is used for later analysis and updating the information base. The input is a query-response pair, and the output is stored as information accumulated in the database. Specifically, the server builds a dataset and stores it in storage.

[0091] Step 7:

[0092] The server assesses the complexity of the inquiry and forwards it to the appropriate person as needed. The input is the analysis result, and the output is a forwarding instruction to the appropriate person. Specifically, the server notifies the appropriate person based on the forwarding protocol.

[0093] (Application Example 1)

[0094] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0095] With the increasing use of electronic payment services in modern society, there has been a significant rise in user inquiries. In particular, there is a demand for prompt and accurate support regarding payment problems and technical issues. However, conventional customer support systems struggle to analyze inquiries and respond quickly, and complex inquiries, in particular, rely on human resources, leading to delays. Therefore, there is a need for a new system that can efficiently process user inquiries and enable rapid responses in electronic payment services.

[0096] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0097] In this invention, the server includes an input means for receiving inquiries from users, an information processing means for analyzing inquiries using natural language processing, and an answer generation means for referencing a knowledge base based on the analysis results and generating an appropriate response. This enables a rapid and accurate response to inquiries by providing immediate answers to payment-related problems faced by users and, if necessary, automatically transferring complex issues to human operators.

[0098] An "input method" is an interface for receiving inquiries from users.

[0099] "Information processing means" refers to a mechanism for analyzing user inquiries using natural language processing technology.

[0100] A "response generation means" is a device that generates an appropriate response for the user based on analysis results and by referring to a knowledge base.

[0101] "Response provision means" refers to an output device or interface for presenting the generated response to the user.

[0102] "Communication methods" refer to communication functions and processes for forwarding complex inquiries to human operators.

[0103] "Storage means" refers to a database or storage that records the history of inquiries and responses and stores them for later reference and analysis.

[0104] A "response support device" is a device that analyzes the intent of an inquiry and provides support functions that automatically generate a response using a knowledge base.

[0105] "Process automation means" refers to procedures and technologies for automatically escalating complex problems to human operators.

[0106] This invention is a system for responding quickly and accurately to user inquiries. The system consists of the following main elements:

[0107] The server first receives user inquiries through an input method. This input method often utilizes the user interface of a smartphone or computer. The received inquiries are then processed by information processing tools using natural language processing techniques to analyze the user's intent and question content. In this process, generative AI models such as OpenAI's GPT-3 (registered trademark) are used to perform text-based intent analysis.

[0108] Next, the server uses a response generation mechanism to refer to a knowledge base based on the analysis results and generate an appropriate response. The knowledge base includes past inquiry history and specialized information, enabling it to provide precise solutions to users. The generated response is immediately presented to the user through the response delivery mechanism.

[0109] If the server determines that an inquiry is complex, it is automatically forwarded to a human operator via a communication system. This process is streamlined using RPA (Robotic Process Automation) technology. All inquiry and response history is recorded in a database by a storage system, which is used for future reference and knowledge base updates.

[0110] For example, when a user inquires about a payment error, the server can analyze the payment details and error message to quickly identify the cause and provide a solution. An example of a prompt used in such a system is as follows:

[0111] "User inquiry: {User's question}. Analyze the intent and propose the best solution."

[0112] This invention provides users with a means to quickly resolve problems related to electronic payments.

[0113] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0114] Step 1:

[0115] The terminal receives user inquiries through an input device. This input is sent to the server in text format. The server receives this text data and starts processing it as a trigger for initiating the inquiry.

[0116] Step 2:

[0117] The server processes the received text data using information processing tools and performs natural language processing using a generative AI model. Specifically, it analyzes the user's intent and inquiry content from the received text. This process uses the OpenAI GPT-3 model. The input is the user's inquiry, and the output is data containing the intent of the inquiry and the results of the analysis.

[0118] Step 3:

[0119] The server receives the analysis results from the information processing means and, based on that, uses the response generation means to refer to the knowledge base. The knowledge base records patterns and solutions learned from past queries. The server extracts the relevant information and generates an appropriate response. In this process, it utilizes prompt statements. The input for response generation is the analysis results, and the output is the text of the response provided to the user.

[0120] Step 4:

[0121] The generated response is returned to the user from the server through the response delivery mechanism. The user can view this response on their terminal and use it to resolve the problem and take the next action. The output contains information about a solution that is useful to the user.

[0122] Step 5:

[0123] If the server determines that a query is complex, it automatically forwards the query to a human operator using a communication method. In this phase, the server also sends the query history and analysis data. However, the user is notified in advance that operator confirmation is required.

[0124] Step 6:

[0125] Finally, the server records the history of all queries and responses in a storage system. This record is stored in a database and used for future reference and system improvement. The input is the data of all queries and response processes, and the output is the stored history for future analysis and improvement.

[0126] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0127] This invention relates to a customer support system that generates and provides responses to user inquiries using natural language processing and an emotion engine. This system recognizes the user's emotional state and reflects it in the response, thereby achieving more appropriate and effective customer support.

[0128] When a user enters an inquiry into the chatbot UI via their device, the device receives the inquiry. This input data is sent to a server, where it is analyzed by a natural language processing engine. Here, along with analyzing the content of the inquiry, an emotion engine determines the user's emotions. For example, it identifies whether the user is angry or sad based on the expression and wording of the entered text.

[0129] The server integrates analysis results obtained through natural language processing and sentiment data from the sentiment engine, and generates the optimal response by referring to a knowledge base. In this process, for example, if the user is expressing dissatisfaction, the tone of the response is softened and an apology is added, making adjustments according to the user's emotions.

[0130] The generated response is sent to the device and presented to the user. This allows the user to receive a response that reflects their emotions, not just information, thus improving the quality of the customer support experience.

[0131] Furthermore, the server records all interactions, including not only the content of inquiries but also emotional data. This allows for future analysis of consultations and improvement of response policies. It also has an emotional prioritization function, designed to automatically transfer urgent inquiries to operators.

[0132] For example, if a user makes an inquiry such as, "I'm very worried because my order hasn't arrived," the server recognizes the user's distress and provides an appropriate response such as, "We apologize for your inconvenience. We will investigate this matter immediately." In this way, efficient support that takes the user's feelings into consideration is achieved.

[0133] The following describes the processing flow.

[0134] Step 1:

[0135] The user enters an inquiry into the chatbot UI via their device. The device receives this input as text data and immediately sends it to the server.

[0136] Step 2:

[0137] The server passes the received text data to the natural language processing engine. The natural language processing engine analyzes the content of the query and identifies the user's intent. At the same time, it extracts necessary information based on keywords and context.

[0138] Step 3:

[0139] The server uses an emotion engine to analyze the emotions contained in the user's input. The emotion engine identifies the user's emotional state from their word choice and tone of voice, and quantifies and evaluates emotions such as joy, anger, and sadness.

[0140] Step 4:

[0141] The server integrates the analysis results obtained from natural language processing with the sentiment evaluation from the sentiment engine. Based on this information, it queries the knowledge base and retrieves the relevant response data.

[0142] Step 5:

[0143] The server generates a response to the user based on the response data it has acquired. During this process, the tone and content of the response are adjusted based on the sentiment evaluation. For example, if the sentiment score is high, the response may include gentle language or an apology.

[0144] Step 6:

[0145] The server sends the generated response to the terminal. The terminal displays the response to the user, who then reviews its contents.

[0146] Step 7:

[0147] If the user confirms the response and makes another inquiry, the terminal receives that input again and repeats the process from step 2.

[0148] Step 8:

[0149] The server records all inquiries, responses, and sentiment data in a database. This allows for analysis of inquiry history and sentiment trends, which can be used to improve the system in the future and optimize customer support strategies.

[0150] (Example 2)

[0151] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0152] Modern customer service systems require efficient methods for responding quickly and appropriately to user inquiries. However, traditional systems often fail to provide responses that consider the user's emotional state, resulting in a decline in service quality. Furthermore, they lack sufficient mechanisms for handling complex inquiries and effectively utilizing inquiry history. Therefore, there is a need to realize a customer support system that can respond efficiently and flexibly while considering the user's emotions.

[0153] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0154] In this invention, the server includes means for analyzing queries using natural language processing technology, means for generating an optimal response by referring to knowledge information based on the analysis results and sentiment data obtained through sentiment recognition, and means for displaying and providing the generated response. This makes it possible to quickly generate responses that reflect the user's emotions and improve the quality of service.

[0155] A "user" refers to an individual or legal entity that makes an inquiry using the system.

[0156] A "terminal" refers to a device used by users to input and submit inquiries.

[0157] "Natural language processing technology" refers to methods for computers to understand and analyze human language input.

[0158] "Emotion recognition" refers to analytical techniques used to identify a user's emotional state based on their input.

[0159] "Knowledge information" refers to an information database that is referenced to generate appropriate responses to queries.

[0160] "Response" refers to the reply that the system generates and provides to the user in response to an analyzed query.

[0161] An "operator" refers to a person who receives and handles urgent inquiries that are forwarded by the system.

[0162] "History information" refers to data that records and stores user inquiries and responses from the system.

[0163] The customer support system according to the present invention provides efficient customer service that enables flexible responses that take into account the user's emotions. This system mainly consists of a terminal that receives inquiries from users, a server that analyzes the content of inquiries, and means for generating and providing the optimal response.

[0164] In the system's operation, the user inputs and sends a query through a terminal. The terminal sends this data to the server. The server processes the input text using natural language processing technology (e.g., spaCy and NLTK) for analysis, extracting keywords and intent, and uses sentiment recognition technology (e.g., IBM Watson® Tone Analyzer) to determine the user's emotional state. This allows the server to understand the emotional nuances contained in the query and utilize this information to generate an appropriate response.

[0165] Specifically, for example, using a generative AI model (e.g., OpenAI GPT), the server references pre-built knowledge information, forms prompts based on the results of natural language processing and sentiment recognition, and uses these to generate responses. The responses are adjusted according to the sentiment entered by the user. For example, in response to an inquiry such as "I am in a lot of trouble," the server provides a flexible response such as "I apologize for your trouble. We will investigate immediately."

[0166] Furthermore, the system records all inquiry and response history information and retains it for future improvements. This history information is also used for analysis and automatic updates of knowledge, serving as foundational data to improve the overall accuracy of the system's responses. For inquiries deemed urgent, the server automatically transfers them to an operator, enabling prompt human response.

[0167] A concrete example of a prompt might be the instruction, "If the user is in great distress, what kind of empathetic response would be appropriate?" This would enable the system to automatically generate appropriate, emotion-based responses, thereby improving the user experience.

[0168] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0169] Step 1:

[0170] The user enters their inquiry into the device. Specifically, the user directly types text into the chat interface and presses the send button. The data entered at this point is a text message containing the user's questions or requests.

[0171] Step 2:

[0172] The terminal sends the inquiry data to the server. Upon receiving user input, the terminal converts the text data into JSON format and securely sends it to the server using HTTPS. This prepares the input data for processing on the server side.

[0173] Step 3:

[0174] The server analyzes the query using a natural language processing engine. The server parses the received JSON data and extracts the necessary information using natural language processing tools such as "spaCy" or "NLTK". Specifically, it extracts keywords from the text and identifies the user's intent. The output of this step is the analyzed intent and related keywords.

[0175] Step 4:

[0176] The server uses an emotion analysis engine to determine the user's emotions. Specifically, it uses tools such as "IBM Watson Tone Analyzer" to analyze the tone of the inquiry and identify the emotional state. For example, if the phrase "I'm troubled" is included, it will be determined that the user is confused. This analysis result is output as emotion data.

[0177] Step 5:

[0178] The server generates responses using a generative AI model. It integrates analyzed intent and sentiment data and references a knowledge base to create prompts that form the optimal response. Based on these prompts, it uses a generative AI model (e.g., OpenAI GPT) to generate appropriate responses. The generated text is the output of this step.

[0179] Step 6:

[0180] The server sends the generated response to the terminal. The server then converts the response text back into JSON format and sends it to the terminal via HTTPS. This ensures that an optimized response is delivered to the user's terminal.

[0181] Step 7:

[0182] The terminal displays a response to the user. The terminal analyzes the received response and converts it into a user-friendly format for display in the user interface. This allows the user to confirm the appropriate answer to their inquiry.

[0183] Step 8:

[0184] The server records the history of inquiries and responses in a database. The server meticulously records all interactions to aid in future analysis and system improvements. This history also includes user sentiment data.

[0185] (Application Example 2)

[0186] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0187] The current customer support system has a problem in that it is difficult to properly analyze the emotions of users when handling inquiries and adjust the tone of responses accordingly. As a result, the user experience may be insufficient, potentially leading to decreased satisfaction. Another challenge is the lack of means to quickly transfer urgent inquiries to the appropriate person.

[0188] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0189] In this invention, the server includes emotion analysis means for analyzing the user's emotional state, response adjustment means for adjusting the tone of the response based on the user's emotional state, and prioritization means for prioritizing queries based on emotion. This makes it possible to provide responses that take the user's emotions into consideration and to provide prompt and appropriate support.

[0190] "User emotional state" is a concept that refers to the type and intensity of emotions a user expresses when making an inquiry.

[0191] An "emotion analysis tool" is a system that determines and analyzes the type and intensity of emotions based on user input information.

[0192] A "response adjustment mechanism" is a system for appropriately adjusting the tone and content of a response statement based on the analyzed emotional state.

[0193] A "prioritization mechanism" is a system that takes into account the user's emotional state and determines the priority of processing based on the urgency of the inquiry.

[0194] A "transfer method" is a mechanism for transferring specific inquiries to a human representative as needed.

[0195] A "recording mechanism" is a system that stores all relevant information, including query and response data, in a database.

[0196] This system operates via the user's smartphone or computer terminal. First, the user enters their inquiry into the application's user interface. The terminal sends this data to the server. The server analyzes the user's emotional state using programming languages ​​such as Python and related libraries (e.g., TextBlob, transformers). In doing so, it utilizes natural language processing techniques to understand the structure of the user's input language.

[0197] Once the sentiment analysis is complete, the response tone is adjusted based on the results. The response adjustment mechanism selects a template corresponding to a specific emotion and generates a response message. If the prioritization mechanism determines that the response is urgent, it is quickly forwarded to the appropriate person. For normal inquiries, the response is sent to the terminal and presented to the user.

[0198] All inquiry data, response data, and sentiment analysis results are stored in a database via recording devices. This allows for future quality improvements and automated updates of the knowledge base. For example, if a user makes an inquiry such as "I'm worried because my product hasn't arrived," the server detects the emotion "worry" and provides a response such as "We apologize for the inconvenience; we will check the situation immediately."

[0199] An example of a prompt for the generating AI model is, "When a user inquires, 'My item hasn't arrived yet,' please provide an appropriate, emotionally sensitive response." Based on this example, it becomes possible to provide appropriate customer support that is sensitive to the user's emotions.

[0200] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0201] Step 1:

[0202] The user enters their inquiry into the customer support application's user interface using their device. The entered inquiry data is then sent to the server.

[0203] Step 2:

[0204] The server passes the received query data to a natural language processing engine, which analyzes the text structure. This process extracts grammatical elements and keywords from the input data to understand the query's content. The output is the analyzed text data.

[0205] Step 3:

[0206] The server sends the analyzed text data to the sentiment analysis system. The sentiment analysis system uses TextBlob or a similar library to determine the user's emotional state. It calculates the type (e.g., anger, relief) and intensity of the emotion from the input and outputs an emotion score.

[0207] Step 4:

[0208] Based on the sentiment score, the server selects an appropriate response template using a prompt message through a response adjustment mechanism. This prompt message allows the generative AI model to generate the optimal response. As a result, a customized response is output that is tailored to the analyzed sentiment and content.

[0209] Step 5:

[0210] The generated response is evaluated for urgency using a prioritization mechanism. Based on the sentiment analysis results, if the inquiry is deemed highly urgent, it is automatically forwarded to the appropriate person. The selection of the forwarding destination is then output.

[0211] Step 6:

[0212] After a response is generated and forwarded to the appropriate person as needed, the server saves all relevant data (inquiry, response, sentiment score) to a database using recording mechanisms. This completes the database for future improvements and reference. The output of this step is a record that can be used for future data analysis.

[0213] Step 7:

[0214] The terminal receives the generated response and presents it to the user through the user interface. This allows the user to receive emotionally appropriate and optimal support. The output is a response message that the user can view.

[0215] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0216] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0217] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0218] [Second Embodiment]

[0219] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0220] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0221] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0222] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0223] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0224] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0225] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0226] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0227] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0228] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0229] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0230] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0231] This invention relates to a customer support system using generative AI and RPA, and aims to build a system that efficiently processes user inquiries. This system begins when a user makes an inquiry to customer support using a terminal.

[0232] When a user enters a query through a terminal, the terminal receives the query and sends it to the server as text data. The server then uses natural language processing (NLP) techniques to analyze the query. The analyzed data is used as input to generate the optimal response from the knowledge base, based on the user's intent and the information requested.

[0233] The server consults the knowledge base and generates an appropriate answer based on the analysis results. The generated answer is then sent back to the terminal and displayed to the user. This allows the user to get a quick response to their inquiry.

[0234] On the other hand, if the server encounters a complex inquiry that cannot be handled by its generation AI or knowledge base, it will escalate the inquiry to an operator using a transfer mechanism. In this case, the server provides the operator with detailed information about the inquiry and its history to support a smooth handover.

[0235] Furthermore, the server records all queries and their responses in a database. This recorded data is used for later analysis, knowledge base updates, and new employee training. This allows for the accumulation of query-related information, contributing to future system improvements and increased efficiency.

[0236] As a concrete example, consider the following support scenario: If a user requests to reset their account password, the server uses natural language processing to retrieve information about password resets from the knowledge base. This allows the system to provide the user with detailed instructions on how to reset their password. On the other hand, if the inquiry concerns details of an unexpected software error, the analysis detects the complexity and automatically transfers the request to a human operator for expert assistance.

[0237] The following describes the processing flow.

[0238] Step 1:

[0239] The user enters an inquiry into the chatbot UI via their device. The device receives this input and sends it to the server as text data.

[0240] Step 2:

[0241] The server passes the received query data to a natural language processing engine for analysis. Specifically, it identifies the user's intent and extracts the main keywords and related information.

[0242] Step 3:

[0243] The server uses the analysis results to execute a search query against the knowledge base, thereby retrieving relevant answer information.

[0244] Step 4:

[0245] The server generates a user-appropriate answer based on information retrieved from the knowledge base. The answer is structured and adapted to a format that is easy for the user to understand.

[0246] Step 5:

[0247] The server sends the generated response to the device. The device then displays that response to the user on the chatbot UI.

[0248] Step 6:

[0249] The user reviews their answers and enters additional questions if necessary. The terminal receives this input again and repeats the process from step 2.

[0250] Step 7:

[0251] If the server determines that an inquiry is complex or difficult to handle using its knowledge base, it will transfer the inquiry to a human operator. The server will then provide the operator with detailed information and a history of the inquiry.

[0252] Step 8:

[0253] The server records all queries and responses in a database. This accumulates data that can be used for historical reference and future improvements.

[0254] (Example 1)

[0255] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0256] Many current customer support systems face challenges in efficiently and quickly processing user inquiries. Furthermore, language diversity and the complexity of inquiries can impact the speed and quality of support. Additionally, the utilization of inquiry history and the automated updating and optimization of information bases are often insufficient. These challenges need to be overcome to improve the user experience.

[0257] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0258] In this invention, the server includes receiving means for receiving user inquiries via a digital device, analyzing means for analyzing inquiries using natural language processing technology, and generating means for referencing an information base based on the analysis results and generating an optimal response. This enables rapid multilingual support and effective forwarding of complex inquiries. Furthermore, by automatically improving and updating the information base based on historical data stored in an information recording device, it becomes possible to optimize responses to future inquiries.

[0259] "Receiving means" refers to a device or process that has the function of acquiring user inquiries via a digital device.

[0260] "Analysis means" refers to a technology or process for understanding and breaking down acquired queries using natural language processing techniques to identify the user's intent.

[0261] A "generation means" is a device or system that has the function of constructing an optimal response based on analyzed information and by referring to an information base.

[0262] "Presentation means" refers to technology that has the function of displaying or providing information through a digital device in order to convey the generated response to the user in an appropriate format.

[0263] A "transfer method" refers to a process or system for routing complex inquiries that are difficult to process automatically to the appropriate person.

[0264] "Recording means" refers to a technology or apparatus for systematically storing inquiries and their responses in an information recording device.

[0265] "Update method" refers to a technology or technique for automatically improving and updating an information base based on stored historical data to enhance the quality of the service.

[0266] This invention begins with a user making an inquiry to customer support using a digital device. The user enters their inquiry, for example, using a chat window on a web browser or a mobile application. This entered text data is transmitted to a server via the internet through the terminal's communication protocol. The server implements a secure protocol such as SSL / TLS to receive this data.

[0267] The server analyzes incoming queries using software specialized for natural language processing, such as generative AI models. This analysis aims to understand the user's intentions or the information they are seeking. Techniques such as tokenization and intent recognition are used for this purpose. The analysis results are then compared with data stored in the information base to derive the optimal solution.

[0268] Next, the server uses the analysis results to select appropriate answers from the knowledge base and then uses a generative AI model to format them into a format suitable for user understanding. This process utilizes a pre-trained natural language model and is provided in a way that users can easily and quickly understand.

[0269] For example, if a user requests to reset their account password, the server retrieves information related to password resets from its knowledge base and provides instructions on how to reset it. The prompt might look something like, "The user is requesting a password reset. Please provide detailed reset instructions."

[0270] Furthermore, if an inquiry is complex and the server determines that an automated response is insufficient, it will be forwarded to a human resource representative. The server will then provide the representative with details and history of the inquiry, helping to ensure that the user receives appropriate support.

[0271] Furthermore, this system stores all inquiries and responses in an information recording device, retaining them as structured data for later analysis and automated improvement of the information base. This achieves increased efficiency and improved quality in support.

[0272] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0273] Step 1:

[0274] The user enters their inquiry through a digital device. The entered text is formatted as data within the terminal and sent to the server via a communication protocol. The input is the user's question, and the output is the text data sent to the server. Specifically, the user types their message into the chat window and presses the send button.

[0275] Step 2:

[0276] The server receives text data sent from the terminal. It then analyzes the received data using natural language processing techniques. A generative AI model is used for analysis, extracting intent and identifying keywords. The input is the text data of the submitted query, and the output is the analysis result. Specifically, the server divides the data into tokens and processes them using an intent recognition algorithm.

[0277] Step 3:

[0278] The server searches for relevant answers in its information base based on the analysis results. It selects the most appropriate answer based on the data in the information base. The input is the analysis results, and the output is the selected answer information. Specifically, the server executes database queries to retrieve relevant answers.

[0279] Step 4:

[0280] The server uses an AI model to generate the acquired response information in a user-friendly format. Based on this prompt, text generation takes place, creating the final response. The input is the selected response information, and the output is formatted text for presentation to the user. Specifically, the server uses the AI ​​model to express the response in natural language.

[0281] Step 5:

[0282] The server sends the generated response to the terminal. The terminal displays the received text data and presents it to the user. The input is the formatted response text, and the output is the answer displayed on the user terminal. As a specific operation, the terminal pops up and displays the answer content on the screen.

[0283] Step 6:

[0284] The server records the inquiry and its corresponding response in the database. This record is utilized for subsequent analysis and updating of the information base. The input is a pair of an inquiry and a response, and the output is stored as the information accumulated in the database. As a specific operation, the server constructs a dataset and stores it in the storage.

[0285] Step 7:

[0286] The server determines the complexity of the inquiry and transfers it to the responsible person if necessary. The input is the analysis result, and the output is the transfer instruction to the responsible person. As a specific operation, the server notifies the responsible person based on the transfer protocol.

[0287] (Application Example 1)

[0288] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0289] With the expansion of the use of electronic payment services in modern society, the increase in inquiries from users is remarkable. In particular, for payment troubles and technical problems, quick and accurate support is required. However, in the conventional customer support system, it is difficult to analyze inquiries and speed up responses. Especially for complex inquiries, since it depends on human resources, the delay in response is cited as an issue. Therefore, there is a need for a new system that can efficiently process inquiries from users in electronic payment services and enable quick responses.

[0290] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0291] In this invention, the server includes an input means for receiving inquiries from users, an information processing means for analyzing inquiries using natural language processing, and an answer generation means for referencing a knowledge base based on the analysis results and generating an appropriate response. This enables a rapid and accurate response to inquiries by providing immediate answers to payment-related problems faced by users and, if necessary, automatically transferring complex issues to human operators.

[0292] An "input method" is an interface for receiving inquiries from users.

[0293] "Information processing means" refers to a mechanism for analyzing user inquiries using natural language processing technology.

[0294] A "response generation means" is a device that generates an appropriate response for the user based on analysis results and by referring to a knowledge base.

[0295] "Response provision means" refers to an output device or interface for presenting the generated response to the user.

[0296] "Communication methods" refer to communication functions and processes for forwarding complex inquiries to human operators.

[0297] "Storage means" refers to a database or storage that records the history of inquiries and responses and stores them for later reference and analysis.

[0298] A "response support device" is a device that analyzes the intent of an inquiry and provides support functions that automatically generate a response using a knowledge base.

[0299] "Process automation means" refers to procedures and technologies for automatically escalating complex problems to human operators.

[0300] This invention is a system for responding quickly and accurately to user inquiries. The system consists of the following main elements:

[0301] The server first receives user inquiries through an input method. This input method often utilizes the user interface of a smartphone or computer. The received inquiries are then processed by information processing tools using natural language processing techniques to analyze the user's intent and question content. In this process, generative AI models such as OpenAI's GPT-3 are used to perform text-based intent analysis.

[0302] Next, the server uses a response generation mechanism to refer to a knowledge base based on the analysis results and generate an appropriate response. The knowledge base includes past inquiry history and specialized information, enabling it to provide precise solutions to users. The generated response is immediately presented to the user through the response delivery mechanism.

[0303] If the server determines that an inquiry is complex, it is automatically forwarded to a human operator via a communication system. This process is streamlined using RPA (Robotic Process Automation) technology. All inquiry and response history is recorded in a database by a storage system, which is used for future reference and knowledge base updates.

[0304] For example, when a user inquires about a payment error, the server can analyze the payment details and error message to quickly identify the cause and provide a solution. An example of a prompt used in such a system is as follows:

[0305] "User inquiry: {User's question}. Analyze the intent and propose the best solution."

[0306] According to this invention, users can obtain means to quickly solve problems related to electronic payment.

[0307] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0308] Step 1:

[0309] The terminal receives an inquiry from the user through the input means. This input is sent to the server in text format. The server receives this text data and starts processing as a trigger for the start of the inquiry.

[0310] Step 2:

[0311] The server processes the received text data by information processing means and performs natural language processing using the generated AI model. Specifically, it analyzes the user's intention and the content of the inquiry from the received text. The GPT-3 model of OpenAI is used for this processing. The input is the inquiry sentence from the user, and the output is the data of the intention of the inquiry and the analyzed result.

[0312] Step 3:

[0313] The server receives the analysis result from the information processing means and refers to the knowledge base using the answer generation means based on it. The knowledge base records patterns and solutions learned from past inquiries. The server extracts the corresponding information and generates an appropriate response. At this time, prompt sentences are utilized. The input for response generation is the analysis result, and the output is the text of the answer provided to the user.

[0314] Step 4:

[0315] The generated response is returned to the user from the server through the response delivery mechanism. The user can view this response on their terminal and use it to resolve the problem and take the next action. The output contains information about a solution that is useful to the user.

[0316] Step 5:

[0317] If the server determines that a query is complex, it automatically forwards the query to a human operator using a communication method. In this phase, the server also sends the query history and analysis data. However, the user is notified in advance that operator confirmation is required.

[0318] Step 6:

[0319] Finally, the server records the history of all queries and responses in a storage system. This record is stored in a database and used for future reference and system improvement. The input is the data of all queries and response processes, and the output is the stored history for future analysis and improvement.

[0320] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0321] This invention relates to a customer support system that generates and provides responses to user inquiries using natural language processing and an emotion engine. This system recognizes the user's emotional state and reflects it in the response, thereby achieving more appropriate and effective customer support.

[0322] When a user enters an inquiry into the chatbot UI via their device, the device receives the inquiry. This input data is sent to a server, where it is analyzed by a natural language processing engine. Here, along with analyzing the content of the inquiry, an emotion engine determines the user's emotions. For example, it identifies whether the user is angry or sad based on the expression and wording of the entered text.

[0323] The server integrates analysis results obtained through natural language processing and sentiment data from the sentiment engine, and generates the optimal response by referring to a knowledge base. In this process, for example, if the user is expressing dissatisfaction, the tone of the response is softened and an apology is added, making adjustments according to the user's emotions.

[0324] The generated response is sent to the device and presented to the user. This allows the user to receive a response that reflects their emotions, not just information, thus improving the quality of the customer support experience.

[0325] Furthermore, the server records all interactions, including not only the content of inquiries but also emotional data. This allows for future analysis of consultations and improvement of response policies. It also has an emotional prioritization function, designed to automatically transfer urgent inquiries to operators.

[0326] For example, if a user makes an inquiry such as, "I'm very worried because my order hasn't arrived," the server recognizes the user's distress and provides an appropriate response such as, "We apologize for your inconvenience. We will investigate this matter immediately." In this way, efficient support that takes the user's feelings into consideration is achieved.

[0327] The following describes the processing flow.

[0328] Step 1:

[0329] The user enters an inquiry into the chatbot UI via their device. The device receives this input as text data and immediately sends it to the server.

[0330] Step 2:

[0331] The server passes the received text data to the natural language processing engine. The natural language processing engine analyzes the content of the query and identifies the user's intent. At the same time, it extracts necessary information based on keywords and context.

[0332] Step 3:

[0333] The server uses an emotion engine to analyze the emotions contained in the user's input. The emotion engine identifies the user's emotional state from their word choice and tone of voice, and quantifies and evaluates emotions such as joy, anger, and sadness.

[0334] Step 4:

[0335] The server integrates the analysis results obtained from natural language processing with the sentiment evaluation from the sentiment engine. Based on this information, it queries the knowledge base and retrieves the relevant response data.

[0336] Step 5:

[0337] The server generates a response to the user based on the response data it has acquired. During this process, the tone and content of the response are adjusted based on the sentiment evaluation. For example, if the sentiment score is high, the response may include gentle language or an apology.

[0338] Step 6:

[0339] The server sends the generated response to the terminal. The terminal displays the response to the user, who then reviews its contents.

[0340] Step 7:

[0341] If the user confirms the response and makes another inquiry, the terminal receives that input again and repeats the process from step 2.

[0342] Step 8:

[0343] The server records all inquiries, responses, and sentiment data in a database. This allows for analysis of inquiry history and sentiment trends, which can be used to improve the system in the future and optimize customer support strategies.

[0344] (Example 2)

[0345] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0346] Modern customer service systems require efficient methods for responding quickly and appropriately to user inquiries. However, traditional systems often fail to provide responses that consider the user's emotional state, resulting in a decline in service quality. Furthermore, they lack sufficient mechanisms for handling complex inquiries and effectively utilizing inquiry history. Therefore, there is a need to realize a customer support system that can respond efficiently and flexibly while considering the user's emotions.

[0347] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0348] In this invention, the server includes means for analyzing queries using natural language processing technology, means for generating an optimal response by referring to knowledge information based on the analysis results and sentiment data obtained through sentiment recognition, and means for displaying and providing the generated response. This makes it possible to quickly generate responses that reflect the user's emotions and improve the quality of service.

[0349] A "user" refers to an individual or legal entity that makes an inquiry using the system.

[0350] A "terminal" refers to a device used by users to input and submit inquiries.

[0351] "Natural language processing technology" refers to methods for computers to understand and analyze human language input.

[0352] "Emotion recognition" refers to analytical techniques used to identify a user's emotional state based on their input.

[0353] "Knowledge information" refers to an information database that is referenced to generate appropriate responses to queries.

[0354] "Response" refers to the reply that the system generates and provides to the user in response to an analyzed query.

[0355] An "operator" refers to a person who receives and handles urgent inquiries that are forwarded by the system.

[0356] "History information" refers to data that records and stores user inquiries and responses from the system.

[0357] The customer support system according to the present invention provides efficient customer service that enables flexible responses that take into account the user's emotions. This system mainly consists of a terminal that receives inquiries from users, a server that analyzes the content of inquiries, and means for generating and providing the optimal response.

[0358] In the system's operation, the user inputs and sends a query through a terminal. The terminal sends this data to the server. The server processes the input text using natural language processing technology (e.g., spaCy or NLTK) to extract keywords and intent, and uses sentiment recognition technology (e.g., IBM Watson Tone Analyzer) to determine the user's emotional state. This allows the server to understand the emotional nuances contained in the query and utilize this information to generate an appropriate response.

[0359] Specifically, for example, using a generative AI model (e.g., OpenAI GPT), the server references pre-built knowledge information, forms prompts based on the results of natural language processing and sentiment recognition, and uses these to generate responses. The responses are adjusted according to the sentiment entered by the user. For example, in response to an inquiry such as "I am in a lot of trouble," the server provides a flexible response such as "I apologize for your trouble. We will investigate immediately."

[0360] Furthermore, the system records all inquiry and response history information and retains it for future improvements. This history information is also used for analysis and automatic updates of knowledge, serving as foundational data to improve the overall accuracy of the system's responses. For inquiries deemed urgent, the server automatically transfers them to an operator, enabling prompt human response.

[0361] A concrete example of a prompt might be the instruction, "If the user is in great distress, what kind of empathetic response would be appropriate?" This would enable the system to automatically generate appropriate, emotion-based responses, thereby improving the user experience.

[0362] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0363] Step 1:

[0364] The user enters their inquiry into the device. Specifically, the user directly types text into the chat interface and presses the send button. The data entered at this point is a text message containing the user's questions or requests.

[0365] Step 2:

[0366] The terminal sends the inquiry data to the server. Upon receiving user input, the terminal converts the text data into JSON format and securely sends it to the server using HTTPS. This prepares the input data for processing on the server side.

[0367] Step 3:

[0368] The server analyzes the query using a natural language processing engine. The server parses the received JSON data and extracts the necessary information using natural language processing tools such as "spaCy" or "NLTK". Specifically, it extracts keywords from the text and identifies the user's intent. The output of this step is the analyzed intent and related keywords.

[0369] Step 4:

[0370] The server uses an emotion analysis engine to determine the user's emotions. Specifically, it uses tools such as "IBM Watson Tone Analyzer" to analyze the tone of the inquiry and identify the emotional state. For example, if the phrase "I'm troubled" is included, it will be determined that the user is confused. This analysis result is output as emotion data.

[0371] Step 5:

[0372] The server generates responses using a generative AI model. It integrates analyzed intent and sentiment data and references a knowledge base to create prompts that form the optimal response. Based on these prompts, it uses a generative AI model (e.g., OpenAI GPT) to generate appropriate responses. The generated text is the output of this step.

[0373] Step 6:

[0374] The server sends the generated response to the terminal. The server then converts the response text back into JSON format and sends it to the terminal via HTTPS. This ensures that an optimized response is delivered to the user's terminal.

[0375] Step 7:

[0376] The terminal displays a response to the user. The terminal analyzes the received response and converts it into a user-friendly format for display in the user interface. This allows the user to confirm the appropriate answer to their inquiry.

[0377] Step 8:

[0378] The server records the history of inquiries and responses in a database. The server meticulously records all interactions to aid in future analysis and system improvements. This history also includes user sentiment data.

[0379] (Application Example 2)

[0380] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0381] The current customer support system has a problem in that it is difficult to properly analyze the emotions of users when handling inquiries and adjust the tone of responses accordingly. As a result, the user experience may be insufficient, potentially leading to decreased satisfaction. Another challenge is the lack of means to quickly transfer urgent inquiries to the appropriate person.

[0382] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0383] In this invention, the server includes emotion analysis means for analyzing the user's emotional state, response adjustment means for adjusting the tone of the response based on the user's emotional state, and prioritization means for prioritizing queries based on emotion. This makes it possible to provide responses that take the user's emotions into consideration and to provide prompt and appropriate support.

[0384] "User emotional state" is a concept that refers to the type and intensity of emotions a user expresses when making an inquiry.

[0385] An "emotion analysis tool" is a system that determines and analyzes the type and intensity of emotions based on user input information.

[0386] A "response adjustment mechanism" is a system for appropriately adjusting the tone and content of a response statement based on the analyzed emotional state.

[0387] A "prioritization mechanism" is a system that takes into account the user's emotional state and determines the priority of processing based on the urgency of the inquiry.

[0388] A "transfer method" is a mechanism for transferring specific inquiries to a human representative as needed.

[0389] A "recording mechanism" is a system that stores all relevant information, including query and response data, in a database.

[0390] This system operates via the user's smartphone or computer terminal. First, the user enters their inquiry into the application's user interface. The terminal sends this data to the server. The server analyzes the user's emotional state using programming languages ​​such as Python and related libraries (e.g., TextBlob, transformers). In doing so, it utilizes natural language processing techniques to understand the structure of the user's input language.

[0391] Once the sentiment analysis is complete, the response tone is adjusted based on the results. The response adjustment mechanism selects a template corresponding to a specific emotion and generates a response message. If the prioritization mechanism determines that the response is urgent, it is quickly forwarded to the appropriate person. For normal inquiries, the response is sent to the terminal and presented to the user.

[0392] All inquiry data, response data, and sentiment analysis results are stored in a database via recording devices. This allows for future quality improvements and automated updates of the knowledge base. For example, if a user makes an inquiry such as "I'm worried because my product hasn't arrived," the server detects the emotion "worry" and provides a response such as "We apologize for the inconvenience; we will check the situation immediately."

[0393] An example of a prompt for the generating AI model is, "When a user inquires, 'My item hasn't arrived yet,' please provide an appropriate, emotionally sensitive response." Based on this example, it becomes possible to provide appropriate customer support that is sensitive to the user's emotions.

[0394] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0395] Step 1:

[0396] The user enters their inquiry into the customer support application's user interface using their device. The entered inquiry data is then sent to the server.

[0397] Step 2:

[0398] The server passes the received query data to a natural language processing engine, which analyzes the text structure. This process extracts grammatical elements and keywords from the input data to understand the query's content. The output is the analyzed text data.

[0399] Step 3:

[0400] The server sends the analyzed text data to the sentiment analysis system. The sentiment analysis system uses TextBlob or a similar library to determine the user's emotional state. It calculates the type (e.g., anger, relief) and intensity of the emotion from the input and outputs an emotion score.

[0401] Step 4:

[0402] Based on the sentiment score, the server selects an appropriate response template using a prompt message through a response adjustment mechanism. This prompt message allows the generative AI model to generate the optimal response. As a result, a customized response is output that is tailored to the analyzed sentiment and content.

[0403] Step 5:

[0404] The generated response is evaluated for urgency using a prioritization mechanism. Based on the sentiment analysis results, if the inquiry is deemed highly urgent, it is automatically forwarded to the appropriate person. The selection of the forwarding destination is then output.

[0405] Step 6:

[0406] After a response is generated and forwarded to the appropriate person as needed, the server saves all relevant data (inquiry, response, sentiment score) to a database using recording mechanisms. This completes the database for future improvements and reference. The output of this step is a record that can be used for future data analysis.

[0407] Step 7:

[0408] The terminal receives the generated response and presents it to the user through the user interface. This allows the user to receive emotionally appropriate and optimal support. The output is a response message that the user can view.

[0409] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0410] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0411] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0412] [Third Embodiment]

[0413] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0414] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0415] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0416] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0417] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0418] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0419] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0420] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0421] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0422] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0423] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0424] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0425] This invention relates to a customer support system using generative AI and RPA, and aims to build a system that efficiently processes user inquiries. This system begins when a user makes an inquiry to customer support using a terminal.

[0426] When a user enters a query through a terminal, the terminal receives the query and sends it to the server as text data. The server then uses natural language processing (NLP) techniques to analyze the query. The analyzed data is used as input to generate the optimal response from the knowledge base, based on the user's intent and the information requested.

[0427] The server consults the knowledge base and generates an appropriate answer based on the analysis results. The generated answer is then sent back to the terminal and displayed to the user. This allows the user to get a quick response to their inquiry.

[0428] On the other hand, if the server encounters a complex inquiry that cannot be handled by its generation AI or knowledge base, it will escalate the inquiry to an operator using a transfer mechanism. In this case, the server provides the operator with detailed information about the inquiry and its history to support a smooth handover.

[0429] Furthermore, the server records all queries and their responses in a database. This recorded data is used for later analysis, knowledge base updates, and new employee training. This allows for the accumulation of query-related information, contributing to future system improvements and increased efficiency.

[0430] As a concrete example, consider the following support scenario: If a user requests to reset their account password, the server uses natural language processing to retrieve information about password resets from the knowledge base. This allows the system to provide the user with detailed instructions on how to reset their password. On the other hand, if the inquiry concerns details of an unexpected software error, the analysis detects the complexity and automatically transfers the request to a human operator for expert assistance.

[0431] The following describes the processing flow.

[0432] Step 1:

[0433] The user enters an inquiry into the chatbot UI via their device. The device receives this input and sends it to the server as text data.

[0434] Step 2:

[0435] The server passes the received query data to a natural language processing engine for analysis. Specifically, it identifies the user's intent and extracts the main keywords and related information.

[0436] Step 3:

[0437] The server uses the analysis results to execute a search query against the knowledge base, thereby retrieving relevant answer information.

[0438] Step 4:

[0439] The server generates a user-appropriate answer based on information retrieved from the knowledge base. The answer is structured and adapted to a format that is easy for the user to understand.

[0440] Step 5:

[0441] The server sends the generated response to the device. The device then displays that response to the user on the chatbot UI.

[0442] Step 6:

[0443] The user reviews their answers and enters additional questions if necessary. The terminal receives this input again and repeats the process from step 2.

[0444] Step 7:

[0445] If the server determines that an inquiry is complex or difficult to handle using its knowledge base, it will transfer the inquiry to a human operator. The server will then provide the operator with detailed information and a history of the inquiry.

[0446] Step 8:

[0447] The server records all queries and responses in a database. This accumulates data that can be used for historical reference and future improvements.

[0448] (Example 1)

[0449] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0450] Many current customer support systems face challenges in efficiently and quickly processing user inquiries. Furthermore, language diversity and the complexity of inquiries can impact the speed and quality of support. Additionally, the utilization of inquiry history and the automated updating and optimization of information bases are often insufficient. These challenges need to be overcome to improve the user experience.

[0451] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0452] In this invention, the server includes receiving means for receiving user inquiries via a digital device, analyzing means for analyzing inquiries using natural language processing technology, and generating means for referencing an information base based on the analysis results and generating an optimal response. This enables rapid multilingual support and effective forwarding of complex inquiries. Furthermore, by automatically improving and updating the information base based on historical data stored in an information recording device, it becomes possible to optimize responses to future inquiries.

[0453] "Receiving means" refers to a device or process that has the function of acquiring user inquiries via a digital device.

[0454] "Analysis means" refers to a technology or process for understanding and breaking down acquired queries using natural language processing techniques to identify the user's intent.

[0455] A "generation means" is a device or system that has the function of constructing an optimal response based on analyzed information and by referring to an information base.

[0456] "Presentation means" refers to technology that has the function of displaying or providing information through a digital device in order to convey the generated response to the user in an appropriate format.

[0457] A "transfer method" refers to a process or system for routing complex inquiries that are difficult to process automatically to the appropriate person.

[0458] "Recording means" refers to a technology or apparatus for systematically storing inquiries and their responses in an information recording device.

[0459] "Update method" refers to a technology or technique for automatically improving and updating an information base based on stored historical data to enhance the quality of the service.

[0460] This invention begins with a user making an inquiry to customer support using a digital device. The user enters their inquiry, for example, using a chat window on a web browser or a mobile application. This entered text data is transmitted to a server via the internet through the terminal's communication protocol. The server implements a secure protocol such as SSL / TLS to receive this data.

[0461] The server analyzes incoming queries using software specialized for natural language processing, such as generative AI models. This analysis aims to understand the user's intentions or the information they are seeking. Techniques such as tokenization and intent recognition are used for this purpose. The analysis results are then compared with data stored in the information base to derive the optimal solution.

[0462] Next, the server uses the analysis results to select appropriate answers from the knowledge base and then uses a generative AI model to format them into a format suitable for user understanding. This process utilizes a pre-trained natural language model and is provided in a way that users can easily and quickly understand.

[0463] For example, if a user requests to reset their account password, the server retrieves information related to password resets from its knowledge base and provides instructions on how to reset it. The prompt might look something like, "The user is requesting a password reset. Please provide detailed reset instructions."

[0464] Furthermore, if an inquiry is complex and the server determines that an automated response is insufficient, it will be forwarded to a human resource representative. The server will then provide the representative with details and history of the inquiry, helping to ensure that the user receives appropriate support.

[0465] Furthermore, this system stores all inquiries and responses in an information recording device, retaining them as structured data for later analysis and automated improvement of the information base. This achieves increased efficiency and improved quality in support.

[0466] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0467] Step 1:

[0468] The user enters their inquiry through a digital device. The entered text is formatted as data within the terminal and sent to the server via a communication protocol. The input is the user's question, and the output is the text data sent to the server. Specifically, the user types their message into the chat window and presses the send button.

[0469] Step 2:

[0470] The server receives text data sent from the terminal. It then analyzes the received data using natural language processing techniques. A generative AI model is used for analysis, extracting intent and identifying keywords. The input is the text data of the submitted query, and the output is the analysis result. Specifically, the server divides the data into tokens and processes them using an intent recognition algorithm.

[0471] Step 3:

[0472] The server searches for relevant answers in its information base based on the analysis results. It selects the most appropriate answer based on the data in the information base. The input is the analysis results, and the output is the selected answer information. Specifically, the server executes database queries to retrieve relevant answers.

[0473] Step 4:

[0474] The server uses an AI model to generate the acquired response information in a user-friendly format. Based on this prompt, text generation takes place, creating the final response. The input is the selected response information, and the output is formatted text for presentation to the user. Specifically, the server uses the AI ​​model to express the response in natural language.

[0475] Step 5:

[0476] The server sends the generated response to the terminal. The terminal displays the received text data and presents it to the user. The input is the formatted response text, and the output is the answer displayed on the user's terminal. Specifically, the terminal displays the answer as a pop-up on the screen.

[0477] Step 6:

[0478] The server records queries and their responses in a database. This record is used for later analysis and updating the information base. The input is a query-response pair, and the output is stored as information accumulated in the database. Specifically, the server builds a dataset and stores it in storage.

[0479] Step 7:

[0480] The server assesses the complexity of the inquiry and forwards it to the appropriate person as needed. The input is the analysis result, and the output is a forwarding instruction to the appropriate person. Specifically, the server notifies the appropriate person based on the forwarding protocol.

[0481] (Application Example 1)

[0482] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0483] With the increasing use of electronic payment services in modern society, there has been a significant rise in user inquiries. In particular, there is a demand for prompt and accurate support regarding payment problems and technical issues. However, conventional customer support systems struggle to analyze inquiries and respond quickly, and complex inquiries, in particular, rely on human resources, leading to delays. Therefore, there is a need for a new system that can efficiently process user inquiries and enable rapid responses in electronic payment services.

[0484] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0485] In this invention, the server includes an input means for receiving inquiries from users, an information processing means for analyzing inquiries using natural language processing, and an answer generation means for referencing a knowledge base based on the analysis results and generating an appropriate response. This enables a rapid and accurate response to inquiries by providing immediate answers to payment-related problems faced by users and, if necessary, automatically transferring complex issues to human operators.

[0486] An "input method" is an interface for receiving inquiries from users.

[0487] "Information processing means" refers to a mechanism for analyzing user inquiries using natural language processing technology.

[0488] A "response generation means" is a device that generates an appropriate response for the user based on analysis results and by referring to a knowledge base.

[0489] "Response provision means" refers to an output device or interface for presenting the generated response to the user.

[0490] "Communication methods" refer to communication functions and processes for forwarding complex inquiries to human operators.

[0491] "Storage means" refers to a database or storage that records the history of inquiries and responses and stores them for later reference and analysis.

[0492] A "response support device" is a device that analyzes the intent of an inquiry and provides support functions that automatically generate a response using a knowledge base.

[0493] "Process automation means" refers to procedures and technologies for automatically escalating complex problems to human operators.

[0494] This invention is a system for responding quickly and accurately to user inquiries. The system consists of the following main elements:

[0495] The server first receives user inquiries through an input method. This input method often utilizes the user interface of a smartphone or computer. The received inquiries are then processed by information processing tools using natural language processing techniques to analyze the user's intent and question content. In this process, generative AI models such as OpenAI's GPT-3 are used to perform text-based intent analysis.

[0496] Next, the server uses a response generation mechanism to refer to a knowledge base based on the analysis results and generate an appropriate response. The knowledge base includes past inquiry history and specialized information, enabling it to provide precise solutions to users. The generated response is immediately presented to the user through the response delivery mechanism.

[0497] If the server determines that an inquiry is complex, it is automatically forwarded to a human operator via a communication system. This process is streamlined using RPA (Robotic Process Automation) technology. All inquiry and response history is recorded in a database by a storage system, which is used for future reference and knowledge base updates.

[0498] For example, when a user inquires about a payment error, the server can analyze the payment details and error message to quickly identify the cause and provide a solution. An example of a prompt used in such a system is as follows:

[0499] "User inquiry: {User's question}. Analyze the intent and propose the best solution."

[0500] This invention provides users with a means to quickly resolve problems related to electronic payments.

[0501] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0502] Step 1:

[0503] The terminal receives user inquiries through an input device. This input is sent to the server in text format. The server receives this text data and starts processing it as a trigger for initiating the inquiry.

[0504] Step 2:

[0505] The server processes the received text data using information processing tools and performs natural language processing using a generative AI model. Specifically, it analyzes the user's intent and inquiry content from the received text. This process uses the OpenAI GPT-3 model. The input is the user's inquiry, and the output is data containing the intent of the inquiry and the results of the analysis.

[0506] Step 3:

[0507] The server receives the analysis results from the information processing means and, based on that, uses the response generation means to refer to the knowledge base. The knowledge base records patterns and solutions learned from past queries. The server extracts the relevant information and generates an appropriate response. In this process, it utilizes prompt statements. The input for response generation is the analysis results, and the output is the text of the response provided to the user.

[0508] Step 4:

[0509] The generated response is returned to the user from the server through the response delivery mechanism. The user can view this response on their terminal and use it to resolve the problem and take the next action. The output contains information about a solution that is useful to the user.

[0510] Step 5:

[0511] If the server determines that a query is complex, it automatically forwards the query to a human operator using a communication method. In this phase, the server also sends the query history and analysis data. However, the user is notified in advance that operator confirmation is required.

[0512] Step 6:

[0513] Finally, the server records the history of all queries and responses in a storage system. This record is stored in a database and used for future reference and system improvement. The input is the data of all queries and response processes, and the output is the stored history for future analysis and improvement.

[0514] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0515] This invention relates to a customer support system that generates and provides responses to user inquiries using natural language processing and an emotion engine. This system recognizes the user's emotional state and reflects it in the response, thereby achieving more appropriate and effective customer support.

[0516] When a user enters an inquiry into the chatbot UI via their device, the device receives the inquiry. This input data is sent to a server, where it is analyzed by a natural language processing engine. Here, along with analyzing the content of the inquiry, an emotion engine determines the user's emotions. For example, it identifies whether the user is angry or sad based on the expression and wording of the entered text.

[0517] The server integrates analysis results obtained through natural language processing and sentiment data from the sentiment engine, and generates the optimal response by referring to a knowledge base. In this process, for example, if the user is expressing dissatisfaction, the tone of the response is softened and an apology is added, making adjustments according to the user's emotions.

[0518] The generated response is sent to the device and presented to the user. This allows the user to receive a response that reflects their emotions, not just information, thus improving the quality of the customer support experience.

[0519] Furthermore, the server records all interactions, including not only the content of inquiries but also emotional data. This allows for future analysis of consultations and improvement of response policies. It also has an emotional prioritization function, designed to automatically transfer urgent inquiries to operators.

[0520] For example, if a user makes an inquiry such as, "I'm very worried because my order hasn't arrived," the server recognizes the user's distress and provides an appropriate response such as, "We apologize for your inconvenience. We will investigate this matter immediately." In this way, efficient support that takes the user's feelings into consideration is achieved.

[0521] The following describes the processing flow.

[0522] Step 1:

[0523] The user enters an inquiry into the chatbot UI via their device. The device receives this input as text data and immediately sends it to the server.

[0524] Step 2:

[0525] The server passes the received text data to the natural language processing engine. The natural language processing engine analyzes the content of the query and identifies the user's intent. At the same time, it extracts necessary information based on keywords and context.

[0526] Step 3:

[0527] The server uses an emotion engine to analyze the emotions contained in the user's input. The emotion engine identifies the user's emotional state from their word choice and tone of voice, and quantifies and evaluates emotions such as joy, anger, and sadness.

[0528] Step 4:

[0529] The server integrates the analysis results obtained from natural language processing with the sentiment evaluation from the sentiment engine. Based on this information, it queries the knowledge base and retrieves the relevant response data.

[0530] Step 5:

[0531] The server generates a response to the user based on the response data it has acquired. During this process, the tone and content of the response are adjusted based on the sentiment evaluation. For example, if the sentiment score is high, the response may include gentle language or an apology.

[0532] Step 6:

[0533] The server sends the generated response to the terminal. The terminal displays the response to the user, who then reviews its contents.

[0534] Step 7:

[0535] If the user confirms the response and makes another inquiry, the terminal receives that input again and repeats the process from step 2.

[0536] Step 8:

[0537] The server records all inquiries, responses, and sentiment data in a database. This allows for analysis of inquiry history and sentiment trends, which can be used to improve the system in the future and optimize customer support strategies.

[0538] (Example 2)

[0539] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0540] Modern customer service systems require efficient methods for responding quickly and appropriately to user inquiries. However, traditional systems often fail to provide responses that consider the user's emotional state, resulting in a decline in service quality. Furthermore, they lack sufficient mechanisms for handling complex inquiries and effectively utilizing inquiry history. Therefore, there is a need to realize a customer support system that can respond efficiently and flexibly while considering the user's emotions.

[0541] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0542] In this invention, the server includes means for analyzing queries using natural language processing technology, means for generating an optimal response by referring to knowledge information based on the analysis results and sentiment data obtained through sentiment recognition, and means for displaying and providing the generated response. This makes it possible to quickly generate responses that reflect the user's emotions and improve the quality of service.

[0543] A "user" refers to an individual or legal entity that makes an inquiry using the system.

[0544] A "terminal" refers to a device used by users to input and submit inquiries.

[0545] "Natural language processing technology" refers to methods for computers to understand and analyze human language input.

[0546] "Emotion recognition" refers to analytical techniques used to identify a user's emotional state based on their input.

[0547] "Knowledge information" refers to an information database that is referenced to generate appropriate responses to queries.

[0548] "Response" refers to the reply that the system generates and provides to the user in response to an analyzed query.

[0549] An "operator" refers to a person who receives and handles urgent inquiries that are forwarded by the system.

[0550] "History information" refers to data that records and stores user inquiries and responses from the system.

[0551] The customer support system according to the present invention provides efficient customer service that enables flexible responses that take into account the user's emotions. This system mainly consists of a terminal that receives inquiries from users, a server that analyzes the content of inquiries, and means for generating and providing the optimal response.

[0552] In the system's operation, the user inputs and sends a query through a terminal. The terminal sends this data to the server. The server processes the input text using natural language processing technology (e.g., spaCy or NLTK) to extract keywords and intent, and uses sentiment recognition technology (e.g., IBM Watson Tone Analyzer) to determine the user's emotional state. This allows the server to understand the emotional nuances contained in the query and utilize this information to generate an appropriate response.

[0553] Specifically, for example, using a generative AI model (e.g., OpenAI GPT), the server references pre-built knowledge information, forms prompts based on the results of natural language processing and sentiment recognition, and uses these to generate responses. The responses are adjusted according to the sentiment entered by the user. For example, in response to an inquiry such as "I am in a lot of trouble," the server provides a flexible response such as "I apologize for your trouble. We will investigate immediately."

[0554] Furthermore, the system records all inquiry and response history information and retains it for future improvements. This history information is also used for analysis and automatic updates of knowledge, serving as foundational data to improve the overall accuracy of the system's responses. For inquiries deemed urgent, the server automatically transfers them to an operator, enabling prompt human response.

[0555] A concrete example of a prompt might be the instruction, "If the user is in great distress, what kind of empathetic response would be appropriate?" This would enable the system to automatically generate appropriate, emotion-based responses, thereby improving the user experience.

[0556] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0557] Step 1:

[0558] The user enters their inquiry into the device. Specifically, the user directly types text into the chat interface and presses the send button. The data entered at this point is a text message containing the user's questions or requests.

[0559] Step 2:

[0560] The terminal sends the inquiry data to the server. Upon receiving user input, the terminal converts the text data into JSON format and securely sends it to the server using HTTPS. This prepares the input data for processing on the server side.

[0561] Step 3:

[0562] The server analyzes the query using a natural language processing engine. The server parses the received JSON data and extracts the necessary information using natural language processing tools such as "spaCy" or "NLTK". Specifically, it extracts keywords from the text and identifies the user's intent. The output of this step is the analyzed intent and related keywords.

[0563] Step 4:

[0564] The server uses an emotion analysis engine to determine the user's emotions. Specifically, it uses tools such as "IBM Watson Tone Analyzer" to analyze the tone of the inquiry and identify the emotional state. For example, if the phrase "I'm troubled" is included, it will be determined that the user is confused. This analysis result is output as emotion data.

[0565] Step 5:

[0566] The server generates responses using a generative AI model. It integrates analyzed intent and sentiment data and references a knowledge base to create prompts that form the optimal response. Based on these prompts, it uses a generative AI model (e.g., OpenAI GPT) to generate appropriate responses. The generated text is the output of this step.

[0567] Step 6:

[0568] The server sends the generated response to the terminal. The server then converts the response text back into JSON format and sends it to the terminal via HTTPS. This ensures that an optimized response is delivered to the user's terminal.

[0569] Step 7:

[0570] The terminal displays a response to the user. The terminal analyzes the received response and converts it into a user-friendly format for display in the user interface. This allows the user to confirm the appropriate answer to their inquiry.

[0571] Step 8:

[0572] The server records the history of inquiries and responses in a database. The server meticulously records all interactions to aid in future analysis and system improvements. This history also includes user sentiment data.

[0573] (Application Example 2)

[0574] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0575] The current customer support system has a problem in that it is difficult to properly analyze the emotions of users when handling inquiries and adjust the tone of responses accordingly. As a result, the user experience may be insufficient, potentially leading to decreased satisfaction. Another challenge is the lack of means to quickly transfer urgent inquiries to the appropriate person.

[0576] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0577] In this invention, the server includes emotion analysis means for analyzing the user's emotional state, response adjustment means for adjusting the tone of the response based on the user's emotional state, and prioritization means for prioritizing queries based on emotion. This makes it possible to provide responses that take the user's emotions into consideration and to provide prompt and appropriate support.

[0578] "User emotional state" is a concept that refers to the type and intensity of emotions a user expresses when making an inquiry.

[0579] An "emotion analysis tool" is a system that determines and analyzes the type and intensity of emotions based on user input information.

[0580] A "response adjustment mechanism" is a system for appropriately adjusting the tone and content of a response statement based on the analyzed emotional state.

[0581] A "prioritization mechanism" is a system that takes into account the user's emotional state and determines the priority of processing based on the urgency of the inquiry.

[0582] A "transfer method" is a mechanism for transferring specific inquiries to a human representative as needed.

[0583] A "recording mechanism" is a system that stores all relevant information, including query and response data, in a database.

[0584] This system operates via the user's smartphone or computer terminal. First, the user enters their inquiry into the application's user interface. The terminal sends this data to the server. The server analyzes the user's emotional state using programming languages ​​such as Python and related libraries (e.g., TextBlob, transformers). In doing so, it utilizes natural language processing techniques to understand the structure of the user's input language.

[0585] Once the sentiment analysis is complete, the response tone is adjusted based on the results. The response adjustment mechanism selects a template corresponding to a specific emotion and generates a response message. If the prioritization mechanism determines that the response is urgent, it is quickly forwarded to the appropriate person. For normal inquiries, the response is sent to the terminal and presented to the user.

[0586] All inquiry data, response data, and sentiment analysis results are stored in a database via recording devices. This allows for future quality improvements and automated updates of the knowledge base. For example, if a user makes an inquiry such as "I'm worried because my product hasn't arrived," the server detects the emotion "worry" and provides a response such as "We apologize for the inconvenience; we will check the situation immediately."

[0587] An example of a prompt for the generating AI model is, "When a user inquires, 'My item hasn't arrived yet,' please provide an appropriate, emotionally sensitive response." Based on this example, it becomes possible to provide appropriate customer support that is sensitive to the user's emotions.

[0588] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0589] Step 1:

[0590] The user enters their inquiry into the customer support application's user interface using their device. The entered inquiry data is then sent to the server.

[0591] Step 2:

[0592] The server passes the received query data to a natural language processing engine, which analyzes the text structure. This process extracts grammatical elements and keywords from the input data to understand the query's content. The output is the analyzed text data.

[0593] Step 3:

[0594] The server sends the analyzed text data to the sentiment analysis system. The sentiment analysis system uses TextBlob or a similar library to determine the user's emotional state. It calculates the type (e.g., anger, relief) and intensity of the emotion from the input and outputs an emotion score.

[0595] Step 4:

[0596] Based on the sentiment score, the server selects an appropriate response template using a prompt message through a response adjustment mechanism. This prompt message allows the generative AI model to generate the optimal response. As a result, a customized response is output that is tailored to the analyzed sentiment and content.

[0597] Step 5:

[0598] The generated response is evaluated for urgency using a prioritization mechanism. Based on the sentiment analysis results, if the inquiry is deemed highly urgent, it is automatically forwarded to the appropriate person. The selection of the forwarding destination is then output.

[0599] Step 6:

[0600] After a response is generated and forwarded to the appropriate person as needed, the server saves all relevant data (inquiry, response, sentiment score) to a database using recording mechanisms. This completes the database for future improvements and reference. The output of this step is a record that can be used for future data analysis.

[0601] Step 7:

[0602] The terminal receives the generated response and presents it to the user through the user interface. This allows the user to receive emotionally appropriate and optimal support. The output is a response message that the user can view.

[0603] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0604] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0605] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0606] [Fourth Embodiment]

[0607] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0608] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0609] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0610] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0611] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0612] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0613] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0614] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0615] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0616] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0617] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0618] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0619] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0620] This invention relates to a customer support system using generative AI and RPA, and aims to build a system that efficiently processes user inquiries. This system begins when a user makes an inquiry to customer support using a terminal.

[0621] When a user enters a query through a terminal, the terminal receives the query and sends it to the server as text data. The server then uses natural language processing (NLP) techniques to analyze the query. The analyzed data is used as input to generate the optimal response from the knowledge base, based on the user's intent and the information requested.

[0622] The server consults the knowledge base and generates an appropriate answer based on the analysis results. The generated answer is then sent back to the terminal and displayed to the user. This allows the user to get a quick response to their inquiry.

[0623] On the other hand, if the server encounters a complex inquiry that cannot be handled by its generation AI or knowledge base, it will escalate the inquiry to an operator using a transfer mechanism. In this case, the server provides the operator with detailed information about the inquiry and its history to support a smooth handover.

[0624] Furthermore, the server records all queries and their responses in a database. This recorded data is used for later analysis, knowledge base updates, and new employee training. This allows for the accumulation of query-related information, contributing to future system improvements and increased efficiency.

[0625] As a concrete example, consider the following support scenario: If a user requests to reset their account password, the server uses natural language processing to retrieve information about password resets from the knowledge base. This allows the system to provide the user with detailed instructions on how to reset their password. On the other hand, if the inquiry concerns details of an unexpected software error, the analysis detects the complexity and automatically transfers the request to a human operator for expert assistance.

[0626] The following describes the processing flow.

[0627] Step 1:

[0628] The user enters an inquiry into the chatbot UI via their device. The device receives this input and sends it to the server as text data.

[0629] Step 2:

[0630] The server passes the received query data to a natural language processing engine for analysis. Specifically, it identifies the user's intent and extracts the main keywords and related information.

[0631] Step 3:

[0632] The server uses the analysis results to execute a search query against the knowledge base, thereby retrieving relevant answer information.

[0633] Step 4:

[0634] The server generates a user-appropriate answer based on information retrieved from the knowledge base. The answer is structured and adapted to a format that is easy for the user to understand.

[0635] Step 5:

[0636] The server sends the generated response to the device. The device then displays that response to the user on the chatbot UI.

[0637] Step 6:

[0638] The user reviews their answers and enters additional questions if necessary. The terminal receives this input again and repeats the process from step 2.

[0639] Step 7:

[0640] If the server determines that an inquiry is complex or difficult to handle using its knowledge base, it will transfer the inquiry to a human operator. The server will then provide the operator with detailed information and a history of the inquiry.

[0641] Step 8:

[0642] The server records all queries and responses in a database. This accumulates data that can be used for historical reference and future improvements.

[0643] (Example 1)

[0644] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0645] Many current customer support systems face challenges in efficiently and quickly processing user inquiries. Furthermore, language diversity and the complexity of inquiries can impact the speed and quality of support. Additionally, the utilization of inquiry history and the automated updating and optimization of information bases are often insufficient. These challenges need to be overcome to improve the user experience.

[0646] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0647] In this invention, the server includes receiving means for receiving user inquiries via a digital device, analyzing means for analyzing inquiries using natural language processing technology, and generating means for referencing an information base based on the analysis results and generating an optimal response. This enables rapid multilingual support and effective forwarding of complex inquiries. Furthermore, by automatically improving and updating the information base based on historical data stored in an information recording device, it becomes possible to optimize responses to future inquiries.

[0648] "Receiving means" refers to a device or process that has the function of acquiring user inquiries via a digital device.

[0649] "Analysis means" refers to a technology or process for understanding and breaking down acquired queries using natural language processing techniques to identify the user's intent.

[0650] A "generation means" is a device or system that has the function of constructing an optimal response based on analyzed information and by referring to an information base.

[0651] "Presentation means" refers to technology that has the function of displaying or providing information through a digital device in order to convey the generated response to the user in an appropriate format.

[0652] A "transfer method" refers to a process or system for routing complex inquiries that are difficult to process automatically to the appropriate person.

[0653] "Recording means" refers to a technology or apparatus for systematically storing inquiries and their responses in an information recording device.

[0654] "Update method" refers to a technology or technique for automatically improving and updating an information base based on stored historical data to enhance the quality of the service.

[0655] This invention begins with a user making an inquiry to customer support using a digital device. The user enters their inquiry, for example, using a chat window on a web browser or a mobile application. This entered text data is transmitted to a server via the internet through the terminal's communication protocol. The server implements a secure protocol such as SSL / TLS to receive this data.

[0656] The server analyzes incoming queries using software specialized for natural language processing, such as generative AI models. This analysis aims to understand the user's intentions or the information they are seeking. Techniques such as tokenization and intent recognition are used for this purpose. The analysis results are then compared with data stored in the information base to derive the optimal solution.

[0657] Next, the server uses the analysis results to select appropriate answers from the knowledge base and then uses a generative AI model to format them into a format suitable for user understanding. This process utilizes a pre-trained natural language model and is provided in a way that users can easily and quickly understand.

[0658] For example, if a user requests to reset their account password, the server retrieves information related to password resets from its knowledge base and provides instructions on how to reset it. The prompt might look something like, "The user is requesting a password reset. Please provide detailed reset instructions."

[0659] Furthermore, if an inquiry is complex and the server determines that an automated response is insufficient, it will be forwarded to a human resource representative. The server will then provide the representative with details and history of the inquiry, helping to ensure that the user receives appropriate support.

[0660] Furthermore, this system stores all inquiries and responses in an information recording device, retaining them as structured data for later analysis and automated improvement of the information base. This achieves increased efficiency and improved quality in support.

[0661] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0662] Step 1:

[0663] The user enters their inquiry through a digital device. The entered text is formatted as data within the terminal and sent to the server via a communication protocol. The input is the user's question, and the output is the text data sent to the server. Specifically, the user types their message into the chat window and presses the send button.

[0664] Step 2:

[0665] The server receives text data sent from the terminal. It then analyzes the received data using natural language processing techniques. A generative AI model is used for analysis, extracting intent and identifying keywords. The input is the text data of the submitted query, and the output is the analysis result. Specifically, the server divides the data into tokens and processes them using an intent recognition algorithm.

[0666] Step 3:

[0667] The server searches for relevant answers in its information base based on the analysis results. It selects the most appropriate answer based on the data in the information base. The input is the analysis results, and the output is the selected answer information. Specifically, the server executes database queries to retrieve relevant answers.

[0668] Step 4:

[0669] The server uses an AI model to generate the acquired response information in a user-friendly format. Based on this prompt, text generation takes place, creating the final response. The input is the selected response information, and the output is formatted text for presentation to the user. Specifically, the server uses the AI ​​model to express the response in natural language.

[0670] Step 5:

[0671] The server sends the generated response to the terminal. The terminal displays the received text data and presents it to the user. The input is the formatted response text, and the output is the answer displayed on the user's terminal. Specifically, the terminal displays the answer as a pop-up on the screen.

[0672] Step 6:

[0673] The server records queries and their responses in a database. This record is used for later analysis and updating the information base. The input is a query-response pair, and the output is stored as information accumulated in the database. Specifically, the server builds a dataset and stores it in storage.

[0674] Step 7:

[0675] The server assesses the complexity of the inquiry and forwards it to the appropriate person as needed. The input is the analysis result, and the output is a forwarding instruction to the appropriate person. Specifically, the server notifies the appropriate person based on the forwarding protocol.

[0676] (Application Example 1)

[0677] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0678] With the increasing use of electronic payment services in modern society, there has been a significant rise in user inquiries. In particular, there is a demand for prompt and accurate support regarding payment problems and technical issues. However, conventional customer support systems struggle to analyze inquiries and respond quickly, and complex inquiries, in particular, rely on human resources, leading to delays. Therefore, there is a need for a new system that can efficiently process user inquiries and enable rapid responses in electronic payment services.

[0679] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0680] In this invention, the server includes an input means for receiving inquiries from users, an information processing means for analyzing inquiries using natural language processing, and an answer generation means for referencing a knowledge base based on the analysis results and generating an appropriate response. This enables a rapid and accurate response to inquiries by providing immediate answers to payment-related problems faced by users and, if necessary, automatically transferring complex issues to human operators.

[0681] An "input method" is an interface for receiving inquiries from users.

[0682] "Information processing means" refers to a mechanism for analyzing user inquiries using natural language processing technology.

[0683] A "response generation means" is a device that generates an appropriate response for the user based on analysis results and by referring to a knowledge base.

[0684] "Response provision means" refers to an output device or interface for presenting the generated response to the user.

[0685] "Communication methods" refer to communication functions and processes for forwarding complex inquiries to human operators.

[0686] "Storage means" refers to a database or storage that records the history of inquiries and responses and stores them for later reference and analysis.

[0687] A "response support device" is a device that analyzes the intent of an inquiry and provides support functions that automatically generate a response using a knowledge base.

[0688] "Process automation means" refers to procedures and technologies for automatically escalating complex problems to human operators.

[0689] This invention is a system for responding quickly and accurately to user inquiries. The system consists of the following main elements:

[0690] The server first receives user inquiries through an input method. This input method often utilizes the user interface of a smartphone or computer. The received inquiries are then processed by information processing tools using natural language processing techniques to analyze the user's intent and question content. In this process, generative AI models such as OpenAI's GPT-3 are used to perform text-based intent analysis.

[0691] Next, the server uses a response generation mechanism to refer to a knowledge base based on the analysis results and generate an appropriate response. The knowledge base includes past inquiry history and specialized information, enabling it to provide precise solutions to users. The generated response is immediately presented to the user through the response delivery mechanism.

[0692] If the server determines that an inquiry is complex, it is automatically forwarded to a human operator via a communication system. This process is streamlined using RPA (Robotic Process Automation) technology. All inquiry and response history is recorded in a database by a storage system, which is used for future reference and knowledge base updates.

[0693] For example, when a user inquires about a payment error, the server can analyze the payment details and error message to quickly identify the cause and provide a solution. An example of a prompt used in such a system is as follows:

[0694] "User inquiry: {User's question}. Analyze the intent and propose the best solution."

[0695] This invention provides users with a means to quickly resolve problems related to electronic payments.

[0696] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0697] Step 1:

[0698] The terminal receives user inquiries through an input device. This input is sent to the server in text format. The server receives this text data and starts processing it as a trigger for initiating the inquiry.

[0699] Step 2:

[0700] The server processes the received text data using information processing tools and performs natural language processing using a generative AI model. Specifically, it analyzes the user's intent and inquiry content from the received text. This process uses the OpenAI GPT-3 model. The input is the user's inquiry, and the output is data containing the intent of the inquiry and the results of the analysis.

[0701] Step 3:

[0702] The server receives the analysis results from the information processing means and, based on that, uses the response generation means to refer to the knowledge base. The knowledge base records patterns and solutions learned from past queries. The server extracts the relevant information and generates an appropriate response. In this process, it utilizes prompt statements. The input for response generation is the analysis results, and the output is the text of the response provided to the user.

[0703] Step 4:

[0704] The generated response is returned to the user from the server through the response delivery mechanism. The user can view this response on their terminal and use it to resolve the problem and take the next action. The output contains information about a solution that is useful to the user.

[0705] Step 5:

[0706] If the server determines that a query is complex, it automatically forwards the query to a human operator using a communication method. In this phase, the server also sends the query history and analysis data. However, the user is notified in advance that operator confirmation is required.

[0707] Step 6:

[0708] Finally, the server records the history of all queries and responses in a storage system. This record is stored in a database and used for future reference and system improvement. The input is the data of all queries and response processes, and the output is the stored history for future analysis and improvement.

[0709] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0710] This invention relates to a customer support system that generates and provides responses to user inquiries using natural language processing and an emotion engine. This system recognizes the user's emotional state and reflects it in the response, thereby achieving more appropriate and effective customer support.

[0711] When a user enters an inquiry into the chatbot UI via their device, the device receives the inquiry. This input data is sent to a server, where it is analyzed by a natural language processing engine. Here, along with analyzing the content of the inquiry, an emotion engine determines the user's emotions. For example, it identifies whether the user is angry or sad based on the expression and wording of the entered text.

[0712] The server integrates analysis results obtained through natural language processing and sentiment data from the sentiment engine, and generates the optimal response by referring to a knowledge base. In this process, for example, if the user is expressing dissatisfaction, the tone of the response is softened and an apology is added, making adjustments according to the user's emotions.

[0713] The generated response is sent to the device and presented to the user. This allows the user to receive a response that reflects their emotions, not just information, thus improving the quality of the customer support experience.

[0714] Furthermore, the server records all interactions, including not only the content of inquiries but also emotional data. This allows for future analysis of consultations and improvement of response policies. It also has an emotional prioritization function, designed to automatically transfer urgent inquiries to operators.

[0715] For example, if a user makes an inquiry such as, "I'm very worried because my order hasn't arrived," the server recognizes the user's distress and provides an appropriate response such as, "We apologize for your inconvenience. We will investigate this matter immediately." In this way, efficient support that takes the user's feelings into consideration is achieved.

[0716] The following describes the processing flow.

[0717] Step 1:

[0718] The user enters an inquiry into the chatbot UI via their device. The device receives this input as text data and immediately sends it to the server.

[0719] Step 2:

[0720] The server passes the received text data to the natural language processing engine. The natural language processing engine analyzes the content of the query and identifies the user's intent. At the same time, it extracts necessary information based on keywords and context.

[0721] Step 3:

[0722] The server uses an emotion engine to analyze the emotions contained in the user's input. The emotion engine identifies the user's emotional state from their word choice and tone of voice, and quantifies and evaluates emotions such as joy, anger, and sadness.

[0723] Step 4:

[0724] The server integrates the analysis results obtained from natural language processing with the sentiment evaluation from the sentiment engine. Based on this information, it queries the knowledge base and retrieves the relevant response data.

[0725] Step 5:

[0726] The server generates a response to the user based on the response data it has acquired. During this process, the tone and content of the response are adjusted based on the sentiment evaluation. For example, if the sentiment score is high, the response may include gentle language or an apology.

[0727] Step 6:

[0728] The server sends the generated response to the terminal. The terminal displays the response to the user, who then reviews its contents.

[0729] Step 7:

[0730] If the user confirms the response and makes another inquiry, the terminal receives that input again and repeats the process from step 2.

[0731] Step 8:

[0732] The server records all inquiries, responses, and sentiment data in a database. This allows for analysis of inquiry history and sentiment trends, which can be used to improve the system in the future and optimize customer support strategies.

[0733] (Example 2)

[0734] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0735] Modern customer service systems require efficient methods for responding quickly and appropriately to user inquiries. However, traditional systems often fail to provide responses that consider the user's emotional state, resulting in a decline in service quality. Furthermore, they lack sufficient mechanisms for handling complex inquiries and effectively utilizing inquiry history. Therefore, there is a need to realize a customer support system that can respond efficiently and flexibly while considering the user's emotions.

[0736] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0737] In this invention, the server includes means for analyzing queries using natural language processing technology, means for generating an optimal response by referring to knowledge information based on the analysis results and sentiment data obtained through sentiment recognition, and means for displaying and providing the generated response. This makes it possible to quickly generate responses that reflect the user's emotions and improve the quality of service.

[0738] A "user" refers to an individual or legal entity that makes an inquiry using the system.

[0739] A "terminal" refers to a device used by users to input and submit inquiries.

[0740] "Natural language processing technology" refers to methods for computers to understand and analyze human language input.

[0741] "Emotion recognition" refers to analytical techniques used to identify a user's emotional state based on their input.

[0742] "Knowledge information" refers to an information database that is referenced to generate appropriate responses to queries.

[0743] "Response" refers to the reply that the system generates and provides to the user in response to an analyzed query.

[0744] An "operator" refers to a person who receives and handles urgent inquiries that are forwarded by the system.

[0745] "History information" refers to data that records and stores user inquiries and responses from the system.

[0746] The customer support system according to the present invention provides efficient customer service that enables flexible responses that take into account the user's emotions. This system mainly consists of a terminal that receives inquiries from users, a server that analyzes the content of inquiries, and means for generating and providing the optimal response.

[0747] In the system's operation, the user inputs and sends a query through a terminal. The terminal sends this data to the server. The server processes the input text using natural language processing technology (e.g., spaCy or NLTK) to extract keywords and intent, and uses sentiment recognition technology (e.g., IBM Watson Tone Analyzer) to determine the user's emotional state. This allows the server to understand the emotional nuances contained in the query and utilize this information to generate an appropriate response.

[0748] Specifically, for example, using a generative AI model (e.g., OpenAI GPT), the server references pre-built knowledge information, forms prompts based on the results of natural language processing and sentiment recognition, and uses these to generate responses. The responses are adjusted according to the sentiment entered by the user. For example, in response to an inquiry such as "I am in a lot of trouble," the server provides a flexible response such as "I apologize for your trouble. We will investigate immediately."

[0749] Furthermore, the system records all inquiry and response history information and retains it for future improvements. This history information is also used for analysis and automatic updates of knowledge, serving as foundational data to improve the overall accuracy of the system's responses. For inquiries deemed urgent, the server automatically transfers them to an operator, enabling prompt human response.

[0750] A concrete example of a prompt might be the instruction, "If the user is in great distress, what kind of empathetic response would be appropriate?" This would enable the system to automatically generate appropriate, emotion-based responses, thereby improving the user experience.

[0751] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0752] Step 1:

[0753] The user enters their inquiry into the device. Specifically, the user directly types text into the chat interface and presses the send button. The data entered at this point is a text message containing the user's questions or requests.

[0754] Step 2:

[0755] The terminal sends the inquiry data to the server. Upon receiving user input, the terminal converts the text data into JSON format and securely sends it to the server using HTTPS. This prepares the input data for processing on the server side.

[0756] Step 3:

[0757] The server analyzes the query using a natural language processing engine. The server parses the received JSON data and extracts the necessary information using natural language processing tools such as "spaCy" or "NLTK". Specifically, it extracts keywords from the text and identifies the user's intent. The output of this step is the analyzed intent and related keywords.

[0758] Step 4:

[0759] The server uses an emotion analysis engine to determine the user's emotions. Specifically, it uses tools such as "IBM Watson Tone Analyzer" to analyze the tone of the inquiry and identify the emotional state. For example, if the phrase "I'm troubled" is included, it will be determined that the user is confused. This analysis result is output as emotion data.

[0760] Step 5:

[0761] The server generates responses using a generative AI model. It integrates analyzed intent and sentiment data and references a knowledge base to create prompts that form the optimal response. Based on these prompts, it uses a generative AI model (e.g., OpenAI GPT) to generate appropriate responses. The generated text is the output of this step.

[0762] Step 6:

[0763] The server sends the generated response to the terminal. The server then converts the response text back into JSON format and sends it to the terminal via HTTPS. This ensures that an optimized response is delivered to the user's terminal.

[0764] Step 7:

[0765] The terminal displays a response to the user. The terminal analyzes the received response and converts it into a user-friendly format for display in the user interface. This allows the user to confirm the appropriate answer to their inquiry.

[0766] Step 8:

[0767] The server records the history of inquiries and responses in a database. The server meticulously records all interactions to aid in future analysis and system improvements. This history also includes user sentiment data.

[0768] (Application Example 2)

[0769] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0770] The current customer support system has a problem in that it is difficult to properly analyze the emotions of users when handling inquiries and adjust the tone of responses accordingly. As a result, the user experience may be insufficient, potentially leading to decreased satisfaction. Another challenge is the lack of means to quickly transfer urgent inquiries to the appropriate person.

[0771] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0772] In this invention, the server includes emotion analysis means for analyzing the user's emotional state, response adjustment means for adjusting the tone of the response based on the user's emotional state, and prioritization means for prioritizing queries based on emotion. This makes it possible to provide responses that take the user's emotions into consideration and to provide prompt and appropriate support.

[0773] "User emotional state" is a concept that refers to the type and intensity of emotions a user expresses when making an inquiry.

[0774] An "emotion analysis tool" is a system that determines and analyzes the type and intensity of emotions based on user input information.

[0775] A "response adjustment mechanism" is a system for appropriately adjusting the tone and content of a response statement based on the analyzed emotional state.

[0776] A "prioritization mechanism" is a system that takes into account the user's emotional state and determines the priority of processing based on the urgency of the inquiry.

[0777] A "transfer method" is a mechanism for transferring specific inquiries to a human representative as needed.

[0778] A "recording mechanism" is a system that stores all relevant information, including query and response data, in a database.

[0779] This system operates via the user's smartphone or computer terminal. First, the user enters their inquiry into the application's user interface. The terminal sends this data to the server. The server analyzes the user's emotional state using programming languages ​​such as Python and related libraries (e.g., TextBlob, transformers). In doing so, it utilizes natural language processing techniques to understand the structure of the user's input language.

[0780] Once the sentiment analysis is complete, the response tone is adjusted based on the results. The response adjustment mechanism selects a template corresponding to a specific emotion and generates a response message. If the prioritization mechanism determines that the response is urgent, it is quickly forwarded to the appropriate person. For normal inquiries, the response is sent to the terminal and presented to the user.

[0781] All inquiry data, response data, and sentiment analysis results are stored in a database via recording devices. This allows for future quality improvements and automated updates of the knowledge base. For example, if a user makes an inquiry such as "I'm worried because my product hasn't arrived," the server detects the emotion "worry" and provides a response such as "We apologize for the inconvenience; we will check the situation immediately."

[0782] An example of a prompt for the generating AI model is, "When a user inquires, 'My item hasn't arrived yet,' please provide an appropriate, emotionally sensitive response." Based on this example, it becomes possible to provide appropriate customer support that is sensitive to the user's emotions.

[0783] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0784] Step 1:

[0785] The user enters their inquiry into the customer support application's user interface using their device. The entered inquiry data is then sent to the server.

[0786] Step 2:

[0787] The server passes the received query data to a natural language processing engine, which analyzes the text structure. This process extracts grammatical elements and keywords from the input data to understand the query's content. The output is the analyzed text data.

[0788] Step 3:

[0789] The server sends the analyzed text data to the sentiment analysis system. The sentiment analysis system uses TextBlob or a similar library to determine the user's emotional state. It calculates the type (e.g., anger, relief) and intensity of the emotion from the input and outputs an emotion score.

[0790] Step 4:

[0791] Based on the sentiment score, the server selects an appropriate response template using a prompt message through a response adjustment mechanism. This prompt message allows the generative AI model to generate the optimal response. As a result, a customized response is output that is tailored to the analyzed sentiment and content.

[0792] Step 5:

[0793] The generated response is evaluated for urgency using a prioritization mechanism. Based on the sentiment analysis results, if the inquiry is deemed highly urgent, it is automatically forwarded to the appropriate person. The selection of the forwarding destination is then output.

[0794] Step 6:

[0795] After a response is generated and forwarded to the appropriate person as needed, the server saves all relevant data (inquiry, response, sentiment score) to a database using recording mechanisms. This completes the database for future improvements and reference. The output of this step is a record that can be used for future data analysis.

[0796] Step 7:

[0797] The terminal receives the generated response and presents it to the user through the user interface. This allows the user to receive emotionally appropriate and optimal support. The output is a response message that the user can view.

[0798] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0799] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0800] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0801] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0802] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0803] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0804] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0805] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0806] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0807] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0808] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0809] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0810] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0811] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0812] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0813] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0814] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0815] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0816] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0817] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0818] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0819] The following is further disclosed regarding the embodiments described above.

[0820] (Claim 1)

[0821] A terminal means for receiving inquiries from users,

[0822] A processing means for analyzing queries using natural language processing,

[0823] A response generation means that references a knowledge base based on the analysis results and generates an appropriate response,

[0824] A response provision means that presents the generated response to the user,

[0825] A transfer method for forwarding complex inquiries to an operator,

[0826] A recording means for recording the history of inquiries and responses,

[0827] A system that includes each of these.

[0828] (Claim 2)

[0829] The system according to claim 1, which can analyze inquiry content in multiple languages.

[0830] (Claim 3)

[0831] The system according to claim 1, which has a function to automatically update a knowledge base based on recorded history.

[0832] "Example 1"

[0833] (Claim 1)

[0834] A receiving means for receiving user inquiries via a digital device,

[0835] An analysis method that analyzes queries using natural language processing technology,

[0836] A generation means that references an information base based on the analysis results and generates an optimal response,

[0837] A presentation means that transmits and presents the generated response to the user's digital device,

[0838] Based on the analysis, a transfer method is provided to forward inquiries deemed complex to the appropriate person,

[0839] Recording means for storing all inquiries and their responses in an information recording device,

[0840] An update method that uses information analysis technology to analyze inquiry history and response results and update the information base,

[0841] A system that includes this.

[0842] (Claim 2)

[0843] The system according to claim 1, which analyzes the content of an inquiry in multiple languages ​​and provides the generated response in the corresponding language.

[0844] (Claim 3)

[0845] The system according to claim 1, which automatically improves the information base based on recorded historical data and optimizes future inquiry handling.

[0846] "Application Example 1"

[0847] (Claim 1)

[0848] An input means for receiving user inquiries,

[0849] An information processing means that analyzes queries using natural language processing,

[0850] A response generation means that references a knowledge base based on the analysis results and generates an appropriate response,

[0851] A means of providing an answer that presents the generated response to the user,

[0852] A means of forwarding complex inquiries to operators,

[0853] A means for recording and storing the history of inquiries and responses,

[0854] A response support means including analysis of the inquiry intent and automatic generation of responses using a knowledge base,

[0855] A process automation method that automatically transfers complex problems to human operators,

[0856] A means to quickly resolve payment problems by providing users with immediate responses,

[0857] A system that includes this.

[0858] (Claim 2)

[0859] The system according to claim 1, which can analyze inquiry content in multiple languages.

[0860] (Claim 3)

[0861] The system according to claim 1, which has a function to automatically update a knowledge base based on recorded history.

[0862] "Example 2 of combining an emotion engine"

[0863] (Claim 1)

[0864] A device that receives user inquiries via input,

[0865] A device that analyzes queries using natural language processing technology,

[0866] A device that generates the optimal response by referencing knowledge information based on analysis results and emotional data obtained through emotion recognition,

[0867] A device that adjusts and provides responses according to the user's emotional state,

[0868] A device that displays and provides the generated response,

[0869] A device that automatically transfers urgent inquiries to an operator,

[0870] A device for recording and storing historical information of inquiries and responses,

[0871] A system that includes this.

[0872] (Claim 2)

[0873] The system according to claim 1, which can analyze the content of an inquiry using multilingual processing.

[0874] (Claim 3)

[0875] The system according to claim 1, which has a function to automatically update knowledge information based on recorded historical data.

[0876] "Application example 2 when combining with an emotional engine"

[0877] (Claim 1)

[0878] A means of analyzing the emotional state of a user,

[0879] A response adjustment means that adjusts the tone of the response based on the user's emotional state,

[0880] Prioritization methods that assign emotional priorities to inquiries,

[0881] A response generation means for generating a response to present to the user,

[0882] A transfer method for forwarding complex or high-priority inquiries to human staff,

[0883] A recording means for recording inquiries, responses, and sentiment data,

[0884] A system that includes this.

[0885] (Claim 2)

[0886] The system according to claim 1, which analyzes the content of an inquiry in multiple languages ​​and provides a response adjusted based on the emotional state.

[0887] (Claim 3)

[0888] The system according to claim 1, which automatically updates knowledge information, including sentiment data, based on recorded historical data, and improves the quality of responses. [Explanation of Symbols]

[0889] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. An input means for receiving user inquiries, An information processing means that analyzes queries using natural language processing, A response generation means that references a knowledge base based on the analysis results and generates an appropriate response, A means of providing an answer that presents the generated response to the user, A means of forwarding complex inquiries to operators, A means for recording and storing the history of inquiries and responses, A response support means including analysis of the inquiry intent and automatic generation of responses using a knowledge base, A process automation method that automatically transfers complex problems to human operators, A means to quickly resolve payment problems by providing users with immediate responses, A system that includes this.

2. The system according to claim 1, which can analyze inquiry content in multiple languages.

3. The system according to claim 1, which has a function to automatically update a knowledge base based on recorded history.

Images