system

Abstract

Provide a system. A means for acquiring a communication history with a customer; means for analyzing a communication history with the customer and generating a summary; means for utilizing generative artificial intelligence to generate answers to user queries based on said summaries; means for providing said generated answers to a user or customer; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern companies, email exchanges with customers are commonplace, but it is not easy to respond to customer inquiries quickly and accurately based on this history. Especially when multiple exchanges are required, the involvement of sales representatives takes time and effort, and there is also the risk of human error. Given these circumstances, there is a growing need for systems that can improve the efficiency and accuracy of customer responses.

[0005] With the previous system, it took a lot of time and effort to manually search and analyze email history with customers and then prepare appropriate responses. Furthermore, because it relied on the knowledge and experience of the sales representative, there were also issues with inconsistency and quality in responses. As a result, issues such as a decline in customer satisfaction and an increased workload for sales representatives were raised.

[0006] The present invention aims to solve these problems and significantly improve the efficiency and automation of customer support processes. [Means for solving the problem]

[0007] The present invention solves the above problems by providing a system including the following means.

[0008] First, it provides a means for acquiring communication history with customers. This allows the system to appropriately collect the necessary data and prepare it for analysis. It also includes a means for analyzing communication history with customers and generating summaries. This makes it possible to extract important information from large amounts of historical data and organize it efficiently.

[0009] Next, the system provides a means for generating an answer to a user's inquiry based on the summary using generative artificial intelligence (AI), thereby enabling the system to automatically generate a quick and accurate answer. The system further includes a means for providing the generated answer to the user or customer, thereby enabling the user or customer to receive an appropriate answer quickly.

[0010] Furthermore, the system provides an interface for users or customers to send inquiries and includes a means for receiving inquiries, allowing users and customers to easily access the system and send their own inquiries. Furthermore, by including a means for searching and retrieving appropriate communication history based on received inquiries, the system automatically selects the most appropriate data and improves processing efficiency.

[0011] Additionally, generative artificial intelligence (AI) will use natural language processing techniques and include a means to generate appropriate responses using the query and generated summary as prompts, enabling more sophisticated, natural conversational responses and improving customer satisfaction.

[0012] "Customer communication history" refers to a series of data related to interactions with customers, such as emails, phone calls, and chats.

[0013] A "summary" is a sentence or piece of information that succinctly summarizes the content and meaning of the original text.

[0014] "Generative artificial intelligence (AI)" refers to artificial intelligence that uses machine learning algorithms to automatically create meaningful answers or products from input data.

[0015] "User" means any individual or entity that operates the System and uses certain features and services.

[0016] An "interface" is a software or hardware component that provides a means or method for a user to interact with a system.

[0017] A "prompt" is input information or instructions given to a generative artificial intelligence (AI) in a specific format.

[0018] "Natural language processing technology" is a technology that enables computers to understand, interpret, and generate human language. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] This invention is a system in which generative artificial intelligence (AI) provides appropriate answers when confirmation matters arise based on the history of customer interactions. Detailed embodiments of this system are described below.

[0041] Overall overview

[0042] This system acquires, analyzes, and summarizes communication history with customers, and then uses AI to generate appropriate answers to user inquiries. It also provides an interface that allows customers to make inquiries directly, automating tasks that do not require the intervention of sales staff.

[0043] Main components of the system

[0044] Server: Collects and stores communication history with customers, receives and processes inquiries from users or customers.

[0045] Generative artificial intelligence (AI): Analyzes collected communication history, generates summaries, and generates appropriate answers based on user inquiries.

[0046] User terminal: A device through which a user accesses and queries the system, from which the user submits queries through the interface.

[0047] Customer terminal: A device that allows a customer to directly query the system.

[0048] Program processing (natural language explanation)

[0049] History capture and storage

[0050] The server stores email and chat history with customers in a database. This history includes customer identification information and the content of the interactions. For example, information such as "Order X was placed on 2023-01-01, and Order Y was placed on 2023-02-15" is stored as an interaction with a customer.

[0051] Generate a summary

[0052] The server uses natural language processing techniques to summarize key information using the accumulated historical data, preparing the summary for rapid response to subsequent queries.

[0053] Receiving and parsing queries

[0054] A user or customer sends a query through the interface to make an inquiry. The server receives the query, analyzes it based on the appropriate communication history, and sends it to the generative AI.

[0055] Generate and provide answers

[0056] The generative AI analyzes the received query and summarized historical data as prompts to generate the optimal answer. The server then provides the generated answer to the user or customer. For example, in response to the query "What is the status of my recent orders?", the generated answer is "I placed order Y on 2023-02-15."

[0057] Specific examples

[0058] Example 1: User usage

[0059] The user sends a query from the interface saying, "Please tell me the status of Customer A's recent orders." The server retrieves Customer A's historical data and sends it to the generative AI. The AI ​​generates the optimal answer based on the historical data and the query, providing the answer, "Customer A placed order Y on 2023-02-15."

[0060] Example 2: Direct customer inquiry

[0061] A customer submits a query saying, "I want to know the payment status of my recent invoices." The server retrieves the customer's payment history and sends it to the generative AI. The AI ​​generates an answer based on the historical data and the query, providing the answer, "Regarding recent invoices, invoice B was paid on 2023-02-20."

[0062] In this way, the system can respond quickly and accurately to customer and user inquiries. The distinctive features of this invention are the generation of a summary of the history and the generation of answers using generative AI, which can improve the efficiency and accuracy of customer responses.

[0063] The processing flow will be explained below.

[0064] Step 1:

[0065] The server collects past email and chat history with customers and stores it in a database, including customer identification information, dates, and specific exchanges.

[0066] Step 2:

[0067] The server periodically analyzes historical data and automatically generates summaries, using natural language processing techniques to extract key points and create concise summaries.

[0068] Step 3:

[0069] A user or customer makes an inquiry. A query is sent from the user or customer terminal through the interface. This query contains information that the user wants to know or a matter that needs confirmation.

[0070] Step 4:

[0071] The server analyzes queries received through the interface and searches for and retrieves historical data for the relevant customers, including, if necessary, retrieving the latest summaries from the database.

[0072] Step 5:

[0073] The server sends the acquired history data and the received query to a generative artificial intelligence (AI). The prompt sent to the AI ​​includes the summarized history data and the specific query content.

[0074] Step 6:

[0075] Generative artificial intelligence (AI) generates appropriate responses based on prompts. AI uses natural language processing techniques to create the most appropriate response to a query.

[0076] Step 7:

[0077] The server returns the answer received from the generative artificial intelligence (AI) to the user or customer, and the answer is displayed through the interface so that the user or customer can easily check it.

[0078] Step 8:

[0079] If the user or customer needs to make further inquiries or clarifications based on the answers they receive, they can send the query again, and the server will repeat the same process to continue providing appropriate answers.

[0080] Example 1

[0081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0082] With conventional systems, it was difficult to efficiently utilize communication history with customers and respond quickly and accurately to inquiries from users and customers. Responding to inquiries also required a large amount of manpower, creating a demand for greater operational efficiency. Therefore, there is a need for a system that automates and efficiently performs operations while maintaining the quality of customer service.

[0083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0084] In this invention, the server includes means for acquiring a communication history with a customer, means for saving the communication history with the customer, means for analyzing the communication history with the customer and summarizing important information, means for accepting an inquiry from a user or customer through an interface, means for searching and acquiring an appropriate communication history based on the received inquiry, means for generating an optimal answer based on the summary and the inquiry using generative artificial intelligence, and means for providing the generated answer to the user or customer, thereby enabling prompt and accurate responses to inquiries from customers or users.

[0085] "Customer communication history" refers to records of emails, chats, phone calls, and other interactions with customers.

[0086] A "server" is a computer system that provides functions such as data processing, storage, and network communication.

[0087] "Natural language processing technology" is a technology that uses computers to analyze, understand, and generate human language.

[0088] "Generative artificial intelligence (AI)" is artificial intelligence that has the ability to generate new information and answers based on input data.

[0089] An "enquiry" is a question or request that a user or customer enters into the system.

[0090] An "interface" is the means or method by which a user or customer interacts with a system.

[0091] A "query" is a question or search request issued to the system.

[0092] A "prompt" is input data or instructions given to generative artificial intelligence, which allows the AI ​​to generate an appropriate answer.

[0093] An "answer" is response information generated by a generative artificial intelligence based on a query.

[0094] The "summary" is a concise summary of important information extracted from the communication history with the customer.

[0095] This invention is a system in which generative artificial intelligence (AI) provides appropriate answers based on communication history with customers. The system of the present invention is specifically implemented as follows.

[0096] Overall structure

[0097] The main components of this system are a server, a generative AI, a user terminal, and a customer terminal. The server collects, stores, and analyzes data, and the generative AI generates appropriate answers. The user terminal and customer terminal provide the interface for users and customers to access the system and make inquiries.

[0098] Collecting and storing customer communication history

[0099] The server automatically collects communication history with customers, such as emails and chats, and stores it in a database. Standard server machines are used for the hardware and software used include the Python imaplib library and the Slack API.

[0100] Data analysis and summary generation

[0101] The server analyzes the stored historical data and summarizes key information using natural language processing (NLP) techniques, such as Python libraries nltk and spacy, to help generate the summaries.

[0102] Receiving inquiries

[0103] Users or customers access the interface through their terminals and send queries. The server receives these queries and searches and retrieves relevant historical data from the database. The interface is often implemented using popular web frameworks such as Flask and Django.

[0104] Generate answers

[0105] Generative artificial intelligence (AI) generates optimal answers based on the received query and summarized historical data. In this process, the query and related historical data are input as prompts into a generative AI model (e.g., OpenAI's GPT series).

[0106] Providing answers

[0107] The server provides the generated answer to the user or client terminal, which displays the answer through an interface.

[0108] Specific examples

[0109] Example 1: User usage

[0110] The user sends a query from the interface saying, "Please tell me the status of Customer A's recent orders." The server retrieves Customer A's historical data and sends it to the generative AI. The AI ​​generates the optimal answer based on the historical data and the query, providing the answer, "Customer A placed order Y on 2023-02-15."

[0111] Example 2: Direct customer inquiry

[0112] A customer submits a query saying, "I want to know the payment status of my recent invoices." The server retrieves the customer's payment history and sends it to the generative AI. The AI ​​generates an answer based on the historical data and the query, providing the answer, "Regarding recent invoices, invoice B was paid on 2023-02-20."

[0113] This system makes it possible to respond quickly and accurately to inquiries from customers and users. In addition, the system's unique features of this invention, such as the generation of summaries of communication history and the generation of answers using generative AI, make it possible to improve the efficiency and accuracy of customer support.

[0114] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0115] Step 1:

[0116] The server retrieves communication history with customers. The input is a data source such as a mail server or chat logs. The server retrieves unread emails and chat messages using Python's imaplib library or the Slack API. The output is the retrieved interaction data.

[0117] Specific behavior:

[0118] The server connects to the mail server and retrieves the unread emails.

[0119] The email body, sender information, and receipt date and time are obtained and stored as internal temporary data.

[0120] Step 2:

[0121] The server stores the acquired communication history with the customer in a database. The input is the acquired interaction data. The output is the communication history stored in the database. The database uses MySQL (registered trademark) or PostgreSQL, etc.

[0122] Specific behavior:

[0123] Create a table in the database with columns such as customer ID, transaction text, date and time.

[0124] Store the temporary data in the database using the INSERT statement.

[0125] Step 3:

[0126] The server analyzes the historical data stored in the database and summarizes key information using natural language processing (NLP) techniques. The input is data from the database of customer communication history. The output is summarized text data.

[0127] Specific behavior:

[0128] Use the SELECT statement to retrieve customer history data from the database.

[0129] Use Python's nltk and spacy libraries to tokenize text and tag parts of speech.

[0130] Keyword extraction and summary generation algorithms are used to output important information as a summary.

[0131] Step 4:

[0132] A user or customer sends a query. The input is a query from the user terminal or customer terminal. The output is the query information received by the server, which processes it appropriately.

[0133] Specific behavior:

[0134] A query sent from a user terminal is received by a web framework (e.g., Flask).

[0135] Stores received queries as internal temporary data.

[0136] Step 5:

[0137] The server searches and retrieves the appropriate communication history based on the received query. The input is the user or customer query and the database. The output is the communication history data related to the query.

[0138] Specific behavior:

[0139] The keywords contained in the query are analyzed and search conditions are generated to retrieve appropriate historical data from the database.

[0140] Based on the search criteria, relevant data is retrieved from the database.

[0141] Step 6:

[0142] Generative AI generates optimal answers based on queries and related historical data. The input is the user or customer query and summarized historical data. The output is the generated answer.

[0143] Specific behavior:

[0144] The server sends queries and historical data as prompts to the generative AI model.

[0145] The generative AI model generates an answer and sends it back to the server.

[0146] Step 7:

[0147] The server provides the generated answer to the user or client terminal. The input is the generated answer. The output is the answer displayed on the interface.

[0148] Specific behavior:

[0149] The server transmits the generated answer to the user or client terminal.

[0150] The answers are displayed in the interface for the user or customer to review.

[0151] (Application example 1)

[0152] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0153] Customer service in brick-and-mortar stores requires providing information quickly and accurately, but traditional methods make it difficult for staff to grasp all product information and customer history, resulting in problems that lead to lower customer satisfaction and reduced productivity. Particularly in busy store environments, staff lack the support they need to quickly provide appropriate answers.

[0154] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0155] In this invention, the server includes: means for acquiring a communication history with a customer; means for analyzing the communication history with the customer and generating a summary; means for utilizing generative artificial intelligence (AI) to generate an answer to a user's inquiry based on the summary; means for providing the generated answer to the user or customer; and means for receiving an inquiry via an information display device operated by a user in a store and presenting the generated answer to the user in real time. This enables staff to respond to customer inquiries quickly and accurately, even in a physical store, thereby improving customer satisfaction and business efficiency.

[0156] "Customer communication history" refers to a record of all interactions, inquiries, purchase history, etc. that a customer has with a company or service provider.

[0157] A "summary" is information that extracts important information from the communication history with a customer and summarizes it concisely.

[0158] "Generative artificial intelligence (AI)" is an AI that has the ability to generate appropriate answers to user inquiries based on collected data.

[0159] An "information display device" is a device worn or used by a user, which displays answers to inquiries in real time.

[0160] The term "user" refers to an employee such as a store staff member who operates the system of the present invention and deals with customers.

[0161] A "server" is a computer system that collects, stores, analyzes, and generates answers using generative artificial intelligence (AI).

[0162] "Interface" refers to the means or screen through which a user or customer makes inquiries to the system.

[0163] A "prompt" is data that provides the underlying information or conditions that a generative artificial intelligence (AI) uses to generate an appropriate answer.

[0164] The present invention provides a system that enables store staff to use smart glasses or other information display devices to quickly and accurately interact with customers. Detailed embodiments of the system are described below.

[0165] Overall overview

[0166] This system acquires and analyzes communication history with customers, and then uses generative artificial intelligence (AI) to summarize the results.The user (store staff) can then provide appropriate answers to customer inquiries in real time via an information display device.

[0167] System Components

[0168] server

[0169] The server collects, stores, analyzes, and summarizes the communication history with the customer, which includes the following steps:

[0170] 1. History capture and storage:

[0171] The server stores email and chat history with customers in a database, preferably using MySQL or similar database.

[0172] 2. Generate a summary:

[0173] The server uses natural language processing technologies such as Python's NLTK and Gensim to analyze the accumulated historical data and summarize important information.

[0174] Generative artificial intelligence (AI)

[0175] 3. Receiving and parsing queries:

[0176] The server analyzes queries received through smart glasses or information display devices and searches for appropriate communication history.

[0177] 4. Generate and provide answers:

[0178] The server uses a generative AI model (e.g., GPT-3 (registered trademark)) to generate an optimal answer based on the analysis results. The generated answer is then processed using deep learning libraries such as TENSORFLOW (registered trademark) and PyTorch.

[0179] The server presents this answer in real time to the user's smart glasses or information display device.

[0180] User terminal

[0181] The user terminal is a device worn by the user, such as smart glasses, that displays the generated answers in real time.

[0182] Specific examples

[0183] Example 1: Customer Inquiry

[0184] A customer asks, "Are there any other products similar to this one?" The user sends this inquiry to the server via their smart glasses. The server retrieves the customer's past purchase history and inquiry history and sends it to the generative AI. The generative AI analyzes this and generates an answer such as, "Products B and C are similar to this product." This answer is displayed in real time on the user's smart glasses.

[0185] Prompt Sentence Examples

[0186] Here is an example of a prompt for a generative AI model:

[0187] User query: "Are there any other products like this?"

[0188] Past purchase history: Product A (2023-01-10), Product B (2023-02-15)

[0189] Past inquiry history: "I'd like to know about similar products" "Please recommend some products"

[0190] Example of an answer it might generate: "Products similar to this one are Product B and Product C."

[0191] This system will enable staff at physical stores to respond quickly and accurately to a wide range of customer questions, improving customer satisfaction and operational efficiency.

[0192] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0193] Step 1:

[0194] A user receives a customer inquiry via smart glasses.

[0195] Input: Customer inquiry (e.g., "Do you have anything similar to this product?")

[0196] Output: Query data sent to the server

[0197] How it works: The user speaks their inquiry into the smart glasses' microphone, which is then converted into text data and sent to the server. The voice recognition uses Google's Speech-to-Text API and other technologies.

[0198] Step 2:

[0199] The server analyzes the received query.

[0200] Input: Query data sent by the user (a textual query)

[0201] Output: The parsed results of the query

[0202] Specific operation: The server analyzes the received query using natural language processing techniques (such as NLTK or SpaCy) and prepares it to query the database in an appropriate manner. Typical analysis includes understanding the meaning of the query and extracting keywords.

[0203] Step 3:

[0204] The server searches and retrieves the customer's communication history.

[0205] Input: query analysis results, customer identification information

[0206] Output: Customer communication history data

[0207] Specific operation: Based on the parsed query and customer identification information, the server searches the database (e.g., MySQL) for relevant communication history and retrieves the required data, including past purchase history and inquiry history.

[0208] Step 4:

[0209] Summarize historical data obtained by the server.

[0210] Input: Customer communication history data

[0211] Output: Summarized historical data

[0212] How it works: The server uses Python libraries such as NLTK and Gensim to extract key information from the acquired historical data and generate summaries, which are then used as prompts for the generative AI model.

[0213] Step 5:

[0214] The server sends prompts to the generative AI based on the summary data and the query.

[0215] Input: summarized historical data, query text

[0216] Output: prompt to generative AI

[0217] How it works: The server combines the summarized history data with the parsed query and formats it as a prompt for a generative AI model (e.g., GPT-3). Examples of prompts include: "User query: 'Are there any other products similar to this one?'," "Past purchase history: Product A (2023-01-10), Product B (2023-02-15)," and "Past inquiry history: 'I'd like to know about similar products,' and 'What products do you recommend?'" These are then sent to the generative AI model.

[0218] Step 6:

[0219] A generative AI model generates appropriate answers.

[0220] Input: prompt to generative AI

[0221] Output: Generated answer (text format)

[0222] How it works: A generative AI model (e.g., GPT-3) generates the best answer based on the prompt provided, using natural language processing and deep learning techniques in the process.

[0223] Step 7:

[0224] The server sends the generated answer to the user's smart glasses.

[0225] Input: Generated answer (text format)

[0226] Output: Answer displayed on smart glasses

[0227] Specific operation: The server sends the generated answer to the user's smart glasses in real time, allowing the user to visually confirm it. The generated answer is displayed on the display of the smart glasses.

[0228] Step 8:

[0229] The user provides the answer to the customer.

[0230] Input: Answer displayed on smart glasses

[0231] Output: Providing a verbal response to the customer

[0232] Specific operation: The user reads out the answer displayed on the smart glasses and provides the answer to the customer verbally, allowing the customer to obtain the information quickly and accurately.

[0233] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0234] The present invention combines a system that analyzes communication history with customers and generates summaries with an emotion engine that recognizes the user's emotions. This system takes into account the user's emotional state and can generate more appropriate and effective responses. An embodiment of this system will be described in detail below.

[0235] Main components of the system

[0236] Server: Collects and stores communication history with customers, receives and processes inquiries from users or customers.

[0237] Generative artificial intelligence (AI): Analyzes collected historical data, generates summaries, and generates appropriate answers based on user queries.

[0238] User terminal: A device through which a user accesses and queries the system.

[0239] Customer terminal: A device that allows a customer to directly query the system.

[0240] Emotion engine: Analyzes the emotional state of the user's query and adjusts the generated answer based on the results.

[0241] Program processing (natural language explanation)

[0242] History capture and storage

[0243] The server collects and stores in a database the history of past emails and chats with customers, including the customer's identity and the content of those conversations.

[0244] Generate a summary

[0245] The server uses natural language processing techniques to summarize key information using the accumulated historical data, preparing the summary for rapid response to subsequent queries.

[0246] Query Reception and Emotion Recognition

[0247] A user or customer makes an inquiry. The query is sent from the user terminal or customer terminal through the interface. The server analyzes the query received through the interface and analyzes the user's emotional state using an emotion engine.

[0248] Emotion-Based Adjustment

[0249] The emotion engine analyzes the user's emotional state and dynamically changes the prompts sent to the generative artificial intelligence (AI) based on the results. For example, if the user is angry, the prompts will be set to generate responses with a more polite and calm tone.

[0250] Generate and provide answers

[0251] The generative AI generates an appropriate answer based on the prompts adjusted by the emotion engine. The server then provides the generated answer to the user or customer. For example, if the user is feeling anxious in response to the query "Please tell me the status of my recent order," the server will provide an answer in the form of "Sorry to keep you waiting. I will explain the status of my recent order. ..."

[0252] Specific examples

[0253] Example 1: User usage

[0254] The user sends a query from the interface saying, "Please tell me the status of Customer A's recent orders." The server retrieves Customer A's historical data and analyzes the user's emotional state using the emotion engine. The generative AI uses a prompt adjusted based on the analysis results of the emotion engine to generate and provide the answer, "Customer A placed order Y on 2023-02-15 as their most recent order."

[0255] Example 2: Direct customer inquiry

[0256] A customer submits a query saying, "I want to know the payment status of my recent invoices." The server retrieves the customer's payment history and analyzes their emotional state using an emotion engine. The generative AI uses a prompt tailored based on the analysis results of the emotion engine to generate and provide an answer such as, "Regarding recent invoices, I paid invoice B on 2023-02-20."

[0257] The role of the emotional engine

[0258] By analyzing the user's emotional state, the emotion engine can adjust the tone and content of responses to deliver a more personalized experience, improving customer satisfaction and building better customer relationships.

[0259] In this way, by combining emotion engines, it is possible to achieve more advanced and flexible customer response than conventional automated response systems.

[0260] The processing flow will be explained below.

[0261] Step 1:

[0262] The server collects past email and chat history with the customer and stores it in a database. This data includes customer identification information, the date of the interaction, and the specific content. For example, it stores information that "Customer A" placed "Order X" on "2023-01-01."

[0263] Step 2:

[0264] The server uses natural language processing techniques to extract key information from the stored historical data and generate a summary, which serves as a preliminary step for quickly responding to future inquiries.

[0265] Step 3:

[0266] A user or customer submits a query through the interface to make an inquiry. For example, a customer might type, "I want to know the status of my recent bill payment."

[0267] Step 4:

[0268] The server analyzes queries received through the interface and retrieves the relevant customer history data from the database, ensuring that the relevant data is up to date.

[0269] Step 5:

[0270] The server sends the acquired historical data and the query to the emotion engine, which analyzes the user's emotional state. For example, it determines whether the user is feeling "anxious" or "angry" based on the query's style and wording.

[0271] Step 6:

[0272] The emotion engine analyzes the user's emotional state and sends the results back to the server. For example, the analysis result may say, "The user is feeling anxious."

[0273] Step 7:

[0274] The server then adjusts the prompts sent to the generative artificial intelligence (AI) based on the emotion engine's analysis, for example, "If you're feeling anxious, add more detailed explanations and reassuring words."

[0275] Step 8:

[0276] Generative artificial intelligence (AI) generates appropriate answers based on tailored prompts, such as, "Regarding your recent invoices, you paid invoice B on 2023-02-20. Please rest assured about the details."

[0277] Step 9:

[0278] The server provides the generated answer to the user or customer. The answer is displayed through the interface so that the user or customer can easily check it. By displaying it on the terminal, the user can quickly obtain an appropriate answer to their inquiry.

[0279] Step 10:

[0280] If the user or customer is not satisfied with the answer they received and needs further inquiry or confirmation, they can send the query again, and the server will repeat the same process and continue to generate the best answer.

[0281] In this way, a system incorporating an emotion engine can provide more flexible and accurate customer service while taking into account the user's emotional state.

[0282] Example 2

[0283] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0284] Conventional customer service systems provide uniform responses without considering the emotional state of the user or customer, making it difficult to improve customer satisfaction. Furthermore, responses that ignore the emotional state can exacerbate customer dissatisfaction and anxiety, risking a deterioration in customer relationships. The challenge with conventional systems is the difficulty of properly assessing the emotional state of the user or customer and providing more personalized responses.

[0285] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for acquiring a communication history with a customer; means for saving the communication history with the customer; means for analyzing the communication history with the customer and generating a summary; means for a user or customer to send an inquiry; means for searching and acquiring an appropriate communication history based on the received inquiry; means for receiving the inquiry; means for using an emotion engine including means for analyzing the emotional state of the user in addition to the summary; means for adjusting a prompt to be sent to a generative artificial intelligence (AI) based on the emotional state of the user; means for utilizing the generative artificial intelligence (AI) to generate an appropriate answer based on the adjusted prompt; and means for providing the generated answer to the user or customer. This enables more personalized responses that take into account the emotional state of the user or customer.

[0286] "Customer communication history" refers to exchanges between customers and companies, such as emails, chats, phone recordings, and social media messages.

[0287] "Database" refers to a structured collection of data for managing and storing collected information.

[0288] "Natural language processing technology" refers to technologies and methods for understanding, analyzing, and generating human language.

[0289] An "emotion engine" refers to a software component that analyzes the emotional state of a user or customer (e.g., joy, anger, anxiety, etc.) from text data.

[0290] "Generative artificial intelligence (AI)" refers to artificial intelligence that automatically generates documents and information that humans can understand and use in natural language based on given prompts.

[0291] A "prompt" refers to input information that instructs a generative artificial intelligence (AI) on what kind of answer it should generate.

[0292] "User's emotional state" refers to the emotional state (e.g., joy, anger, anxiety, etc.) of the user when they input a query.

[0293] "Interface" refers to the device or software mechanism through which a user or customer accesses the system and sends and receives inquiries.

[0294] A "summary" is a concise summary of complex data or documents, extracting only the most important information.

[0295] "Enquiry" means a question or request made by a user or customer to the system.

[0296] The present invention combines a system that analyzes communication history with customers and generates summaries with an emotion engine that recognizes the user's emotions. This system takes the user's emotional state into account and generates more appropriate and effective answers.

[0297] Main components of the system

[0298] Server: Collects and stores customer communication history, receives and processes inquiries from users or customers. Examples: AWS EC2 instances, Google Cloud Compute Engine.

[0299] Database: A system for storing and managing collected historical data. Examples: MySQL, PostgreSQL.

[0300] Generative Artificial Intelligence (AI): Analyzes collected historical data, generates summaries, and generates appropriate answers based on user queries. Examples: OpenAI GPT-3, Google BERT.

[0301] User terminal: A device through which a user accesses and queries the system. Examples: web browser, smartphone app.

[0302] Customer terminal: A device that allows a customer to directly query the system.

[0303] Emotion engine: Analyzes the emotional state of the user's query and adjusts the generated answer accordingly. Examples: IBM Watson(R), Azure(R) Text Analytics.

[0304] Program processing

[0305] History capture and storage

[0306] The server collects past communication history with the customer using the specified mailbox or chat service API (e.g., Gmail API or Slack API). The collected data is stored in a database (e.g., MySQL or PostgreSQL) along with the customer's identification information and metadata.

[0307] Generate a summary

[0308] The server analyzes the accumulated historical data using natural language processing (NLP) techniques (e.g., NLTK, SpaCy), extracts important information, and generates summaries, which are then stored in a database as a preliminary step for responding quickly to future inquiries.

[0309] Receiving a query

[0310] Users or customers make inquiries from a web browser or smartphone app. The inquiries are sent to the server as HTTP requests, and the server receives them and stores them in a database in text format.

[0311] Emotion recognition

[0312] The server sends the text data of the received query to an emotion engine (e.g., IBM Watson NLP API, Azure Cognitive Services) to analyze the user's emotional state. The emotion engine returns a score to the server, such as "anxiety: 0.75, joy: 0.10, anger: 0.15."

[0313] Emotion-Based Adjustment

[0314] The server adjusts the prompts sent to the generative AI based on the emotion score received from the emotion engine. For example, if the anxiety score is high, the server generates a prompt that reads, "The user is feeling anxious. Please generate a reassuring response."

[0315] Generate and provide answers

[0316] The generative AI receives the tailored prompt and generates an appropriate answer based on it. For example, in response to the query "What is the status of Customer A's recent orders?", it generates the answer "Customer A recently placed order Y on 2023-02-15." The server then provides the generated answer to the user or customer through the same interface that received the query.

[0317] Specific examples

[0318] Example 1: User usage

[0319] When a user sends a query from the interface saying, "Please tell me the status of Customer A's recent orders," the server retrieves Customer A's historical data and analyzes the user's emotional state using the emotion engine. The generative AI uses a prompt adjusted based on the analysis results of the emotion engine to generate and provide the answer, "Customer A placed order Y on 2023-02-15 as their most recent order."

[0320] Example 2: Direct customer inquiry

[0321] When a customer sends a query such as "I want to know the payment status of my recent invoices," the server retrieves the customer's payment history and analyzes their emotional state using an emotion engine. The generative AI uses prompts tailored based on the analysis results of the emotion engine to generate and provide an answer such as "Regarding recent invoices, I paid invoice B on 2023-02-20."

[0322] Prompt Sentence Examples

[0323] Emotional state is anxious: "The user is feeling anxious. Please generate a polite and reassuring response."

[0324] Angry emotional state: "The user is angry. Please set a calm and measured tone in your response."

[0325] In this way, by combining emotion engines, it is possible to achieve more advanced and flexible customer response than conventional automated response systems.

[0326] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0327] Step 1: Collecting History

[0328] The server collects communication history with customers. As input, it obtains data from mailboxes and chat service APIs (e.g., Gmail API, Slack API). The specific process for collecting data involves sending queries to the API and analyzing the returned data. Through this operation, the contents of customer emails and chats are collected on the server.

[0329] Step 2: Save history

[0330] The server saves the collected customer communication history in a database. The input is the data collected in step 1. The server generates a document containing the customer's identification information and metadata and saves it in a database (e.g., MySQL, PostgreSQL). This operation records the customer's communication history in the database.

[0331] Step 3: Generate a summary

[0332] The server analyzes the accumulated historical data using natural language processing (NLP) techniques (e.g., NLTK, SpaCy). The input is the data saved in step 2. The server extracts important information and performs text analysis and semantic extraction as specific processes to generate a summary. The summary generated as output is saved in a database. This operation adds the summary information to the database.

[0333] Step 4: Receiving a query

[0334] A user or a client sends a query through the interface. The input is a query text from the terminal. The server receives the query as an HTTP request and stores it in a database in text format. This operation allows the server to manage the received queries.

[0335] Step 5: Recognize emotions

[0336] The server sends the text data of the received query to an emotion engine (e.g., IBM Watson NLP API, Azure Cognitive Services). The input is the query text received in step 4. The emotion engine performs specific processing to analyze the user's emotional state from the text. As an output, an emotion score (e.g., "Anxiety: 0.75, Joy: 0.10, Anger: 0.15") is returned to the server. This operation analyzes the user's emotional state.

[0337] Step 6: Emotional Adjustment

[0338] The server adjusts the prompt based on the emotion score received from the emotion engine. The input is the emotion score obtained in step 5 and the customer summary information stored in the database. The server uses the emotion score to dynamically generate a prompt to send to the generative AI (e.g., OpenAI GPT-3, Google BERT). For example, a prompt might be generated that reads, "The user is feeling anxious. Please generate a polite and reassuring response." This operation improves the accuracy of the generated response.

[0339] Step 7: Generate and serve answers

[0340] The generative AI receives the adjusted prompt and generates an appropriate answer based on it. The input is the prompt adjusted in step 6. The generative AI performs the specific process of generating an answer according to the prompt and returns the answer as output to the server. The server sends the generated answer to the user or customer. The answer is provided through the same interface that received the inquiry. This operation provides an appropriate answer to the user or customer.

[0341] The above is a specific description of the processing steps in the system.

[0342] (Application example 2)

[0343] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0344] Conventional automated response systems generate responses without considering the emotional state of the customer, limiting their ability to improve customer satisfaction. Furthermore, they lacked the processes for summarizing information and analyzing emotions to provide prompt and appropriate responses, making it difficult to provide effective customer support.

[0345] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0346] In this invention, the server includes means for acquiring a communication history with a customer, means for analyzing the communication history with the customer and generating a summary, means for utilizing an emotion engine that analyzes the emotional states of the user and the customer, means for utilizing generative artificial intelligence (AI) that generates an answer to the user's inquiry based on the summary and the emotion analysis results, and means for providing the generated answer to the user or customer. This makes it possible to provide a quick and accurate answer that takes into account the emotional state of the customer.

[0347] "Customer communication history" refers to past conversation data, such as emails and chats, between the customer and the system.

[0348] A "summary" is a concise summary of important information extracted from the collected communication history.

[0349] An "emotion engine" refers to a system that analyzes emotions from input sentences of a user or customer and recognizes their emotional state.

[0350] "Generative artificial intelligence (AI)" refers to artificial intelligence that uses natural language processing technology to generate appropriate answers based on user inquiries.

[0351] "Interface" refers to the screen or device through which a user or customer submits a query to the system.

[0352] A "prompt" refers to the input information that generative artificial intelligence (AI) uses to generate an appropriate answer.

[0353] "Answer" means a response generated and provided by a generative artificial intelligence (AI) in response to a user or customer inquiry.

[0354] To implement this invention, a system consisting of the following means is required. The main components of the system are a server, a user terminal, a customer terminal, an emotion engine, and a generative artificial intelligence (AI).

[0355] Main components of the system

[0356] server

[0357] The server has a database that stores and manages past communication history with customers, and also provides the computing resources to analyze inquiries and generate appropriate answers.

[0358] User and customer terminals

[0359] These terminals refer to devices that users or customers use to access the system and send inquiries, such as PCs, smartphones, and tablets.

[0360] Emotion Engine

[0361] The emotion engine uses the Samsung Emotional Engine API and other tools to analyze emotions from received text.

[0362] Generative artificial intelligence (AI)

[0363] As generative artificial intelligence (AI), we use advanced natural language generation models such as GPT-3.

[0364] System Overview

[0365] The server acquires past communication history and stores it in a database. When a user or customer sends an inquiry, the server receives the inquiry. After receiving the inquiry, the server uses an emotion engine to analyze emotions from the inquiry text.

[0366] Based on the analysis results of the emotion engine, the server uses generative artificial intelligence (AI) to generate an appropriate answer. When the generative artificial intelligence (AI) generates an answer, it adjusts the prompt based on the analysis results of the emotion engine to provide a more appropriate answer.

[0367] Specific examples

[0368] For example, consider a food delivery service application. If a customer asks, "Why hasn't my order arrived yet?", the server receives the inquiry. If the emotion engine detects "anger," the generative artificial intelligence (AI) can generate a response using prompts like the following:

[0369] Example prompt sentence:

[0370] Customer: "My order hasn't arrived yet."

[0371] Emotion analysis result: "Anger"

[0372] Generative AI prompt: "The customer seems upset. Please generate a response that explains the delay in an appropriate and polite tone."

[0373] The appropriate response generated by this would be something like, "We're very sorry. We're currently checking the tracking information for your order, so please wait a moment." This system makes it possible to provide quick and accurate responses that take into account the customer's feelings.

[0374] Software and hardware used

[0375] Hardware

[0376] Server (e.g. AWS EC2), user device (e.g. smartphone, tablet), customer device (e.g. PC, smartphone)

[0377] software

[0378] Emotion engines (e.g., Samsung Emotional Engine API), generative artificial intelligence (e.g., GPT-3), database management systems (e.g., MySQL)

[0379] This configuration makes it possible to provide flexible and accurate automatic responses that respond to customer emotions when dealing with customers in food delivery services and the like.

[0380] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0381] Step 1: Capture and save history

[0382] The server retrieves and stores the customer's past communication history from the database. Specifically, the server collects data such as emails, chats, and order history, and stores it in a database (e.g., MySQL). At this time, the customer's identification information and the content of the communication are stored in the database.

[0383] Input: Email, chat, order history

[0384] Output: Communication history stored in a database

[0385] Step 2: Receiving and analyzing inquiries

[0386] The server receives queries from users or customers. The queries are sent from the user or customer terminal and received by the server through an interface. The received queries are analyzed by an emotion engine (e.g., Samsung Emotional Engine API) to identify the emotional state.

[0387] Input: User or customer inquiry

[0388] Output: Emotion analysis results

[0389] Step 3: History search and summary generation

[0390] The server searches and retrieves relevant historical data from the database based on the sentiment analysis results and the query. The retrieved historical data is summarized using natural language processing techniques (e.g., BERT). The summary extracts only the important information and summarizes it concisely. This information is used in subsequent processing steps.

[0391] Input: Sentiment analysis results, query text

[0392] Output: Summarized historical data

[0393] Step 4: Generate and refine prompts

[0394] The server generates prompts based on the emotion analysis and summarized historical data, which are then sent to a generative artificial intelligence (AI). The generated prompts are tailored based on the emotion analysis. For example, if the user's emotion is anger, a prompt that responds in a polite tone is set.

[0395] Input: Sentiment analysis results, summarized historical data

[0396] Output: Adjusted prompt

[0397] Step 5: Generate an answer

[0398] Generative artificial intelligence (AI) generates appropriate responses based on tailored prompts. Specifically, an AI model (e.g., GPT-3) analyzes the prompts and generates the best possible response for the user or customer. This model is pre-trained to generate natural-sounding sentences with a high degree of accuracy.

[0399] Input: Adjusted prompt

[0400] Output: Generated answer

[0401] Step 6: Provide your answers

[0402] The server then provides the generated response to the user or customer via the user's or customer's terminal, sending a properly formatted response so that the user or customer can immediately obtain the appropriate information.

[0403] Input: Generated answer

[0404] Output: Provided to the user or customer

[0405] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0406] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0407] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0408] [Second embodiment]

[0409] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0410] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0411] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0412] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0413] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0414] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0415] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0416] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0417] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0418] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0419] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0420] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0421] This invention is a system in which generative artificial intelligence (AI) provides appropriate answers when confirmation matters arise based on the history of customer interactions. Detailed embodiments of this system are described below.

[0422] Overall overview

[0423] This system acquires, analyzes, and summarizes communication history with customers, and then uses AI to generate appropriate answers to user inquiries. It also provides an interface that allows customers to make inquiries directly, automating tasks that do not require the intervention of sales staff.

[0424] Main components of the system

[0425] Server: Collects and stores communication history with customers, receives and processes inquiries from users or customers.

[0426] Generative artificial intelligence (AI): Analyzes collected communication history, generates summaries, and generates appropriate answers based on user inquiries.

[0427] User terminal: A device through which a user accesses and queries the system, from which the user submits queries through the interface.

[0428] Customer terminal: A device that allows a customer to directly query the system.

[0429] Program processing (natural language explanation)

[0430] History capture and storage

[0431] The server stores email and chat history with customers in a database. This history includes customer identification information and the content of the interactions. For example, information such as "Order X was placed on 2023-01-01, and Order Y was placed on 2023-02-15" is stored as an interaction with a customer.

[0432] Generate a summary

[0433] The server uses natural language processing techniques to summarize key information using the accumulated historical data, preparing the summary for rapid response to subsequent queries.

[0434] Receiving and parsing queries

[0435] A user or customer sends a query through the interface to make an inquiry. The server receives the query, analyzes it based on the appropriate communication history, and sends it to the generative AI.

[0436] Generate and provide answers

[0437] The generative AI analyzes the received query and summarized historical data as prompts to generate the optimal answer. The server then provides the generated answer to the user or customer. For example, in response to the query "What is the status of my recent orders?", the generated answer is "I placed order Y on 2023-02-15."

[0438] Specific examples

[0439] Example 1: User usage

[0440] The user sends a query from the interface saying, "Please tell me the status of Customer A's recent orders." The server retrieves Customer A's historical data and sends it to the generative AI. The AI ​​generates the optimal answer based on the historical data and the query, providing the answer, "Customer A placed order Y on 2023-02-15."

[0441] Example 2: Direct customer inquiry

[0442] A customer submits a query saying, "I want to know the payment status of my recent invoices." The server retrieves the customer's payment history and sends it to the generative AI. The AI ​​generates an answer based on the historical data and the query, providing the answer, "Regarding recent invoices, invoice B was paid on 2023-02-20."

[0443] In this way, the system can respond quickly and accurately to customer and user inquiries. The distinctive features of this invention are the generation of a summary of the history and the generation of answers using generative AI, which can improve the efficiency and accuracy of customer responses.

[0444] The processing flow will be explained below.

[0445] Step 1:

[0446] The server collects past email and chat history with customers and stores it in a database, including customer identification information, dates, and specific exchanges.

[0447] Step 2:

[0448] The server periodically analyzes historical data and automatically generates summaries, using natural language processing techniques to extract key points and create concise summaries.

[0449] Step 3:

[0450] A user or customer makes an inquiry. A query is sent from the user or customer terminal through the interface. This query contains information that the user wants to know or a matter that needs confirmation.

[0451] Step 4:

[0452] The server analyzes queries received through the interface and searches for and retrieves historical data for the relevant customers, including, if necessary, retrieving the latest summaries from the database.

[0453] Step 5:

[0454] The server sends the acquired history data and the received query to a generative artificial intelligence (AI). The prompt sent to the AI ​​includes the summarized history data and the specific query content.

[0455] Step 6:

[0456] Generative artificial intelligence (AI) generates appropriate responses based on prompts. AI uses natural language processing techniques to create the most appropriate response to a query.

[0457] Step 7:

[0458] The server returns the answer received from the generative artificial intelligence (AI) to the user or customer, and the answer is displayed through the interface so that the user or customer can easily check it.

[0459] Step 8:

[0460] If the user or customer needs to make further inquiries or clarifications based on the answers they receive, they can send the query again, and the server will repeat the same process to continue providing appropriate answers.

[0461] Example 1

[0462] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0463] With conventional systems, it was difficult to efficiently utilize communication history with customers and respond quickly and accurately to inquiries from users and customers. Responding to inquiries also required a large amount of manpower, creating a demand for greater operational efficiency. Therefore, there is a need for a system that automates and efficiently performs operations while maintaining the quality of customer service.

[0464] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0465] In this invention, the server includes means for acquiring a communication history with a customer, means for saving the communication history with the customer, means for analyzing the communication history with the customer and summarizing important information, means for accepting an inquiry from a user or customer through an interface, means for searching and acquiring an appropriate communication history based on the received inquiry, means for generating an optimal answer based on the summary and the inquiry using generative artificial intelligence, and means for providing the generated answer to the user or customer, thereby enabling prompt and accurate responses to inquiries from customers or users.

[0466] "Customer communication history" refers to records of emails, chats, phone calls, and other interactions with customers.

[0467] A "server" is a computer system that provides functions such as data processing, storage, and network communication.

[0468] "Natural language processing technology" is a technology that uses computers to analyze, understand, and generate human language.

[0469] "Generative artificial intelligence (AI)" is artificial intelligence that has the ability to generate new information and answers based on input data.

[0470] An "enquiry" is a question or request that a user or customer enters into the system.

[0471] An "interface" is the means or method by which a user or customer interacts with a system.

[0472] A "query" is a question or search request issued to the system.

[0473] A "prompt" is input data or instructions given to generative artificial intelligence, which allows the AI ​​to generate an appropriate answer.

[0474] An "answer" is response information generated by a generative artificial intelligence based on a query.

[0475] The "summary" is a concise summary of important information extracted from the communication history with the customer.

[0476] This invention is a system in which generative artificial intelligence (AI) provides appropriate answers based on communication history with customers. The system of the present invention is specifically implemented as follows.

[0477] Overall structure

[0478] The main components of this system are a server, a generative AI, a user terminal, and a customer terminal. The server collects, stores, and analyzes data, and the generative AI generates appropriate answers. The user terminal and customer terminal provide the interface for users and customers to access the system and make inquiries.

[0479] Collecting and storing customer communication history

[0480] The server automatically collects communication history with customers, such as emails and chats, and stores it in a database. Standard server machines are used for the hardware and software used include the Python imaplib library and the Slack API.

[0481] Data analysis and summary generation

[0482] The server analyzes the stored historical data and summarizes key information using natural language processing (NLP) techniques, such as Python libraries nltk and spacy, to help generate the summaries.

[0483] Receiving inquiries

[0484] Users or customers access the interface through their terminals and send queries. The server receives these queries and searches and retrieves relevant historical data from the database. The interface is often implemented using popular web frameworks such as Flask and Django.

[0485] Generate answers

[0486] Generative artificial intelligence (AI) generates optimal answers based on the received query and summarized historical data. In this process, the query and related historical data are input as prompts into a generative AI model (e.g., OpenAI's GPT series).

[0487] Providing answers

[0488] The server provides the generated answer to the user or client terminal, which displays the answer through an interface.

[0489] Specific examples

[0490] Example 1: User usage

[0491] The user sends a query from the interface saying, "Please tell me the status of Customer A's recent orders." The server retrieves Customer A's historical data and sends it to the generative AI. The AI ​​generates the optimal answer based on the historical data and the query, providing the answer, "Customer A placed order Y on 2023-02-15."

[0492] Example 2: Direct customer inquiry

[0493] A customer submits a query saying, "I want to know the payment status of my recent invoices." The server retrieves the customer's payment history and sends it to the generative AI. The AI ​​generates an answer based on the historical data and the query, providing the answer, "Regarding recent invoices, invoice B was paid on 2023-02-20."

[0494] This system makes it possible to respond quickly and accurately to inquiries from customers and users. In addition, the system's unique features of this invention, such as the generation of summaries of communication history and the generation of answers using generative AI, make it possible to improve the efficiency and accuracy of customer support.

[0495] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0496] Step 1:

[0497] The server retrieves communication history with customers. The input is a data source such as a mail server or chat logs. The server retrieves unread emails and chat messages using Python's imaplib library or the Slack API. The output is the retrieved interaction data.

[0498] Specific behavior:

[0499] The server connects to the mail server and retrieves the unread emails.

[0500] The email body, sender information, and receipt date and time are obtained and stored as internal temporary data.

[0501] Step 2:

[0502] The server stores the acquired communication history with the customer in a database. The input is the acquired interaction data. The output is the communication history stored in the database. The database is typically MySQL or PostgreSQL.

[0503] Specific behavior:

[0504] Create a table in the database with columns such as customer ID, transaction text, date and time.

[0505] Store the temporary data in the database using the INSERT statement.

[0506] Step 3:

[0507] The server analyzes the historical data stored in the database and summarizes key information using natural language processing (NLP) techniques. The input is data from the database of customer communication history. The output is summarized text data.

[0508] Specific behavior:

[0509] Use the SELECT statement to retrieve customer history data from the database.

[0510] Use Python's nltk and spacy libraries to tokenize text and tag parts of speech.

[0511] Keyword extraction and summary generation algorithms are used to output important information as a summary.

[0512] Step 4:

[0513] A user or customer sends a query. The input is a query from the user terminal or customer terminal. The output is the query information received by the server, which processes it appropriately.

[0514] Specific behavior:

[0515] A query sent from a user terminal is received by a web framework (e.g., Flask).

[0516] Stores received queries as internal temporary data.

[0517] Step 5:

[0518] The server searches and retrieves the appropriate communication history based on the received query. The input is the user or customer query and the database. The output is the communication history data related to the query.

[0519] Specific behavior:

[0520] The keywords contained in the query are analyzed and search conditions are generated to retrieve appropriate historical data from the database.

[0521] Based on the search criteria, relevant data is retrieved from the database.

[0522] Step 6:

[0523] Generative AI generates optimal answers based on queries and related historical data. The input is the user or customer query and summarized historical data. The output is the generated answer.

[0524] Specific behavior:

[0525] The server sends queries and historical data as prompts to the generative AI model.

[0526] The generative AI model generates an answer and sends it back to the server.

[0527] Step 7:

[0528] The server provides the generated answer to the user or client terminal. The input is the generated answer. The output is the answer displayed on the interface.

[0529] Specific behavior:

[0530] The server transmits the generated answer to the user or client terminal.

[0531] The answers are displayed in the interface for the user or customer to review.

[0532] (Application example 1)

[0533] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0534] Customer service in brick-and-mortar stores requires providing information quickly and accurately, but traditional methods make it difficult for staff to grasp all product information and customer history, resulting in problems that lead to lower customer satisfaction and reduced productivity. Particularly in busy store environments, staff lack the support they need to quickly provide appropriate answers.

[0535] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0536] In this invention, the server includes: means for acquiring a communication history with a customer; means for analyzing the communication history with the customer and generating a summary; means for utilizing generative artificial intelligence (AI) to generate an answer to a user's inquiry based on the summary; means for providing the generated answer to the user or customer; and means for receiving an inquiry via an information display device operated by a user in a store and presenting the generated answer to the user in real time. This enables staff to respond to customer inquiries quickly and accurately, even in a physical store, thereby improving customer satisfaction and business efficiency.

[0537] "Customer communication history" refers to a record of all interactions, inquiries, purchase history, etc. that a customer has with a company or service provider.

[0538] A "summary" is information that extracts important information from the communication history with a customer and summarizes it concisely.

[0539] "Generative artificial intelligence (AI)" is an AI that has the ability to generate appropriate answers to user inquiries based on collected data.

[0540] An "information display device" is a device worn or used by a user, which displays answers to inquiries in real time.

[0541] The term "user" refers to an employee such as a store staff member who operates the system of the present invention and deals with customers.

[0542] A "server" is a computer system that collects, stores, analyzes, and generates answers using generative artificial intelligence (AI).

[0543] "Interface" refers to the means or screen through which a user or customer makes inquiries to the system.

[0544] A "prompt" is data that provides the underlying information or conditions that a generative artificial intelligence (AI) uses to generate an appropriate answer.

[0545] The present invention provides a system that enables store staff to use smart glasses or other information display devices to quickly and accurately interact with customers. Detailed embodiments of the system are described below.

[0546] Overall overview

[0547] This system acquires and analyzes communication history with customers, and then uses generative artificial intelligence (AI) to summarize the results.The user (store staff) can then provide appropriate answers to customer inquiries in real time via an information display device.

[0548] System Components

[0549] server

[0550] The server collects, stores, analyzes, and summarizes the communication history with the customer, which includes the following steps:

[0551] 1. History capture and storage:

[0552] The server stores email and chat history with customers in a database, preferably using MySQL or similar database.

[0553] 2. Generate a summary:

[0554] The server uses natural language processing technologies such as Python's NLTK and Gensim to analyze the accumulated historical data and summarize important information.

[0555] Generative artificial intelligence (AI)

[0556] 3. Receiving and parsing queries:

[0557] The server analyzes queries received through smart glasses or information display devices and searches for appropriate communication history.

[0558] 4. Generate and provide answers:

[0559] The server uses a generative AI model (e.g., GPT-3) to generate the optimal answer based on the analysis results, which is then processed using deep learning libraries such as TensorFlow and PyTorch.

[0560] The server presents this answer in real time to the user's smart glasses or information display device.

[0561] User terminal

[0562] The user terminal is a device worn by the user, such as smart glasses, that displays the generated answers in real time.

[0563] Specific examples

[0564] Example 1: Customer Inquiry

[0565] A customer asks, "Are there any other products similar to this one?" The user sends this inquiry to the server via their smart glasses. The server retrieves the customer's past purchase history and inquiry history and sends it to the generative AI. The generative AI analyzes this and generates an answer such as, "Products B and C are similar to this product." This answer is displayed in real time on the user's smart glasses.

[0566] Prompt Sentence Examples

[0567] Here is an example of a prompt for a generative AI model:

[0568] User query: "Are there any other products like this?"

[0569] Past purchase history: Product A (2023-01-10), Product B (2023-02-15)

[0570] Past inquiry history: "I'd like to know about similar products" "Please recommend some products"

[0571] Example of an answer it might generate: "Products similar to this one are Product B and Product C."

[0572] This system will enable staff at physical stores to respond quickly and accurately to a wide range of customer questions, improving customer satisfaction and operational efficiency.

[0573] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0574] Step 1:

[0575] A user receives a customer inquiry via smart glasses.

[0576] Input: Customer inquiry (e.g., "Do you have anything similar to this product?")

[0577] Output: Query data sent to the server

[0578] How it works: The user speaks their inquiry into the microphone of the smart glasses, which is then converted into text data and sent to the server. The voice recognition uses the Google Speech-to-Text API and other tools.

[0579] Step 2:

[0580] The server analyzes the received query.

[0581] Input: Query data sent by the user (a textual query)

[0582] Output: The parsed results of the query

[0583] Specific operation: The server analyzes the received query using natural language processing techniques (such as NLTK or SpaCy) and prepares it to query the database in an appropriate manner. Typical analysis includes understanding the meaning of the query and extracting keywords.

[0584] Step 3:

[0585] The server searches and retrieves the customer's communication history.

[0586] Input: query analysis results, customer identification information

[0587] Output: Customer communication history data

[0588] Specific operation: Based on the parsed query and customer identification information, the server searches the database (e.g., MySQL) for relevant communication history and retrieves the required data, including past purchase history and inquiry history.

[0589] Step 4:

[0590] Summarize historical data obtained by the server.

[0591] Input: Customer communication history data

[0592] Output: Summarized historical data

[0593] How it works: The server uses Python libraries such as NLTK and Gensim to extract key information from the acquired historical data and generate summaries, which are then used as prompts for the generative AI model.

[0594] Step 5:

[0595] The server sends prompts to the generative AI based on the summary data and the query.

[0596] Input: summarized historical data, query text

[0597] Output: prompt to generative AI

[0598] How it works: The server combines the summarized history data with the parsed query and formats it as a prompt for a generative AI model (e.g., GPT-3). Examples of prompts include: "User query: 'Are there any other products similar to this one?'," "Past purchase history: Product A (2023-01-10), Product B (2023-02-15)," and "Past inquiry history: 'I'd like to know about similar products,' and 'What products do you recommend?'" These are then sent to the generative AI model.

[0599] Step 6:

[0600] A generative AI model generates appropriate answers.

[0601] Input: prompt to generative AI

[0602] Output: Generated answer (text format)

[0603] How it works: A generative AI model (e.g., GPT-3) generates the best answer based on the prompt provided, using natural language processing and deep learning techniques in the process.

[0604] Step 7:

[0605] The server sends the generated answer to the user's smart glasses.

[0606] Input: Generated answer (text format)

[0607] Output: Answer displayed on smart glasses

[0608] Specific operation: The server sends the generated answer to the user's smart glasses in real time, allowing the user to visually confirm it. The generated answer is displayed on the display of the smart glasses.

[0609] Step 8:

[0610] The user provides the answer to the customer.

[0611] Input: Answer displayed on smart glasses

[0612] Output: Providing a verbal response to the customer

[0613] Specific operation: The user reads out the answer displayed on the smart glasses and provides the answer to the customer verbally, allowing the customer to obtain the information quickly and accurately.

[0614] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0615] The present invention combines a system that analyzes communication history with customers and generates summaries with an emotion engine that recognizes the user's emotions. This system takes into account the user's emotional state and can generate more appropriate and effective responses. An embodiment of this system will be described in detail below.

[0616] Main components of the system

[0617] Server: Collects and stores communication history with customers, receives and processes inquiries from users or customers.

[0618] Generative artificial intelligence (AI): Analyzes collected historical data, generates summaries, and generates appropriate answers based on user queries.

[0619] User terminal: A device through which a user accesses and queries the system.

[0620] Customer terminal: A device that allows a customer to directly query the system.

[0621] Emotion engine: Analyzes the emotional state of the user's query and adjusts the generated answer based on the results.

[0622] Program processing (natural language explanation)

[0623] History capture and storage

[0624] The server collects and stores in a database the history of past emails and chats with customers, including the customer's identity and the content of those conversations.

[0625] Generate a summary

[0626] The server uses natural language processing techniques to summarize key information using the accumulated historical data, preparing the summary for rapid response to subsequent queries.

[0627] Query Reception and Emotion Recognition

[0628] A user or customer makes an inquiry. The query is sent from the user terminal or customer terminal through the interface. The server analyzes the query received through the interface and analyzes the user's emotional state using an emotion engine.

[0629] Emotion-Based Adjustment

[0630] The emotion engine analyzes the user's emotional state and dynamically changes the prompts sent to the generative artificial intelligence (AI) based on the results. For example, if the user is angry, the prompts will be set to generate responses with a more polite and calm tone.

[0631] Generate and provide answers

[0632] The generative AI generates an appropriate answer based on the prompts adjusted by the emotion engine. The server then provides the generated answer to the user or customer. For example, if the user is feeling anxious in response to the query "Please tell me the status of my recent order," the server will provide an answer in the form of "Sorry to keep you waiting. I will explain the status of my recent order. ..."

[0633] Specific examples

[0634] Example 1: User usage

[0635] The user sends a query from the interface saying, "Please tell me the status of Customer A's recent orders." The server retrieves Customer A's historical data and analyzes the user's emotional state using the emotion engine. The generative AI uses a prompt adjusted based on the analysis results of the emotion engine to generate and provide the answer, "Customer A placed order Y on 2023-02-15 as their most recent order."

[0636] Example 2: Direct customer inquiry

[0637] A customer submits a query saying, "I want to know the payment status of my recent invoices." The server retrieves the customer's payment history and analyzes their emotional state using an emotion engine. The generative AI uses a prompt tailored based on the analysis results of the emotion engine to generate and provide an answer such as, "Regarding recent invoices, I paid invoice B on 2023-02-20."

[0638] The role of the emotional engine

[0639] By analyzing the user's emotional state, the emotion engine can adjust the tone and content of responses to deliver a more personalized experience, improving customer satisfaction and building better customer relationships.

[0640] In this way, by combining emotion engines, it is possible to achieve more advanced and flexible customer response than conventional automated response systems.

[0641] The processing flow will be explained below.

[0642] Step 1:

[0643] The server collects past email and chat history with the customer and stores it in a database. This data includes customer identification information, the date of the interaction, and the specific content. For example, it stores information that "Customer A" placed "Order X" on "2023-01-01."

[0644] Step 2:

[0645] The server uses natural language processing techniques to extract key information from the stored historical data and generate a summary, which serves as a preliminary step for quickly responding to future inquiries.

[0646] Step 3:

[0647] A user or customer submits a query through the interface to make an inquiry. For example, a customer might type, "I want to know the status of my recent bill payment."

[0648] Step 4:

[0649] The server analyzes queries received through the interface and retrieves the relevant customer history data from the database, ensuring that the relevant data is up to date.

[0650] Step 5:

[0651] The server sends the acquired historical data and the query to the emotion engine, which analyzes the user's emotional state. For example, it determines whether the user is feeling "anxious" or "angry" based on the query's style and wording.

[0652] Step 6:

[0653] The emotion engine analyzes the user's emotional state and sends the results back to the server. For example, the analysis result may say, "The user is feeling anxious."

[0654] Step 7:

[0655] The server then adjusts the prompts sent to the generative artificial intelligence (AI) based on the emotion engine's analysis, for example, "If you're feeling anxious, add more detailed explanations and reassuring words."

[0656] Step 8:

[0657] Generative artificial intelligence (AI) generates appropriate answers based on tailored prompts, such as, "Regarding your recent invoices, you paid invoice B on 2023-02-20. Please rest assured about the details."

[0658] Step 9:

[0659] The server provides the generated answer to the user or customer. The answer is displayed through the interface so that the user or customer can easily check it. By displaying it on the terminal, the user can quickly obtain an appropriate answer to their inquiry.

[0660] Step 10:

[0661] If the user or customer is not satisfied with the answer they received and needs further inquiry or confirmation, they can send the query again, and the server will repeat the same process and continue to generate the best answer.

[0662] In this way, a system incorporating an emotion engine can provide more flexible and accurate customer service while taking into account the user's emotional state.

[0663] Example 2

[0664] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0665] Conventional customer service systems provide uniform responses without considering the emotional state of the user or customer, making it difficult to improve customer satisfaction. Furthermore, responses that ignore the emotional state can exacerbate customer dissatisfaction and anxiety, risking a deterioration in customer relationships. The challenge with conventional systems is the difficulty of properly assessing the emotional state of the user or customer and providing more personalized responses.

[0666] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for acquiring a communication history with a customer; means for saving the communication history with the customer; means for analyzing the communication history with the customer and generating a summary; means for a user or customer to send an inquiry; means for searching and acquiring an appropriate communication history based on the received inquiry; means for receiving the inquiry; means for using an emotion engine including means for analyzing the emotional state of the user in addition to the summary; means for adjusting a prompt to be sent to a generative artificial intelligence (AI) based on the emotional state of the user; means for utilizing the generative artificial intelligence (AI) to generate an appropriate answer based on the adjusted prompt; and means for providing the generated answer to the user or customer. This enables more personalized responses that take into account the emotional state of the user or customer.

[0667] "Customer communication history" refers to exchanges between customers and companies, such as emails, chats, phone recordings, and social media messages.

[0668] "Database" refers to a structured collection of data for managing and storing collected information.

[0669] "Natural language processing technology" refers to technologies and methods for understanding, analyzing, and generating human language.

[0670] An "emotion engine" refers to a software component that analyzes the emotional state of a user or customer (e.g., joy, anger, anxiety, etc.) from text data.

[0671] "Generative artificial intelligence (AI)" refers to artificial intelligence that automatically generates documents and information that humans can understand and use in natural language based on given prompts.

[0672] A "prompt" refers to input information that instructs a generative artificial intelligence (AI) on what kind of answer it should generate.

[0673] "User's emotional state" refers to the emotional state (e.g., joy, anger, anxiety, etc.) of the user when they input a query.

[0674] "Interface" refers to the device or software mechanism through which a user or customer accesses the system and sends and receives inquiries.

[0675] A "summary" is a concise summary of complex data or documents, extracting only the most important information.

[0676] "Enquiry" means a question or request made by a user or customer to the system.

[0677] The present invention combines a system that analyzes communication history with customers and generates summaries with an emotion engine that recognizes the user's emotions. This system takes the user's emotional state into account and generates more appropriate and effective answers.

[0678] Main components of the system

[0679] Server: Collects and stores customer communication history, receives and processes user or customer inquiries. Examples: AWS EC2 instances, Google Cloud Compute Engine.

[0680] Database: A system for storing and managing collected historical data. Examples: MySQL, PostgreSQL.

[0681] Generative Artificial Intelligence (AI): Analyzes collected historical data, generates summaries, and generates appropriate answers based on user queries. Examples: OpenAI GPT-3, Google BERT.

[0682] User terminal: A device through which a user accesses and queries the system. Examples: web browser, smartphone app.

[0683] Customer terminal: A device that allows a customer to directly query the system.

[0684] Emotion engine: Analyzes the emotional state of the user's query and adjusts the generated answer accordingly. Examples: IBM Watson, Azure Text Analytics.

[0685] Program processing

[0686] History capture and storage

[0687] The server collects past communication history with the customer using the specified mailbox or chat service API (e.g., Gmail API or Slack API). The collected data is stored in a database (e.g., MySQL or PostgreSQL) along with the customer's identification information and metadata.

[0688] Generate a summary

[0689] The server analyzes the accumulated historical data using natural language processing (NLP) techniques (e.g., NLTK, SpaCy), extracts important information, and generates summaries, which are then stored in a database as a preliminary step for responding quickly to future inquiries.

[0690] Receiving a query

[0691] Users or customers make inquiries from a web browser or smartphone app. The inquiries are sent to the server as HTTP requests, and the server receives them and stores them in a database in text format.

[0692] Emotion recognition

[0693] The server sends the text data of the received query to an emotion engine (e.g., IBM Watson NLP API, Azure Cognitive Services) to analyze the user's emotional state. The emotion engine returns a score to the server, such as "anxiety: 0.75, joy: 0.10, anger: 0.15."

[0694] Emotion-Based Adjustment

[0695] The server adjusts the prompts sent to the generative AI based on the emotion score received from the emotion engine. For example, if the anxiety score is high, the server generates a prompt that reads, "The user is feeling anxious. Please generate a reassuring response."

[0696] Generate and provide answers

[0697] The generative AI receives the tailored prompt and generates an appropriate answer based on it. For example, in response to the query "What is the status of Customer A's recent orders?", it generates the answer "Customer A recently placed order Y on 2023-02-15." The server then provides the generated answer to the user or customer through the same interface that received the query.

[0698] Specific examples

[0699] Example 1: User usage

[0700] When a user sends a query from the interface saying, "Please tell me the status of Customer A's recent orders," the server retrieves Customer A's historical data and analyzes the user's emotional state using the emotion engine. The generative AI uses a prompt adjusted based on the analysis results of the emotion engine to generate and provide the answer, "Customer A placed order Y on 2023-02-15 as their most recent order."

[0701] Example 2: Direct customer inquiry

[0702] When a customer sends a query such as "I want to know the payment status of my recent invoices," the server retrieves the customer's payment history and analyzes their emotional state using an emotion engine. The generative AI uses prompts tailored based on the analysis results of the emotion engine to generate and provide an answer such as "Regarding recent invoices, I paid invoice B on 2023-02-20."

[0703] Prompt Sentence Examples

[0704] Emotional state is anxious: "The user is feeling anxious. Please generate a polite and reassuring response."

[0705] Angry emotional state: "The user is angry. Please set a calm and measured tone in your response."

[0706] In this way, by combining emotion engines, it is possible to achieve more advanced and flexible customer response than conventional automated response systems.

[0707] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0708] Step 1: Collecting History

[0709] The server collects communication history with customers. As input, it obtains data from mailboxes and chat service APIs (e.g., Gmail API, Slack API). The specific process for collecting data involves sending queries to the API and analyzing the returned data. Through this operation, the contents of customer emails and chats are collected on the server.

[0710] Step 2: Save history

[0711] The server saves the collected customer communication history in a database. The input is the data collected in step 1. The server generates a document containing the customer's identification information and metadata and saves it in a database (e.g., MySQL, PostgreSQL). This operation records the customer's communication history in the database.

[0712] Step 3: Generate a summary

[0713] The server analyzes the accumulated historical data using natural language processing (NLP) techniques (e.g., NLTK, SpaCy). The input is the data saved in step 2. The server extracts important information and performs text analysis and semantic extraction as specific processes to generate a summary. The summary generated as output is saved in a database. This operation adds the summary information to the database.

[0714] Step 4: Receiving a query

[0715] A user or a client sends a query through the interface. The input is a query text from the terminal. The server receives the query as an HTTP request and stores it in a database in text format. This operation allows the server to manage the received queries.

[0716] Step 5: Recognize emotions

[0717] The server sends the text data of the received query to an emotion engine (e.g., IBM Watson NLP API, Azure Cognitive Services). The input is the query text received in step 4. The emotion engine performs specific processing to analyze the user's emotional state from the text. As an output, an emotion score (e.g., "Anxiety: 0.75, Joy: 0.10, Anger: 0.15") is returned to the server. This operation analyzes the user's emotional state.

[0718] Step 6: Emotional Adjustment

[0719] The server adjusts the prompt based on the emotion score received from the emotion engine. The input is the emotion score obtained in step 5 and the customer summary information stored in the database. The server uses the emotion score to dynamically generate a prompt to send to the generative AI (e.g., OpenAI GPT-3, Google BERT). For example, a prompt might be generated that reads, "The user is feeling anxious. Please generate a polite and reassuring response." This operation improves the accuracy of the generated response.

[0720] Step 7: Generate and serve answers

[0721] The generative AI receives the adjusted prompt and generates an appropriate answer based on it. The input is the prompt adjusted in step 6. The generative AI performs the specific process of generating an answer according to the prompt and returns the answer as output to the server. The server sends the generated answer to the user or customer. The answer is provided through the same interface that received the inquiry. This operation provides an appropriate answer to the user or customer.

[0722] The above is a specific description of the processing steps in the system.

[0723] (Application example 2)

[0724] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0725] Conventional automated response systems generate responses without considering the emotional state of the customer, limiting their ability to improve customer satisfaction. Furthermore, they lacked the processes for summarizing information and analyzing emotions to provide prompt and appropriate responses, making it difficult to provide effective customer support.

[0726] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0727] In this invention, the server includes means for acquiring a communication history with a customer, means for analyzing the communication history with the customer and generating a summary, means for utilizing an emotion engine that analyzes the emotional states of the user and the customer, means for utilizing generative artificial intelligence (AI) that generates an answer to the user's inquiry based on the summary and the emotion analysis results, and means for providing the generated answer to the user or customer. This makes it possible to provide a quick and accurate answer that takes into account the emotional state of the customer.

[0728] "Customer communication history" refers to past conversation data, such as emails and chats, between the customer and the system.

[0729] A "summary" is a concise summary of important information extracted from the collected communication history.

[0730] An "emotion engine" refers to a system that analyzes emotions from input sentences of a user or customer and recognizes their emotional state.

[0731] "Generative artificial intelligence (AI)" refers to artificial intelligence that uses natural language processing technology to generate appropriate answers based on user inquiries.

[0732] "Interface" refers to the screen or device through which a user or customer submits a query to the system.

[0733] A "prompt" refers to the input information that generative artificial intelligence (AI) uses to generate an appropriate answer.

[0734] "Answer" means a response generated and provided by a generative artificial intelligence (AI) in response to a user or customer inquiry.

[0735] To implement this invention, a system consisting of the following means is required. The main components of the system are a server, a user terminal, a customer terminal, an emotion engine, and a generative artificial intelligence (AI).

[0736] Main components of the system

[0737] server

[0738] The server has a database that stores and manages past communication history with customers, and also provides the computing resources to analyze inquiries and generate appropriate answers.

[0739] User and customer terminals

[0740] These terminals refer to devices that users or customers use to access the system and send inquiries, such as PCs, smartphones, and tablets.

[0741] Emotion Engine

[0742] The emotion engine uses the Samsung Emotional Engine API and other tools to analyze emotions from received text.

[0743] Generative artificial intelligence (AI)

[0744] As generative artificial intelligence (AI), we use advanced natural language generation models such as GPT-3.

[0745] System Overview

[0746] The server acquires past communication history and stores it in a database. When a user or customer sends an inquiry, the server receives the inquiry. After receiving the inquiry, the server uses an emotion engine to analyze emotions from the inquiry text.

[0747] Based on the analysis results of the emotion engine, the server uses generative artificial intelligence (AI) to generate an appropriate answer. When the generative artificial intelligence (AI) generates an answer, it adjusts the prompt based on the analysis results of the emotion engine to provide a more appropriate answer.

[0748] Specific examples

[0749] For example, consider a food delivery service application. If a customer asks, "Why hasn't my order arrived yet?", the server receives the inquiry. If the emotion engine detects "anger," the generative artificial intelligence (AI) can generate a response using prompts like the following:

[0750] Example prompt sentence:

[0751] Customer: "My order hasn't arrived yet."

[0752] Emotion analysis result: "Anger"

[0753] Generative AI prompt: "The customer seems upset. Please generate a response that explains the delay in an appropriate and polite tone."

[0754] The appropriate response generated by this would be something like, "We're very sorry. We're currently checking the tracking information for your order, so please wait a moment." This system makes it possible to provide quick and accurate responses that take into account the customer's feelings.

[0755] Software and hardware used

[0756] Hardware

[0757] Server (e.g. AWS EC2), user device (e.g. smartphone, tablet), customer device (e.g. PC, smartphone)

[0758] software

[0759] Emotion engines (e.g., Samsung Emotional Engine API), generative artificial intelligence (e.g., GPT-3), database management systems (e.g., MySQL)

[0760] This configuration makes it possible to provide flexible and accurate automatic responses that respond to customer emotions when dealing with customers in food delivery services and the like.

[0761] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0762] Step 1: Capture and save history

[0763] The server retrieves and stores the customer's past communication history from the database. Specifically, the server collects data such as emails, chats, and order history, and stores it in a database (e.g., MySQL). At this time, the customer's identification information and the content of the communication are stored in the database.

[0764] Input: Email, chat, order history

[0765] Output: Communication history stored in a database

[0766] Step 2: Receiving and analyzing inquiries

[0767] The server receives queries from users or customers. The queries are sent from the user or customer terminal and received by the server through an interface. The received queries are analyzed by an emotion engine (e.g., Samsung Emotional Engine API) to identify the emotional state.

[0768] Input: User or customer inquiry

[0769] Output: Emotion analysis results

[0770] Step 3: History search and summary generation

[0771] The server searches and retrieves relevant historical data from the database based on the sentiment analysis results and the query. The retrieved historical data is summarized using natural language processing techniques (e.g., BERT). The summary extracts only the important information and summarizes it concisely. This information is used in subsequent processing steps.

[0772] Input: Sentiment analysis results, query text

[0773] Output: Summarized historical data

[0774] Step 4: Generate and refine prompts

[0775] The server generates prompts based on the emotion analysis and summarized historical data, which are then sent to a generative artificial intelligence (AI). The generated prompts are tailored based on the emotion analysis. For example, if the user's emotion is anger, a prompt that responds in a polite tone is set.

[0776] Input: Sentiment analysis results, summarized historical data

[0777] Output: Adjusted prompt

[0778] Step 5: Generate an answer

[0779] Generative artificial intelligence (AI) generates appropriate responses based on tailored prompts. Specifically, an AI model (e.g., GPT-3) analyzes the prompts and generates the best possible response for the user or customer. This model is pre-trained to generate natural-sounding sentences with a high degree of accuracy.

[0780] Input: Adjusted prompt

[0781] Output: Generated answer

[0782] Step 6: Provide your answers

[0783] The server then provides the generated response to the user or customer via the user's or customer's terminal, sending a properly formatted response so that the user or customer can immediately obtain the appropriate information.

[0784] Input: Generated answer

[0785] Output: Provided to the user or customer

[0786] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0787] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0788] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0789] [Third embodiment]

[0790] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0791] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0792] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0793] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0794] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0795] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0796] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0797] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0798] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0799] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0800] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0801] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0802] This invention is a system in which generative artificial intelligence (AI) provides appropriate answers when confirmation matters arise based on the history of customer interactions. Detailed embodiments of this system are described below.

[0803] Overall overview

[0804] This system acquires, analyzes, and summarizes communication history with customers, and then uses AI to generate appropriate answers to user inquiries. It also provides an interface that allows customers to make inquiries directly, automating tasks that do not require the intervention of sales staff.

[0805] Main components of the system

[0806] Server: Collects and stores communication history with customers, receives and processes inquiries from users or customers.

[0807] Generative artificial intelligence (AI): Analyzes collected communication history, generates summaries, and generates appropriate answers based on user inquiries.

[0808] User terminal: A device through which a user accesses and queries the system, from which the user submits queries through the interface.

[0809] Customer terminal: A device that allows a customer to directly query the system.

[0810] Program processing (natural language explanation)

[0811] History capture and storage

[0812] The server stores email and chat history with customers in a database. This history includes customer identification information and the content of the interactions. For example, information such as "Order X was placed on 2023-01-01, and Order Y was placed on 2023-02-15" is stored as an interaction with a customer.

[0813] Generate a summary

[0814] The server uses natural language processing techniques to summarize key information using the accumulated historical data, preparing the summary for rapid response to subsequent queries.

[0815] Receiving and parsing queries

[0816] A user or customer sends a query through the interface to make an inquiry. The server receives the query, analyzes it based on the appropriate communication history, and sends it to the generative AI.

[0817] Generate and provide answers

[0818] The generative AI analyzes the received query and summarized historical data as prompts to generate the optimal answer. The server then provides the generated answer to the user or customer. For example, in response to the query "What is the status of my recent orders?", the generated answer is "I placed order Y on 2023-02-15."

[0819] Specific examples

[0820] Example 1: User usage

[0821] The user sends a query from the interface saying, "Please tell me the status of Customer A's recent orders." The server retrieves Customer A's historical data and sends it to the generative AI. The AI ​​generates the optimal answer based on the historical data and the query, providing the answer, "Customer A placed order Y on 2023-02-15."

[0822] Example 2: Direct customer inquiry

[0823] A customer submits a query saying, "I want to know the payment status of my recent invoices." The server retrieves the customer's payment history and sends it to the generative AI. The AI ​​generates an answer based on the historical data and the query, providing the answer, "Regarding recent invoices, invoice B was paid on 2023-02-20."

[0824] In this way, the system can respond quickly and accurately to customer and user inquiries. The distinctive features of this invention are the generation of a summary of the history and the generation of answers using generative AI, which can improve the efficiency and accuracy of customer responses.

[0825] The processing flow will be explained below.

[0826] Step 1:

[0827] The server collects past email and chat history with customers and stores it in a database, including customer identification information, dates, and specific exchanges.

[0828] Step 2:

[0829] The server periodically analyzes historical data and automatically generates summaries, using natural language processing techniques to extract key points and create concise summaries.

[0830] Step 3:

[0831] A user or customer makes an inquiry. A query is sent from the user or customer terminal through the interface. This query contains information that the user wants to know or a matter that needs confirmation.

[0832] Step 4:

[0833] The server analyzes queries received through the interface and searches for and retrieves historical data for the relevant customers, including, if necessary, retrieving the latest summaries from the database.

[0834] Step 5:

[0835] The server sends the acquired history data and the received query to a generative artificial intelligence (AI). The prompt sent to the AI ​​includes the summarized history data and the specific query content.

[0836] Step 6:

[0837] Generative artificial intelligence (AI) generates appropriate responses based on prompts. AI uses natural language processing techniques to create the most appropriate response to a query.

[0838] Step 7:

[0839] The server returns the answer received from the generative artificial intelligence (AI) to the user or customer, and the answer is displayed through the interface so that the user or customer can easily check it.

[0840] Step 8:

[0841] If the user or customer needs to make further inquiries or clarifications based on the answers they receive, they can send the query again, and the server will repeat the same process to continue providing appropriate answers.

[0842] Example 1

[0843] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0844] With conventional systems, it was difficult to efficiently utilize communication history with customers and respond quickly and accurately to inquiries from users and customers. Responding to inquiries also required a large amount of manpower, creating a demand for greater operational efficiency. Therefore, there is a need for a system that automates and efficiently performs operations while maintaining the quality of customer service.

[0845] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0846] In this invention, the server includes means for acquiring a communication history with a customer, means for saving the communication history with the customer, means for analyzing the communication history with the customer and summarizing important information, means for accepting an inquiry from a user or customer through an interface, means for searching and acquiring an appropriate communication history based on the received inquiry, means for generating an optimal answer based on the summary and the inquiry using generative artificial intelligence, and means for providing the generated answer to the user or customer, thereby enabling prompt and accurate responses to inquiries from customers or users.

[0847] "Customer communication history" refers to records of emails, chats, phone calls, and other interactions with customers.

[0848] A "server" is a computer system that provides functions such as data processing, storage, and network communication.

[0849] "Natural language processing technology" is a technology that uses computers to analyze, understand, and generate human language.

[0850] "Generative artificial intelligence (AI)" is artificial intelligence that has the ability to generate new information and answers based on input data.

[0851] An "enquiry" is a question or request that a user or customer enters into the system.

[0852] An "interface" is the means or method by which a user or customer interacts with a system.

[0853] A "query" is a question or search request issued to the system.

[0854] A "prompt" is input data or instructions given to generative artificial intelligence, which allows the AI ​​to generate an appropriate answer.

[0855] An "answer" is response information generated by a generative artificial intelligence based on a query.

[0856] The "summary" is a concise summary of important information extracted from the communication history with the customer.

[0857] This invention is a system in which generative artificial intelligence (AI) provides appropriate answers based on communication history with customers. The system of the present invention is specifically implemented as follows.

[0858] Overall structure

[0859] The main components of this system are a server, a generative AI, a user terminal, and a customer terminal. The server collects, stores, and analyzes data, and the generative AI generates appropriate answers. The user terminal and customer terminal provide the interface for users and customers to access the system and make inquiries.

[0860] Collecting and storing customer communication history

[0861] The server automatically collects communication history with customers, such as emails and chats, and stores it in a database. Standard server machines are used for the hardware and software used include the Python imaplib library and the Slack API.

[0862] Data analysis and summary generation

[0863] The server analyzes the stored historical data and summarizes key information using natural language processing (NLP) techniques, such as Python libraries nltk and spacy, to help generate the summaries.

[0864] Receiving inquiries

[0865] Users or customers access the interface through their terminals and send queries. The server receives these queries and searches and retrieves relevant historical data from the database. The interface is often implemented using popular web frameworks such as Flask and Django.

[0866] Generate answers

[0867] Generative artificial intelligence (AI) generates optimal answers based on the received query and summarized historical data. In this process, the query and related historical data are input as prompts into a generative AI model (e.g., OpenAI's GPT series).

[0868] Providing answers

[0869] The server provides the generated answer to the user or client terminal, which displays the answer through an interface.

[0870] Specific examples

[0871] Example 1: User usage

[0872] The user sends a query from the interface saying, "Please tell me the status of Customer A's recent orders." The server retrieves Customer A's historical data and sends it to the generative AI. The AI ​​generates the optimal answer based on the historical data and the query, providing the answer, "Customer A placed order Y on 2023-02-15."

[0873] Example 2: Direct customer inquiry

[0874] A customer submits a query saying, "I want to know the payment status of my recent invoices." The server retrieves the customer's payment history and sends it to the generative AI. The AI ​​generates an answer based on the historical data and the query, providing the answer, "Regarding recent invoices, invoice B was paid on 2023-02-20."

[0875] This system makes it possible to respond quickly and accurately to inquiries from customers and users. In addition, the system's unique features of this invention, such as the generation of summaries of communication history and the generation of answers using generative AI, make it possible to improve the efficiency and accuracy of customer support.

[0876] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0877] Step 1:

[0878] The server retrieves communication history with customers. The input is a data source such as a mail server or chat logs. The server retrieves unread emails and chat messages using Python's imaplib library or the Slack API. The output is the retrieved interaction data.

[0879] Specific behavior:

[0880] The server connects to the mail server and retrieves the unread emails.

[0881] The email body, sender information, and receipt date and time are obtained and stored as internal temporary data.

[0882] Step 2:

[0883] The server stores the acquired communication history with the customer in a database. The input is the acquired interaction data. The output is the communication history stored in the database. The database is typically MySQL or PostgreSQL.

[0884] Specific behavior:

[0885] Create a table in the database with columns such as customer ID, transaction text, date and time.

[0886] Store the temporary data in the database using the INSERT statement.

[0887] Step 3:

[0888] The server analyzes the historical data stored in the database and summarizes key information using natural language processing (NLP) techniques. The input is data from the database of customer communication history. The output is summarized text data.

[0889] Specific behavior:

[0890] Use the SELECT statement to retrieve customer history data from the database.

[0891] Use Python's nltk and spacy libraries to tokenize text and tag parts of speech.

[0892] Keyword extraction and summary generation algorithms are used to output important information as a summary.

[0893] Step 4:

[0894] A user or customer sends a query. The input is a query from the user terminal or customer terminal. The output is the query information received by the server, which processes it appropriately.

[0895] Specific behavior:

[0896] A query sent from a user terminal is received by a web framework (e.g., Flask).

[0897] Stores received queries as internal temporary data.

[0898] Step 5:

[0899] The server searches and retrieves the appropriate communication history based on the received query. The input is the user or customer query and the database. The output is the communication history data related to the query.

[0900] Specific behavior:

[0901] The keywords contained in the query are analyzed and search conditions are generated to retrieve appropriate historical data from the database.

[0902] Based on the search criteria, relevant data is retrieved from the database.

[0903] Step 6:

[0904] Generative AI generates optimal answers based on queries and related historical data. The input is the user or customer query and summarized historical data. The output is the generated answer.

[0905] Specific behavior:

[0906] The server sends queries and historical data as prompts to the generative AI model.

[0907] The generative AI model generates an answer and sends it back to the server.

[0908] Step 7:

[0909] The server provides the generated answer to the user or client terminal. The input is the generated answer. The output is the answer displayed on the interface.

[0910] Specific behavior:

[0911] The server transmits the generated answer to the user or client terminal.

[0912] The answers are displayed in the interface for the user or customer to review.

[0913] (Application example 1)

[0914] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0915] Customer service in brick-and-mortar stores requires providing information quickly and accurately, but traditional methods make it difficult for staff to grasp all product information and customer history, resulting in problems that lead to lower customer satisfaction and reduced productivity. Particularly in busy store environments, staff lack the support they need to quickly provide appropriate answers.

[0916] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0917] In this invention, the server includes: means for acquiring a communication history with a customer; means for analyzing the communication history with the customer and generating a summary; means for utilizing generative artificial intelligence (AI) to generate an answer to a user's inquiry based on the summary; means for providing the generated answer to the user or customer; and means for receiving an inquiry via an information display device operated by a user in a store and presenting the generated answer to the user in real time. This enables staff to respond to customer inquiries quickly and accurately, even in a physical store, thereby improving customer satisfaction and business efficiency.

[0918] "Customer communication history" refers to a record of all interactions, inquiries, purchase history, etc. that a customer has with a company or service provider.

[0919] A "summary" is information that extracts important information from the communication history with a customer and summarizes it concisely.

[0920] "Generative artificial intelligence (AI)" is an AI that has the ability to generate appropriate answers to user inquiries based on collected data.

[0921] An "information display device" is a device worn or used by a user, which displays answers to inquiries in real time.

[0922] The term "user" refers to an employee such as a store staff member who operates the system of the present invention and deals with customers.

[0923] A "server" is a computer system that collects, stores, analyzes, and generates answers using generative artificial intelligence (AI).

[0924] "Interface" refers to the means or screen through which a user or customer makes inquiries to the system.

[0925] A "prompt" is data that provides the underlying information or conditions that a generative artificial intelligence (AI) uses to generate an appropriate answer.

[0926] The present invention provides a system that enables store staff to use smart glasses or other information display devices to quickly and accurately interact with customers. Detailed embodiments of the system are described below.

[0927] Overall overview

[0928] This system acquires and analyzes communication history with customers, and then uses generative artificial intelligence (AI) to summarize the results.The user (store staff) then provides appropriate answers to customer inquiries in real time via an information display device.

[0929] System Components

[0930] server

[0931] The server collects, stores, analyzes, and summarizes the communication history with the customer, which includes the following steps:

[0932] 1. History capture and storage:

[0933] The server stores email and chat history with customers in a database, preferably using MySQL or similar database.

[0934] 2. Generate a summary:

[0935] The server uses natural language processing technologies such as Python's NLTK and Gensim to analyze the accumulated historical data and summarize important information.

[0936] Generative artificial intelligence (AI)

[0937] 3. Receiving and parsing queries:

[0938] The server analyzes queries received through smart glasses or information display devices and searches for appropriate communication history.

[0939] 4. Generate and provide answers:

[0940] The server uses a generative AI model (e.g., GPT-3) to generate the optimal answer based on the analysis results, which is then processed using deep learning libraries such as TensorFlow and PyTorch.

[0941] The server presents this answer in real time to the user's smart glasses or information display device.

[0942] User terminal

[0943] The user terminal is a device worn by the user, such as smart glasses, that displays the generated answers in real time.

[0944] Specific examples

[0945] Example 1: Customer Inquiry

[0946] A customer asks, "Are there any other products similar to this one?" The user sends this inquiry to the server via their smart glasses. The server retrieves the customer's past purchase history and inquiry history and sends it to the generative AI. The generative AI analyzes this and generates an answer such as, "Products B and C are similar to this product." This answer is displayed in real time on the user's smart glasses.

[0947] Prompt Sentence Examples

[0948] Here is an example of a prompt for a generative AI model:

[0949] User query: "Are there any other products like this?"

[0950] Past purchase history: Product A (2023-01-10), Product B (2023-02-15)

[0951] Past inquiry history: "I'd like to know about similar products" "Please recommend some products"

[0952] Example of an answer it might generate: "Products similar to this one are Product B and Product C."

[0953] This system will enable staff at physical stores to respond quickly and accurately to a wide range of customer questions, improving customer satisfaction and operational efficiency.

[0954] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0955] Step 1:

[0956] A user receives a customer inquiry via smart glasses.

[0957] Input: Customer inquiry (e.g., "Do you have anything similar to this product?")

[0958] Output: Query data sent to the server

[0959] How it works: The user speaks their inquiry into the microphone of the smart glasses, which is then converted into text data and sent to the server. The voice recognition uses the Google Speech-to-Text API and other tools.

[0960] Step 2:

[0961] The server analyzes the received query.

[0962] Input: Query data sent by the user (a textual query)

[0963] Output: The parsed results of the query

[0964] Specific operation: The server analyzes the received query using natural language processing techniques (such as NLTK or SpaCy) and prepares it to query the database in an appropriate manner. Typical analysis includes understanding the meaning of the query and extracting keywords.

[0965] Step 3:

[0966] The server searches and retrieves the customer's communication history.

[0967] Input: query analysis results, customer identification information

[0968] Output: Customer communication history data

[0969] Specific operation: Based on the parsed query and customer identification information, the server searches the database (e.g., MySQL) for relevant communication history and retrieves the required data, including past purchase history and inquiry history.

[0970] Step 4:

[0971] Summarize historical data obtained by the server.

[0972] Input: Customer communication history data

[0973] Output: Summarized historical data

[0974] How it works: The server uses Python libraries such as NLTK and Gensim to extract key information from the acquired historical data and generate summaries, which are then used as prompts for the generative AI model.

[0975] Step 5:

[0976] The server sends prompts to the generative AI based on the summary data and the query.

[0977] Input: summarized historical data, query text

[0978] Output: prompt to generative AI

[0979] How it works: The server combines the summarized history data with the parsed query and formats it as a prompt for a generative AI model (e.g., GPT-3). An example prompt is as follows: "User query: 'Are there any other products similar to this one?'," "Past purchase history: Product A (2023-01-10), Product B (2023-02-15)," and "Past inquiry history: 'I'd like to know about similar products' and 'What products do you recommend?'" The content is then sent to the generative AI model.

[0980] Step 6:

[0981] A generative AI model generates appropriate answers.

[0982] Input: prompt to generative AI

[0983] Output: Generated answer (text format)

[0984] How it works: A generative AI model (e.g., GPT-3) generates the best answer based on the prompt provided, using natural language processing and deep learning techniques in the process.

[0985] Step 7:

[0986] The server sends the generated answer to the user's smart glasses.

[0987] Input: Generated answer (text format)

[0988] Output: Answer displayed on smart glasses

[0989] Specific operation: The server sends the generated answer to the user's smart glasses in real time, allowing the user to visually confirm it. The generated answer is displayed on the display of the smart glasses.

[0990] Step 8:

[0991] The user provides the answer to the customer.

[0992] Input: Answer displayed on smart glasses

[0993] Output: Providing a verbal response to the customer

[0994] Specific operation: The user reads out the answer displayed on the smart glasses and provides the answer to the customer verbally, allowing the customer to obtain the information quickly and accurately.

[0995] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0996] The present invention combines a system that analyzes communication history with customers and generates summaries with an emotion engine that recognizes the user's emotions. This system takes into account the user's emotional state and can generate more appropriate and effective responses. An embodiment of this system will be described in detail below.

[0997] Main components of the system

[0998] Server: Collects and stores communication history with customers, receives and processes inquiries from users or customers.

[0999] Generative artificial intelligence (AI): Analyzes collected historical data, generates summaries, and generates appropriate answers based on user queries.

[1000] User terminal: A device through which a user accesses and queries the system.

[1001] Customer terminal: A device that allows a customer to directly query the system.

[1002] Emotion engine: Analyzes the emotional state of the user's query and adjusts the generated answer based on the results.

[1003] Program processing (natural language explanation)

[1004] History capture and storage

[1005] The server collects and stores in a database the history of past emails and chats with customers, including the customer's identity and the content of those conversations.

[1006] Generate a summary

[1007] The server uses natural language processing techniques to summarize key information using the accumulated historical data, preparing the summary for rapid response to subsequent queries.

[1008] Query Reception and Emotion Recognition

[1009] A user or customer makes an inquiry. The query is sent from the user terminal or customer terminal through the interface. The server analyzes the query received through the interface and analyzes the user's emotional state using an emotion engine.

[1010] Emotion-Based Adjustment

[1011] The emotion engine analyzes the user's emotional state and dynamically changes the prompts sent to the generative artificial intelligence (AI) based on the results. For example, if the user is angry, the prompts will be set to generate responses with a more polite and calm tone.

[1012] Generate and provide answers

[1013] The generative AI generates an appropriate answer based on the prompts adjusted by the emotion engine. The server then provides the generated answer to the user or customer. For example, if the user is feeling anxious in response to the query "Please tell me the status of my recent order," the server will provide an answer in the form of "Sorry to keep you waiting. I will explain the status of my recent order. ..."

[1014] Specific examples

[1015] Example 1: User usage

[1016] The user sends a query from the interface saying, "Please tell me the status of Customer A's recent orders." The server retrieves Customer A's historical data and analyzes the user's emotional state using the emotion engine. The generative AI uses a prompt adjusted based on the analysis results of the emotion engine to generate and provide the answer, "Customer A placed order Y on 2023-02-15 as their most recent order."

[1017] Example 2: Direct customer inquiry

[1018] A customer submits a query saying, "I want to know the payment status of my recent invoices." The server retrieves the customer's payment history and analyzes their emotional state using an emotion engine. The generative AI uses a prompt tailored based on the analysis results of the emotion engine to generate and provide an answer such as, "Regarding recent invoices, I paid invoice B on 2023-02-20."

[1019] The role of the emotional engine

[1020] By analyzing the user's emotional state, the emotion engine can adjust the tone and content of responses to deliver a more personalized experience, improving customer satisfaction and building better customer relationships.

[1021] In this way, by combining emotion engines, it is possible to achieve more advanced and flexible customer response than conventional automated response systems.

[1022] The processing flow will be explained below.

[1023] Step 1:

[1024] The server collects past email and chat history with the customer and stores it in a database. This data includes customer identification information, the date of the interaction, and the specific content. For example, it stores information that "Customer A" placed "Order X" on "2023-01-01."

[1025] Step 2:

[1026] The server uses natural language processing techniques to extract key information from the stored historical data and generate a summary, which serves as a preliminary step for quickly responding to future inquiries.

[1027] Step 3:

[1028] A user or customer submits a query through the interface to make an inquiry. For example, a customer might type, "I want to know the status of my recent bill payment."

[1029] Step 4:

[1030] The server analyzes queries received through the interface and retrieves the relevant customer history data from the database, ensuring that the relevant data is up to date.

[1031] Step 5:

[1032] The server sends the acquired historical data and the query to the emotion engine, which analyzes the user's emotional state. For example, it determines whether the user is feeling "anxious" or "angry" based on the query's style and wording.

[1033] Step 6:

[1034] The emotion engine analyzes the user's emotional state and sends the results back to the server. For example, the analysis result may say, "The user is feeling anxious."

[1035] Step 7:

[1036] The server then adjusts the prompts sent to the generative artificial intelligence (AI) based on the emotion engine's analysis, for example, "If you're feeling anxious, add more detailed explanations and reassuring words."

[1037] Step 8:

[1038] Generative artificial intelligence (AI) generates appropriate answers based on tailored prompts, such as, "Regarding your recent invoices, you paid invoice B on 2023-02-20. Please rest assured about the details."

[1039] Step 9:

[1040] The server provides the generated answer to the user or customer. The answer is displayed through the interface so that the user or customer can easily check it. By displaying it on the terminal, the user can quickly obtain an appropriate answer to their inquiry.

[1041] Step 10:

[1042] If the user or customer is not satisfied with the answer they received and needs further inquiry or confirmation, they can send the query again, and the server will repeat the same process and continue to generate the best answer.

[1043] In this way, a system incorporating an emotion engine can provide more flexible and accurate customer service while taking into account the user's emotional state.

[1044] Example 2

[1045] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1046] Conventional customer service systems provide uniform responses without considering the emotional state of the user or customer, making it difficult to improve customer satisfaction. Furthermore, responses that ignore the emotional state can exacerbate customer dissatisfaction and anxiety, risking a deterioration in customer relationships. The challenge with conventional systems is the difficulty of properly assessing the emotional state of the user or customer and providing more personalized responses.

[1047] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for acquiring a communication history with a customer; means for saving the communication history with the customer; means for analyzing the communication history with the customer and generating a summary; means for a user or customer to send an inquiry; means for searching and acquiring an appropriate communication history based on the received inquiry; means for receiving the inquiry; means for using an emotion engine including means for analyzing the emotional state of the user in addition to the summary; means for adjusting a prompt to be sent to a generative artificial intelligence (AI) based on the emotional state of the user; means for utilizing the generative artificial intelligence (AI) to generate an appropriate answer based on the adjusted prompt; and means for providing the generated answer to the user or customer. This enables more personalized responses that take into account the emotional state of the user or customer.

[1048] "Customer communication history" refers to exchanges between customers and companies, such as emails, chats, phone recordings, and social media messages.

[1049] "Database" refers to a structured collection of data for managing and storing collected information.

[1050] "Natural language processing technology" refers to technologies and methods for understanding, analyzing, and generating human language.

[1051] An "emotion engine" refers to a software component that analyzes the emotional state of a user or customer (e.g., joy, anger, anxiety, etc.) from text data.

[1052] "Generative artificial intelligence (AI)" refers to artificial intelligence that automatically generates documents and information that humans can understand and use in natural language based on given prompts.

[1053] A "prompt" refers to input information that instructs a generative artificial intelligence (AI) on what kind of answer it should generate.

[1054] "User's emotional state" refers to the emotional state (e.g., joy, anger, anxiety, etc.) of the user when they input a query.

[1055] "Interface" refers to the device or software mechanism through which a user or customer accesses the system and sends and receives inquiries.

[1056] A "summary" is a concise summary of complex data or documents, extracting only the most important information.

[1057] "Enquiry" means a question or request made by a user or customer to the system.

[1058] The present invention combines a system that analyzes communication history with customers and generates summaries with an emotion engine that recognizes the user's emotions. This system takes the user's emotional state into account and generates more appropriate and effective answers.

[1059] Main components of the system

[1060] Server: Collects and stores customer communication history, receives and processes user or customer inquiries. Examples: AWS EC2 instances, Google Cloud Compute Engine.

[1061] Database: A system for storing and managing collected historical data. Examples: MySQL, PostgreSQL.

[1062] Generative Artificial Intelligence (AI): Analyzes collected historical data, generates summaries, and generates appropriate answers based on user queries. Examples: OpenAI GPT-3, Google BERT.

[1063] User terminal: A device through which a user accesses and queries the system. Examples: web browser, smartphone app.

[1064] Customer terminal: A device that allows a customer to directly query the system.

[1065] Emotion engine: Analyzes the emotional state of the user's query and adjusts the generated answer accordingly. Examples: IBM Watson, Azure Text Analytics.

[1066] Program processing

[1067] History capture and storage

[1068] The server collects past communication history with the customer using the specified mailbox or chat service API (e.g., Gmail API or Slack API). The collected data is stored in a database (e.g., MySQL or PostgreSQL) along with the customer's identification information and metadata.

[1069] Generate a summary

[1070] The server analyzes the accumulated historical data using natural language processing (NLP) techniques (e.g., NLTK, SpaCy), extracts important information, and generates summaries, which are then stored in a database as a preliminary step for responding quickly to future inquiries.

[1071] Receiving a query

[1072] Users or customers make inquiries from a web browser or smartphone app. The inquiries are sent to the server as HTTP requests, and the server receives them and stores them in a database in text format.

[1073] Emotion recognition

[1074] The server sends the text data of the received query to an emotion engine (e.g., IBM Watson NLP API, Azure Cognitive Services) to analyze the user's emotional state. The emotion engine returns a score to the server, such as "anxiety: 0.75, joy: 0.10, anger: 0.15."

[1075] Emotion-Based Adjustment

[1076] The server adjusts the prompts sent to the generative AI based on the emotion score received from the emotion engine. For example, if the anxiety score is high, the server generates a prompt that reads, "The user is feeling anxious. Please generate a reassuring response."

[1077] Generate and provide answers

[1078] The generative AI receives the tailored prompt and generates an appropriate answer based on it. For example, in response to the query "What is the status of Customer A's recent orders?", it generates the answer "Customer A recently placed order Y on 2023-02-15." The server then provides the generated answer to the user or customer through the same interface that received the query.

[1079] Specific examples

[1080] Example 1: User usage

[1081] When a user sends a query from the interface saying, "Please tell me the status of Customer A's recent orders," the server retrieves Customer A's historical data and analyzes the user's emotional state using the emotion engine. The generative AI uses a prompt adjusted based on the analysis results of the emotion engine to generate and provide the answer, "Customer A placed order Y on 2023-02-15 as their most recent order."

[1082] Example 2: Direct customer inquiry

[1083] When a customer sends a query such as "I want to know the payment status of my recent invoices," the server retrieves the customer's payment history and analyzes their emotional state using an emotion engine. The generative AI uses prompts tailored based on the analysis results of the emotion engine to generate and provide an answer such as "Regarding recent invoices, I paid invoice B on 2023-02-20."

[1084] Prompt Sentence Examples

[1085] Emotional state is anxious: "The user is feeling anxious. Please generate a polite and reassuring response."

[1086] Angry emotional state: "The user is angry. Please set a calm and measured tone in your response."

[1087] In this way, by combining emotion engines, it is possible to achieve more advanced and flexible customer response than conventional automated response systems.

[1088] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1089] Step 1: Collecting History

[1090] The server collects communication history with customers. As input, it obtains data from mailboxes and chat service APIs (e.g., Gmail API, Slack API). The specific process for collecting data involves sending queries to the API and analyzing the returned data. Through this operation, the contents of customer emails and chats are collected on the server.

[1091] Step 2: Save history

[1092] The server saves the collected customer communication history in a database. The input is the data collected in step 1. The server generates a document containing the customer's identification information and metadata and saves it in a database (e.g., MySQL, PostgreSQL). This operation records the customer's communication history in the database.

[1093] Step 3: Generate a summary

[1094] The server analyzes the accumulated historical data using natural language processing (NLP) techniques (e.g., NLTK, SpaCy). The input is the data saved in step 2. The server extracts important information and performs text analysis and semantic extraction as specific processes to generate a summary. The summary generated as output is saved in a database. This operation adds the summary information to the database.

[1095] Step 4: Receiving a query

[1096] A user or a client sends a query through the interface. The input is a query text from the terminal. The server receives the query as an HTTP request and stores it in a database in text format. This operation allows the server to manage the received queries.

[1097] Step 5: Recognize emotions

[1098] The server sends the text data of the received query to an emotion engine (e.g., IBM Watson NLP API, Azure Cognitive Services). The input is the query text received in step 4. The emotion engine performs specific processing to analyze the user's emotional state from the text. As an output, an emotion score (e.g., "Anxiety: 0.75, Joy: 0.10, Anger: 0.15") is returned to the server. This operation analyzes the user's emotional state.

[1099] Step 6: Emotional Adjustment

[1100] The server adjusts the prompt based on the emotion score received from the emotion engine. The input is the emotion score obtained in step 5 and the customer summary information stored in the database. The server uses the emotion score to dynamically generate a prompt to send to the generative AI (e.g., OpenAI GPT-3, Google BERT). For example, a prompt might be generated that reads, "The user is feeling anxious. Please generate a polite and reassuring response." This operation improves the accuracy of the generated response.

[1101] Step 7: Generate and serve answers

[1102] The generative AI receives the adjusted prompt and generates an appropriate answer based on it. The input is the prompt adjusted in step 6. The generative AI performs the specific process of generating an answer according to the prompt and returns the answer as output to the server. The server sends the generated answer to the user or customer. The answer is provided through the same interface that received the inquiry. This operation provides an appropriate answer to the user or customer.

[1103] The above is a specific description of the processing steps in the system.

[1104] (Application example 2)

[1105] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1106] Conventional automated response systems generate responses without considering the emotional state of the customer, limiting their ability to improve customer satisfaction. Furthermore, they lacked the processes for summarizing information and analyzing emotions to provide prompt and appropriate responses, making it difficult to provide effective customer support.

[1107] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1108] In this invention, the server includes means for acquiring a communication history with a customer, means for analyzing the communication history with the customer and generating a summary, means for utilizing an emotion engine that analyzes the emotional states of the user and the customer, means for utilizing generative artificial intelligence (AI) that generates an answer to the user's inquiry based on the summary and the emotion analysis results, and means for providing the generated answer to the user or customer. This makes it possible to provide a quick and accurate answer that takes into account the emotional state of the customer.

[1109] "Customer communication history" refers to past conversation data, such as emails and chats, between the customer and the system.

[1110] A "summary" is a concise summary of important information extracted from the collected communication history.

[1111] An "emotion engine" refers to a system that analyzes emotions from input sentences of a user or customer and recognizes their emotional state.

[1112] "Generative artificial intelligence (AI)" refers to artificial intelligence that uses natural language processing technology to generate appropriate answers based on user inquiries.

[1113] "Interface" refers to the screen or device through which a user or customer submits a query to the system.

[1114] A "prompt" refers to the input information that generative artificial intelligence (AI) uses to generate an appropriate answer.

[1115] "Answer" means a response generated and provided by a generative artificial intelligence (AI) in response to a user or customer inquiry.

[1116] To implement this invention, a system consisting of the following means is required. The main components of the system are a server, a user terminal, a customer terminal, an emotion engine, and a generative artificial intelligence (AI).

[1117] Main components of the system

[1118] server

[1119] The server has a database that stores and manages past communication history with customers, and also provides the computing resources to analyze inquiries and generate appropriate answers.

[1120] User and customer terminals

[1121] These terminals refer to devices that users or customers use to access the system and send inquiries, such as PCs, smartphones, and tablets.

[1122] Emotion Engine

[1123] The emotion engine uses the Samsung Emotional Engine API and other tools to analyze emotions from received text.

[1124] Generative artificial intelligence (AI)

[1125] As generative artificial intelligence (AI), we use advanced natural language generation models such as GPT-3.

[1126] System Overview

[1127] The server acquires past communication history and stores it in a database. When a user or customer sends an inquiry, the server receives the inquiry. After receiving the inquiry, the server uses an emotion engine to analyze emotions from the inquiry text.

[1128] Based on the analysis results of the emotion engine, the server uses generative artificial intelligence (AI) to generate an appropriate answer. When the generative artificial intelligence (AI) generates an answer, it adjusts the prompt based on the analysis results of the emotion engine to provide a more appropriate answer.

[1129] Specific examples

[1130] For example, consider a food delivery service application. If a customer asks, "Why hasn't my order arrived yet?", the server receives the inquiry. If the emotion engine detects "anger," the generative artificial intelligence (AI) can generate a response using prompts like the following:

[1131] Example prompt sentence:

[1132] Customer: "My order hasn't arrived yet."

[1133] Emotion analysis result: "Anger"

[1134] Generative AI prompt: "The customer seems upset. Please generate a response that explains the delay in an appropriate and polite tone."

[1135] The appropriate response generated by this would be something like, "We're very sorry. We're currently checking the tracking information for your order, so please wait a moment." This system makes it possible to provide quick and accurate responses that take into account the customer's feelings.

[1136] Software and hardware used

[1137] Hardware

[1138] Server (e.g. AWS EC2), user device (e.g. smartphone, tablet), customer device (e.g. PC, smartphone)

[1139] software

[1140] Emotion engines (e.g., Samsung Emotional Engine API), generative artificial intelligence (e.g., GPT-3), database management systems (e.g., MySQL)

[1141] This configuration makes it possible to provide flexible and accurate automatic responses that respond to customer emotions when dealing with customers in food delivery services and the like.

[1142] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1143] Step 1: Capture and save history

[1144] The server retrieves and stores the customer's past communication history from the database. Specifically, the server collects data such as emails, chats, and order history, and stores it in a database (e.g., MySQL). At this time, the customer's identification information and the content of the communication are stored in the database.

[1145] Input: Email, chat, order history

[1146] Output: Communication history stored in a database

[1147] Step 2: Receiving and analyzing inquiries

[1148] The server receives queries from users or customers. The queries are sent from the user or customer terminal and received by the server through an interface. The received queries are analyzed by an emotion engine (e.g., Samsung Emotional Engine API) to identify the emotional state.

[1149] Input: User or customer inquiry

[1150] Output: Emotion analysis results

[1151] Step 3: History search and summary generation

[1152] The server searches and retrieves relevant historical data from the database based on the sentiment analysis results and the query. The retrieved historical data is summarized using natural language processing techniques (e.g., BERT). The summary extracts only the important information and summarizes it concisely. This information is used in subsequent processing steps.

[1153] Input: Sentiment analysis results, query text

[1154] Output: Summarized historical data

[1155] Step 4: Generate and refine prompts

[1156] The server generates prompts based on the emotion analysis and summarized historical data, which are then sent to a generative artificial intelligence (AI). The generated prompts are tailored based on the emotion analysis. For example, if the user's emotion is anger, a prompt that responds in a polite tone is set.

[1157] Input: Sentiment analysis results, summarized historical data

[1158] Output: Adjusted prompt

[1159] Step 5: Generate an answer

[1160] Generative artificial intelligence (AI) generates appropriate responses based on tailored prompts. Specifically, an AI model (e.g., GPT-3) analyzes the prompts and generates the best possible response for the user or customer. This model is pre-trained to generate natural-sounding sentences with a high degree of accuracy.

[1161] Input: Adjusted prompt

[1162] Output: Generated answer

[1163] Step 6: Provide your answers

[1164] The server then provides the generated response to the user or customer via the user's or customer's terminal, sending a properly formatted response so that the user or customer can immediately obtain the appropriate information.

[1165] Input: Generated answer

[1166] Output: Provided to the user or customer

[1167] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1168] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1169] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1170] [Fourth embodiment]

[1171] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1172] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1173] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1174] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1175] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1176] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1177] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1178] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1179] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1180] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1181] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1182] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1183] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1184] This invention is a system in which generative artificial intelligence (AI) provides appropriate answers when confirmation matters arise based on the history of customer interactions. Detailed embodiments of this system are described below.

[1185] Overall overview

[1186] This system acquires, analyzes, and summarizes communication history with customers, and then uses AI to generate appropriate answers to user inquiries. It also provides an interface that allows customers to make inquiries directly, automating tasks that do not require the intervention of sales staff.

[1187] Main components of the system

[1188] Server: Collects and stores communication history with customers, receives and processes inquiries from users or customers.

[1189] Generative artificial intelligence (AI): Analyzes collected communication history, generates summaries, and generates appropriate answers based on user inquiries.

[1190] User terminal: A device through which a user accesses and queries the system, from which the user submits queries through the interface.

[1191] Customer terminal: A device that allows a customer to directly query the system.

[1192] Program processing (natural language explanation)

[1193] History capture and storage

[1194] The server stores email and chat history with customers in a database. This history includes customer identification information and the content of the interactions. For example, information such as "Order X was placed on 2023-01-01, and Order Y was placed on 2023-02-15" is stored as an interaction with a customer.

[1195] Generate a summary

[1196] The server uses natural language processing techniques to summarize key information using the accumulated historical data, preparing the summary for rapid response to subsequent queries.

[1197] Receiving and parsing queries

[1198] A user or customer sends a query through the interface to make an inquiry. The server receives the query, analyzes it based on the appropriate communication history, and sends it to the generative AI.

[1199] Generate and provide answers

[1200] The generative AI analyzes the received query and summarized historical data as prompts to generate the optimal answer. The server then provides the generated answer to the user or customer. For example, in response to the query "What is the status of my recent orders?", the generated answer is "I placed order Y on 2023-02-15."

[1201] Specific examples

[1202] Example 1: User usage

[1203] The user sends a query from the interface saying, "Please tell me the status of Customer A's recent orders." The server retrieves Customer A's historical data and sends it to the generative AI. The AI ​​generates the optimal answer based on the historical data and the query, providing the answer, "Customer A placed order Y on 2023-02-15."

[1204] Example 2: Direct customer inquiry

[1205] A customer submits a query saying, "I want to know the payment status of my recent invoices." The server retrieves the customer's payment history and sends it to the generative AI. The AI ​​generates an answer based on the historical data and the query, providing the answer, "Regarding recent invoices, invoice B was paid on 2023-02-20."

[1206] In this way, the system can respond quickly and accurately to customer and user inquiries. The distinctive features of this invention are the generation of a summary of the history and the generation of answers using generative AI, which can improve the efficiency and accuracy of customer responses.

[1207] The processing flow will be explained below.

[1208] Step 1:

[1209] The server collects past email and chat history with customers and stores it in a database, including customer identification information, dates, and specific exchanges.

[1210] Step 2:

[1211] The server periodically analyzes historical data and automatically generates summaries, using natural language processing techniques to extract key points and create concise summaries.

[1212] Step 3:

[1213] A user or customer makes an inquiry. A query is sent from the user or customer terminal through the interface. This query contains information that the user wants to know or a matter that needs confirmation.

[1214] Step 4:

[1215] The server analyzes queries received through the interface and searches for and retrieves historical data for the relevant customers, including, if necessary, retrieving the latest summaries from the database.

[1216] Step 5:

[1217] The server sends the acquired history data and the received query to a generative artificial intelligence (AI). The prompt sent to the AI ​​includes the summarized history data and the specific query content.

[1218] Step 6:

[1219] Generative artificial intelligence (AI) generates appropriate responses based on prompts. AI uses natural language processing techniques to create the most appropriate response to a query.

[1220] Step 7:

[1221] The server returns the answer received from the generative artificial intelligence (AI) to the user or customer, and the answer is displayed through the interface so that the user or customer can easily check it.

[1222] Step 8:

[1223] If the user or customer needs to make further inquiries or clarifications based on the answers they receive, they can send the query again, and the server will repeat the same process to continue providing appropriate answers.

[1224] Example 1

[1225] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1226] With conventional systems, it was difficult to efficiently utilize communication history with customers and respond quickly and accurately to inquiries from users and customers. Responding to inquiries also required a large amount of manpower, creating a demand for greater operational efficiency. Therefore, there is a need for a system that automates and efficiently performs operations while maintaining the quality of customer service.

[1227] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1228] In this invention, the server includes means for acquiring a communication history with a customer, means for saving the communication history with the customer, means for analyzing the communication history with the customer and summarizing important information, means for accepting an inquiry from a user or customer through an interface, means for searching and acquiring an appropriate communication history based on the received inquiry, means for generating an optimal answer based on the summary and the inquiry using generative artificial intelligence, and means for providing the generated answer to the user or customer, thereby enabling prompt and accurate responses to inquiries from customers or users.

[1229] "Customer communication history" refers to records of emails, chats, phone calls, and other interactions with customers.

[1230] A "server" is a computer system that provides functions such as data processing, storage, and network communication.

[1231] "Natural language processing technology" is a technology that uses computers to analyze, understand, and generate human language.

[1232] "Generative artificial intelligence (AI)" is artificial intelligence that has the ability to generate new information and answers based on input data.

[1233] An "enquiry" is a question or request that a user or customer enters into the system.

[1234] An "interface" is the means or method by which a user or customer interacts with a system.

[1235] A "query" is a question or search request issued to the system.

[1236] A "prompt" is input data or instructions given to generative artificial intelligence, which allows the AI ​​to generate an appropriate answer.

[1237] An "answer" is response information generated by a generative artificial intelligence based on a query.

[1238] The "summary" is a concise summary of important information extracted from the communication history with the customer.

[1239] This invention is a system in which generative artificial intelligence (AI) provides appropriate answers based on communication history with customers. The system of the present invention is specifically implemented as follows.

[1240] Overall structure

[1241] The main components of this system are a server, a generative AI, a user terminal, and a customer terminal. The server collects, stores, and analyzes data, and the generative AI generates appropriate answers. The user terminal and customer terminal provide the interface for users and customers to access the system and make inquiries.

[1242] Collecting and storing customer communication history

[1243] The server automatically collects communication history with customers, such as emails and chats, and stores it in a database. Standard server machines are used for the hardware and software used include the Python imaplib library and the Slack API.

[1244] Data analysis and summary generation

[1245] The server analyzes the stored historical data and summarizes key information using natural language processing (NLP) techniques, such as Python libraries nltk and spacy, to help generate the summaries.

[1246] Receiving inquiries

[1247] Users or customers access the interface through their terminals and send queries. The server receives these queries and searches and retrieves relevant historical data from the database. The interface is often implemented using popular web frameworks such as Flask and Django.

[1248] Generate answers

[1249] Generative artificial intelligence (AI) generates optimal answers based on the received query and summarized historical data. In this process, the query and related historical data are input as prompts into a generative AI model (e.g., OpenAI's GPT series).

[1250] Providing answers

[1251] The server provides the generated answer to the user or client terminal, which displays the answer through an interface.

[1252] Specific examples

[1253] Example 1: User usage

[1254] The user sends a query from the interface saying, "Please tell me the status of Customer A's recent orders." The server retrieves Customer A's historical data and sends it to the generative AI. The AI ​​generates the optimal answer based on the historical data and the query, providing the answer, "Customer A placed order Y on 2023-02-15."

[1255] Example 2: Direct customer inquiry

[1256] A customer submits a query saying, "I want to know the payment status of my recent invoices." The server retrieves the customer's payment history and sends it to the generative AI. The AI ​​generates an answer based on the historical data and the query, providing the answer, "Regarding recent invoices, invoice B was paid on 2023-02-20."

[1257] This system makes it possible to respond quickly and accurately to inquiries from customers and users. In addition, the system's unique features of this invention, such as the generation of summaries of communication history and the generation of answers using generative AI, make it possible to improve the efficiency and accuracy of customer support.

[1258] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1259] Step 1:

[1260] The server retrieves communication history with customers. The input is a data source such as a mail server or chat logs. The server retrieves unread emails and chat messages using Python's imaplib library or the Slack API. The output is the retrieved interaction data.

[1261] Specific behavior:

[1262] The server connects to the mail server and retrieves the unread emails.

[1263] The email body, sender information, and receipt date and time are obtained and stored as internal temporary data.

[1264] Step 2:

[1265] The server stores the acquired communication history with the customer in a database. The input is the acquired interaction data. The output is the communication history stored in the database. The database is typically MySQL or PostgreSQL.

[1266] Specific behavior:

[1267] Create a table in the database with columns such as customer ID, transaction text, date and time.

[1268] Store the temporary data in the database using the INSERT statement.

[1269] Step 3:

[1270] The server analyzes the historical data stored in the database and summarizes key information using natural language processing (NLP) techniques. The input is data from the database of customer communication history. The output is summarized text data.

[1271] Specific behavior:

[1272] Use the SELECT statement to retrieve customer history data from the database.

[1273] Use Python's nltk and spacy libraries to tokenize text and tag parts of speech.

[1274] Keyword extraction and summary generation algorithms are used to output important information as a summary.

[1275] Step 4:

[1276] A user or customer sends a query. The input is a query from the user terminal or customer terminal. The output is the query information received by the server, which processes it appropriately.

[1277] Specific behavior:

[1278] A query sent from a user terminal is received by a web framework (e.g., Flask).

[1279] Stores received queries as internal temporary data.

[1280] Step 5:

[1281] The server searches and retrieves the appropriate communication history based on the received query. The input is the user or customer query and the database. The output is the communication history data related to the query.

[1282] Specific behavior:

[1283] The keywords contained in the query are analyzed and search conditions are generated to retrieve appropriate historical data from the database.

[1284] Based on the search criteria, relevant data is retrieved from the database.

[1285] Step 6:

[1286] Generative AI generates optimal answers based on queries and related historical data. The input is the user or customer query and summarized historical data. The output is the generated answer.

[1287] Specific behavior:

[1288] The server sends queries and historical data as prompts to the generative AI model.

[1289] The generative AI model generates an answer and sends it back to the server.

[1290] Step 7:

[1291] The server provides the generated answer to the user or client terminal. The input is the generated answer. The output is the answer displayed on the interface.

[1292] Specific behavior:

[1293] The server transmits the generated answer to the user or client terminal.

[1294] The answers are displayed in the interface for the user or customer to review.

[1295] (Application example 1)

[1296] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1297] Customer service in brick-and-mortar stores requires providing information quickly and accurately, but traditional methods make it difficult for staff to grasp all product information and customer history, resulting in problems that lead to lower customer satisfaction and reduced productivity. Particularly in busy store environments, staff lack the support they need to quickly provide appropriate answers.

[1298] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1299] In this invention, the server includes: means for acquiring a communication history with a customer; means for analyzing the communication history with the customer and generating a summary; means for utilizing generative artificial intelligence (AI) to generate an answer to a user's inquiry based on the summary; means for providing the generated answer to the user or customer; and means for receiving an inquiry via an information display device operated by a user in a store and presenting the generated answer to the user in real time. This enables staff to respond to customer inquiries quickly and accurately, even in a physical store, thereby improving customer satisfaction and business efficiency.

[1300] "Customer communication history" refers to a record of all interactions, inquiries, purchase history, etc. that a customer has with a company or service provider.

[1301] A "summary" is information that extracts important information from the communication history with a customer and summarizes it concisely.

[1302] "Generative artificial intelligence (AI)" is an AI that has the ability to generate appropriate answers to user inquiries based on collected data.

[1303] An "information display device" is a device worn or used by a user, which displays answers to inquiries in real time.

[1304] The term "user" refers to an employee such as a store staff member who operates the system of the present invention and deals with customers.

[1305] A "server" is a computer system that collects, stores, analyzes, and generates answers using generative artificial intelligence (AI).

[1306] "Interface" refers to the means or screen through which a user or customer makes inquiries to the system.

[1307] A "prompt" is data that provides the underlying information or conditions that a generative artificial intelligence (AI) uses to generate an appropriate answer.

[1308] The present invention provides a system that enables store staff to use smart glasses or other information display devices to quickly and accurately interact with customers. Detailed embodiments of the system are described below.

[1309] Overall overview

[1310] This system acquires and analyzes communication history with customers, and then uses generative artificial intelligence (AI) to summarize the results.The user (store staff) can then provide appropriate answers to customer inquiries in real time via an information display device.

[1311] System Components

[1312] server

[1313] The server collects, stores, analyzes, and summarizes the communication history with the customer, which includes the following steps:

[1314] 1. History capture and storage:

[1315] The server stores email and chat history with customers in a database, preferably using MySQL or similar database.

[1316] 2. Generate a summary:

[1317] The server uses natural language processing technologies such as Python's NLTK and Gensim to analyze the accumulated historical data and summarize important information.

[1318] Generative artificial intelligence (AI)

[1319] 3. Receiving and parsing queries:

[1320] The server analyzes queries received through smart glasses or information display devices and searches for appropriate communication history.

[1321] 4. Generate and provide answers:

[1322] The server uses a generative AI model (e.g., GPT-3) to generate the optimal answer based on the analysis results, which is then processed using deep learning libraries such as TensorFlow and PyTorch.

[1323] The server presents this answer in real time to the user's smart glasses or information display device.

[1324] User terminal

[1325] The user terminal is a device worn by the user, such as smart glasses, that displays the generated answers in real time.

[1326] Specific examples

[1327] Example 1: Customer Inquiry

[1328] A customer asks, "Are there any other products similar to this one?" The user sends this inquiry to the server via their smart glasses. The server retrieves the customer's past purchase history and inquiry history and sends it to the generative AI. The generative AI analyzes this and generates an answer such as, "Products B and C are similar to this product." This answer is displayed in real time on the user's smart glasses.

[1329] Prompt Sentence Examples

[1330] Here is an example of a prompt for a generative AI model:

[1331] User query: "Are there any other products like this?"

[1332] Past purchase history: Product A (2023-01-10), Product B (2023-02-15)

[1333] Past inquiry history: "I'd like to know about similar products" "Please recommend some products"

[1334] Example of an answer it might generate: "Products similar to this one are Product B and Product C."

[1335] This system will enable staff at physical stores to respond quickly and accurately to a wide range of customer questions, improving customer satisfaction and operational efficiency.

[1336] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1337] Step 1:

[1338] A user receives a customer inquiry via smart glasses.

[1339] Input: Customer inquiry (e.g., "Do you have anything similar to this product?")

[1340] Output: Query data sent to the server

[1341] How it works: The user speaks their inquiry into the microphone of the smart glasses, which is then converted into text data and sent to the server. The voice recognition uses the Google Speech-to-Text API and other tools.

[1342] Step 2:

[1343] The server analyzes the received query.

[1344] Input: Query data sent by the user (a textual query)

[1345] Output: The parsed results of the query

[1346] Specific operation: The server analyzes the received query using natural language processing techniques (such as NLTK or SpaCy) and prepares it to query the database in an appropriate manner. Typical analysis includes understanding the meaning of the query and extracting keywords.

[1347] Step 3:

[1348] The server searches and retrieves the customer's communication history.

[1349] Input: query analysis results, customer identification information

[1350] Output: Customer communication history data

[1351] Specific operation: Based on the parsed query and customer identification information, the server searches the database (e.g., MySQL) for relevant communication history and retrieves the required data, including past purchase history and inquiry history.

[1352] Step 4:

[1353] Summarize historical data obtained by the server.

[1354] Input: Customer communication history data

[1355] Output: Summarized historical data

[1356] How it works: The server uses Python libraries such as NLTK and Gensim to extract key information from the acquired historical data and generate summaries, which are then used as prompts for the generative AI model.

[1357] Step 5:

[1358] The server sends prompts to the generative AI based on the summary data and the query.

[1359] Input: summarized historical data, query text

[1360] Output: prompt to generative AI

[1361] How it works: The server combines the summarized history data with the parsed query and formats it as a prompt for a generative AI model (e.g., GPT-3). Examples of prompts include: "User query: 'Are there any other products similar to this one?'," "Past purchase history: Product A (2023-01-10), Product B (2023-02-15)," and "Past inquiry history: 'I'd like to know about similar products,' and 'What products do you recommend?'" These are then sent to the generative AI model.

[1362] Step 6:

[1363] A generative AI model generates appropriate answers.

[1364] Input: prompt to generative AI

[1365] Output: Generated answer (text format)

[1366] How it works: A generative AI model (e.g., GPT-3) generates the best answer based on the prompt provided, using natural language processing and deep learning techniques in the process.

[1367] Step 7:

[1368] The server sends the generated answer to the user's smart glasses.

[1369] Input: Generated answer (text format)

[1370] Output: Answer displayed on smart glasses

[1371] Specific operation: The server sends the generated answer to the user's smart glasses in real time, allowing the user to visually confirm it. The generated answer is displayed on the display of the smart glasses.

[1372] Step 8:

[1373] The user provides the answer to the customer.

[1374] Input: Answer displayed on smart glasses

[1375] Output: Providing a verbal response to the customer

[1376] Specific operation: The user reads out the answer displayed on the smart glasses and provides the answer to the customer verbally, allowing the customer to obtain the information quickly and accurately.

[1377] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1378] The present invention combines a system that analyzes communication history with customers and generates summaries with an emotion engine that recognizes the user's emotions. This system takes into account the user's emotional state and can generate more appropriate and effective responses. An embodiment of this system will be described in detail below.

[1379] Main components of the system

[1380] Server: Collects and stores communication history with customers, receives and processes inquiries from users or customers.

[1381] Generative artificial intelligence (AI): Analyzes collected historical data, generates summaries, and generates appropriate answers based on user queries.

[1382] User terminal: A device through which a user accesses and queries the system.

[1383] Customer terminal: A device that allows a customer to directly query the system.

[1384] Emotion engine: Analyzes the emotional state of the user's query and adjusts the generated answer based on the results.

[1385] Program processing (natural language explanation)

[1386] History capture and storage

[1387] The server collects and stores in a database the history of past emails and chats with customers, including the customer's identity and the content of those conversations.

[1388] Generate a summary

[1389] The server uses natural language processing techniques to summarize key information using the accumulated historical data, preparing the summary for rapid response to subsequent queries.

[1390] Query Reception and Emotion Recognition

[1391] A user or customer makes an inquiry. The query is sent from the user terminal or customer terminal through the interface. The server analyzes the query received through the interface and analyzes the user's emotional state using an emotion engine.

[1392] Emotion-Based Adjustment

[1393] The emotion engine analyzes the user's emotional state and dynamically changes the prompts sent to the generative artificial intelligence (AI) based on the results. For example, if the user is angry, the prompts will be set to generate responses with a more polite and calm tone.

[1394] Generate and provide answers

[1395] The generative AI generates an appropriate answer based on the prompts adjusted by the emotion engine. The server then provides the generated answer to the user or customer. For example, if the user is feeling anxious in response to the query "Please tell me the status of my recent order," the server will provide an answer in the form of "Sorry to keep you waiting. I will explain the status of my recent order. ..."

[1396] Specific examples

[1397] Example 1: User usage

[1398] The user sends a query from the interface saying, "Please tell me the status of Customer A's recent orders." The server retrieves Customer A's historical data and analyzes the user's emotional state using the emotion engine. The generative AI uses a prompt adjusted based on the analysis results of the emotion engine to generate and provide the answer, "Customer A placed order Y on 2023-02-15 as their most recent order."

[1399] Example 2: Direct customer inquiry

[1400] A customer submits a query saying, "I want to know the payment status of my recent invoices." The server retrieves the customer's payment history and analyzes their emotional state using an emotion engine. The generative AI uses a prompt tailored based on the analysis results of the emotion engine to generate and provide an answer such as, "Regarding recent invoices, I paid invoice B on 2023-02-20."

[1401] The role of the emotional engine

[1402] By analyzing the user's emotional state, the emotion engine can adjust the tone and content of responses to deliver a more personalized experience, improving customer satisfaction and building better customer relationships.

[1403] In this way, by combining emotion engines, it is possible to achieve more advanced and flexible customer response than conventional automated response systems.

[1404] The processing flow will be explained below.

[1405] Step 1:

[1406] The server collects past email and chat history with the customer and stores it in a database. This data includes customer identification information, the date of the interaction, and the specific content. For example, it stores information that "Customer A" placed "Order X" on "2023-01-01."

[1407] Step 2:

[1408] The server uses natural language processing techniques to extract key information from the stored historical data and generate a summary, which serves as a preliminary step for quickly responding to future inquiries.

[1409] Step 3:

[1410] A user or customer submits a query through the interface to make an inquiry. For example, a customer might type, "I want to know the status of my recent bill payment."

[1411] Step 4:

[1412] The server analyzes queries received through the interface and retrieves the relevant customer history data from the database, ensuring that the relevant data is up to date.

[1413] Step 5:

[1414] The server sends the acquired historical data and the query to the emotion engine, which analyzes the user's emotional state. For example, it determines whether the user is feeling "anxious" or "angry" based on the query's style and wording.

[1415] Step 6:

[1416] The emotion engine analyzes the user's emotional state and sends the results back to the server. For example, the analysis result may say, "The user is feeling anxious."

[1417] Step 7:

[1418] The server then adjusts the prompts sent to the generative artificial intelligence (AI) based on the emotion engine's analysis, for example, "If you're feeling anxious, add more detailed explanations and reassuring words."

[1419] Step 8:

[1420] Generative artificial intelligence (AI) generates appropriate answers based on tailored prompts, such as, "Regarding your recent invoices, you paid invoice B on 2023-02-20. Please rest assured about the details."

[1421] Step 9:

[1422] The server provides the generated answer to the user or customer. The answer is displayed through the interface so that the user or customer can easily check it. By displaying it on the terminal, the user can quickly obtain an appropriate answer to their inquiry.

[1423] Step 10:

[1424] If the user or customer is not satisfied with the answer they received and needs further inquiry or confirmation, they can send the query again, and the server will repeat the same process and continue to generate the best answer.

[1425] In this way, a system incorporating an emotion engine can provide more flexible and accurate customer service while taking into account the user's emotional state.

[1426] Example 2

[1427] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1428] Conventional customer service systems provide uniform responses without considering the emotional state of the user or customer, making it difficult to improve customer satisfaction. Furthermore, responses that ignore the emotional state can exacerbate customer dissatisfaction and anxiety, risking a deterioration in customer relationships. The challenge with conventional systems is the difficulty of properly assessing the emotional state of the user or customer and providing more personalized responses.

[1429] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for acquiring a communication history with a customer; means for saving the communication history with the customer; means for analyzing the communication history with the customer and generating a summary; means for a user or customer to send an inquiry; means for searching and acquiring an appropriate communication history based on the received inquiry; means for receiving the inquiry; means for using an emotion engine including means for analyzing the emotional state of the user in addition to the summary; means for adjusting a prompt to be sent to a generative artificial intelligence (AI) based on the emotional state of the user; means for utilizing the generative artificial intelligence (AI) to generate an appropriate answer based on the adjusted prompt; and means for providing the generated answer to the user or customer. This enables more personalized responses that take into account the emotional state of the user or customer.

[1430] "Customer communication history" refers to exchanges between customers and companies, such as emails, chats, phone recordings, and social media messages.

[1431] "Database" refers to a structured collection of data for managing and storing collected information.

[1432] "Natural language processing technology" refers to technologies and methods for understanding, analyzing, and generating human language.

[1433] An "emotion engine" refers to a software component that analyzes the emotional state of a user or customer (e.g., joy, anger, anxiety, etc.) from text data.

[1434] "Generative artificial intelligence (AI)" refers to artificial intelligence that automatically generates documents and information that humans can understand and use in natural language based on given prompts.

[1435] A "prompt" refers to input information that instructs a generative artificial intelligence (AI) on what kind of answer it should generate.

[1436] "User's emotional state" refers to the emotional state (e.g., joy, anger, anxiety, etc.) of the user when they input a query.

[1437] "Interface" refers to the device or software mechanism through which a user or customer accesses the system and sends and receives inquiries.

[1438] A "summary" is a concise summary of complex data or documents, extracting only the most important information.

[1439] "Enquiry" means a question or request made by a user or customer to the system.

[1440] The present invention combines a system that analyzes communication history with customers and generates summaries with an emotion engine that recognizes the user's emotions. This system takes the user's emotional state into account and generates more appropriate and effective answers.

[1441] Main components of the system

[1442] Server: Collects and stores customer communication history, receives and processes user or customer inquiries. Examples: AWS EC2 instances, Google Cloud Compute Engine.

[1443] Database: A system for storing and managing collected historical data. Examples: MySQL, PostgreSQL.

[1444] Generative Artificial Intelligence (AI): Analyzes collected historical data, generates summaries, and generates appropriate answers based on user queries. Examples: OpenAI GPT-3, Google BERT.

[1445] User terminal: A device through which a user accesses and queries the system. Examples: web browser, smartphone app.

[1446] Customer terminal: A device that allows a customer to directly query the system.

[1447] Emotion engine: Analyzes the emotional state of the user's query and adjusts the generated answer accordingly. Examples: IBM Watson, Azure Text Analytics.

[1448] Program processing

[1449] History capture and storage

[1450] The server collects past communication history with the customer using the specified mailbox or chat service API (e.g., Gmail API or Slack API). The collected data is stored in a database (e.g., MySQL or PostgreSQL) along with the customer's identification information and metadata.

[1451] Generate a summary

[1452] The server analyzes the accumulated historical data using natural language processing (NLP) techniques (e.g., NLTK, SpaCy), extracts important information, and generates summaries, which are then stored in a database as a preliminary step for responding quickly to future inquiries.

[1453] Receiving a query

[1454] Users or customers make inquiries from a web browser or smartphone app. The inquiries are sent to the server as HTTP requests, and the server receives them and stores them in a database in text format.

[1455] Emotion recognition

[1456] The server sends the text data of the received query to an emotion engine (e.g., IBM Watson NLP API, Azure Cognitive Services) to analyze the user's emotional state. The emotion engine returns a score to the server, such as "anxiety: 0.75, joy: 0.10, anger: 0.15."

[1457] Emotion-Based Adjustment

[1458] The server adjusts the prompts sent to the generative AI based on the emotion score received from the emotion engine. For example, if the anxiety score is high, the server generates a prompt that reads, "The user is feeling anxious. Please generate a reassuring response."

[1459] Generate and provide answers

[1460] The generative AI receives the tailored prompt and generates an appropriate answer based on it. For example, in response to the query "What is the status of Customer A's recent orders?", it generates the answer "Customer A recently placed order Y on 2023-02-15." The server then provides the generated answer to the user or customer through the same interface that received the query.

[1461] Specific examples

[1462] Example 1: User usage

[1463] When a user sends a query from the interface saying, "Please tell me the status of Customer A's recent orders," the server retrieves Customer A's historical data and analyzes the user's emotional state using the emotion engine. The generative AI uses a prompt adjusted based on the analysis results of the emotion engine to generate and provide the answer, "Customer A placed order Y on 2023-02-15 as their most recent order."

[1464] Example 2: Direct customer inquiry

[1465] When a customer sends a query such as "I want to know the payment status of my recent invoices," the server retrieves the customer's payment history and analyzes their emotional state using an emotion engine. The generative AI uses prompts tailored based on the analysis results of the emotion engine to generate and provide an answer such as "Regarding recent invoices, I paid invoice B on 2023-02-20."

[1466] Prompt Sentence Examples

[1467] Emotional state is anxious: "The user is feeling anxious. Please generate a polite and reassuring response."

[1468] Angry emotional state: "The user is angry. Please set a calm and measured tone in your response."

[1469] In this way, by combining emotion engines, it is possible to achieve more advanced and flexible customer response than conventional automated response systems.

[1470] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1471] Step 1: Collecting History

[1472] The server collects communication history with customers. As input, it obtains data from mailboxes and chat service APIs (e.g., Gmail API, Slack API). The specific process for collecting data involves sending queries to the API and analyzing the returned data. Through this operation, the contents of customer emails and chats are collected on the server.

[1473] Step 2: Save history

[1474] The server saves the collected customer communication history in a database. The input is the data collected in step 1. The server generates a document containing the customer's identification information and metadata and saves it in a database (e.g., MySQL, PostgreSQL). This operation records the customer's communication history in the database.

[1475] Step 3: Generate a summary

[1476] The server analyzes the accumulated historical data using natural language processing (NLP) techniques (e.g., NLTK, SpaCy). The input is the data saved in step 2. The server extracts important information and performs text analysis and semantic extraction as specific processes to generate a summary. The summary generated as output is saved in a database. This operation adds the summary information to the database.

[1477] Step 4: Receiving a query

[1478] A user or a client sends a query through the interface. The input is a query text from the terminal. The server receives the query as an HTTP request and stores it in a database in text format. This operation allows the server to manage the received queries.

[1479] Step 5: Recognize emotions

[1480] The server sends the text data of the received query to an emotion engine (e.g., IBM Watson NLP API, Azure Cognitive Services). The input is the query text received in step 4. The emotion engine performs specific processing to analyze the user's emotional state from the text. As an output, an emotion score (e.g., "Anxiety: 0.75, Joy: 0.10, Anger: 0.15") is returned to the server. This operation analyzes the user's emotional state.

[1481] Step 6: Emotional Adjustment

[1482] The server adjusts the prompt based on the emotion score received from the emotion engine. The input is the emotion score obtained in step 5 and the customer summary information stored in the database. The server uses the emotion score to dynamically generate a prompt to send to the generative AI (e.g., OpenAI GPT-3, Google BERT). For example, a prompt might be generated that reads, "The user is feeling anxious. Please generate a polite and reassuring response." This operation improves the accuracy of the generated response.

[1483] Step 7: Generate and serve answers

[1484] The generative AI receives the adjusted prompt and generates an appropriate answer based on it. The input is the prompt adjusted in step 6. The generative AI performs the specific process of generating an answer according to the prompt and returns the answer as output to the server. The server sends the generated answer to the user or customer. The answer is provided through the same interface that received the inquiry. This operation provides an appropriate answer to the user or customer.

[1485] The above is a specific description of the processing steps in the system.

[1486] (Application example 2)

[1487] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1488] Conventional automated response systems generate responses without considering the emotional state of the customer, limiting their ability to improve customer satisfaction. Furthermore, they lacked the processes for summarizing information and analyzing emotions to provide prompt and appropriate responses, making it difficult to provide effective customer support.

[1489] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1490] In this invention, the server includes means for acquiring a communication history with a customer, means for analyzing the communication history with the customer and generating a summary, means for utilizing an emotion engine that analyzes the emotional states of the user and the customer, means for utilizing generative artificial intelligence (AI) that generates an answer to the user's inquiry based on the summary and the emotion analysis results, and means for providing the generated answer to the user or customer. This makes it possible to provide a quick and accurate answer that takes into account the emotional state of the customer.

[1491] "Customer communication history" refers to past conversation data, such as emails and chats, between the customer and the system.

[1492] A "summary" is a concise summary of important information extracted from the collected communication history.

[1493] An "emotion engine" refers to a system that analyzes emotions from input sentences of a user or customer and recognizes their emotional state.

[1494] "Generative artificial intelligence (AI)" refers to artificial intelligence that uses natural language processing technology to generate appropriate answers based on user inquiries.

[1495] "Interface" refers to the screen or device through which a user or customer submits a query to the system.

[1496] A "prompt" refers to the input information that generative artificial intelligence (AI) uses to generate an appropriate answer.

[1497] "Answer" means a response generated and provided by a generative artificial intelligence (AI) in response to a user or customer inquiry.

[1498] To implement this invention, a system consisting of the following means is required. The main components of the system are a server, a user terminal, a customer terminal, an emotion engine, and a generative artificial intelligence (AI).

[1499] Main components of the system

[1500] server

[1501] The server has a database that stores and manages past communication history with customers, and also provides the computing resources to analyze inquiries and generate appropriate answers.

[1502] User and customer terminals

[1503] These terminals refer to devices that users or customers use to access the system and send inquiries, such as PCs, smartphones, and tablets.

[1504] Emotion Engine

[1505] The emotion engine uses the Samsung Emotional Engine API and other tools to analyze emotions from received text.

[1506] Generative artificial intelligence (AI)

[1507] As generative artificial intelligence (AI), we use advanced natural language generation models such as GPT-3.

[1508] System Overview

[1509] The server acquires past communication history and stores it in a database. When a user or customer sends an inquiry, the server receives the inquiry. After receiving the inquiry, the server uses an emotion engine to analyze emotions from the inquiry text.

[1510] Based on the analysis results of the emotion engine, the server uses generative artificial intelligence (AI) to generate an appropriate answer. When the generative artificial intelligence (AI) generates an answer, it adjusts the prompt based on the analysis results of the emotion engine to provide a more appropriate answer.

[1511] Specific examples

[1512] For example, consider a food delivery service application. If a customer asks, "Why hasn't my order arrived yet?", the server receives the inquiry. If the emotion engine detects "anger," the generative artificial intelligence (AI) can generate a response using prompts like the following:

[1513] Example prompt sentence:

[1514] Customer: "My order hasn't arrived yet."

[1515] Emotion analysis result: "Anger"

[1516] Generative AI prompt: "The customer seems upset. Please generate a response that explains the delay in an appropriate and polite tone."

[1517] The appropriate response generated by this would be something like, "We're very sorry. We're currently checking the tracking information for your order, so please wait a moment." This system makes it possible to provide quick and accurate responses that take into account the customer's feelings.

[1518] Software and hardware used

[1519] Hardware

[1520] Server (e.g. AWS EC2), user device (e.g. smartphone, tablet), customer device (e.g. PC, smartphone)

[1521] software

[1522] Emotion engines (e.g., Samsung Emotional Engine API), generative artificial intelligence (e.g., GPT-3), database management systems (e.g., MySQL)

[1523] This configuration makes it possible to provide flexible and accurate automatic responses that respond to customer emotions when dealing with customers in food delivery services and the like.

[1524] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1525] Step 1: Capture and save history

[1526] The server retrieves and stores the customer's past communication history from the database. Specifically, the server collects data such as emails, chats, and order history, and stores it in a database (e.g., MySQL). At this time, the customer's identification information and the content of the communication are stored in the database.

[1527] Input: Email, chat, order history

[1528] Output: Communication history stored in a database

[1529] Step 2: Receiving and analyzing inquiries

[1530] The server receives queries from users or customers. The queries are sent from the user or customer terminal and received by the server through an interface. The received queries are analyzed by an emotion engine (e.g., Samsung Emotional Engine API) to identify the emotional state.

[1531] Input: User or customer inquiry

[1532] Output: Emotion analysis results

[1533] Step 3: History search and summary generation

[1534] The server searches and retrieves relevant historical data from the database based on the sentiment analysis results and the query. The retrieved historical data is summarized using natural language processing techniques (e.g., BERT). The summary extracts only the important information and summarizes it concisely. This information is used in subsequent processing steps.

[1535] Input: Sentiment analysis results, query text

[1536] Output: Summarized historical data

[1537] Step 4: Generate and refine prompts

[1538] The server generates prompts based on the emotion analysis and summarized historical data, which are then sent to a generative artificial intelligence (AI). The generated prompts are tailored based on the emotion analysis. For example, if the user's emotion is anger, a prompt that responds in a polite tone is set.

[1539] Input: Sentiment analysis results, summarized historical data

[1540] Output: Adjusted prompt

[1541] Step 5: Generate an answer

[1542] Generative artificial intelligence (AI) generates appropriate responses based on tailored prompts. Specifically, an AI model (e.g., GPT-3) analyzes the prompts and generates the best possible response for the user or customer. This model is pre-trained to generate natural-sounding sentences with a high degree of accuracy.

[1543] Input: Adjusted prompt

[1544] Output: Generated answer

[1545] Step 6: Provide your answers

[1546] The server then provides the generated response to the user or customer via the user's or customer's terminal, sending a properly formatted response so that the user or customer can immediately obtain the appropriate information.

[1547] Input: Generated answer

[1548] Output: Provided to the user or customer

[1549] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1550] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1551] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1552] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1553] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1554] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1555] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1556] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1557] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1558] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1559] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1560] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1561] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1562] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1563] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1564] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1565] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1566] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1567] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1568] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1569] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1570] The following is further disclosed regarding the above embodiment.

[1571] (Claim 1)

[1572] A means of obtaining communication history with customers;

[1573] means for analyzing a communication history with the customer and generating a summary;

[1574] means for utilizing generative artificial intelligence (AI) to generate answers to user queries based on said summaries;

[1575] means for providing said generated answers to a user or customer;

[1576] A system including:

[1577] (Claim 2)

[1578] means for providing an interface for said user or customer to submit an inquiry;

[1579] means for receiving the inquiry;

[1580] means for searching and acquiring an appropriate communication history based on the received inquiry;

[1581] The system of claim 1 further comprising:

[1582] (Claim 3)

[1583] The generative artificial intelligence (AI) uses natural language processing technology;

[1584] means for using the query and the generated summary together as a prompt to generate an appropriate response;

[1585] 10. The system of claim 1, comprising:

[1586] "Example 1"

[1587] (Claim 1)

[1588] A means of obtaining communication history with customers;

[1589] a means for storing a communication history with the customer;

[1590] means for analyzing communication history with said customer and summarizing important information;

[1591] means for accepting inquiries from users or customers through an interface;

[1592] means for searching and acquiring an appropriate communication history based on the received inquiry;

[1593] means for generating an optimal answer based on the abstract and the query using generative artificial intelligence;

[1594] means for providing said generated answers to a user or customer;

[1595] A system including:

[1596] (Claim 2)

[1597] means for providing said interface;

[1598] and means for associating and searching historical data based on the received inquiry.

[1599] 10. The system of claim 1.

[1600] (Claim 3)

[1601] The generative artificial intelligence uses natural language processing technology;

[1602] means for generating an appropriate answer using the query and the generated summary as prompts;

[1603] 10. The system of claim 1, comprising:

[1604] "Application Example 1"

[1605] (Claim 1)

[1606] A means of obtaining communication history with customers;

[1607] means for analyzing a communication history with the customer and generating a summary;

[1608] means for utilizing generative artificial intelligence (AI) to generate answers to user queries based on said summaries;

[1609] means for providing said generated answers to a user or customer;

[1610] a means for receiving an inquiry via an information display device operated by a user at a store and presenting the generated answer to the user in real time;

[1611] A system including:

[1612] (Claim 2)

[1613] means for providing an interface for said user or customer to submit an inquiry;

[1614] means for receiving the inquiry;

[1615] means for searching and acquiring an appropriate communication history based on the received inquiry;

[1616] The system of claim 1 further comprising:

[1617] (Claim 3)

[1618] The generative artificial intelligence (AI) uses natural language processing technology;

[1619] means for using the query and the generated summary together as a prompt to generate an appropriate response;

[1620] 10. The system of claim 1, comprising:

[1621] "Example 2: Combining Emotion Engines"

[1622] (Claim 1)

[1623] A means of obtaining communication history with customers;

[1624] a means for storing a communication history with the customer;

[1625] means for analyzing a communication history with the customer and generating a summary;

[1626] a means by which users or customers can submit inquiries;

[1627] means for searching and acquiring an appropriate communication history based on the received inquiry;

[1628] means for receiving the inquiry;

[1629] means for using an emotional engine including means for analyzing the emotional state of a user in addition to said summary;

[1630] means for adjusting prompts sent to a generative artificial intelligence (AI) based on the emotional state of the user;

[1631] a means for utilizing generative artificial intelligence (AI) to generate appropriate responses based on the tailored prompts;

[1632] means for providing said generated answers to a user or customer;

[1633] A system including:

[1634] (Claim 2)

[1635] means for providing an interface for users or customers to submit queries;

[1636] means for receiving a query via said interface;

[1637] means for adjusting prompts based on the user's emotional state obtained from the interface;

[1638] means for transmitting the tailored prompt to a generative artificial intelligence (AI);

[1639] The system of claim 1 further comprising:

[1640] (Claim 3)

[1641] The generative artificial intelligence (AI) uses natural language processing technology;

[1642] means for using the user's emotional state and the generated summary together as prompts to generate an appropriate response;

[1643] 10. The system of claim 1, comprising:

[1644] "Application example 2 when combining emotion engines"

[1645] (Claim 1)

[1646] A means of obtaining communication history with customers;

[1647] means for analyzing a communication history with the customer and generating a summary;

[1648] a means for utilizing an emotion engine for analyzing the emotional state of users and customers;

[1649] a means for utilizing a generative artificial intelligence (AI) to generate an answer to a user's inquiry based on the summary and the sentiment analysis result;

[1650] means for providing said generated answers to a user or customer;

[1651] A system including:

[1652] (Claim 2)

[1653] means for providing an interface for said user or customer to submit an inquiry;

[1654] means for receiving the query and performing sentiment analysis;

[1655] A means for searching and acquiring an appropriate communication history based on the received inquiry and sentiment analysis results;

[1656] The system of claim 1 further comprising:

[1657] (Claim 3)

[1658] The generative artificial intelligence (AI) uses natural language processing technology;

[1659] means for combining the query, the generated summary, and the sentiment analysis results and using them as prompts to generate an appropriate answer;

[1660] 10. The system of claim 1, comprising: [Explanation of symbols]

[1661] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of obtaining communication history with customers; means for analyzing a communication history with the customer and generating a summary; means for utilizing generative artificial intelligence to generate answers to user queries based on said summaries; means for providing said generated answers to a user or customer; A system including:

2. means for providing an interface for said user or customer to submit an inquiry; means for receiving the inquiry; means for searching and acquiring an appropriate communication history based on the received inquiry; The system of claim 1 further comprising:

3. The generative artificial intelligence uses natural language processing technology; means for using the query and the generated summary together as a prompt to generate an appropriate response; The system of claim 1 , comprising:

Images